Impact Ratios, the Four Fifths Rule and Legal Thresholds

Understanding Impact Ratios

Impact ratios measure the difference in outcomes between different groups when assessing AI system performance. These ratios compare the rate of positive outcomes for a protected group against the rate for the reference group. For example, if an AI system approves 80% of loan applications from group A but only 60% from group B, the impact ratio equals 0.75.

This calculation helps identify potential disparities that may indicate bias.

Organisations use impact ratios to monitor fairness across various demographic categories including age, gender, ethnicity, or disability status. The ratio compares approval rates, rejection rates, or success rates between these groups. A ratio of 1.0 indicates equal treatment, while values below 1.0 suggest disadvantage for the group being measured.

Practitioners must calculate these ratios for each protected characteristic identified in their organisation’s diversity policy.

• Impact ratios work best when calculated using large enough sample sizes to ensure statistical significance
• Organisations should establish baseline ratios before implementing AI systems to identify changes
• Multiple ratios should be calculated for different outcomes to capture complete bias patterns

The Four Fifths Rule as Legal Benchmark

The Four Fifths Rule serves as a widely accepted standard for determining whether disparities in AI outcomes constitute adverse impact. This rule states that if the selection rate for any group is less than 80% of the rate for the group with the highest selection rate, there may be evidence of discrimination.

The 80% threshold originates from US employment law but has gained international recognition as a practical measure.

Consider a recruitment AI system that processes applications from candidates aged 25-35 and 36-50. If 40% of younger candidates receive job offers while only 25% of older candidates receive similar outcomes, the ratio equals 0.625. This falls below the 0.80 threshold, indicating potential adverse impact.

Practitioners must apply this rule consistently across all protected characteristics identified in their organisation’s equal opportunities framework.

Legal practitioners often reference this rule when conducting fairness assessments. The rule provides a clear numerical boundary that makes bias easier to identify and measure. Courts and regulatory bodies have used similar thresholds when evaluating claims of discriminatory impact.

The rule helps practitioners make objective decisions about whether observed disparities require further investigation or remediation.

Setting Legal Thresholds for Practical Application

Organisations must establish their own legal thresholds based on regulatory requirements and industry standards. The four fifths rule provides a starting point, but practitioners should consider additional factors such as sector-specific regulations or internal diversity commitments. Financial services firms may apply stricter thresholds due to regulatory oversight, while other industries might use more lenient measures.

Legal thresholds should account for reasonable accommodation requirements and legitimate business needs. For example, a healthcare AI system might have different thresholds for clinical outcomes compared to administrative processes. Practitioners must document their threshold decisions and maintain consistency across similar applications.

The thresholds must align with existing organisational policies regarding equal opportunities and anti-discrimination measures.

• Thresholds should be clearly documented and communicated to relevant stakeholders
• Regular review of thresholds ensures they remain appropriate for changing circumstances
• Legal thresholds must comply with existing employment and data protection legislation

Practitioners should calculate impact ratios for each protected group identified in their organisation’s diversity framework. The four fifths rule provides a consistent approach to determining whether disparities reach legally significant levels. Setting appropriate thresholds helps organisations identify bias early and take corrective action before legal issues arise.

Regular monitoring of these metrics ensures ongoing compliance with equal opportunities requirements. The combination of impact ratios and established thresholds creates a practical framework for measuring AI fairness that practitioners can defend in regulatory or legal contexts.