Group Fairness Metrics: Parity, Equal Opportunity and Equalised Odds

Understanding Fairness Through Mathematical Parity

Group fairness metrics measure how well an AI system treats different demographic groups equally. The simplest approach focuses on parity conditions that compare outcomes across groups. Statistical parity examines whether the probability of positive outcomes remains consistent across protected attributes such as gender, ethnicity, or age.

For example, a hiring algorithm should offer equal opportunities to male and female candidates. If 30% of male applicants receive job offers while only 15% of female applicants receive similar treatment, this indicates a parity violation.

Parity conditions apply to various scenarios in practice. In credit scoring, financial institutions must ensure similar approval rates across different demographic groups. In healthcare, diagnostic algorithms should maintain consistent accuracy rates for patients from diverse backgrounds.

The mathematical approach involves calculating group-specific metrics such as selection rates or positive predictive values. These measures help identify systematic disparities that may indicate bias in automated decision-making processes.

• Statistical parity compares overall positive outcome rates across groups
• Demographic parity focuses on equal representation in favorable outcomes
• Calibration parity ensures similar prediction accuracy across demographic segments

Equal Opportunity: Balancing True Positives Across Groups

Equal opportunity addresses the challenge of ensuring fair treatment when outcomes matter for different groups. This metric specifically targets the true positive rate across protected attributes. In criminal justice applications, equal opportunity requires that algorithms identify guilty individuals at similar rates regardless of race or ethnicity.

The focus remains on maintaining consistent detection of relevant outcomes rather than overall group representation.

Practical implementation involves measuring recall or sensitivity for each demographic group. For instance, a fraud detection system must identify fraudulent transactions at equal rates among different customer segments. If the system correctly identifies 80% of fraud cases among white customers but only 60% among Black customers, this reveals an equal opportunity problem.

The metric compares these true positive rates to determine whether the algorithm treats all groups fairly in detecting meaningful outcomes.

Implementation requires careful attention to data quality and algorithmic design. Healthcare screening systems must maintain equal opportunity when identifying disease markers across patient demographics. Employment assessment tools should ensure similar success rates in identifying qualified candidates from different backgrounds.

The approach involves calculating group-specific true positive rates and comparing these values against established thresholds.

Equalised Odds: The Complete Picture of Fairness

Equalised odds combines equal opportunity with equalised false positive rates to create a fuller measure of group fairness. This approach requires that both true positive rates and false positive rates remain consistent across demographic groups. In immigration processing, equalised odds ensure that the system identifies genuine threats at similar rates while maintaining equal error rates for different nationality groups.

The metric provides a balanced view of algorithmic performance across both correct and incorrect classifications.

Real-world applications demonstrate the importance of equalised odds in preventing compounding bias. In college admissions, equalised odds prevent algorithms from disproportionately rejecting qualified candidates from certain backgrounds. The approach involves measuring both true positives and false positives for each demographic group.

For example, a university admissions algorithm must maintain similar rates of accepting qualified students while also ensuring equal rejection rates for students who do not meet criteria.

Implementation involves calculating four key metrics for each group: true positives, false positives, true negatives, and false negatives. These values form the basis for determining whether equalised odds conditions are met. The approach helps identify when algorithms perform well for some groups but poorly for others.

In employment screening, equalised odds prevent systems from appearing fair on surface measurements while creating hidden disparities in final outcomes.

Organisations implementing these metrics must consider practical limitations and data constraints. The measurement process requires sufficient data samples for each demographic group to ensure reliable results. Regular monitoring helps identify when fairness conditions change over time or with new data inputs.

These approaches provide concrete frameworks for practitioners to evaluate and improve algorithmic decision-making processes.