Fairness Metrics and Why They Cannot All Be Satisfied at Once

Fairness in machine learning is not a single concept with one metric. It is a collection of competing definitions, each expressing a different notion of what it means for a model to treat groups equitably. You cannot optimise for all fairness metrics at once.

You must choose which notion of fairness matters most for your specific use case and document that choice carefully.

One definition of fairness is equal accuracy across groups: if the model is 90 percent accurate on men and 85 percent accurate on women, it is not equally accurate and treats the groups differently. This can occur in hiring systems where a model correctly identifies qualified candidates at different rates depending on gender.

For example, a recruitment algorithm may successfully identify 90% of qualified male candidates but only 85% of qualified female candidates. This discrepancy can lead to underrepresentation of women in roles even when their qualifications are comparable.

Another definition is equal positive rate (demographic parity): if the model approves 70 percent of men and 60 percent of women, it is not equally treating the groups despite your intent. This is often applied in lending scenarios. For instance, a mortgage approval system that approves 70% of male applicants and 60% of female applicants may appear to treat both groups fairly on the surface, but it fails to account for underlying differences in base rates.

In a real-world case, a UK bank using such a model might find that although the approval rate is close, the underlying qualification levels differ significantly between genders, resulting in unintended bias.

A third is equal false positive rate: if the model incorrectly approves unqualified applicants 5 percent of the time among men and 8 percent of the time among women, false positive rates differ across groups. This is critical in domains such as criminal justice or medical diagnosis.

An example would be a predictive policing model that flags individuals for increased surveillance. If it flags 5% of men and 8% of women incorrectly, it may lead to over-policing of women even when their risk levels are similar. This highlights how even well-intentioned models can produce disparate impacts.

The mathematical constraint is called impossibility of equalization. If two groups have different base rates (60 percent of male applicants are creditworthy, 50 percent of female applicants are), you cannot simultaneously achieve equal accuracy across groups, equal approval rates across groups and equal false positive rates across groups.

If you optimise to approve the same fraction of men and women to achieve demographic parity, you must approve some unqualified women and decline some qualified men. This appears unfair to the qualified men but was necessary to achieve parity of approval rates. Every fairness choice involves trade-offs.

For loan approval, your notion of fairness might mean: “Equal approval rate for applicants with equal creditworthiness.” This is often applied in consumer lending. A UK credit card company may implement a model that ensures that individuals with similar credit scores receive similar treatment, regardless of gender or ethnicity.

However, this approach must be balanced against other metrics such as false positive rates to avoid approving unqualified applicants.

For medical diagnosis, it might mean: “Equal false negative rate across ethnic groups so we miss illness equally often for each group.” This is essential in healthcare systems where disparities in diagnosis can lead to unequal treatment. A hospital using an AI diagnostic tool must ensure that the model does not miss illnesses more frequently in one ethnic group than another.

For example, a model that misses lung cancer in 5% of white patients and 10% of Black patients would be considered unfair and potentially harmful.

For employment screening, it might mean: “Equal accuracy across age groups so the model performs equally for older and younger workers.” This is particularly relevant in sectors where age discrimination is a concern. A UK tech company using an automated screening process must ensure that the model does not penalise older candidates even if their experience is comparable.

A practical example is a model that rates candidates based on their years of experience and education, but fails to account for the fact that older candidates may have had fewer opportunities to gain certain certifications.

Some fairness metrics assume you have a true label (the applicant did or did not repay the loan; the patient did or did not have the disease). But if the model is only used for screening and no one with a low score is ever given the opportunity to prove they are creditworthy, you cannot measure whether the model is fair to low-scoring applicants.

You have no ground truth for them. This is a common challenge in hiring systems. For example, a company may use an AI model to shortlist candidates, but those not shortlisted never get a chance to demonstrate their suitability.

This means the model’s fairness cannot be evaluated for those who are rejected, even if their performance in the actual role is unknown.

You must document which fairness definition you chose, why it is appropriate for your specific use case, and what trade-offs it involves with other fairness metrics. You must acknowledge explicitly that your choice implies accepting some degree of inequality in other fairness metrics. For example, a UK financial institution may choose to prioritise equal false positive rates over equal accuracy to reduce the risk of approving unqualified borrowers.

This decision must be clearly communicated to stakeholders and auditors. A practical step is to create a fairness impact assessment that outlines the chosen metric, its rationale, and how it aligns with the company’s ethical and regulatory responsibilities. This documentation is essential for transparency and accountability in AI decision-making processes.