Subgroup Analysis and Reporting Performance Where It Is Weakest

Overall accuracy masks subgroup performance differences. A model can have 90 percent accuracy overall but 75 percent accuracy for a subgroup that is underrepresented or different in other ways. If you report only overall accuracy to stakeholders and regulators, you are hiding the fact that the model fails for some groups. This is not acceptable in regulated AI. For example, in a UK employment screening model, overall accuracy may be 90%, but for candidates aged over 50, accuracy drops to 70%. This discrepancy could indicate age bias and may breach the Equality Act 2010. Stakeholders must be made aware of such disparities to ensure compliance and ethical deployment.

Subgroup analysis means calculating performance metrics separately for each subgroup of interest: men and women, different age groups, different ethnicities, different income levels, different geographic regions. You must then compare these metrics to see if any subgroup has substantially worse performance. This is how you discover whether your model is fair or whether it discriminates systematically against some groups. In a healthcare diagnostic model, for instance, performance may be 95% for white patients but only 80% for Black patients. This difference must be flagged and investigated. Subgroup analysis is not optional; it is a regulatory requirement in many domains, including finance, healthcare, and public services in the UK.

A subgroup with worse performance is not automatically a reason to reject the model and never deploy it. It is a reason to understand why performance is worse and then to decide consciously whether that performance gap is acceptable. Perhaps the subgroup is much smaller in the training data so the model has fewer examples to learn from. Perhaps the subgroup’s characteristics are structurally different and the model has learned patterns that do not generalise. Perhaps there is true unfairness in the data itself and the model should not be deployed. For example, a UK lending model may perform poorly for rural borrowers due to limited historical data. This is not necessarily a reason to abandon the model, but it is a signal that further investigation and potentially targeted data collection is required.

The threshold for acceptable performance difference depends on your specific use case. In hiring, accepting a 10 percentage point lower accuracy for one age group might be unacceptable because age discrimination is illegal. In content recommendation, a similar performance gap might be tolerable because the stakes are lower. You must make this decision explicitly during development and document it. A UK e-commerce company using a recommendation engine may accept a 5% performance gap for users in rural areas, as their primary concern is user engagement rather than fairness. However, a model used for mortgage lending must meet stricter thresholds to avoid discriminatory outcomes. This decision-making process must be transparent and auditable.

Reporting performance where it is weakest means you do not report only the best-case accuracy. You report the worst subgroup performance as a floor. If the model is 90 percent accurate overall but 60 percent accurate for one subgroup, you report both numbers. Regulators and auditors expect to see the worst-case performance so they can evaluate whether that is acceptable. Hiding weak subgroup performance is deceptive. In a UK government digital service, a model used for benefit eligibility screening must report performance for all ethnic groups, even if one performs significantly worse. This transparency is essential for public trust and regulatory compliance.

You must also investigate why performance is worse for some subgroups. Is it because of data quality issues specific to that subgroup’s data? Is it because the subgroup is underrepresented in training? Is it because the subgroup’s characteristics are structurally different from the majority group? Each explanation points to a different response and each has different implications for deployment. For example, a UK fraud detection model may underperform for users in certain regions due to underrepresentation in training data. This can be addressed by collecting more diverse data or using techniques like synthetic data generation. Alternatively, if performance differences are due to structural differences in user behaviour, the model may need to be retrained or adapted to account for those variations. A practical step is to create a subgroup performance dashboard that tracks metrics over time and flags anomalies. This allows teams to respond quickly and maintain model fairness and performance.