Article 10 Data Governance: Representativeness, Errors and Gaps

Article 10 addresses data governance for high risk AI systems, requiring organisations to implement procedures ensuring that training, validation and testing data meets specific quality standards. Data quality directly affects system performance, fairness and compliance. Poor data governance leads to models that fail on certain populations, make biased decisions or misclassify edge cases.

The first requirement concerns representativeness. Training data must be representative of the population or contexts where the system will be deployed. If a high risk recruitment system is trained primarily on historical hiring data from a specific geographic region, it may not perform equally well when deployed nationally. The system might be accurate for the region where training data came from but biased or less accurate for other regions. Article 10 requires organisations to assess whether training data represents the relevant populations and use cases.

Representativeness testing must identify gaps. If training data underrepresents certain demographic groups, certain job categories or certain geographic regions, the organisation must document these gaps and assess their consequences. A system trained on majority population data may not recognise faces from minority populations accurately. A recruitment system trained on data from particular industries may make poor decisions for other industries. Documenting representativeness gaps is the first step; then the organisation must decide how to address them.

The second requirement covers data quality errors. Training and validation data must be checked for errors that could affect system performance. This includes labelling errors where training examples are incorrectly categorised, missing values where important information is absent, outliers that represent data quality problems rather than legitimate edge cases, and duplicates that might skew statistical inference. A high risk system for employment decisions must have training data where job performance labels are accurate. A system for medical diagnosis must have training data where diagnoses are correctly verified.

Error detection and correction procedures must be documented. The organisation must describe how data quality is verified, what error thresholds trigger correction or retraining and who is responsible for data quality oversight. Simply stating that data quality is important does not satisfy Article 10. The organisation must implement specific procedures and document their application to the specific system.

The third requirement addresses data gaps. Article 10 requires identification and documentation of any known gaps or limitations in training, validation and testing datasets. A system for credit decisions trained on data from customers in developed economies may not work well for customers in developing markets. A system for predicting equipment failure trained on data from a specific manufacturer might not generalise to other manufacturers. Documenting these gaps is important so deployers understand system limitations.

Data versioning and traceability matter. The organisation must be able to identify exactly which data version was used to train, validate and test each model version. This becomes important when model performance degrades and retraining is needed. Understanding which data versions correspond to which model versions helps troubleshoot problems. If a deployed model’s performance drops, the organisation must be able to trace back to the training data to identify whether the problem is data drift in production or a problem with the training data itself.

Article 10 also addresses data documentation. Organisations should maintain data cards or dataset documentation describing the data, including its provenance, its characteristics, how it was collected, what preprocessing was applied and known limitations. This documentation supports compliance demonstration and helps deployers understand what training data means for system behaviour.

Consider practical implementation. A healthcare organisation develops a high risk AI system for patient triage. Article 10 requires them to document the training dataset’s representativeness across age groups, gender, ethnicity and geographic regions where the system will be deployed. They conduct representativeness analysis and find underrepresentation of elderly patients and certain ethnic groups. They document this gap and decide to acquire additional training data for underrepresented populations. They implement data quality procedures checking for labelling errors in diagnoses and missing vital signs. They document the data versioning procedure used in their machine learning pipeline. They maintain dataset documentation describing the data source, collection procedures and known limitations.

This is not a one-time exercise. As the system is deployed and used with real patient data, the organisation monitors whether the production data characteristics match training data. If they find distribution shifts (new disease patterns, new patient demographics), they update their Article 10 assessment and implement retraining or model adjustment.