Safe AI for Data Scientists: Evidence, Documentation and Testing Regulators Accept

This online course teaches data scientists how to develop artificial intelligence systems that meet regulatory requirements and gain approval from oversight bodies. The programme targets professionals working with machine learning models, including data analysts, machine learning engineers, and AI researchers who need to demonstrate their systems comply with safety standards. Students learn to apply evidence-based approaches to AI development, create proper documentation for regulatory submissions, and implement testing procedures that regulators find acceptable. The curriculum covers essential topics such as risk assessment, model validation, audit trails, and compliance frameworks that apply across different industries and jurisdictions.

The course consists of fifteen lessons, each containing a graded quiz to test understanding, plus a final examination to assess overall knowledge. All materials are available at no cost, making high-quality AI regulation education accessible to anyone interested in responsible machine learning development. Upon completion, learners can confidently apply regulatory principles to their AI projects, prepare documentation for approval processes, and ensure their machine learning systems meet required safety standards. Graduates will have practical skills for working with regulatory bodies and implementing quality assurance measures that support trustworthy artificial intelligence deployment.

Frequently asked questions

What documentation does a machine learning model need for compliance?

Under the EU AI Act and most regulatory frameworks, a model needs a description of its intended purpose, records of the data used including provenance and any known quality issues, documentation of how you assessed fairness and bias, evidence that you used sound data science practices, and a model card describing performance across subgroups. The key principle is that you must produce evidence for every material decision you made during development, not just a final accuracy number.

How do you test a model for bias?

Test for bias by calculating performance metrics separately for each subgroup: men and women, different age groups, ethnicities, income levels and other relevant categories. Compare accuracy, false positive rate, false negative rate and other metrics across subgroups. If some subgroups have materially worse performance, investigate why. Consider whether to exclude the model from that subgroup, collect more data for it or accept the limitation consciously. Never report only overall performance; subgroup performance is the real test of fairness.

Why can fairness metrics not all be satisfied at the same time?

This is a mathematical constraint called impossibility of equalization. If populations have different base rates (for example, different loan default rates), you cannot simultaneously achieve equal accuracy, equal approval rates and equal false positive rates across groups. If you optimise for equal approval rates, you must accept worse accuracy for some groups. You must choose which fairness definition matters most and document that choice.

What is model validation and who should do it?

Model validation is independent review by someone who did not build the model and has authority to block deployment. The validator checks that data is appropriate, that metrics are well-chosen, that fairness analysis is thorough and that all documentation is honest. Validation is not a checkbox. Good validators are sceptical and ask tough questions. They ensure the model development process was sound before it goes live.

What is data leakage in machine learning?

Data leakage occurs when information from the test set influences the training process. Examples include using a variable that is only known after the decision is made (reporting that an account was fraudulent after fraud already happened), or preprocessing data using statistics calculated from the entire dataset rather than from training data only. Leakage makes a model appear better than it actually is and causes failures in production.

How do you document a training dataset?

Document the source of the data, who collected it and for what purpose, the size and scope of the dataset, the licence under which it is available, any known quality issues or errors, and whether you have rights to use it for model training. If the data includes underrepresented populations, document those gaps. If data collection changed over time, document when. This documentation allows auditors to assess whether the data is appropriate for the task.

What is model calibration and why does it matter?

Calibration means that predicted probabilities are honest about uncertainty. A model that predicts 0.7 probability of an event should be correct about 70 percent of the time on similar cases. If a model is miscalibrated, its probabilities are unreliable. In high-stakes decisions like lending, miscalibration can lead to systematic errors. You can measure calibration using a calibration curve and adjust it using post-hoc methods like Platt scaling.

Do regulators accept explainability methods such as SHAP?

Explainability methods like SHAP and LIME show what a model does but do not prove fairness or safety. Regulators increasingly reject the claim that a model is safe simply because it is explainable. Explainability is useful for debugging and oversight but does not substitute for thorough fairness testing, validation and thorough documentation. Use explainability as part of responsible AI practice, not as a substitute for rigorous evaluation.

What does the EU AI Act require of data used to train models?

The EU AI Act requires that data used to train high-risk AI systems is appropriate, of adequate quality, and representative of the population the system will serve. You must document the data, assess any biases it contains and test the model for fairness across subgroups. Since 2 August 2026, Article 4 on AI literacy requires that providers ensure persons affected by high-risk AI systems can exercise their rights. You must be transparent about how data was used and how the model makes decisions.