Writing the Model Card and the Technical File
A model card is a short structured document describing a machine learning model, its intended use, performance, limitations and recommendations for appropriate use. Model cards are increasingly expected by regulators and stakeholders. A good model card is honest about limitations and does not oversell the model’s capabilities. For example, a model card for a credit scoring system should clearly state that the model is intended to predict creditworthiness based on historical financial data and not to make decisions about personal circumstances such as family status or social behaviour. This transparency helps prevent misuse and ensures that the model is applied only in contexts where it is appropriate and reliable.
A model card should include: intended use (what decision this model makes), performance metrics and results (reported separately for all subgroups), limitations (coverage gaps, known weaknesses), recommendations for use (appropriate and inappropriate contexts) and information about the data and training process. In a real-world scenario, a model card for an automated hiring tool must detail that the model is designed to screen CVs for suitability based on job requirements. Performance metrics should be broken down by gender, age group and educational background to show whether the model treats all candidates fairly. If performance varies significantly across these groups, this must be clearly stated. This level of detail is essential for building trust and accountability in AI systems.
The EU AI Act and other regulations require technical documentation for high-risk AI systems. A technical file is more detailed than a model card and is compiled during development, not written retroactively afterward. It includes: a description of the AI system and its purpose, data used for training and validation, methodology for assessing risks (fairness, bias, performance), technical specifications, performance metrics and evidence that the system meets safety and fairness requirements. For instance, a technical file for a medical diagnosis AI must outline how the model was trained on anonymised patient data, what specific medical conditions it was designed to detect, and how bias was tested and mitigated. This documentation must be complete and precise to meet regulatory standards.
For regulators, a technical file must include evidence, not assertions. Do not write “the model is fair.” Write “we tested fairness using these metrics, found these performance gaps in these subgroups, and accepted those gaps because…” Regulators want to see data, not claims. They want evidence of your process and reasoning. An example of this is a technical file for a facial recognition system that includes detailed fairness testing results. The file would state that the system was tested on a dataset of 10000 images, with accuracy rates of 95% for white males and 80% for Black females. The reasoning behind accepting this gap would be explained, such as the limited diversity in the training data and steps taken to improve it in future versions. This level of detail is critical for regulatory compliance and stakeholder confidence.
A technical file should live in a shared repository accessible to compliance and audit teams. It should be versioned (v1.0, v1.1, v2.0) so there is a clear record of what was known about the model at each point in time. If the model was deployed under v1.0 and you later discover a fairness issue in v1.1, you know which deployment that issue applies to and can trace consequences. For example, a financial services company deploying a fraud detection model must ensure that all versions of the technical file are stored and accessible. If a new version reveals that the model underperforms for certain user demographics, this can be traced back to the specific version and deployment, allowing for prompt corrective action. This version control is essential for maintaining transparency and accountability.
Keep both documents up to date. If the model is updated, update the model card and technical file. If you discover a new limitation after deployment, add it to the documentation. These documents are living records of the model throughout its entire life, not static documents created once at launch. A practical example is a customer service chatbot that was initially designed to handle basic queries. After deployment, it was found that the model struggled with complex emotional language. This limitation was added to the model card and technical file, along with recommendations that the chatbot should not be used for sensitive customer interactions. This update ensures that future users of the model are aware of its limitations and can adjust their use accordingly.
Creating and maintaining these documents also involves collaboration between multiple teams. Data scientists must work closely with compliance officers and legal teams to ensure that all technical and regulatory requirements are met. For example, a team developing an AI-powered recommendation engine must include input from privacy experts to ensure that user data is handled appropriately. The model card and technical file must reflect this multidisciplinary effort. Regular reviews and updates should be scheduled, perhaps quarterly or after each model update, to ensure that the documentation remains accurate and relevant. This process also helps in identifying potential issues before they become critical, allowing for proactive rather than reactive management of model performance and compliance.