Article 11 and Annex IV Technical Documentation Contents
Article 11 requires high risk AI systems to be accompanied by technical documentation. This is not marketing collateral or user manuals. The technical documentation specified in Annex IV is a detailed compendium of information about the AI system’s design, development, testing and deployment characteristics. This documentation is the evidence that a system is ready for deployment and conformity assessment.
Annex IV lists the specific contents required. The documentation must include a general description of the AI system, including its intended purpose, the decision-making logic and how it uses data. For a high risk employment decision system, this means describing what employment decisions the system supports (hiring, promotion, disciplinary action), how the algorithm makes decisions and what data it uses. This description must be written so that deployers, regulators and conformity assessors can understand what the system does.
The documentation must specify the performance characteristics of the AI system. This includes accuracy, reliableness, cybersecurity and any other relevant metrics. Documentation should report accuracy as evaluated on representative test data, broken down by demographic groups and use cases where the system will be deployed. Accuracy alone is not sufficient; the documentation must also address reliableness, describing how the system performs when data characteristics change, when adversarial examples are present or when operational conditions deviate from normal circumstances.
The documentation must describe the training, validation and testing data, including information about data sources, data volume, representativeness across relevant populations and any known limitations or gaps. This builds on Article 10 data governance. The technical documentation provides the detailed evidence that Article 10’s representativeness and error management requirements have been met.
Instructions for use must be included in the documentation. These are written procedures for deployers explaining how to use the system correctly, what decisions the system is designed to support, what decisions it is not designed for, and what human oversight is required. The instructions must be specific and actionable, not vague. For a credit decision system, instructions must explain how to integrate the system into the lending process, when human loan officers must review system recommendations and what to do if the system produces an unusual or concerning recommendation.
The documentation must address human oversight measures implemented to mitigate risks. If the system is configured to require human review of decisions affecting a certain threshold amount, this must be documented. If certain types of decisions must always be reviewed by humans, this must be specified. The documentation describes the technical mechanisms for human oversight (for example, alerts that require human confirmation, decision logs that are reviewed periodically).
Risk management procedures must be documented. This is the evidence of Article 9 compliance. The documentation must identify the main risks associated with the system, explain the mitigations implemented and describe the procedures for ongoing monitoring. The documentation should include the results of testing for performance under different conditions, testing for bias and fairness, and any adversarial reliableness testing.
Quality management procedures are documented. Under Article 17, organisations must have quality management systems. The technical documentation describes these systems: how the organisation keeps records, how changes to the model or data are tracked, how retraining decisions are made and who has authority for model changes.
Logs and monitoring procedures are described. Under Article 12, the system must generate logs of decisions. Under Article 9, the organisation must monitor performance. The documentation explains what logs the system generates, how long they are retained, what monitoring metrics are tracked and how performance degradation is detected and addressed.
The documentation must also address model transparency to the extent possible. If the system uses a deep learning model that is inherently difficult to interpret, the documentation must acknowledge this and explain what approaches have been used to provide decision transparency (feature importance analysis, example-based explanations, decision rules) even if the model itself cannot be fully explained.
Practical implementation: A healthcare organisation develops a high risk system for patient triage. Annex IV documentation includes a general description explaining the system accepts patient symptoms and vital signs and assigns patients to urgency categories. Performance characteristics are documented, including accuracy rates by urgency level and by patient demographics. Training data is described, including sources (hospital records from the past three years), volume (125,000 patient encounters), representativeness analysis showing coverage of all age groups and geographic areas served. Instructions for use explain that the system supports triage nurses’ decisions and that nurses retain authority to override recommendations based on clinical judgment. Human oversight measures include a requirement that system recommendations must be reviewed by a qualified nurse who confirms or modifies the urgency assignment. Risk management documentation describes the identified risks (incorrect urgency assignment, performance variance across demographics) and mitigations (model validation on held-out data by demographics, continuous performance monitoring with alerts if accuracy drops). Quality management procedures describe the model versioning system and the procedures for retraining when performance degrades.