Building the High Risk Evidence Pack and Post Market Monitoring Plan
The final stage of high risk compliance is compiling the evidence pack and establishing the post-market monitoring plan that will be submitted to the conformity assessment body and maintained throughout the system’s lifecycle.
The evidence pack is a complete compilation of all documentation demonstrating compliance with high risk requirements. It should be organised logically and indexed for easy reference. Start with the technical documentation prepared per Article 11 and Annex IV. This is the foundation. Ensure the technical documentation clearly describes the system’s intended purpose, performance characteristics, training data, validation procedures and any known limitations.
Include the risk assessment documentation from Article 9. This should identify the principal risks associated with the system, explain the probability and severity of each risk and document the mitigations implemented. Include evidence of risk validation (testing results, validation reports) demonstrating that mitigations actually reduce risk as expected.
Include data governance documentation from Article 10. Document the training data, including its sources, volume, representativeness analysis across relevant populations and any identified gaps. Include documentation of data quality procedures, error detection and labelling verification processes. Include dataset documentation or data cards describing the data completely.
Include the instructions for use as required by Article 13. Ensure these are written clearly so deployers can actually follow them. Include quality management system documentation from Article 17, including procedures for development, deployment, monitoring and change management. Include evidence of performance monitoring results showing the system meets documented targets and procedures for handling incidents.
Include human oversight procedures and evidence that they work in practice, including authority assigned to reviewers, training provided and deployment logs showing humans are exercising genuine oversight. Include cybersecurity documentation describing security measures and any penetration testing results.
Include evidence of Article 4 AI literacy measures documenting how deployers and affected individuals are informed, and evidence of Article 26 deployer duties and Article 27 impact assessments if applicable.
The post-market monitoring plan describes how the organisation will continue to monitor the system’s performance, detect problems and implement corrections after deployment. This is not optional. The Conformity Assessment Body will review the monitoring plan to assess whether the organisation has adequate procedures for ongoing compliance.
The monitoring plan must specify which metrics will be tracked (accuracy by demographic group, performance consistency, incident rates, override rates), the frequency of monitoring (monthly for accuracy, quarterly for demographic breakdowns, annual reviews), and responsibility for monitoring and reporting. Thresholds must trigger investigation: what accuracy drop requires action, and procedures must specify how to determine whether degradation stems from data shift, model drift or other causes. Escalation procedures should define when to notify management, contact the provider or pause the system.
The plan should address bias and fairness monitoring, anomalies in human oversight, feedback collection from affected individuals and individuals, serious incident reporting procedures, model retraining and testing before deployment, and end-of-life procedures including record retention and log archiving.
Consider a practical example. A transportation company implements a high risk system for driver monitoring and safety assessments. The evidence pack includes technical documentation describing the system’s accuracy (94 percent) for predicting safety-relevant driving behaviours. Risk assessment documentation identifies risks of false positives (falsely flagging safe drivers as unsafe) and biases in predictions across different age groups and regions. The pack includes data governance documentation showing that training data represents drivers from all regions where the system operates. Quality management procedures describe monthly accuracy reviews and quarterly demographic breakdowns. Human oversight procedures document that supervisors must review all driver safety recommendations before they affect employment decisions.
The post-market monitoring plan specifies monthly accuracy reviews, quarterly demographic breakdowns, annual complete reviews and management reporting. Thresholds trigger investigation: if accuracy drops below 90 percent or demographic accuracy gaps exceed 4 percentage points. Investigation procedures specify how root causes will be determined (data shift, model drift, operational changes). Escalation procedures specify that persistent problems trigger consultation with the system provider and possible system pause pending remediation.
Building the evidence pack requires time and resources. Organisations should plan backwards from the 2 December 2027 deadline, allowing time for documentation preparation, internal review and remediation of any gaps before conformity assessment begins. Waiting until late 2027 to begin this work will result in non-compliance. The evidence pack is not created in weeks; it requires months of systematic work to gather, organise and verify all the required documentation and evidence of compliance.