A.6 Continued: Deployment, Operation and Retirement Records

This lesson covers the second half of control A.6, addressing deployment, operation and retirement of AI systems. These are the activities that happen after a system is built and where many organisations have control weaknesses.

Deployment readiness

Before deploying an AI system into production, your organisation should confirm that it is ready. Deployment readiness includes confirming that all planned verifications have been completed, that performance is acceptable in a production-like environment, and that staff who will operate the system understand how to use it and what to do if problems occur.

Many organisations create a deployment checklist or gate. This checklist includes items like “verification tests complete”, “documentation reviewed and approved”, “operators trained”, “monitoring configured” and “incident response plan documented”. The checklist is signed off by the person responsible for the system before it goes live.

Deployment records should include who approved deployment, when it occurred and what configuration was deployed. Many organisations maintain a deployment log or change management record. During audit, auditors will ask to see evidence of the deployment decision and who made it. A system that was deployed without documented approval and no deployment record is a control weakness.

Operating and monitoring systems

Control A.6 requires that you maintain records of how AI systems perform during operation. This includes monitoring key performance metrics, checking that outputs remain acceptable and verifying that the system is being used as intended.

For some systems, monitoring might be manual. If a system makes hiring recommendations, a manager might review outputs weekly to check that they look reasonable. For other systems, monitoring might be automated. A model that generates text responses might be monitored through automated checks for toxic language or factual errors, with alerts when anomalies are detected.

Your monitoring plan should define what will be monitored, how often and what will trigger escalation if a system is not performing as expected. A common audit finding is “systems are deployed but no one is checking whether they still work”. This is a significant control failure because you have no visibility into whether your system is meeting its objectives or harming people.

Retirement and ongoing records

Eventually systems are retired, either because they are replaced or because a risk becomes unacceptable. Control A.6 requires that you document why systems are retired and what happens to their data and models. Are models deleted or archived? How long is operational data retained? Do you need to notify users that a system will no longer be available?

Your organisation should also maintain records of all AI systems for a defined period. These records might be kept for compliance investigation, incident analysis or to understand what happened when a system fails. Many organisations keep records of systems and their performance for three to five years after retirement.

Building the life cycle narrative

An effective approach is to maintain one life cycle record for each AI system that covers the system from inception through retirement. This could be a single document or a file folder. It includes the objectives statement, design documentation, verification plan and results, deployment record, monitoring records and retirement documentation. During audit, this single record allows auditors to understand the entire history of the system’s management.