Auditing Monitoring, Drift Detection and Alert Response

After a model is deployed, it continues running in production and making decisions about real users. The world changes in ways the training data did not anticipate. Users behave differently, external events shift patterns, and over time model performance degrades. This stage is called the production phase, and controls here focus on monitoring whether the model is still working as intended, detecting when performance degrades, and ensuring the team responds quickly when something goes wrong.

Performance Monitoring and Metrics

The first control is continuous monitoring of model performance. At minimum, the organisation should track the same metrics used to evaluate the model before deployment. If the model was evaluated on accuracy, precision and recall, then production monitoring should track these metrics continuously and alert if they drop below acceptable thresholds. Many organisations deploy monitoring that tracks only technical metrics (latency, error rate, throughput) and miss that model performance itself is degrading.

Audit monitoring by asking: what metrics are tracked in production? Are they the same metrics used in testing? How frequently are they calculated and reported? Are there alert thresholds, and do they trigger when performance drops? If the organisation does not track model-specific metrics like accuracy or fairness, that is a significant gap. Request performance dashboards and review the last twelve months of data. You should see monitoring data regularly updated. If monitoring dashboards are stale or not updated, then monitoring control is not working.

Drift Detection

Drift is a specific and common failure mode. After deployment, the distributions of inputs and outputs change. Perhaps the customer base shifts and the model receives data with different characteristics than the training data. Or the product changes and uses of the model expand to new use cases the model was never trained for. When inputs drift, the model’s assumptions are violated and performance degrades. When outputs drift, it signals the model is not performing as before.

Audit for drift detection by asking: does the system monitor for input drift? Does it monitor for output drift? Are drift alerts sent to the model owner? Has drift been detected in the past year, and if so, what happened? Some organisations define drift detection conceptually but do not implement it, or detect drift but do not act on it. Both are control failures.

Threshold and Alert Response

Even when monitoring is in place, organisations sometimes fail to respond to alerts. A model owner receives a notification that accuracy has dropped from 95 percent to 88 percent but assumes it is temporary and does not investigate. This can allow poor decisions to continue for weeks. Audit response procedures by sampling alerts from the past year. Pull alerts that were raised. For each, track what happened next: was the alert acknowledged? Was it investigated? If performance had degraded, what action was taken?

You might find alerts that were ignored, investigated too late, or addressed with a band-aid fix instead of root cause analysis. If the organisation has an incident response process, verify it was followed. If it has no incident response process, that is a finding.

Fairness Monitoring and Bias Drift

Beyond traditional performance monitoring, audit whether the organisation monitors fairness metrics in production. Does monitoring track whether the model performs equally well for all demographic groups? Or does monitoring only track overall metrics, missing that performance has degraded for a specific group? Fairness drift is a particularly harmful failure because it affects the most vulnerable populations first and often goes undetected the longest.

Retraining and Model Refresh

When performance degradation is detected, the response is usually to retrain the model on newer data. Audit for retraining procedures. How frequently does the model get retrained? Who decides when retraining is necessary? Does retraining follow the same development and validation procedures as the original model, or is it treated as a quick patch? Many organisations retrain more loosely than they built the original model, creating new risks.