Validation by Someone Who Did Not Build the Model
The person who built the model is deeply invested in its success. Unconscious bias can lead model developers to overlook problems or interpret ambiguous evidence as supporting deployment. Independent validation by someone with fresh perspective, no stake in deployment and authority to block deployment is essential for defensible model governance.
For example, a data scientist building a credit scoring model may unconsciously favour features that align with their own assumptions about creditworthiness, even if those features are not statistically dependable. An independent reviewer can challenge those assumptions and ensure the model is not biased against certain demographics or geographic regions.
Independent validation means a different person or team conducts testing and evaluation. Ideally this person was not involved in model development and has authority to reject the model if they find problems. In small organisations, independent validation might mean a peer review from another data scientist who did not work on the project.
In larger organisations, it might mean a dedicated model risk team or compliance team with sufficient authority and expertise. A junior analyst in a fintech company, for instance, may be asked to review a model created by a senior team. This junior analyst can bring a fresh perspective and challenge assumptions that the senior team may have overlooked due to their deep involvement in the project.
Independent validation should cover multiple areas: checking that the model was built using sound data science practices, verifying that the dataset is appropriate and representing the target population, checking that fairness testing was thorough and interpreted correctly, confirming that the model’s assumptions match the real-world use case, and making an independent decision about whether the model should be deployed.
A healthcare AI model designed to predict patient readmission must be validated not only for accuracy but also for fairness across different age groups and ethnicities. A validator must ensure that the model does not disproportionately flag patients from certain backgrounds, which could lead to discriminatory treatment.
The validator should have access to all documentation: the problem statement, data sources, choice of metrics and why they were chosen, training methodology, fairness analysis and test results. They should be able to ask questions and request additional analysis. The validator is not a rubber stamp.
A validator reviewing a fraud detection model must be able to examine how the model was trained on historical transaction data and whether it was tested on a representative sample of fraud cases. They must also be able to challenge the choice of metrics, such as whether precision or recall is more critical in this specific use case.
A common mistake is treating validation as a checkbox exercise. The validator glances at a summary report and approves the model without asking hard questions. True validation is sceptical and thorough.
A good validator asks: “How do you know the data is representative? How do you know this metric is the right one? What if this key assumption is wrong?
What if this subgroup performs much worse?” For example, a model predicting employee performance may appear accurate overall, but a validator must probe whether it performs equally well across different departments or job roles. If performance varies significantly, it may indicate a hidden bias that was not caught during initial development.
Documentation makes validation possible. If there is no written record of why you chose certain data or metrics, an independent validator cannot assess whether choices were sound. If there is no documented record of fairness testing, the validator cannot verify it happened.
Without documentation, the validator is forced to trust your word, which is not true independent validation. A model used in recruitment must have clear documentation of how the training data was collected and whether it reflects the diversity of the applicant pool. Without such documentation, a validator cannot determine whether the model is prone to gender or racial bias.
A lack of transparency in model development can lead to regulatory penalties and reputational damage.
Effective independent validation also requires clear communication between the model creator and the validator. A model developer must be prepared to explain their reasoning and justify their decisions. For instance, a model used in customer churn prediction must be validated not only for its predictive accuracy but also for its alignment with business objectives.
A validator must understand how the model will be used in practice and whether it aligns with the company’s ethical guidelines. This process can be enhanced by using standardised validation frameworks such as those provided by the UK’s Financial Conduct Authority or the European Union’s AI Act, which outline specific requirements for model transparency and accountability.
Organisations should also consider the timing of validation. Independent validation should occur before model deployment, not after. A model that has already been implemented and is producing results may be difficult to revise, especially if it is already integrated into critical business processes.
A validator must be able to challenge the model at the point of development, before it is too late to make changes. For example, a model used in a lending platform must be validated before it is rolled out to customers. If the model is found to be biased or inefficient after deployment, it can lead to customer dissatisfaction and regulatory scrutiny.
Therefore, validation must be an integral part of the model development lifecycle, not an afterthought.
Finally, the validator must be equipped to take action. They must have the authority to recommend rejection or modification of the model. A validator who is merely asked to review a model and provide feedback without the ability to influence the outcome is not truly independent.
In a large financial institution, for example, a model risk team must be able to halt deployment if they identify critical flaws. This capability building ensures that validation is not just a formality but a meaningful safeguard against flawed or biased models. Without this authority, even the most thorough validation process can fail to protect the organisation from unintended consequences.