Procurement, Supplier Testing and Independent Evaluation

Procurement Considerations for AI Tools

When purchasing artificial intelligence systems for policing purposes, organisations must carefully evaluate supplier claims and technical specifications. The procurement process should include detailed assessment of how algorithms make decisions, what data they use, and how they handle edge cases. For example, a facial recognition system must demonstrate accuracy across diverse demographic groups rather than just meeting a single performance metric.

• Request detailed technical documentation including algorithmic architecture and data processing methods
• Ensure suppliers provide evidence of testing across varied demographic populations
• Verify that systems comply with relevant data protection legislation including GDPR
• Confirm supplier commitments to ongoing monitoring and system updates

Organisations should also examine supplier governance frameworks and their approach to addressing bias or errors. A supplier who cannot demonstrate how they identify and correct algorithmic bias presents significant risk. The procurement contract must clearly specify performance requirements, including accuracy thresholds, false positive rates, and response times.

These metrics should reflect real-world operational conditions rather than laboratory settings.

Supplier Testing Requirements

Before deployment, AI systems must undergo rigorous testing that mirrors actual policing environments. Testing should include scenarios that reflect typical operational challenges such as poor lighting conditions, partial occlusion, or diverse demographic representation. For instance, a predictive policing tool should be tested using historical data that reflects current community demographics rather than outdated patterns.

• Conduct testing in environments that replicate real operational conditions
• Include diverse demographic samples in all testing phases
• Test edge cases and unusual circumstances that may occur in practice
• Verify system performance under various environmental conditions

Independent testing by third-party organisations provides additional assurance. These tests should be conducted by specialists with expertise in both AI systems and policing contexts. The testing process must include validation of accuracy claims, assessment of bias mitigation measures, and evaluation of system reliability over extended periods.

Test results should be publicly available or accessible to oversight bodies to maintain transparency.

Organisations must also verify that suppliers maintain appropriate data security measures. This includes encryption protocols, access controls, and data retention policies. The supplier’s approach to data minimisation and purpose limitation directly affects compliance with data protection principles.

Testing should confirm that data is not retained longer than necessary and that access is appropriately restricted.

Independent Evaluation Processes

Independent evaluation involves third-party assessment of AI systems after procurement but before full deployment. These evaluations focus on operational effectiveness, legal compliance, and potential risks to individuals. The evaluation team should include specialists from diverse backgrounds including legal experts, ethicists, and community representatives where possible.

• Engage independent specialists with relevant expertise
• Ensure evaluation covers both technical and operational aspects
• Include assessment of potential discriminatory impacts
• Verify compliance with existing legal frameworks

Organisations should establish clear evaluation criteria that align with operational needs and legal requirements. These criteria might include accuracy rates, false positive and false negative outcomes, system reliability, and user experience. The evaluation process must be documented and results shared with relevant stakeholders including oversight committees and senior management.

Regular re-evaluation of deployed systems remains essential. AI tools should undergo periodic assessment to ensure continued effectiveness and compliance. This includes monitoring for degradation in performance, identification of new bias patterns, and verification that systems continue to meet operational requirements.

For example, a biometric identification system may perform well initially but show declining accuracy over time due to changing demographic patterns or environmental factors.

The evaluation process should also consider feedback from frontline officers who use these systems daily. Their practical experience provides valuable insights into real-world effectiveness and potential issues that may not appear in controlled testing environments. Regular feedback loops between operational staff and technical teams help identify problems early and ensure systems remain fit for purpose.

Organisations must maintain records of all evaluation activities and decisions made based on these assessments. These records support accountability and provide evidence of due diligence if questions arise about system performance or compliance. The evaluation framework should be flexible enough to accommodate evolving threats and changing operational requirements while maintaining consistent standards for AI tool assessment.