Evidence Types: Testing, Analysis, Process and Field Data

Types of Evidence in AI Safety Cases

When constructing safety cases for AI systems, practitioners must understand that evidence comes in various forms. The strength and reliability of evidence directly impacts the persuasiveness of safety arguments. Evidence types fall into distinct categories that serve different purposes in demonstrating system safety.

Testing evidence forms the backbone of most safety cases. This includes formal testing procedures such as unit testing, integration testing, and system testing. For example, when evaluating an AI-powered medical diagnostic tool, testing might involve clinical trials with diverse patient populations.

The testing process should cover edge cases, failure scenarios, and normal operating conditions. Test results must demonstrate that the system performs within acceptable safety margins. Testing evidence is particularly valuable when it shows consistent performance across multiple scenarios and datasets.

Analysis and Process Evidence

Analysis evidence involves systematic examination of system behaviour through mathematical proofs, formal verification, or statistical analysis. This type of evidence often appears in technical documentation and design reviews. Consider a self-driving vehicle system where formal verification techniques prove that collision avoidance algorithms meet specific safety requirements.

Analysis evidence can include computational models, risk assessments, or failure mode analyses. The key advantage of this evidence type is its ability to provide mathematical guarantees about system behaviour under specified conditions.

Process evidence documents how safety was maintained throughout development and deployment. This encompasses quality assurance procedures, change management protocols, and continuous monitoring systems. For instance, a financial services AI system might have process evidence showing regular audits, incident reporting procedures, and staff training records.

Process evidence demonstrates that safety considerations were integrated into daily operations rather than being an afterthought. The evidence should show that processes were followed consistently and that any deviations were properly documented and addressed.

Field Data and Real-World Performance

Field data represents real-world observations of system performance in operational environments. This evidence type provides insight into how systems behave outside controlled testing conditions. For example, an AI chatbot deployed in customer service might generate field data through user interaction logs, error rates, and satisfaction scores.

Field data evidence must be collected systematically and analyzed for patterns that indicate safety performance. The data should demonstrate that the system operates safely across diverse usage scenarios and user demographics.

Effective safety cases combine multiple evidence types to create a complete picture of system safety. Testing provides controlled validation, analysis offers mathematical assurance, processes ensure consistent implementation, and field data confirms real-world performance. Each evidence type serves a distinct purpose in addressing different aspects of safety concerns.

• Testing evidence should cover both normal operations and failure scenarios
• Analysis evidence must demonstrate mathematical or computational guarantees
• Process evidence shows that safety was embedded throughout development
• Field data provides real-world validation of system performance

Practitioners must ensure that evidence is relevant, reliable, and sufficient for their specific safety arguments. The quality of evidence depends on proper documentation, appropriate testing methodologies, and consistent data collection practices. Evidence should be traceable back to specific safety requirements or design constraints.

Regular updates to evidence are necessary as systems evolve or new safety concerns emerge.

When presenting evidence in safety cases, clarity and accessibility matter significantly. Technical specialists must communicate findings in ways that other stakeholders can understand. This involves explaining the significance of testing results, the limitations of analytical approaches, and the relevance of field observations.

The goal is to build confidence in the safety case through transparent presentation of evidence sources and methodologies.

Organizations should establish clear protocols for gathering, documenting, and maintaining evidence throughout the system lifecycle. This includes defining who collects evidence, how it is stored, and what standards apply to its quality. Regular reviews of evidence ensure that safety arguments remain valid as systems develop and change.

The evidence must demonstrate that safety was considered at every stage of development and deployment.