AISI: the state builds an evaluation lab

The UK government established the AI Security Institute (AISI) in 2024 as a dedicated government directorate to address the growing risks associated with advanced artificial intelligence. The institute was formed through a rebrand from the AI Safety Institute, shifting focus toward national security and criminal misuse.

It has an annual budget of approximately 66 million pounds, access to over 1.5 billion pounds of compute resources, and employs more than 100 technical experts. This evolution reflects a strategic pivot in national AI governance towards proactive risk mitigation.

Core Infrastructure and Open Source Contributions

The AISI has become a central node in the global AI safety ecosystem through its open-source initiatives. It has developed and maintains the Inspect evaluation framework, which currently has over 2,400 GitHub stars and includes more than 200 pre-built evaluations. The institute also maintains the inspect_evals benchmark library, a tool used to assess the robustness of AI systems.

In collaboration with Redwood Research, it developed ControlArena, a control-experiment library for evaluating AI behaviour under controlled conditions. Additionally, AISI has open-sourced sandboxing tools designed to isolate and test AI systems safely.

Evidence-Based Research and Policy Influence

The AISI produces high-impact research that shapes both policy and public understanding of AI risks. In December 2025, it published the Frontier AI Trends Report, which documented a significant increase in apprentice-level cyber task success rates. Specifically, the report found that success in such tasks rose from 10 to 50 percent over two years, with model capabilities doubling approximately every 8 months.

In 2026, the institute co-authored the International AI Safety Report, which provided a global overview of AI safety challenges and policy responses.

Pre-Deployment Evaluation and Industry Coordination

A core function of the AISI is to evaluate frontier AI models before their release. It operates under formal agreements to carry out these assessments, ensuring that high-risk systems are vetted before deployment. For example, it published evaluations of OpenAI’s o1 model and conducted coordinated testing of Anthropic’s Claude models in collaboration with the US AI safety institute.

These evaluations are not advisory but form a critical part of the regulatory and safety infrastructure guiding responsible AI development.

Large-Scale Research Funding and Collaboration

To drive innovation in AI alignment and safety, the AISI launched the Alignment Project in July 2025. This initiative is a collaborative effort involving AWS, Anthropic, CIFAR, UKRI, and ARIA. The project has awarded over 27 million pounds in grants to 60 or more research teams.

These grants range from 50,000 to 1,000,000 pounds and support work aimed at mitigating risks in AI systems. The initiative is structured to fund both foundational and applied research in alignment and safety.

Research Output and Educational Impact

The AISI’s blog has become a primary source of technical AI safety discourse. Notable contributions include RepliBench, a study on autonomous replication (arXiv:2504.18565), and several 2026 papers focusing on lie detection, sycophancy, and misalignment entrenchment. These publications are frequently cited in academic and policy circles.

The AISI also supports educational efforts by offering practical training through its evaluation course, which provides hands-on experience with the tools and frameworks it has developed.

What to take away

The AI Security Institute plays a central role in the UK’s national AI safety strategy, combining open-source tooling, rigorous evaluation, and large-scale research funding. It contributes to global AI governance through reports and benchmarks, and its work influences both academic research and regulatory practice. The institute’s impact is evident in its infrastructure, publications, and collaborative efforts with international partners.

Reference

Sources and further reading

• Inspect framework
• Frontier AI Trends Report
• The Alignment Project