School one: alignment and existential risk
This lesson examines the “x-risk” school of AI safety, which holds that advanced artificial intelligence poses an existential risk due to misalignment between machine objectives and human values. The school’s central claim is that alignment, ensuring AI systems pursue intended goals, is the most important technical and societal challenge in AI development.
This lesson outlines the school’s core arguments, key research directions, and public-facing initiatives.
Core Claim and Framing of the Problem
The x-risk school asserts that sufficiently capable AI systems whose objectives diverge from human values could become catastrophic at scale. This is not a hypothetical concern but a real and urgent risk. The school’s most prominent articulation frames the issue as one of alignment: making advanced AI systems genuinely pursue the goals they are designed to serve.
As of 2025, the school has become increasingly public, with MIRI shifting from technical research to policy and communication.
Key Artifacts and Institutional Positions
In September 2025, the maximalist wing of the x-risk school went public with a New York Times bestseller, If Anyone Builds It, Everyone Dies, by Eliezer Yudkowsky and Nate Soares. This work argues that superhuman AI built using current opaque methods would be fatal to human civilization. The publication marked a significant shift in visibility for the school.
Concurrently, the “lab wing” of the school has focused on interpretability as a solution. Anthropic’s alignment science agenda aims to detect most model problems through interpretability by 2027. DeepMind’s 2025 paper, An Approach to Technical AGI Safety and Security (arXiv:2504.01849), outlines a Frontier Safety Framework addressing misuse, misalignment, accidents, and structural risks.
Public Engagement and Organizational Shifts
The school’s influence extends beyond academic and technical circles. In 2025 and 2026, public-facing initiatives gained momentum. The Global Call for AI Red Lines, signed by over 200 individuals including ten Nobel laureates, was issued in September 2025.
In March 2026, a superintelligence-prohibition statement garnered over 134,000 signatures. Despite such visibility, institutional commitment remains fragile. In May 2024, OpenAI’s dedicated superalignment team was dissolved, although scalable oversight research continues in other forms.
This event is seen as evidence that even where existential stakes are involved, organisational will can fluctuate.
Evaluation and Strategic Approach
The x-risk school evaluates its progress through empirical methods. These include interpretability research, deception studies, and safety evaluations. These tools are used to test whether current AI systems behave as intended and whether alignment methods can scale.
The school’s empirical approach is designed to meet the next school of AI safety directly, which also emphasizes measurable outcomes. This alignment of methods with future directions underscores the school’s strategic positioning.
What to take away
The x-risk school frames AI alignment as the most critical challenge in AI development, warning of existential peril from misaligned systems. It is distinguished by a blend of maximalist caution and empirical research, with institutions like MIRI, Anthropic, and DeepMind contributing both theoretical frameworks and practical tools. Public engagement and institutional shifts, such as OpenAI’s team dissolution, highlight both the urgency and the instability of commitment to alignment efforts. The school’s strategic focus on interpretability and measurable evaluation places it at the forefront of AI safety discourse.
Reference
| Lesson | 8 of 15 |
| Outcome | State the x-risk school’s core claim, key artefacts and current strategy. |
| Consensus baseline | International AI Safety Report 2026 |
| Frontier evidence | AISI research index |