California SB 53 and Frontier Model Safety Reporting
California Senate Bill 53, enacted in 2024, introduces new requirements for organisations deploying artificial intelligence systems within the state. The law compels businesses to submit detailed reports to the California AI Safety Commission regarding their AI systems, including risk assessments, governance frameworks, and mitigation strategies. These reporting obligations apply to any entity that develops, deploys, or uses AI systems that may pose significant risks to individuals or society. The Commission is equipped to request additional information or conduct audits to ensure compliance.
Key Reporting Obligations
- Organisations must submit an initial report within 90 days of deploying an AI system that meets specified thresholds.
- Annual updates are required, including any changes to the system or identified risks.
- Reports must include data on system usage, outcomes, and any incidents or harms caused by the AI.
- Organisations must detail their AI governance structures, including oversight committees or designated AI stewards.
For example, a US-based fintech company operating in California must submit a report detailing its credit-scoring AI model, including how it was trained, what data was used, and any identified biases or limitations. The report must also explain how the company monitors for discriminatory outcomes or unfair treatment of applicants. The Commission may request further clarification or data, such as audit logs or explanations of algorithmic decisions.
Frontier Model Safety Reporting
The bill also introduces specific reporting requirements for “frontier AI models,” defined as large language models or other AI systems that have been trained on vast datasets and have capabilities exceeding those of standard models. These models must be reported to the Commission, including information on their training data, computational resources, and potential risks. The Commission evaluates these models to ensure they meet safety and transparency standards before being deployed or marketed in California.
Organisations deploying frontier models must submit detailed documentation, such as:
- A description of the model’s architecture and training process
- Information on data sources, including any copyrighted or sensitive data
- Details of any testing or validation performed
- Measures taken to reduce harmful outputs or biases
A US-based AI research lab that develops a new large language model must submit these details to the Commission before launching the model publicly. The Commission may then assess whether the model meets safety thresholds or requires additional safeguards. If the Commission identifies potential risks, it may require the organisation to halt deployment or make modifications before the model is released.
Organisations must also maintain records of any AI-related incidents or harms caused by their systems. These records must be retained for at least two years and provided upon request. The Commission may use these records to identify systemic issues or to develop future regulatory guidance. For example, if a healthcare AI tool misdiagnoses patients, the organisation must report this incident, including clinical data, system outputs, and any corrective actions taken.
Compliance with these reporting obligations is enforced through the California AI Safety Commission, which has the authority to issue warnings, impose fines, or require corrective actions. The Commission’s powers are aligned with those of similar EU regulatory frameworks, such as the EU AI Act, which applies to AI systems placed on the EU market from February 2025. The Commission may also collaborate with other US state regulators or federal agencies to ensure consistency in AI governance.
Organisations must ensure that their AI governance frameworks align with these reporting requirements. This includes appointing AI stewards, conducting risk assessments, and implementing monitoring systems. The ISO/IEC 42001:2023 standard provides a framework for AI management systems, which can support compliance efforts. The Commission may refer to this standard when evaluating an organisation’s AI governance practices.
As of August 2025, the Commission has begun accepting initial reports from qualifying organisations. Companies that have not yet submitted reports must do so within 90 days of deploying qualifying AI systems. The Commission’s enforcement powers, including penalties, are expected to be active from August 2026. The Commission’s approach reflects broader global trends towards AI regulation, including the EU AI Act’s provisions on transparency, AI literacy, and model safety reporting.