Advising on AI Incidents and Regulatory Notification
When something goes wrong with an AI system, your firm needs to know how to advise clients and whether to notify regulators. This lesson covers the process.
An AI incident is any event where an AI system causes or risks causing harm. Examples include: a fabricated citation used in a court submission, a data breach where training data is exposed, an AI system that discriminates and causes harm to affected individuals, an AI recruiting system that malfunctions and incorrectly rejects qualified candidates, or an AI system that generates unsafe advice that a user acts on.
The first step when an incident occurs is assessment. What happened? Was it an error in the AI system? Was it user error? Was it a security breach? Was it a failure of human oversight? Understanding what happened is essential because it determines what notification is required.
The second step is containment. If the incident is ongoing, stop it. If an AI system is generating biased decisions, stop using it. If data has been breached, secure access. If incorrect advice has been given, contact affected clients immediately. Containment means preventing further harm.
The third step is documentation. Create a detailed record of the incident. Record what happened, when it was discovered, what steps were taken, who was involved and what the outcome was. This documentation is important for internal purposes and may be required for regulatory notification.
The fourth step is determining whether notification is required. This depends on several factors. Is the incident a personal data breach? UK GDPR requires notification to the regulator (Information Commissioner’s Office) within 72 hours if the breach creates risk of harm to affected individuals. Is the incident a financial harm event? Financial regulators might require notification. Is the incident related to professional conduct? The Law Society or Bar Standards Board might need to be notified.
For personal data breaches, the test is whether the breach creates risk of harm. Not all breaches require notification. For example, if encrypted data is breached, notification may not be required if the encryption cannot be broken. If unencrypted personal data is breached, notification is almost always required.
The timing for data breach notification is 72 hours from discovery. This is a strict deadline. Firms must have a process to identify breaches and notify the regulator within this window.
For professional conduct issues, notification is less formal but important. If an AI system caused harm to a client and the firm’s handling of it raises questions about professional competence or integrity, the firm might need to report the matter to the Law Society. Some firms take the conservative approach of self-reporting issues they discover. This reduces liability if the regulator later investigates.
The fifth step is notification. If notification is required, prepare a clear account of the incident and submit it to the appropriate regulator. For data breaches, use the ICO’s online notification system. For professional conduct issues, contact the Law Society.
The sixth step is remediation. What was done to fix the underlying problem? If the problem was an AI tool that was malfunctioning, was it replaced? If the problem was inadequate verification, were verification processes improved? If the problem was lack of client consent, was consent obtained? Remediation demonstrates that you have taken the incident seriously and have fixed the underlying issue.
The seventh step is client notification. If clients were affected by the incident, they must be told. In data breaches, individuals affected must be notified. In other incidents, clients need to know that something went wrong and what steps have been taken. The notification should be prompt, clear and honest.
The eighth step is learning. What can be learned from this incident to prevent recurrence? Should the policy be updated? Should tools be changed? Should training be improved? Incidents are opportunities to improve processes.
For your firm’s own incidents involving AI, you should advise the firm to report to the Law Society if the incident involves potential breach of professional conduct rules. For clients you advise, you should help them understand their notification obligations and timeline.