Intimate Image Deepfake Offences and Platform Response

Intimate image deepfake offences represent a growing concern for online platforms, particularly those hosting or generating content. These deepfakes, which create realistic but false depictions of individuals in sexual scenarios, raise serious privacy, consent, and safety issues. The Online Safety Act and related EU AI regulations place clear obligations on platforms to respond to such content. The EU AI Act, which became enforceable in February 2025, prohibits certain AI practices including those that generate or manipulate intimate images without consent. Platforms must ensure they do not host or promote such content, and must take steps to identify and remove it when discovered.

Platform Responsibilities and AI Governance

Under the EU AI Act, platforms must implement measures to prevent the misuse of AI systems, including deepfake generation tools. The Act’s Article 5 lists prohibited practices, such as the use of AI to create misleading or harmful content. This includes deepfakes that depict individuals in sexual or humiliating scenarios. Platforms must monitor their systems and content for such misuse, and must have processes in place to respond swiftly. The duty to act applies to both content hosted on the platform and content generated through AI tools that the platform makes available or supports. The AI Act also requires platforms to maintain AI governance frameworks that align with ISO/IEC 42001:2023, which sets out requirements for AI management systems. This includes documenting AI risks, implementing controls, and ensuring accountability.

• Platforms must identify and remove deepfake content that depicts individuals in sexual or harmful scenarios
• AI governance frameworks must be aligned with ISO/IEC 42001:2023
• Content moderation systems must be capable of detecting AI-generated deepfakes
• Staff must be trained on AI literacy as required by Article 4 of the EU AI Act

Content Detection, Reporting, and User Safeguards

Effective detection of deepfake content requires investment in AI-based tools and human moderation. Platforms must develop or adopt systems that can identify deepfakes through visual, audio, or metadata analysis. These tools must be trained on diverse datasets to reduce false positives and ensure accuracy. Where deepfakes are identified, platforms must have clear reporting pathways for users. This includes enabling users to flag content, providing feedback mechanisms, and ensuring that reports are reviewed promptly. User education is also important. Platforms must make it clear to users that deepfake content is prohibited, and must explain how to identify such content. The EU AI Act’s transparency obligations, which apply from 2 August 2026, require platforms to provide machine-readable markings for generative AI systems. This helps users understand when content was generated by AI, including deepfakes.

In practice, this means platforms must develop or adopt AI-based detection tools, train moderation staff, and implement user-facing reporting systems. For example, a social media platform might use machine learning models to flag deepfake content, then have human reviewers confirm the findings. The platform must also ensure that these tools are updated regularly to respond to new deepfake techniques. The Digital Omnibus on AI, which came into force in July 2026, defers some obligations such as high-risk AI requirements to later dates. However, platforms must still prepare for these future deadlines. The platform must also ensure that any AI tools used for deepfake detection or generation are compliant with EU AI Act requirements. This includes ensuring that such tools are not misused to create harmful content. The AI Act’s penalties, which can reach up to 35 million EUR or 7 percent of global turnover, make compliance essential.

Lastly, platforms must maintain records of their AI governance efforts, including how they identify, monitor, and respond to deepfake content. These records must align with ISO/IEC 42001:2023 and support any regulatory scrutiny. The certification of AI management systems through bodies such as BSI, which became the first UKAS-accredited certification body in January 2026, is an option for platforms seeking formal validation of their efforts. The focus remains on practical, effective responses to deepfake threats, aligned with evolving regulatory expectations and AI capabilities.