User to User, Search and Pornography Duties Explained

The Online Safety Act and AI regulations place specific duties on platforms regarding user-generated content, including search functions and pornography. These obligations apply to platforms that host or generate content, including those using AI systems. The focus of this lesson is on the responsibilities towards users, including search and pornography-related duties, as defined by these regulations.

User to User Duties

Platforms must ensure that user-generated content is monitored and managed to prevent harm. This includes addressing illegal content, such as child sexual abuse material, terrorist content, or harassment. The duty extends to platforms that allow users to interact directly through comments, forums, or messaging. For example, a social media platform must have systems in place to identify and remove posts that promote violence or hate speech. The platform must also respond to user reports and take appropriate action. The duty applies regardless of whether the content is posted by a user or generated through AI tools. The AI systems used must not amplify harmful content or fail to detect prohibited material.

• Platforms must implement effective reporting mechanisms for users to flag harmful content.
• User-generated content must be reviewed for compliance with legal requirements.
• AI tools must not be used to promote or amplify illegal or harmful content.

Search and AI-Generated Content

Search platforms have special responsibilities when displaying results that may include AI-generated or misleading content. The EU AI Act requires platforms to ensure that AI systems used in search do not mislead users or promote prohibited practices. For example, a search engine must not rank AI-generated content as if it were factual news or expert advice. The platform must also ensure that machine-readable markings are applied to AI-generated content, particularly for systems placed on the market before 2 August 2026. This obligation applies to content that is clearly identified as AI-generated or derived from AI models. The platform must make these markings visible to users through appropriate interfaces or metadata. The Digital Omnibus on AI defers some high-risk obligations to 2 December 2027, but transparency duties apply immediately from 2 August 2026.

• Search platforms must ensure AI-generated or misleading content is not presented as factual or authoritative.
• Content must be clearly marked as AI-generated or derived from AI models.
• Platforms must apply machine-readable markings by 2 December 2026 for systems placed on the market before that date.

Pornography and Age Verification

Platforms must take steps to prevent access to pornography by minors. This includes implementing age verification systems and removing or restricting access to such content. The duty applies to platforms that host or promote pornography, or those that allow users to upload such content. For example, a video-sharing platform must ensure that users cannot access or upload pornography without proper verification. The platform must also monitor for prohibited content and respond to reports. The AI systems used must not promote or recommend pornography to minors. The platform must have processes to identify and remove such content, including through automated tools or human review. The EU AI Act’s prohibited practices clause applies to systems that fail to prevent access by minors or that promote harmful content.

• Age verification must be implemented to prevent access to pornography by minors.
• Content must be reviewed and removed if it is identified as pornography or harmful to minors.
• AI systems must not promote or recommend pornography to users under 18.

Compliance with these duties requires platforms to have clear policies, effective monitoring systems, and processes for addressing user reports. The AI management system standard ISO/IEC 42001:2023 provides a framework for implementing these responsibilities. The first UKAS-accredited certification body was BSI, which began operations in January 2026. The standard helps organisations demonstrate that their AI systems meet regulatory expectations. The Digital Omnibus on AI introduces further obligations, including those related to transparency and high-risk AI systems. These obligations must be implemented progressively, with deadlines such as 2 December 2027 for high-risk systems and 2 August 2028 for embedded AI products. The penalties for non-compliance can be significant, including fines of up to 35 million EUR or 7 percent of global turnover. The focus remains on protecting users, particularly vulnerable groups such as children, through responsible AI use and content management.