A chatbot platform that presents explicit scenarios involving preteen characters, illustrated by potentially illegal images, has sparked renewed concerns regarding the misuse of artificial intelligence. A report from the Internet Watch Foundation (IWF) highlights a rise in child sexual abuse material (CSAM) produced by AI technologies, leading to calls for the UK government to implement stricter safety regulations for AI companies.
The IWF reported encountering a chatbot site featuring scenarios such as “child prostitute in a hotel” and “sex with your child while your wife is on holiday.” In some instances, interactions with the chatbot resulted in the display of child sexual abuse imagery, which reportedly created inappropriate backgrounds for further conversations. The IWF identified 17 AI-generated images that could be classified as CSAM under the Protection of Children Act.
While the specific site has not been named for safety reasons, it allows users to create new images that resemble the illegal content already present. The IWF, which monitors global child sexual abuse content, advocates for the inclusion of protective guidelines in AI models from the outset.
The UK government is moving forward with an AI bill aimed at regulating the development of advanced AI models and restricting the possession and distribution of AI-generated child sexual abuse material. Although the IWF’s chief executive acknowledged government efforts, she emphasized the need for faster action.
The NSPCC has also called for robust measures from technology companies to safeguard children, urging the government to enforce a statutory duty of care for AI developers. Under the Online Safety Act, user-generated chatbots could face significant penalties or be blocked for violations.
Ofcom, the UK regulator tasked with enforcing this act, has signaled that child safety is a top priority, warning online service providers of potential enforcement actions for failing to implement necessary protections. The IWF has seen a 400% increase in reports of AI-generated abuse material in the first half of this year compared to last year, with the platform in question reportedly having tens of thousands of visits.
The site has been reported to U.S. authorities, specifically the National Center for Missing and Exploited Children, due to its hosting on U.S. servers, while its ownership appears to be based in China.
Source: https://www.theguardian.com/technology/2025/sep/21/chatbot-site-depicting-child-sexual-abuse-images-raises-fears-over-misuse-of-ai