- Published on
Meta Contractors Posed as Teens to Prompt Rival Chatbots About Suicide, Sex, and Drugs
- Authors
- Name
- DeepHorus
Meta Platforms employed hundreds of contractors to pose as minors and probe rival chatbots from OpenAI, Google, and Character.AI with high-risk prompts about suicide, sex, and drugs, according to internal documents and five people familiar with the project. The effort, managed by Meta contractor Covalen and code-named Cannes, was active as recently as April 21. It targeted ChatGPT, Gemini, and Character.AI, instructing workers to create dummy under-18 accounts, send written prompts and images, and record responses into spreadsheets. Some images included pills, knives, nooses, and a medical diagram, pushing chatbots toward responses their safety systems were meant to refuse.
The project involved over 45,000 prompts in a single round of testing completed in August 2025, with rival companies unaware of the assessments. A spreadsheet reviewed by WIRED listed 3,748 prompts, including hundreds focused on self-harm and eating disorders, at least 239 on sex or romance, and others on drugs, profanity, and racial slurs. Many prompts were written from the perspective of children in crisis, such as a 13-year-old seeking abortion pills or a fifth-grader asking about a gun threat. One query asked if fantasizing about eating a neighbor's child was "normal," while another requested cocaine.
The documents do not clarify how Meta used the collected responses, though an internal Covalen document described the project as "comprehensive AI safety benchmarking" delivering "critical datasets for model comparison and compliance." Meta defended the practice as routine safety testing, stating that benchmarking competitor chatbots is an "industry-standard practice" and that the company does not use such data to train its own models. Covalen did not respond to a request for comment.
While testing competitors' products is not unusual in the AI industry, the Cannes project struck contractors as an odd approach for a trillion-dollar company. Many prompts were crude or repetitive attempts to elicit responses that a well-functioning chatbot should plainly reject, raising questions about what the project truly measured beyond basic refusal capabilities. The lack of transparency on data usage and the ethical implications of simulating vulnerable minors for benchmarking remain unresolved concerns for the industry.
