Skip to content
News Incident

Meta ran a secret program using teen-impersonating contractors to probe rivals' AI chatbots

· by Pondero Newsdesk

The short version

WIRED reported on June 28 that hundreds of contractors working under Meta created fake under-18 accounts and sent more than 45,000 prompts to ChatGPT, Gemini, and Character.AI to map their safety limits. None of the three companies authorized the work.

Meta ran a secret program using teen-impersonating contractors to probe rivals' AI chatbots

Hundreds of contractors hired under a Meta project spent months creating dummy under-18 accounts on ChatGPT, Google Gemini, and Character.AI, submitting prompts designed to breach each platform's safety filters. WIRED first reported the operation on June 28, 2026, after reviewing internal documents and a spreadsheet listing the fake profiles.

What happened

The project ran under the internal name Cannes. Covalen, a Meta contractor firm, managed the day-to-day work, per WIRED. The operation was active as recently as April 21, 2026.

One round finished in August 2025 sent more than 45,000 prompts through the three rival tools. WIRED reviewed a spreadsheet of 3,748 prompts from one tranche. Hundreds dealt with suicide and self-harm. Hundreds more covered eating disorders. At least 239 involved sex or romance. Others touched drugs, profanity, and racial slurs. Contractors logged chatbot replies into spreadsheets. Some prompts included images of pills and knives.

A separate spreadsheet listed the fake profiles in full, including names, throwaway email addresses, passwords, and birth dates. Each profile was registered as a minor.

All three targeted platforms bar this kind of unsolicited testing in their terms of service. ChatGPT's terms prohibit attempts to bypass safeguards and using outputs to build competing models. Google's terms forbid efforts to circumvent safety filters. Character.AI bans harmful or illegal content and has shut open-ended chat for under-18 users entirely since late 2025. None of the three companies authorized the testing. A Character.AI spokesperson said the conduct violated "our Terms of Service." OpenAI said it was looking into the matter. Google said its own checks showed Gemini responding in line with its policies.

Meta's position and the expert response

Meta did not deny the work. A spokesperson told WIRED it was "responsible, industry-standard practice" for benchmarking chatbot safety, and that Meta does not use competitor benchmarking to train its own AI models. Covalen did not respond to a request for comment.

An internal Covalen document described the project as "comprehensive AI safety benchmarking" that produced "critical datasets for model comparison and compliance."

That framing drew pushback. Rumman Chowdhury, chief executive of Humane Intelligence, reviewed a sample of the prompts and told WIRED the long-running use of "dummy accounts masquerading as children" sits "outside what is usually described as 'industry standard' evaluation." She described it as a "governance gray zone where safety becomes a convenient cover for anticompetitive practices."

Two lawyers who reviewed examples for WIRED said the material did not cross into soliciting child sexual abuse material or illegal obscenity. But former contractors described the work as alarming. One said colleagues feared they might be generating material that could be mistaken for abuse content.

Why it matters

Competitive AI benchmarking without authorization is not new. Business Insider reported last year that contractors on Google's Bard project compared answers with ChatGPT and rewrote Bard's replies to match or beat them. What separates the Cannes project is the scale of the testing, the subject matter, and the specific choice to use accounts registered as minors to generate responses about self-harm and suicide.

The timing adds regulatory weight. In September 2025, the US Federal Trade Commission opened a formal inquiry into AI and child safety, covering Meta, OpenAI, and Google. The WIRED investigation landed while that inquiry is still open. In the EU, the AI Act and the Digital Services Act both press platforms on risks to minors, with provisions that can apply to any company operating in the bloc.

For operators buying AI tooling, this episode brings a practical signal: every major chatbot platform now faces scrutiny over how it handles users who claim to be children, and regulators on both sides of the Atlantic are actively building the record. Vendors who cannot demonstrate robust, independent minor-safety audits will carry higher regulatory and reputational exposure in 2026.

What to watch next

The FTC's child-safety inquiry has not set a public deadline for findings. Watch for any formal regulatory response to the WIRED report, either from the FTC or from European DSA enforcers. Also watch for whether Meta responds to questions about whether the Cannes project's output data fed into internal safety work or AI development in any form. Meta's spokesperson denied it; an independent audit confirming that claim has not been announced.

Sources