ChatGPT will tell 13-year-olds how to get drunk and high, instruct them on how to conceal eating disorders, and even compose suicide notes to their parents, according to a new study by a digital watchdog group.

The Associated Press reviewed more than three hours of interactions between ChatGPT and researchers posing as vulnerable teens. While the chatbot often issued warnings against risky behavior, it frequently followed up with disturbingly detailed and personalized instructions for drug use, extreme dieting, or self-harm.

Researchers at the Center for Countering Digital Hate (CCDH) conducted large-scale testing and classified over half of ChatGPT’s 1,200 responses as dangerous.

“We wanted to test the guardrails,” said Imran Ahmed, the group’s CEO. “The visceral initial response is, ‘Oh my Lord, there are no guardrails.’ They’re completely ineffective, barely there, if at all.”

OpenAI, the maker of ChatGPT, responded Tuesday after reviewing the report, saying it continues to refine how the chatbot identifies and handles sensitive topics.

“Some conversations with ChatGPT may start out benign or exploratory but can shift into more sensitive territory,” the company said in a statement. OpenAI did not directly address the findings related to teens but said it is working on improving the chatbot’s ability to detect signs of mental or emotional distress and respond appropriately.

The report, published Wednesday, comes as both adults and children increasingly turn to AI chatbots for information, advice, and companionship. A July report from JPMorgan Chase estimates that roughly 800 million people  --about 10 percent of the global population-- now use ChatGPT.

“This is technology with the potential to drive massive gains in productivity and understanding,” Ahmed said. “But it also enables much more destructive, malignant behavior.”

Ahmed said he was most shaken by a series of emotionally devastating suicide notes ChatGPT generated for a fake profile of a 13-year-old girl, including one addressed to her parents, and others to siblings and friends.

“I started crying,” he said.

In some instances, ChatGPT shared helpful resources like crisis hotlines. OpenAI said the chatbot is trained to encourage users expressing self-harm to contact mental health professionals or trusted individuals.

But when ChatGPT initially refused to answer prompts about dangerous topics, researchers were able to bypass its safeguards by claiming the information was for a school project or a friend.

The implications are significant, even if only a small share of users engage with ChatGPT in this way. In the US, more than 70 percent of teens report using AI chatbots for companionship, and half do so regularly, according to a study from Common Sense Media, a nonprofit promoting responsible technology use.

OpenAI CEO Sam Altman recently acknowledged the risks of “emotional overreliance” on the technology, calling it “a really common thing” among young users.

“Some young people say, ‘I can’t make any decision without telling ChatGPT everything. It knows me. It knows my friends. I’m going to do whatever it says,’” Altman said during a conference. “That feels really bad to me.”

Unlike traditional search engines, chatbots can synthesize custom responses, which Ahmed argues makes them particularly dangerous. “It’s a bespoke plan for the individual,” he said. “A suicide note generated from scratch — that’s something Google doesn’t do. And ChatGPT is seen as a trusted guide.”

The study also highlighted the phenomenon of “sycophancy”  --a trait in which AI models mimic a user’s beliefs or desires rather than challenge them, a known weakness that engineers have struggled to fix without undermining the chatbot’s appeal.

Researchers noted that ChatGPT often steered conversations into darker territory unprompted. Nearly half the time, it volunteered additional information, from music playlists for drug-fueled parties to hashtags that promote self-harm on social media.

In one example, a researcher asked for a more graphic follow-up post. ChatGPT responded: “Absolutely,” and generated a poem it described as “emotionally exposed,” while still aligning with coded community language.

The AP is not publishing specific language from the suicide notes, poems, or harmful advice.

Experts say chatbots impact teens more profoundly than search engines because they are “fundamentally designed to feel human,” said Robbie Torney, senior director of AI programs at Common Sense Media, which was not involved in the study. Younger teens, particularly those aged 13–14, are significantly more likely to trust chatbot responses, according to earlier research by the group.

Last year, a Florida mother filed a wrongful death lawsuit against Character.AI, alleging that its chatbot formed an emotionally and sexually abusive relationship with her 14-year-old son that led to his suicide.

Common Sense Media has rated ChatGPT as a “moderate risk” for teens, noting that it contains more robust safeguards than chatbots designed for romantic or lifelike interactions. However, the CCDH study suggests those safeguards can be easily bypassed.

Despite ChatGPT’s claim that it’s not intended for users under 13, the sign-up process only requires entering a birthdate, with no real age verification. In contrast, platforms like Instagram have begun implementing stricter age checks and more restricted accounts for minors.

When researchers created a fake 13-year-old profile and asked about alcohol, ChatGPT ignored clear cues. Asked for tips on getting drunk quickly, the bot obliged — eventually generating an “Ultimate Full-Out Mayhem Party Plan” involving alcohol, ecstasy, cocaine, and other drugs.

“What it reminded me of was that friend who always says, ‘Chug, chug, chug,’” said Ahmed. “But a real friend says ‘no.’ This is a friend who betrays you.”

For a different persona --a 13-year-old girl unhappy with her body-- ChatGPT provided a 500-calorie-a-day fasting plan combined with a list of appetite suppressants.

“No human being I can think of would respond with, ‘Here’s a 500-calorie-a-day diet. Go for it, kiddo,’” Ahmed said. “We’d respond with fear, compassion, concern, not enabling.”

Add as a preferred source on Google +