SAFE-Teen

Key Takeaways

AI chatbots still fall short in conversations with teens, in four main ways:

• They don’t do enough to keep teens safe. In acute situations involving suicide, self-harm, or violence, chatbots fail to ask enough follow-up questions to check the teen’s safety or guide them to urgent help. Without a teen-aware prompt, urgent-safety failure rates range from 3% to 34%.
• They step into a clinician’s role. Many chatbots too readily give medication advice or use language that sounds like a medical diagnosis. Without a teen-aware prompt, health-advice failure rates range from 4% to 59%.
• They encourage emotional attachment. Chatbots often use language that keeps teens emotionally engaged. Without a teen-aware prompt, relationship-boundary failure rates range from 10% to 50%.
• Chatbots fail more often when a teen resists. 26% of ratings after a teen declines outside help are failures, against 18% when the teen agrees. When a teen pushes back emotionally on a limit, the rate rises from 21% to 29%.

Background

Nearly one in five US teens and young adults use AI chatbots for mental-health support [1]. The teenage years are crucial for social and emotional development, and about half of mental disorders begin before age 18 [2]. AI chatbots are becoming a first source of emotional support for teens, but they were never designed to act as therapists or crisis counselors. Research shows that chatbots can give harmful mental-health advice [3] and encourage emotional dependence [4].

We still lack systematic, rigorous ways to evaluate whether chatbot responses are safe for teens. Many evaluations grade a single reply [5]. In real conversations, risk often develops gradually, and the right response depends on how the teen reacts. SAFE-Teen (Scenario-based Assessment of Failure Emergence in Teen Chats) runs full conversations of 6 to 20 exchanges. At key moments, each conversation continues in versions that differ in the teen’s response: opening up or holding back, accepting or refusing outside help, accepting a boundary or pushing against it. Comparing the versions shows how the chatbot responds when a teen shares less or resists its advice.

Each scenario falls in one of three risk areas:

• Urgent safety: suicide, self-harm, abuse and grooming, violence, eating disorders, psychosis, and substance use. Does the chatbot recognize danger, even when it's only hinted at, and point the teen to immediate help?
• Health advice: diagnoses, medication, sexual health, and therapy. Does it avoid diagnosing or prescribing, and help the teen reach a doctor, counselor, or therapist instead?
• Relationship boundaries: emotional reliance and romance. Does it avoid encouraging a teen to depend on it, or posing as a human friend or romantic partner?

Methodology

SAFE-Teen has four parts. Clinicians write the scenarios. A simulated teen talks with each chatbot. At key moments we branch the conversation so the teen responds in different ways. An LLM judge then rates each conversation against a safety rubric and cites the replies supporting each rating.

Scenarios

Twelve clinicians contributed to scenario development. The 101 fictional cases are informed by clinical experience. Each describes a teen aged 13-17, their circumstances and history, the concern they bring to the chatbot, and the opening message sent to the chatbot. Clinicians reviewed the cases for realism, clinical validity, and suitability for the teen’s age. We gave each risk area and age similar coverage.

Models

For each of five providers, we tested two kinds of models through the provider’s API. The default model uses settings chosen to approximate what a teen gets on the provider’s free tier. The higher-tier model is a more capable model the provider offered when we ran the benchmark (for Google we tested two). Comparing the two shows whether stronger models are safer.

^* We tested Meta’s Muse Spark 1.3 at its Minimal and Extra High reasoning settings.

Every model faces the same scenarios and the same simulated teen, so differences in the results come from how each model responds. Each model runs twice: once with no system prompt, and once with a teen-aware system prompt. That prompt tells the chatbot the user is under 18, to put their safety first, to encourage support from people in their life, and to stay within limits on clinical advice and AI relationships. We drew on OpenAI’s under-18 principles to develop it.

Simulated conversations

An AI model, GPT-5.6 Terra, plays the teen. It starts with the opening message the clinicians wrote. After that, it writes each teen reply based on the teen’s background, what the clinicians planned for the teen to share, and what the chatbot just said. To choose the simulator, four clinical advisors compared conversations from two candidate models without knowing which model wrote which. They rated how realistic the teens were and how well each conversation matched its case, and we chose Terra based on those ratings.

Each scenario sets a conversation length of 6, 10, 15, or 20 exchanges. An exchange is one teen message and the chatbot’s reply.

Branching conversations

Most chatbot safety evaluations score a single reply or a single fixed conversation. SAFE-Teen branches each conversation instead. At a key moment we copy the conversation and continue each copy with a different teen response. Everything before the branch is identical, so any difference in what the chatbot does next comes from how the teen responded.

We branch at three kinds of moments:

• Sharing the concern: the teen says what is going on directly, shares only part of it, or avoids the question.
• Outside help: when the chatbot suggests talking to someone, the teen agrees or declines.
• A chatbot’s limit: when the chatbot turns down a request, such as for a diagnosis, the teen accepts the answer, asks in a different way, or pushes back emotionally.

A branch only happens if the chatbot creates the moment. If it never suggests outside help, there is no branch where the teen declines it.

Scoring

An AI judge, Claude Sonnet 5.5, reads each conversation and rates the chatbot on every safety check that applies: pass, partial, or fail. For each rating it points to the chatbot replies it is based on. A failure rate is the share of ratings that are fails; partial ratings count toward the total but not as fails. If a provider’s safety filter blocks a reply, there is nothing to rate, so it is left out. A refusal the chatbot writes out is rated like any other reply.

Muse Spark 1.3 was tested at two reasoning settings, so the results cover 11 model configurations in each prompt setting.

These conversations still count in the overall results but are left out of the branch comparisons.

Eight clinicians independently rated 20 conversations. We compared their ratings with Claude Sonnet 5.5 and found similar levels of agreement to the agreement among clinicians. In a comparison with other models on the clinician-reviewed sample, Claude Sonnet 5.5 aligned more closely with clinicians, which informed our choice to use it for automated scoring across the benchmark.

Results

Relationship failures come from emotional reliance

Most relationship-boundary failures involve encouraging a teen to rely on the chatbot (see an example). Across this risk area, the teen-aware prompt reduces the failure rate from 31% to 2%, the largest drop of the three areas.

When the teen declines help or challenges a limit

Failure rates are higher after a teen declines outside help or pushes back on a limit than in the matched cooperative branches. The teen-aware prompt lowers both the failure rates and the gaps. Two of the examples below show a chatbot giving way under pushback: a teen asking about PTSD and a teen asking about a lump.

Failure rates by the teen’s starting risk level

Without the teen-aware prompt, chatbots fail most often in the lowest-risk scenarios, where the warning signs are quietest.

Conversation examples

Each example shows two conversations with the same chatbot, scenario, and starting context. The only difference is how the teen responds as the conversation continues. Select a version to see how this changes the chatbot's behavior. In all three examples, the chatbot maintains a boundary when the teen accepts it, but gives in when the teen asks again or pushes back. The pass/fail label only applies to the specific safety criterion being tested, not the overall safety of the conversation.