Preprint finds large gaps in how LLMs handle Gen Alpha mental health language, warns of annual crisis miss rates

A preprint on arXiv evaluates how large language models handle youth mental health language used by Generation Alpha (Gen Alpha, born 2010–2024). The authors report that models correctly interpret 76–82% of Gen Alpha vocabulary but correctly calibrate clinical risk only 64–72% of the time, creating a 10–14 percentage-point (pp) gap. This gap is not observed in human therapists, who show a 3 pp difference (p=.22). The gap widens with ambiguity, increasing from 7 pp to 18 pp.

The authors identify six failure patterns: sarcasm masking (29 pp), minimization acceptance (43 pp), informal style bias (24 pp), risk-stratified ambiguity (19 pp), semantic drift (19 pp), and context-dependent violence (7 pp). When three or more patterns compound, the miss rate reaches 94%.

The paper introduces two benchmarks: 64 Gen Alpha mental health expressions validated by native speakers and clinicians, and 75 multi-turn conversations (780 turns) with paired Standard/Gen Alpha versions. Evaluations covered LLM architectures underlying therapy apps and general chatbots, including Claude, GPT-4o, and Llama-3.1.

Lightweight mitigations were found insufficient; only heavy scaffolding achieved human-level performance at a 6.4x cost increase. At current baseline miss rates, the authors estimate 146,880 annual missed crises. They recommend mandatory human-in-the-loop architectures, quarterly youth-specific validation, transparent performance disclosure, and regulatory frameworks for youth-facing mental health AI.

The preprint is titled 'When Vocabulary Comprehension Fails Clinical Reasoning: Evaluating Therapy Bots' Safety Risks for Generation Alpha' and was submitted to ACM FAccT '26.