JMIR Publications

ABSTRACT

AI mental health tools increasingly market a single benchmark score as evidence of safety, despite failures that range widely in severity, from cold or insufficient responses to knowingly enabling harm. We argue that safety claims should specify which severity category they address and be supported by evidence appropriate to that category. We define a systems role in a mental health crisis as a bridge to human care rather than a treatment, introduce a four-category severity ladder for the ways a system can fail that role, and describe four types of evidence, from internal testing to real-world deployment, needed to credibly support a claim. Mapping the two together shows that current practice, a single score applied uniformly across every severity, systematically understates what has actually been demonstrated about a system's most severe failures. We propose a sequenced path for generating evidence at these categories without exposing at-risk users to unvalidated systems, and argue that vendors and buyers alike need a shared standard for what a safety claim should specify before it can be trusted.