JMIR Publications

ABSTRACT

Background:

Prevention of suicide is a global health priority. Around 800,000 individuals die by suicide yearly, and for every death, there are another 20 estimated suicide attempts. Large language models (LLMs) hold the potential to enhance scalable, accessible, and affordable digital services for suicide prevention and self-harm interventions. However, their use also raises clinical and ethical questions that require careful consideration.

Objective:

This scoping review aimed to identify emergent trends in applications of LLMs within the field of suicide and self-harm research. Additionally, it summarizes key clinical and ethical considerations relevant to this nascent area of research.

Methods:

Searches were conducted in four databases. Eligible studies described the application of LLMs for suicide or self-harm prevention, detection, or management. English-language peer-reviewed articles and conference proceedings were included, with no date restrictions. This review adhered to PRISMA-ScR standards.

Results:

Of the 533 studies identified, 36 met inclusion criteria, and an additional 7 more were identified through citation chaining, resulting in a total of 43 studies for review. A narrative synthesis approach was used to synthesize study characteristics, objectives, models, data sources, proposed clinical applications, and ethical considerations. Studies showed a bifurcation of publication fields with varying publication norms between computer science and mental health. While most studies (77%) focused on identifying suicide risk, newer applications leveraging generative functions (e.g., support, education, and training) are emerging. Social media was the most common source of LLM training data. BERT (Bidirectional Encoder Representation Transformer) was the predominant model used, although GPT (Generative Pre-trained Transformer) featured prominently in generative applications. Clinical applications of LLMs were reported in 60% of studies, often for suicide risk detection or as clinical assistance tools. Ethical considerations were reported in 33% of studies, with privacy, confidentiality, and consent strongly represented.

Conclusions:

This evolving research area, bridging computer science and mental health, demands a multi-disciplinary approach. While open access models and datasets will likely shape this field, documenting their limitations and potential biases is crucial. High-quality training data is essential for refining these models and mitigating unwanted biases. Policies that address ethical concerns – particularly related to privacy and security when using social media data – are imperative. The emergence of generative AI signals a shift in approach, particularly in applications related to care, support, and education. Ongoing human oversight, whether through human-in-the-loop testing or expert external validation, is essential for responsible development and use.