JMIR Publications

ABSTRACT

Background:

This study aims to detect self-harm and/or suicidal ideation (SH/S) language used by youth (ages 13–21) in their private Instagram conversations. While automated mental health tools have shown promise, there remains a gap in understanding how nuanced youth language around SH/S can be effectively identified. Our work focuses on developing interpretable models that go beyond binary classification to recognize the spectrum of SH/S expressions.

Objective:

Our work focuses on developing interpretable models that go beyond binary classification to recognize the spectrum of SH/S expressions.

Methods:

We analyzed a dataset of Instagram private conversations donated by youth. A range of traditional machine learning models (SVM, Random Forest, Naive Bayes, XGBoost) and transformer-based architectures (BERT, DistilBERT) were trained and evaluated. In addition to raw text, we incorporated contextual, psycholinguistic (LIWC), sentiment (VADER), and lexical (TF-IDF) features to improve detection accuracy. We further explored how increasing conversational context—from message-level to sub-conversation level—affected model performance.

Results:

DistilBERT demonstrated a good performance in identifying the presence of SH/S behaviors within individual messages, achieving an accuracy of 99%. However, when tasked with a more fine-grained classification—differentiating among “Self” (personal accounts of self-harm or suicide), “Other” (references to SH/S experiences involving others), and “Hyperbole” (sarcastic, humorous, or exaggerated mentions not indicative of genuine risk)—the model's accuracy declined to 89%. Notably, by expanding the input window to include a broader conversational context, the model's performance on these granular categories improved to 91%, highlighting the importance of contextual understanding when distinguishing between subtle variations in SH/S discourse.

Conclusions:

Our findings underscore the importance of designing SH/S automatic detection systems sensitive to the dynamic language of youth and social media. Contextual and sentiment-aware models improve detection and provide a nuanced understanding of SH/S risk expression. This research lays the foundation for developing inclusive and ethically grounded interventions, while also calling for future work to validate these models across platforms and populations.