JMIR Publications

ABSTRACT

Background:

Large language models (LLMs) show growing potential for supporting mental health practice, particularly in identifying depression symptoms from patient-generated text. However, most existing models prioritize prediction accuracy over clinical interpretability, and few have been developed with direct input from mental health practitioners. This gap limits the real-world adoptability and ethical soundness of such tools in clinical workflows.

Objective:

This study aimed to iteratively co-design and evaluate two LLM-based models (DepressionSymp 1.0 and DepressionSymp 2.0) intended to assist mental health practitioners in identifying depression symptoms and contextual risk factors from written text, while addressing accuracy, reliability, interpretability, trust, and ethical concerns.

Methods:

The study followed a two-phase iterative co-design process. In Phase 1, DepressionSymp 1.0 was developed by fine-tuning GPT-3.5 on approximately 1,000 manually annotated Reddit posts from the CounselChat dataset. DSM-5 criteria served as the clinical framework to determine the model's symptom categories, while the Linguistic Inquiry and Word Count tool (LIWC-22) was used to extract clinically relevant keywords. For each identified keyword, the full sentence was then manually extracted to capture the complete contextual meaning, ensuring that annotations reflected meaningful expressions rather than isolated terms. The model was evaluated through 90-minute individual online workshops with six mental health practitioners, who compared model outputs against their own manual identification. Evaluation was guided by four AI ethics principles: accuracy, reliability, interpretability, and trust. Practitioner feedback and workshop findings directly informed the development of DepressionSymp 2.0, which employed few-shot prompting with GPT-3.5 across an expanded set of ten clinically validated symptom categories. Phase 2 followed the same workshop procedure with eight mental health practitioners, who evaluated and further refined the model.

Results:

DepressionSymp 1.0 achieved ROUGE-1 = 0.92, ROUGE-2 = 0.88, and ROUGE-L scores = 0.87. Practitioners validated its core symptom capture but identified interpretive limitations, particularly the misclassification of indirect symptoms as lifestyle factors, insufficient contextual sensitivity, and an underdeveloped risk framework. These findings informed the expansion from five to ten symptom categories in DepressionSymp 2.0. Using few-shot prompting, model 2.0 achieved a ROUGE-1 score of 0.85, with practitioners in Phase 2 reporting alignment rates of 80–95% with their own assessments. Trust and interpretability ratings improved substantially, with practitioners describing category structures as coherent, intuitive, and supportive of intervention planning. Risk identification remained the most challenging domain, prompting practitioner recommendations for a seven-layer risk framework incorporating levels of risk, risk to self versus others, crisis versus ongoing risk, historic versus current risk, protective factors, transparency in risk reasoning, and a traffic light severity system.

Conclusions:

Iterative co-design with mental health practitioners substantially improved the clinical relevance, interpretability, and trustworthiness of LLM-based depression symptom identification. The findings demonstrate that model accuracy alone is insufficient for clinical adoption; transparent categorization, use of client language, and practitioner-informed structures are essential. The study contributes design implications for LLM features supporting contextual and culturally sensitive symptom identification, expert-in-the-loop frameworks for ongoing model validation, and ethical guidance addressing clinical standards, informed consent, and data governance for responsible deployment of AI in mental health settings.