JMIR Publications

ABSTRACT

Background:

Depression is underdiagnosed worldwide, and clinicians rely on interpreting patients' subjective speech. Qualitative analysis of patient language does not scale to routine care, and existing computational approaches describe topics with keyword lists that miss clinical nuance.

Objective:

We evaluated whether large language models (LLMs) can identify clinically relevant topics across multilingual cohorts by clustering spontaneous speech transcripts with LLM embeddings and generating fine-grained natural-language cluster descriptions for interpretability. We further examined which interview questions best elicit clinically discriminant content and how sociodemographic factors modulate cluster membership.

Methods:

We analyzed spontaneous speech transcripts from four independent cohorts totaling 2,067 participants: a French general population sample (n=1,809) and three clinical samples in Italian (n=116), Chinese (n=52), and Spanish (n=90). Responses to open-ended questions were transcribed, embedded with a multilingual language model, dimensionally reduced, grouped by density-based clustering, and each cluster was summarized by a language model into a natural-language description. Cluster membership was tested for association with validated clinical scales (PHQ-9, GAD-7, AIS, MFI, MADRS, C-SSRS) and sociodemographic factors (age, education, sex).

Results:

Unsupervised clustering yielded semantically coherent clusters significantly associated with clinical scores across all four cohorts. In the French general population, clusters discriminated depression (PHQ-9 η²=0.17, P<.001), anxiety (GAD-7), insomnia (AIS), and fatigue (MFI) scores (η² 0.13–0.14, all P<.01). In the clinical cohorts, clusters discriminated depression status in the Italian sample (MADRS; Cramér's V=0.73, P<.01), MDD diagnosis in the Chinese sample (V=0.40–0.55, P<.05), and suicide risk in the Spanish sample (C-SSRS; V=0.50, P<.01). The question "Describe how you're feeling and how your nights have been" was most effective at eliciting clinically discriminant content, with clusters distinguishing qualitatively different experiences (e.g., anxiety-driven insomnia vs. age-related nocturia). In contrast, questions about past or future events yielded small effect sizes (η²≤0.03). Age and sex were independently associated with cluster membership (η²=0.25 and 0.21, respectively, for "Describe your last 24 hours"; both P<.001).

Conclusions:

Fine-grained LLM-based topic modeling can scale qualitative analysis of patient speech across languages while preserving clinical interpretability. Certain interview questions elicit clinically discriminant content far more than others, and sociodemographic factors shape topic content independently of clinical status, an often-overlooked confound in computational psychiatry. This approach may support screening, personalized treatment planning, and culturally sensitive assessment.