JMIR Publications

ABSTRACT

Background:

Language, which is a crucial element of human communication, is influenced by the complex interplay between thoughts, emotions, and experiences. Psychiatric disorders have an impact on cognitive and emotional processes, which in turn affect the content and way individuals with these disorders communicate using language. The recent rapid rise in performance of large language models (LLMs) suggests that leveraging them for quantitative analysis of language usage has the potential to become a useful method for objectively diagnosing and distinguishing between various psychiatric disorders.

Objective:

Here, we present a novel application of LLMs to analyze patterns of language usage from spontaneous communication related to psychiatric disorders.

Methods:

We utilized embeddings derived from the GRIT-7b LLM for posts originating from subreddits dedicated to seven common conditions: schizophrenia (Scz), borderline personality disorder (BPD), depression (Dep), attention deficit hyperactivity disorder (ADHD), anxiety (Anx), post-traumatic stress disorder (PTSD) and bipolar disorder (Bip). Using these embeddings, we trained a cross-validated, multi-class XGBoost classifier to label which subreddit each post came from.

Results:

The 10-fold cross-validated classifier has weighted average precision, recall, F1, and accuracy scores were of 0.73, 0.68, 0.70 and 0.73, respectively, for the multi-class objective. In a one-versus-rest task, individual category AUCs ranged from 0.89-0.97, with an micro-average one-versus-rest AUC of 0.95.

Conclusions:

Our results suggest that LLMs can provide valuable insights into the linguistic patterns that distinguish between different psychiatric disorders, and that these patterns can be used to develop more objective, efficient, and patient-centered strategies for assessment, monitoring, and research.