JMIR Publications

ABSTRACT

Background:

The ability to automatically detect anxiety disorders from speech could be useful as a screening tool for an anxiety disorder. Prior studies have shown that individual words in textual transcripts of speech have an association with anxiety severity. Transformer-based neural networks are models that have been recently shown to have powerful predictive capabilities, based on multiple input words. Transformers detect linguistic patterns and can be separately trained to make specific predictions based on those patterns.

Objective:

To determine if a transformer-based language model can be used to screen for Generalized Anxiety Disorder from impromptu speech transcripts.

Methods:

A total of N=2,000 participants provided an impromptu speech sample in response to a modified version of the Trier Social Stress Test. They also completed the Generalized Anxiety Disorder-7 item scale (GAD-7). A transformer-based neural-network model (pre-trained on large textual corpora) was fine-tuned on the speech transcripts and the GAD-7 to predict above or below a screening threshold of the GAD-7. We report the area under the receiver operating characteristic (AUROC) on test data and compare the results with a baseline logistic regression model using the Linguistic Inquiry and Word Count (LIWC) features as input. Using the Integrated Gradient method to determine specific words that strongly affect the predictions, we infer specific linguistic patterns that influence the predictions.

Results:

The baseline LIWC-based logistic regression model had an AUROC value of 0.58. The AUROC of the fine-tuned transformer model achieved an AUROC value of 0.64. Specific words that were often implicated in the predictions were also dependent on the context. For example, the first-person singular pronoun “I,” influenced towards an anxious prediction 88% of the time while it influenced towards nonanxious 12% of the time, depending on the context. Silent pauses in speech, also often implicated in predictions, influenced the prediction of anxious 20% of the time and nonanxious 80% of the time.

Conclusions:

There is evidence that a transformer-based neural network model has increased predictive power compared to the single-word-based LIWC model. We have also shown that specific words in a specific context – a linguistic pattern – is part of the reason for the better prediction. This suggests that such transformer-based models could play a useful role in anxiety screening systems.