Accepted for/Published in: JMIR Formative Research
Date Submitted: Aug 28, 2022
Date Accepted: Dec 5, 2022
Using vocal characteristics to classify psychological distress in adult helpline callers: A retrospective observational study
ABSTRACT
Background:
Elevated psychological distress has demonstrated impacts on individuals’ health. Reliable and efficient ways to detect distress are key to early intervention. Artificial Intelligence has the potential to detect states of emotional distress in an accurate, efficient and timely manner.
Objective:
To automatically classify short segments of speech obtained from callers to national suicide prevention helpline services, according to low versus high psychological distress, using a range of voice biomarkers in combination with machine learning approaches.
Methods:
One hundred and twenty telephone call recordings were initially converted to 16-bit Pulse Code Modulation format. Short variable length segments of each call were rated on psychological distress using the Distress Thermometer by the responding counsellor and a second team of psychologists blinded to the initial ratings (n=6). Nineteen voice biomarkers were derived from 40ms speech frames within each segment. Candidate biomarkers were reduced using Lasso regression, validated by generalised additive mixed effects regression, accounting for non-linearity, autocorrelation and moderation by sex. Speech frames were then grouped using k-means clustering based on the selected biomarkers. Finally, component-wise gradient boosting was used to classify each speech frame according to low versus high psychological distress. Classification accuracy was confirmed via leave one out cross validation ensuring that speech segments from single callers were not used in both the training and test data.
Results:
Using twelve voice biomarkers, 686 of 747 speech segments were successfully classified, AUCROC = 93.8% (95% CI = 91.47, 94.6) and AUCPR = 94.8 (95% CI = 93.0, 96.5). When experiencing elevations in psychological distress, male speakers spoke with increasing loudness, higher frequencies in the 75th percentile of frequencies, but with poorer clarity of speech. In contrast, when experiencing psychological distress, the frequencies with which female callers spoke decreased in the highest quartile of frequencies, but exhibited increased clarity of speech.
Conclusions:
The high level of accuracy achieved suggests possibilities for real-time detection of psychological distress in helpline settings and has potential uses in pre-emptive triage. Clinical Trial: ANZCTR, ACTRN12622000486729, https://www.anzctr.org.au/ACTRN12622000486729.aspx
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.