JMIR Publications

ABSTRACT

Background:

Mood disorders have high relapse rates and existing monitoring relies on infrequent clinical assessments and self-report, limiting timely intervention

Objective:

To evaluate the feasibility of a smartphone-based system that passively infers mood from naturalistic phone conversations using speech signal processing and artificial intelligence, and to examine alignment with self-reported mood and clinical measures

Methods:

We deployed a background smartphone app to capture speech during routine calls and prompted post-call mood ratings. Encrypted features (speaker diarisation, prosody, Mel-Frequency Cepstral Coefficients (MFCCs), Word2Vec embeddings) were processed on secure servers. Phase 1 validated inferred mood against post-call self-ratings; Phase 2 compared daily mood trajectories with the Montgomery–Åsberg Depression Rating Scale (MADRS) and Early Warning Signs Questionnaire (EWSQ). The MADRS, routinely used in the hospital service, was employed to maintain continuity with clinical practice and minimise disruption to workflows.

Results:

Eleven participants completed Phase 1. The inferred mood demonstrated a moderate correlation with self-reported ratings, with performance improving as call volumes increased. The pipeline operated across heterogeneous devices and preserved privacy via feature-vector transmission

Conclusions:

Speech-based mood inference from naturalistic phone calls is feasible and aligns with subjective and clinical indicators, especially with sufficient call activity. Privacy-preserving design and multimodal features facilitate real-world deployment while promoting proactive relapse prevention. Clinical Trial: No Trial