JMIR Publications

ABSTRACT

Background:

Predicting the outcomes of individual patients for treatment interventions appears central for making mental healthcare more tailored and effective. Machine Learning (ML) has been proven to be able to make such predictions with notable accuracy. However, little work has been done to investigate the performance of such ML-based predictions within digital mental health (DMH) interventions. Implementing ML approaches in such a context would be quite easy as data is readily available for large patient populations.

Objective:

This study evaluates the performance of ML in predicting treatment outcomes in a DMH intervention designed for treating depression and anxiety.

Methods:

Several algorithms were trained based on the data of 970 patients to predict significant reduction in depression and anxiety symptoms, by using clinical and sociodemographic variables. As a Random Forest Classifier (RF) performed best over cross-validation, it was used to predict the outcomes of 279 new patients.

Results:

The RF achieved an accuracy of 0.71 for the testset (base-rate: 0.67, AUC: 0.60, P = .001, balanced accuracy: 0.60). Additionally, predicted non-responders showed less average reduction of their PHQ-9 (-2.7 , P = .004) and GAD-7 values (-3.7, P < .001) compared to responders. Besides pre-treatment PHQ and GAD values, the self-reported motivation, type of referral into the program (self versus healthcare provider) as well as Work Productivity and Activity Impairment Questionnaire (WPAI) items contributed most to the predictions.

Conclusions:

This study highlights that, also within DMH, social-demographic and clinical variables can be used for ML to predict therapy outcomes. Despite the overall moderate performance, this appears promising as these predictions can potentially improve the outcomes of nonresponders by monitoring their progress or by offering alternative or additional treatment. Behavioural patterns measured by smartphone-based interventions, such as app-usage, as well as biological data from wearable devices in DMH interventions are highlighted as paths towards improved predictive performance.