JMIR Publications

ABSTRACT

Background:

It remains a challenge to predict which treatment will work for which patient in mental healthcare.

Objective:

The aims of this multi-site study were two-fold: 1) to predict patient’s response to treatment, during treatment, in Dutch basic mental healthcare using commonly available data from routine care; and 2) to compare the performance of these machine learning models across three different mental healthcare organizations in the Netherlands by using clinically interpretable models.

Methods:

Using anonymized datasets from three different mental healthcare organizations in the Netherlands (n = 6,452), we applied three times a lasso regression to predict treatment outcome. The algorithms were internally validated with cross-validation within each site and externally validated on the data from the other sites.

Results:

The performance of the algorithms, measured by the AUC of the internal validations as well as the corresponding external validations, were in the range of 0.77 to 0.80.

Conclusions:

Machine learning models provide a robust and generalizable approach in automated risk signaling technology to identify cases at risk of poor treatment outcome. Results of this study hold substantial implications for clinical practice by demonstrating that model performance of a model derived from one site is similar when applied to another site (i.e. good external validation).