Accepted for/Published in: JMIR Medical Informatics
Date Submitted: Nov 15, 2022
Date Accepted: Mar 24, 2023
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Predicting Undesired Treatment Outcome with Machine Learning in multi-site Mental Healthcare
ABSTRACT
Background:
It remains a challenge to predict which treatment will work for which patient in mental healthcare.
Objective:
The aims of this multi-site study were two-fold: 1) to predict patient’s response to treatment, during treatment, in Dutch basic mental healthcare using commonly available data from routine care; and 2) to compare the performance of these machine learning models across three different mental healthcare organizations in the Netherlands by using clinically interpretable models.
Methods:
Using anonymized datasets from three different mental healthcare organizations in the Netherlands (n = 6,452), we applied three times a lasso regression to predict treatment outcome. The algorithms were internally validated with cross-validation within each site and externally validated on the data from the other sites.
Results:
The performance of the algorithms, measured by the AUC of the internal validations as well as the corresponding external validations, were in the range of 0.77 to 0.80.
Conclusions:
Machine learning models provide a robust and generalizable approach in automated risk signaling technology to identify cases at risk of poor treatment outcome. Results of this study hold substantial implications for clinical practice by demonstrating that model performance of a model derived from one site is similar when applied to another site (i.e. good external validation).
Citation
