JMIR Publications

ABSTRACT

Background:

Machine learning systems in healthcare have the potential to enhance decision-making but often fail to address critical issues such as prediction explainability, confidence, and robustness in a context-based and easily interpretable manner.

Objective:

This study aims to design and evaluate a decision support system (DSS) for clinical psychopathological treatment assessments incorporating a novel machine learning (ML) model that is inherently interpretable and transparent. It aims to enhance clinical explainability and trust through a transparent, hierarchical model structure that progresses from questions to scores to classification predictions. Model confidence and robustness are addressed by applying Monte Carlo Dropout (MCD), a probabilistic method that reveals model uncertainty and confidence.

Methods:

A DSS for clinical psychopathological treatment assessments was developed, incorporating a novel ML model structure. The DSS was aimed at enhancing graphical interpretation of the model outputs. The model was aimed at addressing issues of prediction explainability, confidence, and robustness. Using patient questionnaire answers and demographics from an online treatment service in Denmark (n=1088), the proposed ML model was trained and validated.

Results:

The five-fold cross validation accuracy was: 0.75 (0.019). The precision was greater than or equal to 76% for all four prediction classes (depression, panic, social phobia, specific phobia). The area under the curve (AUC) for the four classes was: 0.94, 0.91, 0.91, 0.96 respectively.

Conclusions:

A mental health treatment DSS has been demonstrated that supports a graphical interpretation of the prediction class probability distributions. Their spread and overlap can inform the clinician of competing treatment possibilities for the patient and uncertainty in the treatment prediction. With the ML model achieving greater than 75% accuracy, it is expected that the model will be clinically useful in both screening new patients and informing clinical interviews.