JMIR Publications

ABSTRACT

Background:

Patient safety incidents are a leading cause of harm in psychiatric settings, yet early warning systems tailored to mental health remain underdeveloped. Traditional risk tools like DASA-IV offer limited predictive accuracy and are reactive rather than proactive.

Objective:

We introduce the Mental Health Adverse Event Prediction (MAP) tool, a deep learning-based EWS trained on longitudinal psychiatric EMR data to anticipate adverse events in 24-hour windows.

Methods:

A retrospective and prospective cohort study using routinely collected electronic medical record (EMR) data to train and validate ML models for short-term risk prediction. This study took place at Waypoint Centre for Mental Healthcare, a large inpatient psychiatric hospital in Ontario, Canada, serving both high-secure forensic and non-forensic patient populations. A total of 4,651 patients and 403,098 encounters from January 2020 to August 2024. For prospective evaluation, the 2024 test set included 900 patients and 48,313 encounters. MAP was trained using recurrent neural networks with attention mechanisms on multivariate time-series data. The model employed an autoregressive design to forecast risk based on 7 days of prior patient data and was benchmarked against the DASA-IV clinical tool and other machine learning baselines. The primary outcome was the occurrence of an adverse mental health event recorded in the EMR within the next 24 hours. Model performance was assessed using area under the receiver operating characteristic curve (AUC) and recall, alongside subgroup analyses and interpretability assessments using integrated gradients.

Results:

The LSTM with attention mechanism achieved the highest predictive performance (AUC = 0.83), outperforming existing tools such as the DASA-IV by 20 AUC points (0.81 vs. 0.61)​​, and demonstrating the potential of ML-based models to support proactive risk management in mental health settings.

Conclusions:

MAP is one of the first prospectively evaluated deep learning-based early warning systems for psychiatric inpatient care. By outperforming existing clinical tools and providing interpretable, rolling predictions, MAP offers a pathway toward safer, more proactive mental health interventions. Future work should assess its equity implications and integration into routine psychiatric workflows.