JMIR Publications

ABSTRACT

Background:

Suicidal behavior is a major public health problem worldwide. The exact etiology remains unclear representing a complex problem involving multiple factors. Evidence indicates that around 50-80% of people who die by suicide have had contact with the healthcare system in the year prior to their death.

Objective:

We present the IDICIUS project, whose objective is to develop a Clinical Decision Support System (CDSS) working as early warning system (EWS) that applies artificial intelligence to prevent suicide risk using anonymized electronic health record (EHR).

Methods:

The present study shows the first two stages of the IDICIUS project, where real world data (RWD) from four public sources were integrated, curated, standardized, and anonymized in phase 1, and analysis and modeling of suicide risk were performed in phase 2. This study retrospective, population-based study included 41,557 adult patients receiving mental health care at a large hospital. We evaluated the performance of state-of-the-art ML classifiers with increasing complexity: logistic regression, elastic net, decision trees and random forests, bagging and boosting ensemble methods (GradientBoosting, XGBoost, CatBoost, RandomForest, AdaBoost), support vector machines, and deep neural networks. To address class imbalance, we tested several balancing techniques, with RandomUndersampling providing the best results.

Results:

Phase 1 yielded a useful database of 32,661 mental health patients with 112 features. Of those, 2,764 patients exhibited suicidal behavior (target), while 29,897 did not (control). The undersampling reduced the class distribution to 4,422 vs. 2,211 (control vs. target). The most prevalent features in the target class were psychiatric conditions, anxiety episodes (82.1%) and depressive episodes (60.9%), along with demographic and behavioral factors: female gender (60.5%), alcohol consumption (27.9%), and conduct disorder (25.7%). The largest patient subgroups were females with combined depression and anxiety (10.9%), females with anxiety disorders only (6.6%), males with anxiety disorders only (4.3%), and males with combined depression and anxiety (3.8%). Phase 2 revealed that ensemble methods (Gradient Boosting, XGBoost, CatBoost, Random Forest, and AdaBoost) outperformed traditional approaches, achieving ROC-AUC scores around 0.95. Gradient Boosting and XGBoost emerged as the top performers. While these models showed moderate precision in identifying true positives (0.51-0.58), they demonstrated high sensitivity in detecting at-risk patients, with recall scores of 0.85 and 0.84, respectively. Both models achieved a good balance between precision and recall (F1-scores: 0.68 and 0.67). The strongest predictors were corticosteroid use, psychiatric medications (olanzapine, lorazepam), female sex, and depression.

Conclusions:

We successfully integrated multiple EHR data sources and applied artificial intelligence to develop, model, and optimize ensemble algorithms for suicide prevention, maximizing effectiveness, efficiency, and generalizability. These AI-generated algorithms, based on readily available and well-structured data, may advance the identification of at-risk patients and function as EWS, thereby increasing opportunities for timely preventive interventions.