JMIR Publications

ABSTRACT

Background:

Artificial intelligence (AI) offers potential solutions to address the challenges faced by a strained mental healthcare system, such as increasing demand for care, staff shortages and pressured accessibility. While developing AI-based tools for clinical practice is technically feasible and has the potential of producing real-world impact, only few are actually implemented into clinical practice. Implementation starts at the algorithm development phase, as this phase bridges theoretical innovation and practical application. The design and the way the AI tool is developed may either facilitate or hinder later implementation and use.

Objective:

This qualitative study examines the challenges encountered by a development team in creating a suicide risk prediction algorithm using real-world electronic health record (EHR) data for clinical use. It explores how the team addressed these challenges and identifies key considerations for bridging technical and clinical needs in algorithm development. This work aims to contribute to the body of knowledge concerning the effective integration of AI into mental healthcare.

Methods:

An exploratory, multimethod case study was conducted, employing a hybrid approach with both inductive and deductive analysis. Data were collected through desk research, reflective team meetings, and iterative feedback sessions with the development team. Thematic analysis was used to identify development challenges and the team’s responses, as well as to derive key considerations for future implementations.

Results:

Key challenges included defining, operationalizing, and measuring suicide incidents within EHRs due to issues such as missing data, underreporting, and differences between data sources. Predicting factors were identified by consulting clinical experts, however, psychosocial variables had to be constructed as they could not directly be extracted from EHR data. Risk of bias occurred when traditional suicide prevention questionnaires, unequally distributed across patients, were used as input. Analyzing unstructured data by Natural Language Processing (NLP) was challenging due to data noise but ultimately enabled successful sentiment analysis, which provided dynamic, clinically relevant information for the algorithm. A complex model (XGBoost) enhanced predictive accuracy but posed challenges regarding understandability, which was highly valued by clinicians.

Conclusions:

To advance mental healthcare as a data-driven field, several critical considerations must be addressed: ensuring robust data governance and quality, fostering cultural shifts in data documentation practices, establishing mechanisms for continuous monitoring of AI tool usage, mitigating risks of bias, balancing predictive performance with explainability, and maintaining a clinician "in-the-loop" approach. Future research should prioritize sociotechnical aspects related to the implementation and daily use of AI in mental healthcare practice.