JMIR Publications

ABSTRACT

Background:

Given the costs of machine learning implementation, a systematic approach to prioritizing which models to implement into clinical practice may be valuable.

Objective:

Primary objective was to determine the healthcare attributes respondents at two pediatric institutions rate as important when prioritizing machine learning model implementation. Secondary objective was to describe their perspectives on implementation using a qualitative approach.

Methods:

In this mixed methods study, we distributed a survey to health system leaders, physicians and data scientists at two pediatric institutions. We asked respondents to rank the following five attributes in terms of implementation usefulness: the clinical problem was common, the clinical problem caused substantial morbidity and mortality, risk stratification led to different actions that could reasonably improve patient outcomes, reducing physician workload, and saving money. Important attributes were those ranked as first or second most important. Individual qualitative interviews were conducted with a subsample of respondents.

Results:

Among 613 eligible respondents, 275 (44.9%) responded. Qualitative interviews were conducted with 17 respondents. The most common important attributes were risk stratification leading to different actions (205/275, 74.5%) and clinical problem causing substantial morbidity or mortality (177/275, 64.4%). The attributes considered least important were reducing physician workload and saving money. Qualitative interviews consistently prioritized implementations that improved patient outcomes.

Conclusions:

Respondents prioritized machine learning model implementation where risk stratification would lead to different actions and clinical problems that caused substantial morbidity and mortality. Implementations that improved patient outcomes were prioritized. These results can help provide a framework for machine learning model implementation.