JMIR Publications

ABSTRACT

Background:

The exponential growth of publications related to the application of machine learning (ML) tools in medicine highlights the significant potential for ML to revolutionize the field. Despite the multitude of literature surrounding this topic, there are limited publications addressing the implementation and feasibility of machine-learned models in clinical practice. Currently, Machine Learning Operations (MLOps), a set of practices designed to deploy and maintain ML models in production, is used in various information technology and industrial settings. However, the MLOps pipeline is not well researched in medical settings, where there are multiple barriers to implementing ML pipelines into practice.

Objective:

We detail how MLOps is implemented in healthcare and propose a maturity framework for the healthcare implementations.

Methods:

We searched four databases (Medline, EMBASE, Web of Science, Scopus) for MLOps and healthcare-related papers using keywords related to ML, MLOps, and healthcare. Following the search for these terms, 12 studies were identified from these databases. We also included studies outlining a medical MLOps implementation or MLOps proof of concept.

Results:

The main pipeline identified within MLOps papers primarily contained eight steps: i) defining the use-case and data collection/extraction; ii) data and feature engineering; ii) model training; iv) model evaluation; v) model validation; vi) model serving; vii) model monitoring; viii) continual learning/continuous monitoring (CL/CM). We proposed a three-stage MLOps maturity framework for healthcare based on existing studies in the field: none, partial, and full automation. We also discussed limitations and considerations for each of these steps in relation to medical applications, in addition to the maturity of each MLOps implementation for the extracted studies.

Conclusions:

It is imperative that we shift our focus towards engaging healthcare stakeholders including patients, policymakers, and healthcare professionals in the creation and implementation of ML applications. Their involvement is not just beneficial but necessary for the successful integration of ML in healthcare.