JMIR Publications

ABSTRACT

Background:

Translation of artificial intelligence/machine learning (AI/ML)-based medical modeling software (MMS) into clinical settings requires rigorous evaluation by interdisciplinary teams and across the AI lifecycle. The fragmented nature of available resources to support MMS documentation limits the transparent reporting of scientific evidence to support MMS, creating barriers and impeding the translation of software from code to bedside.

Objective:

The aim of this paper is to scope AI/ML-based MMS documentation practices and define the role of documentation in facilitating safe and ethical MMS translation into clinical workflows.

Methods:

A scoping review was conducted in accordance with PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. MEDLINE (PubMed) was searched using MeSH key concepts of AI/ML, ethical considerations, and explainability to identify publications detailing AI/ML-based MMS documentation, in addition to snowball sampling of selected reference lists. To include the possibility of implicit documentation practices not explicitly labeled as such, we did not use "documentation" as a key concept but rather as an inclusion criterion. A two-stage screening process (title and abstract screening and full-text review) was conducted by an independent reviewer. A data extraction template was utilized to record publication-related information, barriers to developing ethical and explainable MMS, available standards, regulations, frameworks, or governance strategies related to documentation, and recommendations for documentation for papers that met inclusion criteria.

Results:

Of the total 115 papers, 21 (18%) articles met the requirements for inclusion. Data regarding the current state and challenges of AI/ML-based documentation was synthesized and themes including bias, accountability, governance, and interpretability were identified.

Conclusions:

Our findings suggest that AI/ML-based MMS documentation practice is siloed across the AI life cycle and there exists a gray area for tracking and reporting of non-regulated MMS. Recommendations from the literature call for proactive evaluation, standards, frameworks, and transparency and traceability requirements to address ethical and explainability barriers, enhance documentation efforts, provide support throughout the AI lifecycle, and promote translation of MMS. If prioritized across multidisciplinary teams and across the AI lifecycle, AI/ML-based MMS documentation may serve as a method of coordinated communication and reporting toward resolution of AI translation barriers related to bias, accountability, governance, and interpretability.