JMIR Publications

ABSTRACT

Background:

The number of artificial intelligence (AI) studies in medicine has exponentially increased recently. However, there is no clear quantification of clinical benefit when AI-assisted tools are implemented in patient care.

Objective:

We aim to systematically review all published randomized controlled trials (RCTs) of AI-assisted tools to characterize their performance in clinical practice.

Methods:

CINAHL, Cochrane Central, Embase, Medline and PubMed were searched to identify relevant RCTs comparing the performance of AI-assisted tool against conventional clinical management without AI-assistance published up to July 2021. We evaluated the primary endpoints of each study to determine which were clinically relevant.

Results:

Among 11,839 articles searched, only 38 RCTs identified were included. These RCTs were conducted in a roughly equal distribution from North America, Europe, and Asia. AI-assisted tools were implemented in 13 different clinical specialties. Most RCTs were published in the field of Gastroenterology, with 15 studies on AI-assisted endoscopy. The majority of RCTs studied image-based AI-assisted tools, and a minority of RCTs studied AI-assisted tools that drew from tabular patient. In 29 out of 38 RCTs, AI-assisted interventions outperformed usual clinical care, and clinically relevant outcomes improved with AI assisted intervention in 21 out of 29 studies. Small sample size and single-centre design limit the generalizability of these studies.

Conclusions:

There is growing evidence supporting the implementation of AI-assisted tool in daily clinical practice, yet the number of available RCTs is limited and heterogeneous. Future studies are needed to quantify the benefit of AI-assisted tools in clinical practice. Clinical Trial: This study was registered on PROSPERO (ID: CRD42021286539).