Accepted for/Published in: Journal of Medical Internet Research
Date Submitted: Jul 14, 2020
Date Accepted: Nov 30, 2020
Date Submitted to PubMed: Dec 2, 2020
Deep neural network-based machine learning reduces the screening workload for systematic review: Investigation based on recent clinical guidelines
ABSTRACT
Background:
Performing systematic reviews is a time-consuming and resource-intensive process.
Objective:
We investigated whether a machine learning system could perform systematic reviews more efficiently.
Methods:
All systematic reviews and meta-analyses of interventional randomized controlled trials cited in recent clinical guidelines from the American Diabetes Association, American College of Cardiology, American Heart Association (x2), and American Stroke Association were assessed. After reproducing the primary screening dataset according to the published search strategy, we extracted correct articles (actually reviewed) and incorrect articles (not reviewed) from the dataset. Next, these two sets of articles were used to train a neural network-based artificial intelligence engine (Concept Encoder, FRONTEO Inc.). The primary endpoint was work saved over sampling at 95% recall.
Results:
Among 145 candidates, 8 reviews of randomized controlled trials fulfilled the inclusion criteria. Using these 8 reviews, the machine learning system significantly reduced the literature screening workload by at least 6-fold versus manual screening. When machine learning was initiated using two correct articles randomly selected by a researcher, it achieved a 10-fold reduction in workload versus manual screening with high sensitivity for eligible studies. The area under the receiver operating characteristics curve increased dramatically every time the algorithm learned a correct article.
Conclusions:
The Concept Encoder achieved a 10-fold or better reduction of the screening workload for systematic review after learning two randomly selected studies on the target topic. However, few meta-analyses of randomized controlled trials were included. The Concept Encoder could facilitate the acquisition of evidence for clinical guidelines. Clinical Trial: UMIN000032663
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.