JMIR Publications

ABSTRACT

Background:

Systematic reviews and meta-analyses are important to evidence-based medicine (EBM), but the information retrieval and literature screening procedures are time-consuming tasks. Rapid Medical Evidence Synthesis (RMES) is a software designed to support information retrieval, literature screening, and data extraction for EBM.

Objective:

Our objective was to evaluate the accuracy of RMES for literature screening with reference to published systematic reviews.

Methods:

We used RMES to automatically screen the titles and abstracts of PubMed-indexed articles included in 12 systematic reviews across six medical fields, by applying four filters: (1) study type; (2) study type + disease; (3) study type + intervention; and (4) study type + disease + intervention. We determined the numbers of articles correctly included by each filter relative to those included by the authors of each systematic review. Only PubMed-indexed articles were assessed.

Results:

Across the 12 reviews, the number of articles analyzed by RMES ranged from 46 to 5612. The number of PubMed-cited articles included in the reviews ranged from 4 to 47. The median (range) percentage of articles correctly labeled by RMES using filters 1–4 were: 80.9% (57.1%–100.0%), 65.2% (34.1%–81.8%), 70.5% (0.0%–100.0%), and 58.6% (0.0%–81.8%), respectively.

Conclusions:

This study demonstrated good performance and accuracy of RMES for the initial screening of the titles and abstracts of articles for use in systematic reviews. RMES has the potential to reduce the workload and time involved in the initial screening of published studies.