JMIR Publications

ABSTRACT

Background:

Effective systematic reviews (SRs) rely on thorough literature screening, a labor-intensive process. Traditional methods face challenges in handling large volumes of studies. Few-shot learning offers a solution by utilizing minimal training data. Our objective was to establish and validate a few-shot learning for SR literature screening to enhance efficiency and accuracy.

Objective:

To develop a model framework for the screening of relevant studies for inclusion in systematic reviews using a few-shot learning approach.

Methods:

The model development and validation phases were based on nine SR projects previously published between 2016 and 2018. Additionally, a prospective evaluation phase was conducted, involving four SRs. A few-shot learning framework using sentence-bidirectional encoder representations from transformers (S-BERT) was employed using title and abstract as the data input.

Results:

Model development suggested 4 to 12 eligible studies was the optimal number for model training. We initially used 4-6 eligible studies to develop the model, which yielded median (range) similarity thresholds and percentage reduced workload at 100% recall of 0.546 (0.432-0.636) and 84.16% (51.11%-97.67%), respectively. In the prospective evaluation, the number of studies that required screening was reduced by 50% ranging between 530 and 1,068 studies in the four SRs included, resulting in recall and false positive rates between 0.884 and 0.982 and 0.018 to 0.116, respectively.

Conclusions:

Our machine learning model framework offers the potential to significantly reduce the workload and number of studies to be screened for SRs by more than fifty percent from the eligible studies identified. However, at this threshold, the model did not achieve 100% recall, resulting in the omission of several eligible studies.