JMIR Publications

ABSTRACT

Background:

A barrier to practicing evidence-based medicine is the rapidly increasing body of biomedical literature. Use of method terms to limit the search can help reduce the burden of screening articles for clinical relevance, however, it is limited by their partial dependence on indexing terms and usually produces low precision, especially when high sensitivity is required. Machine learning has been applied to the identification of high-quality literature with the potential to achieve high precision without sacrificing sensitivity. The use of artificial intelligence has shown promise to improve efficiency for identifying sound evidence.

Objective:

The primary objective of this research is to derive and validate deep learning machine models using iterations of Bidirectional Encoder Representations from Transformers (BERT) to retrieve high-quality, high relevance evidence for clinical consideration from the biomedical literature.

Methods:

Using the HuggingFace transformers library, we will experiment with variations of BERT models, including BERT, BioBERT, BlueBERT, and PubMedBERT, to determine which have the best performance in article identification based on quality criteria. Our experiments will utilize a large dataset of over 150,000 PubMed citations which have been manually labeled based on their methodological rigor for clinical use. We will evaluate and report on the performance of the classifiers in categorizing articles based on their likelihood of meeting quality criteria.

Results:

Model performance characteristics will include recall (sensitivity), specificity, precision, accuracy, F-score, and classification probability scores.

Conclusions:

The experiments will be devised to improve the precision of the retrieval of high-quality articles by applying a machine learning classifier to PubMed searching.