JMIR Publications

ABSTRACT

Background:

Eligibility criteria of clinical trials are the main strategy for screening appropriate participants. Automatic analysis of eligibility criteria of clinical trials for electrical screening leveraging natural language processing techniques can actively improve the recruitment efficiency and reduce costs for promoting clinical research and practice.

Objective:

We proposes a natural language processing model to automatically classify the eligibility criteria classification for clinical trial.

Methods:

A classifier for short-text eligibility criteria based on ensemble learning was proposed, where a set of pre-trained models was integrated. The pre-trained models contain a list of state-of-the-art deep learning methods including BERT, XLNET, and Roberta, for training and classification. Then the classification results gathering by the integrated models were combined as new features for training the LightGBM classifier in the model for eligibility criteria classification.

Results:

Our proposed method obtained an accuracy of 0.846, a precision of 0.803, and a recall of 0.817 on a standard dataset from a shared task in an international conference. The macro F1 value is 0.807, outperforming the best performance in the shared task.

Conclusions:

We designed a model for screening short text classification criteria for clinical trials based on multi-model ensemble learning. Through experiments, we can conclude that in the case of model ensemble, the result is often significantly improved compared to the single model. Based on this, the introduction of Focal Loss can further reduce the impact of class imbalance and get a better result.