JMIR Publications

ABSTRACT

Background:

Clinical Natural Language Processing (NLP) has become an emerging technology in healthcare that leverages a large amount of free-text data in electronic health records (EHRs) to improve patient care, support clinical decisions, and facilitate clinical and translational science research. Recently, deep learning has achieved state-of-the-art performance in many clinical NLP tasks. However, training deep learning models usually requires large annotated datasets, which are normally not publicly available and can be time-consuming to build in clinical domains. Working with smaller annotated datasets is typical in clinical NLP and therefore, ensuring that deep learning models perform well is crucial for the models to be used in real-world applications. A widely adopted approach is fine-tuning existing Pre-trained Language Models (PLMs), but these attempts fall short when the training dataset contains only a few annotated samples. Few-Shot Learning (FSL) has recently been investigated to tackle this problem. Siamese Neural Network (SNN) has been widely utilized as an FSL approach in computer vision, but has not been studied well in NLP. Furthermore, the literature on its applications in clinical domains is scarce.

Objective:

To propose an FSL training and evaluation algorithm and to compare the performance of the proposed approach to that of baseline transformer models.

Methods:

We propose two SNN-based FSL approaches for clinical NLP, including pre-trained SNN (PT-SNN) and SNN with second-order embeddings (SOE-SNN). We evaluated the proposed approaches on two clinical tasks, namely clinical text classification and clinical named-entity recognition. We tested three few-shot settings including 4-shot, 8-shot, and 16-shot learning. Both clinical NLP tasks were benchmarked using three PLMs, including Bidirectional Encoder Representations from Transformers (BERT), Bidirectional Encoder Representations from Transformers for Biomedical Text Mining (BioBERT), and BioBERT trained from clinical texts (BioClinicalBERT).

Results:

FSL-based approach outperformed baseline transformer models in both tasks. For the sentence classification task, the performance difference between the proposed approach and baseline transformer models was statistically significant. For the named-entity recognition task, while the difference in performance was not statistically significant, it was still noticeable.

Conclusions:

The experimental results verified the effectiveness of the proposed SNN-based FSL approaches in both clinical NLP tasks. FSL-based approaches outpeformed baseline models in both sentence classification and NER.