Accepted for/Published in: JMIR Medical Informatics
Date Submitted: Jul 28, 2020
Date Accepted: Nov 20, 2020
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Extracting Family History of Patients from Clinical Narratives: Using Deep Learning Models
ABSTRACT
Background:
Patients’ family history (FH) is a critical risk factor associated with numerous diseases. However, FH information is not well captured in the structured database but often documented in clinical narratives. Natural language processing (NLP) is the key technology to extract patients’ FH from clinical narratives. In 2019, the National NLP Clinical Challenge (N2C2) organized shared tasks to solicit NLP methods for FH information extraction.
Objective:
This study presents our end-to-end FH extraction system developed during the 2019 N2C2 challenge as well as the new transformer-based models that we developed after the challenge.
Methods:
We developed deep learning-based systems for FH concept extraction and relation identification. We explored deep learning models including Long-short term memory – conditional random fields (LSTM-CRFs) and Bidirectional Encoder Representations from Transformers (BERT) as well as developed ensemble models using a majority voting strategy. To further optimize performance, we systematically compared three different strategies to use BERT output representations for relation identification.
Results:
Our system was among the top-ranked systems in the challenge. Our best system submitted during this challenge achieved micro-averaged F1-scores of 0.7944 and 0.6544 for concept extraction and relation identification, respectively. After challenge, we further explored new transformer-based models and improved the performances of both subtasks to 0.8249 and 0.6775, respectively. For relation identification, our system achieved a performance comparable to the best system (0.6810) reported in the challenge.
Conclusions:
This study demonstrated the feasibility of utilizing deep learning methods to extract family history information from clinical narratives automatically.
Citation
