JMIR Publications

ABSTRACT

Background:

The electronic health record (EHR) contains a wealth of medical experience. An organized EHR can greatly help doctors treat patients. In some cases, only limited patient information is collected to help doctors make treatment decisions. Based on the EHR’s medical experience as a reference for this limited information, doctors’ treatment capabilities can be enhanced. Natural language processing (NLP) and deep learning methods can help organize the knowledge behind the EHR information into medical experience.

Objective:

We aim to create a model to extract concept embeddings from EHRs for disease pattern retrieval and further classification tasks.

Methods:

We collected 1,040,989 emergency department (ED) visits from the National Taiwan University Hospital Integrated Medical Database and 305,897 samples from the National Hospital and Ambulatory Medical Care Survey Emergency Department data. After data cleansing and preprocessing, the data sets were divided into training, validation, and test sets. We proposed a Transformer-based model to embed EHRs and used Bidirectional Encoder Representations from Transformers (BERT) to extract features from free text and concatenate with structural data as input to our proposed model. Then, Deep InfoMax (DIM) and Simple Contrastive Learning of Visual Representations (SimCLR) were used for the unsupervised embedding of the disease concept. The pretrained disease concept-embedding (EDisease) model was further finetuned to adapt to the critical care outcome prediction task. We evaluated the performance of embedding using t-SNE to perform dimension reduction for visualization. The performance of the finetuned predictive model was evaluated against published models using the area under the receiver operating characteristic (AUROC).

Results:

A visualization of disease embeddings revealed clustering at specific triage levels. The results of the new patient disease pattern retrieval in the embedded space were meaningful. Relevant symptoms and possible diseases could be found among the top five results. The performance of our model on the outcome prediction had the highest AUROC 0.876. In the ablation study, the use of a smaller data set or fewer unsupervised methods for pretraining deteriorated prediction performance. On the smaller finetuning set, the AUROCs were 0.806, 0.808, 0.813, and 0.815 for the model without pretraining, the model pretrained by only SimCLR, the model pretrained by only DIM, and the proposed model, respectively.

Conclusions:

Through contrastive learning methods, the disease concept can be embedded meaningfully. Moreover, these methods can be used for disease retrieval tasks to enhance clinical practice capabilities. The disease concept model is also suitable as a pretrained model for subsequent prediction tasks.