JMIR Publications

ABSTRACT

Background:

When analyzing electronic health records, proper labeling of outcomes is mandatory. To obtain proper information from radiologic reports, several studies conducted to classify radiologic reports used deep learning. However, the classification of pneumonia in radiologic reports written in bi-lingual languages has not previously been conducted.

Objective:

The goal of this research is to classify pneumonia by a deep learning method.

Methods:

As a retrospective study, our dataset included radiology reports for chest CT and chest X-rays of surgical patients from January 2008 to January 2018 in Asan Medical Center in Korea. The classification performance of our attention-LSTM model was compared with various deep learning and machine learning methods. Sensitivity, specificity, accuracy and F1-score for the models were compared.

Results:

A total of 5450 radiologic reports were included in which at least one pneumonia-related word was contained. In the test set (n=1090), our proposed model showed 91% accuracy and the top 3 performances of the models were based on FastText or LSTM. The CNN based model showed lower accuracy (73%) than the other two algorithms. Classifying negative results had an F1-score of 0.96; however, classifying positive and uncertain results showed a lower performance (negative: 0.83, uncertain: 0.62 F1-scores). In the extra-validation set, our model showed 80% accuracy, 0.92 precision, and 0.87 recall.

Conclusions:

Our method of deep natural language process showed significant performance in classifying pneumonia in bi-lingual radiologic reports. The method could enrich the research about pneumonia by obtaining exact outcomes from electronic health data.