JMIR Publications

ABSTRACT

Background:

Health research frequently requires manual chart review to identify patients for the study-specific cohort and examine their clinical outcomes. Manual chart review is a labour-intensive process requiring significant time investment for clinical researchers.

Objective:

This study aimed to evaluate the feasibility and accuracy of an assisted chart review program, using an in-house natural language processing (NLP) program, to identify patients who developed radiation pneumonitis (RP) after receiving curative radiotherapy.

Methods:

A retrospective manual chart review was completed for patients who received curative radiotherapy for stage II-III lung cancer from January 1, 2013 to December 31, 2015 at BC Cancer Kelowna. In the manual chart review, RP diagnosis and grading were recorded using Common Terminology Criteria for Adverse Events (CTCAE) v5.0. From the charts of 50 sample patients, a total of 1413 clinical documents were extracted for review from the Cancer Agency Information System (CAIS). The NLP program was built using the Natural Language Toolkit Python platform. Python version 3.7.2. was used to run the NLP program. The output of the NLP program is a list of the full sentences containing the key terms, the document ID’s and dates from which these sentences were extracted. The result from the manual review was used as the gold standard in this study, with which the result of the NLP program was compared.

Results:

Twenty-five out of the 50 sample patients developed RP grade 1 or greater; the NLP program was able to ascertain 23 out of these 25 patients (sensitivity = 0.92, 95%CI:0.74-0.99; specificity = 0.36, 95%CI:0.18-0.57). Furthermore, the NLP program was able to correctly identify all 9 patients with RP grade 2 or greater, which are patients with clinically significant symptoms (sensitivity = 1.0, 95%CI: 0.66-1.0; specificity = 0.27, 95%CI:0.14-0.43). The NLP program was useful in distinguishing patients with RP from those without RP. The NLP program in this study would avoid unnecessary manual review of 22% of the sample patients (n=11), as these patients were identified as RP grade 0 and will not require further manual review in subsequent studies.

Conclusions:

This feasibility study showed that the NLP program was able to assist with the identification of patients who developed RP after curative radiotherapy. The NLP program streamlines the manual chart review further by identifying key sentences of interest. This work has a potential to improve future clinical research, as the NLP program shows promise in performing chart review in a more time efficient manner, compared to the traditional labor-intensive manual chart review.