JMIR Publications

ABSTRACT

Clinical natural language processing (cNLP) techniques are commonly developed and used to extract information from clinical notes to facilitate decision making and research. However, they are less established for rare diseases such as lymphoid malignancies due to the lack of annotated data as well as the heterogeneity and complexity of how clinical information is documented. In addition, there is increasing evidence that cNLP techniques may be prone to biases embedded in clinical documentation or model development. These biases can result in disparities in performance when extracting clinical information or predicting patient outcomes. In this paper, we report the development and validation of a cNLP pipeline that extracts cancer-related clinical information such as performance status, staging, and diagnosis as well as less common information such as substance use and military environmental exposures from clinical notes of veterans with lymphoid malignancies. Due to the low transferability of pre-trained language models from non-cancer domains to our study data, we developed a rule-based cNLP pipeline that integrates domain expertise. We tested and compared the performance of the cNLP pipeline on notes from two veteran patient cohorts: one from non-Hispanic White veterans and the other one from non-Hispanic Black veterans. We found that while the pipeline has robust performance across the two patient groups, it had a significantly lower performance in extracting substance use from notes of Black patients (p-value = 0.03967). This may be due to less frequent documentation of clinical substance use in Black patients. We further discuss the challenges encountered in developing and deploying the cNLP pipeline on the Department of Veterans Affairs (VA) data for rare cancers and future directions to enhance the performance of cNLP systems and to avoid biases.