JMIR Publications

ABSTRACT

Background:

Despite their growing use in healthcare, pre-trained language models (PLMs) often lack clinical relevance due to insufficient domain expertise and poor interpretability. A key strategy to overcome these challenges is integrating external knowledge into PLMs, enhancing their adaptability and clinical usefulness. Current biomedical knowledge graphs like UMLS, SNOMED CT, and HPO, while comprehensive, fail to effectively connect general biomedical knowledge with physician insights. Equally important is the need for a model that integrates diverse knowledge in a way that is both unified and compartmentalized. This approach not only addresses the heterogeneous nature of domain knowledge but also recognizes the unique data and knowledge repositories of individual healthcare institutions, necessitating careful and respectful management of proprietary information.

Objective:

This study aims to enhance the clinical relevance and interpretability of PLMs by integrating external knowledge in a manner that respects the diversity and proprietary nature of healthcare data. We hypothesize that domain knowledge, if captured and distributed as standalone modules, can be effectively re-integrated into PLMs to improve their adaptability and utility in healthcare settings.

Methods:

We demonstrate that via adapters, small and lightweight neural networks that enable the integration of extra information without full model fine-tuning, we can inject diverse sources of external domain knowledge into language models and improve the overall performance with an increased level of interpretability. As a practical application of this methodology, we introduce a novel task, structured as a case study, that endeavors to capture physician knowledge in assigning cardiovascular diagnoses from clinical narratives, where we extract diagnosis-comment pairs from Electronic Health Records (EHRs) and cast the problem as text classification.

Results:

The study demonstrates that integrating domain knowledge into PLMs significantly improves their performance. While improvements with ClinicalBERT are more modest, likely due to its pre-training on clinical texts, BERT adaptations surprisingly match or exceed ClinicalBERT in several metrics. This underscores the effectiveness of knowledge adapters and highlights their potential in settings with strict data privacy constraints. This approach also increases the level of interpretability of these models in a clinical context.

Conclusions:

This research provides a basis for creating health knowledge graphs infused with physician knowledge, marking a significant step forward for PLMs in healthcare. Notably, the model balances integrating knowledge both comprehensively and selectively, addressing the heterogeneous nature of medical knowledge and the privacy needs of healthcare institutions.