JMIR Publications

ABSTRACT

Background:

Although much progress has been made in AI, several challenges remain substantial obstacles to the development and translation of AI systems into clinical practice. Even the LLMs (Large Language Models) which show excellent performance on various tasks, have progressed slowly in clinical practice task. Providing precise and "explainable" treatment plans with personalized details remains a big challenge for AI systems due to both the highly professional medical knowledge and the patient’s complicated condition.

Objective:

This study aimed to develop an explainable and efficient decision support system, named H-SYSTEM, to assist neurosurgeons in diagnosing and treating HICH patients. The system was designed to address the limitations of existing AI systems by integrating a medical domain knowledge graph (HKG) to enhance decision-making accuracy and explainability.

Methods:

The H-SYSTEM consists of three main modules, the key-named entity identification module, the semantic analysis and representation module, and the reasoning module. Furthermore, we constructed a medical domain knowledge graph for HICH, named HKG, which is served as an “external knowledge brain” of H-SYSTEM to enhance its text recognition and automated decision-making capability. The HKG is exploited to guide the training of semantic analysis and representation module and reasoning module, which makes the output of H-SYSTEM more “explainable”. To assess the performance of the H-SYSTEM, we compared it with doctors and different LLMs.

Results:

The outputs based on HKG showed a reliable performance as compared with the neurosurgical doctor (ND), with an overall accuracy of 94.87%. The BERT-IDCNN-BiLSTM-CRF model was used as the key-named entity identification module of H-SYSTEM due to its fast convergence and efficient extraction of key named entities, achieved the highest performance among 7 Key Named Entity Identification (KENI) models with P=92.03, R=90.22, and F1=91.11, significantly outperforming the others. The H-SYSTEM achieved an overall accuracy of 91.74% in treatment plans, showing significant consistency with the "gold standard" (P<0.05), with diagnostic measures achieving 88.18% accuracy, 97.03% AUC, and 0.874 κ; surgical therapy achieving 98.53% accuracy, 98.53% AUC, and 0.971 κ; and rescue therapies achieving 89.50% accuracy, 94.67% AUC, and 0.923 κ (all P<0.05). Furthermore, the H-SYSTEM showed high reliability and efficiency when compared to doctors, ChatGPT, achieving statistically higher accuracy (95.26% vs. 91.48%, p < 0.05). Additionally, The H-SYSTEM achieved a total accuracy of 92.22% (ranging from 91.14% to 95.35%) in treatment plans for 605 additional patients from six different medical centers.

Conclusions:

The H-SYSTEM shows significantly high efficiency and generalization capacity in processing EMRs, and provides explainable and elaborate treatment plans. Therefore, it has the potential to provide neurosurgeons with rapid and reliable decision support, especially in emergency conditions. The knowledge graph enhanced deep-learning model can exhibit excellent performance in the clinical practice task.