JMIR Publications

ABSTRACT

Background:

The evaluation and improvement of medical Large Language Models (LLMs) are critical for their real-world deployment, particularly in ensuring accuracy, safety, and ethical alignment. However, existing frameworks are inadequate for dissecting domain-specific error patterns or addressing cross-modal challenges.

Objective:

To address the limitations of current evaluation methods, this study aims to develop and apply a granular error taxonomy to systematically identify critical weaknesses in leading medical LLMs. The ultimate goal is to establish an actionable roadmap for enhancing their clinical robustness, safety, and overall trustworthiness.

Methods:

This study introduces a granular error taxonomy developed through a systematic analysis of 10 top-performing models on MedBench (specifically: AntAngelMed, Citrus-2.0, INF-Med, WHU_Med, zhuomuniao-Med, TeleChat2, hunyuan-med, UNI-GPT, fusiontech-Med, and GPT-4). Incorrect responses were categorized into eight distinct types: Omissions, Hallucination, Format Mismatch, Causal Reasoning Deficiency, Contextual Inconsistency, Unanswered, Output Error, and Deficiency in Medical Language Generation. Based on these findings, we propose a tiered optimization strategy spanning four levels, from prompt engineering and knowledge-augmented retrieval to hybrid neuro-symbolic architectures and causal reasoning frameworks.

Results:

Based on a comprehensive error analysis of 42766 question responses generated by the top 10 models, the evaluation using the eight defined metrics revealed significant vulnerabilities in the leading models. Despite achieving a high accuracy of 0.86 in medical knowledge recall, analysis across the eight error categories identified Omissions as the most prevalent issue, exhibiting a staggering 96.3% omission rate in critical reasoning tasks. Furthermore, safety and ethics evaluations showed alarming inconsistency under option-shuffled conditions, with a low robustness score of 0.79. Our analysis uncovers systemic weaknesses in the models' ability to enforce knowledge boundaries and perform multi-step reasoning.

Conclusions:

This work establishes an actionable roadmap for developing more clinically robust LLMs. By providing error-driven insights, it redefines evaluation paradigms, ultimately advancing the safety and trustworthiness of AI in high-stakes medical environments and promoting its responsible application.