JMIR Publications

ABSTRACT

Background:

Large language models show promise for enhancing diagnostic accuracy and clinical decision-making. However, prevailing evaluations rely on exam-based benchmarks like MedQA, which may test rote memorization over clinical judgment. In medicine, clinical logic must be as sound as the conclusion, yet there has been little systematic investigation into benchmark quality or reasoning-trace failures.

Objective:

Investigate failure modes of reasoning-based Large Language Models (LLMs) in medicine by auditing benchmark quality, building a clinically-informed taxonomy of reasoning errors, and testing a mechanistic intervention.

Methods:

We evaluated OpenAI o1 on the MedQA (n=1,273) and cross-referenced incorrect answers against source materials to flag benchmark flaws. For 37 confirmed model failures, we developed a reasoning error taxonomy through inductive coding and validated it on three additional LLMs (GPT-4.5-preview, o3-mini, and DeepSeek-R1). We developed a sparse autoencoder (SAE) to isolate reasoning-specific features, steered these features with a positive bias, and measured effects on accuracy and reasoning length across MedQA, MedMCQA, and PubMedQA. Hallucination in reasoning traces was evaluated with an LLM-as-a-judge.

Results:

Forty-one percent of initial errors reflected benchmark problems, including missing figures (22%) and ambiguities (19%). Our taxonomy classified failures into four categories: Information Synthesis, Therapeutic Decision, Diagnostic Reasoning, and Foundational Principle Errors, revealing distinct failure profiles across models. Steering reasoning-specific features significantly improved accuracy across all benchmarks, but increased reasoning length. We identified five functional categories whose roles aligned with the taxonomy.

Conclusions:

The reliability of medical LLMs is limited by flawed evaluations and recurrent reasoning failures. Isolating and steering reasoning-specific features via SAEs could mechanistically correct these patterns, advancing interpretable, “glass-box” clinical AI.