Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Multi-Agent Cross-Review of Large Language Models for Medical Question Answering Under Noisy Clinical Text: Development and Evaluation Study
ABSTRACT
Background:
Clinical text in electronic health records frequently contains heterogeneous noise, such as typographical errors, speech-recognition and optical character recognition (OCR) errors, and copy-paste duplication, that can degrade the reliability of AI-driven clinical decision support. Existing approaches target single noise types or require costly retraining and prior knowledge of the noise.
Objective:
We aimed to develop a multi-agent framework that provides robustness to multiple noise types at inference time, without model retraining.
Methods:
We propose Medical Multi-agent Cross-Review (MedMCR), in which two reasoning agents generate independent rationales, cross-review each other, and an adjudicator selects the final answer. We used Qwen2.5-32B-Instruct and gemma-3-27b-it as the reasoning agents and gpt-oss-20B as the adjudicator. On MedQA, MedMCR was compared with zero-shot and chain-of-thought (CoT) baselines under six noise types — typographical errors, word deletion, word duplication, OCR errors, shuffling, and word substitution — at four word error rate (WER) levels. A linear probability model with a WER-by-method interaction assessed robustness.
Results:
MedMCR consistently outperformed the zero-shot and CoT baselines, improving accuracy from 66.1% to 77.1% under clean conditions (P<.001, McNemar test), with gains sustained across all six noise types. Using only 32B-class models, MedMCR matched Qwen2.5-72B CoT within 1.4 percentage points. Interaction analysis showed significantly more gradual degradation for typographical, OCR, shuffling, and duplication noise, whereas MedMCR degraded more steeply than the baseline under word substitution. Cross-review scores were positively associated with response accuracy, supporting their validity as a quality signal.
Conclusions:
MedMCR enhances inference-time robustness to multiple noise types without retraining, and the vulnerability of large language models differs across noise types. Multi-agent cross-review is therefore a promising strategy for improving clinical AI reliability in noisy real-world environments.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.