Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Currently submitted to: JMIR Medical Informatics

Date Submitted: May 11, 2026

Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.

Improving Medical Question Answering in Online Health Consultations Through Retrieval-Augmented Semantic Aggregation of Multiple Physician Responses

  • Hanshuo Xing; 
  • Qingsheng Yin; 
  • Xiangyu Ji; 
  • Na Zhao; 
  • Zheng Zhu; 
  • Xin Wang; 
  • Yanjun Zhang; 
  • Pengwei Zhuang

ABSTRACT

Background:

Large language models (LLMs) have shown promise for medical question answering, but existing systems often underuse the diverse and complementary information contained in the common “one question, multiple answers” setting, in which multiple physicians respond to the same patient query.

Objective:

This study aimed to develop and evaluate a retrieval-augmented medical question answering framework enhanced by MedSAGE, a Medical Semantic Aggregation for Generative Question Answering module, to better synthesize multiple physician responses and improve the coherence, completeness, and clinical relevance of system-generated answers.

Methods:

We developed a retrieval-augmented generation framework incorporating MedSAGE, a medical semantic aggregation module that jointly models sentence-level consensus, query relevance, and semantic diversity across physician responses. The framework combines dense vector retrieval with structured semantic aggregation to transform fragmented multi-physician inputs into unified medical answers. We evaluated the framework on the publicly available cMedQA2 dataset, which contains more than 120,000 real-world medical question-answer pairs across multiple clinical departments. Performance was assessed using automated natural language generation metrics, terminology-specific analyses, physician ratings, and ablation experiments across multiple large language models.

Results:

The proposed framework consistently outperformed both base generation and standard retrieval-augmented generation across the evaluated models. For Qwen-32B, BLEU-2 increased from 1.10% to 13.35%, BERTScore-F1 increased from 57.05% to 70.99%, METEOR increased from 19.63% to 55.54%, and CIDEr increased from 7.04% to 24.44%. In the ablation analysis, Distinct-2 increased from 6.55% to 9.46%. Physician evaluation further showed improved answer quality, with the highest answer accuracy reaching 4.08 out of 5.0, alongside gains in actionability and overall satisfaction. The framework also improved the coverage of key medical terminology categories, including disease, drug, treatment, and examination entities.

Conclusions:

MedSAGE-enhanced semantic aggregation can strengthen retrieval-augmented medical question answering by transforming fragmented physician responses into more coherent, complete, and clinically meaningful outputs. This framework may support digital health consultation systems and related medical informatics applications that require structured synthesis of heterogeneous physician-generated responses. Clinical Trial: Not applicable.


 Citation

Please cite as:

Xing H, Yin Q, Ji X, Zhao N, Zhu Z, Wang X, Zhang Y, Zhuang P

Improving Medical Question Answering in Online Health Consultations Through Retrieval-Augmented Semantic Aggregation of Multiple Physician Responses

JMIR Preprints. 11/05/2026:101021

DOI: 10.2196/preprints.101021

URL: https://preprints.jmir.org/preprint/101021

Download PDF


Request queued. Please wait while the file is being generated. It may take some time.

© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.