Currently submitted to: JMIR Medical Informatics
Date Submitted: Sep 1, 2026
Open Peer Review Period: Sep 8, 2026 - Nov 3, 2026
(currently open for review)
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
A multi-component retrieval-augmented framework for pediatric knowledge construction and question answering: development study
ABSTRACT
Background:
Pediatric clinical question answering needs current knowledge and sound reasoning. Cloud-based large language models (LLMs) raise patient-privacy concerns, while locally deployed small models lack medical coverage and struggle with complex reasoning. Existing retrieval-augmented generation and knowledge graph methods still fall short in automatic structured knowledge extraction, incremental updating, and reasoning over mixed question formats.
Objective:
To build PediaRAG, a fully local, multi-component retrieval-augmented framework that brings local small models close to cloud LLMs on pediatric question answering. Offline, PediaRAG compiles an incrementally updatable three-representation knowledge base (knowledge graph, structured Wiki, vector index) and fuses evidence from several retrieval sources.
Methods:
PediaRAG works in two stages: offline knowledge base construction and online retrieval inference. In the offline stage, a locally deployed DeepSeek-R1-14B extracted 21,533 knowledge graph triples from 71 pediatric textbooks, guidelines, expert consensus, review articles, and expert interpretations, produced 2,156 structured Wiki pages and 3,047 vector indices, and ran a four-level consistency check (rejection rate 13.2%) to keep the knowledge base reliable. Online, single-choice questions draw on Reciprocal Rank Fusion with a knowledge-graph-weighted hybrid (weights 1.0:1.0:1.5). Multiple-choice questions use source-wise concatenation, disambiguation prompts, and three-temperature majority voting. Ablation experiments (A0-A7) on a self-built 200-question pediatric dataset isolated each component's contribution, and PediaRAG was further tested on PediaBench.
Results:
On the self-built dataset, PediaRAG lifted a 9B model's single-choice accuracy from 35.6% (zero-shot) to 77.8%, leaving a 10.0-percentage-point gap to the cloud DeepSeek-V4-Flash API (87.8%). Its multiple-choice exact-match rate reached 45.0%, 25.0 percentage points above the cloud API. Overall accuracy was 74.5%, near the cloud API's 81.0% (P=.053). Ablation studies showed knowledge graph retrieval contributed most, and the three-channel combination beat every single- or dual-channel setup. On PediaBench, PediaRAG reached 75.45% accuracy.
Conclusions:
With its three-representation knowledge base and multi-source retrieval, PediaRAG lets local small models reach cloud-level performance on pediatric question answering without sending data to the cloud. This fits clinical decision support, where knowledge changes often and reasoning must be exact. Clinical Trial: Not applicable
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.