Accepted for/Published in: JMIR Medical Informatics
Date Submitted: Apr 14, 2026
Date Accepted: Jul 14, 2026
Large Language Model–Based Clinical Decision Support for Antibiotic Selection and Dose Recommendation in Hospitalized Patients With Pneumonia: Multicenter Retrospective Study
ABSTRACT
Background:
Pneumonia is a common infectious disease, and antibiotic treatment in hospitalized patients must balance efficacy, safety, and resistance risk. However, antibiotic selection and dose adjustment still rely heavily on clinician experience. Although large language models (LLMs) are promising for clinical reasoning, their direct use for medication recommendation is limited by hallucinations and weak adherence to clinical constraints.
Objective:
This study aimed to develop and externally validate a constrained, human-in-the-loop, LLM-based CDS pipeline for antibiotic selection and dose recommendation in hospitalized patients with pneumonia.
Methods:
We conducted a multicenter retrospective study using electronic health record (EHR) narratives, antibiotic orders, and laboratory indicators of hepatic and renal function from 331 hospitalized patients with pneumonia from 2 hospitals in China. The development cohort included 233 patients, and the external validation cohort included 98 patients. The pipeline integrated dual-branch retrieval (similar-case vector retrieval plus guideline-based knowledge-graph retrieval), clinician-defined rule constraints, and hybrid-context reasoning. DeepSeek-v3, GLM-4.6, and GPT-4o were evaluated using F1 score and Jaccard accuracy.
Results:
On the internal test set, the full pipeline with DeepSeek-v3 achieved the best performance, with an F1 score of 0.8110 and Jaccard accuracy of 0.7624 for antibiotic selection, and an F1 score of 0.7538 and Jaccard accuracy of 0.7076 for joint antibiotic-plus-dosing recommendation. On the external validation set, performance remained high (F1=0.8605, Jaccard accuracy=0.8571 for antibiotic selection; F1=0.8503, Jaccard accuracy=0.8469 for antibiotic-plus-dosing recommendation). The system also provided traceable evidence and rule-trigger information to support clinician review.
Conclusions:
A constrained, retrieval-augmented LLM pipeline improved the consistency, interpretability, and cross-site generalizability of antibiotic selection and dose recommendation for hospitalized patients with pneumonia.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.