Currently submitted to: Journal of Medical Internet Research
Date Submitted: Jun 8, 2026
Open Peer Review Period: Jun 9, 2026 - Aug 4, 2026
(currently open for review)
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Improving Drug–Drug Interaction Detection in Surgical Inpatients: Evaluation of a Healthcare-Specific Large Language Model.
ABSTRACT
Background:
Medication-related harm is a major cause of preventable morbidity and mortality in hospitalised patients, particularly among older individuals with polypharmacy. Healthcare-specific large language models (LLMs) trained on validated pharmacological sources may provide more reliable and clinically relevant drug–drug interaction (DDI) detection than general-purpose systems.
Objective:
To evaluate the accuracy and processing time of Katana AI, a novel healthcare-specific large language model, compared with pharmacist-led medication review and a general-purpose large language model for DDI detection in surgical inpatients.
Methods:
Medication charts from surgical inpatients were prospectively reviewed between September 2025 and February 2026. DDIs identified by Katana AI, pharmacist-led review, and ChatGPT were compared against the British National Formulary (BNF) reference standard. Interactions were classified by severity and level of supporting evidence. Detection accuracy and processing time were recorded and compared.
Results:
Thirty-nine surgical inpatients were included, comprising 293 prescribed medications. The median age was 70 years (IQR 60–75), with 69% aged over 65 years. Katana AI identified 125 DDIs, of which 117 were clinically accurate (93.6%), compared with 85.9% for pharmacist review and substantially lower accuracy for ChatGPT. Katana AI demonstrated significantly higher accuracy than ChatGPT (p<0.001) and a modest but statistically significant improvement over pharmacist review (p=0.041). Mean processing time was 32.1 seconds for Katana AI, comparable to ChatGPT (30.7 seconds) and significantly faster than pharmacist review (227 seconds; p<0.001).
Conclusions:
Katana AI demonstrated high accuracy and rapid detection of clinically relevant DDIs, outperforming a general-purpose large language model and showing a modest improvement over pharmacist review. These findings support the potential role of healthcare-specific large language models as clinical decision-support tools to enhance medication safety and prescribing efficiency. Further multi-centre validation is warranted. Clinical Trial: N/A
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.