JMIR Publications

ABSTRACT

Background:

Medication errors are among the most common and preventable causes of hospital patient harm. Computerized provider order entry (CPOE) and electronic health record (EHR) systems with clinical decision support (CDS) are the primary technological response, yet hospital medication safety remains uneven, both in catch rates for genuinely fatal orders and in alert volumes that drive alert fatigue. Large language models (LLMs) are now entering production medication-management workflows, but existing LLM medication-safety evaluations typically test isolated subtasks on small or synthetic datasets, assess only one or two models, and report a single aggregate accuracy or F1 metric on a class-imbalanced scenario set, an approach that obscures sensitivity-specificity tradeoffs and inflates apparent performance on the dominant class.

Objective:

To introduce PsiBench, a clinically validated medication-safety benchmark for evaluating large language models (LLMs) against the standards used to certify hospital CPOE and EHR systems, together with a non-overlapping three-tier evaluation framework that separates highest-stakes discrimination, the operational CDS regime, and category-correct alerting.

Methods:

PsiBench comprises 492 medication-safety scenarios across 11 safety categories, created by clinical pharmacology experts whose work underpins an annualized testing procedure used by more than 2000 US hospitals. The three-tier framework partitions the scenarios non-overlappingly: Discrimination (98 scenarios; 50 fatal versus 48 deception, near-balanced 51%/49%); Operational (394 scenarios; 261 serious unsafe plus 133 safe, the latter including 41 Excessive Alerts reclassified as operational negatives); and Attribution (311 alert-required scenarios). We evaluated 40 frontier LLMs from 10 providers over 3 runs per scenario at temperature 0.2 (or the provider default where temperature is not configurable), yielding 59,040 evaluations conducted April 21-23, 2026.

Results:

Headline binary performance on the full benchmark spanned a wide range across the 40 models: F1 78.5%-92.3%, accuracy 65.4%-89.8%, sensitivity 81.4%-100.0%, and specificity 6.1%-81.8% (Youden's J 0.06-0.75). Leading models by F1 (o4-mini 92.3%; o3 92.2%) paired high sensitivity with meaningful specificity, whereas three models saturated sensitivity at 100% but fell below 25% specificity, making them indistinguishable from a naive always-alert classifier. This wide spread on any single headline metric demonstrates that single-metric evaluation collapses three clinically distinct questions into one number and motivates the tier-specific analyses developed in a separate clinical paper.

Conclusions:

PsiBench and the three-tier framework operationalize a rigorous evaluation rubric for LLM medication safety, grounded in two decades of national hospital audit experience. The framework generalizes to any binary medication-safety classifier (rule-based, conventional machine learning, or LLM-driven) and supports tier-aware model selection and postdeployment surveillance. A high benchmark score demonstrates capability on a curated scenario set but does not by itself guarantee performance on a specific hospital population or formulary; prospective local validation and ongoing post-market surveillance remain necessary before clinical deployment.