JMIR Publications

ABSTRACT

Background:

Electronic health record (EHR)-derived process measures, such as vital-sign record density, are increasingly used for quality monitoring, risk prediction, and cross-system benchmarking. Whether these measures retain consistent meaning across sites and data streams is rarely tested.

Objective:

To determine whether vital-sign record density varies primarily by data stream, database, contributing site, or care unit, and what this implies for comparative use.

Methods:

We retrospectively analyzed MIMIC-IV version 3.1 and eICU-CRD version 2.0. Record density and timing metrics were computed for the first 24 ICU hours from nurse-charted streams and, in eICU-CRD, from the monitor stream. Heart rate was the primary variable; oxygen saturation and respiratory rate were replication variables. Variance was decomposed by database, hospital, and unit using eta-squared with 500 hospital-clustered bootstrap replicates.

Results:

A prior MIMIC-IV heart-rate count included two alarm-limit items (99.3% reconstruction agreement), indicating that the exposure combined observations and documentation events. Across 117,572 eICU-CRD stays present in both streams, heart-rate bottom-decile overlap was 10.6% observed vs 10.5% expected under independence (observed-to-chance ratio 1.01; Spearman ρ=0.024). Pairing MIMIC-IV with the monitor stream placed the database component of record-count variance at 0.960; pairing it with the nurse stream reduced this to 0.026. In pooled nurse-stream data, hospital contributed 42.6%-74.3% of variance for record count and interval-structure metrics, database 1.6%-7.1%, and unit within hospital 0.4%-2.8%; intervals exceeding 30 minutes were an exception (database 24.9%). A pooled bottom-decile rule had only 51.0% positive agreement with hospital-specific rules, and five of 68 hospitals supplied half of the pooled bottom decile. The cross-stream and variance patterns were replicated for oxygen saturation and respiratory rate. An admission-hour association fell from OR 3.67 (95% CI 3.29-4.09) to 1.25 (1.13-1.38) after restriction to stays completing the 24-hour window and was absent in the eICU-CRD nurse stream (OR 1.03, P=.70).

Conclusions:

Vital-sign record density was strongly site-dependent. Apparent cross-database differences could be reproduced by the choice of source table, and an apparent temporal association was substantially explained by analytic-window truncation. Comparative use requires specifying the generating stream, enforcing the analytic window, and explicitly modeling site-level variation.