JMIR Publications

ABSTRACT

Background:

Retrieval over clinical free text underpins record search, cohort identification, and retrieval-augmented language systems. Retrieval performance is conventionally summarized by a single mean across a corpus, which can conceal systematic heterogeneity within the population it represents. Documentation of more deranged patients may contain more competing problems, interventions, and

Objective:

This study tested whether known-item retrieval over intensive care unit (ICU) discharge summaries varies across strata of an early physiological derangement composite measured independently of the text, whether any gradient is shared by dense and lexical retrieval, and whether measured clinical, documentary, or query characteristics attenuate it.

Methods:

We linked 4,078 MIMIC-IV discharge summaries to an existing adult medical ICU cohort and built a pooled, length-matched index of 2,956 documents drawn equally from four quartiles of a prespecified derangement composite. A held-out two-sentence passage had to retrieve its originating summary after removal from the target. Eight contrastively trained dense retrievers, five masked language model encoders as negative controls, and BM25 were evaluated. Reciprocal rank at 10 was regressed on exposure quartile with patient-clustered standard errors and Holm correction. We examined retrieval geometry, alternative query sampling, same-patient competition, an age-free exposure, a lab-only organ dysfunction stratification with a matched exposure comparison, and covariate adjustment.

Results:

All 8 dense retrievers performed worse as derangement increased (mean decline from lowest to highest quartile 37.2%, range 29.1% to 43.9%; all 8 slopes Holm-significant), while BM25 declined by 13.4%. No difference in absolute slopes was detected (difference +0.0016, 95% CI −0.0110 to +0.0141), but dense retrieval declined more in proportional terms (difference −0.074, 95% CI −0.109 to −0.036). Deterioration was mainly associated with falling target similarity rather than rising distractor similarity. Joint covariate adjustment attenuated the mean coefficient by 21.6%, and all 40 slopes across alternative query draws were negative. Organ dysfunction slopes were directionally similar but less consistently supported, and the matched difference between exposures included zero in all eight models.

Conclusions:

Retrieval performance over ICU discharge summaries in this corpus was heterogeneous with respect to an independently measured early patient-state signal. Corpus-level means concealed patient-associated variation in retrievability. The findings are specific to MIMIC-IV discharge summaries and to a known-item assay; they do not establish a failure rate for deployed retrieval-augmented systems, and the pathway linking measured physiology to the discharge narrative remains unidentified.