Previously submitted to: JMIR Medical Informatics (no longer under consideration since Nov 19, 2026)
Date Submitted: Nov 18, 2026
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Longitudinal Electronic Health Record Models for Short-Horizon Prediction of In-Hospital Mortality: Retrospective Development, Internal Validation, and Data-Completeness Study
ABSTRACT
Background:
Longitudinal electronic health record models may support early recognition of clinical deterioration, but incomplete measurements and imprecise outcome times complicate comparisons with clinical early warning scores.
Objective:
This study aimed to compare longitudinal prediction models and clinical early warning scores within a model-agnostic evaluation framework that examines data completeness and outcome timing.
Methods:
This retrospective, single-center study used an outcome-enriched cohort comprising 264,523 eight-hour windows from 7638 episodes (7078 patients) from years 2022 to 2025. Day-discretized terminal discharge served as a death-time proxy; 24 hours was primary, 48 and 72 hours complementary, and 8 hours exploratory. Using patient-disjoint partitions and time-available predictors, we compared penalized logistic regression, gradient boosting (GBM), and random forest using engineered temporal features, a sequence Transformer, and a Transformer–GBM ensemble against adapted Modified Early Warning Score (MEWS), National Early Warning Score (NEWS), and the local VIDA early warning score. Evaluation covered discrimination, calibration, data provenance, common-support comparisons, and alert budgets.
Results:
At 24 hours, area under the receiver operating characteristic curve (AUROC) ranged from 0.852 to 0.885 across longitudinal models and from 0.607 to 0.705 for MEWS, NEWS, and VIDA; average precision (AP) ranged from 0.256 to 0.338 and from 0.073 to 0.094, respectively. GBM and the ensemble had the highest point estimates (AUROC 0.884, 95% CI 0.874-0.895, and 0.885, 95% CI 0.874-0.896; AP 0.333, 95% CI 0.304-0.366, and 0.338, 95% CI 0.309-0.371, respectively). The ensemble showed no clear gain in discrimination over GBM. Prespecified VIDA reconstruction increased episode-level availability from 4889 to 6881 without establishing equivalence to recorded scores. A phase-only 8-hour control achieved AUROC 0.833, demonstrating an endpoint-discretization artifact.
Conclusions:
These comparisons highlight the relevance of temporal representation, measurement availability, and endpoint definition when assessing predictive performance. Representative temporal/external validation, prevalence-aware recalibration, and prospective clinical evaluation are required before clinical use.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.