Accepted for/Published in: JMIR Medical Informatics
Date Submitted: Feb 2, 2024
Date Accepted: Jul 21, 2024
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
The dawn of targeted development and validation of clinical prediction models in secondary care settings – opportunities and challenges for electronic health records data
ABSTRACT
Upon deploying a clinical prediction model (CPM) in clinical practice, the performance and generalizability of a CPM needs to be demonstrated in the population of intended use. This is also called ‘targeted validation’. In this viewpoint we consider targeted validation and the use of clinical prediction models (CPM) in a secondary care setting using Electronic Health Record (EHR) data. More than half of the patients with (complex) care needs are treated in leading clinical teaching hospitals in secondary care settings. The growing influx of patients combined with the worldwide problem of health workforce shortage requires efficient and effective organization of care. Moreover, the healthcare field increasingly moves towards health(care) decisions to be made by both the clinician and the patient (i.e. shared decision making). Against that background, CPMs may have particular use in a secondary care setting. Yet, CPMs are not always validated and if validation occurred, it is often not ‘targeted’ and/or focusses solely on discrimination of a CPM. The accuracy of predicted risks (calibration) is far less often assessed, while this is pivotal for transition of a CPMs into clinical practice. The introduction of artificial intelligence based software applications in secondary care settings enables the creation of statistically powerful datasets from unstructured EHRs. This comes with both technical and legal challenges. Upon using EHR data for the development and validation of CPMs, alongside the widely accepted checklists, we propose to additionally consider the three practical steps: 1) let a local EHR expert (clinician, nurse) be involved in the data extraction process, 2) perform validity checks on the generated datasets and 3) provide metadata on how variables were constructed from EHRs. If successful, such datasets are statistically powerful and opens the gates for targeted development and validation of CPMs in secondary care settings, filling a major gap in prediction modelling research and appropriately advancing CPM’s into clinical practice.
Citation
