Accepted for/Published in: Journal of Medical Internet Research
Date Submitted: Sep 1, 2020
Date Accepted: Nov 19, 2020
Date Submitted to PubMed: Nov 23, 2020
Development and External Validation of a Machine Learning Tool to Rule Out COVID-19 Among Adults in the Emergency Department Using Routine Blood Tests: A Large, Multicenter, Real-World Study
ABSTRACT
Background:
Conventional diagnosis of COVID-19 with polymerase chain reaction (PCR) testing is associated with prolonged time to diagnosis and costs of running the test. The SARS-CoV-2 virus might lead to characteristic patterns in levels of widely-available, routine blood tests results that could be identified with machine learning methodologies. Machine learning modalities integrating findings from these common laboratory test results might accelerate ruling out emergency department patients for COVID-19.
Objective:
We sought to develop and externally validate a machine learning model to rule out COVID-19 using only routine blood tests among adults in emergency departments.
Methods:
Using clinical data from emergency departments (EDs) from 66 US hospitals before the pandemic (before the end of December 2019) or during the pandemic (March-July 2020), we included patients aged ≥20 years in the study timeframe or missing laboratory results. Model development used 2,183 PCR-confirmed positive cases from 43 hospitals during the pandemic as positive controls; negative controls were 10,000 pre-pandemic patients from the same hospitals. External validation used 23 hospitals with 1,020 pandemic PCR-positive cases and 171,734 pre-pandemic negative controls. The main outcome was COVID-19 status predicted using same-day routine laboratory results. Model performance was assessed with area under the receiving operating characteristic curve (AUROC) as well as sensitivity, specificity, negative predictive value (NPV).
Results:
Of 184,937 patients included (median [IQR] age deciles 50.0 [30.0-60.0] years, 40.5% male), AUROC for development and external validation was 0.91 (95% CI, 0.90-0.92). Using a risk score cutoff of 1.0 (out of 100) in the validation dataset, the model achieved sensitivity of 95.9% and specificity of 41.7%; with a cutoff of 2.0, sensitivity of 92.5% and specificity of 60%. At the cutoff of 2, the NPVs at prevalences of 1%, 10%, and 20% were 99.9%, 98.6%, and 97%.
Conclusions:
A machine learning model developed with multicenter clinical data integrating commonly collected ED laboratory data demonstrated high rule-out accuracy for COVID-19 status, and might inform selective use of PCR-based testing. Clinical Trial: N/A
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.