Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Accepted for/Published in: JMIR Medical Informatics

Date Submitted: Mar 24, 2022
Date Accepted: May 13, 2022

The final, peer-reviewed published version of this preprint can be found here:

Error and Timeliness Analysis for Using Machine Learning to Predict Asthma Hospital Visits: Retrospective Cohort Study

Zhang X, Luo G

Error and Timeliness Analysis for Using Machine Learning to Predict Asthma Hospital Visits: Retrospective Cohort Study

JMIR Med Inform 2022;10(6):e38220

DOI: 10.2196/38220

PMID: 35675129

PMCID: 9218884

Error and Timeliness Analysis for Using Machine Learning to Predict Asthma Hospital Visits: Retrospective Cohort Study

  • Xiaoyi Zhang; 
  • Gang Luo

ABSTRACT

Background:

A significant burden on health care comes from asthma hospital visits including emergency department visits and inpatient stays. To leverage preventive care more effectively in managing asthma, we formerly employed machine learning and the University of Washington Medicine (UWM) data to build the world’s most accurate model to forecast which asthma patients will encounter asthma hospital visits during the successive 12 months.

Objective:

Currently, two questions remain regarding our model’s performance. First, for a patient who will encounter asthma hospital visits in the future, how timely can our model identify the risk for the first time? Second, if our model erroneously predicts a patient to encounter asthma hospital visits at the UWM during the successive 12 months, how likely will the patient encounter ≥1 asthma hospital visit somewhere else or have ≥1 surrogate of a poor outcome? This work aims to answer these two questions.

Methods:

The patient cohort covered every adult asthma patient who received care at the UWM during 2011-2018. Using the UWM data, our model made predictions on the asthma patients in 2018. For every such patient with ≥1 asthma hospital visit at the UWM in 2019, we computed the number of days of advanced warning that our model gave on the patient for the first time. For every such patient erroneously projected to encounter ≥1 asthma hospital visit at the UWM in 2019, we used PreManage and the UWM data to check whether the patient had ≥1 asthma hospital visit outside of the UWM in 2019 or any surrogate of a poor outcome. Surrogates of poor outcomes included order of systemic corticosteroids during the successive 12 months, any type of visit for asthma exacerbation during the successive 12 months, and asthma hospital visit between the successive 13-24 months.

Results:

Among the 218 asthma patients in 2018 with asthma hospital visits at the UWM in 2019, 61.9% (135/218) were given risk warnings for such visits for the first time ≥3 months ahead by our model and 84.4% (184/218) were given risk warnings ≥1 day ahead. Among the 1,310 asthma patients in 2018 erroneously projected to encounter asthma hospital visits at the UWM in 2019, 29.01% (380/1,310) had asthma hospital visits outside of the UWM in 2019 or surrogates of poor outcomes.

Conclusions:

Our model gave timely risk warnings for most asthma patients with poor outcomes. 29.01% (380/1,310) of asthma patients for whom our model gave false-positive predictions had asthma hospital visits somewhere else during the successive 12 months or surrogates of poor outcomes, and were reasonable candidates for preventive interventions. There is still significant room for improving our model to give more accurate and more timely risk warnings.


 Citation

Please cite as:

Zhang X, Luo G

Error and Timeliness Analysis for Using Machine Learning to Predict Asthma Hospital Visits: Retrospective Cohort Study

JMIR Med Inform 2022;10(6):e38220

DOI: 10.2196/38220

PMID: 35675129

PMCID: 9218884

Download PDF


Request queued. Please wait while the file is being generated. It may take some time.

© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.