JMIR Publications

ABSTRACT

Background:

Asthma causes numerous hospital encounters including emergency department visits and hospitalizations annually. To improve patient outcomes and cut the number of these encounters, predictive models are widely used to prospectively pinpoint high-risk patients with asthma for preventive care via care management. But, the prior models do not have adequate accuracy to achieve this goal well. Adopting the modeling guideline of checking extensive candidate features, we recently constructed a machine learning model on Intermountain Healthcare data to predict asthma-related hospital encounters in patients with asthma. Although this model is more accurate than the prior models, it remains unknown whether our modeling guideline is generalizable to other healthcare systems.

Objective:

This study aims to assess our modeling guideline’s generalizability to Kaiser Permanente Southern California (KSPC).

Methods:

The patient cohort included a random sample of 70% of patients with asthma who were enrolled in a KPSC health plan for any duration between 2015 and 2018. Via secondary analysis of 987,506 KPSC data instances from 2012 to 2017 and checking 337 candidate features, we produced a machine learning model to project asthma-related hospital encounters in the succeeding 12-month period in patients with asthma.

Results:

Our model reached an area under the receiver operating characteristics curve of 0.820. When the cutoff point for doing binary classification was put at the top 10.00% (20,474/204,744) of patients with asthma having the largest predicted risk, our model achieved an accuracy of 90.08% (184,435/204,744), a sensitivity of 51.90% (2,259/4,353), and a specificity of 90.91% (182,176/200,391).

Conclusions:

Our modeling guideline exhibited acceptable generalizability to KPSC and resulted in a model that is more accurate than those formerly built by others. After further enhancement, our model could be used to guide asthma care management.