JMIR Publications

ABSTRACT

Background:

Asthma brings a huge burden to patients and healthcare systems. To facilitate preventive care for asthma management and improve patient outcomes, we recently developed two machine learning models, one on Intermountain Healthcare data and the other on Kaiser Permanente Southern California (KSPC) data, to forecast asthma-related hospital visits including emergency department visits and hospitalizations in the succeeding 12 months in patients with asthma. As is typical for machine learning approaches, these two models give no explanation of their forecasting results. To address black-box models’ interpretability issue, we designed an automatic method to supply rule-format explanations for any machine learning model’s forecasting results on imbalanced tabular data and to suggest customized interventions with no accuracy loss. Our method worked well for explaining our Intermountain Healthcare model’s forecasting results, but its generalizability to other healthcare systems stays unknown.

Objective:

This study aims to evaluate our automatic explanation method’s generalizability to KPSC for forecasting asthma-related hospital visits.

Methods:

Through secondary analysis of 987,506 data instances from 2012 to 2017 at KSPC, we employed our method to explain our KPSC model’s forecasting results and to suggest customized interventions. The patient cohort covered a random sample of 70% of patients with asthma who had a KPSC health plan for any period between 2015 and 2018.

Results:

Our method explained the forecasting results for 97.57% (2,204/2,259) of the patients with asthma our KPSC model correctly forecasted to undergo asthma-related hospital visits in the succeeding 12 months.

Conclusions:

For forecasting asthma-related hospital visits, our automatic explanation method exhibited decent generalizability to KPSC.