JMIR Publications

ABSTRACT

Background:

Asthma hospital encounters impose a heavy burden on the health care system. To improve preventive care and outcomes for asthma patients, we recently developed a black-box machine learning model to predict whether an asthma patient will have 1 or more asthma hospital encounters in the succeeding 12 months. Our model is more accurate than previous models. However, black-box machine learning models do not explain their predictions, forming a barrier to widespread clinical adoption. To solve this issue, we previously developed a method to automatically provide rule-based explanations for the model’s predictions and to suggest tailored interventions without sacrificing model performance. For an average patient correctly predicted by our model to have future asthma hospital encounters, our explanation method generated over 5,000 rule-based explanations, if any. However, the user of the automated explaining function, often a busy clinician, wants to quickly obtain the most useful information for a patient by viewing just the top few explanations. Therefore, we need a methodology to appropriately rank the explanations generated for a patient. This is currently an open problem.

Objective:

This work aims to develop a method to appropriately rank the rule-based explanations that our automated explaining method generates for a patient.

Methods:

We developed a ranking method that struck a balance among multiple factors. Through a secondary analysis of 82,888 data instances of adults with asthma from the University of Washington Medicine between 2011 and 2018, we demonstrated our ranking method on the test case of predicting asthma hospital encounters in asthma patients.

Results:

For each patient predicted to have asthma hospital encounters in the succeeding 12 months, the top few explanations returned by our ranking method typically have high quality and low redundancy. Many top-ranked explanations give useful insights on various aspects of the patient’s situation, which cannot be easily obtained by viewing the patient’s data in the current electronic health record system.

Conclusions:

The explanation ranking module is an essential component of the automated explaining function, which addresses the interpretability issue that deters the widespread adoption of machine learning predictive models in clinical practice. In the next few years, we plan to test our explanation ranking method on predictive modeling problems addressing other diseases as well as on data from other health care systems.