JMIR Publications

ABSTRACT

Background:

Heart disease (HD) is a leading global cause of mortality, emphasizing the need for effective diagnostic and predictive tools to enable early intervention. Machine learning (ML) advancements have shown great promise in improving the accuracy and efficiency of HD risk prediction. This study aims to develop a five-variable logistic classification model (SHAP-LR) that integrates data preprocessing, feature engineering, and advanced ML models to screen for prevalent HD in ambulatory settings.

Objective:

The primary objective of this study is to develop and evaluate a robust and interpretable logistic regression model (SHAP-LR) for predicting heart disease (HD) risk. This model integrates data preprocessing, feature engineering, and advanced machine learning techniques to enhance the accuracy and efficiency of HD risk prediction. Additionally, the study aims to create a user-friendly HD scoring system based on the best-performing model, optimized through hyperparameter tuning, to provide healthcare providers with a practical tool for quantifying patient risk. The study also seeks to validate the model's performance across diverse datasets and clinical settings to ensure its generalizability and applicability in real-world scenarios. Through the use of SHAP (SHapley Additive exPlanations) values, the study further aims to enhance the model's interpretability, allowing clinicians to understand the factors contributing to HD risk and thereby facilitating informed decision-making in clinical practice.

Methods:

The study utilized three publicly accessible datasets: BRFSS_2015, Cleveland, Hungary, Switzerland, VA Long Beach, and Statlog (Heart) datasets. Multiple ML models, including Decision Tree, AdaBoost, Gradient Boost, Bagging, CatBoost, Extra Trees, and Logistic Regression, were evaluated. Model performance was assessed using accuracy, precision, recall, F1-score, and AUROC. A user-friendly HD scoring system was developed based on the best-performing model, logistic regression, which was further optimized through hyperparameter tuning. Real-world performance was assessed in a single-center, retrospective validation using a prespecified split-then-rebalance workflow. The Framingham Risk Score (FRS) served as a clinical benchmark.

Results:

Logistic regression outperformed other models, achieving an accuracy of 90.54% and an AUROC of 80.27% on the BRFSS_2015 dataset. After hyperparameter tuning, the model's performance improved further, with accuracy reaching 90.55% and AUROC increasing to 81.09%. The SHAP value analysis revealed that age, high blood pressure, and high cholesterol were the most significant predictors of HD. The developed scoring system provided a quantifiable risk assessment tool, enabling clinicians to stratify patients based on their HD risk effectively. External validation also showed improved discrimination on UCI Heart and Statlog after tuning. In the clinical validation test subset, SHAP-LR achieved AUROC 0.83, exceeding FRS 0.70, and calibration was near-ideal.

Conclusions:

This study demonstrates the effectiveness of ML in predicting HD, with logistic regression emerging as the most reliable model. The integration of SHAP values enhanced the model's interpretability, making it a valuable tool for clinical decision-making. The developed HD scoring system offers a practical and efficient method for risk assessment, potentially improving early diagnosis and intervention strategies. Future work will focus on validating the model in diverse clinical settings and exploring additional features to further enhance predictive accuracy.