JMIR Publications

ABSTRACT

Background:

Bone metastasis patients suffered from a very limited survival time, and a life expectancy of less than three months (early death) was considered as a contraindication of large invasive surgery. Accurate survival prediction is critically important to guide make clinical decisions.

Objective:

The aim of this study was to develop a more accurate prediction model based on machine learning to assess the probability of early death among bone metastasis patients.

Methods:

This study enrolled 118,227 patients with bone metastasis between 2010 and 2019 from the National Cancer Database. The entire cohort of patients was randomly split 9:1 into a training group (n=106492) and a validation group (n=11735). Patients in the training group were used to train and optimize prediction models; Patients in the validation group were used to assess and validate the prediction performance of models. Six approaches including logistic regression, XGBoosting machine, decision tree, random forest, neural network, and gradient boosting machine were introduced in the study. Prediction performance of models was assessed using twelve measures, and among each measure, approaches were scored by sort according to prediction performance. The optimal model was the approach with the highest sum score of prediction performance. Patients (n=332) from a teaching hospital was regarded as the external validation group, and external validation was performed using the optimal model. Local interpretable model-agnostic explanation (LIME) was employed to make model’s explainability to promote clinical utility and transparency in the best model.

Results:

In the entire cohort, up to 36.6% of patients had early death. According to multivariate analysis, age, sex, race, marital status, rural-urban continuum, primary site, tumor stage (T stage), node stage (N stage), brain metastasis, liver metastasis, lung metastasis, cancer-directed surgery, radiation, and chemotherapy were significantly associated with early death and were included for model training and optimization. The gradient boosting machine had the highest score of prediction performance (59 points), followed by the neural network (56 points) and XGBoosting machine (56 points). Area under curve (AUC) of the gradient boosting machine was up to 0.858 (95% confident interval [CI]: 0.851-0.865), calibration slope was 1.02, and intercept-in-large was -0.02, all of which indicated favorable discrimination and calibration. Explainability of models was conducted to rank variables and visualize their contributions to early death among individuals based on the gradient boosting machine. Risk stratification of patients was achieved in the study and patients could be divided into two risk groups according to the threshold (37.00%). Based on the model developed by the gradient boosting machine, patients in the high-risk group (71.96%) were 4.5 times more likely to develop early death than patients in the low-risk group (15.62%). External validation showed that the AUC was up to 0.847 (95% CI: 0.798-0.895), calibration slope was 1.06, and intercept-in-large was 0.17.

Conclusions:

The optimal model can be a pragmatic prediction tool with the ability of identify bone metastasis patients at a high risk of early death, thus facilitating clinical decision-making and communication between patients and doctors. Risk stratification of patients was achieved in the study and patients could be divided into two risk groups. Patients in the high-risk group should better be treated with radiotherapy alone, best supportive care, or minimal invasive techniques such as cementoplasty to palliatively alleviate pain.