JMIR Publications

ABSTRACT

Background:

Prediction of medical outcomes may potentially benefit from using advanced statistical modeling techniques. We aimed to externally validate modeling strategies for the prediction of the disability of community-dwelling older people.

Objective:

We aimed to externally validate modeling strategies for the prediction of the disability of community-dwelling older people.

Methods:

We analyzed individual patient data from five studies including community-dwelling older people. We considered a set of fourteen binary predictors as measured with the Tilburg Frailty Indicator (TFI). With this set, we predicted the continuous total disability score as measured with the Groningen Activity Restriction Scale (GARS) using five statistical modeling techniques: general linear model (GLM), support vector machine (SVM), neural net (NN), recursive partitioning (RP), and random forest (RF). For external validation, a model developed on one of the five data sets was applied to each of the four remaining sets. This process was repeated four times for a total of twenty validations. Calibration characteristics, the correlation coefficient, and the root of the mean squared error were used to assess the performance of the models.

Results:

All models, except the NN model, showed satisfactory performance characteristics when validated on the validation data sets. The use of a deviating data set for the development and the validation of the models lead to poor performance characteristics for all models due to the deviating baseline characteristics of that data set compared to the baseline characteristics of the other data sets.

Conclusions:

All models showed satisfactory performance characteristics on the development data sets. The performance of the models GLM, SVM, RP and RF on the validation data sets was also satisfactory, except when the models were developed on the data set with deviating baseline characteristics compared to the characteristics of the other data sets we used in this study. The performance of the NN models on the validation data sets was much worse compared to the initial performance on the development data sets.