Accepted for/Published in: JMIR mHealth and uHealth
Date Submitted: Aug 28, 2020
Date Accepted: May 18, 2021
A Comparison of the Validity and Generalisability of Machine Learning Algorithms for the Prediction of Energy Expenditure: A Validation Study
ABSTRACT
Background:
Accurate solutions for the estimation of physical activity and energy expenditure (EE) at scale are needed for a range of medical and health research fields. Machine learning techniques show promise in research-grade accelerometers and some evidence indicates these techniques can be applied to more scalable commercial devices.
Objective:
This study tests the validity and out-of-sample generalisability of algorithms for the prediction of EE in a number of wearables (Fitbit charge 2, ActiGraph GT3-x, SenseWear Armband Mini and Polar H7) using two laboratory datasets comprised of different activities.
Methods:
Two laboratory studies (combined n = 89) in which participants performed a sequential lab-based activity protocol were combined in this study. In both studies, accelerometer and physiological data were collected alongside EE by indirect calorimetry. Three regression algorithms were used to predict Metabolic Equivalents (METs) (random forest, gradient boosting and neural network) and 5 classification algorithms were used for physical activity intensity classification as sedentary, light or moderate to vigorous (K-Nearest neighbor, support vector machine, random forest, gradient boosting and neural network). Algorithms were evaluated in leave-one-out cross validations and out-of-sample validations.
Results:
Root mean squared error (RMSE) was lowest for gradient boosting and random forest applied to SenseWear and Polar H7 combined (0.89 METs) and in the classification task gradient boost applied to SenseWear and Polar H7 was most accurate (85 %). Fitbit models achieved a RMSE of 1.33 METs and 78% for classification. Errors increased in out-of-sample tests with the SWA Gradient Boost achieving RMSE values of 1.17 METs and accuracy of 80%.
Conclusions:
Algorithms trained on combined datasets demonstrate high predictive accuracy, with a tendency for superior performance of random forests and gradient boosting for most but not all wearable devices. Predictions were poorer in the between study validations evidencing the benefit of combining more than one data source.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.