Accepted for/Published in: JMIR Biomedical Engineering
Date Submitted: Oct 21, 2022
Date Accepted: Jan 19, 2023
Validation study for an algorithm to classify real-world ambulatory status from a wearable device using multimodal and demographically diverse data
ABSTRACT
Background:
Measuring physical activity amounts and patterns using wearable sensor technology in real-world settings can provide critical insights into health status.
Objective:
We trained an algorithm that classifies binary ambulatory status (yes or no) on accelerometer signal from a wrist-worn Biometric Monitoring Technology (BioMeT) and tested its analytical validity and generalizability.
Methods:
BioMeT algorithm validation traditionally relies on large numbers of self-reported labels or on periods of high-resolution monitoring with reference devices. We used both methods on data collected from two distinct studies for algorithm training and testing, one with precise ground-truth labels from a reference device (N=75), and the second with participant-reported ground-truth labels from a more diverse, larger sample (N=1691); in total, we collected data from 16.7 million 10-second epochs. We trained a neural network on a combined dataset, measuring performance in multiple held-out testing datasets, overall and in demographically-stratified subgroups.
Results:
The algorithm was accurate classifying ambulatory status on 10-second epochs (AUC = 0.938; 95% CI, 0.921-0.958) and on daily-aggregate metrics (daily Mean Absolute Percentage Error [MAPE] = 18%; 95% CI, 15-20%), without significant performance differences across subgroups.
Conclusions:
Our algorithm can accurately classify ambulatory status using a wrist-worn device in real-world settings, with generalizability across demographic subgroups. Clinical Trial: NA
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.