JMIR Publications

ABSTRACT

Background:

Floodlight Open is a global, open-access, fully remote, digital-only study designed to understand the drivers and barriers in deployment and persistence of use of a smartphone app for measuring functional impairment in a naturalistic setting and broad study population.

Objective:

To assess measurement equivalence properties of the Floodlight Open app across operating system (OS) platforms, OS versions and smartphone device models.

Methods:

Floodlight Open enrolled adult participants with and without self-declared multiple sclerosis (MS). The study utilized the Floodlight Open app, a “bring-your-own-device” (BYOD) solution that remotely measured MS-related functional ability via smartphone sensor-based active tests. Measurement equivalence was assessed in all evaluable participants by comparing the performance on the six active tests (i.e., tests requiring active input from the user) included in the app across OS platforms (iOS vs Android), OS versions (iOS versions 11–15 and separately Android versions 8–10 [comparing each OS version with the other two OS versions pooled together]), and device models (comparing each device model with all remaining device models pooled together). The tests in scope were: Information Processing Speed (IPS), IPS Digit–Digit (IPS DD, measuring reaction speed), Pinching Test (PT), Static Balance Test (SBT), U-Turn Test (UTT) and Two-Minute Walk Test (2MWT). Group differences were assessed by permutation test for the mean difference after adjusting for age, sex and self-declared MS disease status.

Results:

Overall, 1,976 participants using 206 different device models were included in the analysis. Differences in test performance between subgroups were very small or small, with percent differences generally being ≤5% on the IPS, IPS DD, UTT, and 2MWT; <20% on the PT; and <30% on the SBT). No statistical significant differences were observed between OS platforms other than the PT (P<.001). Similarly, differences across iOS or across Android versions were non-significant after correcting for multiple comparison using false discovery rate correction (all adjusted P>.05). Comparing the different device models revealed a statistically significant difference only on the PT for 4/17 models (adjusted P=<.001–.03).

Conclusions:

Consistent with the hypothesis that smartphone sensor-based measurements obtained with different devices are equivalent, this study showed no evidence for a systematic lack of measurement equivalence across OS platforms, OS versions and device models on six active tests included in the Floodlight Open app. These results are compatible with the use of smartphone-based tests in a BYOD setting but more formal tests of equivalence would be needed.