JMIR Publications

ABSTRACT

Background:

The Floodlight Open app is a digital health technology tool (DHTT) that comprises of remote, smartphone sensor-based tests (“daily activities”) for assessing symptoms of multiple sclerosis (MS). User acquisition, engagement, and retention remains a barrier in the successful deployment of such tools.

Objective:

To quantitatively and qualitatively investigate key user experience (UX) factors associated with the Floodlight Open app.

Methods:

Floodlight Open is a global, open-access, digital-only study designed to understand the drivers and barriers in deploying a DHTT in a naturalistic setting without supervision and onboarding by a clinician. Daily activities included tests assessing cognition (Information Processing Speed [IPS], IPS Digit-Digit), hand-motor function (Pinching Test, Draw a Shape Test), and postural stability and gait (Static Balance Test [SBT], U-Turn Test [UTT], 2-Minute Walk Test [2MWT]). All daily activities except the 2MWT were taken in a fixed sequence. Qualitative UX was studied through semi-structured interviews in a substudy of US participants with MS. The quantitative UX analysis investigated the impact of new UX design features on user engagement and retention in US participants for three separate test series: all daily activities included in the fixed sequence (DA), all daily activities included in the fixed sequence except the SBT and UTT (DA*), and the 2MWT.

Results:

The qualitative UX substudy (n=22) revealed the need for two new UX design features: a more seamless user journey during the activation process that eliminates the requirement of switching back and forth between the app and the email that the participants received upon registration, and configurable reminders and push notifications to help plan and remind the participants to complete their daily activities. Both UX design features were assessed in the quantitative UX analysis. Introducing the more seamless user journey (original user journey: n=608; more seamless user journey: n=481) improved the conversion rate of participants who enrolled in the study and proceeded to successfully activate the app from 54.05% to 74.63%. Introduction of reminders and push notifications (n=350 with and n=172 without reminders and notifications) improved continuous usage time (proportion of participants with ≥3 consecutive days of usage: ~30% vs ~12% for DA and DA*, ~30% vs ~20% for 2MWT), test completion rates (maximum number of test series completed: 279 vs 64 for DA, 283 vs 126 for DA*, 302 vs 76 for 2MWT), and user retention rates (at Day 30: 30.61% vs 9.76% for DA, 30.61% vs 17.18% for DA*, 22.44% vs 6.25% for 2MWT). Inactivity times remained comparable.

Conclusions:

The remote assessment of MS with DHTTs is a relatively nascent but growing field of research. The continued assessment of UX and improvement of UX design features can play a crucial role in the successful long-term adoption of new DHTTs.