Accepted for/Published in: JMIR mHealth and uHealth
Date Submitted: Aug 17, 2019
Date Accepted: Feb 7, 2020
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
A Questionnaire for Assessing User Satisfaction with mHealth Applications: development using Rasch measurement theory
ABSTRACT
Background:
Mobile Health (mHealth) applications (apps) offer great opportunities to conduct large-scale, cost-efficient digital solutions for implementing lifestyle change. Furthermore, many mHealth apps act as medical devices. Yet, there is little research on how to assess user satisfaction an with mHealth solution.
Objective:
To examine the measurement properties in the development of the mHealth Satisfaction Questionnaire.
Methods:
Respondents who took part in the Health Integrator Study and were randomized to use the Health Integrator smartphone app for lifestyle changes (n=112), with and without additional telephone coaching, rated their satisfaction with the app using the new 14-item mHealth Satisfaction Questionnaire. The ratings were done using a 5-step Likert scale and measurement properties were evaluated with Rasch measurement theory.
Results:
Optimal scoring was reached when response options 2, 3 and 4 were collapsed, giving 3 response categories. After omitting two items that did not fit into the scale, fit residuals were within, or close, to the recommended range +2.5. There was no differential item functioning (DIF) between intervention group, age group or sex. The Person Separation Index was 0.79 indicating that the scale's ability to discriminate correctly between person leniency was acceptable for group comparisons, but not for individual evaluations. The scale did not meet the criterion of unidimensionality, 16 % of the respondents were outside the desired range of −1.96 to 1.96. In addition, several items showed local dependency and three underlying dimensions emerged: negative experiences, positive experiences, and lifestyle consequences of using the mHealth solution.
Conclusions:
In times where mHealth apps and digital solutions are given more attention, the mHealth Satisfaction Questionnaire provides a new possibility to measure user satisfaction to ensure usability and improve development of new apps. Our study is one of only few cases where Rasch measurement theory has been used to evaluate usability of such an instrument. There is, though, need for further development of the mHealth Satisfaction Questionnaire, including adding more items and considering further response options. The mHealth Satisfaction Questionnaire should also be evaluated in a larger sample and with other mHealth apps and in other contexts. Clinical Trial: ClinicalTrials.gov Identifier: NCT03579342
Citation
