Accepted for/Published in: JMIR mHealth and uHealth
Date Submitted: Nov 26, 2019
Date Accepted: May 14, 2020
Date Submitted to PubMed: May 18, 2020
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
User engagement in the wild: Effective use of digital ecological momentary assessment
ABSTRACT
Background:
User interaction event logs provide rich and large datasets that can reveal valuable insights into how people engage with technology. Additionally, approaches such as ecological momentary assessment can be used to gather accurate real-time data in an individual’s native environment by asking questions ‘in the moment’.
Objective:
The purpose of the study is to evaluate engagement and response to ecological momentary assessment questions using a case study of an app used by people with dementia and their carers for reminiscence. The overall goal of this research is to inform ecological momentary assessment use in app design.
Methods:
A feasibility trial was conducted in which participants (n=56) used the app over a 12-week period. Half of the participants were people living with dementia (n=28) and half were carers (n=28) with an average age of 73 ± 13 (SD). Questions were presented to individuals at various points which they could chose to answer or dismiss. Presentation and dismissal rates for questions were explored over hours of the day and across trial weeks. Engagement and response to questions presented to users following reminiscence with photos, videos and music was investigated.
Results:
Overall compliance with ecological momentary assessment was high, with 69.1% of questions answered when presented. Questions presented in the evening had the lowest dismissal rate. The dismissal rate for questions presented at 9pm (10%) was significantly lower compared to 11am (50%) (χ2=21.4, P < .001). Dismissal rates were high at the beginning of the trial but decreased after a few weeks, for example, the dismissal rate in trial week 8 (10%) was significantly lower than in trial week 2 (55%) (χ2=19.2, P < .001). Questions asked following reminiscence with personal media had the highest dismissal rate compared to generic photos, video and music (P < .001 for all). In contrast, questions asked after the user had listened to generic music had significantly lower dismissal rates compared to personal music, photos and videos (P < .001 for all).
Conclusions:
The main limitation of our study was the generalisability of results to a larger population given the sample size, quasi-experimental design, and older demographic where half of participants were people living with dementia. However, this study shows that older people are willing to participate and engage in ecological momentary assessment. Based on this study we propose a series of recommendations for app design to increase user engagement with ecological momentary assessment. These include presenting questions after 8pm in the evening, only after the individual has been using the app for several weeks, and only if the user is not trying to complete a task within the app.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.