JMIR Publications

ABSTRACT

Background:

Ecological momentary assessments (EMA) are short, repeated surveys designed to collect information on experiences in real-time, real-life contexts. Embedding periodic bursts of EMA within cohort studies enables the study of experiences on multiple timescales and could greatly enhance the accuracy of self-reported information. However, burden on participants may be high and should be minimised to optimise EMA response rates.

Objective:

To evaluate the effects of study design features on EMA response rates.

Methods:

Embedded within an ongoing cohort study (Health@NUS), 3 bursts of EMAs were implemented over a 7-month period (April to Oct 2021). The response rate (% of completed EMA surveys from all sent EMA surveys) for each burst was examined. Following a low response rate in Burst 1, changes were made to the subsequent implementation strategy (text message announcements instead of emails). In addition, two consecutive randomised controlled trials (RCTs) were conducted to evaluate the efficacy of (1) four different reward structures (with fixed and bonus components) and (2) two different schedule lengths (7 or 14 days), on changes to the EMA response rate. Analyses were conducted in 2021-2022 using ANOVA and ANCOVA to examine group differences, and mixed models to assess changes across all three bursts.

Results:

Participants (N=384) were university students (60% female, mean age 23 (SD 1.3) years). Changing the reward structure did not significantly change the response rate (F(3, 380) = 1.75, p = 0.157). Changing the schedule length did significantly change the response rate (F(1, 382) = 6.23, p = 0.013); the response rate was higher for the longer (14-day) schedule (M = 48.34% (SD 33.17)), as compared to the shorter (7-day) schedule (M = 38.52% (SD 33.44)). The average response rate was higher in Bursts 2 and 3 (50.56 (SD 33.61), 48.34 (SD 33.17), respectively) as compared to Burst 1 (25.78 (SD 30.12)) and the difference was statistically significant (F = 93.83, p = <0.001).

Conclusions:

Small changes to the implementation strategy (text messages instead of emails) may have contributed to increasing the response rate over time. Changing the available rewards did not lead to a significant difference in the response rate, whereas schedule length did. Our study provides novel insights on how to implement EMA surveys in ongoing cohort studies. This knowledge is essential for conducting high-quality studies using EMA surveys. Clinical Trial: Clinical Trials identifier: NCT05154227