JMIR Publications

ABSTRACT

Background:

Ecological momentary assessment (EMA) is widely used in mental health research to capture affective states and well-being in everyday life. Intensive longitudinal EMA studies require sustained participation, and low engagement and dropout can introduce bias. Frequent notifications are used to encourage engagement, but their associations with short-term engagement and longer-term dropout remain insufficiently understood.

Objective:

We examined (a) how notification order within a measurement window relates to the timing and likelihood of questionnaire opening, and (b) whether the total number of notifications relates to permanent disengagement, in a 12-month, multi-burst smartphone EMA study of adolescents’ well-being.

Methods:

We analyzed paradata from the FUTURE project, an observational EMA study of 201 adolescents (13–17 years). Four 14-day bursts each contained up to four short questionnaires per day, with up to 10 notifications per questionnaire window. Time from first notification to opening was modeled with Cox survival models with respondent-level frailty. Permanent dropout was modeled with logistic regression at the person-burst level. We tested whether demographics, timing, and the momentary phone context (screen activity and app use) moderated the notification-order association. Analyses used Stata 18.

Results:

A total of 26,007 questionnaires were notified. Of these, 17,668 (67.9%) were opened. Within a measurement window, later reminders were associated with a slightly higher chance of opening (hazard ratio [HR] 1.13, 95% CI 1.065-1.190; P<.001), and this estimate was stable across all models, and we found no sign of within-window reminder fatigue. The notification-order association varied only marginally by gender, age, and socioeconomic status. It was flatter when the screen was already on (HR 0.862, 95% CI 0.846–0.878), or when social media (HR 0.884, 95% CI 0.856–0.913) or messaging (HR 0.884, 95% CI 0.844–0.926) had been in use in the minute before the first prompt (all P<.001). Regarding study dropout, 86 of 201 participants (43%) disengaged permanently. More reminders within a burst were associated with slightly lower odds of study dropout (fully adjusted odds ratio [OR] 0.981, 95% CI 0.965–0.996; P=.01), a small association that was stable across all model specifications (OR 0.978–0.984).

Conclusions:

In this adolescent EMA study, most openings occurred shortly after the first prompt. Among windows still unopened, later reminders were associated with a slightly higher chance of opening, and we found no evidence of reminder fatigue. This association varied little across sociodemographic groups. Screen activity in the minute before the first prompt (and, less strongly, social-media or messaging use) was associated with faster opening. In these contexts, the chance of opening changed little across successive reminders. More within-burst reminders were associated with slightly lower dropout odds. Both context-timed prompting and targeted retention should be tested experimentally in adolescent mental health EMA.