JMIR Publications

ABSTRACT

Background:

Ecological Momentary Assessment (EMA) is pivotal in longitudinal health research in youth, but potential bias associated with non-participation, omitted reports, or dropout threaten its clinical validity. Prior meta-analytic evidence is inconsistent regarding specific determinants of missing data.

Objective:

This meta-analysis aims to update and expand upon previous research by examining key participation metrics—acceptance, compliance, and retention—in youth EMA studies. Additionally, it seeks to identify potential moderators among sample and design characteristics, with the goal of better understanding and mitigating the impact of missing data.

Methods:

We revised the bibliographic database search to identify EMA studies involving children and adolescents published from 2001 to November 2023. Eligible studies utilized mobile-delivered EMA protocols in samples with average age up to 18 years. We conducted separate meta-analyses for acceptance, compliance, and retention rates, and performed meta-regressions to address sample and design characteristics. Furthermore, we extracted and pooled sample-level effect sizes related to correlates of response compliance. Risk of publication bias was assessed using funnel plots, regression tests, and sensitivity analyses targeting inflated compliance rates.

Results:

We identified 285 samples including 17,441 participants, aged 5 to 17.96 years (mean age 14.22, SD 2.24, mean %-female 55.7%). Pooled estimates were 67.27% (k=88, 95% CI 62.39 to 71.96) for acceptance, 71.97% (k=216, 95% CI 69.83 to 74.11) for compliance, and 96.57% (k=169, 95% CI 95.42 to 97.56) for retention. Despite overall poor moderation of participation metrics, acceptance rates decreased with an increase in EMA items (log-transformed, β = -0.115, SE = 0.036; 95% CI -0.185 to -0.045, P=.001, R2=19.98), compliance rates declined by 0.8% per year of publication (SE 0.25, 95% CI -1.3 to -0.3, P=.002, R2=4.17), and retention rates dropped with increasing study durations (log-transformed, β = -0.061,SE 0.015, 95% CI -0.091 to -0.091, P=<.001, R2=10.06). Benefits of monetary incentivization on response compliance attenuated with increasing proportion of female participants (β=-0.002, SE 0.001; 95% CI -0.003 to -0.001, P=.003, R2=9.47). Within-sample analyses showed a small but significant effect indicating higher compliance in girls compared to boys (k=25, g=0.18, 95% CI 0.06-0.31, P=.003), but no significant age-related effects were found (k=14, z score = 0.05, 95% CI -0.01 to 0.16).

Conclusions:

Despite a 5-fold increase in included effect sizes compared to the initial review, the variability in rates of missing data based one can expect based on specific sample and design characteristics remains substantial. The inconsistency in identifying robust moderators highlights the need for greater attention to missing data and its impact on study results. To eradicate any health-related bias in EMA studies, researchers should collectively increase transparent reporting practices, intensify primary methodological research, involve participants’ perspectives on missing data. Clinical Trial: PROSPERO CRD42022376948; https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42022376948