JMIR Publications

ABSTRACT

Background:

Ecological momentary assessment (EMA) is a measurement methodology that involves the repeated collection of real-time data on participants’ behavior and experience in their natural environment. While EMA allows researchers to gain valuable insights into dynamic behavioral processes, the need for frequent self-report can be burdensome and disruptive. Compliance with EMA protocols is important for accurate, unbiased sampling, yet there is no “gold standard” for EMA study design to promote compliance.

Objective:

The purpose of this study was to use a factorial design to identify optimal study design factors, or combinations of factors, for achieving the highest completion rates for smartphone-based EMAs.

Methods:

Participants recruited from across the United States were randomized to one of two levels on each of 5 design factors in a 2x2x2x2x2 design (32 conditions): Factor 1: Number of questions per EMA survey [15 vs. 25], Factor 2: Number of EMAs per day [2 vs. 4], Factor 3: EMA prompting schedule [random vs. fixed times], Factor 4: Payment type [$1 paid per EMA vs. payment based on the percentage of EMAs completed], and Factor 5: EMA response scale type [i.e., slider-type response scale vs. Likert-type response scale; this is the only within-person factor; each participant was randomized to complete slider or Likert-type questions for the first 14 days or second 14 days of the study period]. All participants were asked to complete prompted EMAs for 28 days. The effect of each factor on EMA completion was examined, as well as the effects of factor interactions on EMA completion. Finally, relations between demographic and socioenvironmental factors and EMA completion were examined.

Results:

Participants (N=411) were 48.4 (SD=12.1) years of age, 75.7% (311/411) female, 72.5% (298/411) White (74/411, 18.0% Black or African American, 11/411, 2.7% Asian, 6/411, 1.5% American Indian or Alaska Native, 22/411, 5.4% more than one race), and 9.6% (38/396) Hispanic/Latino. On average, participants completed 83.8% (28,948/34,552) of scheduled EMAs, and 96.6% (397/411) of participants completed the follow-up survey. Results indicated that there were no significant main effects of the design factors on compliance and also no significant interactions. Analyses also indicated that older adults, those without a history of substance use problems, and those without current depression tended to complete more EMAs than their counterparts. No other demographic or socioenvironmental factors were related to EMA completion rates. Finally, the app was well liked (i.e., System Usability Scale score = 82.7), and there was a statistically significant positive association between liking the app and EMA compliance (P=0.005).

Conclusions:

Study results have broad implications for developing best practices guidelines for future studies that employ EMA methodologies. Clinical Trial: Clinicaltrials.gov number: NCT05194228