JMIR Publications

ABSTRACT

Background:

Ecological Momentary Assessment (EMA) has the potential to minimize recall bias by having people report on their experiences in the moment (momentary model) or over short periods of time (coverage model). This potential hinges on the assumption that participants provide ratings based on the reporting timeframe instructions prescribed in the EMA items. However, it is unclear what timeframes participants are actually using when they answer EMA questions and whether participant training improves participants’ adherence to the reporting instructions.

Objective:

The objectives of this study are to investigate the reporting timeframes participants used when answering EMA questions and whether participant training improves participants’ adherence to the EMA reporting timeframe instructions.

Methods:

This study used telephone-based cognitive interviews to investigate this question. In a 2x2 factorial design, participants (n=100) were assigned to receive either basic or enhanced EMA training and also randomized to rate their experiences using a momentary (at the moment you were called) or coverage (since the last phone call) model. Participants received 5 calls over the course of one day to provide ratings; after each rating, participants were immediately interviewed about the timeframe that they used to answer the EMA questions. Two raters independently coded the momentary interview responses into timeframe categories (Cohen’s kappa = 0.64 (95%CI: 0.55-0.73)).

Results:

Results from the momentary conditions showed that most of the calls referred to the period during the call (28.6%) or just before the call (49.2%) to provide ratings; the remainder were from longer reporting periods. Multinomial logistic regression results indicated a significant training effect (χ2 (1, 199)=16.61, p<0.001), where the enhanced training condition yielded more reports within the intended reporting timeframes for momentary EMA reports. Cognitive interview data from the coverage model did not lend themselves to reliable coding and were not analyzed.

Conclusions:

These findings provide the first evidence about adherence to EMA instructions to reporting periods, and that enhanced participant training improves adherence to the timeframe specified in momentary EMA studies.