JMIR Publications

ABSTRACT

Background:

Experience sampling methods (ESM) are increasingly being used to study ecological emotion dynamics in daily functioning through repeated assessments taken over several days. However, most of these ESM approaches are only based in self-report assessments and therefore, the study of the ecological trajectories of their underlying mechanisms is scarce (ie, cognitive biases), which require to be evaluated through experimental tasks. We developed a novel ESM tool that integrates self-reported measures of emotion and emotion regulation together with an online cognitive task that allows to assess underlying mechanisms during daily functioning.

Objective:

The objective of the study was to validate this new tool, studying its usability and the possible factors related to the compliance with it, in terms of latency and missing responses. Among compliance predictors, we considered psychological and time-related variables, as well as usability, measurement reactivity and participants’ satisfaction with the tool.

Methods:

We carried out two ESM studies, Study 1 (N=84; 3 assessments per day for 5 days), and Study 2 (N=135; 3 assessments per day for 10 days).

Results:

In both studies, participants found the tool highly usable (average usability score > 81). Mixed regression models were conducted to test predictors of compliance. In Study 1, latency and missing responses were significantly predicted by day (P<.001, and P=.03, respectively). Participants showed slower responses to the notification and missing assessments as days of the study progressed. Also, latency was significantly predicted by app usefulness (P=.02), indicating participants who found the app as more useful showed slower responses. In Study 2, latency was significantly predicted by day (P=.003) and app overload (P=.03), while missing responses were predicted by time of the day (P=.014) and stress reactivity to the use of the tool (P=.04). Thus, participants also reported slower responses to notifications as days of the study progressed and when participants perceived higher app overload. Higher missing responses were registered at later notifications as well as by participants showing higher reactive stress.

Conclusions:

The new tool had high levels of usability. Further, the study of compliance is of enormous importance when implementing an ESM design as ours, including an online cognitive task. The main predictors of latency and missing responses found across studies are methodology-related variables. Future research integrating tasks in ESM designs should take these results into consideration, facilitating methodological quality and performing a more accurate estimation of participants’ response rates.