JMIR Publications

ABSTRACT

Background:

Digital health interventions (DHIs) hold promise for improving health outcomes. Tough Talks COVID-19 (TT-C) was a culturally tailored, community-informed DHI designed to support Black young adults (YAs) in the US South in making informed COVID-19 vaccination decisions.

Objective:

To understand how user engagement, captured through paradata, was associated with improved vaccine attitudes in the TT-C DHI.

Methods:

We analyzed paradata from 299 Black YAs in Alabama, Georgia, and North Carolina who used the TT-C DHI in a randomized controlled trial. The DHI included 62 self-paced activities that were analyzed using rapid content analysis to identify the specific vaccine uptake-related domain they covered (hesitancy, confidence, conspiracy beliefs, and knowledge). Intervention engagement was defined as number of activities and time spent in the DHI. We used regression models to evaluate associations between intervention engagement and changes in vaccine-related measures after three months of use.

Results:

Overall engagement with the DHI was high.. Greater engagement with the DHI, both in total time and number of activities completed, was associated with larger improvements for all four vaccine outcomes at three months after download (P < 0.05). The effects of the TT-C DHI were amplified when participants engaged in activities that were directly related to the domain of interest, particularly with conspiracy beliefs, which had an effect size three times larger for domain-specific engagement (β: -0.067 [95% CI: -0.130, -0.005]) compared to domain-agnostic engagement (β: -0.019 [95% CI: -0.035, -0.002])

Conclusions:

Our findings underscore the value of using paradata to assess DHI effectiveness and highlight that the type of content engaged with is associated with change. These insights can guide future DHI design and real-time adaptive interventions to maximize impact. Clinical Trial: ClinicalTrials.gov NCT05490329; https://clinicaltrials.gov/study/NCT05490329