JMIR Publications

ABSTRACT

Background:

With the proliferation of digital mental health interventions (DMHIs) guided by relational agents, little is known about the behavioral, cognitive and affective engagement components associated with symptom improvement over time. Obtaining a better understanding could lend clues about recommended use for particular subgroups of the population, the potency of different intervention components, and the mechanisms underlying the intervention’s success.

Objective:

This exploratory study applied clustering techniques to a range of engagement indicators mapped to the intervention’s active components and the Connect, Attend, Participate, and Enact (CAPE) model to examine the prevalence and characterization of each identified cluster among users of a relational agent-guided DMHI.

Methods:

Adults aged 18 or older were invited via social media to participate in an 8-week intervention of a DMHI guided by the natural language processing (NLP)-supported relational agent, Woebot. Users completed assessments of affective and cognitive engagement: working alliance as measured by Goal and Task working alliance subscale scores and enactment (i.e., application of therapeutic recommendations in real-world settings). The app collected behavioral engagement (i.e., utilization). We applied agglomerative hierarchical clustering analysis to the engagement indicators to identify the number of clusters best fit to the data collected, characterized them, and then examined associations with baseline demographic and clinical characteristics as well as mental health outcomes at Week 8.

Results:

Analyses (n=202) supported 3 clusters: 1) “Typical Utilizers” (n=81, 40%), who had intermediate levels of behavioral engagement; 2) “Early Utilizers” (n=58, 29%), who had the highest levels of behavioral engagement in week 1; and 3) “Efficient Engagers” (n=63, 31%)., who had significantly higher levels of affective and cognitive engagement, but the lowest level of behavioral engagement. With respect to mental health baseline and outcomes measures, “Efficient Engagers” had significantly higher levels of baseline resilience (p=.0004) and greater declines in depressive symptoms (p=.01) and stress (p=.01) from baseline to Week 8 compared to Typical Utilizers. Significant differences across clusters were found by age, gender identity, race/ethnicity, sexual orientation, education, and insurance coverage. The main analytic findings remained robust in sensitivity analyses.

Conclusions:

Three distinct, robust engagement clusters were identified, which were characterized by varying baseline demographic and clinical characteristics and differing mental health outcomes. Additional research is needed to inform fine-grained recommendations regarding optimal engagement and to determine the best sequence of particular intervention components with known potency. The findings represent an important first step in disentangling the complex interplay between different affective, cognitive, and behavioral engagement indicators and outcomes associated with use of a DMHI incorporating an NLP-supported relational agent. Clinical Trial: This study was retrospectively registered on ClinicalTrials.gov (#NCT05672745).