JMIR Publications

ABSTRACT

Background:

Digital mental health interventions (DMHIs) using conversational artificial intelligence (AI) agents are increasingly adopted as scalable alternatives to traditional care. Engagement is typically measured using volume-based metrics (e.g., session counts, total time on a platform). These metrics overlook engagement patterns over time, which are not well understood in relation to mental health outcomes.

Objective:

The purpose of this cross-sectional study was to explore how different patterns of engagement with Mental’s AI conversational agent relate to self-reported anxiety and depression. We aimed to 1) identify and describe engagement profiles based on patterns of interaction depth and temporal consistency, 2) compare anxiety and depression symptoms across engagement profiles, and 3) explore whether engagement profiles were associated with mental health symptom severity.

Methods:

This cross-sectional observational study linked survey responses to backend app usage data from 112 Mental app users who completed at least 5 sessions with the conversational AI agent. Engagement profiles were derived using median splits on interaction depth (alpha parameter) and temporal consistency (Gini coefficient). Depression was assessed using the PHQ-8 and anxiety using the GAD-7. One-way ANOVAs compared symptoms across profiles. Linear regression models examined associations between profiles and symptom severity, adjusting for age, gender, and total duration of use.

Results:

We identified four distinct engagement profiles based on interaction depth and temporal consistency: extended and episodic (Profile 1; n=25), extended and consistent (Profile 2; n=31), brief and episodic (Profile 3; n=31), and brief and consistent (Profile 4; n=25). Users in Profile 1 (extended and episodic) reported the lowest anxiety (mean 2.68, SD 2.43) and depression (mean 3.48, SD 4.06), while Profile 4 (brief and consistent) reported the highest anxiety (mean 10.00, SD 7.03) and depression (mean 10.60, SD 8.75). Significant differences were observed for anxiety (F(3, 108)=8.07, P<.001, η²=.18) and depression (F(3, 108)=5.47, P=.002, η²=.13). In adjusted models, engagement profile was significantly associated with depression (R²=.16, F(7, 104)=2.87, P=.009) and anxiety (R²=.21, F(7, 104)=4.04, P<.001). Compared to Profile 1, users in Profiles 2 and 4 reported significantly higher depression and anxiety. Profile 3 differed from Profile 1 for anxiety only (β=3.08, P=.048).

Conclusions:

Users with longer, clustered sessions reported the lowest symptoms, while those with brief, evenly distributed use reported the highest, suggesting concentrated engagement may be more impactful than fragmented use. These findings highlight the importance of considering how engagement unfolds over time, and that pattern-based measurement could improve understanding of user outcomes in AI-delivered mental health care. Future work should examine whether tailoring interventions to different engagement patterns can improve outcomes.