JMIR Publications

ABSTRACT

Background:

Consumers are increasingly seeking mental health advice and therapeutic support through large language model (LLM)–based chatbots due to their high accessibility; however, existing systems often lack systematic integration and fail to explain the mechanisms through which AI interactive features impacts users; this disconnects between conversational features and therapeutic mechanisms limits their potential for sustainable mental health improvements.

Objective:

This study aimed to evaluate the effectiveness of a fine-tuned LLM-based conversational agent (ActiAgent) for reducing sub-clinical depression and anxiety. Specifically, we sought to examine coping self-efficacy as a key mediating mechanism and explore early behavioral commitment as a predictor of psychological improvement.

Methods:

We conducted a 6-week randomized crossover intervention study among university students with sub-clinical depressive and anxiety symptoms (N=131). Participants were randomly assigned to either an immediate LLM-based intervention group or a waitlist control group. Depressive symptoms (PHQ‑9), anxiety symptoms (GAD‑7), and coping self-efficacy were assessed weekly. We systematically coded conversational features from human‑AI interaction logs, including early behavioral commitment, empathy density, question density, and advice density. Mediation analysis was performed to examine whether coping self-efficacy mediated the intervention effects on depressive and anxiety symptoms. Hierarchical linear regression was further used to test whether early behavioral commitment predicted improvements in coping self-efficacy.

Results:

Of 131 participants, 66 were assigned to the immediate intervention group and 65 to the waitlist control group. At Week 3, the ActiAgent group showed significantly greater reductions in depressive symptoms (t=–2.137; P=.034; d=0.37) and anxiety symptoms (t=–2.111; P=.037; d=0.37) compared with the waitlist group. In the crossover phase, the waitlist group demonstrated significant improvements after receiving ActiAgent (depression: t=3.29; P=.002; d=0.41; anxiety: t=2.14; P=.036; d=0.27), and groups did not differ at Week 6 (all P>.05). The intervention group exhibited significantly greater increases in coping self-efficacy at Week 3 (F=4.413; P=.038). Coping self-efficacy significantly mediated symptom reduction for depression (indirect effect=–0.44; 95% CI –1.06 to –0.03) and anxiety (indirect effect=–0.39; 95% CI –0.87 to –0.02). Early behavioral commitment at Week 1 was positively associated with coping self-efficacy gains at Week 2 (r=0.250; P=.047) and uniquely explained an additional 6.4% of variance in self-efficacy growth (β=0.252; t=2.030; P=.047)

Conclusions:

This study demonstrates that the theory-informed LLM counseling agent ActiAgent effectively reduces sub-clinical depressive and anxiety symptoms, with therapeutic gains mediated by improvements in coping self-efficacy. Early behavioral commitment during initial intervention sessions significantly predicts subsequent increases in coping self-efficacy, highlighting a key mechanism of therapeutic action. These findings support the value of theory-driven design and conversational feature engineering for LLM-based mental health tools and underscore the need for intentional engagement strategies to foster sustained psychological improvement.