JMIR Publications

ABSTRACT

Background:

Mental health disorders are widespread, but access to evidence-based mental health interventions is limited. Conversational agents (CAs) or chatbots are computer programs that mimic human conversation. They have the potential to improve access to mental health interventions through automated, scalable, and personalized delivery of psychotherapeutic content through a human-like interaction. However, digital health interventions, including those delivered by CAs, often have high attrition rates. Identifying the factors associated with attrition is critical to improving future clinical trials and leveraging the potential of these interventions.

Objective:

This review aimed to estimate the overall and differential rates of attrition in CA-delivered mental health interventions, evaluate the impact of study design and intervention-related aspects on attrition, and describe study design features aimed at reducing or mitigating study attrition.

Methods:

Systematic review and meta-analyses of overall and differential attrition of CA-delivered mental health intervention RCTs was conducted in June 2022. Five databases were searched, and a grey literature search was also conducted. We included randomized-controlled trials (RCTs) that compared CA-delivered mental health intervention against control groups and excluded studies that lasted for one session only and used “Wizard-of-Oz” interventions. We also assessed the risk of bias in the included studies using Risk-of-Bias Tool 2.0. Random-effects proportional meta-analysis was applied to calculate the pooled dropout rates in intervention groups. Random-effects meta-analysis was used to compare the attrition in the intervention compared to control groups. We used a narrative review to summarize the findings.

Results:

The systematic search retrieved 5,197 records from peer-reviewed databases and citation searches and 41 RCTs met the inclusion criteria. The mean meta-analytic overall attrition rate in the intervention group was 21.84% (95% CI [16.74, 27.36]), I2 = 94%. Short-term studies that lasted 8 weeks or shorter showed a lower attrition rate at 18.05% (95% CI [9.91, 27.76]), I2 = 94.6, compared to long-term studies that lasted more than 8 weeks, 26.59% (95% CI [20.09, 33.63]), I2 = 93.89. Intervention group participants were more likely to attrit compared to control group participants for short-term, log OR = 1.22 (95% CI [0.99, 1.50]), I2 = 21.89%, and long-term studies, log OR = 1.33 (95% CI [1.08, 1.65]), I2 = 49.43%. Intervention-related characteristics associated with higher attrition include stand-alone CA intervention without human support, not having a symptoms tracker feature, no visual representation of the CA, and comparing CA intervention with wait-list controls. No participant-level factor reliably predicted attrition.

Conclusions:

Our results indicated that about one-fifth of the participants will drop out from CA-delivered interventions for short-term studies. This was comparable to face-to-face mental health intervention and less than most digital health interventions based on other meta-analyses. However, there were high heterogeneities which made it difficult to generalize the findings. Our results suggested that future CA-delivered mental health interventions should adopt a blended design with human support, use symptom tracking, compare CA interventions against active controls rather than wait-list, and include a visual representation of the CA to reduce the attrition rate. Clinical Trial: CRD42022341415