JMIR Publications

ABSTRACT

Background:

To date, there is no comprehensive paper that systematically synthesizes the effect of generative AI chatbot’s impact on mental health. Can generative AI chatbots help reduce our psychological distress?

Objective:

To comprehensively assess existing evidence, a systematic review and meta-analysis is essential to evaluate the overall effectiveness, identify gaps, and guide future research in this evolving field. This paper aims to: 1) synthesize current evidence on generative AI chatbot interventions targeting mental health issues, 2) quantify the effectiveness of these interventions via a meta-analysis of randomized controlled trials (RCTs), and examine key moderators of intervention effectiveness.

Methods:

This systematic review included 26 studies for narrative synthesis, out of which 12 randomized controlled trials were included in the meta-analysis.

Results:

The systematic synthesis revealed that 1) generative AI-chatbot interventions mostly took place in non-WEIRD countries (Western, Educated, Industrialized, Rich, and Democratic) and 2) there is a lack of studies focusing on young children and older adults. The meta-analysis showed a statistically significant effect (ES = 0.36, p = .039), which means that generative AI chatbots are, on average, effective in reducing negative mental health issues. Among moderators, we found statistically significant and higher effect sizes among interventions that have an active control group, conducted in WEIRD countries, recruited non-clinical populations, older age, majority female, non-personalized, with human assistance, and social-oriented.

Conclusions:

In conclusion, this comprehensive review has highlighted the potential of generative AI chatbots in addressing anxiety, depression, negative mood, and stress. The findings indicate that generative AI interventions are particularly beneficial in WEIRD countries, among non-clinical populations, older adults, and females. Human-assisted and social-oriented programs, as opposed to fully autonomous or task-oriented ones, demonstrate greater effectiveness. Meanwhile, non-personalized chatbots appear to yield more effective outcomes than personalized systems.