JMIR Publications

ABSTRACT

Background:

Conversational artificial intelligence (CAI) emerges as a promising new digital technology for mental healthcare. CAI applications, like psychotherapeutic chatbots, are already available in app stores.

Objective:

This scoping review aims to provide a comprehensive overview of the ethical considerations surrounding the use of CAI as a therapist for individuals with mental health disorders. The secondary aim is to delineate future research directions in this evolving field.

Methods:

We conducted a systematic search in PubMed, Embase, APA PsycINFO, Web of Science, Scopus, The Philosopher’s Index, and ACM Digital Library. Our search comprised three elements concerning embodied AI, ethics, and mental health, separated by AND commands. We defined CAI as a conversational agent that interacts with a person and uses NLP to formulate output. We included articles discussing ethical challenges related to AI-driven conversational agents that are aimed at functioning as a therapist for individuals with mental health issues. We added additional articles through snowball searching. We only included articles in English or Dutch. Additionally, all types of articles were considered except abstracts of symposia . Screening for eligibility was done by two independent researchers (MRM and TS). An initial charting form was made based on the expected considerations and further revised and complemented during the charting process. The ethical challenges were divided into different themes. When a certain concern occurred in more than two articles, we identified it as a distinct theme.

Results:

We included 73 articles, of which 90% were published in 2018 or later. Most were reviews (27%) followed by articles that used empirical data collection methods such as surveys or other qualitative methods (14%). The following 10 themes were distinguished: (1) Harm (reduction) and safety (discussed in 52% of articles), the most common topics within this theme were suicidality and crisis management, harmful or wrong suggestions, and the risk of dependency on CAI; (2) Explicability, transparency, and trust (25%), including topics such as the effects of “black-box” algorithms on trust; (3) Responsibility and accountability (26%); (4) Empathy and humanness (21%); (5) Justice (33%), including themes such as health inequalities due to differences in digital literacy; (6) Anthropomorphisation and deception (18%); (7) Autonomy (11%); (8) Effectiveness (30%); (9) Privacy and confidentiality (64%); and (10) Concerns for healthcare workers’ jobs (12%). Other themes were discussed in 14% of articles.

Conclusions:

Our scoping review has comprehensively covered various ethical aspects of CAI in mental healthcare. However, certain themes, including the climate impact of AI, the responsibility gap, and especially the nuanced examination of therapeutic processes, are less explored . Additionally, the scarcity of qualitative studies and underrepresentation of key stakeholders highlight areas for future research to deepen our understanding of the ethical implications of CAI in mental health.