Accepted for/Published in: Journal of Medical Internet Research
Date Submitted: Jul 6, 2023
Date Accepted: Nov 24, 2023
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Evaluation of the Efficacy of Artificial Intelligence Chatbots for Digital Health: A Scoping Review
ABSTRACT
Background:
Artificial Intelligence (AI) conversational agents, such as chatbots, have become ubiquitous in our daily lives, enabling natural language conversations with users through various modes of communication. Conversational agents have the potential to play a significant role in promoting health and well-being. As the number of studies and available products related to conversational agents continues to rise, there is a critical need to assess the product features to enhance the design of conversational agents that effectively promote health and behavioral change.
Objective:
This scoping review provides a comprehensive assessment of the efficacy of AI chatbots for digital health, including the chatbots’ characteristics, user backgrounds, communication models, building relational capacity, personalization, interaction, and responses to suicidal thoughts. The second study objective is to understand the users’ in-app experiences during app usage.
Methods:
This review follows Arksey and O’Malley's scoping review methodology. Two different approaches were employed to identify relevant chatbots and studies: search for AI chatbot apps in the iOS and Android app stores and in scientific journals or conference articles through a search strategy designed by a librarian. After screening the chatbots, 36 chatbots were identified and evaluated by 10 research assistants for analysis through simulated conversations.
Results:
We compiled a dataset of 36 chatbots and evaluated their features, conversational capabilities, and user experiences. Most chatbots were rated for all ages or teenagers, and their sizes and developers varied. Chatbots utilized text, animations, speech, images, and emojis for communication. Personalization options and relational capacity varied, with some chatbots demonstrating empathy and humor. Approximately 44% of chatbots addressed suicidal thoughts effectively. These findings provide insights for improving the design and functionality of AI chatbots in digital health interventions.
Conclusions:
AI chatbots have the potential to revolutionize digital health interventions by offering scalable and personalized solutions for behavior change. Future research should focus on addressing limitations, exploring real-world user experiences, and evaluating long-term effectiveness to optimize their impact on healthcare delivery and digital health interventions.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.