Accepted for/Published in: Journal of Medical Internet Research
Date Submitted: Jul 12, 2023
Date Accepted: Sep 29, 2023
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Conversational agents in healthcare: expert-informed definition, classification, and conceptual framework
ABSTRACT
Background:
Conversational agents (CAs), or chatbots, are computer programs that simulate conversations with humans. The use of CAs in healthcare settings is recent and rapidly increasing, which often translates to poor reporting of the CA development and evaluation processes, and unreliable research findings. We developed and published a conceptual framework for Designing, developing, evaluating, and Implementing a Smartphone-delivered, rule-based COnVERsational agent (DISCOVER) consisting of three iterative stages of CA’s design, development, evaluation, and implementation, complemented by two cross-cutting themes (user-centered design and data privacy and security).
Objective:
This study aimed to perform in-depth, semi-structured interviews with multidisciplinary experts in healthcare CAs to share their views on the definition and classification of healthcare CAs and evaluate and validate the DISCOVER conceptual framework.
Methods:
We conducted one-on-one, semi-structured interviews via Zoom with 12 multidisciplinary CA experts using an interview guide based on our framework. The interviews were audio recorded, transcribed by the research team, and analyzed using thematic analysis.
Results:
Following participants input, we defined CAs as digital interfaces that use natural language to engage in a synchronous dialogue using one or more communication modalities such as text, voice, images, or video. CAs were classified into 13 categories: response generation method, input and output modalities, CA purpose, deployment platform, CA development modality, appearance, length of interaction, type of CA-user interaction, dialogue initiation, communication style, CA personality, human support, and type of healthcare intervention. Experts considered that the conceptual framework could be adapted for AI-based CAs. However, despite recent advances in AI, including large language models, the technology is not able to ensure safety and reliability in healthcare settings. Finally, aligned with participants’ feedback, we present an updated iteration of the Conceptual framework for Healthcare conversational AgenTs (CHAT) with key considerations for CA design, development, evaluation, and implementation, complemented by three cross-cutting themes, ethics, user involvement and data privacy and security.
Conclusions:
We presented an expanded, validated Conceptual framework for Healthcare conversational AgenTs (CHAT) aimed to guide researchers from a variety of backgrounds and expertise in the design, development, evaluation, and implementation of rule based CAs in healthcare settings.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.