Accepted for/Published in: JMIR Human Factors
Date Submitted: Jul 12, 2022
Date Accepted: Jan 1, 2023
The Effects of a Healthcare Chatbot’s Complexity and Persona on User Trust, Perceived Usability, and Effectiveness: Mixed Methods Study
ABSTRACT
Background:
This rising adoption of telehealth provides new opportunities for more efficacious and equitable healthcare information mediums. The ability of chatbots to provide a conversational, personal, and comprehendible avenue for learning about healthcare information make them a promising tool for addressing healthcare inequity as healthcare trends continue toward online and remote processes. While chatbots have been studied in the healthcare domain for their efficacy for smoking cessation, diet recommendation, and other assistive applications, few studies have examined how specific design characteristics influence the effectiveness of chatbots at providing health information.
Objective:
Our objective was to investigate the influence of different design considerations on the effectiveness of an educational healthcare chatbot.
Methods:
A 2 x 3 between-subjects study was performed with two independent variables: a chatbot’s complexity of responses (technical or non-technical language) and the presented qualifications of the chatbot’s persona (doctor, nurse, or nursing student). Regression models were used to evaluate the impact of these variables on three outcome measures: effectiveness, usability, and trust. A qualitative transcript review was also done to review how participants engaged with the chatbot.
Results:
Analysis of 71 participants found that participants who received technical language responses were significantly more likely to be in the high effectiveness group (odds ratio [OR] 2.84, 95% CI 1.10-7.70, P=.03) Participants with higher health literacy (OR 1.88, 95% CI 1.05-3.54, P=.04), and participants who perceived the system as usable (OR 6.62, 95% CI 1.93-30.3, P=.006) were significantly more likely to trust the chatbot. Participants who trusted the chatbot were significantly more likely to perceive that chatbot as more useable (P=.003). The participants engaged with the chatbot in a variety of ways with some taking a conversational approach and others treating the chatbot more like search engine.
Conclusions:
Given their increasing popularity, it is vital that we consider how chatbots are designed and implemented. Factors like chatbot persona and language complexity are just two of the design characteristics of a chatbot that may influence their ability to provide efficacious healthcare information.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.