Accepted for/Published in: JMIR Human Factors
Date Submitted: Nov 15, 2023
Open Peer Review Period: Nov 15, 2023 - Dec 13, 2023
Date Accepted: Feb 18, 2024
(closed for review but you can still tweet)
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Feasibility of a Voice-only Chatbot with Anthropomorphic Features Compared to a Text-based Chatbot as a Responder to Questions on Mental Health: a Randomized, Controlled Trial among Healthy Participants
ABSTRACT
Background:
The use of chatbots in mental health support has increased exponentially in recent years, with studies showing that they may be effective in treating mental health problems. More recently, the use of voice-controlled visual avatars called digital humans has been introduced, which use machine learning and mimicry of facial emotions to build emotional engagement with users. It is important to study the difference in emotional response and usability preferences between text-based chatbots and digital humans for interacting with mental health services.
Objective:
The aim of this study was to explore to what extent a voice-only and a text-only chatbot interface differed on usability when tested by healthy participants, using BETSY (Behavior, Emotion, Therapy System, and You) which employs two distinct interfaces: a voice-only user interface with anthropomorphic features and a text-only user interface. We also set out to explore how chatbot-generated conversations on mental health (specific to each interface) affected self-reported feelings and biometrics.
Methods:
We explored to what extent a voice-only chatbot with anthropomorphic features differed from a traditional text-only chatbot regarding perception of usability though the System Usability Scale (SUS-10), emotional reactions though electroencephalography, and feeling of closeness. Healthy participants (n=45) were randomized to two groups that used a voice-only chatbot with anthropomorphic features (n=25) or a text-only chatbot with no such features (n=20). The groups were compared by linear regression analysis and t-tests.
Results:
No differences were observed between the text-only and voice-only group regarding demographic features. Mean (SD) SUS-10 score was 75.34 (10.01) [range, 57-90] for text-only chatbot versus 64.80 (14.14) (range, 40-90) for the voice-only chatbot. Both groups scored their respective chatbot interfaces as average or above average in usability. Women were more likely to report feeling annoyed by BETSY.
Conclusions:
The text-only chatbot was perceived as significantly more user-friendly than the voice-only chatbot, although there were no significant differences in EEG measurements. Male participants exhibited lower levels of annoyance with both interfaces contrary to previously reported findings.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.