Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Comparing AI Roles and Support Types in Digital Health: A Randomized Factorial Experiment with Patients and Providers
ABSTRACT
Background:
The increasing demand for digital health services has highlighted the need for innovative solutions to enhance accessibility and quality of care. AI has emerged as a promising tool, offering capabilities such as personalized treatment plans and virtual consultations. Despite these advancements, the acceptance of AI in digital health remains a critical concern, with potential ethical and practical challenges.
Objective:
The objective of this study is to evaluate how different AI roles and support types influence the perceptions of AI tools among both patients and healthcare providers.
Methods:
We recruited healthcare providers (N=251) and the general public (N=396) to examine their perceptions of AI healthcare chatbots. Participants were randomly assigned to interact with one of six LLM-based chatbots designed within a 2×3 factorial framework: AI roles (assisting patients vs. assisting providers) and support type (task, companion, or control). Following the interaction, participants reported perceived warmth and competence, emotional and informational support, and technology acceptance using validated scales. Chat histories were also recorded, from which we extracted linguistic features and conversation topics. We ran a factorial analysis of covariance (ANCOVA) on each perception construct and each linguistic feature, testing the main effects of role and support type and their interaction. We also fitted separate linear regression models for providers and patients to examine how perceptions varied with participants’ demographic characteristics and prior beliefs about AI.
Results:
We found that providers were generally more conservative than patients, reporting lower scores on five of the seven constructs. Independent of role, the task chatbot was rated highest on competence, informational support, attitudes, and intention to adopt, while the companion chatbot was rated warmer than the task chatbot but no higher than the general-purpose chatbot on warmth or emotional support. Participants’ conversations followed the same pattern, with those talking to the task chatbot asking more questions and those talking to the companion chatbot focusing more on their feelings. Across both groups, participants’ general trust in AI predicted their perceptions of every construct, and among patients, those who had a regular doctor, were female, had less education, or knew more about AI also gave higher ratings.
Conclusions:
This study contributes to understanding both providers’ and patients’ perceptions of using chatbots in different roles and support types, ultimately suggesting that future healthcare AI systems should build on domain-specific expertise and users’ prior trust in AI, while finding better ways to convey warmth that users actually perceive.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.