Accepted for/Published in: Journal of Medical Internet Research
Date Submitted: Oct 10, 2024
Open Peer Review Period: Oct 10, 2024 - Dec 5, 2024
Date Accepted: Jan 15, 2025
(closed for review but you can still tweet)
Perspectives and Experiences with Large Language Models in Healthcare: Survey insights from the workforce
ABSTRACT
Background:
Large Language Models (LLMs) can potentially transform how we use data, including within the healthcare sector. Understanding stakeholder perspectives on new technology is crucial for successful implementation.
Objective:
1. Investigate users' uptake, perceptions, and experiences regarding LLMs in healthcare. 2. Analyse survey responses based on demographic profiles.
Methods:
We administered an electronic survey to elicit stakeholder experiences and views of LLMs. Survey domains included: Demographics, user experiences of LLMs, motivations for using LLMs, and perceived impact on functional roles. The survey targeted healthcare providers, support staff, healthcare students and academics in health-related fields. Respondents were adults (>18 years) aware of LLMs.
Results:
We received n=1083 responses, of which n=845 were analysable. Users primarily adopted LLMs for speed, convenience, and productivity. Non-users were more likely to be healthcare workers (p<0.001), older (p<0.001), and female (p<0.01). While 79% agreed that the user experience was positive, 46% found the generated content unhelpful. Non-users were less likely to report LLMs meeting unmet needs (45% versus 65%) or improving functional roles (63% versus 75%). Free-text opinions highlighted concerns regarding autonomy, outperformance, and reduced demand. Respondents also predicted changes to human interactions, including fewer but higher quality interactions and a change in consumer needs, which would require provider adaptation.
Conclusions:
Despite the reported benefits of LLMs, the non-user profile—healthcare workers, older individuals, and females—underscores the need for targeted education and support to address adoption barriers and ensure the successful integration of LLMs in healthcare. Anticipated role changes, evolving human interactions, and the risk of the digital divide further emphasise the need for careful integration and ongoing evaluation of LLMs in healthcare to ensure equity and sustainability. Clinical Trial: N/A
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.