Accepted for/Published in: JMIR Nursing
Date Submitted: Jan 1, 2024
Open Peer Review Period: Jan 4, 2024 - Feb 29, 2024
Date Accepted: May 24, 2024
(closed for review but you can still tweet)
Artificial intelligence-assisted decision making in long-term care: qualitative study on prerequisites for responsible innovation
ABSTRACT
Background:
While use of artificial intelligence (AI)-based technologies such as decision-support systems (AI-DSSs) could help sustaining and improving the quality and efficiency of care, their deployment also creates ethical and social challenges. In recent years, there has been a growing prevalence of high-level guidelines and frameworks to provide guidance on responsible AI innovation. However, few studies specify how AI-based technologies such as AI-DSSs can be responsibly embedded in specific contexts such as the nursing process in the long-term care (LTC) for older adults.
Objective:
Opportunities and prerequisites for responsible AI-assisted decision-making in the nursing process were explored from the perspectives of nurses and other professional stakeholders in LTC.
Methods:
Semi-structured interviews were conducted with 24 care professionals in Dutch LTC, including nurses, care coordinators, data specialists and care centralists. Two imaginary scenarios about the future use of AI-DSSs were developed beforehand and used to enable participants to articulate their expectations regarding the opportunities and risks of AI-assisted decision-making. After first openly discussing opportunities and possible risks associated with both scenarios, six high-level principles for responsible AI were used as probing themes to evoke further consideration on risks of using AI-DSSs in LTC. Further, participants were asked to brainstorm about possible strategies and actions in the design, implementation and use of AI-DSSs to address or mitigate the mentioned risks. A thematic analysis was carried out to identify opportunities and prerequisites for responsible innovation in this area.
Results:
Professionals’ stance towards the use of AI-DSSs is not a matter of purely positive or negative expectations, but rather a nuanced interplay of positive and negative elements that lead to a weighed perception of opportunities and prerequisites for responsible AI-assisted decision-making. Both opportunities and risks were identified in relation to early identification of care needs, guidance in devising care strategies, shared decision-making, and caregivers’ workload and work experience. To optimally balance opportunities and risks of AI-assisted decision-making, seven categories of prerequisites for responsible AI-assisted decision-making in the nursing process were identified: (1) regular deliberation on data collection, (2) a balanced proactive nature of AI-DSSs, (3) incremental advancements aligned with trust and experience, (4) customization for all user groups including clients and caregivers, (5) measures to counteract bias and narrow perspectives, (6) human-centric learning loops, and (7) routinization of using AI-DSSs.
Conclusions:
Opportunities of AI-assisted decision-making in the nursing process could turn into drawbacks, depending on the specific shaping of the design and the deployment of AI-DSSs. Therefore, we recommend viewing the responsible use of AI-DSSs as a balancing act. Moreover, given the interrelatedness of the identified prerequisites, we call for various actors, including developers and users of AI-DSSs, to cohesively address different factors important to the responsible embedding of AI-DSSs in practice.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.