JMIR Publications

ABSTRACT

The role of artificial intelligence (AI), particularly large language models (LLM) like Chat Generative Pre-Trained Transformer (ChatGPT), has garnered significant attention in healthcare. This paper focuses on how user expertise and their trust in the technology can influence LLMs' effectiveness in healthcare. Our arguments instigate the following questions: are we and our healthcare system ready to integrate LLMs? If yes, is there a policy explicitly stating in what capacity it could be used to reduce clinical workload before its dissemination? Will the ease of generating content with AI stifle the development of creativity and critical thinking in medical students accustomed to technology providing immediate solutions? Additionally, we elucidate risk factors such as the possibility of a self-referential loop and accountability problems emerging due to LLMs in healthcare. While these problems have yet to materialize, they represent a likely challenge as LLMs advance and proliferate in healthcare. A thoughtful, deliberate approach to integrating LLMs into healthcare can mitigate risks associated with overreliance and deskilling, ensuring that it complements rather than compromises the quality of care. By leveraging AI's strengths and compensating for its limitations through human oversight, healthcare can harness the benefits of this technology to improve outcomes, enhance patient care, and support healthcare professionals in their vital work. Thus, the path forward involves embracing generative AI's potential while remaining vigilant about its limitations, ensuring that its integration enhances rather than diminishes the human element in healthcare.