JMIR Publications

ABSTRACT

Background:

A large language model (LLM) is a type of artificial intelligence (AI) algorithm that uses deep learning techniques and massively large data sets to understand, summarize, generate and predict new content. Modern LLMs use transformer-based models or short "transformers", which are neural networks and have been tested for various tasks in Natural Language Processing (NLP). In late 2022, such models gained widespread awareness with the release of ChatGPT which uses generative pre-trained transformer (GPT) models.

Objective:

The aim of this adapted Delphi study was to gain insights into opinions of how researchers think LLMs might influence healthcare and what are the strengths, weaknesses, opportunities and threats (SWOT) of the use of LLMs in healthcare.

Methods:

We invited researchers in the field of health informatics, nursing informatics, and medical NLP to share their opinions on the use of LLMs in healthcare. We started the first round with open questions based on our SWOT framework. In the second and third round, the participants scored these items.

Results:

The first, second, and third rounds had 28, 23, and 21 participants, respectively. Almost all participants were affiliated with academic institutions. Agreement was reached on 103 items related to use cases, benefits, risks, reliability, adoption aspects, and the future of LLMs in healthcare. Participants offered a multitude of use cases showing the potential value of LLMs; however, many shortcomings were also identified.

Conclusions:

Future research related to LLMs should not only focus on testing their possibilities for natural language related tasks, but also should consider the workflows the methods could contribute to and the requirements regarding quality, integration, and regulations needed for successful implementation in practice.