JMIR Publications

ABSTRACT

When large language models (LLMs) were introduced to the public at large in late 2022 with ChatGPT, the interest was unprecedented with more than 1 billion unique users within 90 days. Until the introduction of GPT-4 in March 2023, these LLMs were single mode—text. As medicine is a multimodal discipline, the potential future versions of LLMs that can handle multimodality — meaning they could interpret and generate not only text but also images, videos, sound, and even comprehensive documents — can be conceptualized as a significant evolution in the field of AI. This paper zooms in on the new potential of generative artificial intelligence (AI) by the achievement of multimodal inputs of text, images, and speech. We present several futuristic patient scenarios to help illustrate the potential path forward. It is important to point out though that despite the unprecedented potential of generative AI in the form of M-LLMs, the human touch in medicine remains irreplaceable. AI should be seen as a tool that can augment healthcare professionals, rather than replace them. It's also important to consider the human aspects of healthcare - empathy, understanding, and the doctor-patient relationship - when deploying AI.