JMIR Publications

ABSTRACT

Background:

Prompt engineering, focusing on crafting effective prompts to large language models (LLMs), has garnered attention for its capabilities at harnessing potential of LLMs. This is even more crucial in the medical domain, due to its specialized terminology and language technicity. Clinical natural language processing (NLP) applications must navigate complex language and ensure privacy compliance. Prompt engineering offers a novel approach by designing tailored prompts to guide models in exploiting clinically relevant information from complex medical texts. Despite its promise, the efficacy of prompt engineering in the medical domain remains to be fully explored.

Objective:

To review research efforts and technical approaches in prompt engineering for medical applications as well as provide an overview of opportunities and challenges for clinical practice.

Methods:

Databases indexing the fields of medicine, computer science, and medical informatics were queried in order to identify relevant published articles. Since prompt engineering is an emerging field, preprint databases were also considered. Multiple data were extracted, such as the prompt paradigm, the involved LLMs, the languages of the study, the domain of the topic, the baselines, and several learning, design, and architecture strategies specific to prompt engineering.

Results:

We included 114 recent studies that apply prompt engineering based methods to the medical domain, published between 2022 and 2024, covering multiple prompt paradigms such as prompt learning (PL), prompt tuning (PT), and prompt design (PD). Among the three prompt paradigms, we have observed that PD is the most prevalent (78 articles). In 12 papers, PD, PL, and PT terms were used interchangeably. While ChatGPT is the most commonly used LLM, we have identified seven studies utilizing this LLM on a sensitive clinical dataset. Chain-of-Thought, present in 17 studies, emerges as the most frequent prompt engineering technique. While PL and PT articles typically provide a baseline for evaluating prompt-based approaches, 64% of the PD studies do not report any non-prompt-related baseline. Lastly, we individually examine each of the key prompt engineering specific information reported across articles, and find that many studies neglect to explicitly mention them, posing a challenge for advancing prompt engineering research.

Conclusions:

In addition to reporting on trends and the scientific landscape of prompt engineering, we provide reporting guidelines for future studies to help advance research in the medical field. We also disclose tables and figures summarizing medical prompt engineering papers available, and hope that future contributions will leverage these existing works to better advance the field.