JMIR Publications

ABSTRACT

Background:

Preprocessing serves to standardize clinical documentation and create consistency throughout all clinical notes. An essential aspect of this process is text normalization, which mainly entails recognizing abbreviations and translating them into their full forms. A range of methods, including Natural Language Processing (NLP) and Machine Learning (ML) algorithms, are employed to identify and interpret abbreviations found in clinical notes. Recently, fine-tuning Large Language Models (LLMs) and applying prompt engineering techniques have been utilized to carry out various tasks involving clinical notes. Each method presents its own set of benefits and drawbacks.

Objective:

To utilize LLMs to detect and decipher abbreviations in clinical notes effectively and efficiently.

Methods:

A framework is proposed for the detection and deciphering of abbreviations within clinical text, using LLMs. This framework consists of four phases. First, the task and sub-tasks are identified. Second, the relevant properties of task and sub-tasks within the designated domain are clearly defined. Third, these properties are utilized to generate optimized examples. The fourth stage involves the application of these properties in LLMs, which can serve either as engineering prompts enhanced by examples or be employed in the creation of a labeled dataset for fine tuning. Finally, two case studies are conducted; the first involves prompt engineering enhanced by optimized examples is employed as input for ChatGPT utilizing the GPT-4 language model, while the second experiment entails the fine-tuning of both GPT-2 and T5 models using the previously established labeled dataset.

Results:

The findings emphasize a distinct advantage of example-based prompts, whether examples are selected randomly or through optimized selection, in comparison to basic prompts. The results indicate that basic prompts consistently yield lower accuracy. Furthermore, prompts enhanced by the optimized selection of 5-shots examples demonstrate significantly superior performance relative to those utilizing randomly selected examples. In contrast to the example-based prompt approach, the result indicates that the T5 models, which are fine-tuned with the labeled dataset using the proposed framework, exhibit enhanced performance when compared to the example-based prompt.

Conclusions:

This research underscores the potential of LLMs for the proficient detection and interpretation of abbreviations within clinical notes. This capability is particularly evident when optimized examples are utilized in the preparation of the labeled dataset for the fine-tuning of T5 models, resulting in reduced costs and enhanced performance. Further investigation is necessary to develop an LLM model specifically designed for the detection and interpretation of abbreviations in clinical notes.