JMIR Publications

ABSTRACT

Large Language Models (LLMs) have been utilized across a multitude of applications, demonstrating enormous potential in processing and comprehending complex datasets in healthcare. One area yet to be thoroughly explored is the application of LLMs for the reliable and reproducible evaluation of medical documents. Automatic evaluation of these documents, if achieved effectively, has the potential to improve healthcare, enhance patient safety, reduce the risk of cognitive and other biases, and refine the training process of LLMs. Importantly, it is essential that the system's reasoning process is a) transparent and comprehensible to human evaluators such as a checklist completion, and b) is guided by established medical guidelines proven to increase patient safety and the gold standard for implementing clinical care, thereby elevating the overall performance and applicability of AI-driven healthcare. In this study, we introduce a framework which is based on a multi-step approach for medical record evaluation that incorporates guidelines directly into the evaluation process, a concept we term 'guideline-in-the-loop'. Our proposed algorithm, named MedCheckLLM, is an LLM-driven structured, layered reasoning mechanism designed to automate the evaluation of medical records, with a particular emphasis on the evaluation against evidence-based guidelines. Crucially, the guidelines are deterministally accessed by the LLM as out-of-training data. This rigorous separation of LLM and guidelines is expected to lead to increased validity and interpretability of the evaluations and offers flexibility for updating guidelines. The primary objective of this research is to introduce the conceptual framework and assess its feasibility. This approach is expected to have significant implications on healthcare quality and the transparent and efficient application of LLMs in clinical settings.