JMIR Publications

ABSTRACT

Background:

The world has witnessed increased adoption of Large Language Models (LLMs) in the last year. Although the products developed using LLMs have the potential to solve accessibility and efficiency problems in healthcare, there is a lack of guidelines available for developing LLMs for healthcare and especially medical education.

Objective:

The study aims to identify and prioritize the enablers for developing successful LLMs for medical education. The study also discusses the relationship among these identified enablers.

Methods:

The study first identifies key enablers for LLM development using the narrative review of extant literature. The next opinion of users of LLMs was taken to determine the relative importance of these enablers using the multi-criteria decision-making method called the Analytical Hierarchy Process. Further, Total Interpretive Structural Modelling (TISM) was used to analyze product developers' perspectives and ascertain the relationship and hierarchy among these enablers. Finally, Cross-impact matrix multiplication was applied to classification (MICMAC) to find these enablers' relative driving and dependence power. The non-probabilistic purposive sampling was used for the study.

Results:

The result of AHP concluded that credibility, with a priority weight of 0.37, is the most important enabler, followed by Accountability (0.27642) and Fairness (0.10572). In contrast, usability, with a priority weight of 0.04, has negligible importance. The results of TISM concur with the findings of the AHP. The only striking difference from the user's preference was that product developers gave the least importance to cost. The development of the MICMAC analysis suggests that cost has a strong influence on other enablers. The inputs of the focus group were found reliable, with a consistency ratio (CR=0.084) less than 0.1.

Conclusions:

The study is the first to identify, prioritize, and analyze the relationship of enablers for effective LLMs for medical education. The study provides an easy to comprehendible prescriptive framework CUCFATE (Cost, Usability, Credibility, Fairness, Accountability, Transparency, and Explainability) for the same. The study findings are useful for healthcare professionals, health technology experts, medical technology regulators, and policymakers.