JMIR Publications

ABSTRACT

Background:

Large language models (LLMs) are rapidly emerging in healthcare, offering opportunities in decision support, education, and research, but raising critical concerns about safety, reliability, and ethics. While several guidelines for trustworthy AI exist in business and technology, few systematic reviews have applied them to medical contexts.

Objective:

This study aimed to conduct a systematic review of LLM research in healthcare, applying the AI Guidelines for Business as a framework across twelve domains including safety, reliability, ethics, transparency, fairness, inclusiveness, privacy, security, robustness, data quality, and verifiability.

Methods:

Following PRISMA methodology, databases such as PubMed, Web of Science, Scopus, arXiv, and IEEE Xplore were searched. The literature was systematically searched on January 15, 2025. Eligible studies were classified and independently verified. A total of 247 studies were included. Each article was analyzed to determine the corresponding medical specialty, classification as surgical, internal medicine, or emergency care, the intended purpose of use, and the study population. Performance measures were converted into percentages on a 0–100 scale for comparability.

Results:

Research was concentrated on internal medicine (23.1%), surgery (23.1%), and radiology (14.2%), while many specialties were underrepresented. Evaluation domains emphasized accuracy (57.9%), fairness and inclusiveness (37.7%), data quality (19.0%), and misinformation control (17.8%). Reported performance was moderate to high, with accuracy at 75.0%, fairness at 58.1%, data quality at 75.9%, and misinformation control at 73.8%. Privacy protection (n=2) and security assurance (n=0) were almost entirely absent from the literature reviewed.

Conclusions:

The review highlights strengths but also methodological gaps and disciplinary imbalances, underscoring the need for broader specialty engagement and comprehensive, guideline-based evaluation to ensure safe, reliable, and equitable implementation of LLMs in healthcare.