JMIR Publications

ABSTRACT

The integration of Large Language Models (LLMs) into healthcare presents transformative opportunities alongside significant risks, necessitating a proactive approach to balance innovation with patient safety. This paper advocates for a harm reduction framework to address the dual-edged nature of LLM use in medicine, where patients and clinicians face distinct vulnerabilities. For patients, risks include misinformation, privacy breaches, biased outputs, and delayed care due to over-reliance on unverified LLM-generated advice. For clinicians, challenges encompass diagnostic errors, liability concerns, workflow disruptions, and the erosion of clinical judgment. Rather than advocating prohibition, this work proposes evidence-based strategies to mitigate harm while maximizing utility. Key recommendations include: fostering critical health literacy and verification habits among patients; enforcing institutional guidelines for clinicians that prioritize "human-in-the-loop" validation; enhancing transparency in LLM outputs and data practices; and implementing secure, bias-mitigated AI tools tailored to medical contexts. The paper underscores the necessity of thoughtful use—emphasizing LLMs as assistive, never autonomous, tools—and highlights collaborative efforts among developers, policymakers, healthcare institutions, and educators to address challenges like regulatory ambiguity, equity gaps, and rapid technological evolution. By embedding accountability, transparency, and continuous oversight, this framework aims to steer the ethical integration of LLMs toward enhancing—not undermining—core medical values: patient safety, trust, and equitable care delivery.