JMIR Publications

ABSTRACT

Personal health large language models (PH-LLMs) have rapidly transitioned from research prototypes to consumer-facing, data-linked systems that support symptom triage, medication inquiries, mental health check-ins, and longitudinal self-management. Yet their direct-to-consumer deployment without clinical oversight creates a distinct ethical risk profile that is not adequately addressed by generic AI guidance. This Viewpoint synthesizes PH-LLM–specific challenges across six domains: privacy, accuracy, equity, transparency, human–AI interaction, and regulatory governance. We explain why these risks are amplified by health literacy gaps, longitudinal data aggregation, persuasive conversational interfaces, and fragmented oversight across the consumer–clinical boundary. Building on biomedical principlism, we propose a governance framework that operationalizes beneficence, nonmaleficence, respect for autonomy, and justice into auditable design and deployment controls. Beneficence is advanced through health-literacy–aligned communication and risk-stratified care navigation; nonmaleficence through validated crisis-response protocols, pharmacological contraindication safeguards, and hallucination mitigation pipelines; autonomy through explicit role and scope disclosure, granular data sovereignty, and risk-proportionate consent mechanisms; and justice through predeployment fairness audits, postdeployment disparity monitoring, and inclusive access architecture. We translate these principles into implementable mechanisms, including a user-centered safety certification checklist for market entry, a tiered accountability architecture assigning responsibility by capacity across developers and platforms, and postdeployment digital pharmacovigilance via centralized adverse-event reporting, transparency reporting, and independent safety advisory boards. Collectively, these measures aim to preserve innovation while establishing practical safeguards that protect users and support responsible integration of PH-LLMs into personal health management.