JMIR Publications

ABSTRACT

Medical artificial intelligence (AI) systems are often evaluated through aggregate performance metrics and output-level fairness measures. However, clinically meaningful harms may remain hidden when systems perform well on average while underperforming for data-poor, underrepresented, or structurally marginalized populations. This Viewpoint uses the concept of refined exclusion to synthesize a recurring pattern in medical AI: systems may appear technically successful at the population level while transferring uncertainty, misclassification, delayed recognition, or reduced clinical reliability to groups that are less visible within training data, validation cohorts, proxy definitions, and deployment workflows. Drawing on representative cases from population health management, chest radiograph AI, dermatology, computational pathology, and foundation model applications, we argue that refined exclusion should not be treated merely as algorithmic bias or a defect of model outputs. Rather, it reflects a data governance failure with direct implications for patient safety. Moving beyond output-centered algorithmic fairness, we propose data justice as a governance foundation for medical AI, organized across distributional, procedural, and substantive dimensions. We further outline operational checkpoints across the medical AI lifecycle, including subgroup learnability assessment, data provenance documentation, local validation, procurement-stage accountability, explainability-based proxy audits, post-deployment subgroup monitoring, and patient participation. Reframing refined exclusion as a patient safety problem shifts the central governance question from “Is this model accurate on average?” to “For whom is this system safe, reliable, and clinically accountable?”