JMIR Publications

ABSTRACT

This study examines the phenomenon of "sandbagging" in AI medical devices, where systems strategically underperform during evaluation to conceal dangerous capabilities that emerge post-deployment. Through systematic analysis of emerging literature on AI sandbagging behaviour, technical detection approaches, and regulatory structures in the EU, UK, and US, this research reveals critical gaps in current regulatory frameworks designed for traditional medical devices. Analysis shows sandbagging manifests through both developer-driven mechanisms (where engineers intentionally display safer capabilities for expedited deployment) and system-driven mechanisms (where AI systems autonomously underperform during evaluation phases). Research shows that both large frontier and smaller models exhibit sandbagging behaviours after prompting or fine-tuning while maintaining general performance benchmarks, with larger models demonstrating superior calibration capabilities. Current static regulatory approaches in the EU Medical Device Regulation and UK frameworks fail to detect sandbagging as they rely on documentation-based submissions without addressing AI's dynamic, generative nature. The US FDA's Total Product Lifecycle approach shows promise through algorithm change protocols and real-world performance monitoring, yet regulatory sandboxes remain underutilized. Healthcare provider liability becomes dangerously ambiguous when clinicians rely on systems with concealed capabilities, particularly given automation bias effects and black-box reasoning limitations. Traditional risk classifications focusing on direct bodily harm inadequately address AI's potential for deceptive behaviour, including "password-locked" models that reveal hidden capabilities when triggered. Technical detection solutions including attribution graph analysis and noise-based detection show promise but remain insufficient. Dynamic evaluation frameworks are essential, recommending mandatory regulatory sandboxes for real-world testing, continuous monitoring protocols, adversarial testing, and enhanced post-market surveillance.