JMIR Publications

ABSTRACT

Background:

Generative artificial intelligence (GenAI) is increasingly integrated into scientific workflows. While offering significant utility, its capacity to fabricate convincing images in specific biomedical experimental settings raises serious concerns for research integrity. Specifically, the ability to seamlessly alter experimental trends to support false conclusions poses a critical, yet under-evaluated, threat.

Objective:

This study aimed to assess whether a contemporary high-fidelity generative model (ChatGPT Images 2.0) could generate “conclusion-altered” images from authentic biomedical figures in two selected modalities-western blot and subcutaneous xenograft tumor images-that preserve visual characteristics while reversing apparent experimental outcomes, and to determine whether such images could be reliably identified by uncalibrated human experts or commercial AI-detection tools.

Methods:

We constructed a benchmark dataset of 104 images, comprising 52 authentic images from 16 published biomedical studies and 52 matched AI-manipulated counterparts in western blot and subcutaneous xenograft tumor subgroups. Authentic images were iteratively refined using natural-language prompts via “ChatGPT Images 2.0” without post-processing. The diagnostic performance of three commercial AI-detection tools (“Illuminarty”, “Is It AI”, and “AI or Not”) was evaluated using receiver operating characteristic (ROC) analyses. In parallel, 24 PhD-level biomedical researchers (active peer reviewers familiar with these image types) independently classified images under routine conditions without prior forensic training.

Results:

In the evaluated settings, human readers showed near-chance performance in distinguishing authentic from AI-manipulated images, with mean accuracy of 50.5% (SD 6.9%), sensitivity of 38.5% (SD 18.9%), specificity of 61.5% (SD 17.7%), and mean Youden index of 0. Among automated tools, “AI or Not” achieved the best discrimination (AUC=0.790; 95% CI 0.695-0.885), followed by “Is It AI” (AUC=0.733; 95% CI 0.645-0.821); “Illuminarty” performed at chance level (AUC=0.485; 95% CI 0.372-0.598). Although some tools showed moderate discrimination, performance remained insufficient for reliable application in these settings, given false-positive risks and unclear thresholds.

Conclusions:

In the evaluated western blot and xenograft settings, current high-fidelity GenAI can produce conclusion-altered images that evade routine human expert scrutiny and challenge tested detection tools. These findings highlight vulnerabilities in specific biomedical image modalities and underscore the need for enhanced safeguards. We propose a three-step editorial workflow for these settings: mandating original uncropped raw images at submission, screening metadata during triage, and instructing reviewers to focus on biological inconsistencies.