JMIR Publications

ABSTRACT

Background:

Medical large language models can perform well on static questions yet alter clinically relevant outputs when users introduce false premises, misleading evidence, authority claims, or repeated disagreement.

Objective:

We systematically reviewed how sycophancy and pressure resistance have been evaluated in patient- and clinician-facing medical settings.

Methods:

The registered review record reported searches of MEDLINE/PubMed, Embase, IEEE Xplore, arXiv, and CENTRAL through 31 May 2026 and yielded a frozen corpus of 31 benchmark and simulation studies. Studies were organized post hoc as direct sycophancy or pressure-resistance evidence (n=16), direct-challenge evidence (n=1), or adjacent interactive-robustness evidence (n=14). Fourteen were archival publications, 1 was accepted, 2 were nonarchival workshop reports, and 14 were preprints as of 4 September 2026. We extracted user role, counterfactual, comparator, analytic unit, denominator, model snapshot, native endpoint, uncertainty, and publication status.

Results:

Direct evaluations documented user-concordant output shifts under pressure in controlled settings. However, construct definitions, comparators, conditioning rules, analytic units, turn structures, model versions, and endpoints were incompatible; no common estimand existed, and pooling was not methodologically defensible. No study estimated routine-care incidence or patient outcomes.

Conclusions:

We derived a taxonomy that distinguishes progressive correction from regressive capitulation and propose a minimum evaluation framework covering construct definition, counterfactual, user role, comparator, model identity, conversation protocol, outcome, denominator, analysis, and openness. The evidence supports pressure resistance as a distinct target for benchmark design and lifecycle regression testing but does not establish a universal failure rate or protection conferred by release year, medical specialization, or a reasoning label.