JMIR Publications

ABSTRACT

Background:

Hepatopancreatobiliary (HPB) malignancies require complex treatment planning that often relies on multidisciplinary team (MDT) discussions. Large language models (LLMs) have recently been explored for clinical decision support, but their performance within real-world multidisciplinary decision environments remains unclear. In particular, the stability of LLM-generated recommendations—whether a model produces the same answer when given the same clinical input—has rarely been examined.

Objective:

This study evaluated the feasibility of LLMs in MDT-based oncologic decision-making, with a focus on response stability.

Methods:

This retrospective study included consecutive cases discussed at a single-center HPB MDT conference between September 1, 2024, and August 31, 2025. Standardized clinical case summaries derived from MDT records were provided to 4 large language models (GPT-4.0, GPT-5.2, Gemini-3.0, and Claude Sonnet 4.5). Each model generated treatment recommendations among predefined MDT treatment options. Identical queries were repeated 4 times to evaluate response stability. Concordance with MDT decisions was assessed using the first model response.ted the feasibility of LLMs in MDT-based oncologic decision-making, with a focus on response stability.

Results:

A total of 107 MDT cases were analyzed. Stability of LLM-generated recommendations differed significantly across models (P = .01), with Gemini-3.0 showing the lowest discordance rate (12.8% [SD 2.3%]) and GPT-4.0 the highest (30.2% [SD 6.5%]). Concordance with MDT decisions ranged from 48.6% to 72.9% across models. Complete discordance occurred in 17 cases (15.9%). Pancreatic tumor location (adjusted OR 7.23; P = .004) and low MDT agreement level (adjusted OR 15.31; P = .014) were independently associated with complete discordance.

Conclusions:

LLM-generated treatment recommendations demonstrated moderate alignment with MDT decisions in HPB oncology. Importantly, response stability varied substantially across models, indicating that concordance alone is insufficient for evaluating LLMs as clinical decision support tools. These findings suggest that LLMs may serve as a reasoning-support layer in MDT-like decision environments, but their response stability must be systematically characterized before clinical integration.