JMIR Publications

ABSTRACT

Background:

Multidisciplinary Tumour Boards (MDTs) constitute the foundation of modern tumour therapy. Large language models are widely discussed in optimising their recommendations.

Objective:

This is the first prospective study evaluating the implementation of locally run LLMs on real-world cases within a regular head and neck MDT.

Methods:

Seventeen patients participated in the study. The MDT application was processed by two different local LLMs (gemma-3-12b and gpt-oss-20b) to obtain treatment recommendations. The MDT conferred as usual. After the decision was made, the MDT was shown with the LLMs' recommendations. If deemed to be beneficial, the MDT's recommendation was adjusted. The MDT members rated the LLMs' responses inter alia for medical accuracy on a 6-point Likert scale. In addition, a tabular comparison of the MDT's and LLMs' recommendations was carried out.

Results:

In one case (6%), the LLM was able to substantially improve the MDT recommendation by underscoring a follow-up examination that had not yet been performed. Concordance regarding curative or palliative therapy regimen reached 94% for gemma-3-12b and 59% for gpt-oss-20b. Gemma-3-12b stated the same first-line therapy regimen as the MDT as first-line in 35% (n = 6), gpt-oss-20b in 41% (n = 7) of the cases. In 59% of patients (n = 10), gemma-3-12b stated the first-line therapy regimen of the MDT, albeit with a different priority, while for gpt-oss-20b it was 41% (n = 7). Medical adequacy as rated by the MDT members revealed a median of 5 [IQR 2-5] for gemma-3-12b and a median of 4 [IQR 3-5] for gpt-oss-20b. MDT members stated hazardous information in 27% (25/93) of ratings for gemma-3-12b and 17% (14/93) of ratings for gpt-oss-20b.

Conclusions:

Locally run LLMs improved the MDT recommendation in one case and mainly served to verify completeness in others, underscoring that they cannot replace MDTs. However, their observed benefit suggests that more advanced local models may offer safe, rapid, and cost-effective support for MDT decision-making. Clinical Trial: Ethical approval was obtained from the ethics committee of the state medical association (Request number: 2024-17946_2).