JMIR Publications

ABSTRACT

Background:

Multidisciplinary tumor boards (MDTs) are essential for personalized treatment planning in head and neck squamous cell carcinoma (HNSCC) but are resource-intensive. Large language models (LLMs) offer potential for clinical decision support, although concerns regarding reliability, patient safety, and data privacy remain.

Objective:

To evaluate the clinical reliability and guideline concordance of a locally deployed, guideline-informed LLM (Llama 3.3, 70B) for the largest real world data set of pre- and posttreatment HNSCC recommendations and investigate the impact of information asymmetry on model performance.

Methods:

This retrospective single-center study included 676 patients with HNSCC, representing 1,127 clinical scenarios (676 pretreatment and 451 posttreatment). All processing by 70b Llama 3.3 was performed locally, ensuring GDPR-compliant data handling. Analysis was guided by structured summaries of international treatment guidelines, to generate pre- and post-therapeutic recommendations. In 676 real patients (1,127 clinical scenarios), LLM outputs were compared with original and simulated MDT decisions to evaluate information bias.

Results:

Concordance with original MDTs was high (pre-therapeutic: 94.8%; post-therapeutic: 91.0%) and further improved in simulated MDT comparisons (97.3% and 96.1%, respectively). Performance was strongest in advanced disease and in distinguishing curative from palliative strategies (accuracy: 97%). Discrepancies were attributable to missing clinical inputs, particularly resectability and ECOG-PS.

Conclusions:

These findings support the use of guideline-informed, locally deployed LLMs as decision-support tools for MDT preparation, while emphasizing the need for supervised implementation and prospective validation.