JMIR Publications

ABSTRACT

Large language models (LLMs) are rapidly being adopted to augment clinical workflows in gastrointestinal (GI) endoscopy, where vast multimodal data must be interpreted, documented, and translated into guideline-concordant management and patient communication. Early prototypes look promising, but the evidence comes from disparate study designs and evaluation methods that are hard to compare, leaving the real-world value of these systems unclear. In this Viewpoint, we argue that evaluating LLMs task by task obscures how they behave once embedded in the endoscopic process, and that a systems-level perspective is needed. We propose a pipeline-based conceptual framework that organizes LLM applications into four interconnected layers—data structuring, perception and interpretation, clinical decision-making, and patient communication—spanning the full path from raw data to patient interaction. Our key message is that performance is uneven across this pipeline: it is generally higher in text-centric tasks and degrades in complex multimodal reasoning and individualized decision support, and, critically, errors introduced upstream can propagate downstream to compromise clinical decisions and patient-facing outputs. Reading the pipeline as a whole, we surface the cross-layer risks and key barriers that isolated evaluations miss, and outline directions for integrated end-to-end evaluation, prospective real-world validation, stronger multimodal reasoning, and knowledge-grounded architectures. We advance this framework to guide rigorous assessment and the responsible translation of LLMs into routine GI endoscopic care.