JMIR Publications

ABSTRACT

Background:

Post-polypectomy surveillance intervals derived from colonoscopy findings and histopathology should be guideline based, yet these are frequently not adhered to, contributing to under-and over-surveillance. This suggests that a more structured and automated approach would improve adherence.

Objective:

We developed and evaluated an automated, implementable, guideline-aligned decision support tool that extracts colorectal polyp features from free-text colonoscopy and histopathology reports and produces auditable surveillance recommendations

Methods:

We extracted paired colonoscopy and histopathology reports from an NHS trust and designed a four-stage prompting pipeline using open-weight Large Language Models (LLMs) to perform pot-level segmentation, align endoscopic findings to specimens and normalise each pot into a structured schema. The surveillance intervals are then computed by a deterministic rules engine encoding guideline rules. We evaluated extraction performance on 100 procedures (215 pots), assessed surveillance recommendation performance on a same-site cohort (n=100) and performed external validation on a cohort from a second trust (n=100).

Results:

Key extraction fields showed end-to-end accuracy (specimen type 0.995; location 0.981; diagnosis 0.981; excision completeness 0.949), with report-level pot comprehensiveness 0.968±0.112. Surveillance recommendation showed strong class-level performance (F1: no follow-up 0.98/0.97; site-check 0.93/0.91; 3-year 0.86/0.80; cancer alert 1.00/1.00). The comparison to current clinical practice showed that the tool would improve guideline adherence (+10.2 percentage points), primarily by reducing potential under-surveillance (10.2% to 2.4%).

Conclusions:

This hybrid approach using open-weight LLMs for structured extraction and guideline-encoded rules for decision-making can generate accurate, computationally lightweight, auditable and generalisable post-polypectomy surveillance recommendations. Such a tool has potential to improve safety-netting and guideline adherence in NHS colonoscopy workflows.