JMIR Publications

ABSTRACT

Background:

Perioperative computed tomography (CT) imaging is essential for detecting postoperative recurrence and metastasis in cancer patients. However, large-scale automated extraction of oncological outcomes from CT reports remains limited by the unstructured nature of report text and wide variability in reporting styles. Radiology reports frequently contain linguistic ambiguities, including negations, hedging, and expressions conveying diagnostic uncertainty (eg, "cannot exclude recurrence" or "possibly metastatic"). Manual review is labor-intensive and constrains consistent extraction at scale. The inability to systematically account for diagnostic uncertainty represents a major barrier to reliable automated surveillance systems.

Objective:

This study aimed to develop a semi-supervised deep-learning classification framework that explicitly captures diagnostic uncertainty by classifying postoperative recurrence and metastasis into three categories (Positive, Negative, and Uncertain).

Methods:

This retrospective study analyzed 288,076 postoperative CT reports from 86,083 noncardiac surgery patients at Asan Medical Center (2014–2021). Preprocessing identified presumed negatives through keyword filtering and unsupervised clustering. The semi-supervised framework incorporated human-in-the-loop validation across three cycles—with clinicians reviewing approximately 2,000 samples per cycle (<1% of total reports)—and integrated rule-based algorithms (RA), medical BERT (MedEmbed and PubMedBERT). A report-level train–validation split was employed, as preprocessing reduces each report to sentence-level fragments that preclude patient-level linkage. Performance was evaluated against RA and multiple BERT variants under both naive and simulated real-world class distributions. Maximum mean discrepancy testing confirmed distributional integrity of the sampled data. Model interpretability was assessed using Integrated Gradients.

Results:

The cohort included 11 cancer types, predominantly gastrointestinal (34.2%), hepatobiliary and pancreas (17.8%), and genitourinary (16.3%). Under simulated conditions, PubMedBERT achieved 92.58% accuracy for recurrence, and MedEmbed 93.25% for metastasis. The framework achieved accuracies of 97.33% (multiclass) and 99.33% (binary class) for recurrence and 95.00% (multiclass) and 96.67% (binary class) for metastasis, compared with human inter-revision consistencies of 96.88% and 93.80% (recurrence and metastasis for multiclass), reflecting concordance with the clinician-derived consensus standard. The framework captured diagnostic uncertainty in 1.4% of recurrence and 6.9% of metastasis cases. Notably, the RA outperformed several sophisticated deep-learning models in metastasis classification.

Conclusions:

The proposed framework achieves clinician-concordant classification while requiring minimal expert annotation (<1% of reports). By explicitly modeling diagnostic uncertainty and combining rule-based and deep-learning approaches, it demonstrates the potential for automated cancer surveillance and clinical decision support in real-world settings; however, generalizability to other institutions requires prospective multicenter validation.