JMIR Publications

ABSTRACT

Background:

Multiple Instance Learning (MIL) is widely used for slide-level classification in digital pathology without requiring expert annotations. However, even partial expert annotations offer valuable supervision, yet few studies have effectively leveraged this information within MIL frameworks.

Objective:

This study aims to develop and evaluate a ranking-aware MIL framework, called Rank Induction, that effectively incorporates partial expert annotations to improve slide-level classification performance under realistic annotation constraints.

Methods:

We developed Rank Induction, a MIL approach that incorporates expert annotations using a pairwise rank loss inspired by RankNet. The method encourages the model to assign higher attention scores to annotated regions than to unannotated ones, guiding it to focus on diagnostically relevant patches. We evaluated Rank Induction on two public datasets (Camelyon16, DigestPath2019) and an in-house dataset (SMF-stomach) and tested its robustness under three real world conditions: low-data regimes, coarse within-slide annotations, and sparse slide-level annotations.

Results:

Rank Induction outperformed existing methodologies, achieving an Area Under the Receiver Operating Characteristic curve (AUROC) of 0.839 on Camelyon16, 0.995 on DigestPath2019 and 0.875 on SMF-stomach. It remained robust under low-data conditions, maintaining an AUROC of 0.761 with only 60% (n=130) of the training data. When using coarse annotations (with 2240-pixel padding), performance slightly declined to 0.823. Remarkably, annotating just 20% (n=22) of the slides was enough to reach near-saturated performance (AUROC: 0.806 vs. 0.839 with full annotations).

Conclusions:

Incorporating expert annotations through ranking-based supervision significantly improves MIL-based classification. Rank Induction remains robust even with limited, coarse, or sparsely available annotations, demonstrating its practicality in real-world scenarios.