JMIR Publications

ABSTRACT

Background:

The number of publications has demonstrated deep learning (DL) algorithms matched or outperformed clinicians in image-based cancer diagnostics, and are frequently considered as opponents rather partners. Despite clinicians-in-the-loop DL approach has great potential, no study systematically quantified the diagnostic accuracy of clinicians with and without the assistance of DL in image-based cancer identification.

Objective:

We systematically quantified the diagnostic accuracy of clinicians with and without the assistance of DL in image-based cancer identification.

Methods:

PubMed, Embase, IEEEXplore, and the Cochrane Library were searched for studies published between 1st Jan, 2012, and the 7th Dec, 2021. Any type of study design was permitted that focused on assessing clinicians versus DL-assisted clinicians in cancer identification using medical imaging. Studies using medical waveform data graphics material and those investigating image segmentation rather than classification were excluded. Studies providing binary diagnostic accuracy data and contingency tables were included for further meta-analysis. Two subgroups were defined and analyzed, including cancer type and imaging modality.

Results:

In total, 9,796 records were identified, of which 48 studies were deemed eligible for systematic review. 25 of these studies made comparisons between clinicians and DL-assisted clinicians, and provided sufficient data for statistical synthesis. We found a pooled sensitivity of 83% (95%CI 80-86%) for clinicians alone and 88% (86-90%) for DL-assisted clinicians. Pooled specificity was 86% (83-88%) for clinicians and 88% (85-90%) for DL-assisted clinicians. The pooled sensitivity and specificity for DL-assisted clinicians were higher than for unassisted clinicians at a ratio of 1.07 (1.05-1.09) and 1.03 (1.02-1.05), respectively. Similar diagnostic performances by DL-assisted clinicians were also observed across the predefined subgroups.

Conclusions:

The diagnostic performance of DL-assisted clinicians appears better than unassisted clinicians in image-based cancer identification. However, caution should be exercised because the evidence provided in the reviewed studies does not cover all minutiae involved in real-world clinical practice. Combining qualitative insights from clinical practice with data science approaches may improve DL-assisted practice although further research is required. Clinical Trial: Prospero, CRD42021281372