JMIR Publications

ABSTRACT

Background:

Accurate diagnosis of middle ear diseases remains challenging due to the specialized nature of otoscopic interpretation and the global shortage of otolaryngologists.

Objective:

We aimed to evaluate the impact of a deep learning-based artificial intelligence (AI) support system on the diagnostic performance of clinicians with varying levels of expertise, specifically focusing on its ability to bridge the gap between trainees and experts.

Methods:

In this retrospective reader study, a deep learning model (EfficientNet-B4) was developed using a large-scale dataset of 7,335 endoscopic images. The model was designed for multi-class classification, simultaneously identifying primary categories (Chronic Otitis Media, Otitis Media with Effusion, or None [indicating the absence of COM and OME]) and secondary features (attic retraction, myringitis, otomycosis, and ventilation tube). To assess clinical impact, diagnostic performance was compared among three otology professors (OP), five senior residents (SR), and five junior residents (JR) using an independent clinical assessment set of 100 images. Participants performed diagnoses both with and without AI-generated predictions, with a time constraint of one minute per image to simulate real-world clinical pressure.

Results:

In 5-fold cross-validation, the deep learning model achieved 94.8% accuracy for the primary classification. The standalone AI achieved diagnostic accuracies ranging from 94% to 99% in the clinical assessment set. AI support significantly enhanced diagnostic accuracy across all groups for primary classification, with improvements of 11.0% for OP, 28.8% for SR, and 32.2% for JR (P<.001). Accuracy increased from 78.7% to 89.7% for OP, from 65.0% to 93.8% for SR, and from 54.4% to 86.6% for JR. The accuracy gap between the most experienced (OP) and least experienced (JR) groups was reduced from 24.3% to 3.1%. For secondary categories, significant improvements were observed in diagnosing attic retraction and myringitis among residents. AI-supported accuracy for attic retraction was numerically higher in senior residents than with standalone AI, although the 95% confidence intervals overlapped substantially.

Conclusions:

Our findings demonstrate that deep learning-based AI support substantially improves the diagnostic accuracy of middle ear diseases, particularly for less experienced clinicians. By narrowing the performance gap between trainees and experts, this system shows high potential as a valuable adjunct for clinical decision-making and medical education in otology.