JMIR Publications

ABSTRACT

Background:

Most AI models for thyroid nodules are designed to screen for malignant ones to guide further interventions but have not yet been fully implemented in clinical practice.

Objective:

This study aims to evaluate AI in real clinical settings for identifying potentially benign thyroid nodules initially deemed at malignant risk by radiologists, reducing unnecessary FNA, and optimizing nodule management.

Methods:

This retrospective multi-center study utilized a validation cohort to assess the AI's performance in identifying benign nodules initially classified as malignant by radiologists and a comparison cohort to evaluate the AI's performance against radiologists.

Results:

4572 thyroid nodules were collected in the validation cohort.AI correctly identified 2,719 (86.8% among benign nodules) and reduced unnecessary FNAs from 68.5% to 9.1%. However, 123 malignant nodules (8.6% of malignant cases) were mistakenly identified as benign, with the majority of these being of low or intermediate suspicion. In the comparison cohort, AI successfully identified 81.4% of benign nodules. It outperformed junior and senior radiologists, who identified only 55.0% and 44.0%, respectively. The AUCs of AI (0.882, 95% CI, 0.849-0.912) were better than those of senior (0.625, 95% CI, 0.550–0.698) and junior radiologists (0.425, 95% CI, 0.358–0.490), respectively (p < 0.01).

Conclusions:

Compared to radiologists, AI can better serve as a "goalkeeper" in reducing unnecessary FNAs by identifying benign nodules that were initially assessed as malignant by radiologists. However, active surveillance is still necessary for all these nodules since a very small number of low-aggressive malignant nodules may be mistakenly identified. Clinical Trial: The research protocol was registered at www.chictr.org.cn (ChiCTR2200066755).