JMIR Publications

ABSTRACT

Background:

With the increasing application of computer-aided diagnostic (CAD) technologies such as the S-Detect system in breast ultrasound imaging, clinical workflow has been partially optimized and radiologists’ workload reduced. While numerous studies have evaluated the diagnostic accuracy of S-Detect, limited evidence exists regarding its reproducibility, an essential attribute for clinical reliability.

Objective:

To evaluate the intra‑system reproducibility of S-Detect system, a computer‑aided diagnostic (CAD) module integrated in a commercial ultrasound platform, for differentiating benign from malignant breast nodules on identical static ultrasound images.

Methods:

This retrospective study consecutively enrolled female patients with breast masses who underwent surgery at the Department of Breast and Thyroid Surgery, Third Xiangya Hospital (2019-2020). Static images of the largest cross‑section of each lesion were acquired by a single experienced sonographer and stored. Each stored image was independently analyzed twice by the S‑Detect module on the same workstation: once immediately after acquisition (S‑Detect 1) and once later (S‑Detect 2) by the same operator blinded to the first result. Agreement between the two analyses was assessed by concordance rate and Cohen’s kappa (κ). Diagnostic performance against histopathology was summarized using sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), accuracy, and area under the receiver operating characteristic curve (AUC).

Results:

A total of 398 nodules from 261 patients were included. S‑Detect classified 156 nodules as ‘‘possibly benign’’ and 242 as ‘‘possibly malignant’’ in the first analysis. On repeat analysis, 149 were classified as ‘‘possibly benign’’ and 249 as ‘‘possibly malignant’’. Seven lesions (1.76%) initially labeled ‘‘possibly benign’’ were reclassified as ‘‘possibly malignant’’ on repeat analysis; no lesions were downgraded from malignant to benign. Overall concordance between S‑Detect 1 and S‑Detect 2 was 98.24% with κ = 0.95 (95% CI, 0.94-0.99; P < 0.001), indicating almost perfect agreement. Compared with histopathology (170 benign, 228 malignant), S‑Detect 1 yielded sensitivity 96.05% (92.91-98.02), specificity 86.47% (80.29-91.12), PPV 90.50% (86.08–93.78), NPV 94.23% (89.10-97.14), accuracy 91.96% (88.91-94.28), and AUC 0.913 (0.887-0.939). S‑Detect 2 produced comparable performance (sensitivity 96.93%, specificity 83.53%, PPV 88.76%, NPV 95.30%, accuracy 91.21%, AUC 0.902); differences in AUCs were not statistically significant (P > 0.05).

Conclusions:

Under standardized image‑acquisition and analysis conditions, the S‑Detect CAD demonstrated excellent intra‑system reproducibility with negligible intra‑module variability. These findings support the system’s reliability as an adjunct for breast ultrasound interpretation. Clinical Trial: This retrospective, registered, blinded reproducibility study was approved by the Ethics Committee of the Third Xiangya Hospital, Central South University (Approval No. R19004) and was registered with the Chinese Clinical Trial Registry (ChiCTR‑1800019649).