JMIR Publications

ABSTRACT

Background:

Systematic assessment of competency in liver contrast-enhanced ultrasound (CEUS) is critical, especially for novices. However, standardized tools for evaluating theoretical and cognitive skills are currently lacking. Despite increasing use of CEUS for liver examinations, no standardized assessment tool exists for trainees. The only existing CEUS competency scale lacks organ specificity and external validation. Current subjective or fragmented tests fail to capture multidimensional expertise (e.g., scanning protocols, artifact recognition, clinical decision-making), undermining training quality. Closing this gap is critical, as operator competency affects diagnostic accuracy and patient safety.

Objective:

This study developed and validated the CEUS Competency Assessment for the Liver (CCAL), a tool to standardize evaluation of theoretical and cognitive skills in liver CEUS and provide feedback to trainees and educators.

Methods:

This retrospective cohort study used a four‑stage process.Stage 1: item generation from literature/guidelines. Stage 2: three Delphi rounds (21 experts, 57% female, mean age 51.0 years) and cognitive testing (10 trainees) for content validity (>85% agreement) and clarity. Internal validation on 100 sonography trainees (66.0%, 27.0%, 7.0% for assistant/associate/professors; 70% with no prior CEUS experience) with ≥1 month training at 4 hospitals (2018–2020). Stage 4: external validation on 579 sonography trainees (51.6%, 31.6%, 16.8% for assistant/associate/professors; 75.8% no prior CEUS experience) with ≥1 month training at 25 multicenter sites (2018–2020). Primary outcomes for validation were reliability (Cronbach’s α, Guttman split‑half) and validity (exploratory factor analysis, item difficulty/discrimination); secondary outcome was objective question pass rate.

Results:

The final CCAL comprised 74 items across three domains: Demographics (8 items), Self-evaluations (13 Likert-scale items), and Objective questions (53 multiple-choice). Internal consistency was excellent (Cronbach’s α = 0.916; Guttman split-half = 0.908). Exploratory factor analysis confirmed a three-factor structure explaining 74.6% of cumulative variance. Item analysis showed balanced difficulty (66% easy, 32.1% moderate, 1.9% difficult) . Item revealed balanced difficulty distribution (66% easy, 32.1% moderate, 1.9% difficult items). Scores varied significantly by CEUS training duration (p < 0.01); trainees with 4–6 months or more than 6 months of training achieved higher self‑evaluation and objective question scores than those with less than 3 months of training. Prior ultrasound teaching experience was also associated with higher scores (p < 0.01). Competency gaps were identified: pass rate was lowest for image-guided intervention (32.5%, 188/579) and highest for indications/contraindications (81.4%, 471/579).

Conclusions:

The CCAL is the first rigorously externally validated liver‑specific competency assessment tool for CEUS, providing higher‑level evidence that fills a critical gap in radiology education and offers a reproducible framework for competency‑based liver imaging assessment.In practice, the CCAL may facilitate targeted curriculum improvement and enhance quality assurance across training programs, and could potentially inform credentialing processes pending further validation. Clinical Trial: This study involved human participants and adhered to the Declaration of Helsinki. Ethical approval was obtained from the Institutional Review Board of The Third Affiliated Hospital of Sun Yat-sen University AN and was was registered with the Chinese Clinical Trial Registry ( ChiCTR2000035750).