JMIR Publications

ABSTRACT

Background:

While Artificial Intelligence (AI) is increasingly adopted in healthcare, clinicians face barriers including insufficient understanding, limited trust, and interpretation challenges. Existing frameworks, such as the UNESCO AI Competency Framework, lack clinical specificity. Additionally, there remains limited evidence on structured, framework-based training programs designed to advance AI literacy among medical professionals.

Objective:

This study aimed to (1) develop and validate a Medical AI Competency Framework and (2) demonstrate the framework’s practical application through the design and pilot implementation of an AI training program.

Methods:

We first drafted a Medical AI Competency Framework by integrating the UNESCO AI framework with Miller’s pyramid model. Expert feedback and validation involved 24 stakeholders (six hospital administrators, eight medical professionals, and ten university instructors). A five-module AI training program was designed incorporating problem-based learning (PBL) and flipped classroom methodology. A two-round Delphi process with nine educators in instructional design, medical education, and AI validated the program design using consensus criteria (Round 1: IQR≤1, AS≥75%, FS > threshold; Round 2: AS≥80%). A pilot mini-workshop with 28 participants and 4 instructors assessed the feasibility of the training program by measuring participants’ satisfaction, engagement, and self-confidence.

Results:

A six-dimension, four-level Medical AI Competency Framework was developed. Expert validation showed strong emphasis on AI foundations (79.17%) and application skills (95.83%) of the framework. Based on the framework, a five-module AI training program was designed. Each module included five elements: content, learning goals, teaching activities, learning resources, and assessment. The Delphi process achieved complete consensus across all 25 elements of the training program. Pilot implementation surveys suggested participants’ high satisfaction (Mean = 4.00), strong engagement across behavioral, emotional, and cognitive dimensions (Mean = 3.80–4.05), and positive self-confidence in applying AI in medical contexts (Mean = 3.63).

Conclusions:

This study presents an empirically informed framework and demonstrates its practical value through a structured training program. It provides a scalable model for integrating AI into medical curricula, enhancing medical professionals’ readiness for AI-driven healthcare. Future work should expand the framework and training program to new regions and delivery formats (e.g., semester-long courses and continuing medical education) and evaluate their long-term impact through longitudinal, multi-institutional studies.