Currently submitted to: JMIR Medical Education
Date Submitted: Jun 6, 2026
Open Peer Review Period: Aug 4, 2026 - Oct 4, 2026
(currently open for review)
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
A Pilot Study on Measuring Medical AI Education Competency among Hospital Staff in the Republic of Korea
ABSTRACT
Background:
The clinical integration of artificial intelligence (AI) within digital healthcare ecosystems necessitates practical competencies among healthcare professionals for safe and effective operation. However, traditional subjective perception surveys are limited in quantifying the specific, behavior-based competency deficits required in actual clinical workflows.
Objective:
This study aimed to develop a medical AI self-assessment tool utilizing Behaviorally Anchored Rating Scales (BARS) across diverse hospital occupational groups and to conduct a pilot evaluation to analyze current competency profiles and specific educational needs.
Methods:
A cross-sectional online survey was conducted in November 2025 among 116 healthcare professionals employed at tertiary and regional general hospitals. Three core domains and six detailed competency items were established for each occupational group. Respondents rated their practical performance on a BARS scale from 0 (no experience) to 5 (expert/leader). Data were analyzed based on demographic characteristics, prior AI training status, and competency score distributions across occupations.
Results:
The overall mean AI competency score was 2.44, indicating a foundational utilization phase. A distinct polarization was observed: physicians (mean 2.82) and nurses (mean 2.71) demonstrated higher proficiency, utilizing AI primarily as an assistive tool for clinical tasks and monitoring. Conversely, medical technicians (mean 1.99) and administrative staff (mean 2.24) remained at a basic literacy level, showing significant deficits in quality control and governance. The tool demonstrated high internal consistency, with Cronbach alpha values ranging from 0.882 to 0.970.
Conclusions:
Clinical AI competencies vary significantly by occupational role, with critical deficits identified among non-clinical support and technical staff. To ensure safe and sustainable clinical AI integration, standardized technical training should be replaced with customized, role-specific competency programs and multidisciplinary governance frameworks anchored in objective behavioral indicators. Clinical Trial: Not applicable
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.