JMIR Publications

ABSTRACT

Background:

Artificial intelligence (AI) is increasingly shaping medical student learning, yet institutions have limited data to guide the integration of these tools into medical education. As part of a quality improvement initiative, we evaluated students’ AI use to inform institutional transformation toward AI-supported competency-based medical education (CBME).

Objective:

To characterize how first-year medical students use AI, identify opportunities for improving responsible AI-supported learning, and guide curricular, faculty development, and policy initiatives.

Methods:

We conducted a program evaluation survey of 191 first-year medical students following a neuroscience module. The survey assessed students’ AI use, perceived effects on learning, and verification practices. To contextualize the findings and monitor for unintended consequences, cohort-level examination performance was compared with that of prior cohorts.

Results:

Of 191 students, 175 (91.6%) reported using AI. Students most commonly used AI for examination preparation, content review, concept clarification, and practice question generation. Most reported improved study efficiency (89.1%), understanding of difficult concepts (86.9%), and active self-testing (82.9%). However, 22.2% rarely or never verified AI-generated information, identifying an important target for educational improvement. Cohort-level examination performance was similar to or modestly higher than that of prior cohorts.

Conclusions:

This quality improvement initiative demonstrated that AI was already deeply embedded in students’ learning while identifying verification and responsible use as priorities for improvement. The findings informed the development of AI-supported CBME curricula, faculty development, and institutional policies. Ongoing evaluation will assess the effects of these initiatives and guide their iterative refinement.