JMIR Publications

ABSTRACT

Background:

Virtual patients (VPs) are widely used in medical education, but creating high-quality cases is resource-intensive. Generative artificial intelligence may accelerate VP development, yet direct comparisons between AI-generated and clinician-authored cases within the same educational setting remain limited.

Objective:

To compare medical students’ perceptions of AI-generated vs. clinician-developed virtual cardiology patient scenarios.

Methods:

A randomized, participant-blinded two-arm comparative controlled study was conducted using two VP scenarios: a clinician-developed VP (CD-VP) and a ChatGPT-generated VP (GPT-VP). The study was conducted in October 2025 during scheduled fifth-year Emergency Medicine course sessions. Eligible participants were fifth-year medical students who had completed the cardiology course in the previous academic year. Students were assigned 1:1 to complete either a CD-VP (n = 64) or a GPT-VP (n = 67). Both scenarios followed a shared acute coronary syndrome blueprint and were delivered through the same mobile-optimized application (MobiViP) with identical interface structure and branching logic. The GPT-VP was initially generated using ChatGPT-5 and refined through iterative follow-up prompting under expert review, with direct manual rewriting of scenario content explicitly prohibited. After scenario completion, students completed a mandatory 19-item questionnaire comprising four prespecified subscales: Realism and Validity, Cognitive Load, Self-reported Learning and Reflection, and Engagement and Satisfaction. Statistical analyses included reliability testing (Cronbach’s alpha), one-sample tests against a neutral midpoint, between-group comparisons (Mann–Whitney U), and correlation analyses (Spearman’s rho).

Results:

A total of 131 complete questionnaires were analyzed, with no item-level missing data. On a 5-point Likert scale, mean subscale scores in the CD-VP group were 3.81 for Realism and Validity, 3.67 for Cognitive Load, 3.64 for Self-reported Learning and Reflection, and 4.16 for Engagement and Satisfaction; corresponding scores in the GPT-VP group were 4.11, 3.03, 3.86, and 4.28. Compared with the CD-VP, the GPT-VP received significantly higher ratings for Realism and Validity (U = 1503.0, p = 0.003) and significantly more favorable Cognitive Load ratings, indicating lower perceived burden (U = 1070.5, p < 0.001). Self-reported Learning and Reflection was marginally higher for the GPT-VP (U = 1719.0, p = 0.050), whereas Engagement and Satisfaction did not differ significantly (U = 1922.5, p = 0.306). Internal consistency was acceptable to good for most subscales in both groups, although Realism and Validity in the GPT-VP group was moderate (Cronbach’s α = 0.67).

Conclusions:

The cardiology GPT-VP, following expert review, was rated at least comparably to the CD-VP and received more favorable ratings in selected learner-perceived domains. These findings support the educational acceptability and practical potential of AI-assisted VP development in undergraduate medical education, while underscoring the continued importance of expert oversight. Future research should examine objective learning outcomes, long-term retention, and scalability across topics as well as cross-institution comparisons. Clinical Trial: The study was not prospectively registered because it was conducted as a minimal-risk educational evaluation embedded within routine teaching and used anonymized post-scenario learner perception data rather than patientlevel clinical outcomes.