JMIR Publications

ABSTRACT

Background:

Artificial intelligence (AI)-powered virtual patient systems provide medical students with repeatable practice environments for history-taking training. However, user acceptance of such systems and the experience dimensions associated with that acceptance lack systematic mixed-methods evidence.

Objective:

This study aimed to (1) examine the associations of system experience and learning experience/intrinsic motivation with overall acceptance among undergraduate medical students using an AI-Powered Medical History-Taking Training and Evaluation System (AMTES); (2) explore user experience patterns through open-ended questions; and (3) integrate quantitative and qualitative findings to inform system refinement and pedagogical implementation.

Methods:

A cross-sectional convergent mixed-methods design was used. Sixty-six undergraduate medical students at a Chinese medical college completed a post-use questionnaire after AMTES training. The primary outcome was an overall acceptance composite combining use intention, recommendation intention, and overall satisfaction. Associations with system experience and learning experience/intrinsic motivation were examined using linear regression with HC3 robust standard errors and bootstrap confidence intervals. Sensitivity analyses included covariate adjustment, single-outcome models, a fractional logit model, and content-overlap sensitivity checks for the system-experience composite. Open-ended responses were analyzed using codebook-oriented thematic analysis and integrated with quantitative findings through a joint display.

Results:

Both system experience and learning experience/intrinsic motivation were positively associated with overall acceptance (standardized β=0.526 and 0.377, respectively; both P≤.002; R²=0.662). Sensitivity analyses supported the robustness of both positive associations. Qualitative analysis showed that the most frequently nominated benefits clustered within the Practice Accessibility and Feedback Support theme, especially self-directed practice (42.4%) and immediate feedback (37.9%). Refinement priorities clustered around four dialogue-quality and assessment dimensions, including semantic understanding, contextual consistency, conversational naturalness, and scoring logic. These dimensions were each cited by 21% to 35% of respondents. The mixed-methods joint display confirmed contextual alignment on dialogue and scoring concerns and revealed that perceived scoring-feedback discrepancies may erode student trust in the feedback function. This represents a cross-dimensional pattern that was not apparent from the quantitative model alone.

Conclusions:

In this exploratory cohort of undergraduate medical students, both system interaction quality and perceived learning value were positively associated with overall acceptance of AMTES, with system interaction quality showing the stronger association. Dialogue coherence, semantic understanding, and scoring-feedback alignment emerged as the most frequently nominated refinement priorities and are plausible candidate targets for improving acceptance-related perceptions. Unlike prior work focused on technical reliability or educational effectiveness, this study extends the focus to implementation-level acceptance, suggesting that interaction quality may be associated with constrained acceptance even when learning value is recognized. These findings may inform system refinement priorities and the curricular integration of AI-powered history-taking training systems.