JMIR Publications

ABSTRACT

Background:

Clinical reasoning is a fundamental process that students must learn and be able to put into practice in medical school training. This requires the development of multiple abilities through practice with patients. Nevertheless, in contexts like emergency services the access is restricted for students. Virtual patients have shown to be a useful tool in the acquisition of knowledge and decision-making training for medical students. In recent years, large language models (LLMs) have emerged as an alternative that functions as conversational virtual patients allowing the students to refine their clinical reasoning abilities in a safe environment.

Objective:

The objective of our study was to evaluate the perception of medical students about PRAXIA in terms of communicative realism, case consistency, and its utility as a complementary pedagogical tool in medical student training.

Methods:

We employed a Design Thinking approach to develop PRAXIA, an LLM-based virtual simulation environment designed to enhance clinical reasoning. We assessed student performance using a custom rubric integrating the Revised-IDEA framework with the Dreyfus model of skill acquisition. Additionally, we designed a validation survey to measure communicative realism, story consistency, situational fidelity, feedback value, and technical usability. In a prospective study, 28 medical students from the Universidad de Chile completed a 10-minute simulated history and physical exam, followed by clinical decision-making tasks. Post-simulation, participants completed the user experience survey. We strictly observed data privacy and ethical standards.

Results:

Twenty-eight medical students from Universidad de Chile participated in the study and reported high overall satisfaction across all five prototype dimensions. Highest scores were for conversational realism (mean 3.58/4.00) and case coherence/consistency (mean 3.73/4.00 for both story consistency and situational fidelity), while the lowest was for formative feedback value (mean 3.29/4.00). Qualitative feedback from students estimated the experience as "useful," highlighting the virtual patient's fluency and human-like messages, but suggested the need for more specific feedback. Our psychometric analysis revealed acceptable to excellent internal consistency using the corrected item-total correlation (CITC). The "Formative value of the feedback" dimension demonstrated a Cronbach's alpha of 0.82, showing high internal consistency.

Conclusions:

PRAXIA, an LLM-based virtual patient for emergency medicine, shows high acceptability, educational value and promising usability. Its focus on realism, contextual fidelity, and formative feedback complements existing simulations and provides a basis for future studies on its effect on clinical reasoning.