Currently submitted to: JMIR Human Factors
Date Submitted: Nov 9, 2026
Open Peer Review Period: Nov 9, 2026 - Jan 4, 2027
(currently open for review)
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Usability and Perceived Realism of an Artificial Intelligence Assisted Acute Care Simulator for Medical Students: Prospective Survey and Qualitative Evaluation
ABSTRACT
Background:
Artificial intelligence (AI) assisted simulation can combine patient dialogue with changing clinical observations. Its usefulness in medical education depends on how learners experience the whole system, including the interface, scenario behavior, and teaching support.
Objective:
To evaluate the usability, acceptability, and perceived realism of BristolSimLab during supervised acute care teaching, and identify design features and limitations that could inform further development.
Methods:
We evaluated supervised use by 42 third-year medical students during a prehospital emergency medicine field trip. Students completed two acute care scenarios in pairs, with AI-generated feedback and support from clinical teaching fellows. Pre-session (n=40) and post-session (n=42) records assessed confidence before and after use, and usability, realism, and acceptability afterward. Free-text responses underwent descriptive content analysis. A concurrent incident log recorded detected system issues.
Results:
The mean System Usability Scale score was 81.3 (SD 10.4), and 41/42 respondents (97.6%) considered the activity acceptable. Clinically realistic responses to interventions were endorsed by 39/42 (92.9%), while 28/42 (66.7%) endorsed resemblance to managing a real acutely ill patient. Free-text categories included responsiveness or progression (16/42), temporal features (12/42), absent visual cues (12/42), and delegation through the nurse role (7/42). Confidence ratings were higher after use at the cohort level (P<.001). Mean scores on the different knowledge forms were both 6.5/8, which could not establish knowledge change.
Conclusions:
BristolSimLab received favorable usability and acceptability ratings in supervised teaching. Learner feedback identified responsive scenario behavior as a strength and visual cues, role interaction, and feedback explanation as priorities for refinement. These findings support further task-based evaluation; they do not establish individual learning gains or a general system safety profile.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.