Large Language Model-powered AI-assisted Simulated Patient for Nursing Education: Quasi-Experimental Study
ABSTRACT
Large Language Model (LLM)-powered virtual patient systems are gaining popularity in nursing education but also face persistent challenges, such as hallucinations and role inconsistency. This paper reports the model architecture design, technical considerations, implementation challenges, and evaluation of an LLM-powered AI-assisted simulated patient integrated into a nursing triage simulation game. The system couples a Unity game engine client with OpenAI GPT-4o to support real-time Cantonese-language clinical dialogues using Traditional Chinese characters and idiomatic spoken syntax. Key technical contributions include a layered hallucination mitigation architecture that combines grounded prompt inclusion, few-shot conversational exemplars, negative role constraints, and calibrated model parameters, reducing script-inconsistent hallucinations from approximately 25% to below 5% across test cases. The implementation process also addresses stateless API memory management, cloud-based safety filter calibration for domain-specific clinical prompts, and prompt engineering strategies to prevent role drift and hidden scenario information leakage. The system was evaluated with 522 nursing students, demonstrating that GPT-4o maintained 85–90% script adherence across 50+ test interactions per case scenario, significantly outperforming alternative models tested. This study presents a replicable technical framework encompassing key considerations, recurring implementation challenges, and practical suggestions, culturally and linguistically contextualized for nursing education, with implications for broader adoption in non-Anglophone healthcare education settings.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.