JMIR Publications

ABSTRACT

Background:

Clinical communication skills are a core yet complex competency in medical education and practice. However, early-stage medical students often struggle to apply communication frameworks in practice due to limited opportunities for structured simulation training. Generative artificial intelligence (GenAI) supported training has the potential to offer a scalable and innovative approach for clinical communication skills training.

Objective:

This study examined the longitudinal development of medical students’ clinical communication skills over one academic year, and investigated whether AI-powered simulation contributed to differential improvement relative to conventional training. In addition, the study explored motivational, behavioral, and affective mechanisms associated with communication skills development.

Methods:

A non-blinded randomized controlled design was employed, with students assigned to the AI group (n = 22) or the control group (n = 28). Communication skills performance was assessed using standardized Objective Structured Clinical Examinations (OSCE) scores relative to pass marks to enable comparison across time points. Paired t-tests and difference-in-differences (DID) were conducted between groups and over time to examine changes in communication skills. Latent change score modeling was used to identify predictors of communication skills improvement. To examine the mechanism of AI group’s communication skills development, sentiment analysis was conducted on 6,120 comments from AI-student conversations. Finally, a system quality survey was used to understand participants’ feedback on the AI-powered simulator.

Results:

Medical students’ intrinsic motivation and behavioral learning could significantly impact their communication skills development. The AI group presented greater enhancements in communication skills than the control group, even though it was statistically insignificant. Moreover, positive sentiment was significantly associated with the AI group’s communication skills improvement. The AI group’s confidence level was negatively associated with communication improvement, while the group nevertheless reported a generally high level of satisfaction with the system.

Conclusions:

This AI simulator presented an early form of conversational intelligence: capable of responding appropriately but limited in capturing the full complexity of human affect, vulnerability, and narrative unpredictability that characterize real clinical encounters, which offers significant opportunity for future research as this emerging technology becomes more capable and widespread. In conclusion, this study suggests that GenAI holds promise to support foundational clinical communication training, particularly by fostering motivation, behavioral engagement, and providing positive practice environments. Careful instructional and system design are required to ensure the training supports not only fluency but also reflective and emotionally responsive communication skills. Clinical Trial: NA