JMIR Publications

ABSTRACT

Background:

Training social verbal interactions is crucial for medical first responders (MFRs) to assess a patient’s condition and perform urgent treatment during emergency medical services (EMS). Integrating Conversational agents (CAs) in Virtual patients (VP), i.e., digital simulations, for medical communication training is a cost-effective alternative to resource-intensive human role playing. CAs have also shown moderate evidence to more effectively improve communication skills when used with instructional interventions. However, more recent generative pre-trained transformer (GPT) AI produces richer, more diverse and natural responses than previous CAs, and can also control prosodic voice qualities like pitch and duration. These functionalities have the potential to better match the interaction expectations of MFRs, i.e., the habitability.

Objective:

We aimed to study how the integration of GPT AI in a mixed reality (MR)-VP could support communication training of MFRs.

Methods:

We developed an MR simulation of a traffic accident with a VP. OpenAI ChatGPT was integrated in the VP and prompted with verified characteristics of accident’s victims. MFRs (n=24) were instructed on how to interact with the MR scenario. After assessing and treating the VP, the MFRs were administered the Mean Opinion Scale (MOS) X2 and the Subjective Assessment of Speech System Interfaces (SASSI) questionnaires to study their perception of the voice quality and the usability of the voice interactions correspondingly. Open questions took place after the questionnaires. The observed and logged interactions with the VP, descriptive statistics of the questionnaires and the output of the open questions are reported.

Results:

The usability assessment of the VP resulted in moderate positive ratings specially in habitability (M=5.25) and annoyance (M=1.8). It was noted that interactions were negatively affected by the approximately 3 seconds processing-time delays of the responses. MFRs acknowledged the naturalness of determining physiological states of the VP through verbal communication, e.g., Where does it hurt? However, the question-answer dynamic in the verbal exchange with the VP and the lack of the VP’s ability to start the verbal exchange were noticed. Noteworthy insights highlight the potential of domain-knowledge prompt engineering to steer the actions of MFRs for effective training.

Conclusions:

Generative AI in VPs facilitates MFRs’ training, but continues to rely on instructional interventions for effective verbal interactions. Therefore, the capabilities of the GPT-VP and a detailed training protocol need to be communicated to trainees. Future interactions should allow triggers by keyword recognition, e.g., the VP pointing to the hurting area, implementing conversational turn-taking techniques and VP’s ability to start a verbal exchange. Furthermore, a local AI server, chunk processing and lowering the audio resolution of the VP´s voice, could ameliorate the delay in response and privacy concerns. Prompting could be used in future studies to create a virtual MFR capable of assisting trainees.