JMIR Publications

ABSTRACT

Background:

Immersive virtual reality (iVR) has emerged as a training method to prepare medical first responders (MFRs) for mass casualty incidents (MCIs) and disasters in a resource-efficient, flexible, and safe manner. However, systematic evaluations and validations of potential performance indicators for virtual MCI training are still lacking.

Objective:

This study aimed at evaluating the usefulness of different performance indicators based on visual attention, triage performance, and information transmission in virtual MCI scenarios to discriminate between different levels of expertise. Furthermore, we aimed to investigate the extent to which such objective parameters correlate with subjective performance assessments.

Methods:

Seventy-six participants (age M = 25.54 years, SD = 6.01, 59% male) with different medical expertise (MFRs: paramedics and emergency doctors; non-MFRs: medical students, in-hospital nurses, and other doctors) participated in five virtual MCI scenarios of varying complexity in a randomized order. Tasks involved assessing the situation, triaging virtual patients, and transmitting relevant information to a control center. Performance measures included eye-tracking based attention, triage accuracy, triage speed, information transmission efficiency, and self-assessment of performance. Expertise was determined based on the occupational group (39 MFRs vs. 37 non-MFRs) and a knowledge test with patient vignettes.

Results:

Triage accuracy (d = 0.48), triage speed (d = 0.42), and information transmission efficiency (d = 1.13) differentiated significantly between MFRs and non-MFRs. Additionally, higher triage accuracy was significantly associated with higher triage knowledge test scores (Spearman’s rho = .40). Eye-tracking based attention was not significantly associated with expertise. Furthermore, subjective performance was not correlated with any other performance indicator.

Conclusions:

iVR-based MCI scenarios proved to be a valuable tool for assessing the performance of MFRs. The results emphasize the potential for iVR training to supplement traditional MCI training by providing training scenarios in which different tasks can be practiced in a controlled environment. In particular, performance indicators such as triage accuracy, triage speed, and information transmission efficiency capture multiple aspects of performance and are recommended for integration. While eye-tracking-based attention did not function as a valid performance indicator in this study, future research should further explore gaze behavior as a potential indicator by examining other facets such as gaze patterns. Overall, the results underscore the importance of integrating objective parameters to enhance trainers' feedback and provide trainees with guidance on evaluating and reflecting on their own performance.