JMIR Publications

ABSTRACT

Background:

Extended reality (XR) technologies, including virtual reality (VR), augmented reality (AR), and mixed reality (MR), are increasingly used in medical education to create immersive and interactive learning environments. As these modalities expand, validated instruments are needed to measure learners’ experiences accurately. The Immersive Technology Evaluation Measure (ITEM) was developed in the United Kingdom as a multidomain questionnaire assessing immersion, motivation, cognitive load, usability, and debriefing. However, its interpretability in English-medium medical programs where English is not learners’ first language has not been examined.

Objective:

This study aimed to evaluate the clarity, comprehensibility, and response processes of ITEM among preclinical medical students in a multilingual English-medium medical education context.

Methods:

We conducted a qualitative cognitive interviewing study at the College of Medicine and Health Sciences, United Arab Emirates University, with Year 3 and 4 undergraduate medical students (n=10) after completion of VR-based clinical skills modules. Using concurrent think-aloud and verbal probing techniques, students explained their interpretations of each ITEM question and their reasoning for the selected responses. We analyzed transcripts using content analysis to identify comprehension issues and response-process challenges. We refined unclear items through research team consensus, with input from an English language expert, while preserving the instrument’s conceptual structure.

Results:

Participants identified 47.5% (19/40) of ITEM items as unclear or needing refinement, most commonly due to unfamiliar or abstract terminology and nonspecific references that were interpreted inconsistently. The immersion and usability domains contained the most unclear items. We added brief parenthetical clarifications and explicit references to the “VR activity” to 17 items. We removed no items and made no structural changes to the questionnaire.

Conclusions:

Cognitive interviewing identified targeted linguistic refinements that improved ITEM interpretability for non-native English speakers while preserving the intended constructs. The adapted ITEM is suitable for further pilot testing and psychometric validation in diverse immersive Technology-Enhanced Learning (TEL) settings. Clinical Trial: Not Applicable