Accepted for/Published in: JMIR Serious Games
Date Submitted: Jan 16, 2020
Date Accepted: Apr 19, 2020
Biosensor Real-time Affective Analytics in Virtual and Mixed Reality Medical Education Serious Games.
ABSTRACT
Background:
The role of emotion is crucial to the learning process, since it is linked to motivation, interest and attention. Expression of such affective states has been demonstrated in the brain and in overall biological activity. Bio-signals such as Heart Rate (HR), Electro-Dermal Activity (EDA), and Electroencephalography (EEG) reflect physiological expressions of the human body that change during shifts of emotional state. Thus, analysing changes of bio-signal recordings can point to a person’s emotional state. Contemporary medical education has progressed extensively towards diverse learning resources using Virtual/Mixed Reality (V/MR) applications.
Objective:
This work is the first study of wearable, bio-sensor based affect detection in a learning processes involving a serious game in the Microsoft HoloLens V/MR platform.
Methods:
A wearable array of sensors recording HR, EDA and EEG signals was deployed during two educational activities conducted by 11 subjects of diverse educational level (undergraduate, post-graduate and specialist neurosurgeon doctors). The first scenario was a conventional Virtual Patient case used for establishing the personalized biosignal baselines for the subject. The other was a similar case in a V/MR environment regarding neuroanatomy. The affective measures that we recorded were EEG-theta/beta ratio, alpha rhythm, Heart Rate (HR) and ElectroDermal Activity (EDA)
Results:
Results have been recorded and aggregated across all three groups. Average EEG-theta/beta ratios of the VP vs the MR serious game cases were recorded at 3.49±0.82 vs 3.23±0.94 for students, 2.59±0.96 vs 2.90±1.78 for neurosurgeons and 2.33±0.26 vs 2.56±0.62. for postgraduate medical students. Average alpha rhythm of the VP vs the MR serious game cases were recorded at 7.77±1.62μV vs 8.42±2.56μV for students, 7.03±2.19μV vs 7.15±1.86μV for neurosurgeons and 11.84±6.15μV vs 9.55±3.12μV for postgraduate medical students. Average HR of the VP vs the MR serious game cases were recorded at 87±13 vs 86±12 bpm for students, 81±7 vs 83±7 bpm for neurosurgeons and 81±7 vs 77±6 bpm for postgraduate medical students. Average EDA of the VP vs the MR serious game cases were recorded at 1.198±1.467μS vs 4.097±2.79μS for students, 1.890±2.269μS vs 5.407±5.391μS for neurosurgeons and 0.739±0.509μS vs 2.498±1.72μS for postgraduate medical students. The variations of these metrics have been correlated with existing theoretical interpretations regarding educationally relevant affective analytics such as engagement and educational focus.
Conclusions:
These results demonstrate that this novelsensor configurations provides a bio-signals’ array that can lead to credible affective state detection and can be utilized in platforms like intelligent tutoring systems for providing real-time, evidence-based, affective learning analytics using V/MR deployed medical education resources.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.