Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Accepted for/Published in: JMIR Serious Games

Date Submitted: Jan 16, 2020
Date Accepted: Apr 19, 2020

The final, peer-reviewed published version of this preprint can be found here:

Biosensor Real-Time Affective Analytics in Virtual and Mixed Reality Medical Education Serious Games: Cohort Study

Antoniou PE, Arfaras G, Pandria N, Athanasiou A, Ntakakis G, Babatsikos E, Nigdelis V, Bamidis P

Biosensor Real-Time Affective Analytics in Virtual and Mixed Reality Medical Education Serious Games: Cohort Study

JMIR Serious Games 2020;8(3):e17823

DOI: 10.2196/17823

PMID: 32876575

PMCID: 7495262

Biosensor Real-time Affective Analytics in Virtual and Mixed Reality Medical Education Serious Games.

  • Panagiotis E. Antoniou; 
  • George Arfaras; 
  • Niki Pandria; 
  • Alkinoos Athanasiou; 
  • George Ntakakis; 
  • Emmanouil Babatsikos; 
  • Vasilis Nigdelis; 
  • Panagiotis Bamidis

ABSTRACT

Background:

The role of emotion is crucial to the learning process, since it is linked to motivation, interest and attention. Expression of such affective states has been demonstrated in the brain and in overall biological activity. Bio-signals such as Heart Rate (HR), Electro-Dermal Activity (EDA), and Electroencephalography (EEG) reflect physiological expressions of the human body that change during shifts of emotional state. Thus, analysing changes of bio-signal recordings can point to a person’s emotional state. Contemporary medical education has progressed extensively towards diverse learning resources using Virtual/Mixed Reality (V/MR) applications.

Objective:

This work is the first study of wearable, bio-sensor based affect detection in a learning processes involving a serious game in the Microsoft HoloLens V/MR platform.

Methods:

A wearable array of sensors recording HR, EDA and EEG signals was deployed during two educational activities conducted by 11 subjects of diverse educational level (undergraduate, post-graduate and specialist neurosurgeon doctors). The first scenario was a conventional Virtual Patient case used for establishing the personalized biosignal baselines for the subject. The other was a similar case in a V/MR environment regarding neuroanatomy. The affective measures that we recorded were EEG-theta/beta ratio, alpha rhythm, Heart Rate (HR) and ElectroDermal Activity (EDA)

Results:

Results have been recorded and aggregated across all three groups. Average EEG-theta/beta ratios of the VP vs the MR serious game cases were recorded at 3.49±0.82 vs 3.23±0.94 for students, 2.59±0.96 vs 2.90±1.78 for neurosurgeons and 2.33±0.26 vs 2.56±0.62. for postgraduate medical students. Average alpha rhythm of the VP vs the MR serious game cases were recorded at 7.77±1.62μV vs 8.42±2.56μV for students, 7.03±2.19μV vs 7.15±1.86μV for neurosurgeons and 11.84±6.15μV vs 9.55±3.12μV for postgraduate medical students. Average HR of the VP vs the MR serious game cases were recorded at 87±13 vs 86±12 bpm for students, 81±7 vs 83±7 bpm for neurosurgeons and 81±7 vs 77±6 bpm for postgraduate medical students. Average EDA of the VP vs the MR serious game cases were recorded at 1.198±1.467μS vs 4.097±2.79μS for students, 1.890±2.269μS vs 5.407±5.391μS for neurosurgeons and 0.739±0.509μS vs 2.498±1.72μS for postgraduate medical students. The variations of these metrics have been correlated with existing theoretical interpretations regarding educationally relevant affective analytics such as engagement and educational focus.

Conclusions:

These results demonstrate that this novelsensor configurations provides a bio-signals’ array that can lead to credible affective state detection and can be utilized in platforms like intelligent tutoring systems for providing real-time, evidence-based, affective learning analytics using V/MR deployed medical education resources.


 Citation

Please cite as:

Antoniou PE, Arfaras G, Pandria N, Athanasiou A, Ntakakis G, Babatsikos E, Nigdelis V, Bamidis P

Biosensor Real-Time Affective Analytics in Virtual and Mixed Reality Medical Education Serious Games: Cohort Study

JMIR Serious Games 2020;8(3):e17823

DOI: 10.2196/17823

PMID: 32876575

PMCID: 7495262

Download PDF


Request queued. Please wait while the file is being generated. It may take some time.

© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.