JMIR Publications

ABSTRACT

Background:

While smartphone-based emergency care training is more affordable than traditional avenues of training, it is still in its infancy, remains poorly implemented, and current implementations tend to be invariant to the evolving learning needs of the intended users. In resource-limited settings, the use of such platforms coupled with gamified approaches remain largely unexplored and under-developed. This is despite the lack of traditional training opportunities and continuing high neonatal mortality rates in these settings.

Objective:

The primary aim of this randomised experiment was to assess the effectiveness of offering adaptive versus standard feedback through a smartphone-based serious game on healthcare providers’ knowledge gains when managing a simulated (gamified) medical emergency. A secondary aim was to assess the effects of learner characteristics and learning spacing on individual learning with repeated use of the game using the secondary outcome of individualised normalised learning gain.

Methods:

The experiment was aimed at healthcare workers (physicians, nurses and clinical officers) who provide bedside neonatal care in low-income settings. Data were captured through an Android smartphone-based gamified application installed on the study participants personal phones. The intervention -which was based on successful attempts at a learning task - was adaptive feedback provided within the application to the experimental arm while the control arm received standardised feedback. The primary endpoint was completion of the second learning session within the application. We enrolled 572 participants between February 2019 and July 2019 of which 247(43.18%) reached the primary endpoint. The primary outcome measure was standardised relative change in learning gains between the study arms as measured by the “Morris G” effect size. The secondary outcome was participants individualised and normalised learning gains.

Results:

The effect of adaptive feedback on healthcare providers’ learning gain was found to be g = 0.09 (95% CI: -0.31 – 0.46, p-value = .474). In exploratory analysis, using normalised learning gains, when subject-treatment interaction and differential time effect was controlled for, this effect increased significantly to 0.644 (95% CI: 0.35 - 0.94, p-value <.001) with immediate repetition which is a moderate learning effect but reduced significantly by 0.28 within a week. The overall learning change from LIFE use in both arms was large and may have obscured a direct effect of feedback.

Conclusions:

There is a considerable learning gain between first and second rounds of learning with both forms of feedback and a small added benefit of adaptive feedback after controlling for learner differences. We suggest that linking the adaptive feedback provided to healthcare providers to how they space their repeat learning session(s) may yield higher learning gains. Future work might explore in more depth feedback content: in particular whether explanatory feedback (why answers were wrong) may enhance learning more than reflective feedback (information about what the right answers are). Clinical Trial: Pan African Clinical Trials Registry PACTR201901783811130; https://pactr.samrc.ac.za/TrialDisplay.aspx?TrialID=5836