Accepted for/Published in: JMIR Formative Research
Date Submitted: Aug 25, 2020
Date Accepted: May 16, 2021
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Next level proctored exam for proficiency testing in Primary Care Education: an observatory study on efficiency and accuracy and on exam outcome.
ABSTRACT
Background:
The COVID19 pandemic affected education and assessment and led to a complex planning. Therefore, we organised the proficiency test for admission to Family Medicine as a proctored exam. To prevent from fraud we developed a virtual supervisor app tracking and tracing candidates’ behaviour.
Objective:
To assess efficiency and accuracy of the proctored exam procedure and to test the impact on the exam scores.
Methods:
The app operates on three levels to register events: recording of actions, analyses of behaviour and live supervision. Each suspicious event is given a score. To assess efficiency we inventoried the technical issues and the interventions. To test accuracy we counted the number of suspicious students and behaviours. To test the impact of the supervising app on students’ exam outcome we compared the scores between the proctored and the on campus group. Candidates were free to register for off or on campus participation.
Results:
593 candidates subscribed to the exam: 472 (79%) candidates used the supervisor app and 121 (20%) were on campus. Test results of both groups were comparable. We registered 15 technical issues in off campus context. Two candidates experienced a negative impact on the exam due to the technical issue. The app detected 22 candidates with a suspicious level >1, mainly increased due to background noise. All events occurred without fraud purpose.
Conclusions:
This pilot study demonstrated that a supervisor app with recording and registration behaviour is able to detect suspicious events without an impact on the exam. Background noise was the most critical event. There was no fraud detected. A supervisor app registering and recording behaviour to prevent from fraud during exams is efficient and not affecting the exam outcome. In future research, a controlled design should compare the cost-benefit balance between the complex intervention of the supervisor app and the combination of the candidates’ awareness of being monitored with a safe exam browsing plug in. Clinical Trial: Not applicable
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.