Accepted for/Published in: JMIR mHealth and uHealth
Date Submitted: Oct 29, 2020
Date Accepted: Mar 18, 2021
Measuring quality of clinical skills mobile apps for student learning: A review and comparison of the Mobile App Rating Scale (MARS) and Mobile App Rubric for Learning (MARuL)
ABSTRACT
Background:
Mobile apps are widely used in the health professions, increasing the need for simple methods for determining the quality of an app. In particular, teachers need the ability to curate high quality mobile apps for student learning.
Objective:
The aim was to systematically search the two most common app stores, the iOS App Store and Google Play Store, for clinical skills mobile apps. The quality of apps meeting criteria would be evaluated using two measures; the widely used Mobile Apps Rating Scale (MARS) which measures general app quality, and the Mobile App Rubric for Learning (MARuL) a recently developed instruments which measures the value of apps for student learning.
Methods:
The two mobile app stores were systematically searched using clinical skills terms commonly found in medical education and apps meeting criteria identified using an approach based on PRISMA guidelines. The screening process identified nine apps. Apps were rated independently by two reviewers using the Mobile App Rating Scale (MARS) and Mobile App Rubric for Learning (MARuL).
Results:
: Intraclass correlation coefficients (ICC) for the two raters were similar with ICC scores for the MARS (ICC(2-way) = 0.68, p<.001) and the MARuL (ICC(2-way) = 0.68, p<.001) indicating moderate reliability. Two of the nine apps, Geeky Medics – OSCE revision (MARS Android=3.74/5, MARS iOS=3.68/5, MARuL Android=75/104, MARuL iOS=73/104) and OSCE PASS: Medical Revision (MARS Android=3.79/5, MARS iOS=3.71/5, MARuL Android=69/104, MARuL iOS=73/104), scored highly on both measures of app quality and for both Android and iOS. Both measures also showed agreement for the lowest rated app, Patient Education Institute (MARS Android=2.21/5, MARS iOS=2.11/5, MARuL Android=18/104, MARuL iOS=21.5/104), which had the lowest scores in all categories except Information (MARS) and Professional (MARuL) in both operating systems. For the remaining apps, their rankings differed based on both the measure used and the operating system.
Conclusions:
Both the MARS and MARuL showed similar ability to differentiate the quality of the nine apps. However, the MARuL's incorporation of teaching and learning elements as part of a multidimensional measure of quality may make it more appropriate for use with apps focussed on teaching and learning, whereas the MARS's more general rating of quality may be more appropriate for health apps targeting a general health audience. Ratings of the nine apps by both measures also highlighted the variable quality of clinical skills mobile apps for learning.
Citation
Per the author's request the PDF is not available.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.