Accepted for/Published in: JMIR Human Factors
Date Submitted: Jan 1, 2024
Date Accepted: Jul 7, 2024
Evaluating the Usability and Quality of a Clinical Mobile Application for Assisting Physicians in Head CT Scan Ordering: A Think Aloud and Mobile Apps Rating Scale (MARS) Study
ABSTRACT
Background:
Among the numerous factors contributing to healthcare providers' (HCPs) engagement with mobile apps (apps), including user characteristics (dexterity, anatomy, and attitude) and mobile features (screen and button size), usability and quality of apps were introduced as the most influential factors.
Objective:
Therefore, this study aims to investigate the usability and quality of the Head CT Scan Appropriateness Criteria mobile application (HAC app) for physicians' CT scan ordering.
Methods:
Our study design was primarily based on methodological triangulation by utilizing mixed methods research involving quantitative analysis of the Mobile Apps Rating Scale (MARS) for quality assessment, quantitative and qualitative Think Aloud (TA) usability testing, and debriefing across three phases. Sixteen medical interns participated in quality assessment and testing usability characteristics, including efficiency, effectiveness, learnability, error, and satisfaction with the HAC app.
Results:
The effectiveness of the HAC app was deemed satisfactory, with a rating of 96.9%. MARS assessment scale also indicated the overall favorable (82 out of 100) quality score of the HAC app. Scoring four MARS subscales, "Information" (73.37 out of 100) and "Engagement" (73.48 out of 100) had the lowest scores. Analysis of the items in each MARS subscale revealed that in the "Engagement" subscale, the lowest score of the HAC app was "customization" (63.6 out of 100). In the "Functionality" subscale, the HAC app's lowest value was "Performance" (67.4 out of 100). Qualitative TA usability testing of the HAC app found eight notable usability issues also highlighted in the MARS quality assessment.
Conclusions:
Evaluating the quality and usability of mobile apps using a mixed methods approach provides valuable information about the functionality and disadvantages of the mobile apps and complements each other. It is highly recommended to embrace a more holistic and mixed methods strategy when evaluating mobile apps, as relying solely on one method proves imperfect to reflect trustworthy and reliable information regarding the usability and quality of apps.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.