JMIR Publications

ABSTRACT

Background:

Many frameworks and tools are available to evaluate the quality of mobile health apps (MHA) which are increasingly used by healthcare professionals (HCPs) for accessing medical information, clinical decision support, and communication. However, existing tools are not well-equipped to assess the quality of apps designed for HCPs from their perspectives.

Objective:

We aimed to develop a new tool based on the Mobile App Rating Scale (MARS) to capture the unique priorities of HCPs from their perspective. We then conducted a psychometric analysis of this new questionnaire to determine its effectiveness in assessing the quality of MHAs designed specifically for HCPs from their perspectives.

Methods:

This study was conducted in two phases. In Phase 1, the original MARS tool was adapted for HCPs through expert panel review and subsequent qualitative interviews, resulting in the development of the pMARS tool. This phase focused on establishing face and content validity. Qualitative interviews were conducted with HCPs from a tertiary hospital in Singapore to gather their perspectives on the tool’s structure, clarity, applicability, and usability. Phase 2 invited HCP participants to complete pMARS based on their experience with the LabMed app, a mHealth tool designed to provide medical laboratory-related information to HCPs. We established the construct validity of pMARS through multiple psychometric techniques. Internal consistency reliability was measured using Cronbach’s alpha, while structural equation modeling (SEM) was employed to examine the interrelationships among latent constructs. Additionally, we employed item response theory (IRT) to evaluate each item’s impact on latent constructs of interest, i.e., discriminative performance of individual items within each domain.

Results:

Based on the results from Phase 1, the pMARS comprised 26 items across five domains: engagement, functionality, aesthetics, information, and subjective quality, refined through interviews with ten HCPs. In Phase 2 (n=218), pMARS demonstrated good internal consistency reliability across all domains (Cronbach’s alpha: 0.855-0.931). SEM demonstrated that functionality had the strongest influence on end-user willingness to use/recommend and purchase the MHA (p<.001). IRT identified that customization and interactivity of the engagement domain have a weak impact on latent constructs, whereas entertainment has a higher impact. Ease of use and gestural design had a weak impact on the functionality domain, whereas arrangement and size of content and quantity and quality had a strong impact on aesthetics and information domains, respectively.

Conclusions:

This study reports the development and psychometric analysis of pMARS. This study reports the development and psychometric analysis of pMARS. Our findings demonstrate strong internal consistency reliability and construct validity, supporting its potential use in healthcare. Further research should validate pMARS across diverse MHAs and contexts, and apply IRT to further refine its precision and efficiency. Clinical Trial: Not applicable