Accepted for/Published in: JMIR mHealth and uHealth
Date Submitted: Jan 1, 2020
Date Accepted: Jan 15, 2021
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Development of a tool to assess health applications for healthcare providers: A Delphi Expert Panel Study
ABSTRACT
Background:
The presence of the mobile phone and devices is generating knowledge about the use of applications to support patient care, but there are few recommendations for apps dedicated to healthcare professionals
Objective:
To establish a validated scale to assess healthcare mobile applications is the most efficient step for health care providers and systems. The main goal is to create and validate a tool to evaluate health apps destined to be used by health professionals.
Methods:
A five steps simplified methodology to assess of the scale was followed. The first step consists of building a scale for professionals based on a literature review. Next step would be an expert panel validation by a Delphi method, rating web-based questionnaires to evaluate inclusion and weight of the indicators. It was agreed to carry out, as many iterations as necessary, to reach a consensus of 75%. Finally, a pilot of the score was developed to evaluate the reliability of the scale. For the inter-rater agreement assessment during the pilot, the Cohen Kappa was used.
Results:
After the literature review, a first scale draft was developed. Two rounds of interactions of the local investigation group and the external panel of experts were needed to select final indicators. Seventeen indicators were included in the score. For the pilot test, 280 apps were evaluated and 66 meet the criteria. The interrater agreement was strong (higher than 82% with significant kappa >0.72 per app and item).
Conclusions:
We have developed, with a reproducible methodology, a tool that allows us to evaluate health applications for clinical, surgical and general medical providers. The ISYScore-PRO scale to be reliable and reproducible. The assessment permitted to consolidate every step of the methodology. We were able to reach consensus on the dimensions and items on the scale with only two rounds. The process of validation included two robust methodologies. The ISYScore-PRO scale is reliable and reproducible.
Citation
