JMIR Publications

ABSTRACT

Background:

The presence of the mobile phone and devices is generating knowledge about the use of applications to support patient care. However, there are few specific recommendations for apps dedicated to healthcare professionals

Objective:

To establish a validated scale to assess healthcare mobile applications as the most efficient step for health care providers and systems. The main objective is to create and validate a tool to evaluate health apps intended to be used by health professionals.

Methods:

A five steps simplified methodology to assess the scale was followed. The first step consists of building a scale for professionals based on a literature review. The following step is an expert panel validation by a Delphi method, rating web-based questionnaires to evaluate inclusion and weight of the indicators. It was agreed to carry out as many iterations as necessary to reach a consensus of 75%. Finally, a pilot of the score was developed to evaluate the reliability of the scale. For the inter-rater agreement assessment during the pilot, the Cohen Kappa was used.

Results:

After the literature review, a first scale draft was developed. Two rounds of interactions of the local research group and the external panel of experts were needed. After the group’s consensus, from the 18 indicators, a 17-item scale (ISYScore-Pro) was presented. For the pilot and validation test, 14 clusters from the 10th version of the International Classification of Diseases (ICD-10) were included, 280 apps were screened, 66 evaluated and 18 met the criteria. The interrater agreement was strong (92.2%, kappa 0.840 [0.834-0.847], p<0.0001) and consistent significatively after weighting.

Conclusions:

We have developed, with reproducible methodology, a tool that allows for the evaluation of health applications for health professionals and providers. The ISYScore-PRO scale is reliable and reproducible. The assessment allowed to consolidate every step of the methodology. The process of validation included robust methods. For future research, this tool is adaptable to any language and more specific domains.