JMIR Publications

ABSTRACT

Background:

There are more than 350,000 digital health interventions (DHIs) in the app stores. To ensure that they are effective and safe to use, they should be assessed for compliance with best practice standards.

Objective:

The objective of this paper was to examine the compliance of DHIs with best practice standards and to compute statistical benchmarks for assessing user experience (UX), professional/clinical assurance (PCA) and data privacy (DP). We also compared the quality of DHIs across different healthcare domains.

Methods:

Assessment data were collected from 2040 DHIs using the Organisation for the Review of Care and Health Applications – ORCHA Baseline Review (OBR) assessment tool. As part of the assessment, each DHI received a score out of 100 for each of the abovementioned domains (UX, PCA, DP). The domain scores were combined to make up the overall ORCHA quality score. Inferential statistics, probability distributions, Kruskal-Wallis and Dunn tests were used to conduct the data analysis.

Results:

53.7% of the DHIs demonstrated satisfactory or high quality with scores of 65 or higher. The overall median OBR quality score (ORCHA score) for all DHIs was 63±22 (median ± interquartile range) out of 100. For the three assessment domains, DHI’s scored the highest for the UX assessment (75.21±8.40), followed by DP (65.73±17.54) and PCA (52.60±44.18) assessments. DHIs related to urology (n=20, 74.50±10.50) typically scored highest, followed by DHIs related to respiratory conditions (n=110, 74.00±12.00). DHIs within areas such as women’s health (n=89, 58.00±18.00), ophthalmology (n=69, 49.00±14.00), sexual health (n=69, 57.00±20.00), dental care (n=35, 59.00±21.50) and allergy management (n=17, 54.00±14.00) typically scored the lowest.

Conclusions:

There is a high level of variability in the quality of DHIs across different healthcare domains, with an urgent need to improve compliance with best practices in some areas. Possible explanations for the observed differences might include varied market maturity and commercial interests within the different healthcare domains. More investment to support the development of higher quality DHIs in areas such as women’s health, ophthalmology, dental care and allergy management is needed.