Accepted for/Published in: JMIR mHealth and uHealth
Date Submitted: Mar 6, 2023
Date Accepted: Oct 18, 2023
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Assessing the Quality of Digital Health Interventions Across Different Healthcare Domains Using Data-Driven Benchmarking
ABSTRACT
Background:
There are more than 350,000 digital health interventions (DHIs) in the app stores. To ensure that they are effective and safe to use, they should be assessed for compliance with best practice standards.
Objective:
The objective of this paper was to examine the compliance of DHIs with best practice standards and to compute statistical benchmarks for assessing user experience (UX), professional/clinical assurance (PCA) and data privacy (DP). We also compared the quality of DHIs across different healthcare domains.
Methods:
Assessment data were collected from 2040 DHIs using the Organisation for the Review of Care and Health Applications – ORCHA Baseline Review (OBR) assessment tool. As part of the assessment, each DHI received a score out of 100 for each of the abovementioned domains (UX, PCA, DP). The domain scores were combined to make up the overall ORCHA quality score. Inferential statistics, probability distributions, Kruskal-Wallis and Dunn tests were used to conduct the data analysis.
Results:
53.7% of the DHIs demonstrated satisfactory or high quality with scores of 65 or higher. The overall median OBR quality score (ORCHA score) for all DHIs was 63±22 (median ± interquartile range) out of 100. For the three assessment domains, DHI’s scored the highest for the UX assessment (75.21±8.40), followed by DP (65.73±17.54) and PCA (52.60±44.18) assessments. DHIs related to urology (n=20, 74.50±10.50) typically scored highest, followed by DHIs related to respiratory conditions (n=110, 74.00±12.00). DHIs within areas such as women’s health (n=89, 58.00±18.00), ophthalmology (n=69, 49.00±14.00), sexual health (n=69, 57.00±20.00), dental care (n=35, 59.00±21.50) and allergy management (n=17, 54.00±14.00) typically scored the lowest.
Conclusions:
There is a high level of variability in the quality of DHIs across different healthcare domains, with an urgent need to improve compliance with best practices in some areas. Possible explanations for the observed differences might include varied market maturity and commercial interests within the different healthcare domains. More investment to support the development of higher quality DHIs in areas such as women’s health, ophthalmology, dental care and allergy management is needed.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.