Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.
Who will be affected?
Readers: No access to all 28 journals. We recommend accessing our articles via PubMed Central
Authors: No access to the submission form or your user account.
Reviewers: No access to your user account. Please download manuscripts you are reviewing for offline reading before Wednesday, July 01, 2020 at 7:00 PM.
Editors: No access to your user account to assign reviewers or make decisions.
Copyeditors: No access to user account. Please download manuscripts you are copyediting before Wednesday, July 01, 2020 at 7:00 PM.
Hoogendoorn P, Versluis A, van Kampen S, McCay C, Leahy M, Bijlsma M, Bonacina S, Bonten T, Bonthuis MJ, Butterlin A, Cobbaert K, Duijnhoven T, Hallensleben C, Harrison S, Hastenteufel M, Holappa T, Kokx B, Morlion B, Pauli N, Ploeg F, Salmon M, Schnoor K, Sharp M, Sottile PA, Värri A, Williams P, Heidenreich G, Oughtibridge N, Stegwee R, Chavannes N
What Makes a Quality Health App—Developing a Global Research-Based Health App Quality Assessment Framework for CEN-ISO/TS 82304-2: Delphi Study
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
What Makes a Quality Health App? Developing a Global Research-based Health App Quality Assessment Framework for CEN-ISO/TS 82304-2
Petra Hoogendoorn;
Anke Versluis;
Sanne van Kampen;
Charlie McCay;
Matt Leahy;
Marlou Bijlsma;
Stefano Bonacina;
Tobias Bonten;
Marie-José Bonthuis;
Anouk Butterlin;
Koen Cobbaert;
Thea Duijnhoven;
Cynthia Hallensleben;
Stuart Harrison;
Mark Hastenteufel;
Terhi Holappa;
Ben Kokx;
Birgit Morlion;
Norbert Pauli;
Frank Ploeg;
Mark Salmon;
Kymo Schnoor;
Mary Sharp;
Pier Angelo Sottile;
Alpo Värri;
Patricia Williams;
Georg Heidenreich;
Nicholas Oughtibridge;
Robert Stegwee;
Niels Chavannes
ABSTRACT
Background:
Consensus on what makes a quality health app and an international standard for assessing and communicating quality in health apps have been lacking. At the request of the European Commission, international Standard Development Organizations (SDOs) have joined forces to develop a Technical Specification (TS) for the quality and reliability of health and wellness apps.
Objective:
This study aimed to create a trusted, useful, usable, and global health app quality assessment framework.
Methods:
A 2-round Delphi technique with 83 experts participating in one (n=42) or both (n=41) rounds was used to establish consensus on a draft global health app quality assessment framework. A follow-up survey with 27 respondents informed the scoring mechanism. After subsequent alignment with related standards, the quality assessment framework was tested and fine-tuned with manufacturers of 11 COVID-19 symptom apps. National mirror committees participating in the SDO technical committees commented and voted on the TS.
Results:
The final quality assessment framework includes 81 questions, 67 of which impact the scores in 4 overarching quality aspects (Healthy and safe, Easy to use, Secure data and Robust build). The scoring mechanism enables communication of the quality assessment results in a health app quality score and label alongside a detailed report. The manufacturers found the required time to complete the assessment acceptable (2 to 4 days). Publication of CEN-ISO/TS 82304-2:2021 Health software – Part 2: Health and wellness apps – Quality and reliability was approved in May 2021 in a nearly unanimous vote by 34 national SDOs globally.
Conclusions:
A useful and usable global standard in health app quality assessment was developed. This will help health app manufacturers enhance and demonstrate quality of health apps, consumers and healthcare professionals make informed decisions on health apps and insurers in reimbursement decisions on health apps.
Citation
Please cite as:
Hoogendoorn P, Versluis A, van Kampen S, McCay C, Leahy M, Bijlsma M, Bonacina S, Bonten T, Bonthuis MJ, Butterlin A, Cobbaert K, Duijnhoven T, Hallensleben C, Harrison S, Hastenteufel M, Holappa T, Kokx B, Morlion B, Pauli N, Ploeg F, Salmon M, Schnoor K, Sharp M, Sottile PA, Värri A, Williams P, Heidenreich G, Oughtibridge N, Stegwee R, Chavannes N
What Makes a Quality Health App—Developing a Global Research-Based Health App Quality Assessment Framework for CEN-ISO/TS 82304-2: Delphi Study