JMIR Publications

ABSTRACT

Background:

Health apps are increasingly being used to promote health, manage diseases, and deliver healthcare services. Still, there is scarce objective information regarding their quality beyond the required CE mark for medical apps, leading to potential risks for users. To address these challenges, several authorities have developed health app assessment frameworks. In 2017, TIC Salut Social Foundation (FTSS) in Catalonia developed its own health app assessment framework, in use since that year. The publication of CEN ISO/TS 82304-2 (abbreviated as 82304-2), an international standard for assessing health apps, and the co-creation of the related app assessment handbook and scheme provide a unique opportunity to harmonize app assessments across the EU.

Objective:

This study aimed to perform a comparative analysis of the FTSS assessment framework with 82304-2 and explore the integration of 82304-2 in Catalonia. The goal was to identify shared concepts and context-specific concepts. Our broader aim was to provide this methodology for health authorities in other regions and countries to consider a similar integration.

Methods:

For the comparative analysis, a mixed-method approach was used, combining a qualitative case study with a quantitative analysis of the two frameworks. The analysis covered five key areas: framework characteristics, governance, workflows, quality aspects, and quality requirements (criteria). To support a quantitative analysis, the 120 FTSS requirements and 74 quality aspect-related 82304-2 requirements were translated into 97 concepts. A scoring system was applied to identify matches of the frameworks with these concepts, with scores ranging from 0 (no match) to 1 (full match). Integration was evaluated considering a series of scenarios, including using the 82304-2 related assessment scheme, adopting 82304-2 requirements, adapting the requirements to local needs, and maintaining the current FTSS framework.

Results:

The main difference between the frameworks was its app usage-based assessment (FTSS) versus evidence- and app usage-based assessment (82304-2). Overall, a 48% (47/97) match between the frameworks' concepts was found, with an additional 39% (37.5/97) specific to 82304-2 and 13% (12.5/97) to FTSS. The 82304-2 only concepts were found to be relevant to FTSS. Thus, FTSS decided to integrate all eighty-one 82304-2 requirements. Five FTSS-specific requirements were included in the app assessment handbook for 82304-2, while another 4 rigor-enhancing, 1 scope-expanding and 1 context-specific requirement will be assessed on top.

Conclusions:

The integration of 82304-2 in Catalonia offers a more structured, comprehensive, standardized approach to health app assessment. Once the 82304-2 assessment framework is put in practice, and 82304-2 quality labels are issued, the evolved handbook will achieve 93% (81/87) of the FTSS assessment, reducing the assessment workload specific to Catalonia to only 7% (6/87). FTSS encourages other authorities to perform a similar process or to wait until the L2E handbook is in practice to adopt it in full.