Accepted for/Published in: JMIR Formative Research
Date Submitted: Jul 3, 2025
Date Accepted: Nov 17, 2025
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Development of an Evaluation Index System for Health Recommender Systems Based on the Health Technology Assessment Framework
ABSTRACT
Background:
Health Recommender Systems (HRSs) are specialized digital platforms or software applications designed to recommend personalized information, resources or interventions relevant to the user’s specific health needs. Establishing an evaluation indicator system for HRSs helps developers optimize the design and functionality of the applications, while also providing a scientific basis for their assessment.
Objective:
This study was conducted to develop an evaluation index system for HRSs.
Methods:
With reference to the health technology assessment framework, the evaluation index system structure and entries of the HRS were constructed; and through two rounds of Delphi expert consultation, the evaluation index system of the HRS is determined; the Analytic Hierarchy Process (AHP) method was performed to identify the structured indexes and their index weights.
Results:
A total of 18 experts participated in two rounds of the Delphi survey. Based on a literature review, the study was conducted using the Health Technology Assessment (HTA) model. This study preliminarily determined the content of evaluation index system of HRS and formed the expert consultation table containing 5 dimensions and 16 entries. Two rounds of correspondence consultation of the expert positivity were 100% and 88.89%, the expert authority coefficient was 0.883 and 0.884, Kendall coefficient was 0.236~0.360. The final evaluation index system of HRS contained 5 first-level indicators (performance, effectiveness, safety, economy and social appropriateness), and 18 second-level indicators.
Conclusions:
In this study, we constructed an evaluation index system for HRS that includes containing 5 dimensions (effectiveness, safety, economy, and social suitability) and 18 indicators. This tool could provide a scientific and effective tool for the research and development, application evaluation, and promotion of this technology. Clinical Trial: Not applicable.
Citation
