JMIR Publications

ABSTRACT

Digital health interventions (DHI`s) have the potential to improve health care and health promotion. However, there is a lack of guidance in the literature for the development, refinement, and prioritisation of Key Performance Indicators (KPI’s) for evaluation of DHI`s. This paper presents a 4-stage process used in the Gravitate Health project based on stakeholder consultation and consensus for this purpose. The Gravitate Health consortium comprises private and public partners from across Europe and the U.S, which is developing innovative digital health solutions in the form of Federated Open-Source Platform (FOSP) and G-Lens®, to present users with individualised digital information about their medicines. The 1st stage was the consultative process for the development of the KPI`s involved stakeholder (Gravitate Health project leads) consultations at the planning stages of the project which resulted in an extensive list of KPI’s organised into 7 categories. The 2nd stage was conducting a scoping review which confirmed the need for extensive stakeholder consultation in all stages of the KPI development, refinement, and prioritisation process. The 3rd stage was a period of further consultation with all consortium members which resulted in the elimination of 1 category of KPI’s. The 4th stage involved using the Delphi technique for refining and prioritising the remaining 6 categories of KPI’s. It is unusual to use this in a non-research exercise, but it provided a clear consultative framework and structure that facilitated the achievement of consensus within a large consortium of 250 members, on a substantial list of KPI’s for the project. Consortium members ranked the relevance and importance of each KPI. The final list of KPI`s provides substantial indicators sensitive to the needs of a broad group of stakeholders that are being used to capture real-world data in developing and evaluating DHI`s.