Previously submitted to: Journal of Medical Internet Research (no longer under consideration since Dec 05, 2023)
Date Submitted: Mar 15, 2023
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
MEDHI -Tool - Demystifying the WHO framework For Monitoring and Evaluation of Digital Health Interventions
ABSTRACT
Background:
Digital health interventions (DHI), such as conversational agents and chatbots, play a crucial role for health support, promotion and rehabilitation. As any other health interventions, DHIs require monitoring and evaluation (M&E). However, the current evaluation process lacks a standardized approach and necessitates the use of multiple strategies to ensure comprehensiveness. In 2016, the World Health Organization (WHO) proposed a seven-step practical guide for M&E DHIs, to conducting research and assessment, that became the main and most comprehensive reference in this context. Despite its importance, the guide is complex and lacks decision support tools, making its application a challenging task.
Objective:
The aim of this study was to create the MEDHI-Tool to support the use of the WHO M&E guideline for DHIs.
Methods:
This is applied methodological research, carried out in 2 phases: development and evaluation of the MEDHI-Tool; and application of this tool to M&E, a chatbot for guidance, monitoring and stroke tracking, recently made available to the public and which is being used by people from an association of patients with this disease. The development was carried out in Excel and its content was based on the consolidation of the information present among the chapters of the guide, which were classified into categories (description, objective and recommendations) and annotated in tables, to serve as a basis for creating the filters. The product was evaluated by experts for organization, relevance, understanding and applicability in M&E.
Results:
The MEDHI-Tool has four tabs composed and follow by the sequence of steps of the WHO guide: 1 and 4 (first tab); 2; 3; 5 to 7 (fourth tab). The tool provides a standard of guidelines to be applied to all maturity cycles of any DHI profile. The evaluation of the tool yielded a favorable response regarding each analyzed criterion. In the case study, the MEDHI-Tool proved to be a valuable resource in guiding decisions on appropriate metrics, objectives, study types, data collection, and indicators.
Conclusions:
It was proposed the development of a tool to guide the M&E of DHI such as chatbots. The MEDHI- Tool has proven to be useful in providing evidence for planning processes, ensuring effective analysis of data generated and improving the delivery of products provided by DHI.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.