Previously submitted to: JMIR Mental Health (no longer under consideration since Feb 19, 2025)
Date Submitted: Feb 17, 2025
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Validation and selection of criteria for evaluating apps for managing depression: a Delphi study
ABSTRACT
Background:
Despite a lack of evidence relating to effectiveness and safety, the use of apps in the field of mental health is increasing due to their ease of use and accessibility. The aim of the EvalDepApps project is to develop and validate an assessment tool for evaluating depression management apps based on scientific evidence, expert judgment, and end-user needs.
Objective:
The purpose of this study was to determine the most relevant criteria for evaluating apps intended to manage depression through consensus-based assessment.
Methods:
A total of 43 individuals (22 health professionals, 4 health technology experts, and 17 individuals diagnosed with depression) were invited to participate in an online modified Delphi study of 51 criteria identified from the literature. In Round 1, participants rated criteria according to relevance (0="Not at all important" to 6="Very important") and 3 levels of consensus were defined: high level when ≥ 80% of respondents scored the criterium at 5 or 6; medium when 60-79% of respondents scored the criterium at 5 or 6; and low when < 60% of respondents scored the criterium at 5 or 6. In Round 2, participants were asked to re-rate criteria that had achieved the medium level of consensus in Round 1 under the same parameters. Comments by participants were collected and analyzed. The final assessment list consisted of those criteria that had attained the maximum consensus in either Round.
Results:
The response rate was 59.0% (26/43) in Round 1 and 53.4% (23/43) in Round 2. In Round 1, 24 criteria (47.1%) attained the maximum level of consensus, 20 (39.2%) the medium level, and 7 (13.7%) the lowest level. In Round 2, 4 out of 20 criteria (20.0%) attained the maximum consensus. Participant comments reinforced the relevance of the selected criteria. The final list consisted of 28 criteria, the majority relating to Safety and Privacy, and Clinical Effectiveness (25.0% each), followed by Usability and Functionality (17.9%).
Conclusions:
By prioritizing criteria relating to data security and clinical effectiveness, participants in this study emphasized that the assessment of apps for depression management must take both these aspects into full consideration. The results of this study will enable the EvalDepApps project to develop an assessment tool for depression management apps that incorporates the most relevant criteria.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.