JMIR Publications

ABSTRACT

Background:

Digital mental health interventions are being utilized more than ever for the prevention and treatment of psychological problems. Optimizing the implementation aspects of digital mental health is essential to deliver the program to populations in need, but there is a lack of validated implementation outcome measures for digital mental health interventions.

Objective:

The primary aim of this study is to develop implementation outcome scales of digital mental health for different levels of stakeholders involved in the implementation process: users, providers, and managers or policymakers. The secondary aim is to conduct a psychometric assessment of a scale for users.

Methods:

We developed English and Japanese versions of the implementation outcome scales for digital mental health (iOSDMH) based on the literature review, and on panel discussions with experts in implementation research and web-based psychotherapy. The study developed acceptability, appropriateness, feasibility, satisfaction, and harm outcome measures for users, providers, and managers or policymakers, respectively. We conducted an exploratory factor analysis (EFA) to assess the structural validity of the iOSDMH for users using an online survey. Satisfaction, which consisted of a single item, was not included in the EFA.

Results:

The iOSDMH was developed for users, providers, and managers or policymakers. The iOSDMH contains 19 items for users, 11 items for providers, and 14 items for managers or policymakers. Cronbach's α coefficients indicated intermediate internal consistency for acceptability (α=.665), but well above for appropriateness (α=.776), feasibility (α=.832), and harm (α=.777) of the iOSDMH for users. An exploratory factor analysis revealed 3-factor structures, indicating acceptability and appropriateness as close concepts. Despite the similarity between these two concepts, we agreed that acceptability and appropriateness should be used as different factors, following previous studies.

Conclusions:

We developed implementation outcome scales of digital mental health among users, providers, and managers. Psychometric assessment of the scales for users demonstrated acceptable reliability and validity. Evaluating the components of the implementation of digital mental health is a big step forward in implementation science. Clinical Trial: N/A