JMIR Publications

ABSTRACT

Background:

Digital health interventions have demonstrated efficacy for a variety of conditions and are becoming better established and integrated within healthcare. However, the process of making them available to end-users in an efficient and sustained way is still challenging and remains a young area of research. In order to advance this field, comprehensive frameworks have been created.

Objective:

The aim of this study is to compare the Reach, Adoption, Implementation and Maintenance (RE-AIM) and Behavior Interventions using Technology (BIT) frameworks with data collected from the WebMAP Mobile (WMM) hybrid effectiveness-implementation randomized controlled trial.

Methods:

We conducted a hybrid effectiveness-implementation cluster randomized controlled trial with a stepped wedge design in which the intervention was sequentially implemented in 8 clinics, following a usual care period. Participants were 143 youth (mean age 14.5, 82% female) with chronic pain, from which 73 were randomized to receive active intervention. Implementation outcomes were assessed with the RE-AIM and BIT frameworks.

Results:

According to the RE-AIM framework, WMM showed excellent Reach, as indicated by recruiting a sample 19% larger than the size originally planned, consenting 79% of eligible referred adolescents (86% of whom rated the treatment as acceptable). Effectiveness was limited, with only global impression of change showing significantly greater improvements in the treatment group; however greater treatment engagement (number of modules completed) was associated with greater reductions in pain and disability. Adoption was excellent, as evidenced by all the invited clinics participating and referring patients to the study. Implementation at the user level was acceptable, showing good engagement and moderate adherence. Providers showed positive attitudes. Costs were similar to planned with a 7% increase in funds needed to make WMM publicly available. Maintenance at the user level was evidenced by 56 new patients downloading the app during the maintenance period. At the organization level, all clinics agreed to continue making referrals and, all but one, making new referrals. According to the BIT: 86% of adolescents considered it an acceptable treatment demonstrating strong acceptability. Adoption: 94% downloaded the app and all of them used it after their first log-in. Appropriateness at the user level: Two participants were unable to download the application. Perceptions of the appearance, navigation, and theme were positive. At the provider level, they perceived WMM as a good fit for their clinic, beneficial, helpful and resource-efficient. Feasibility: no technical issues were reported. Fidelity: 40% completed the treatment. Implementation costs: 7% above the budget. Penetration: 56 new users accessed the app during the maintenance period. Sustainability: 78% of clinics continued recommending WMM after the end of the study.

Conclusions:

For the first time, a real-world digital health intervention was used as a proof of concept to test all the domains in the RE-AIM and BIT frameworks, allowing for comparisons. Clinical Trial: ClinicalTrials.gov NCT03332563