JMIR Publications

ABSTRACT

Background:

While web-based psycho-educational programs may be an efficient, accessible, and scalable option for improving participant well-being they seldom are sustained beyond trial publication. Implementation evaluations may help optimize program uptake, but few are performed. When the United States Department of Veterans Affairs (VA) launched the online psycho-educational workshop Building Better Caregivers (BBC) for informal caregivers of Veterans nationwide in 2013, the workshop did not enroll as many caregivers as anticipated.

Objective:

To identify initial implementation strengths and weaknesses, strategies likely to improve workshop uptake, whether the VA adopted these strategies, and whether workshop enrollment changed.

Methods:

We used mixed methods and the Promoting Action on Research Implementation in Health Services (PARIHS) implementation evaluation framework. In Stage 1 we conducted semi-structured interviews with caregivers, local staff, and regional and national VA leaders and surveys with caregivers and staff. We collected and analyzed survey and interview data concurrently and integrated results to identify implementation strengths and weakness and strategies likely to improve workshop uptake. In Stage 2 we re-interviewed national leaders to determine whether the VA adopted recommended strategies and used national VA data to determine whether annual workshop enrollment changed over time.

Results:

We completed 54 interviews with caregivers (n=32), staff (n=13), and regional (n=5) and national (n=4) leaders and received survey responses from 23 (72%) caregivers and 10 (77%) local staff. In Stage 1, survey and interview results were consistent across multiple PARIHS constructs. Whereas participants from low enrollment centers reported fewer implementation strengths and a greater number of weaknesses than those from high enrollment centers, qualitative themes were consistent across high and low enrollment centers and across caregiver, staff, and leadership respondent groups. Identified strengths included belief in a positive workshop impact and use of some successful outreach approaches for caregivers. Implementation weaknesses included multiple missed opportunities to improve outreach and to better support local staff. From these, we identified and recommended new or enhanced implementation strategies—increased investment in outreach and marketing capabilities, tailoring of outreach strategies to multiple stakeholder groups, use of campaigns that are personal, repeated, detailed and have diverse delivery options, recurrent training and mentoring for new staff, and comprehensive data management and reporting capabilities. In Stage 2, we determined that the VA had adopted several of these strategies in 2016. In the years before and after adoption cumulative BBC enrollment increased from 2,139 caregivers (2013-2015) to 4,030 (2016-2018).

Conclusions:

This study expands the limited implementation science literature on best practices to use when implementing web-based psycho-educational programs. We found that robust outreach and marketing strategies were critical to implementation success of the BBC workshop. After the VA adopted expanded strategies, workshop enrollment doubled. Other health systems may want to deploy these strategies when implementing web-based programs of their own.