JMIR Publications

ABSTRACT

Background:

While eMental health interventions can have many potential benefits for mental healthcare, implementation outcomes are often disappointing. In order to improve these outcome, there is a need for a better understanding of complex, dynamic interactions between a broad range of implementation-related factors. These interactions and processes should be studied holistically, paying attention to context-, technology-, and people-related factors.

Objective:

The main objective of this mixed-method study is to holistically evaluate the implementation activities and outcomes of an eMental health intervention in an organization for forensic mental healthcare.

Methods:

First, desk research was performed on 18 documents on the implementation process. Second, the intervention’s use by 721 patients and 172 therapists was analyzed via log data. Third, semi-structured interviews were conducted with all 18 therapists of one outpatient clinic to identify a broad range of factors that influence implementation outcomes. The interviews were analyzed via a combination of deductive analysis using the NASSS Framework, and inductive, open coding.

Results:

The timeline that was generated via desk research showed that implementation activities mostly focused on technical skills training of therapists. Log data analyses demonstrated that 1019 modules were started, and 19% of all patients of the forensic hospital started at least one module. Of these patients, 18% completed at least one module. Of the therapists that had the possibility of using the module, 54% sent at least one feedback message to a patient. The median number of feedback messages sent per therapist was 1, with a minimum of 0 and a maximum of 460. The interviews showed that therapists did not always introduce the intervention to patients and that using the intervention was not part of their daily routine. Also, therapists indicated that patients often did not have the required conscientiousness and literacy levels. Furthermore, they had mixed opinions about the design of the intervention. Important organization-related factors were the need for more support and better integration in organizational structures. Finally, therapists stated that despite its current low use, the intervention has the potential to improve the quality of treatment.

Conclusions:

Synthesis of the different types of data showed that implementation outcomes were mostly disappointing. Implementation activities focused mostly on technical training of therapists, while little attention was paid to changes in the organization, design of the technology, and patients’ awareness. A more holistic approach towards implementation strategies - with more attention for the organization, patients, and technology and training of therapists - might have resulted in better implementation outcomes. Overall, adaptivity appears to be an important concept in eHealth implementation: a technology should be easily adaptable to an individual patient, therapists should be trained to able to deal flexibly with an eMental health intervention in their treatment, and organizations have to adapt their implementation strategies and structures to embed a new eHealth intervention.