JMIR Publications

ABSTRACT

Background:

Internet interventions are able to easily generate objective data about program usage. Increasingly, more studies are exploring the relationship between usage and outcomes but they often report different metrics of use and the findings are mixed. Thus, current evaluations fail to demonstrate which metrics should be considered and if it is possible to determine an optimal dose-response relationship which can inform thresholds for adherence and clinically meaningful change.

Objective:

This study aimed to explore the relationship between several usage metrics and outcomes; and determine an optimal dose of usage of an internet intervention for depression.

Methods:

This is a secondary analysis of data from a Randomized Controlled Trial that examined the efficacy of an internet-based Cognitive-Behavioral Therapy (iCBT) program for depression (Space from Depression) in an adult community sample. Space from Depression is a seven-module, supported intervention delivered over a period of 8 weeks. Supporters were trained volunteers who provided feedback to participants on a weekly basis. Different usage metrics (i.e. time spent, modules and activities completed, percentage of program completion) were automatically collected by the platform and composite variables from these (e.g. activities per session) were computed. A breakdown of the usage metrics was obtained by weeks. For the analysis, the sample was divided into those who obtained a reliable change (RC) (Beck Depression Inventory [BDI-II] change >8) and those who did not.

Results:

Data from 216 users who used the intervention and completed pre and post-treatment outcomes were included in the analyses. 89 participants obtained a RC and 127 did not. Those in the RC group significantly spent more time, had more logins, used more tools, viewed a higher percentage of the program and got more reviews from the supporter compared to those who did not obtain a RC. Differences between groups in usage was observed from first week in advance across the different metrics although they vanished over time. In the RC group, the usage was higher during the first four weeks and then a significant decrease was observed. ROC Curve analyses showed that the optimal cut-points for different usage metrics were 7 hours total time spent, 15 sessions, 30 tools used and 50% of program completion.

Conclusions:

Overall, the results showed that those individuals who obtained RC after the intervention had higher levels of exposure to the platform. The usage during the first half of the intervention was higher and differences between groups were observed from the first week. This study also suggests that it is possible to determine an optimal dose and this can be used to inform the minimal usage to establish adherence. These results will help to better understand how to use internet interventions and what optimal level of engagement can most affect outcomes. Clinical Trial: The trial is registered as a controlled trial with ISRCTN (ISRCTN03704676).