JMIR Publications

ABSTRACT

Background:

Engagement with and potential impact of web-based interventions is often studied by unobtrusively tracking user behavior with web analytics. These metrics do provide insights into how users behave, but not why.

Objective:

This study demonstrates how a mixed-methods approach combining ‘how’ and ‘why’ can be used to gain insight into engagement with and potential impact of web-based interventions. The chlamydia page of the Dutch sexual health intervention Sense.info is used as a demonstration case.

Methods:

First, acyclic behavior change diagrams (ABCDs) were used to carry out a theoretical analysis of intended use of the chlamydia page. ABCDs display how behavior change principles are applied in an intervention and which sub-behaviors and target behaviors are (aimed to be) influenced. Second, actual use of the chlamydia page was analyzed with the web analytics tool Matomo, based on which preliminary assumptions about use and impact were formulated. Third, to further explore these assumptions, a study combining the think-aloud method and semi-structured interviews was executed with young individuals aged 16-25 (N = 15). Template analysis was used to analyze interview transcripts.

Results:

Participants perceived the Sense.info chlamydia page as reliable and would visit it mostly for self-diagnosis purposes. The page seemed reasonably motivate testing for sexually transmitted infections (STIs), but participants gave suggestions to lower the threshold.

Conclusions:

We conclude that a mixed-methods approach combining web analytics with a think-aloud method and semi-structured interviews was helpful in assessing engagement with and potential impact of Sense.info. This approach might also have potential for the evaluation of web-based interventions in general.