JMIR Publications

ABSTRACT

Background:

Public health mass media campaigns are increasingly using digital media platforms, such as online advertising and social media; however there is a lack of evidence on how to best use these digital platforms for public health campaigns. To generate this evidence, appropriate campaign evaluations are needed, but with the proliferation of digital media-related metrics, there is no clear consensus on which evaluation metrics should be used. Public health campaigns are diverse in nature, so to facilitate analysis this review has selected tobacco control campaigns as the scope of study.

Objective:

This literature review examines how tobacco control campaigns that use traditional and digital media platforms have been evaluated.

Methods:

Medicine and science databases (Medline, Embase, PsychInfo, Cinahl and Scopus), and a marketing case study database (World Advertising Research Center) were searched for articles published between 2013 and 2018. Two authors established the eligibility criteria and reviewed articles for inclusion. Individual campaigns were identified from the articles, and information on campaigns and their evaluations were supplemented with searches on Google, Google Scholar, and social media platforms. Data about campaign evaluations were tabulated and then mapped to an evaluation logic model.

Results:

In total 17 campaigns were included in this review, reported on by 51 articles, 22 marketing reports and 7 grey literature reports. Most campaigns were from high-income, English-speaking countries, with behavioural change as the primary objective. In the process evaluations, a wide range of metrics were used to assess the reach of digital media campaign activities, making comparison between campaigns difficult. Every campaign in the review, except one, reported some type of ‘engagement’ impact measure, with number of website visits being the most commonly reported metric (11 of the 17 campaigns). Other commonly reported evaluation measures identified in this review included: engagement on social media, changes in attitudes, and number of people contacting smoking cessation services. Of note, only 7 of the 17 campaigns attempted to measure media platform attribution, i.e. where the audience encountered the campaign.

Conclusions:

One of the key findings of this review is the numerous and diverse range of measures and metrics used in tobacco control campaign evaluations. To address this issue, we propose a set of principles to guide selection of digital-related metrics for campaign evaluations, and also outline a logic model to provide a coherent organisation to the diverse range of metrics. Future research is needed to specifically investigate whether there is any association between engagement metrics and desired campaign outcomes, in order to determine whether reporting of engagement metrics is meaningful in campaign evaluations.