JMIR Publications

ABSTRACT

Background:

With a rapidly evolving tobacco retail environment, it is increasingly necessary to understand the point of sale (POS) advertising environment as part of tobacco surveillance and control. Advances in machine learning and image processing suggest the ability for more efficient and more nuanced data capture than previously available.

Objective:

To employ machine learning algorithms to discover both the presence of tobacco advertising in photographs of tobacco POS advertising, as well as their location in the photograph.

Methods:

We first collected images of the interiors of tobacco retailers in West Virginia and the District of Columbia during 2016 and 2018. The clearest photos were selected and used to create a training and test data set. We then used a pre-trained image classification network model, Inception V3,to discover the presence of tobacco logos, as well as a unified object detection system, You Only Look Once (YOLO), to identify logo locations.

Results:

Our model was successful in identifying the presence of advertising within images, with a classification accuracy of over 75% for 8 of the 42 brands. Discovering the location of logos within a given photo was more challenging due to the relatively small training data set, resulting in a mean Average Precision (mAP) score of 72% and Intersection over Union (IOU) of 62%.

Conclusions:

Our research provides evidence for a novel methodological approach that tobacco researchers and other public health practitioners can apply in the collection and processing of data for tobacco or other POS surveillance efforts. The resulting surveillance information can inform policy adoption, implementation, and enforcement. Limitations notwithstanding, our analysis shows the promise of using machine learning as part of a suite of tools to understand the tobacco retail environment, make policy recommendations, and design public health interventions at the municipal or other jurisdictional scale.