JMIR Publications

ABSTRACT

Background:

The decisions that individuals make about the food and beverage products they purchase and consume directly influence their energy intake and dietary quality, and may lead to excess weight gain and obesity. However, gathering and interpreting data on food and beverage purchase patterns can be difficult. Leveraging novel sources of data on food and beverage purchase behavior can provide us with more objective understandings of food consumption behaviors.

Objective:

Food and beverage purchase receipts often include time-stamped location information, which, when associated with product purchase details, can provide a useful behavioral measurement tool. The purpose of this study was to assess the feasibility, reliability, and validity of processing data from fast food restaurant receipts using crowdsourcing via Amazon Mechanical Turk (MTurk).

Methods:

Between 2013 and 2014, receipts (N=12,165) from consumer purchases were collected at 60 different location of five fast food restaurant chains in New Jersey and New York City, i.e., Burger King, KFC, McDonald’s, Subway, and Wendy’s. Data containing the restaurant name, location, receipt ID, food items purchased, price, and other information were manually entered into an MS Access database and checked for accuracy by a second reviewer; this was considered the “gold standard”. To assess the feasibility of coding receipt data via MTurk, a prototype set of receipts (N=196) was selected. For each receipt, five turkers were asked to 1) identify the name of the restaurant, and 2) indicate whether a beverage was listed in the receipt and, if yes, categorize the beverage as cold (e.g., soda, energy drink) or hot (e.g., coffee, tea). Inter-Turker agreement for specific questions (e.g.., restaurant name, beverage inclusion), and agreement between Turk consensus responses and the “gold standard” values in the manually entered data set were calculated.

Results:

Among the 196 receipts completed by turkers, the inter-turker agreement was 100% for restaurant names (e.g., Burger King, McDonald’s, Subway), 98.5% for beverage inclusion (i.e., hot, cold, or none), 92.3% for types of hot beverage (e.g., hot coffee, hot tea), and 87.0% for types of cold beverage (e.g., coke, bottled water), respectively. When compared with the “gold standard” data, the agreement level was 100.0% for restaurant name, 99.5% for beverage inclusion, and 99.5% for beverage types.

Conclusions:

Our findings indicated high inter-rater agreement for questions across difficulty levels, e.g., single vs. binary vs. multiple choice items. As a complement to traditional methods for coding receipt data, MTurk can produce excellent-quality data in a lower-cost, more time-efficient manner.