JMIR Publications

ABSTRACT

Background:

While vaping is considered to be safer than smoking tobacco and can help with successful smoking cessation, there is rising concern that it increases addiction among non-smokers, especially adolescents. Twitter presents a relevant, valuable, and feasible social media platform to study perspectives related to vaping.

Objective:

Our objective is to use Twitter to assess key factors such as sentiment, marketing, procurement, health effects, and policy that will provide unique perspectives related to vaping. We investigated both traditional and deep learning methods to develop classifiers that can identify vaping relevant tweets, commercial tweets, and tweets with pro and anti-vaping sentiments.

Methods:

We continuously collected tweets that matched vaping-related keywords over a seven-month period. From this corpus of tweets, we annotated a randomly selected set of 4,000 tweets for relevance (vape relevant or not), commercial (commercial or not), and sentiment (pro-vape or not). We evaluated several traditional classification methods including logistic regression (LR), random forest (RF), linear support vector machine (SVM), and multinomial naive Bayes (NB). We also evaluated several deep learning methods including convolutional neural network (CNN), long short-term memory (LSTM) network, LSTM-CNN network, and bidirectional LSTM (BiLSTM) network. The deep learning methods employed two types of pre-trained vectors including one that was vaping-specific and another that was not.

Results:

LSTM-CNN performed the best with the highest area under the ROC curve (AUC) score of 0.96, 95% C.I. [0.93-0.98] for relevance; all deep leaning methods including LSTM-CNN performed the best with AUCs of 0.99, 95% C.I. [0.98-0.99] for distinguishing commercial from non-commercial tweets; and BiLSTM performed the with AUC of 0.83, 95% C.I. [0.78-0.89]. Overall, LSTM-CNN performed the best for all three classification tasks.

Conclusions:

We developed and evaluated machine learning classifiers to classify vaping related tweets by relevance, commercial and sentiment polarity. We evaluated both traditional and deep learning classifiers. Overall, deep learning classifiers like LSTM-CNN had superior performance and had the added advantage of requiring no preprocessing. The performance of these classifiers support the development of automated vaping surveillance applications.