Accepted for/Published in: Journal of Medical Internet Research
Date Submitted: Nov 29, 2018
Date Accepted: Mar 29, 2019
(closed for review but you can still tweet)
Modelling spatiotemporal factors associated with sentiment on Twitter: a synthesis and suggestions for improving the identification of localised deviations
ABSTRACT
Background:
Studies examining how sentiment on social media varies depending on timing and location appear to produce inconsistent results, making it hard to design systems that use sentiment to detect localised events for public health applications.
Objective:
Our aim was to measure how common timing and location confounders explain variation in sentiment on Twitter.
Methods:
Using a dataset of 16.54 million English-language tweets from 100 cities posted between 13 July and 30 November 2017, we estimated positive and negative sentiment for each of the cities using dictionary-based sentiment analysis and constructed models to explain differences in sentiment using time of day, day of week, weather, city, and interaction type (conversations or broadcasting) as factors, and found that all factors were independently associated with sentiment.
Results:
In the full multivariable model of positive (Pearson's R in test data 0.236; 95% CI 0.231-0.241), and negative (Pearson’s R in test data 0.306 95% CI 0.301-0.310) sentiment, the city and time of day explained more of the variance than weather and day of week. Models that account for these confounders produce a different distribution and ranking of important events compared to models that do not account for these confounders.
Conclusions:
In public health applications that aim to detect localised events by aggregating sentiment across populations of Twitter users, it is worthwhile accounting for baseline differences before looking for unexpected changes.
Citation
