Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Accepted for/Published in: JMIR Infodemiology

Date Submitted: Jan 28, 2022
Date Accepted: Aug 15, 2022

The final, peer-reviewed published version of this preprint can be found here:

Codeveloping and Evaluating a Campaign to Reduce Dementia Misconceptions on Twitter: Machine Learning Study

Erturk SZ, Hudson G, Jansli SM, Morris D, Odoi CM, Wilson E, Clayton-Turner A, Bray V, Yourston G, Cornwall A, Cummins N, Wykes T, Jilka S

Codeveloping and Evaluating a Campaign to Reduce Dementia Misconceptions on Twitter: Machine Learning Study

JMIR Infodemiology 2022;2(2):e36871

DOI: 10.2196/36871

PMID: 37113444

PMCID: 9987190

Co-developing and evaluating a campaign to reduce dementia misconceptions on Twitter: Machine Learning Study

  • Sinan Zachary Erturk; 
  • Georgie Hudson; 
  • Sonja M Jansli; 
  • Daniel Morris; 
  • Clarissa M Odoi; 
  • Emma Wilson; 
  • Angela Clayton-Turner; 
  • Vanessa Bray; 
  • Gill Yourston; 
  • Andrew Cornwall; 
  • Nicholas Cummins; 
  • Til Wykes; 
  • Sagar Jilka

ABSTRACT

Background:

Dementia misconceptions are common and harmful on Twitter. Machine learning (ML) models co-developed with carers provide a method to identify these and help in evaluating awareness campaigns.

Objective:

To co-develop a machine learning model alongside carers which can identify misconceptions about dementia in tweets and use this to assess the performance of a dementia awareness campaign co-designed with carers.

Methods:

Carers rated dementia misconceptions in 1,414 tweets; these were used to build 4 ML models. Through a standard 80/20 train/test split, we evaluated the 4 models and performed a further blind validation with carers for the best 2 of these 4; from this blind validation we selected the best model overall. We analysed UK tweets (N7,124) to investigate how events influence misconception prevalence. We co-developed an awareness campaign and collected pre-and-post campaign tweets (N4,880), classifying them with our model as misconceptions or not.

Results:

A Random Forest model best identified misconceptions with an accuracy of 80% from blind validation and found 32% (N7124) of UK tweets about dementia were misconceptions. From this, we could track how the percentage of misconceptions changed in response to top UK news stories. Misconceptions significantly rose around particular political topics and were highest (68% of dementia tweets) when there was controversy over the UK government allowing hunting to continue during COVID-19. Misconceptions were lowest (10% of dementia tweets) when a famous UK actor died from Alzheimer’s disease. After our campaign, the use of misconception keywords such as ‘demented’ and ‘senile’ reduced by 9% in misconceptions (χ2 = 12.27, p < .001), but misconception prevalence did not reduce overall.

Conclusions:

ML provides an effective way to predict outcomes and track the impact of news stories in real time. Involvement of the community can support can improve ML performance.


 Citation

Please cite as:

Erturk SZ, Hudson G, Jansli SM, Morris D, Odoi CM, Wilson E, Clayton-Turner A, Bray V, Yourston G, Cornwall A, Cummins N, Wykes T, Jilka S

Codeveloping and Evaluating a Campaign to Reduce Dementia Misconceptions on Twitter: Machine Learning Study

JMIR Infodemiology 2022;2(2):e36871

DOI: 10.2196/36871

PMID: 37113444

PMCID: 9987190

Download PDF


Request queued. Please wait while the file is being generated. It may take some time.

© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.