JMIR Publications

ABSTRACT

Background:

Public health authorities (PHAs) have been recommending interventions such as physical distancing and face masks, to curtail the transmission of coronavirus disease (COVID-19) within the community. Public perceptions towards such interventions are to be identified so that PHAs can effectively address valid concerns. The Health Belief Model (HBM) has been used to characterize user-generated content from social media during previous outbreaks, to understand health behaviors of people.

Objective:

This study is aimed at developing and evaluating deep learning-based text classification models for classifying social media content posted during the COVID-19 outbreak, using the key four constructs of HBM. We specifically focus on content related to the physical distancing interventions put forth by PHAs. We intend to test the model with a real-world case study.

Methods:

The dataset for this study was prepared by analyzing Facebook comments which were posted by the public in response to the COVID-19 posts of three PHAs: Ministry of Health of Singapore (MOH), Centers for Disease Control and Prevention (CDC) and Public Health England (PHE). The comments made in the context of physical distancing were manually classified with a Yes/No flag for each of the four HBM constructs: perceived severity, perceived susceptibility, perceived barriers, and perceived benefits. Using a curated dataset of 16,752 comments, a gated recurrent unit (GRU) based recurrent neural network (RNN) model was trained for text classification. Accuracy and binary cross-entropy loss were used for evaluating the model while specificity, sensitivity and balanced accuracy were the metrics used for evaluating the classification results in the MOH case study.

Results:

The HBM text classification models achieved mean accuracy rates of 0.92, 0.95, 0.91 and 0.94 for the constructs perceived susceptibility, perceived severity, perceived benefits, and perceived barriers, respectively. In the case study with MOH FB comments, specificity was above 96% for all HBM constructs. Sensitivity was 94.3% and 90.9% for perceived severity and perceived benefits while for perceived susceptibility and perceived barriers, it was 79.6% and 81.5%. The classification models were able to accurately predict the trends in the prevalence of the constructs for the examined days in the case study.

Conclusions:

The deep learning-based text classifiers developed in this study help in getting an understanding of the public perceptions towards physical distancing, using the four key constructs of HBM. Health officials can make use of the classification model to characterize health behaviors of public through the lens of social media. In future studies, we intend to extend the model for studying public perceptions on other important interventions of PHAs.