JMIR Publications

ABSTRACT

Background:

The healthcare polycrisis in Canada comprises several concurrent crises namely workforce shortages; food insecurity; housing stress; and health inequities experienced by indigenous peoples, low-income families, and racialized communities. There is a need for a scalable digital surveillance system to monitor these related but distinct health system crises concurrently using Natural Language Processing (NLP), as well as statistical forecasting techniques.

Objective:

The objective of this study will be to utilize a multi-method NLP pipeline to understand discourse dynamics, causal interdependencies between crisis themes, and how responsive policy-making process are to both the discourse and causal interdependencies between crisis theme in Canada’s healthcare system from 2020-2025.

Methods:

We used an NLP pipeline to examine the language and sentiment of 6757 online news headlines regarding Canada’s health care system over the course of approximately 5 years (2020-2025). This included utilizing two different methods for analyzing sentiment: a pre-trained BERT model for fine-tuning with a test accuracy of 0.97, and VADER (a valence-aware dictionary and sEntiment Reasoner) for analyzing valence and subjectivity. Further, we utilized latent dirichlet allocation (LDA) to identify topics or themes within the online news headlines. Then, we utilized time series analysis (tsa) to predict trends in each topic/theme utilizing both ARIMA (AutoRegressive Integrated Moving Average) and ets (error, trend, and seasonality) methods. Finally, we used granger causality testing to determine if past values of one theme could be used to predict future values of another theme.

Results:

Our results indicated that the BERT model achieved a test accuracy of 0.97 when applied to the task of identifying sentiment in the online news headline data set. Our results also indicated that seven major discourse topics (themes) emerged from our analysis. Of those seven themes, The Food Insecurity theme was dominant (accounted for 17.8% of all articles written about it). Furthermore, our results showed strong statistical evidence (f = 17.361; p-value adjusted for multiple comparisons using the FDR = 0.002) that changes in the healthcare system crises were related to changes in discussions regarding housing. Also, our results showed no statistically significant change in sentiment after any policy intervention was implemented based upon our sentiment analysis results (Cohen's d < 0.2). Our forecasting results did indicate that all five themes would continue to grow through 2026.

Conclusions:

These findings illustrate the complexity of issues currently facing Canada and suggest that policymakers should consider implementing digital surveillance systems based upon NLP to monitor and analyze complex relationships between these multiple crises. Further, our findings suggest that there is a persistent policy perception gap with significant implications for designing health informatics surveillance systems.