JMIR Publications

ABSTRACT

Background:

Social media-based digital health surveillance, defined as the systematic collection and interpretation of user-generated data for the timely dissemination of findings so that action can be taken, has made substantial strides with the advent of artificial intelligence and natural language processing (NLP) methods that allow researchers to produce scalable insights from unstructured data. Multi-sector community health networks increasingly rely on these platforms to advance priorities such as obesity and health equity, yet common NLP methods in this space (e.g., topic modeling and sentiment analysis) characterize content by how often a topic is posted and collapse distinct communicative framings into single themes – producing minimal actionable insight and confirming what a network already produces rather than what resonates, and leaving engagement an underexploited signal.

Objective:

As part of our partnership with community networks – aimed at increasing the use of evidence-based policies to address obesity and health inequities – we leveraged Facebook and developed digital health surveillance methods that connect communication framing to audience engagement, generating actionable feedback for the networks.

Methods:

We collected 33,979 Facebook posts from 152 pages across four anonymized U.S. networks (two midwestern, two southeastern states) between February 1, 2022 and January 31, 2023. We proposed a two-stage framework that first used large language models to classify each post into one of four communication approaches – Individual Behavior; Policy or Environmental Approaches; Other Organizational or Program Information; Irrelevant – grounded in the socioecological model and the PSE framework. We then applied engagement-based topic modeling, with "likes" normalized across a sliding temporal window per network to account for temporal fluctuations and follower-base growth.

Results:

Engagement patterns and raw posting volume diverged in ways not predictable from posting alone. Three of the four networks exhibited at least one rank reversal between volume and engagement, localized to specific communication-approach categories rather than appearing uniformly. Additionally, engagement consistently aligned with each network's local audience composition rather than with topics alone. In a predominantly African American network, COVID-related and community-helping content dominated engagement; in a rural network anchored by a VA medical center, veteran support and insurance content led; and in a metropolitan academic and hospital network, health-care-policy content drew roughly 9.5× more engagement per post than annual reports despite far lower posting volume.

Conclusions:

What networks post most frequently is not necessarily what their audiences engage with most, and effective communication is fundamentally dependent on the local context of the specific community each network serves. By leveraging these findings as feedback to our community partners, this framework moves digital health surveillance beyond collecting and summarizing digital data toward translating it into timely, actionable, and translational feedback – ensuring findings reach those who have the right to know so that action can be taken.