JMIR Publications

ABSTRACT

Background:

Personal narratives dominate health communication on social media platforms, integrating medical information with lived experiences and emotional framing. However, conventional computational systems exhibit "narrative blindness" which is an inability to process the causal logic and interpretive dimensions of health narratives, limiting their effectiveness in assessing information credibility and detecting potentially harmful health misinformation being shared in cyberspace.

Objective:

This study introduces and validates the Computational Narrative Builder, a novel multi-task deep learning model that reconstructs unstructured online health narratives into structured, machine-readable representations. By operationalizing Labovian sociolinguistic narrative theory through Agent-Action-Outcome (AAO) triplets enriched with interpretive annotations, we ask: Can transformer-based AAO models achieve superior recall and interpretability versus traditional approaches?

Methods:

We operationalized Labov and Waletzky's six-stage narrative framework into a computational AAO schema, representing narrative events as structured triplets: Agent (actor), Action (intervention), and Outcome (result). From 5253 Reddit health threads with 185,181 unique comments and 2,295,242 words collected across six health-focused subreddits (February 2019–November 2024), we created a gold-standard dataset through expert annotation of 1000 posts (2000 narrative segments), achieving substantial inter-annotator agreement (Fleiss κ=0.76 for narrative segmentation, κ=0.81 for AAO extraction at span level). We developed a multi-task DistilBERT architecture with five parallel task heads: AAO extraction (using Conditional Random Fields), narrative segmentation (six Labovian stages), emotion classification (8 classes), stance classification (5 classes), and tone classification (6 classes). The model was evaluated using 5-fold stratified cross-validation with bootstrap resampling (n=1000) against TF-IDF and single-task BERT baselines.

Results:

The Computational Narrative Builder AAO span extraction achieved micro-averaged F1 = 0.76 (exact-match), outperforming a single-task BERT baseline (0.71; P = .008) and a TF-IDF baseline (0.60; P<.001). Overlap-tolerant scoring yielded micro-F1 = 0.87 (IoU ≥ 0.5). Component-wise F1 was 0.78 (actions), 0.75 (outcomes), and 0.74 (agents). Exact-match triplet accuracy = 0.57. Narrative stage classification reached micro-F1 = 0.84. Interpretive classification was robust: emotion (macro F1 = 0.80), stance (0.75), and tone (0.68). Multi-task learning produced significant gains vs single-task (+4 percentage points for AAO, P = .008; +3 points for segmentation, P = .008). Processing averaged 4.35 narratives/second.

Conclusions:

The Computational Narrative Builder successfully addresses narrative blindness by providing a validated, efficient method for translating unstructured health narratives into structured representations. This enables systematic analysis of health narratives at scale, establishing foundations for advanced applications in credibility assessment and public health surveillance. While validated on English-language Reddit data, the framework's theoretical grounding suggests potential cross-platform generalizability, pending future validation.