JMIR Publications

ABSTRACT

Background:

Early risk stratification in emergency medical services (EMS) is essential for patients presenting with acute cardiopulmonary symptoms, yet prehospital decision-making at the dispatch stage is often based on limited structured information. Free-text dispatch narratives may contain additional clinical signals, but their role in early risk assessment remains insufficiently characterized. This study aims to evaluate whether NLP-driven models can support real-time decision-making in emergency dispatch systems.

Objective:

To develop and temporally validate a natural language processing–based machine learning model using emergency dispatch narratives for early risk stratification, and to assess its potential as an AI-enabled decision-support tool in real-time prehospital care.

Methods:

We conducted a population-based retrospective cohort study using EMS dispatch records from Nanning, China, between 2021 and 2025. Adult patients with suspected cardiopulmonary symptoms were identified based on predefined complaint keywords. After excluding non-medical and incomplete records, 38,523 cases with available free-text narratives were included. To simulate real-world deployment, data from 2021–2024 (n = 28,332) were used for model development, and 2025 data (n = 10,191) served as an independent temporal test cohort. Dispatch narratives were processed using a natural language processing (NLP) pipeline based on character-level n-grams, and combined with structured variables (age, sex, call time) in a multimodal machine learning framework. The primary outcome was a composite of prehospital critical events, including death, clinical deterioration, or lack of response to initial treatment. Model performance was evaluated using area under the receiver operating characteristic curve (AUROC), area under the precision–recall curve (AUPRC), calibration, and decision curve analysis. The modeling framework was designed to simulate real-world deployment using temporally separated training and testing cohorts.

Results:

Structured variables alone showed limited discrimination (AUROC 0.563). Incorporation of narrative features improved performance (AUROC 0.803), with marginal additional gain from multimodal integration (AUROC 0.808, 95% CI 0.798–0.818). In the temporally independent 2025 cohort, the multimodal model achieved an AUPRC of 0.630, exceeding the baseline outcome prevalence (24.2%). Model performance remained consistent across age and sex subgroups, and calibration was acceptable (Brier score 0.1878). In a risk enrichment analysis, the top 10% of predicted high-risk cases accounted for 32.1% of all critical outcomes. Decision curve analysis indicated a higher net clinical benefit compared with treat-all or treat-none strategies across a range of threshold probabilities.

Conclusions:

Free-text dispatch narratives contain clinically relevant information associated with early risk stratification in patients with suspected cardiopulmonary emergencies. Incorporating narrative-derived features into a structured modeling framework may complement existing dispatch systems and support more informed decision-making prior to patient contact. This approach provides a scalable framework for integrating unstructured dispatch data into AI-enabled decision support systems, with potential to improve early triage and resource allocation in emergency care.