JMIR Publications

ABSTRACT

Background:

Collecting information on adverse events following immunization from as many sources as possible is critical for prompt identification of any potential safety concerns, so that appropriate actions can be taken. Febrile convulsions are recognized as an important potential reaction to vaccination in children under the age of 6.

Objective:

The primary aim of this study is to evaluate natural language processing (NLP) techniques and machine learning models for the rapid detection of febrile convulsion presentations in emergency departments Additionally, we examine the deployment requirements for a machine learning model to perform real-time monitoring of emergency department triage notes.

Methods:

We developed a pattern matching approach as a baseline and evaluated several machine learning models for the classification of febrile convulsions in emergency department (ED triage notes, to determine both their training requirements and their effectiveness in detecting febrile convulsions. We measured their performance during training and then compared the deployed models’ result on new incoming ED data.

Results:

Although the best standard neural networks had acceptable performance and were low resource models, transformer-based models outperformed them significantly, justifying their ongoing deployment.

Conclusions:

Employing NLP, particularly with the utilization of large language models, offers significant advantages in syndromic surveillance. Large language models make highly effective classifiers, and their text generation capacity can be used to enhance the quality and diversity of training data.