Previously submitted to: JMIR Public Health and Surveillance (no longer under consideration since Aug 24, 2023)
Date Submitted: May 9, 2023
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Survice-BERT: A BERT (Bidirectional Encoder Representations from Transformers) Model for Biomedical Named Entity Recognition in Disease Surveillance Reports
ABSTRACT
Background:
Since the emergence of the novel coronavirus disease (COVID-19), it is becoming increasingly important to predict infectious diseases to prevent their spread. Researchers have conducted studies on forecasting infectious diseases to control post pandemics. However, the lack of infectious disease surveillance datasets and the difficulty of monitoring outbreaks obstruct the research development.
Objective:
This study aims to create language model which can extract disease outbreak information from periodic reports of infectious diseases. We present a developing approach about our NER (Named Entity Recognition) datasets based on infectious disease surveillance reports and our Survice-BERT (Bidirectional Encoder Representations from Transformers) model fine-tuned by our datasets for immediate pandemics response.
Methods:
We built our datasets by crawling and pre-processing infectious disease surveillance reports. The structure of datasets consist of 7000 sentences and eight NER tags (Year, Month, Week, Day, Location, Disease, Case, and Death of disease). We fine-tuned two BERT model among biomedical domain-specific pre-trained models. Totally, we experimented 40 times by differing batch sizes and learning rates. We evaluated F1-scores of models by NER tags and compared each classifier performance.
Results:
The proposed model achieved a high F1 average of 0.99. We demonstrated this model extracts data with high performance and remarkably improve research forecasting of infectious diseases outbreaks.
Conclusions:
It will help save lives from infectious diseases. The Survice-BERT model will be freely available on our GitHub.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.