JMIR Publications

ABSTRACT

Background:

Robust and flexible infectious disease surveillance is crucial for public health. Event-based surveillance (EBS) was developed to allow timely detection of infectious disease outbreaks using mostly web-based data. Despite its widespread use, EBS has not been evaluated systematically on a global scale in terms of outbreak detection performance.

Objective:

To assess the variation in timing and frequency of EBS reports compared to true outbreaks and to identify the determinants of variability, using the example of seasonal influenza epidemics in 24 countries.

Methods:

We obtained influenza-related reports from two EBS systems, HealthMap and the WHO Epidemic Intelligence from Open Sources (EIOS), and weekly virologic influenza counts from FluNet as a gold standard. Epidemic influenza periods were detected based on report frequency using Bayesian change point analysis. Timely sensitivity, i.e., outbreak detection within the first two weeks before or after an outbreak onset, was calculated along with sensitivity, specificity, positive predictive value, and timeliness of detection. Linear regressions were performed to assess the influence of country-specific factors on EBS performance.

Results:

Overall, monitoring the frequency of EBS reports detected 73.5% of outbreaks, but only 9.2% within two weeks of onset; in the best case, monitoring the frequency of health-related reports identified 29% of outbreaks within two weeks of onset. We observed a large degree of variability in all evaluation metrics across countries. The number of EBS reports available within a country, the human development index, and the country’s geographical location partially explained this variability.

Conclusions:

Monitoring the frequency of EBS reports allowed just under 10% of seasonal influenza outbreaks to be detected in a timely manner in a worldwide analysis, with a large variability in detection capabilities. This article documents the global variation of EBS performance and demonstrates that monitoring report frequency alone in EBS may be insufficient for timely detection of outbreaks. Moreover, factors such as human development index and geographical location of a country may influence the performance of EBS and should be considered in future evaluations.