JMIR Publications

ABSTRACT

Background:

In 2021, European Union registered over 365,000 excess deaths, with over 16,000 excess deaths in Portugal. The Portuguese Directorate-General of Health (DGS) has developed a deep neural network – AUTOCOD – that codifies primary causes of death by analyzing the free text in the physicians' death certificates (DC). Although the performance of AUTOCOD has already been demonstrated, it was not clear if this performance was the same over time, especially during excess mortality periods.

Objective:

Determine the sensitivity of AUTOCOD for classifying the underlying cause of death compared with manual coding to ascertain the specific causes of death, in periods of excess mortality.

Methods:

We included all the DC between 2016 and 2019. We evaluated the performance of AUTOCOD through a confusion matrix, comparing ICD-10 classifications of DC by AUTOCOD with those from the human coders at DGS (gold-standard). Next, we compared the periods without excess mortality with periods of excess, severe and extreme excess mortality. Lastly, we repeated the analyses for the three most common ICD-10 chapters, targeting classification at the block level.

Results:

AUTOCOD showed high sensitivity (≥0.75) for ten ICD-10 chapters studied, with values above 0.90 for the more prevalent chapters (II – neoplasms; IX – diseases of the circulatory system; X – diseases of the respiratory system). These high sensitivity values show no significant differences when comparing the periods without excess mortality with periods of excess, severe and extreme excess mortality. When considering the ICD-10 block classification of the three most common ICD-10 chapters, AUTOCOD again performed well, showing high sensitivity (≥0.75) for 13 ICD-10 blocks, with no significant differences between periods without excess mortality and periods with excess mortality.

Conclusions:

Our results suggest that even during periods of excess and extreme excess mortality, the performance of AUTOCOD is not affected by a potential loss in text quality due to pressure on health services. Thus, AUTOCOD can be reliably used for real-time specific-cause mortality surveillance even in extreme excess mortality.