JMIR Publications

ABSTRACT

Background:

Unintentional injury is the leading cause of death in young children. Emergency department (ED) diagnoses are a useful source of information for epidemiological surveillance. However, ED collection systems often employ free-text to report patient diagnoses. Automatic text classification using machine learning techniques (MLTs) is a potentially powerful tool to improve injury surveillance by speeding up coding tasks.

Objective:

This research aims to develop a tool for automatic free text classification of ED diagnoses to identify injury cases.

Methods:

The study includes 283,468 pediatric admissions between 2007 and 2018 to the emergency department of Padova University Hospital, a large referral center in northeast Italy. Each record reports a diagnosis of admission by free text. Free-text records are standard tools for reporting patient diagnoses. An expert pediatrician manually classified a randomly extracted sample of approximately 40,000 diagnoses to serve as the gold standard to train an MLT classifier. After preprocessing, a document-term matrix (DTM) was created. The machine learning classifiers, i.e. decision tree (DT), random forest (RF), gradient boosting (GBM), and support vector machine (SVM), were tuned by four-fold cross-validation. The injury diagnoses were classified into three hierarchical classification tasks: A) injury versus non-injury, B) intentional versus unintentional injury, and C) type of unintentional injury, according to the WHO classification of injuries.

Results:

The SVM classifier achieved the highest performance accuracy (94·14%) in classifying injury vs. non-injury cases (Task A). The GBM method produced the best results (92% accuracy) for the unintentional/intentional injury classification task (Task B). The highest accuracy for the unintentional injury subclassification (Task C) was achieved by the SVM classifier. In the different tasks, the SVM, RF, and GBM algorithms performed similarly against the gold standard.

Conclusions:

Automatic classification of injury data from uncoded patients’ records is feasible and accurate, and this represents a major step toward children’s injuries surveillance.