JMIR Publications

ABSTRACT

Background:

Early recognition of severely injured patients in prehospital settings is of paramount importance for timely treatment and patient transport. The accuracy of dispatching has seldom been addressed in previous studies.

Objective:

In this study, we aimed to build a machine learning-based model through text mining of emergency calls for automated identification of severely injured patients in road accidents.

Methods:

Audio recordings of road accidents in Taipei City in 2018 were retrieved and randomly sampled. Data on transferring calls or non-Mandarin speech were excluded. All the included cases were evaluated by both humans (six dispatchers) and a machine learning model (prehospital activated major trauma (PAMT) model) to predict the major trauma cases identified by emergency medical technicians at the scene. The PAMT model was developed using frequency–inverse document frequency (TF-IDF), rule-based classification, and Bernoulli Naïve Bayes (BNB) classifier, and bootstrapping was applied to evaluate the robustness. The tests of prediction, including sensitivity (SENS), specificity (SPEC), positive predictive value (PPV), negative predictive value (NPV), and accuracy (ACC), for dispatchers and the PAMT model were performed, and the results were compared in terms of the overall performance and among different certainty levels.

Results:

The means for dispatchers vs. the PAMT model were SENS 63.1% vs. 68.0%, SPEC 85.0% vs. 78.0%, PPV 71.7% vs. 60.6%, NPV 80.3% vs. 85.8%, and ACC 76.8% vs. 75.0%, respectively. The mean ACC of dispatchers vs. the PAMT model in the cases from certainty level 0 (the lowest certainty) to 6 (the highest certainty) were 66.7% vs. 83.3%, 64.3% vs. 70.4%, 68.2% vs. 72.7%, 76.4% vs. 91.7%, 56.9% vs. 58.3%, 79.8% vs. 64.3%, and 87.1% vs. 81.3%, respectively. The overall performances of dispatchers and the PAMT model were similar, but the PAMT model had higher accuracy when the dispatchers were less certain of their judgments.

Conclusions:

The results of our study suggest that the machine learning model is not superior to dispatchers in identifying road accident calls with severe trauma cases; however, the model can assist dispatchers when they lack confidence in the judgment of the calls.