JMIR Publications

ABSTRACT

Background:

In the US, language barriers pose challenges to communication in emergency response, and impact emergency care delivery and quality for individuals who are limited English proficient (LEP). There is growing interest among Emergency Medical Service (EMS) personnel in using automated translation tools to improve communications with LEP individuals in the field. However, little is known about whether automated translation software can be used successfully in EMS settings to improve communication with LEP individuals.

Objective:

The objective of this work is to use scenario-based methods with potential EMS providers and non-native English speaking users who identified as LEP (henceforth referred to as LEP participants) to evaluate two automated translation technologies for improving emergency communication.

Methods:

We developed mock emergency scenarios, which were enacted in simulation sessions with EMS personnel and Spanish-speaking and Chinese-speaking (Mandarin) LEP participants using two automated language translation tools: an EMS domain-specific fixed-sentence translation tool (Quickspeak) and a statistical machine translation tool (Google Translate). At the end of the sessions, we gathered feedback from both groups through a post-session questionnaire. EMS participants also completed the System Usability Scale (SUS).

Results:

We conducted a total of five group sessions (3 Chinese, 2 Spanish) with 12 Chinese-speaking LEP participants, 13 Spanish-speaking LEP participants, and 17 EMS personnel. Overall, communications between EMS and LEP participants remained limited even with the use of the two translation tools. QuickSpeak had higher mean SUS scores compared to Google Translate (65.3 vs 48.4, P=.04). Although both tools were deemed less than satisfactory, LEP participants showed preference towards the domain specific system with fixed questions (QuickSpeak) over the free-text translation tool (Google Translate) in terms of understanding the EMS personnel’s questions (Chinese 92% vs 25%; Spanish 86% vs 43%). While both EMS and LEP participants appreciated the flexibility of the free-text tool, multiple translation errors and difficulty responding to questions limited its usefulness.

Conclusions:

Technologies are emerging that have the potential to assist with language translation in emergency response; however, improvements in accuracy and usability are needed before these technologies could be used safely in the field.