JMIR Publications

ABSTRACT

Background:

Deaf individuals often face communication challenges when interacting with those who can hear. Within healthcare settings, these challenges may pose risks to their safety, potentially resulting in misdiagnoses, treatment errors, and decreased quality of care.

Objective:

To systematically review the evidence on the communication systems reported in the literature employing human-computer interaction techniques developed for deaf people who communicate through sign language with hearing health professionals, in a healthcare context, which are already in use or proposed to be used in healthcare contexts and have been tested with human users or videos of human users.

Methods:

A comprehensive search was performed via Medline, Web of Science, Association for Computing Machinery (ACM), Institute of Electrical and Electronic Engineers (IEEE) Xplore, Scopus and Google Scholar in March/2025. The inclusion criteria comprised studies developing a sign language recognition system within a healthcare context and testing with human users. Eligible studies underwent screening by two independent investigators, with any disagreements resolved by a senior researcher.

Results:

The search retrieved 21,778 publications, and screening of reference lists identified two additional studies, resulting in a total of 23 studies meeting the eligibility criteria. Most systems (65.2%) were image-based, while 34.8% relied on sensors (glove-based or depth-sensing). Applications varied across healthcare settings, including general hospital care (43.5%), emergencies (34.8%), and primary care (17.4%). All systems were in the development and testing stage, with no data on security, psychological impacts. Accuracy ranged from 25% to 100% for image-based and 72% to 99.76% for sensor-based systems. Bidirectionality and facial expression recognition, crucial for effective communication, were largely overlooked.

Conclusions:

Image-based systems were more common than sensor-based ones, though both showed wide variability in accuracy in recognizing and interpreting signs. Most systems failed to address critical aspects such as bidirectional communication and the recognition of facial expressions, essential for effective communication. None fully address the requirements for integration into healthcare settings. These findings highlight the need for further research on implementation usability, and impact on the quality of care for deaf patients. Clinical Trial: Marcolino MS, Campos MFM, Prates RO, Reis ZN, de Oliveira LFR, Valle LR, Motta L. Sign language recognition system for deaf patients: a systematic review. 2023. DOI: 10.17605/OSF.IO/FPEMR.