JMIR Publications

ABSTRACT

Background:

Children with hearing loss face significant barriers in speech and language acquisition, especially during critical developmental years. While computer-assisted instruction and gamified training have shown promise, many systems lack adaptability, personalization, and sustained engagement. Serious games, which merge educational goals with immersive game mechanics, have emerged as effective tools for rehabilitation and learning. Advances in large language models (LLMs) now provide opportunities to deliver semantic-level feedback and adaptive game-based content. However, empirical validation of LLM-driven serious games for speech rehabilitation remains limited.

Objective:

This study aimed to design and evaluate Blue Voice, a home-based serious game for speech rehabilitation that integrates LLM-enabled real-time feedback, interactive training modules, and wearable sensing, with the goal of improving rehabilitation outcomes for children with hearing loss.

Methods:

We adopted a mixed-methods approach. (1) Stakeholder interviews and surveys were conducted with 26 participants, including children with hearing loss, their parents, teachers, and speech therapists, to identify needs and barriers. (2) Two experimental studies were carried out. The pilot study involved 22 children between six and twelve years old to examine basic usability and interaction. A larger controlled experiment involved 22 children of the same age group. In this experiment, one group used the Blue Voice system with LLM-enabled real-time feedback, while the other group used the same system without LLM feedback. Usability metrics, engagement levels, and speech performance outcomes were collected. Quantitative data were analyzed using descriptive and inferential statistics, and qualitative feedback was thematically analyzed. Ethical approval was obtained from the institutional review board, and written informed consent was provided by the guardians of all participants.

Results:

Stakeholder feedback indicated strong acceptance of home-based serious games for rehabilitation. In the pilot study with 22 children, usability ratings were high, with “ease of learning” scored at 5.0, “task completion without assistance” at 4.6, and “interface comprehension” at 4.6, while “feedback sufficiency” received a lower score of 3.8, suggesting areas for improvement. In the controlled experiment with 22 children, the experimental group using LLaMA-2–enabled real-time feedback achieved significantly higher speech accuracy in teacher evaluations (week 2: 4.7 ± 0.4 vs 4.1 ± 0.6, t=2.73, P<.05) and system assessments (week 2: 4.6 ± 0.3 vs 4.1 ± 0.5, t=2.56, P<.05) compared with the control group. Engagement ratings were also higher in the experimental group (4.8 ± 0.3) than in the control group (4.2 ± 0.5). Qualitative feedback highlighted that real-time semantic feedback improved children’s confidence, motivation, and expressive ability, while control group participants described the training as repetitive and less engaging.

Conclusions:

Blue Voice demonstrates the potential of serious games in speech rehabilitation for children with hearing loss. By combining LLaMA-2 semantic feedback with game-based mechanics, the system not only improved speech accuracy but also fostered confidence for sustained engagement. The findings suggest that serious games enhanced with large language models can effectively complement traditional therapy and provide a scalable, child-centered digital rehabilitation solution.