JMIR Publications

ABSTRACT

Background:

Clinical documentation burden contributes significantly to physician burnout, with healthcare professionals spending much of their time on electronic health record interactions. Automatic speech recognition (ASR) systems offer a promising solution; however, their application in Korean medical settings faces unique challenges due to widespread Korean-English code-switching, where clinicians routinely alternate between Korean conversational language and English medical terminology within single utterances.

Objective:

This study aimed to develop and evaluate a hybrid post-processing approach combining medical terminology dictionary normalization with large language model (LLM) refinement to improve ASR accuracy for Korean-English code-switched medical speech.

Methods:

We constructed a speech dataset from 23,652 nursing progress notes, with linguistic composition of 67.73% Korean, 23.54% English, and 8.73% numerals or special symbols. Four Korean nurses recorded the notes using four microphone types in an acoustically isolated environment. Speech recognition was performed using OpenAI's gpt-4o-transcribe model. For post-processing, a medical terminology dictionary containing 1,070 mapping entries was constructed from 1,000 nursing progress notes to normalize Korean phonetic renderings of English medical terms. Five LLMs (GPT-4o and four Claude variants) were then evaluated across five temperature settings (0.0–0.8). Performance was assessed using BERTScore (F1), Sentence-BERT cosine similarity, word error rate (WER), and character error rate (CER), comparing post-processed outputs against the original written notes. Statistical significance was determined using Welch t test (α=.05)

Results:

Baseline ASR achieved a BERTScore of 0.9131 and CER of 0.2336. Dictionary-based normalization performed 43,507 word-level substitutions in 70.8% (16,754/23,652) of transcribed sentences. LLM-only post-processing reduced CER by 36.09% (Claude Sonnet 4) and 32.53% (GPT-4o) compared to baseline. The combined dictionary-LLM approach achieved the best performance: Claude Sonnet 4 attained a BERTScore of 0.9638 and CER of 0.0820, representing a 64.9% reduction in CER from baseline (P<.001). Temperature optimization revealed that Claude models exhibited consistent performance across all settings (all P>.05), while GPT-4o showed significant improvement at temperature 0.6 (P<.001 for BERTScore; P=.002 for CER).

Conclusions:

The hybrid approach combining rule-based dictionary normalization with LLM post-processing significantly improves Korean-English code-switched medical ASR accuracy. The temperature invariance of Claude models suggests superior suitability for clinical deployment where reproducibility is essential. This framework provides a practical, modular solution for addressing multilingual challenges in medical ASR systems without requiring model retraining.