JMIR Publications

ABSTRACT

Background:

For clinical nurses, manually entering information into hospital information systems (HIS) remains time consuming and prone to omissions. Although speech recognition can reduce the need for manual entry, its use in clinical settings has historically been limited by code switching, medical terminology, and noisy ward environments. Recent advances in customized automatic speech recognition (ASR) and large language models (LLMs) now make speech based, structured documentation aligned with nursing frameworks such as D.A.R.T. (DaThis study developed and evaluated an integrated ASR and LLM system that transforms spoken nursing input into structured D.A.R.T. notes, evaluating its accuracy, usability, and clinical feasibility within HIS workflows.ta, Action, Response, Teaching) increasingly feasible.

Objective:

This study developed and evaluated an integrated ASR and LLM system that transforms spoken nursing input into structured D.A.R.T. notes, evaluating its accuracy, usability, and clinical feasibility within HIS workflows.

Methods:

A code-switching nursing speech corpus from emergency and ward settings was used to fine tune the Whisper Large v2 model with parameter efficient adaptation[1]. The LLM generated schema constrained D.A.R.T. records from ASR transcripts, which were verified by nurses before being uploaded to the corresponding HIS fields. Evaluation included mixed error rate (MER) for ASR accuracy, F1 scores and agreement statistics for D.A.R.T. classification, hallucination assessments based on factual correctness, and analysis of nurse feedback on system use.

Results:

The fine tuned ASR model reduced the mixed error rate from 31.65% to 6.6%. D.A.R.T. generation achieved an F1 score of 0.82 and met the non-inferiority margin relative to human transcripts (Δ = −0.04).The hallucination rate was 2.51%. During deployment, monthly documentation volumes increased from 32,724 to 65,417, and 76% of the 120 participating nurses reported reduced workload and improved completeness.

Conclusions:

The integrated ASR and LLM system was feasible and showed strong performance, with good acceptance among clinical nurses. It reduced manual documentation burden, improved record completeness, and demonstrated the value of an ASR and LLM supported workflow for nursing documentation.