JMIR Publications

ABSTRACT

Background:

Recent advances in natural language processing and artificial intelligence have led to improvements and a widespread adoption of speech recognition technologies (SR). In healthcare, SR is being used by clinicians and consumers in different settings, especially since the adoption of electronic health records (EHR) and conversational agents (CA). In clinical settings, SR is typically used for dictation, transcription and text entry in documentation and report generation. CAs are also used for data collection in disease management and to facilitate diagnosis. Previous controlled experiments evaluating SR in clinical EHR tasks found it increased errors, the time to complete tasks, and was less usable compared to keyboard and mouse. In consumer applications, CAs have been utilized for a variety of purposes including data collection, decision support and patient monitoring. However, little is known about the safety risks of CAs and few studies have evaluated CAs in the hands of consumers for their efficacy and safety in care delivery. Outside healthcare, cognitive load has been observed to be an important factor affecting the use of SR technologies. Users find it more difficult to speak and think at the same time when compared to typing, pointing and clicking. However, the effects of SR on cognitive load when performing health tasks have not yet been explored. Thus, this study sets out to evaluate SR for documentation in consumer health tasks.

Objective:

To evaluate the effects of speech recognition on problem solving and recall in consumer health tasks.

Methods:

Fifty university staff and students were recruited to undertake four documentation tasks with a simulated health bot in a computer laboratory. Tasks varied in complexity determined by the amount of problem solving and recall (simple and complex) and input modality (SR versus keyboard and mouse). Cognitive load, task completion time, error rate and usability were measured.

Results:

Compared to using keyboard and mouse, SR significantly increased cognitive load for complex tasks (Z=-4.08, P<.001) and simple tasks (Z=-2.24, P=.03). Complex tasks took significantly longer to complete (Z=-2.52, P=.01) and SR was reported to be overall less usable than using keyboard and mouse (Z=-3.30, P=.001). However, there was no effect on errors.

Conclusions:

There is an increased cognitive load for the use of SR to perform consumer health tasks. Therefore, using SR may not be a suitable input modality for complex consumer health tasks that require problem solving and recall. Further studies are needed to investigate the effects of cognitive load on task performance and safety.