Accepted for/Published in: Journal of Medical Internet Research
Date Submitted: Jun 27, 2022
Date Accepted: Sep 16, 2022
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Automatic Assessment of Intelligibility in Noise in Parkinson’s Disease: A Validation Method
ABSTRACT
Background:
Most individuals with Parkinson’s disease (PD) experience a degradation in their speech intelligibility. Research on the use of automatic speech recognition (ASR) to assess intelligibility is still sparse, especially when trying to replicate communication challenges in real-life conditions (i.e., noisy backgrounds). Developing technologies to automatically measure intelligibility in noise can ultimately assist patients in self-managing their voice changes due to the disease.
Objective:
The goal of this study was to pilot-test and validate the use of a customized web-based app to assess speech intelligibility in noise in individuals with dysarthria associated with PD.
Methods:
Twenty individuals with dysarthria associated with PD and 20 healthy controls (HC) recorded a set of sentences using their phones. Google Cloud ASR API was utilized to automatically transcribe the speakers’ sentences. An algorithm was created to embed speakers’ sentences in +6dB SNR multitalker babble. Results from ASR performance were compared to those from 30 listeners who orthographically transcribed the same set of sentences. Data were reduced into a response defined as a success event: whether the AI system transcribed a random speaker/sentence as well-or-better than the average of three randomly chosen human listeners. These data were further analyzed by logistic regression to assess whether AI success differed by speaker group (HC or speakers with dysarthria) or was affected by sentence length. A discriminant analysis was conducted on the human listener data and AI transcriber data independently to compare the ability of each data set to discriminate between HCs and speakers with dysarthria.
Results:
The data analysis indicated a 0.8 probability (0.65, 0.91)95%CI that AI performance would be as-good-or-better than the average human listener. AI transcriber success probability was not found to be dependent on speaker group. AI transcriber success was found to decrease with sentence length, losing an estimate 0.03 probability of transcribing as well as the average human listener for each word increase in sentence length. The AI transcriber data were found to offer the same discrimination of speakers into HC and speakers with dysarthria categories as the human listener data.
Conclusions:
Automatic speech recognition has the potential to assess intelligibility in noise in speakers with dysarthria associated with PD. Results hold promise for the use of artificial intelligence with this clinical population, although a full range of speech severity needs to be evaluated in future work, as well as the effect of different speaking tasks on ASR.
Citation
The author of this paper has made a PDF available, but requires the user to login, or create an account.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.