Currently submitted to: JMIR Human Factors
Date Submitted: Mar 20, 2026
Open Peer Review Period: Apr 6, 2026 - Jun 1, 2026
(currently open for review)
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Developing the SUISQ-RVI for streamlined usability assessment of modern speech interfaces: exploratory factor analysis pilot study
ABSTRACT
Background:
The proliferation of AI-driven voice chatbots necessitates updated methods for usability assessment. While the Speech User Interface Service Quality (SUISQ) questionnaire is a comprehensive tool, its original structure may not capture the nuances of modern speech bots.
Objective:
This pilot study details an exploratory evaluation of the SUISQ and the subsequent development of a reduced version of SUISQ for Voice Interfaces (SUISQ-RVI).
Methods:
Using a dataset (N=48) from a simulated speech bot interaction, we conducted descriptive and exploratory data analyses on the full SUISQ and its previously reduced versions (SUISQ-R and SUISQ-MR).
Results:
Preliminary results indicate that the original four-factor structure may not be optimal for conversational AI contexts. Guided by our exploratory factor analysis, we reduced the original 25-item instrument to a 17-item scale demonstrating a five-factor structure. In this pilot sample, the SUISQ-RVI outperformed previous reduced versions by explaining more variance with better model fit and reliability.
Conclusions:
These initial findings suggest the SUISQ-RVI is a promising, streamlined tool for evaluating modern voice interfaces, though further confirmatory validation with larger cohorts is warranted. Clinical Trial: NOT APPLICABLE
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.