JMIR Publications

ABSTRACT

Background:

Voice disorders are common, yet access to timely specialist assessment is often limited. Automated analysis of short voice recordings may support early screening, but achieving robust performance on heterogeneous clinical datasets remains challenging.

Objective:

To investigate the impact of synthetic speech augmentation on voice disorder recognition and classification using the Saarbruecker Voice Database (SVD) and to develop an adaptive ensemble with learnable weighting and optional sex embedding.

Methods:

Sentence recordings from the SVD were used for binary Voice Disorder Recognition (VDR; n=1,988) and 4-class Voice Disorder Classification (VDC; n=1,190). Two Text-to-Speech (TTS) models (XTTS and Kartoffelbox) generated one synthetic sample per training instance. Synthetic speech quality was assessed in a blinded listening evaluation (40 samples per TTS model). Baseline models included SVM, ResNet-18, LSTM, and transformer-based architectures (HuBERT, Wav2Vec 2.0, WavLM). Ensemble learning and a Learnable Fusion (LF) approach, with and without a sex embedding (LFS), were evaluated using bootstrapped performance estimates and standard classification metrics; AUC was reported for VDR.

Results:

In the blinded evaluation, Kartoffelbox-generated speech was less distinguishable from original recordings than XTTS (52.6% vs 89.5% identified as synthetic) and enabled more accurate perceptual discrimination of pathological status (75.9%/80.0% vs 65.5%/60.0%) and diagnostic class (51.3% vs 35.9%). Among baseline models, HuBERT showed the most balanced VDR performance (accuracy 0.85; F1 0.84). Kartoffelbox-based augmentation further improved performance, with HuBERT achieving 0.90 in sensitivity, 0.87 in accuracy, and 0.86 in F1 score. Learnable Fusion increased robustness (accuracy 0.88; F1 0.88), while adding a sex embedding yielded the best VDR results (accuracy 0.90; AUC 0.92). For VDC, HuBERT was the strongest baseline (accuracy 0.72; median F1 0.63), and fusion achieved the highest overall performance (accuracy 0.73; F1 0.66), with the largest gains for laryngitis and hyperfunctional dysphonia.

Conclusions:

High-quality synthetic speech augmentation improves automated voice disorder analysis across heterogeneous clinical data, with effectiveness varying by TTS model. Adaptive, learnable fusion of complementary architectures provides the most robust performance for both screening and multi-class classification, supporting AI-assisted pre-assessment in clinical and telemedicine settings.