JMIR Publications

ABSTRACT

Background:

Voice-based screening for Parkinson’s disease (PD) using deep learning has reported high diagnostic performance (AUC 0.85-0.97), but these results may be inflated by demographic confounding. Using the Bridge2AI Voice Dataset v3.0.0 (833 participants), we show that a logistic regression classifier using only participant age outperforms a fine-tuned Audio Spectrogram Transformer (AST) for both PD (age-only AUC 0.875 vs AST 0.843; 27.5-year mean age gap) and dementia (0.905 vs 0.895; 30.2-year gap). Because this pattern appears across two independent conditions sharing only a control group, it suggests systematic dataset-level confounding rather than disease-specific signal. To separate age from disease effects, we propose an age-matched evaluation restricting participants to ages 60-80. Under this setting, the AST retains meaningful performance (AUC 0.787; bootstrap 95% CI 0.695-0.863), while age-only classification approaches chance (0.568). Retraining the AST on the age-matched subgroup yields AUC 0.798, confirming a genuine disease-specific vocal signal.

Objective:

To quantify demographic confounding in voice-based PD and dementia screening using the Bridge2AI Voice Dataset v3.0.0, propose an age-matched evaluation protocol to separate age-related voice variation from disease signals, and suggest minimum reporting standards for voice biomarker research

Methods:

Using the Bridge2AI Voice Dataset v3.0.0 (833 participants), we fine-tuned an Audio Spectrogram Transformer (AST) for PD screening (253 participants; 106 PD, 147 controls) and dementia screening (221 participants; 73 dementia, 148 controls) using 5-fold stratified cross-validation at the participant level. A logistic regression classifier using only age and sex served as a demographic-only baseline with identical cross-validation splits. Age-matched analyses restricted evaluation to participants aged 60-80 years. The AST was additionally retrained from scratch on the age-matched PD subgroup (N=129). Supporting analyses included country-level confounding checks, cross-version external validation (Bridge2AI v2.0.1; 442 participants), attention map inspection, and baseline architecture comparisons.

Results:

The age-only classifier outperformed the fine-tuned AST for both PD (AUC 0.875 vs 0.843) and dementia (0.905 vs 0.895), reflecting large case-control age gaps. Under age-matched evaluation (ages 60-80; N=129), AST performance remained substantial (AUC 0.787; bootstrap 95% CI 0.695-0.863) while age-only classification dropped to 0.568. Retraining the AST on the age-matched subgroup yielded AUC 0.798 (95% CI 0.710-0.878), confirming a disease-specific vocal signal. Country-level analysis revealed additional confounding (all 62 Canadian participants were PD cases), and cross-version validation produced an AUC 0.618, indicating limited generalization. A survey of 12 published voice-PD studies found that most do not report age distributions or perform age-matched evaluations.

Conclusions:

Demographic confounding substantially influences voice-based disease screening performance on the Bridge2AI dataset. Age-matched evaluation and retraining show that AST models capture disease-specific vocal biomarkers, though at more modest performance levels than headline metrics suggest, supporting demographic reporting, demographic-only baselines, and age-matched evaluation as minimum standards for voice biomarker research.