JMIR Publications

ABSTRACT

Background:

Parkinson’s Disease (PD) is the second most common neurodegenerative disorder globally and is largely characterized by motor symptoms. Early detection is critical for timely intervention, yet current AI models are usually trained using participants with a confirmed diagnosis and thus later-stage disease.

Objective:

We developed a web platform for structured mouse-tracing data collection through pattern tracing tests. We sought to assess whether models trained on self-reported PD data could generalize to suspected PD individuals, which reflects a low-specificity condition that may indicate early motor symptoms prior to a formal diagnosis of PD or related disorder. We also tested the reverse: whether suspected PD samples could serve as reliable training data for identifying later-stage PD.

Methods:

261 participants (73 self-reported PD, 155 non-PD, and 33 suspected PD) completed three pattern tracing tasks: straight line, sine wave, and spiral wave. During each task, cursor positions, screen dimensions, and an in-target boolean flag were recorded. From these data, we engineered features and generated mouse trace images. We built three categories of classifiers: (1) a feed-forward neural network for engineered features; (2) fine-tuned computer vision models; and (3) multimodal models concatenating a feed-forward neural network with computer vision models. Performance was evaluated for three splits: (1) 5-fold cross-validation on self-reported PD vs. non-PD controls; (2) training on self-reported PD and non-PD controls, testing on suspected PD vs. non-PD controls; and (3) training on suspected PD and non-PD controls, testing on self-reported PD vs non-PD controls.

Results:

The best-performing models were an image-based DenseNet-201 model with an F1 score of 0.9027 ± 0.0332 (split 1), a multimodal ResNet-50 with an F1 score of 0.9353 ± 0.0334 (split 2), and a multimodal ViT with an F1 score of 0.7619 ± 0.0535 (split 3). Training with suspected PD cases led to reasonable performance for predicting PD cases, and vice versa, indicating potential for later-stage PD data to be useful for training early prediction models, and suggesting that a lack of diagnostic specificity in the training data can still be useful for creating screening tools. Image inputs consistently proved to be the most predictive in GradShap analysis.

Conclusions:

A persistent challenge in healthcare AI is that models are often trained on data from individuals with well-established later-stage disease due to the need for diagnostic certainty. However, the clinical goal is typically to detect disease as early as possible. Our findings suggest that, in the case of PD classification using motor data, models trained on later-stage cases may generalize to individuals in earlier stages who have not yet received a formal diagnosis. We provide preliminary support for the use of retrospective clinical data in early detection, though future studies are needed to validate this approach.