JMIR Publications

ABSTRACT

Background:

SymptomGuide® Dementia (SG-D) is a publicly available online symptom tracking tool. The value of these data are enhanced when the specific dementia stage is identified.

Objective:

We aimed to develop a supervised machine learning algorithm to classify dementia stages based on the symptoms tracked in SymptomGuide® Dementia (SG-D).

Methods:

We employed clinical data from 717 people from three sources: 1) a memory clinic; 2) a long-term care study30; and 3) the VASPECT31 clinical trial. Symptoms were captured with SymptomGuide® Dementia (SG-D), a web-based symptom tracking tool aimed to support caregivers of persons living with dementia. A clinician-rated dementia stage classified four groups using either the Functional Assessment Staging Test or the Global Deterioration Scale: Mild Cognitive Impairment, or mild, moderate, or severe dementia. Individualized symptom profiles from the pooled data were used to train machine learning models to predict dementia severity. To ensure unbiased evaluation of model performance, models trained with 6 different machine learning algorithms were compared using nested cross-validation to identify the best performing model. The best performing algorithm was used to train a model optimized for balanced accuracy. Model performance was assessed using measures of balanced accuracy, precision (Positive Predictive Value), sensitivity (recall), Cohen’s Kappa, Area Under the Receiver Operating Characteristic Curve (AUC-ROC), Area Under the Precision-Recall Curve (AUPRC).

Results:

The study population was mostly female (59%), older adults (77.3 ± 10.6 years, range 40-100 years) with mild-moderate dementia (46%). Age, duration of symptoms, 37 unique dementia symptoms and 10 symptom-derived variables were used to distinguish dementia stages. A model trained with a Support Vector Machine Learning algorithm using a one-versus-rest approach showed the best performance. The correct dementia stage was identified with 83% balanced accuracy (Cohen’s Kappa=0.81, AUPRC=0.91, AUC-ROC=0.96). The best performance was seen when classifying severe dementia (AUC-ROC= 0.99).

Conclusions:

A supervised machine learning algorithm exhibited excellent performance in identifying dementia stages based on dementia symptoms reported in an online environment. This novel dementia staging algorithm can be used to describe dementia stage based on user-reported symptoms. This type of symptom recording offers real-world data that reflect important symptoms in people affected by dementia.