JMIR Publications

ABSTRACT

Background:

Recent studies have demonstrated that AI can surpass medical practitioners in diagnostic accuracy, underscoring the increasing importance of AI-assisted diagnosis in healthcare. This research introduces SMART-Pred (Shiny Multi-Algorithm R Tool for Predictive Modeling), an innovative AI-based application for Alzheimer's disease (AD) prediction utilizing handwriting analysis

Objective:

Our objective is to develop and evaluate a non-invasive, cost-effective, and efficient AI tool for early AD detection, addressing the need for accessible and accurate screening methods.

Methods:

Our methodology employs a comprehensive approach to AI-driven Alzheimer's disease (AD) prediction. We begin with Principal Component Analysis for dimensionality reduction, ensuring efficient processing of complex handwriting data. This is followed by the training and evaluation of ten diverse, highly optimized AI models, including logistic regression, Naïve Bayes, random forest, AdaBoost, Support Vector Machine, and neural networks. This multi-model approach allows for a robust comparison of different machine learning techniques in AD prediction. To rigorously assess model performance, we utilize a range of metrics including accuracy, sensitivity, specificity, F1-score, and ROC-AUC. These comprehensive metrics provide a holistic view of each model's predictive capabilities. For validation, we leveraged the DARWIN dataset, which comprises handwriting samples from 174 participants (89 AD patients and 85 healthy controls). This balanced dataset ensures a fair evaluation of our models' ability to distinguish between AD patients and healthy individuals based on handwriting characteristics.

Results:

The random forest model demonstrated strong performance, achieving an accuracy of 88.68% on the test set during comprehensive model analysis. Meanwhile, the AdaBoost algorithm exhibited even higher accuracy, reaching 92.00% after leveraging AI models to identify the most significant variables for predicting Alzheimer's disease. These results surpass current clinical diagnostic tools, which typically achieve around 81.00% accuracy. SMART-Pred's performance aligns with recent AI advancements in AD prediction, such as the Cambridge scientists' AI tool achieving 82.00% accuracy in identifying AD progression within three years using cognitive tests and MRI scans. Furthermore, our comprehensive analysis utilizing SMART-Pred revealed a consistent pattern across all ten AI models employed. The variables "air_time" and "paper_time" consistently stood out as critical predictors for Alzheimer's disease (AD). These two factors were repeatedly identified as the most influential variables in assessing the probability of AD onset, underscoring their potential importance in early detection and risk assessment of the disease.

Conclusions:

Even though some limitations exist with SMART-Pred, it offers several advantages, including being non-invasive, cost-effective, efficient, and customizable for complex datasets and disease diagnostics. The study demonstrates the transformative potential of AI in healthcare, particularly in AD prediction, and may contribute to improved patient outcomes through early detection and intervention. Clinical validation is necessary to confirm whether the key variables identified in this study are sufficient for accurately predicting Alzheimer's disease in real-world medical settings. This step is crucial to ensure the practical applicability and reliability of these findings in clinical practice.