Accepted for/Published in: Journal of Medical Internet Research
Date Submitted: Oct 11, 2024
Date Accepted: Feb 27, 2025
The Autonomous Cognitive Examination: Machine-Learning Based Cognitive Assessment
ABSTRACT
Background:
The rising prevalence of dementia necessitates a scalable solution to cognitive screening and diagnosis. The Autonomous Cognitive Examination (ACoE) is a machine learning-based dementia test which aims to screen all cognitive domains, providing results like established paper-based tests. However, while the ACoE has been internally validated, it has not been externally validated in a clinical population, and its ability to render accurate appraisals of cognition is unknown.
Objective:
To validate the ACoE.
Methods:
To compare the evaluations of the ACoE to established paper-based tests, 46 neurology patients were enrolled into a randomized crossover study and received either the ACoE or a standard paper-based cognitive test. Patients received either Addenbrooke’s Cognitive Examination-3 (ACE-3, n = 35) or the Montreal Cognitive Examination (MoCA, n = 11). We evaluated 3 primary metrics of the ACoE’s performance relative to paper-based tests: 1) inter-rater reliability of overall cognitive scores, 2) inter-rater reliability of cognitive domain scores, and 3) ability to classify patients similarly to paper-based tests.
Results:
The ACoE’s overall cognitive assessments were significantly reliable (ICC = 0.89, p < 0.0001). Each cognitive domain’s assessments were also significantly reliable, including attention (ICC = 0.74, pFWE < 0.00001), language (ICC = 0.89, pFWE < 0.00001), memory (ICC = 0.91, pFWE < 0.00001), fluency (ICC = 0.74, pFWE < 0.00001), and visuospatial function (ICC = 0.78, pFWE < 0.0001). The ACoE was also able to successfully diagnose patients similarly to both paper-based tests (AUC = 0.96, pFWE < 0.0001).
Conclusions:
In this small study, we evaluated if the ACoE could reproduce the assessments of relatively comprehensive standard paper-based cognitive assessments. Overall, while we find the ACoE does reliably reproduce evaluations of cognition, assessments in specific etiologies of larger sample sizes will be necessary to determine utility. Clinical Trial: N/A
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.