Accepted for/Published in: Journal of Medical Internet Research
Date Submitted: Apr 30, 2024
Date Accepted: Mar 3, 2025
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Validation of an Adaptive Assessment of Executive Functions (ACE-X)
ABSTRACT
Background:
Executive functions (EFs) predict positive life outcomes and educational attainment. As such, it is imperative that our measures of EF constructs are both reliable and valid, with advantages for research tools that offer efficiency and remote capabilities.
Objective:
The objective of this study was to evaluate initial reliability and validity evidence for a mobile, adaptive measure of EFs called Adaptive Cognitive Evaluation - Explorer (ACE-X).
Methods:
We collected data from two cohorts of participants: a test-retest sample (N=246, age: mean=35.75, SD=11.74) to assess consistency of ACE-X task performance over repeated administrations, and a validation sample involving child/adolescent (n=5,436, age: mean=12.78, SD=1.60) and adult participants (n=484, age: mean=38.11, SD=14.96) to examine consistency of metrics, internal structures, and invariance of ACE-X task performance. A subset of participants (n=132, age: mean=37.04, SD=13.23) also completed a similar set of cognitive tasks using the Inquisit™ platform to assess the concurrent validity of ACE-X.
Results:
Intraclass correlation coefficients revealed most ACE-X tasks were moderately to very reliable across repeated assessments (ICC=.45-.79, P<.001). Mean comparisons with the literature indicated that patterns of responses were consistent with those reported in previous studies. Moreover, in comparisons of internal structures of ACE-X task performance, model fit indices suggested that a network model based on partial correlations was the best fit to the data (χ2(28)=40.13, P=.064, CFI=.99, RMSEA=.03[.00;.05], BIC=5075.87, AIC=4917.71), and that network edge weights are invariant across both younger and older adult participants. A Spinglass community detection algorithm suggested ACE-X task performance can be described by 3 communities (selected in 85% of replications): set reconfiguration, attentional control, and interference resolution. On the other hand, Pearson correlation coefficients indicated mixed results for the concurrent validity comparisons between ACE-X and Inquisit™ (r=-.05-.62, P<.001-.761).
Conclusions:
These preliminary findings suggest that ACE-X is a reliable and valid research tool for understanding EFs and their relations to outcome measures.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.