JMIR Publications

ABSTRACT

Background:

Remote, scalable cognitive assessment could improve detection and longitudinal monitoring of age-related cognitive change, but few studies have validated comprehensive digital neuropsychological batteries administered entirely at home.

Objective:

To evaluate the feasibility, acceptability, and short-term test–retest reliability of a remotely delivered digital neuropsychological battery (Portable Automated Rapid Testing; PART) in cognitively healthy older adults.

Methods:

We screened 82 English-speaking, cognitively healthy older adults aged 50–85 and mailed configured tablets to participants. Researchers remotely administered a battery of neuropsychological assessments spanning language fluency (Boston naming test, verbal fluency: letter “F”, supermarket items, animals), memory (verbal paired associates, word list recall, logical memory recall), praxis memory (clock drawing, constructional praxis), and executive functioning (trail making test) using the Portable Automated Rapid Testing app twice approximately one month apart. Task tolerability was summarized with a comfort index based on participants' self-reports after completing each task. Reliability analyses included paired t-tests to detect group-level shifts in performance, Pearson correlations for short term stability, and Bland–Altman limits of agreement (LoA) to quantify bias and within-person variability. Performance across all tasks was also compared to comparative samples from large normative studies using equivalent paper-and-pencil versions of these tasks.

Results:

73 completed the first timepoint (T1), 64 completed the second (T2), and the analytic sample comprised N = 63. Overall tolerability was high, as over 75% of participants rated tasks comfortable. Memory and executive function measures showed moderate-to-high test–retest correlations (r range = 0.39–0.73); ns t-tests), while clock drawing and constructional praxis showed low or non-significant associations likely due to ceiling effects. Paired tests indicated small but significant practice/improvement effects for logical memory (Story A: t56 = −2.95, p < 0.01), verbal paired associates (t60 = −3.10, p<0.01), word list (t57 = −2.50, p = 0.02), and trail making part A (t50 = −3.97, p < 0.01) and Part B (t42 = −2.05, p = 0.05). Bland–Altman analyses revealed minimal mean bias for many tasks but notably wide LoA for TMT (Part A: LoA = −44.7 to 29.2 seconds; Part B: LoA = −78.1 to 57.9 seconds). Importantly, medians and quartiles of our sample overlapped within plus or minus one standard deviation of the medians of comparative paper-and-pencil normative in comparable samples.

Conclusions:

PART was well tolerated among healthy older adults and reproduces group-level normative patterns with moderate-to-high short-term reliability for several conventional measures. However, non-trivial within-person variability, practice effects, ceiling effects, and wide limits of agreement for some measures (e.g., TMT, constructional praxis, and clock drawing) limit interpretation of individual change. Future work should validate alternative digital outcome metrics (e.g., stylus interactions, speech recordings), extend sampling to more diverse and lower-education cohorts, and evaluate more ecologically valid designs to improve sensitivity for detecting meaningful within-person cognitive fluctuations.