Currently submitted to: JMIR mHealth and uHealth
Date Submitted: Jun 9, 2026
Open Peer Review Period: Jun 9, 2026 - Aug 4, 2026
(currently open for review)
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Random-Stimulus-Substitution Mobile Memory Monitoring Tool DiaMem for Older Adults: A Randomized Crossover Trial of Reliability, Validity, and Usability
ABSTRACT
Background:
Cognitive decline in older adults can be intervened, yet effective monitoring of community based programs or trials requires assessment tools that are repeatable, reliable, and free of practice effects. To address this need, we developed DiaMem, a WeChat mini program that uses a structured scenario framework while randomly drawing memory items from a large multimodal library to ensure stimulus novelty across repeated tests.
Objective:
To preliminarily evaluate the reliability, validity, and usability of DiaMem in community dwelling older adults.
Methods:
Thirty community dwelling older adults were recruited offline from community centers (closed trial) and completed a 6 day randomized crossover study with three repeated assessments of DiaMem and three of the reference test MemTrax, a continuous recognition test. All assessments were web based (self administered via WeChat) and supervised in person, with no assistance on test content. Reliability was assessed using intraclass correlation coefficient (ICC), minimum detectable change (MDC), and Cronbach’s α. Practice effect was examined by paired t tests (Cohen’s d) between first and third sessions. Criterion related validity was examined against the Auditory Verbal Learning Test (AVLT) and the Montreal Cognitive Assessment (MoCA). Content validity was evaluated by 10 experts using the content validity index (CVI). Usability was measured with the System Usability Scale (SUS).
Results:
All 30 participants completed all scheduled assessments (no attrition). The participants had a mean age of 74.3±9.0 years, and 66.7% were female. DiaMem total score showed good test retest reliability, with ICC(3,1) = 0.836 (95% CI: 0.725–0.912) and Cronbach’s α = 0.898. The MDC was 13.5 points for a single assessment and 7.8 points for the average of three assessments. No significant practice effect was detected (Cohen’s d = 0.02). For criterion related validity, DiaMem total score was moderately correlated with AVLT total score (r = 0.458, P = 0.011) and strongly correlated with MoCA total score (r = 0.766, P < 0.001). Content validity was high (S CVI/Ave = 0.976). The SUS score for DiaMem (69.3±6.3) was significantly higher than that for MemTrax (64.1±11.0; P = 0.020).
Conclusions:
DiaMem showed good test-retest reliability and acceptable validity and usability. Its design mitigates practice effects, supporting repeated assessments in settings where rapid cognitive changes occur (e.g., cognitive training, post stroke) and in clinical trials where averaging multiple measurements may reduce sample size. Clinical Trial: ClinicalTrials.gov NCT07416019
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.