JMIR Publications

ABSTRACT

Background:

Cognitive testing in large population surveys is frequently used to describe cognitive aging and to determine incidence rates, risk factors, and long-term trajectories of the development of cognitive impairment. As the surveys are increasingly administered on Internet-based platforms, web-based, self-administered cognitive testing calls for close investigation.

Objective:

Web-based self-administered versions of two age-sensitive cognitive tests, the Stop and Go Switching Task for executive functioning and the Figure Identification test for perceptual speed, were developed and administered to adult participants in the Understanding America Study. The current study examined the comparability of the cognitive test scores across types of Internet devices, and the extent to which the scores were associated with self-reported distractions in everyday environments in which the participants took the tests. Additionally, national norms on the cognitive scores were provided for the U.S. population.

Methods:

Data were collected from a probability-based Internet panel representative of the U.S. adult population, the Understanding America Study. Participants having access to both a keyboard-and-mouse-based device and a touchscreen-based device were asked to complete each of the cognitive tests twice, in randomized orders across device types, whereas participants having access to only one type of devices were asked to complete the tests twice on the same device. At the end of each test, participants answered questions about interruptions and potential distractions occurred during the test.

Results:

Among those who completed the device ownership survey (7,410 for Stop and Go and 7,216 for Figure Identification), 6,129 (82.7% for Stop and Go) and 6,717 (93.1% for Figure Identification) participants completed the first session and correctly responded to at least 70% of the trials. On average, the standardized differences across device types were small, with the absolute value of Cohen d ranging from 0.05 (for the switch score in Stop and Go and the Figure Identification score) to 0.13 (for the nonswitch score in Stop and Go). Poorer cognitive performances were moderately associated with older ages (the absolute value of r ranged from 0.32 to 0.61) and this relationship was comparable across device types (the absolute value of Cohen q ranged from 0.01 to 0.17). About 10% of participants (779/6123, 12.7% for Stop and Go; 828/6721, 12.3% for Figure Identification) were interrupted during the test. Interruptions predicted poorer cognitive performance (P<.01 for all scores). Specific distractions (e.g., watching TV, listening to music) were inconsistently related to cognitive performance. National norms, as weighted average score using sampling weights, suggested poorer cognitive performance as age increased.

Conclusions:

Cognitive performance assessed by self-administered web-based tests was valid and comparable across keyboard- and touchscreen-based Internet devices. Distraction in everyday environments, especially being interrupted during the test, may result in a non-trivial bias in cognitive testing.