JMIR Publications

ABSTRACT

Background:

Cognitive functioning plays a significant role in individuals’ mental health since fluctuations in cognition influence their daily task performance. Existing digital cognitive assessment tools cannot be administered in the wild and their test sets are not brief enough to capture frequent fluctuations throughout a day. The ubiquitous availability of mobile and wearable devices may form a suitable platform for real-world cognitive assessment.

Objective:

The aims of this study were threefold; (i) to validate a smartwatch-based tool for assessment of cognitive performance; (ii) to evaluate the usability of this tool; (iii) to understand participants’ perception about the application of a smartwatch in cognitive assessment.

Methods:

We developed the Ubiquitous Cognitive Assessment Tool (UbiCAT) as a smartwatch-based tool. UbiCAT implements three cognitive tests; an Arrow test, a Letter test, and a Color test adapted from the two-choice reaction time, N-back, and Stroop tests, respectively. These tests were designed together with domain experts. We validated the UbiCAT test measures against standard computer-based tests with 21 healthy adults by applying statistical analysis significant at 95% level. Usability testing for each UbiCAT app was performed using the Mobile App Rating Scale (MARS) questionnaire. The NASA-TLX questionnaire was used to measure cognitive workload during the N-back test. Participants rated perceived discomfort of wearing a smartwatch during the tests using a 7-point likert-based scale. Upon finishing the experiment, an interview was conducted with each participant. The interviews were transcribed and semantic analysis was performed to group the findings.

Results:

Pearson's correlation analysis between the total correct responses obtained from the UbiCAT and the computer-based tests revealed a significant strong correlation (r= 0.78, p<0.001). One-way ANOVA showed a significant effect of the N-back difficulty level on the participants' performance measures. The study also demonstrated usability ratings above 4 (out of 5) in terms of aesthetics, functionality, and information. Low discomfort (less than 3 out of 7) was reported by our participants after using the UbiCAT. Seven themes were extracted from the transcripts of the interviews conducted with our participants.

Conclusions:

UbiCAT is a smartwatch-based tool with three validated cognitive tests, which is useful for studying human cognitive performance in the wild. Usability ratings showed that participants were engaged in the UbiCAT tests and did not feel any discomfort. The majority of the participants were interested in using UbiCAT, although some preferred computer-based tests, which might be due to the widespread use of personal computers.