JMIR Publications

ABSTRACT

Background:

Persistent cognitive impairment is prevalent in unipolar and bipolar disorders and is associated with decreased quality of life and psychosocial dysfunction. However, there is no digital cognitive screening tool for brief and accurate assessment of cognitive impairments in patients with these affective disorders. The Screen for Cognitive Impairment in Psychiatry (SCIP) test is a validated, short paper-and-pencil instrument for assessment of cognition in affective disorders.

Objective:

To present the design, implementation, and feasibility study of the Internet-based Cognitive Assessment Tool (ICAT) as a web-based cognitive test battery designed based on the SCIP cognitive tasks. The aim of the feasibility study in this early stage was three-folded; among healthy individuals (i) to evaluate the usability of ICAT, (ii) to investigate the feasibility of ICAT as a patient-administered cognitive assessment tool, and (iii) to examine the performance of automatic speech recognition (ASR) for assessment of cognitive function.

Methods:

ICAT was developed in a user-centered design process. The final implementation of ICAT was a web-based tool involving five short cognitive tasks modified from the SCIP to assess immediate and delayed recall, working memory, executive skills, and psychomotor speed. Usability and feasibility studies were conducted with two groups of healthy individuals (N=21 and N=19, respectively). Tests were done for Danish as well as English speaking participants. Each participant of the feasibility study first performed the Danish version of the SCIP test (SCIP-D), and then did the ICAT. Think-aloud method was used during both studies and the participants were asked to fill in the Post-Study System Usability Questionnaire (PSSUQ) upon completing the ICAT. Verbal recall in ICAT was assessed using ASR and performance evaluation criteria was word error rate (WER). Pearson’s two-tailed correlation analysis significant at 0.05 level was applied to investigate the association between SCIP and ICAT total and sub-test scores.

Results:

The psychometric factors of PSSUQ gave scores above 4 (out of 5). Overall, the analysis of the feasibility study revealed moderate to strong correlation between the total scores of SCIP and ICAT (r = 0.63, P= 0.009). There were also moderate to strong correlations between the SCIP and ICAT sub-tests for immediate verbal recall (r=0.67, P=0.002) and psychomotor speed (r=0.71, P=0.001). The associations between the respective sub-tests for working memory, psychomotor speed and delayed recall, however, were not statistically significant. The corresponding WER for English and Danish responses were 17.8% and 6.3%.

Conclusions:

ICAT is the first digital patient-administered screening instrument modified from the SCIP using web-based technology and ASR. The usability ratings revealed that healthy participants found ICAT easy to use. The moderate correlation between the ICAT and SCIP scores suggests that ICAT is a valid tool to assess cognition, although this should be confirmed in a larger study with greater statistical power. There was good accuracy of the ASR for verbal memory assessment. Taken together, the ICAT seems to be a valid online cognitive assessment tool which after some minor modifications and further validation may be used to screen for cognitive impairment in clinical settings.