JMIR Publications

ABSTRACT

Background:

Cognitive assessment employing tangible objects can measure fine-motor and hand-eye coordination skills along with other cognitive domains. Administering such tests is often expensive, labor-intensive, and error-prone due to manual recording and the potential subjectivity. Automating the administration and scoring process can address these difficulties while reducing time and cost. e-Cube is a new vision-based, computerized cognitive assessment tool that integrates computational measures of play complexity and item generators to enable automated and adaptive testing. The e-Cube games employ a set of cubes and the system tracks the movement and location of these cubes manipulated by the player.

Objective:

Primary objectives of the study are to (1) validate the play complexity measures that form the basis of developing the adaptive assessment system and (2) evaluate the preliminary utility and usability of the e-Cube system as an automated cognitive assessment tool.

Methods:

This study employed six e-Cube games, including assembly, shape-matching, sequence-memory, spatial-memory, path-tracking, and maze games, each targeting different cognitive domains. Two versions of the games, the fixed version with a pre-determined set of items for each game and the adaptive version utilizing the autonomous item generators, were prepared for comparative evaluation. Enrolled participants (n=78, aged 18-60) were divided into two groups, Fixed Group (n1=37) and Adaptive Group (n2=41). Each enrolled participant was administered to the six e-Cube games, three subtests of the Wechsler Adult Intelligence Scale, 4th Edition (WAIS-IV) (Block Design (BD), Digit Span (DS), and Matrix Reasoning (MR)), and System Usability Scale (SUS). Statistical analyses at the 95% significance level were applied.

Results:

The play complexity values were correlated with the performance indicators, i.e., correctness and completion time. The adaptive e-Cube games were moderately correlated with the WAIS-IV subtests (P=.001 for assembly and BD, shape-matching and MR, spatial-memory and DS; P=.004 for path-tracking and BD; P=.01 for path-tracking and MR). The fixed version showed weaker correlations with the WAIS-IV subtests. The e-Cube system showed a low false detection rate (0.1%) and was determined usable with the average SUS score of 86.1 (SD 8.84).

Conclusions:

The correlations between the play complexity values and participants’ performance indicators supported the validity of the play complexity measures. Correlations between the adaptive e-Cube games and WAIV-IV subtests showed the potential utility of the e-Cube games for cognitive assessment while a further validation study is needed to confirm. The low false detection rate and high SUS scores indicated that e-Cube is technically reliable and usable.