JMIR Publications

ABSTRACT

Background:

Cognitive assessment employing tangible objects can measure fine motor and hand-eye coordination skills along with other cognitive domains. Administering such tests is often expensive, labor-intensive, and error-prone due to manual recording and the potential subjectivity. Automating the administration and scoring process can address these difficulties while reducing time and cost. e-Cube is a new vision-based, computerized cognitive assessment tool that integrates computational measures of play complexity and item generators to enable automated and adaptive testing. The e-Cube games employ a set of cubes, and the system tracks the movement and location of these cubes manipulated by the player.

Objective:

Primary objectives of the study are to (1) validate the play complexity measures that form the basis of developing the adaptive assessment system and (2) evaluate the preliminary utility and usability of the e-Cube system as an automated cognitive assessment tool.

Methods:

This study employed six e-Cube games, including Assembly, Shape-Matching, Sequence-Memory, Spatial-Memory, Path-Tracking, and Maze, each targeting different cognitive domains. Two versions of the games, the fixed version with pre-determined sets of items and the adaptive version utilizing the autonomous item generators, were prepared for comparative evaluation. Enrolled participants (n=80, aged 18-64) were divided into two groups, Fixed Group (n_1=38) and Adaptive Group (n_2=42). Each was administered the six e-Cube games, three subtests of the Wechsler Adult Intelligence Scale, 4th Edition (WAIS-IV) (Block Design, Digit Span, and Matrix Reasoning), and System Usability Scale (SUS). Statistical analyses at the 95% significance level were applied.

Results:

The play complexity values were correlated with the performance indicators, i.e., correctness and completion time. The adaptive e-Cube games were correlated with the WAIS-IV subtests: r42=0.49 (P<.001),CI=[0.21,0.70] for Assembly and Block Design; r42=0.34 (P=.03),CI=[0.03,0.59] for Shape-Matching and Matrix Reasoning; r42=0.51 (P<.001),CI=[0.24,0.71] for Spatial-Memory and Digit Span; r42=0.45 (P=.003),CI=[0.16,0.67] for Path-Tracking and Block Design; and r42=0.45 (P=.003),CI=[0.16,0.67] for Path-Tracking and Matrix Reasoning. The fixed version showed weaker correlations with the WAIS-IV subtests. The e-Cube system showed a low false detection rate (0.1%) and was determined usable with the average SUS score of 86.01 (SD 8.75).

Conclusions:

The correlations between the play complexity values and participants’ performance indicators supported the validity of the play complexity measures. Correlations between the adaptive e-Cube games and WAIV-IV subtests demonstrated the potential utility of the e-Cube games for cognitive assessment, while a further validation study is needed to confirm. The low false detection rate and high SUS scores indicated that e-Cube is technically reliable and usable.