JMIR Publications

ABSTRACT

Background:

Five to ten percent of elementary school children show delayed development of fine motor skills. To address these problems, detection is required. Current assessment tools are time-consuming, require a trained supervisor and are not motivating for children. Sensor augmented toys and machine learning have been presented as possible solutions.

Objective:

This study examines whether sensor augmented toys can be used to assess children’s fine motor skills. The objectives were to 1) predict the outcome of the fine motor skill part of the Movement Assessment Battery for Children (fine MABC) and 2) to study the influence of the classification model, game, type of data, and level of difficulty of the game on the prediction.

Methods:

Children in elementary school (n=97, age=7.8±0.7) performed the fine motor skill part of the fine MABC and played two games with the sensor augmented toy called “Futuro Cube”. The game “roadrunner” focused on speed while the game “maze” focused on precision. Each game had several levels of difficulty (LoD’s). While playing, both sensor and game data were collected. Four machine learning classifiers were trained with this data to predict the fine MABC outcome: k-nearest neighbor (KNN), logistic regression (LR), decision tree (DT), and support vector machine (SVM). First, we compared the performances of the games and classifiers. Subsequently, we compared the LoD’s and types of data for the classifier and game that performed best on accuracy and F1-score.

Results:

The highest achieved mean accuracy (0.76) was achieved with a DT classifier that was trained on both sensor and game data obtained from playing the easiest and hardest level of the roadrunner game. Significant differences in performance were found in accuracy scores between data obtained from the roadrunner and maze game (DT: P=.01; KNN: P=.02; LR: P=.04; SVM: P=.04). No significant differences in performance were found in accuracy scores between the best performing classifier and the other three classifiers for both the roadrunner game (DT vs KNN: P=.42; DT vs LR: P=.35; DT vs SVM: P=.08) and the maze game (DT vs KNN: P=.15; DT vs LR: P=.62; DT vs SVM: P=.26). The accuracy of the best performing LoD (combination of the easiest and hardest level) achieved with the DT classifier trained with sensor and game data obtained from the roadrunner game was only significantly better than the combination of the easiest and middle level (P=.046).

Conclusions:

The results show that sensor augmented toys can do a good job in predicting the fine MABC score for children in elementary school. Selecting the game type (focusing on speed or precision) and data type (sensor or game data) is more important for the performance than selecting the machine learning classifier or LoD.