JMIR Publications

ABSTRACT

Background:

Autism spectrum disorder (ASD) is a widespread neurodevelopmental condition with a range of potential causes and symptoms. Standard diagnostic mechanisms for ASD, which involve lengthy parent questionnaires and clinical observation, often result in long waiting times for results. Recent advances in computer vision and mobile technology give rise to the possibility of expediting the diagnostic process by computationally analyzing behavioral and social impairments from home videos.

Objective:

In this work, we evaluate whether home videos collected from a game-based mobile application can be used to provide diagnostic insights into ASD. To the best of our knowledge, this is the first study that attempts to identify potential indicators of ASD from mobile phone videos without the use of eye-tracking hardware, manual annotations, and structured clinical environments.

Methods:

Here, we use a mobile health application to collect over 11 hours of video footage depicting 95 children engaged in gameplay in a natural home environment. We utilize automated dataset annotations to analyze two social indicators that have previously been shown to differ between children with ASD and their neurotypical (NT) peers: (1) gaze fixation patterns, which represent regions of an individual’s visual focus, and (2) visual scanning methods, which refer to the ways in which individuals scan their surrounding environment. We compare the gaze fixation and visual scanning methods utilized by children during a 90-second gameplay video in order to identify statistically significant differences between the two cohorts; we then train an LSTM neural network in order to determine if gaze indicators could be predictive of ASD.

Results:

Our work shows that gaze fixation patterns differ between the two cohorts; specifically, we identify one statistically significant region of fixation (P = .00015). In addition, we also demonstrate that there are unique visual scanning patterns that exist for individuals with ASD when compared to NT children (P = .00011). A deep learning model trained on gaze fixation coordinates demonstrates mild predictive power in identifying ASD based on coarse gaze fixation annotations.

Conclusions:

Ultimately, our study demonstrates that heterogeneous video datasets collected from mobile devices hold potential for quantifying visual patterns and providing insights into ASD. We show the importance of automated labeling techniques in generating large-scale datasets while simultaneously preserving the privacy of participants, and we demonstrate that specific social engagement indicators associated with ASD can be identified and characterized using such data.