JMIR Publications

ABSTRACT

Background:

Autism Spectrum Disorder (ASD) is a developmental delay characterized by impairments to language and social-emotional behaviors. The incidence of ASD has increased in recent years; it is now estimated that roughly 1 in 40 children in the United States are affected. Due in part to increasing prevalence, access to treatment has become constrained. Hope lies in mobile solutions that provide therapy through artificial intelligence (AI) approaches, including facial and emotion detection AI models available from mainstream cloud providers available direct-to-consumer. However, these may not be sufficiently well trained for use in pediatric populations.

Objective:

Emotion classifiers are publicly available from Microsoft, Amazon, Google, and SightHound off-the-shelf and if well suited to the pediatric population could be used for developing mobile therapies, perhaps accelerating innovation in this space. Our objective was to test this directly with image data from children with self-reported ASD recruited through crowdsourcing.

Methods:

We used a mobile game called Guess What?, which challenges a child to act out a series of prompts displayed on the screen of the smartphone held on the forehead of his/her care provider. This game is a fun and engaging way for child and parent to interact socially, e.g. with the parent attempting to guess what emotion the child is acting (e.g. surprised, scared, or disgusted). During a 90-second game session, as many as 50 prompts will be shown while the child acts and the video records the actions and expressions of the child. Due in part to the fun nature of the game, it is a viable way to engage pediatric populations including the autism population remotely through crowdsourcing. We recruited 21 children with ASD to play the game and gathered 2602 emotive frames following their game sessions. We used these data to evaluate the accuracy and performance of four state-of-the-art facial emotion classifiers, to develop an understanding of the feasibility of these platforms for pediatric research.

Results:

All classifiers performed poorly for every evaluated emotion except "happy". None of the evaluated classifiers correctly labeled over 60% of the 2602 evaluated frames. Moreover, none of the classifiers correctly identified more than 15% of the frames within the "angry" and "disgust" categories.

Conclusions:

Results suggest that commercial emotion classifiers may be insufficiently trained for use in digital approaches to autism treatment and treatment tracking. Secure, privacy-preserving methods to increase labeled training data will be needed to boost the models’ performance before they can be used in AI-enabled approaches to social therapy of the kind common in autism treatments.