JMIR Publications

ABSTRACT

Background:

Distinguishing autism spectrum disorder (ASD) from social communication disorder (SCD) is clinically challenging because both conditions present with overlapping social communication deficits. Standard caregiver-reported instruments capture surface-level behavioral similarities rather than underlying cognitive differences, motivating the development of digital gamified assessments that measure social cognitive processes directly.

Objective:

This study aimed to develop and evaluate a 2-stage digital assessment pipeline. Stage 1 utilizes a gamified self-report tool (Buddy Plan) for high-sensitivity ASD screening. Stage 2 employs story-based social judgment scenarios (Buddy Drill) to differentiate ASD from SCD.

Methods:

A cross-sectional study was conducted with 275 participants aged 6 to 18 years across 5 diagnostic groups, including ASD (n=51) and SCD (n=54). Participants completed the Buddy Plan (52 items) and Buddy Drill (153 scenarios) modules. A nested cross-validation framework (5×5 stratified folds) was used to train 4 machine learning algorithms. A correlation-based item selection procedure inspired by item response theory (IRT) was applied to select 50 highly discriminative scenarios for Stage 2. Model explainability was assessed using SHapley Additive exPlanations (SHAP).

Results:

For Stage 1 screening (ASD vs neurotypically developing; n=146; ASD n=51, ND n=95), the random forest model achieved a nested area under the receiver operating characteristic curve (AUC) of 0.912 (SD 0.054; bootstrap 95% CI 0.856–0.953). At an optimized threshold of 0.200, the model yielded 96.1% (49/51) sensitivity and 58.9% (56/95) specificity, with 2 ASD false negatives (2/51). For Stage 2 differential diagnosis (ASD vs SCD; n=104 analyzed; ASD n=51, SCD n=53 [1 SCD participant excluded for incomplete Buddy Drill responses]), a regularized logistic regression model using 50 psychometrically selected items achieved a nested AUC of 0.767 (SD 0.111; bootstrap 95% CI 0.682–0.867; 70.6% sensitivity [36/51]; 75.5% specificity [40/53]). Combining self-report items with objective scenarios degraded Stage 2 performance (ΔAUC=−0.069 to −0.090). SHAP analysis identified 5 consensus biomarkers driving Stage 1 classifications, showing predictive value without sex-based bias.

Conclusions:

A 2-stage digital pipeline shows promise for initial ASD screening and moderate, yet clinically informative, accuracy for differentiating ASD from SCD. Combining subjective ratings with objective cognitive tasks degraded differential performance, suggesting that performance-based social reasoning metrics alone yield a clearer diagnostic signal. External validation is needed before clinical deployment.