JMIR Publications

ABSTRACT

Background:

Serious games are increasingly used to study and enhance team performance in organizational and educational settings. While prior research has explored leadership and communication as isolated factors, the multivariate interactions between behavioral indicators remain poorly understood. A deeper understanding of these relationships can reveal which behavioral and demographic factors most strongly predict successful outcomes, offering insights relevant to both scientific research and practical training design.

Objective:

This study aimed to develop machine learning models to predict team success in serious games. Specifically, it sought to identify the behavioural and demographic predictors that most strongly influence team performance outcomes.

Methods:

This study employed a cross-sectional retrospective design. Behavioural and demographic data were analysed from 233 teams participating in escape-room-based serious games delivered by JGM Serious eXperiences in the Netherlands. Teams of 2-8 players (mean age 25.8 years; 53 all-male, 55 all-female, 125 mixed-gender) were scored by trained observers across collaboration, communication, and leadership constructs using Likert-scale indicators. Exploratory Data Analysis (EDA) compared winning (n=141) and losing teams (n=92) using descriptive statistics, Pearson correlations, and significance testing (independent-samples t tests and Mann–Whitney U tests). Mean differences were interpreted with 95% confidence intervals. Four ML models: Logistic Regression (LR), Random Forest (RF), Multi-Layer Perceptron (MLP), and Support Vector Classifier (SVC), were trained using 5-fold cross-validation (F1-score). The best model was interpreted using Shapley Additive Explanations (SHAP).

Results:

Winning teams scored higher on several behavioural constructs, but only four: knowledge sharing, leadership, guidance, and extraversion, showed statistically significant differences between winners and losing teams. These effects were supported by 95% confidence intervals, Shapiro–Wilk tests for normality, and Mann–Whitney U tests where assumptions were violated, indicating that only a subset of behavioural indicators meaningfully distinguishes successful teams. Among the machine learning models, Logistic Regression achieved the highest accuracy (88%), followed by MLP (87%), RF (87%), and SVC (85%). SHAP analysis showed that gender composition and prior escape-room experience were the strongest demographic predictors of success, while “celebrating progress” (extern5) and “taking initiative when the team is stuck” (sturing5) were the most influential behavioural indicators.

Conclusions:

This work demonstrates the usefulness of multivariate analysis in studying and understanding complex human behaviour in serious game environments as opposed to studying isolated behavioural indicators, often described in previous studies. The machine learning models developed using behavioural and demographic features of participating teams showed promising accuracies and their interpretation led to unveiling a set of demographic and behavioural components as most decisive factors leading to team success. This improved understanding of what makes a team win can be potentially translated in terms of improved productivity in business and organizational settings.