JMIR Publications

ABSTRACT

Background:

Explainer videos are an established digital learning medium in higher education. With the increasing use of AI-generated avatars, it remains unclear whether different presentation formats—human presenters versus AI avatars—affect learning outcomes and user experience. Empirical evidence on this question is limited, particularly in engineering education contexts.

Objective:

This study aimed to assess the feasibility of a randomized crossover design to examine learning gains and user experience associated with content-identical explainer videos presented by an AI-generated avatar or a human presenter. Exploratory analyses examined whether short-term learning gains and user experience differed across presentation formats.

Methods:

An observer-blinded, prospective, randomized, noninterventional crossover study was conducted. Thirteen undergraduate engineering students (n=13) watched two content-identical explainer videos introducing fuel cell technology, presented either by an AI-generated avatar or a human presenter in randomized order. Learning gain was assessed using a self-developed 7-item knowledge test at three time points (baseline, after video 1, after video 2). User experience was measured after each video using the standardized AttrakDiff2 questionnaire. Due to the small sample size, inferential analyses were performed using Mann-Whitney U tests.

Results:

Viewing both explainer videos led to clear descriptive learning gains in both groups. During the first learning period, students who watched the AI avatar video achieved a median learning gain of 5 newly correct items. In contrast, students who watched the human-presented video achieved a median increase of 4.5 items. This difference was not statistically significant (Mann-Whitney U=17.5; p=.61; effect size r=.14). No significant differences were observed in the second learning period. Regarding user experience, the human-presented video was consistently rated higher on a descriptive level, particularly in pragmatic quality, hedonic quality related to identification, and overall attractiveness. The AI-based presentation was perceived as largely neutral. None of these differences reached statistical significance.

Conclusions:

This study primarily aimed to assess methodological feasibility rather than to establish definitive effects. The findings indicate that the applied study design is feasible and suitable for investigating learning gain and user experience in AI-based explainer videos. Within this small feasibility sample, no statistically detectable differences in short-term learning gains or user experience were observed between presentation formats. However, given the limited statistical power, these findings should not be interpreted as evidence of equivalence or comparable effectiveness. Instead, they demonstrate that the study design and procedures are feasible and can inform the design of future, adequately powered studies to investigate potential differences. This formative study contributes to the growing evidence base on the educational use of AI-generated avatars in higher education. Clinical Trial: Not applicable (exploratory feasibility study without clinical intervention).