Accepted for/Published in: JMIR Formative Research
Date Submitted: Oct 2, 2025
Date Accepted: Jun 8, 2026
From Click to Chat: A Comparative, Bilingual Study of Autism-Related Information on Websites and General-Purpose AI Chatbots
ABSTRACT
Background:
Parents increasingly consult the internet —both websites and, more recently, AI chatbots—for information on autism spectrum disorder (ASD). Yet the comparative quality of these two source types, especially across languages, remains under-explored.
Objective:
We aimed to assess the completeness and accuracy of ASD information delivered by websites and five popular AI chatbots, and determine whether performance differs between English and Romanian content.
Methods:
In a cross-sectional design, we evaluated 25 English-language and 25 Romanian-language websites and the responses of ChatGPT, Gemini, Claude, Copilot, and DeepSeek. Content was benchmarked against a 24-item checklist, yielding completeness and accuracy scores. Chatbots were tested in two scenarios: a single broad query and 24 item-specific queries.
Results:
Websites achieved higher completeness in English than in Romanian (6.9 vs 5.1; p=0.007) and marginally higher accuracy (6.9 vs 6.1; p=0.045). In Scenario A, chatbots’ completeness (English 5.3 vs Romanian 6.2; p= 0.151), and accuracy (English 6.0 vs Romanian 5.6; p= 0.322) did not show significant differences by language. In the single-query scenario, websites showed higher accuracy than chatbots in both English (6.9 vs 6.0, p=0.19) and Romanian (6.1 vs 5.6, p=0.532), with neither difference reaching statistical significance. Conversely, item-specific questioning favored chatbots, which outperformed websites in English (8.3 vs 6.9, p=0.053) and—significantly—in Romanian (8.0 vs 6.1, p=0.007). Accuracy rose significantly from the single-query to the item-specific scenario (English: 6.0 vs 8.3, p=0.002; Romanian: 5.6 vs 8.0, p=0.001). Between-chatbot variation was statistically significant (One-way repeated measures ANOVA p=0.005), with Gemini and DeepSeek showing superior scores (p=0.016 and 0.023 respectively).
Conclusions:
The quality of online ASD information varies by source type and language. English-language websites were more complete than Romanian ones. Targeted questioning of chatbots yielded significantly more accurate answers in both languages.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.