Previously submitted to: JMIR Pediatrics and Parenting (no longer under consideration since Feb 04, 2026)
Date Submitted: Feb 2, 2026
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Assessing the Accuracy of ChatGPT Responses to Patient Questions in Pediatric Neurology: An Observational Study
ABSTRACT
Background:
ChatGPT has seen increased use by caregivers as a source of medical information. Pediatric neurology has added complexities because there are various disorders, changing diagnoses and potential consequences of misinformation.
Objective:
To evaluate the accuracy, comprehensiveness and reproducibility of ChatGPT responses to frequently asked pediatric neurology questions.
Methods:
In total, 105 unique questions across seven pediatric neurology subspecialties were included. Every question was input twice into ChatGPT (GPT-5) during separate sessions. Two certified pediatric neurologists reviewed the responses and rated them on a 4-point scale: 1-complete and accurate,2- correct but not complete,3- partially incorrect,4-not correct. Reproducibility was considered to be agreement between grading categories (grade 1-2 and grade 3-4) throughout repeated responses.
Results:
60.95%(64/105) of responses were rated as comprehensive, while 31.43%(33/105) were correct but incomplete; fully incorrect responses were uncommon 3.8%(4/105). Accuracy and reproducibility were highest for structured conditions such as cerebral palsy and movement disorders, and lower for more heterogeneous domains including neurodevelopmental delay and epilepsy. Overall reproducibility was high 95.2%(100/105).
Conclusions:
ChatGPT mostly delivers accurate information in pediatric neurology, with variable depth of response depending on the disorder. It is most appropriate for caregiver education with oversight by the clinician. Clinical Trial: not applicable
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.