Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Generative AI for Assessment and Feedback in Health and Life Sciences Higher Education: A Systematic Review of Feedback Quality, Grading Accuracy, and Human Oversight
ABSTRACT
Background:
Generative artificial intelligence (AI) tools are being deployed to grade student assessments and provide feedback across health and life sciences higher education, yet no systematic review has synthesized evidence on their accuracy, feedback quality, and governance requirements in this context.
Objective:
To synthesize evidence on the feedback quality, grading accuracy, rater agreement, and human oversight mechanisms of generative AI tools used for assessment and feedback in health and life sciences higher education.
Methods:
A systematic review was conducted following PRISMA 2020 guidelines and registered on PROSPERO. Five databases (PubMed, Scopus, CINAHL, ERIC, IEEE Xplore) were searched until June 2026. Studies involving generative AI used for grading or providing feedback on work produced by students in undergraduate or postgraduate health and life sciences programs were eligible. Two independent reviewers screened records (Cohen’s κ = 0.76), extracted data, and assessed methodological quality using the Mixed Methods Appraisal Tool (MMAT) version 2018. Certainty of evidence was assessed using the GRADE framework. Findings were synthesized narratively.
Results:
Fifty-three studies published between 2023 and 2026 were included, spanning 27 countries and five health disciplines. Medicine was the predominant discipline (66.0%), and cross-sectional designs predominated (67.9%). Agreement values ranged from κ = −0.156 to ICC = 0.993. Assessment type was the primary moderator of AI-human agreement: simulation and history-taking assessments with structured rubrics consistently achieved excellent agreement (ICC 0.90–0.993), whilst visually-dependent and attitudinal OSCE domains yielded poor agreement. Prompting strategy was the most actionable modifiable determinant within any assessment format. Feedback quality was broadly acceptable to students, with one RCT demonstrating non-inferiority of AI versus expert feedback on learning outcomes (p = 0.06). Human oversight was reported in 94.3% of studies; no study recommended autonomous AI assessment. Methodological quality was predominantly high (50.9%) or medium (45.3%); GRADE certainty was Low to Very Low across all outcome domains.
Conclusions:
Generative AI can achieve excellent grading accuracy for structured health professions assessments but is not yet reliable for complex clinical competencies. A tiered hybrid model, AI for structured formative assessment with mandatory human oversight for summative and complex tasks, is supported by the evidence. Future research prioritizing randomized trials, non-medicine disciplines, and equity considerations are needed before large-scale deployment can be recommended. Clinical Trial: PROSPERO CRD42019123456
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.