Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Currently submitted to: Journal of Medical Internet Research

Date Submitted: Feb 20, 2026
Open Peer Review Period: Feb 21, 2026 - Apr 18, 2026
(currently open for review)

Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.

Methodological Considerations for Conducting an AI-Assisted Systematic Literature Review: The Importance of a Human-in-the-Loop Approach to Maintain Scientific Rigor

  • Ṣẹ̀yẹ AboÌ€gúnrÌ€in; 
  • Marie Lane; 
  • Gemma Carter; 
  • Cornelis Boersma; 
  • Jurjen van der Schans; 
  • Artur Nowak

ABSTRACT

Systematic literature reviews (SLRs) are critical for evidence synthesis and play a central role in supporting research, policy development, and evidence‑based decision‑making across healthcare and related disciplines. However, traditional SLRs are resource-intensive and time-consuming, often requiring months of manual effort to screen thousands of records, extract data, and maintain methodological rigor. As global scientific output continues to grow exponentially, these operational challenges have intensified, contributing to longer completion timelines, greater workforce burden, and a heightened risk that reviews become outdated shortly after publication. In response, there is growing interest in artificial intelligence (AI) approaches to enhance the efficiency and scalability of SLRs. AI tools now offer support for a broad range of review tasks, including assisting with search strategy development, identifying relevant concepts, prioritizing records for screening, and supporting data extraction and risk‑of‑bias assessments. AI can significantly accelerate labor-intensive stages, reduce human error during repetitive tasks, and enable the synthesis of evidence bases that might otherwise be impractical to review manually. However, these efficiencies must be balanced against the risks associated with AI, including bias, lack of transparency, variable outputs, and hallucinations (outputs that appear plausible but are factually incorrect). A human-in-the-loop approach is therefore essential to validate AI outputs and maintain scientific integrity. Human expertise remains critical for defining research questions, validating search strategies, confirming study eligibility, interpreting nuanced data, and making final judgments on quality and risk of bias. Clear methodological guidance is required to support teams in integrating AI tools responsibly, transparently, and reproducibly into SLR workflows. Methodological considerations include selecting appropriate tools, defining oversight strategies, and applying performance metrics such as precision and recall. This paper aims to provide methodological guidance on the effective integration of AI into each stage of the SLR process, drawing on both published literature and the authors’ real-world experience. We outline key considerations for selecting and implementing AI tools while maintaining human oversight. We also discuss how to maintain transparency, auditability, and alignment with established standards, including PRISMA‑P, PRISMA‑trAIce, and emerging guidance from regulators and health technology assessment bodies. We also present future directions for responsible AI use in SLRs. AI should complement, not replace, human judgment. When implemented within a human-in-the-loop framework, AI has the potential to accelerate evidence synthesis, enabling faster, scalable, and rigorous reviews while preserving transparency and reproducibility.


 Citation

Please cite as:

AboÌ€gúnrÌ€in á, Lane M, Carter G, Boersma C, van der Schans J, Nowak A

Methodological Considerations for Conducting an AI-Assisted Systematic Literature Review: The Importance of a Human-in-the-Loop Approach to Maintain Scientific Rigor

JMIR Preprints. 20/02/2026:93165

DOI: 10.2196/preprints.93165

URL: https://preprints.jmir.org/preprint/93165

Download PDF


Request queued. Please wait while the file is being generated. It may take some time.

© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.