JMIR Publications

ABSTRACT

Background:

Large language models (LLMs) are increasingly used in scientific writing, but the conditions under which they can be applied responsibly to evidence synthesis remain poorly defined.

Objective:

To determine which discrete components of the systematic review process can be defensibly delegated to an LLM and which continue to require human judgment.

Methods:

We conducted a two-part proof-of-concept study of selected review subtasks rather than of the systematic review as a whole. First, using a recently published umbrella review as a benchmark, we compared LLM screening decisions with those of the original human authors at both the abstract and full-text stages, applying the published eligibility criteria verbatim and then after iterative rewording; concordance was summarized as percentage agreement and Cohen's κ. Second, in an exploratory blinded evaluation, six board-certified hematopathologists rated two LLM-produced reviews and one published human-authored review on the same topic (5-point scale) and judged whether each had been AI-generated; attribution accuracy was compared against chance by permutation test. We additionally audited each cited claim in the LLM-produced reviews for citation misattribution, with and without text-restriction strategies.

Results:

At the abstract stage, the model adhered to its stated inclusion criteria in all cases examined. At full text, agreement with the human authors reached 86.2% (κ = 0.38), but only after the published criteria had been iteratively reworded; applied verbatim, those criteria led the model to reject every article the original authors had included, indicating that criteria wording rather than model capability alone governed performance. Blinded ratings favored the LLM-produced reviews (means 3.40–3.66 vs 2.60 of 5), and raters did not identify AI involvement above chance (27.8% accuracy; permutation P = 0.84), with the human-authored review most frequently attributed to AI. Citation misattribution affected 4.13–7.06% of cited claims, which text-restriction strategies reduced but did not eliminate.

Conclusions:

LLMs can perform bounded, well-specified subtasks of review production under supervision, whereas verification, disclosure, and final adjudication remain human responsibilities. Given the small rater sample, the evaluation findings are hypothesis-generating rather than definitive. Informed by these observations, we developed ReviewFoundry, an open-source platform supporting human-supervised, AI-assisted review workflows.