JMIR Publications

ABSTRACT

Background:

Large Language Models (LLMs) have raised both interest and concern in the academic community. They offer potential for automating literature search and synthesis for systematic reviews but concerns regarding their reliability and the tendency to generate unsupported ('hallucinated') content persist.

Objective:

To assess the performance of Large Language Models like ChatGPT and Bard to produce references in the context of scientific writing.

Methods:

The performance of ChatGPT and Bard in replicating the results of human-conducted systematic reviews was assessed. Using systematic reviews pertaining to shoulder rotator cuff pathology, these LLMs were tested by providing the same inclusion criteria and comparing the results with original systematic review references. The study utilized three key performance metrics: Recall, Precision, and F1-score, alongside the hallucination rate. Articles were considered “hallucinated” if any two of the following information were wrong: title, first author, or year of publication.

Results:

The LLMs could generate legitimate references, but also produced hallucinated articles at a rate between 28% to 91%. Although ChatGPT 4 demonstrated superior performance among the models tested, all failed significantly in adhering to the established eligibility criteria. Precision and recall ranged from 0% to 13.4% and 0% to 14.7% respectively, highlighting the limitations of these models in replicating human-conducted systematic reviews.

Conclusions:

Given their current performance, it is not recommended for LLMs to be deployed as the primary or exclusive tool for conducting systematic reviews. Any references generated by such models warrant thorough validation by researchers. The high occurrence of hallucinations in LLMs highlights the necessity for refining their training and functionality before confidently employing them for rigorous academic purposes