Accepted for/Published in: JMIR Formative Research
Date Submitted: May 30, 2023
Date Accepted: Oct 29, 2023
AI ChatGPT model GPT-4 for scientific writing. Strengths and weaknesses explored employing medical vitamin B12 topic.
ABSTRACT
Background:
ChatGPT is a large language model developed by OpenAI, designed to generate human-like responses to prompts
Objective:
This study aims to evaluate the ability of GPT-4 in generating scientific content and assisting in scientific writing employing vitamin B12 as a case. Furthermore, the study will compare the performance of GPT-4 to its predecessor, GPT-3.5.
Methods:
The study examined responses from GPT-4 and GPT-3.5 to Vitamin B12-related prompts, focusing on their quality and characteristics and comparing them to established scientific literature.
Results:
Results indicate GPT-4 can potentially streamline scientific writing through its ability to do language editing, abstracts, keyword, and abbreviation lists. However, significant limitations of ChatGPT are revealed, including its inability to identify and address bias, inability to include recent information, lack of transparency, and inaccurate information. Additionally, it cannot check for plagiarism or provide proper references. The accuracy of GPT-4's answers is found to be superior to GPT-3.5.
Conclusions:
ChatGPT can be considered a helpful assistant in the writing process, not a replacement for a scientist's expertise. Researchers must remain aware of its limitations and use it appropriately. The improvements in consecutive ChatGPT versions suggest the possibility of overcoming some present limitations in the near future.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.