JMIR Publications

ABSTRACT

Background:

One aspect of AI that has gained significant attention is natural language processing (NLP) which refers to the ability of computers to understand and generate human language

Objective:

This study aimed to examine the potential and concerns of using Artificial Intelligence (AI) in scientific research by generating high-impact research articles by analyzing the quality of articles generated by ChatGPT and assessing its impact on research framework, data analysis, and literature review. The study also explored concerns around ownership and integrity of research when using AI-generated text.

Methods:

Four articles were generated using ChatGPT and evaluated by 23 reviewers. The researchers developed an evaluation form to assess the quality of the articles generated. Additionally, 500 abstracts were generated using ChatGPT and evaluated for their quality. The data were analyzed using ANOVA and thematic analysis to analyze the qualitative data provided by the reviewers.

Results:

Using detailed prompts with the context of the study would generate high-quality research that can be published in high-impact journals. However, ChatGPT had a slight impact on developing the research framework and data analysis, with the main improvement needed in developing the literature review. Moreover, concerns around ownership and integrity of research when using AI-generated text were highlighted. Nonetheless, ChatGPT has a high potential in increasing human productivity in research and can be used in academic writing.

Conclusions:

AI-generated text has the potential to improve the quality of high-impact research articles, and the study recommends changing the research template to focus only on research methodology and data analysis in-depth, which can be the trend for the next generation of research in the AI era.