JMIR Publications

ABSTRACT

We appreciate Mr. Wang and his colleague's keen interest in our publication [1]. The responses to his comments are as follows. First, Mr. Wang and his colleague argued potential biases from the eight experts' personal preferences, academic or cultural backgrounds, or their familiarity with the subject matter. However, the 30 papers were selected from three different journals and explored various topics (Table S3 in the supplementary data). Therefore, these biases are unlikely to occur due to the significant variation among the articles. Second, we did not use ChatGPT 3.5. We used ChatPDF, which is based on ChatGPT 3.5 but can efficiently analyze all PDF contents. Third, Mr. Wang and his colleague argued that if AI assistance in refining an article does not alter the accuracy of the research and aids in clearer communication of experimental content, such practice should be deemed acceptable. However, in our study, we found that 3 of the 30 ChatGPT-generated abstracts showed wrong conclusions. Finally, Mr. Wang and his colleague are right in cautioning against complete reliance on GPT for labor-intensive tasks, which may lead to a lack of critical thinking and the production of low-quality or erroneous articles. Indeed, when using GPTzero [2] to check their letter, the probability of AI-written text is 81%, not 100%. However, their perspective differs from the purpose of our study. We aimed to assess the applicability of an AI model in generating abstracts for basic preclinical research. Our conclusion is that the quality of the ChatGPT-generated abstracts of basic preclinical research was suboptimal, and the accuracy was not 100%.