Accepted for/Published in: JMIR AI
Date Submitted: May 17, 2023
Open Peer Review Period: May 17, 2023 - Jul 12, 2023
Date Accepted: Oct 15, 2023
(closed for review but you can still tweet)
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Is AI in General Practice a Solution and/or a Risk? A discussion on a conversation with ChatGPT and Microsoft Bing and the use of these models for research. A discussion.
ABSTRACT
Background:
This paper investigates whether these AI models can autonomously generate high-quality academic papers within general practice contexts, assessing coherence, research quality, and evidence-based content, while addressing ethical implications and limitations.
Objective:
This paper investigates whether these AI models can autonomously generate high-quality academic papers within general practice contexts, assessing coherence, research quality, and evidence-based content, while addressing ethical implications and limitations.
Methods:
This study evaluates ChatGPT-4 and Microsoft Bing's performance in assisting with an academic paper on general practice. A prompt was designed to ensure formal and professional responses. Data was collected through interviews, with both models asked to provide a discussion article outline. Evaluation criteria included relevance, accuracy, clarity, and tone/style. AI-generated responses were analyzed independently and comparatively, with results used to determine each model's strengths, weaknesses, and potential areas for improvement.
Results:
Comparing ChatGPT-4 and Microsoft Bing, ChatGPT-4 provides a comprehensive, relevant analysis of AI in healthcare, while Microsoft Bing offers a brief overview. ChatGPT-4 cites 72% accurate peer-reviewed articles, while Microsoft Bing cites 46%. Both models demonstrate clarity, coherence, and appropriate tone for academic papers; however, Microsoft Bing could benefit from providing more details and examples.
Conclusions:
Comparing ChatGPT-4 and Microsoft Bing for academic writing assistance, ChatGPT-4 demonstrates superior relevance and depth, while Microsoft Bing offers conciseness. Both models require improvement. Merging strengths can yield comprehensive answers and up-to-date references. Current AI cannot independently author research articles, but future advancements may enable autonomous research creation. Researchers should critically evaluate AI-assisted outputs and corroborate information to maintain academic rigor.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.