ChatGPT vs. DeepSeek: A Comparative Analysis of AI Models for Breast Cancer Information Retrieval
ABSTRACT
Background:
Artificial intelligence (AI) has revolutionized biomedicine, driving advancements in diagnostics, treatment, and medical data management. Breast cancer, a major global health challenge, highlights the need for accessible and reliable medical information. AI platforms like ChatGPT-4.0 and DeepSeek-V3 have become essential tools for delivering curated medical insights.
Objective:
This study aimed to compare ChatGPT-4.0 and DeepSeek-V3 in retrieving and presenting medical information, with a focus on readability, content quality, and reliability of information sources.
Methods:
The study employed the Flesch-Kincaid Grade Level (FKGL) to assess readability and a 7-point Likert scale to evaluate content quality and reliability. The analysis compared AI-generated responses to expert references, examining consistency, readability, citation efficiency, and global reference diversity.
Results:
AI models often produced simpler responses than expert references, improving accessibility but risking oversimplification. ChatGPT-4.0 demonstrated greater consistency and improved readability in multi-response scenarios, excelling in clarity and depth. DeepSeek-V3 aligned more closely with expert reference readability in single-instance analysis but showed higher variability. DeepSeek-V3 excelled in citation efficiency and global reference diversity but faced challenges such as untagged links, corrupted references, and occasional downtime. Statistical analysis revealed no significant differences between the models, particularly in larger datasets. Neither model consistently matched expert content across all questions.
Conclusions:
The findings reveal that no single AI model consistently matches expert content across all questions, emphasizing the need for careful evaluation to ensure AI-generated information meets diverse user needs. Future improvements should focus on enhancing link accessibility, platform stability, and response consistency to optimize AI-generated medical content for healthcare applications.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.