JMIR Publications

ABSTRACT

Artificial intelligence (AI) has revolutionized biomedicine, driving advancements in diagnostics, treatment, and medical data management. Breast cancer, a significant global health challenge, highlights the need for accessible and reliable medical information. AI platforms like ChatGPT-4.0 and DeepSeek-V3 have become essential tools for delivering curated medical insights. This study compared ChatGPT-4.0 and DeepSeek-V3 in retrieving and presenting medical information, focusing on readability, content quality, and reliability of information sources. Using Flesch-Kincaid Grade Level (FKGL) and a 7-point Likert scale, the analysis revealed that AI models often produce simpler responses than expert references, improving accessibility but risking oversimplification. ChatGPT demonstrated greater consistency and improved readability in multi-response scenarios, excelling in clarity and depth, while DeepSeek aligned more closely with expert reference readability in single-instance analysis but showed higher variability. DeepSeek excelled in citation efficiency and global reference diversity, but faced challenges like untagged links, corrupted references, and occasional downtime. Despite these differences, statistical analysis showed no significant differences between the models, particularly in larger datasets. Both models provided reliable information, but no single model consistently matched expert content across all questions. The findings reveal that no single AI model consistently matches expert content across all questions, emphasizing the need for careful evaluation to ensure AI-generated information meets diverse user needs. Future improvements should focus on enhancing link accessibility, platform stability, and response consistency to optimize AI-generated medical content for healthcare applications.