Previously submitted to: JMIR Dermatology (no longer under consideration since Aug 06, 2024)
Date Submitted: Jul 13, 2024
Open Peer Review Period: Aug 6, 2024 - Aug 6, 2024
(closed for review but you can still tweet)
NOTE: This is an unreviewed Preprint
Warning: This is a unreviewed preprint (What is a preprint?). Readers are warned that the document has not been peer-reviewed by expert/patient reviewers or an academic editor, may contain misleading claims, and is likely to undergo changes before final publication, if accepted, or may have been rejected/withdrawn (a note "no longer under consideration" will appear above).
Peer review me: Readers with interest and expertise are encouraged to sign up as peer-reviewer, if the paper is within an open peer-review period (in this case, a "Peer Review Me" button to sign up as reviewer is displayed above). All preprints currently open for review are listed here. Outside of the formal open peer-review period we encourage you to tweet about the preprint.
Citation: Please cite this preprint only for review purposes or for grant applications and CVs (if you are the author).
Final version: If our system detects a final peer-reviewed "version of record" (VoR) published in any journal, a link to that VoR will appear below. Readers are then encourage to cite the VoR instead of this preprint.
Settings: If you are the author, you can login and change the preprint display settings, but the preprint URL/DOI is supposed to be stable and citable, so it should not be removed once posted.
Submit: To post your own preprint, simply submit to any JMIR journal, and choose the appropriate settings to expose your submitted version as preprint.
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
ChatGPT and The Suspicion of Skin Cancer, a Diagnostic Accuracy Study
ABSTRACT
Background:
While ChatGPT is user-friendly and widely accessible, concerns arise regarding potential delays in diagnosis and false reassurances for patients with suspected skin malignancies.
Objective:
Our study aims to assess the accuracy of AI, specifically ChatGPT, in diagnosing skin malignancies and expressing the urgency to seek medical advice.
Methods:
This diagnostic accuracy study assesses the agreement between dermatologists' final diagnoses and those provided by ChatGPT when patients describe their lesions. Thirty-five patients, suspected of skin cancer (SCC/BCC), provided demographic details and lesion descriptions. Diagnoses were recorded in ChatGPT3.5 and ChatGPT4.0 for analysis.
Results:
Out of 35 lesions suspected by the dermatologist, all were malignant, indicating 100% accuracy. ChatGPT3.5 flagged malignancy in 7 cases (20%), while ChatGPT4.0 did so in 6 cases (17.14%). Consistency was lacking, as only 7 lesions received the same diagnosis from both models. However, both ChatGPT3.5 and ChatGPT4.0 referred patients to dermatologists in all cases.
Conclusions:
Both GPT models performed comparably to each other but were significantly inferior to dermatologists. However, both did not cause delays in referral to a dermatologist. The limitations of these two models include poor accuracy, lack of concordance among each other’s, and reproducibility issues with their answers.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.