JMIR Publications

ABSTRACT

Background:

Case-based scenarios as multiple-choice questions, true or false assessments, and correlation exercises, are widely used for assessing medical students in different aspects of medicine including dermatology domain. Students generally value exposure to a diverse range of lesions, conditions, and diagnostic challenges that closely mirror real-world clinical practice.

Objective:

Our study aims to evaluate the capability of chatbots in generating engaging and educational case reports in dermatology.

Methods:

A questionnaire was developed featuring a mix of AI-generated and non-AI cases and was filled by 45 medical students with the same and required academic grade.

Results:

Findings revealed that 80% of the students-participants of the survey reported using ChatGPT-4, and 56% reported chatbot utilization in their studies. Results showed that students generally struggled to reliably identify the origin of cases (AI vs. non-AI), while in case of conflicting information between a chatbot answer and an Internet source, 68,9% reported that would follow the latter indicating skepticism towards AI-generated text. AI-created cases were characterized by detailed case descriptions, analysis of medical history, clinical examinations, and follow-up questions but lacked the depth, clinical relevance, and motivational elements found in non-AI cases. Non-AI cases were shorter, presented clinical dilemmas, offered direct questions, and included challenging scenarios that students found more educational and engaging

Conclusions:

The study underscores the importance of maintaining clinical and practical relevance in medical education materials and discourages tutors to rely blindly on chatbots for the quick production of exercises for students.