JMIR Publications

ABSTRACT

Background:

Virtual patients that use natural language processing provide a valuable educational experience for learners. Generating a large, varied sample of realistic and appropriate responses for virtual patients is challenging. Artificial intelligence (AI) programs can be a viable source for these responses, but their utility for this purpose has not been explored.

Objective:

We explored the utility of generative AI (ChatGPT) for developing realistic virtual simulated patient (VSP) dialogue to teach prenatal counseling skills.

Methods:

ChatGPT was prompted to generate a list of common areas of concern (AOC) and questions that families expecting preterm delivery at 24 weeks gestation might ask during prenatal counseling. ChatGPT was then prompted to generate two role plays with dialogue between a parent expecting a potential preterm delivery at 24 weeks and their counseling physician using each of the example questions. The prompt was repeated for two unique role plays; one parent was characterized as anxious and the other as having low trust in the medical system. Role play scripts were exported verbatim and independently reviewed by 2 neonatologists with experience in prenatal counseling using a scale of 1-5 on realism, appropriateness, and utility for VSP responses.

Results:

ChatGPT generated 7 areas of concern with 28 example questions. The 35 role play transcripts generated 176 unique parent responses (median 5, IQR 4-6 per role play) with 268 unique sentences. Expert review identified 117 (65%) of the responses as indicating an emotion, either directly or indirectly. Half of the responses had two or more sentences, and half included at least one question. Over half (104, 58%) of role play parent character responses described a feeling e.g., scared, worried, or concerned. The role play of the parent with low trust in the medical system generated the most number of unique sentences (n=50). The majority of responses were found to be realistic, appropriate for variable conversation paths, and usable in a virtual patient program.

Conclusions:

Generative AI programs such as ChatGPT may provide a viable source of training materials to expand virtual patient programs, with careful attention to the concerns and questions of patients and families. Given the potential for unrealistic or inappropriate statements and questions, an expert should review AI chat outputs before deploying them in an educational program.