JMIR Publications

ABSTRACT

Dear Editor, Talwalkar and colleagues addressed a practical problem in medical education. Faculty often observe learner performance but struggle to convert those observations into written feedback. Their randomized evaluation of an ambient artificial intelligence (AI) scribe workflow is timely. The authors also added a useful safety check by reviewing unedited AI summaries for mischaracterization and hallucination [1]. Even so, the workflow’s value as a documentation tool should be distinguished from its value as an educational intervention. The study's central conclusion, that the AI-assisted workflow improved the quality of written narrative feedback without increased instructor effort, warrants cautious interpretation. This claim rests mainly on higher Evaluation of Feedback Captured Tool (EFeCT) scores. Yet the study also reported that human-only narratives were shorter than AI-assisted outputs. Longer narratives may be more likely to contain the elements rewarded by the scoring tool. A higher score may therefore reflect length, structure, or completeness, rather than better educational feedback. Recent related evidence supports this distinction. Kondo and colleagues compared AI-generated feedback with supervisor feedback on clinical clerkship logs. AI feedback was longer and more consistent, and it offered structured, text-anchored comments. Supervisors offered experience-based feedback grounded in clinical context and professional judgment [2]. This contrast is important for interpreting Talwalkar and colleagues’ findings. Rubric alignment and narrative completeness are valuable documentation outcomes, but they do not establish that AI feedback is educationally superior. The educational meaning of the outcome is also uncertain. The EFeCT assesses the written narrative, not learner uptake. Medical students do not simply receive feedback and convert it into action. Spooner and colleagues found that students’ use of feedback depends on self-regulation, beliefs, emotions, and their ability to generate next steps [3]. Molloy and colleagues also cautioned against treating feedback as a simple input, apart from learner involvement and effects beyond the immediate task [4]. These perspectives suggest that learner reception and actionability should be measured before stronger educational claims are made. A further caution concerns feedback volume. Shute’s review supported feedback that is specific and clear, but also warned that elaborated feedback should remain manageable for learners [5]. More text may improve a documentation rubric while making feedback harder to process. Future studies of AI-assisted feedback should therefore separate narrative completeness from educational usefulness. Outcomes should include learner understanding, perceived actionability, subsequent learning behavior, and the burden of reviewing AI-generated inaccuracies. These comments do not lessen the value of the study. They point to the next research step: whether AI-assisted feedback helps learners learn, not only whether it helps documents score better. Sincerely, Amane Endo MD, PhD