JMIR Publications

ABSTRACT

Background:

Large Language Models (LLMs) are rapidly being adopted across the medical field. Emergency medicine is characterized by the need for critical decision-making under high uncertainty, incomplete information, and severe time constraints, facing challenges in LLM implementation distinct from other departments. While surveys targeting general physicians or other specialties exist, large-scale surveys specifically targeting emergency physicians are scarce.

Objective:

This study aimed to assess the usage patterns and perceived issues of LLMs among emergency physicians through a web-based survey conducted by the Japanese Association for Acute Medicine. This study provides foundational knowledge on the current status of LLM use among emergency physicians to promote its safe and effective implementation.

Methods:

An anonymous, cross-sectional, web-based survey was conducted among participants of the Japanese Association for Acute Medicine between June and August 2025. The analysis included 362 emergency physicians. Survey items comprised respondent attributes, LLM usage experience, frequency, purposes of use, service names, and perceived issues (free text). Free-text responses (n=208) were classified into 6 themes (multi-label) using a rule-based text-classification workflow, which was validated against blinded manual coding of a random subsample. Multivariable ordered logistic regression and logistic regression analyses were performed to evaluate the associations of sex, age, and years of clinical experience with multiple LLM-related outcomes, and odds ratios (ORs) were calculated.

Results:

The mean age of the 362 participants was 49.2 years. 290 physicians (80.1%) had experience using LLMs. Of these, 46.2% used them "daily" and 33.8% "weekly," meaning 80.0% used them at least once a week. Usage rates were higher among younger generations: 95.3% for those ≤39 years vs 69.0% for those ≥60 years. Purposes included personal use (71.7%), academic activities (65.2%), education/training (57.2%), operational efficiency/administrative tasks (49.3%), and clinical decision support (43.1%). The regression analyses showed that for each 1-year increase in age, the odds of use frequency significantly decreased (OR 0.97, 95% Confidence Intervals (CI) 0.95-0.99; P=.011), as did the use for clinical decision support (OR 0.97, 95% CI 0.95-0.99; P=.016). The rule-based thematic classification identified "Accuracy/Reliability" (60.6%) as the most frequent concern.

Conclusions:

LLM usage has already widely spread among emergency physicians, with younger physicians showing notably higher frequency especially for clinical decision support. This study helps to understand the current status of LLM usage in emergency settings and to discuss future directions for the development of LLM models and usage guidelines tailored to emergency medicine.