JMIR Publications

ABSTRACT

Background:

Large language models (LLMs) are increasingly used in clinical workflows for evidence retrieval, documentation, and administrative tasks. Data on how clinicians actually use these tools, and on their concerns about reliability, data security, and ethics, remain limited.

Objective:

To describe self-reported use, frequency, perceived usefulness, trust, verification practices, and comparative perceptions of LLMs among Department of Medicine clinicians at a large academic medical center.

Methods:

We conducted a cross-sectional survey of attending physicians, nurse practitioners and physician assistants, residents, and fellows in the Department of Medicine at Mayo Clinic (Rochester, Minnesota), administered December 16, 2025, to February 1, 2026. The instrument covered demographics; awareness; clinical and administrative use; perceived usefulness and ease of use; trust and data security; behavioral intention; verification; comparison with traditional tools; and ethics, using multiple-choice, multi-select, 5-point Likert, and free-text items with skip logic, and was pilot-tested with cognitive interviewing in 10 clinicians. Descriptive statistics were used, and associations between years in practice and use were assessed with Monte Carlo exact tests. Denominators vary by item-level nonresponse; findings apply to respondents only.

Results:

Of 2186 invited clinicians, 254 completed the survey (response rate 11.6%). Among respondents, 247/253 (97.6%) were aware of LLMs and 228/247 (92.3%) reported ever using one in clinical practice. Among users, 103/222 (46.4%) reported daily use and 196/222 (88.3%) at least weekly. OpenEvidence was the most frequently reported clinical tool (187/227, 82.4%); ChatGPT was most frequent for administrative tasks (84/218, 38.5%). The most common reason for clinical use was rapid access to evidence-based answers (174/227, 76.7%), and use was more often outpatient (167/227, 73.6%) than inpatient (120/227, 52.9%). Most respondents agreed that LLM use is ethically appropriate (202/220, 91.8%) and that outputs are reliable (174/219, 79.5%), but fewer were confident that the data they entered were secure (116/219, 53.0%), and only 120/217 (55.3%) reported always or often verifying outputs. When respondents compared their most-used LLM with traditional tools, more rated it better than UpToDate (118/202, 58.4%) or PubMed (102/160, 63.8%); these reflect perceptions of convenience and workflow rather than head-to-head performance. Reported use did not differ significantly by years in clinical practice.

Conclusions:

Among responding Department of Medicine clinicians at a single academic center, self-reported awareness and use of LLMs were high, concentrated in rapid evidence retrieval and support for complex questions. Perceived ethical acceptability and reliability were favorable, but confidence in data security was lower and verification was inconsistent. Given the low response rate and likely self-selection, these findings describe respondents rather than department-wide adoption, should be interpreted as hypothesis-generating, and highlight the need for clear institutional guidance on appropriate use, verification, and data security. Clinical Trial: Na