JMIR Publications

ABSTRACT

Background:

Physicians routinely document specifics of patient encounters in clinic visit notes, a critical but potentially time-consuming task. Ambient-listening artificial intelligence (AI) technology is being integrated into clinical workflows to reduce documentation burden by creating draft visit notes. While this technology is promising, it is not perfect and the potential for patient harm needs to be understood and mitigated. We developed and piloted an efficient, standardized approach to evaluating AI-generated notes for safety concerns in ambulatory care visits.

Objective:

The objective of this study was to develop and pilot an efficient, standardized and scalable approach to evaluating AI-generated notes for safety concerns in ambulatory care visits.

Methods:

During a two-month pilot (July–August 2024), 31 physicians across multiple specialties used an ambient listening AI scribe to assist with clinic note creation. A novel survey instrument was developed to assess note quality, focusing on four error types: accidental inclusions, accidental omissions, hallucinations, and bias. Physicians evaluated 356 AI-generated notes. Where an error was present, physicians rated its severity in terms of its potential to cause patient harm if it was not corrected on a 0–5 scale. Additionally, a vendor-reported metric on the percentage of note content edited by physicians was analyzed.

Results:

Accidental omissions were the most frequent error (18%), followed by hallucinations (11.5%), and accidental inclusions (9.3%). Bias was rare (1%). Most errors were rated as mild to moderate (severity 1–3), with only 2.5% of notes containing errors rated as posing serious or imminent risk (severity 4–5). Editing metrics showed a median of 9.0% of AI-generated words were changed, with 15% of notes left entirely unedited. Physician editing practices varied widely.

Conclusions:

AI-generated clinical notes were generally of high quality, with over 80% free from significant errors. However, because a small number contained errors that carried risk of serious harm if not corrected, careful clinician review of notes remains imperative. Prior to deploying an AI scribe, organizations should pilot the technology and include an efficient review process to understand the nature and type of errors common at their organization. This pilot provides a scalable model for other health systems seeking to implement AI scribe technology responsibly.