JMIR Publications

ABSTRACT

Background:

Longitudinal electronic health record clinical notes contain rich information for artificial intelligence (AI) applications, but they often exceed practical language model context windows and impose substantial computational burden. Clinically informed context reduction may improve scalability by retaining information-rich note sections while removing less relevant text.

Objective:

To develop and evaluate a scalable context-reduction framework that uses a fine-tuned open-source language model to extract clinically relevant information from longitudinal clinical notes for downstream AI applications.

Methods:

We conducted a multiphase study within Kaiser Permanente Southern California using clinical notes from adults with mild asthma. Manual annotation was first performed on a training dataset of 1,500 notes and an independent validation dataset of 250 notes. A Flan-T5 large model was then fine-tuned on the training notes to extract visit reason, chief complaint, and history of present illness. Model outputs from the validation dataset were compared with manually extracted reference standards using contextual precision, recall, relevancy, and faithfulness, with GPT-OSS-20B model serving as the evaluation judge. The fine-tuned model was then applied to 3,815,537 implementation notes from 198,667 patients to extract targeted clinically relevant information at the note level and aggregate it to the patient level.

Results:

The study included 3,817,287 clinical notes from 198,669 patients. Fine-tuning substantially improved extraction performance compared with zero-shot inference, increasing contextual precision from 0.342 to 1.000, recall from 0.305 to 0.958, relevancy from 0.329 to 0.991, and faithfulness from 0.128 to 0.984. In the implementation dataset, mean note length decreased from 213.1 to 27.7 tokens, and mean patient-level token burden decreased from 4,093.6 to 518.1 tokens, corresponding to a 7.90-fold reduction. Context-window feasibility also improved substantially. The proportion of patients with 500 tokens or fewer increased from 4.9% to 65.6%, while the proportion with more than 5,000 tokens decreased from 23.3% to 0.2%.

Conclusions:

A fine-tuned open-source language model accurately retained targeted clinical information while substantially reducing longitudinal clinical text burden at population scale. Clinically informed context reduction may support more efficient and scalable use of electronic health record notes in downstream AI applications.