JMIR Publications

ABSTRACT

Background:

Accurately measuring medication exposure over time is essential for clinical research, adverse-event monitoring, and treatment optimization. However, this is difficult when utilizing electronic health record (EHR) data, especially when prescriptions include variable dosing instructions or frequent dose changes [1]. Glucocorticoid (GC) use in inflammatory disease exemplifies this challenge. GCs are typically initiated at higher doses with a gradual taper and may be altered based on disease response [2, 3]. Additionally, cumulative exposure is clinically important because of its association with substantial metabolic and infectious side effects. [4, 5] While natural language processing (NLP) approaches have been used to extract medication information, many existing methods focus on identifying medications or extracting single-dose information rather than reconstructing dose changes over time [6, 7].

Objective:

This study aimed to develop and validate a large language model (LLM)–assisted NLP pipeline for extracting GC tapering instructions from semi-structured EHR prescription data and calculating longitudinal Patients with International Classification of Diseases, Ninth or Tenth Revision diagnosis codes for giant cell arteritis and/or polymyalgia rheumatica were included if GC prescriptions covered at least 75% of days within a 12-month follow-up period. Prescription data were preprocessed using BigQuery SQL. Gemini-2.5-Flash was prompted using a few-shot strategy to extract tapering instructions, and a Python-based deterministic parsing module converted model outputs into structured daily dose values. The pipeline calculated 365-day cumulative GC exposure, which was compared with manually reviewed reference doses. Performance was evaluated using correlation, classification metrics, Cohen κ, and absolute percentage error.cumulative GC exposure.

Methods:

Patients with International Classification of Diseases, Ninth or Tenth Revision diagnosis codes for giant cell arteritis and/or polymyalgia rheumatica were included if GC prescriptions covered at least 75% of days within a 12-month follow-up period. Prescription data were preprocessed using BigQuery SQL. Gemini-2.5-Flash was prompted using a few-shot strategy to extract tapering instructions, and a Python-based deterministic parsing module converted model outputs into structured daily dose values. The pipeline calculated 365-day cumulative GC exposure, which was compared with manually reviewed reference doses. Performance was evaluated using correlation, classification metrics, Cohen κ, and absolute percentage error.

Results:

After filtering adequate prescription coverage, 100 patients were included in the final analysis. Pipeline-derived cumulative 365-day doses correlated strongly with manually reviewed reference doses (r=0.84; P<.001). When cumulative doses were categorized into low-, medium-, and high-dose groups, the model achieved 80% accuracy, 85% precision, 73% recall, an F1 score of 76%, and an area under the receiver operating characteristic curve of 94% for identifying the highest-risk dose category. The mean individual-level absolute percentage error was 23%, while cohort-level error was 7%. In prescription-level validation, 100 records were annotated for accuracy in dose extraction. The model achieved an accuracy of 96%, a macro F1 score of 97%, and a quadratic-weighted Cohen Kappa of 94% (un-weighted: 93%).

Conclusions:

An LLM–assisted NLP pipeline combined with deterministic parsing can extract complex GC tapering instructions from semi-structured EHR prescription data and estimate longitudinal cumulative medication exposure. This approach offers a scalable solution for medication exposure assessment in clinical research cohorts, particularly when manual review is not feasible. Further validation is needed across other medication classes, clinical settings, and EHR systems. Clinical Trial: N/A