JMIR Publications

ABSTRACT

Background:

Section-specific labels automatically extracted from radiology reports encode structured information and are widely used as supervision targets or evaluation references for tasks such as abnormality classification, report generation, and label refinement. However, the information-selection regularities embedded in these labels have rarely been treated as the primary research object.

Objective:

This study aimed to characterize Findings-to-Impression concept-retention regularities in radiology reports and evaluate whether these recovered regularities are clinically plausible and can support downstream information-selection tasks.

Methods:

We analyzed 48,171 paired Findings and Impression studies from the CheXpert Plus dataset and treated the section-specific concept annotations as large-scale observational evidence for recovering reproducible Findings-to-Impression information-selection regularities. The dataset was randomly split at the patient level into 80% training and 20% held-out test sets. The recovered regularities were explicitly represented using our proposed framework. Predictive utility was evaluated using four progressively specified feature sets implemented with three supervised learning models, whereas transferability was assessed through progressively informed prompting across three LLMs (GPT-5.5, Med42-8B, and OpenBioLLM-8B). Clinical plausibility was assessed through independent review of 200 studies by two radiologists.

Results:

The recovered section-specific information-selection regularities demonstrated predictive utility for traditional models and LLM-guided concept selection. Model discrimination improved with the progressive incorporation of contextual features, reaching its highest performance after the addition of stable directional pair associations (area under the receiver operating characteristic curve 0.707; average precision 0.798). In the LLM experiments, prompting informed by baseline and complexity-specific retention tendencies significantly improved concept-selection precision across all evaluated models compared with zero-shot. Med42-8B showed the broadest overall performance improvement, whereas OpenBioLLM-8B exhibited the greatest increase in precision and the most conservative selection behavior. However, adding directional pair association information did not provide further improvement in LLM performance.

Conclusions:

Automatically extracted section-specific radiology labels provide large-scale observational evidence for reproducible section-aware concept-selection regularities. These regularities provide an interpretable basis for downstream applications and a reusable representation of information-selection knowledge.