Currently submitted to: JMIR Cancer
Date Submitted: Oct 6, 2026
Open Peer Review Period: Nov 4, 2026 - Dec 30, 2026
(currently open for review)
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Pooled Mammographic Malignancy Model Performance Masks Age-Dependent Calibration Heterogeneity: Subgroup Analysis of 961 Masses
ABSTRACT
Background:
Mammographic features and patient age are commonly used to predict breast malignancy, with models demonstrating strong overall discrimination. However, pooled performance may mask clinically relevant calibration differences across subgroups. Age and BI-RADS category may modify the relationship between imaging features and malignancy risk, potentially leading to subgroup-specific miscalibration. This study therefore evaluates calibration across clinically relevant age and BI-RADS strata.
Objective:
To determine whether a logistic regression model predicting mammographic mass malignancy demonstrates uniform calibration across age groups and BI-RADS risk strata, or whether subgroup-level miscalibration emerges despite adequate pooled performance.
Methods:
A total of 961 mammographic masses (445 malignant, 46.3%) were analyzed using logistic regression incorporating age, shape, margin, and density as predictors. Discrimination (AUC) and calibration (slope, expected calibration error [ECE]) were assessed across three age subgroups (<40, 40–59, ≥60 years) and two BI-RADS strata (0–3 and 4–5). Subgroup heterogeneity was evaluated using permutation tests with Bonferroni correction.
Results:
Pooled AUC was 0.86 (95% CI: 0.84–0.89) with near-perfect aggregate calibration (slope 0.97, ECE 0.015). Significant calibration heterogeneity was identified across age subgroups (p = 0.005). Women aged 40–59 years demonstrated overconfident predictions (calibration slope 0.81, AUC 0.79), whereas women under 40 showed underconfident predictions (slope 1.42) with highest discrimination (AUC 0.93; 95% CI: 0.86–0.98). Pairwise comparison confirmed significantly higher AUC in women under 40 versus those aged 40–59 (difference 0.14; 95% CI: 0.07–0.21; p = 0.002). BI-RADS 0–3 masses exhibited poor calibration (ECE 0.17, slope 0.59) compared with BI-RADS 4–5 masses (ECE 0.01, slope 0.99).
Conclusions:
Pooled mammographic model performance metrics conceal clinically significant age-dependent miscalibration. Middle-aged women receive systematically overconfident malignancy predictions. Subgroup-specific recalibration is required prior to clinical implementation.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.