JMIR Publications

ABSTRACT

Background:

Pharmaceutical supply disruptions can compromise treatment continuity and increase health-system burden, yet national surveillance is largely reactive. Conventional next-month status prediction can also be misleading when instability is persistent, because a model may mainly identify products that are already unstable rather than newly emerging events.

Objective:

To develop and internally validate a machine learning model for predicting new-onset pharmaceutical supply-demand instability one month ahead among products that were stable at the time of prediction.

Methods:

We analyzed a national monthly panel from the Republic of Korea (July 2018 to March 2023; 3,245 products; 150,128 product-months). Supply-demand instability was defined using a prespecified operational threshold of the supply-demand instability ratio (SR ≥0.20). The primary outcome was transition from stable to unstable status one month later among at-risk product-months. Thirty-five backward-looking features were derived from supply, use, and distribution-flow series. Ten machine learning algorithms were compared, and a 10-feature random forest was retained as a parsimonious reference model. Internal validation included product-grouped cross-validation and nested product-grouped and time-ordered temporal schemes, with calibration and risk-enrichment characteristics also assessed.

Results:

Instability occurred in 15.0% of product-months, but 74% of instability months with an observed preceding month were continuations of an existing unstable state. New onset occurred in 4.58% of at-risk months. The 10-feature random forest achieved an AUPRC of 0.245 ± 0.019 and an AUROC of 0.844 ± 0.013, compared with 0.182 and 0.774 for logistic regression. In nested internal validation of the modeling pipeline, AUPRC and AUROC were 0.288 ± 0.022 and 0.873 ± 0.008 under product-grouped validation and 0.261 ± 0.020 and 0.855 ± 0.010 under the time-ordered temporal scheme. Expected calibration error was ≤0.003. Approximately half of onsets in each outer test fold were concentrated in its top 10% of predicted risk. Feature-group ablation indicated that performance did not depend on instability-ratio autocorrelation.

Conclusions:

Routine national pharmaceutical monitoring data supported internally validated prediction of newly emerging supply-demand instability when the prediction task was restricted to currently stable products. The results support further evaluation of risk ranking as a decision-support approach for pharmaceutical surveillance. Direct validation against regulator-reported shortages, strictly month-aligned temporal evaluation, and prospective assessment are required before operational use. Clinical Trial: N/A