JMIR Publications

ABSTRACT

Background:

Machine learning has been extensively applied in medical prediction; however, existing machine-assisted pharmacotherapy review systems predominantly rely on structured, codifiable data while neglecting unstructured natural language clinical narratives, thereby allowing latent medication risks to remain undetected.

Objective:

To address the limitations of structured-data-based review, this study proposes and validates a machine learning framework grounded in text classification for the automated identification of "prophylactic" versus "non-prophylactic" medication indications within clinical narratives, thereby assisting in the rationality assessment of prophylactic proton pump inhibitor (PPI) utilization.

Methods:

A total of 3,695 progress notes were independently annotated by two clinical pharmacists to construct a binary dataset for stress-related mucosal disease prophylactic medication indication classification, yielding an inter-annotator agreement Kappa coefficient of 0.8934. Comparative experiments were conducted using traditional machine learning models including logistic regression, support vector machine, random forest, CatBoost, and LightGBM, alongside TextCNN and a Chinese pre-trained BERT model. The dataset was split into training, validation, and test sets on a patient wise basis, and model evaluation was conducted on the independent test set.SHAP analysis was employed to achieve model interpretability and elucidate decision-making processes, ultimately enabling automated rationality assessment of prophylactic proton pump inhibitor administration.

Results:

All models and strategies demonstrated efficacy in medical text classification tasks. The CatBoost models achieved the best overall performance, demonstrating good generalization ability on the independent test set.

Conclusions:

In summary, the text classifier constructed through machine learning and natural language processing effectively extracts prophylactic medication indications for stress-related mucosal disease from unstructured progress notes. Combined with patient-level data partitioning, standardized annotation protocols, and an ethics-compliant framework, this approach provides stable, interpretable, and highly reliable decision support for clinical proton pump inhibitor rationality review.