JMIR Publications

ABSTRACT

Background:

The phenomenon of patients missing booked appointments without cancelling them - known as Do-Not-Show (DNS) or Did Not Attend (DNA) or Failed to Attend (FTA) - has a detrimental effect on patients’ health and results in massive healthcare resources wastage

Objective:

Our objective was to develop Machine Learning (ML) models and evaluate their performance in predicting the likelihood of DNS for hospital outpatient appointments at MidCentral District Health Board (MDHB) in New Zealand.

Methods:

We sourced five years of MDHB outpatient records (a total of 1,080,566 outpatient visits) to build the ML prediction models. We developed three ML models, using Logistic Regression, Random Forest, and XGBoost. 10-fold cross-validation and hyperparameter tuning were deployed to minimize models’ bias and boost all algorithms’ prediction strength. All models were evaluated against accuracy, sensitivity, specificity, and ROC-AUC metrics.

Results:

Based on the five years’ MDHB data, the best prediction classifier was XGBoost. with an Area Under Curve (AUC) of 0.92, a sensitivity of 0.83, and a specificity of 0.85. Patients’ DNS history, age, ethnicity, and appointment lead time significantly contributed to DNS prediction. A machine learning system trained on a large data set can produce useful levels of DNS prediction.

Conclusions:

This research is one of the very first published studies that use ML technologies to assist DNS management in New Zealand. It is a proof of concept and could be used to benchmark the DNS prediction for the MDHB and other District Health Boards. We encourage conducting additional qualitative research to investigate the root cause of DNS issues and potential solutions. Addressing DNS using better strategies potentially can result it better utilization of healthcare resources and improving the health equity.