JMIR Publications

ABSTRACT

Background:

Wait times impact patient satisfaction, treatment effectiveness, and the efficiency of care that the patients receive. Wait time prediction in mental health is a complex task and is impacted by difficulty in predicting the required number of visits for outpatients, high no-show rates, and the possibility of using group treatment sessions. In addition, health-related data are usually highly de-identified by removing both direct and quasi identifiers that makes the task of wait time analyses even more challenging.

Objective:

The first aim of this study was to develop machine learning models to predict the wait time from referral to the first appointment for psychiatric outpatients using real-time data. The second aim was to enhance the performance of these predictive models by utilizing the system’s knowledge while the input data were highly de-identified. The third aim was to identify factors that drive long wait times. And the fourth aim was to build these models such that they were practical and easy-to-implement (and therefore, attractive to care providers).

Methods:

We analyzed retrospective highly de-identified administrative data from eight outpatient clinics at Ontario Shores Centre for Mental Health Sciences in Canada using six machine learning methods to predict first appointment wait time for new outpatients. We used the system’s knowledge to mitigate the low utility (due to being highly de-identified) of our data. The data included 4,187 patients who received care through 30,342 appointments.

Results:

The average wait time varied widely between different types of mental health clinics and for more than half of the clinics, the average wait time was longer than 3 months. The number of scheduled appointments and the rate of no-shows varied widely among clinics. Despite these variations, the Random Forest method provided the minimum RMSE values for four of the eight clinics, and the second minimum RMSE for the other four clinics. Utilizing the system’s knowledge increased the utility of our highly de-identified data and improved the predictive power of the models.

Conclusions:

The Random Forest method enhanced with the system’s knowledge provided reliable wait time predictions for new outpatients regardless of low utility of the highly-deidentified input data and the high variation in wait times across different clinics and patient types. The priority system was identified as a potential reason for long wait times and a fast-track system was suggested as a potential solution.