JMIR Publications

ABSTRACT

Background:

Patient engagement is a critical but challenging public health priority in behavioral healthcare. During telehealth sessions, healthcare providers need to rely more on verbal strategies than typical non-verbal cues to engage patients. Hence, the typical patient engagement behaviors are now different, and provider training on telehealth patient engagement is unavailable or quite limited. Therefore, we explore the application of machine learning for estimating patient engagement to assist psychotherapists in better diagnosis of mental disorders during telemental health sessions.

Objective:

The objective of this study was to examine the ability of machine learning models to estimate patient engagement levels during a telemental health session and understand whether the machine learning approach could support mental disorder diagnosis by psychotherapists.

Methods:

We propose a multimodal learning-based framework MET. We uniquely leverage latent vectors corresponding to Affective and Cognitive features frequently used in psychology literature to understand a person’s level of engagement. Given the labeled data constraints that exist in healthcare, we explore a semi-supervised solution using GANs. To further the development of similar technologies that can be useful for telehealth, we also plan to release a dataset MEDICA containing 1299 video clips, each 3 seconds long and show experiments on the same. The efficacy of our method is also demonstrated through real-world experiments.

Results:

Our framework reports a 40% improvement in RMSE (Root Mean Squared Error) over state-of-the-art methods for engagement estimation. In our real-world tests, we also observed positive correlations between the working alliance inventory scores reported by psychotherapists. This indicates the potential of the proposed model to present patient engagement estimations that aligns well with the engagement measures used by psychotherapists.

Conclusions:

The performance of the framework described here has been compared against other existing engagement detection machine learning models. We also validated the model using a limited sample of real-world data. Patient engagement in literature has been identified to be important to improve therapeutic alliance. But little research has been undertaken to measure it in a telehealth setting wherein the conventional cues are not available to the therapist to take a confident decision. The framework developed is an attempt to model person-oriented engagement modeling theories within machine learning frameworks to estimate the level of engagement of the patient accurately and reliably in telehealth. The results are encouraging and emphasize the value of combining psychology and machine learning to understand patient engagement. Further testing in actual telehealth settings is necessary to fully assess its usefulness in helping therapists gauge patient engagement during virtual sessions. However, the proposed approach and the creation of the new dataset, MEDICA, opens avenues for future research and development of impactful tools for telehealth.