JMIR Publications

ABSTRACT

Background:

Therapeutic homework is a core element of cognitive and behavioral interventions, and greater homework compliance predicts improved treatment outcomes. To date, research in this area relied mostly on therapists’ and clients’ self-report or on studies carried out in academic settings.

Objective:

This study tested whether a machine learning (ML) model using natural language processing could identify homework assignments in behavioral health sessions.

Methods:

We analyzed 34,497 audio-recorded treatment sessions provided in 8 behavioral healthcare programs via an Artificial intelligence (AI) platform designed for therapy. Therapist and client utterances were captured via an artificial intelligence platform. Experts reviewed the homework assigned in 100 sessions, and classifications were created. Next, we sampled 4000 sessions and tagged dialogues that suggest homework to train an unsupervised sentence embedding model. This model was trained on 2.83 million therapist-client dialogues.

Results:

Homework was assigned in 61% of sessions, and in 21% of these cases, more than one homework was provided. Homework addressed practicing skills (37%), taking action (28.5%), journaling (19%), and learning new skills (14%). Our classifier reached 72% F-1, outperforming state-of-the-art ML models.

Conclusions:

Findings suggest that behavioral therapy adherent behaviors can be identified using session audio data. Treatment-specific Artificial intelligence (AI empowered by ML, can help increase fidelity with evidence-based techniques in practice, by reminding clients to complete homework and facilitate therapist homework provision and review in session.