JMIR Publications

ABSTRACT

Background:

Training therapists is both expensive and time consuming. Degree-based training can require tens of thousands of dollars and hundreds of hours of expert instruction. Counseling skills practice often involves role-plays, standardized patients, or practice with real clients. Performance-based feedback is critical for skill development and expertise, but trainee therapists often receive minimal and subjective feedback, which is distal to their skill practice.

Objective:

In this study, we developed and evaluated a patient-like neural conversational agent, which provides real-time feedback to trainees via chat-based interaction.

Methods:

The text-based conversational agent was trained on an archive of 2,354 psychotherapy transcripts and provided specific feedback on the use of basic interviewing/counseling skills (i.e., open questions and reflections - summary statements of what a client has said). One hundred fifty-one (151) non-therapists were randomized to either 1) immediate feedback on their use of open questions and reflections during practice session with ClientBot, or 2) initial education and encouragement on the skills.

Results:

Participants in the ClientBot condition used 91% more reflections during practice with feedback (p < .001) and 76% more reflections after feedback was removed (p < .001) relative to the control group. The treatment group used more open questions during training, but not after feedback was removed, suggesting that certain skills may not improve with performance based feedback. Finally, after feedback was removed the ClientBot group used 31% more listening skills overall (p < .001).

Conclusions:

This proof-of-concept study demonstrates that practice and feedback can improve trainee use of basic counseling skills using a method that can scale beyond the capacity of a single expert trainer.