Accepted for/Published in: JMIR Formative Research
Date Submitted: May 25, 2022
Open Peer Review Period: May 25, 2022 - Jul 20, 2022
Date Accepted: Jul 20, 2022
(closed for review but you can still tweet)
Clinical Guidelines vs. Real-World Treatment Data: The Use of Session Summaries
ABSTRACT
Background:
Although behavioral interventions have been found efficacious and effective in randomized clinical trials for most mental illnesses, the quality and efficacy of mental healthcare delivery remains inadequate in real-world settings, partly due to suboptimal treatment fidelity. This “therapist drift” is an ongoing issue that ultimately reduces the effectiveness of treatments, however until recently there was limited opportunity to assess adherence beyond controlled studies and at scale.
Objective:
This study explored therapists’ use of a standard component that is pertinent across most behavioral treatments - prompting clients to summarize their treatment session as a means for augmenting their understanding of the session and the treatment plan.
Methods:
The dataset for this study comprised 17,607 behavioral treatment sessions given by 322 therapists to 3,519 patients in 37 behavioral healthcare programs across the U.S. Sessions were captured by a therapy-specific artificial intelligence (AI) platform, and an automatic speech recognition system (ASR) transcribed the treatment meeting and separated the data to the therapist and client utterances. A search for possible session summary prompts was then conducted, with two psychologists validating the text that emerged.
Results:
We found that despite clinical recommendations, only 54 sessions (0.30%) included a summary. Exploratory analyses indicated that session summaries mostly addressed relationships (N = 27), work (N = 20), change (N= 6), and alcohol (N = 5). Sessions with meeting summaries also included greater therapist use of validation, complex reflections, and proactive problem-solving techniques.
Conclusions:
Findings suggest that fidelity with the core components of evidence-based psychological interventions as designed is a challenge in real-life settings. Results of this study can inform the development of machine learning and AI algorithms and offer nuanced, timely feedback to providers, thereby improving the delivery of evidence-based practices and quality of mental healthcare services, and facilitating better clinical outcomes in real-world settings. Clinical Trial: N/A
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.