Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.
Who will be affected?
Readers: No access to all 28 journals. We recommend accessing our articles via PubMed Central
Authors: No access to the submission form or your user account.
Reviewers: No access to your user account. Please download manuscripts you are reviewing for offline reading before Wednesday, July 01, 2020 at 7:00 PM.
Editors: No access to your user account to assign reviewers or make decisions.
Copyeditors: No access to user account. Please download manuscripts you are copyediting before Wednesday, July 01, 2020 at 7:00 PM.
Validating the safety and user experience of a generative AI digital mental health intervention
Timothy R Campellone;
Megan Flom;
Robert M Montgomery;
Lauren Bullard;
Maddison C Pirner;
Aaron Pavez;
Michelle Morales;
Devin Harper;
Catherine Oddy;
Tom O'Connor;
Jade Daniels;
Stephanie Eaneff;
Valerie L Forman-Hoffman;
Casey Sackett;
Alison Darcy
ABSTRACT
Background:
General awareness and exposure to generative artificial intelligence (AI) has increased in recent years. This transformative technology has the potential to create a more dynamic and engaging user experience in digital mental health interventions (DMHIs). However, if not appropriately utilized and controlled, this technology can introduce risks to users that may result in harm and erode trust.
Objective:
This randomized controlled trial (RCT) aims to explore the user relationship, satisfaction, safety and technical guardrails of a DMHI using generative AI compared against a rules-based intervention.
Methods:
We conducted a 2-week exploratory RCT of 160 adult participants randomized to receive a generative AI (n = 81) or rules-based (n = 79) version of a DMHI. Self-report measures of the user relationship and satisfaction were collected. Safety monitoring was conducted throughout the trial for adverse events and the success of technical guardrails created for the generative arm was assessed post-trial.
Results:
In general, measures of user relationship and satisfaction appeared to be similar in both the generative and rules-based arms. The generative arm appeared to be more accurate at detecting and responding to user statements with empathy. There were no serious- or device-related adverse events and technical guardrails were shown to be very effective in post-trial review of generated statements.
Conclusions:
This trial provides initial evidence that with the right guardrails and process, generative AI can be successfully used in a digital mental health intervention (DMHI) while maintaining the user experience and relationship. Clinical Trial: The study was first posted on ClinicalTrials.gov (#NCT05948670) on July 17th, 2023.
Citation
Please cite as:
Campellone TR, Flom M, Montgomery RM, Bullard L, Pirner MC, Pavez A, Morales M, Harper D, Oddy C, O'Connor T, Daniels J, Eaneff S, Forman-Hoffman VL, Sackett C, Darcy A
Safety and User Experience of a Generative Artificial Intelligence Digital Mental Health Intervention: Exploratory Randomized Controlled Trial