Previously submitted to: JMIR Mental Health (no longer under consideration since Mar 14, 2024)
Date Submitted: Mar 11, 2024
Open Peer Review Period: Mar 14, 2024 - Mar 14, 2024
(closed for review but you can still tweet)
NOTE: This is an unreviewed Preprint
Warning: This is a unreviewed preprint (What is a preprint?). Readers are warned that the document has not been peer-reviewed by expert/patient reviewers or an academic editor, may contain misleading claims, and is likely to undergo changes before final publication, if accepted, or may have been rejected/withdrawn (a note "no longer under consideration" will appear above).
Peer review me: Readers with interest and expertise are encouraged to sign up as peer-reviewer, if the paper is within an open peer-review period (in this case, a "Peer Review Me" button to sign up as reviewer is displayed above). All preprints currently open for review are listed here. Outside of the formal open peer-review period we encourage you to tweet about the preprint.
Citation: Please cite this preprint only for review purposes or for grant applications and CVs (if you are the author).
Final version: If our system detects a final peer-reviewed "version of record" (VoR) published in any journal, a link to that VoR will appear below. Readers are then encourage to cite the VoR instead of this preprint.
Settings: If you are the author, you can login and change the preprint display settings, but the preprint URL/DOI is supposed to be stable and citable, so it should not be removed once posted.
Submit: To post your own preprint, simply submit to any JMIR journal, and choose the appropriate settings to expose your submitted version as preprint.
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
An artificial intelligence-based virtual human avatar application to assess the mental health of healthcare professionals: A validation study
ABSTRACT
Background:
Many healthcare workers experience high levels of stress, mental health issues and burnout, yet are less likely than average to seek mental health support. Given this challenge, approaches that promote non-stigmatizing access to assessments of mental health for this population are greatly needed to promote both self-awareness and connection to care. Previous researchers have indicated that when users engaged in dialog with a virtual human agent (VHA), they disclosed more information about their mental health compared to similar interactions with a live human or human-as-avatar assessor. However, applications using this approach have not yet been validated and tested for use in the real world with healthcare professionals.
Objective:
Given this unmet need, an application called “BeCalm” was developed to conduct psychological assessments using conversational artificial intelligence (AI) and user interaction with a digital, human-like VHA. The BeCalm application allows for both spoken and written (“chat” based) communication. This pilot study aimed to measure the user experience, acceptability, and convergent validity of the BeCalm application.
Methods:
A cross-sectional one-arm study of the BeCalm application was conducted with 38 healthcare workers (age M = 31.87, SD = 11.28; 84% biologically female).
Results:
Qualitative interviews indicated that the apparent interpersonal connection with the VHA was the most appealing aspect of the BeCalm application, with participants describing the VHA as warm and non-judgmental. The second most highly rated aspect of the application was the resources, summary, and psychoeducation provided at the end of the assessment. Many participants reported that they learned something new from the summary and psychoeducation, and the resources were described as concrete, actionable, and “not something I could just google.” There was also strong convergent validity between the BeCalm assessments and conventional mental health measures across several symptom domains (r’s ranged from .101 to .766), including occupational burnout, work-related trauma, work satisfaction, anxiety, mood, and loneliness. Notably, spoken verbal interactions with the VHA elicited longer participant responses by an average of 60 characters compared to chat interactions. Interestingly, participants with higher levels of loneliness tended to provide lengthier answers across all domains, potentially suggesting a desire for deeper engagement with the VHA.
Conclusions:
These results demonstrate the ability of BeCalm to provide a valid mental health assessment and resources for an underserved, at-risk population, through interactive technology and personalized feedback. The BeCalm application offers a user-friendly, scalable method for assessing healthcare workers’ mental health that could lead to greater self-awareness and behavioral change. Future studies will determine whether BeCalm can improve mental health outcomes among healthcare professionals and other at-risk populations.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.