This paper has been accepted and is currently in production.
It will appear shortly on 10.2196/78288
The final accepted version (not copyedited yet) is in this tab.
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
AI Chatbots for Mental Health Self-Management: A Lived Experience–Centered Qualitative Study on Values and Harms
ABSTRACT
Background:
Large language models (LLMs) now enable chatbots to engage in sensitive mental health conversations, including depression self-management. Yet their rapid deployment often overlooks how well these tools align with the values of people with lived experiences. Without such alignment, chatbots risk causing harm through misinformation, lack of empathy, or inadequate crisis support.
Objective:
This study aims to explore how the values of individuals with lived experiences of depression relate to the potential harms and design implications of LLM-based mental health chatbots.
Methods:
We developed a technology probe—a GPT-4o–based chatbot named Zenny—designed to simulate depression self-management scenarios grounded in prior research. We then conducted interviews with 17 individuals with lived experiences of depression, who interacted with Zenny during the session. Thematic analysis was applied to the interview data to identify key concerns and value-driven insights.
Results:
Our analysis identified five core values that participants prioritized when engaging with the chatbot: informational support, emotional support, personalization, privacy, and crisis management. These values shaped how participants perceived both the benefits and limitations of using LLMs for mental health support.
Conclusions:
This study highlights the importance of aligning AI chatbot design with the values of people with lived experiences to mitigate potential harms. We offer design recommendations that aim to enhance the safety and usefulness of LLM-based tools for depression self-management.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.