Previously submitted to: JMIR mHealth and uHealth (no longer under consideration since May 06, 2025)
Date Submitted: Dec 10, 2024
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
"AI that’s so good it almost seems human": An App Review on the Risk and Hype Cycles of AI and Chatbots in Mobile Mental Health Apps
ABSTRACT
Background:
Interactive AI, such as conversational agents (CAs), is increasingly incorporated into digital mental health mobile apps, offering new ways to support users. As such, there is a need to understand the trends surrounding this evolving innovation landscape.
Objective:
In this study, we aim to assess and identify key themes in claims around the efficacy and utility of interactive AI features such as CAs in mental health perpetuated by mobile app developers and compare those to consumer experiences.
Methods:
We conducted an app review analysing 14 mental health apps, selected out of 320 apps featured on the Google Play Store that claimed to include interactive AI features, performing a qualitative analysis of their app descriptions and user reviews.
Results:
Thematic analysis identified four themes: (i) exaggerated claims by app developers in a saturated market promise “powerful” solutions with AI, (ii) apps as profit-driven platforms often aim to monetise user-centric mental health services through deceptive tactics, and (iii) foster user dependency, and (iv) the human appeal of CAs is often used excessively to attracts users, despite technical limitations and potential harm of overt humanism in CA.
Conclusions:
Our findings suggest that interactive AI and CAs hold potential for mental health care, however require stricter regulatory oversight and more considered design to prioritise user needs effectively. We therefore advocate for safety-focused design, guided by frameworks such as Safety by Design, alongside stricter regulations, to ensure these tools are responsibly developed and centred on user needs.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.