JMIR Publications

ABSTRACT

Background:

Mobile mental health applications are widely used to support diverse mental health needs, yet their perceived usefulness differs across mental health conditions and remains insufficiently examined beyond common conditions such as anxiety and depression. Large-scale analyses of user feedback across diverse conditions and interventions are needed to better understand what users find useful and what undermines usefulness.

Objective:

This study examined perceived usefulness across a large-scale corpus of user reviews of mobile mental health apps and explored how experiences differed among reviews that referenced specific mental health conditions. We also investigated factors shaping perceived usefulness, including usability, reliability, accessibility, and financial barriers.

Methods:

We analyzed MHARD (Mental Health App Reviews Dataset), a publicly available dataset of user reviews from 73 mental health apps on the Google Play Store. Using MHARD, we analyzed 200,972 cleaned English-language reviews and selected a subset of 4,012 of the longest reviews, retaining at least 54 reviews per app, for in-depth thematic analysis. A structured codebook captured perceived usefulness, application features, and mental health conditions mentions, user experience barriers, technical issues, and financial barriers. Four coders independently annotated the data, achieving strong agreement (Cohen’s κ ≥ 0.85). A transformer-based classifier (BART-large) was then fine-tuned on the annotated subset to scale thematic labeling across the full 200,972-review corpus. We report descriptive trends, app-type comparisons, temporal patterns, and exploratory RCT vs non-RCT comparisons using analyses that accounted for app-level clustering where appropriate.

Results:

In the qualitative sample, negative sentiment was the most common category (1,829/4,012, 45%), followed by positive sentiment (1,444/4,012, 35%) and neutral sentiment (739/4,012, 20%). Users valued intervention techniques such as breathing exercises, journaling, and mood tracking, while barriers to usefulness centered on technical instability (e.g., crashes and data loss), limited customization, and financial friction (e.g., paywalls and intrusive advertisements). Condition-specific needs also emerged for underserved groups, including reliability and trust concerns among users referencing PTSD and sensory-friendly, low-stimulation design and flexible reminders among users referencing ADHD. Quantitative app-level analyses showed that perceived usefulness varied across app types. Meditation-focused apps were more frequently associated with anxiety-related reviews, chatbot/AI apps were more often described as helpful, and CBT-based apps showed higher prevalence of helpfulness, anxiety-related, and depression-related reviews. Online therapy apps showed higher negative sentiment, while habit/digital wellbeing apps showed lower positive sentiment, lower helpfulness, and more broken-feature complaints. Review volume and helpfulness-related mentions increased between 2018 and 2020, overlapping with the early COVID-19 period, followed by a post-2020 decline in positive sentiment., suggesting shifts in user expectations and engagement over time. Exploratory RCT vs non-RCT comparisons did not remain statistically significant after accounting for clustering of reviews within apps.

Conclusions:

Users’ experiences with mental health apps are shaped not only by the content of interventions but also by implementation realities—particularly monetization, reliability, and usability. RCT status alone did not explain user-perceived usefulness after accounting for app-level clustering. App type, usability, reliability, and monetization appeared more central to real-world user experience. Improving the real-world impact of digital mental health tools will require aligning evidence-based interventions with user-centered design, transparent pricing models, and reliable functionality, while expanding support for underrepresented mental health conditions such as ADHD, PTSD, bipolar disorder, and OCD. Clinical Trial: N/A