JMIR Publications

ABSTRACT

Background:

Rising mental health challenges among young people have accelerated adoption of digital mental health interventions (DMHIs) to expand access to timely care. Robust, user-informed evaluation is essential to demonstrate the real-world effectiveness of DMHIs, especially given their flexible design, which allows users to decide when, how, and to what extent they engage. In many U.S. states, standardized consumer feedback instruments are used to evaluate publicly funded in-person mental health services, but no parallel requirements exist for DMHIs, leaving a gap in quality improvement and accountability. Establishing systematic feedback processes for DMHIs is critical to evaluate user satisfaction and perceived impact on wellbeing.

Objective:

This study aimed to (1) evaluate user satisfaction and perceived impact of a statewide digital mental health and wellbeing platform, Soluna, and (2) examine whether satisfaction and perceived change varied by user-reported platform engagement.

Methods:

We implemented a user feedback survey, which was administered via Soluna at three time points in 2024. The survey assessed user satisfaction and perceived impact on knowledge, skills and behaviors, based on outcomes articulated in the platform’s theory of change framework. Respondents were categorized into groups based on their self-reported use of the platform’s self-directed, peer-to-peer, or professional support, as well as their duration and frequency of use. Descriptive statistics were used to summarize user satisfaction and perceived impact and group differences were explored using chi-square or Fisher’s exact tests.

Results:

A total of 214 survey responses were submitted in 2024 by participants aged 14 to 25, living in California, who had access to all the features on Soluna. Among all participants, 86.2% agreed they were satisfied with Soluna overall and 97.9% would recommend Soluna to a friend. Findings demonstrated high user satisfaction, regardless of the type of support participants used (79.4%-92.1%), compared to those who had not yet engaged with any feature (66.7%) (p=.030). For proximal perceived impacts, 70.5% to 85.2% of participants reported positive feedback. A difference was observed between the user journey group and the item “Since using Soluna, I feel more connected to others who share and support my values” (X22=7.9, P=.019). Highest rates of agreement were among those in the peer-connected group (81%), followed by professional-support (71.6%) and self-directed (47.8%) groups.

Conclusions:

By embedding a standardized feedback survey into routine service delivery, this paper illustrates how reporting standards for in-person services can be applied to digital contexts. Results provide an early indication of the value of Soluna for supporting youth mental wellbeing. Higher perceived impact among those in the peer-connected and professional-support groups contributes to evidence highlighting the added value of human support to self-guided DMHIs.