JMIR Publications

ABSTRACT

Background:

Young adults with serious mental illness (SMI) have higher rates of smoking and a lower likelihood of achieving abstinence compared to young adults without SMI. Scalable interventions such as smartphone apps with evidence-based content (e.g., the National Cancer Institute’s QuitGuide and quitSTART) could increase access to potentially appealing and effective treatment for this group, but have yet to be tested in this population.

Objective:

The goal of this user-centered design study was to determine usability, acceptability, and user experience of two widely available apps developed by the National Cancer Institute, QuitGuide and quitSTART, among young adult tobacco users with SMI.

Methods:

We conducted usability and acceptability testing of QuitGuide and quitSTART among participants with SMI, aged 18-35, who were stable in community mental health treatment in 2019-2020. Participants were randomly assigned to use QuitGuide or quitSTART on their smartphone. App usability was evaluated at baseline and following a two-week field test of independent use via a video-recorded task-completion protocol. Using a mixed method approach, we triangulated data four data sources: non-participant observation, open-ended interviews, structured interviews (including the System Usability Scale – SUS), and backend app usage data obtained from the National Cancer Institute. Quantitative data were analyzed using descriptive statistics; qualitative data were analyzed using thematic analysis.

Results:

Participants were 17 smokers not interested in quitting, mean age 29 years old (SD=4), and 41 percent with psychotic disorders. Participants smoked an average of 15 cigarettes per day (SD=7). Mean SUS scores for QuitGuide were similar at Visits 1 and 2 (64±18 and 66±18, respectively). In contrast, mean SUS scores for quitSTART numerically increased from Visit 1 (55±20) to Visit 2 (64±16). Acceptability scores followed the same pattern. Observed task completion rates were over 75% for both apps at both visits for all but one task. Regarding app use during the 13-day trial period, QuitGuide users interacted with the app on 4.6 days (SD=2.8) on average, for a mean total of 5.6 interactions (SD=3.8), and responded to a median of 1 notification (range 0-8). In contrast, quitSTART users interacted with the app on 10.8 days (SD=3.5), with 41 interactions (SD=26) total, and responded to a median of 18.5 notifications (range 0-37). Qualitative comments indicated moderate-high satisfaction overall but concerns over the accuracy of the apps’ feedback.

Conclusions:

Both QuitGuide and quitSTART had acceptable levels of usability, and mixed results in user experience among young adults with SMI. The higher level of engagement with quitSTART suggests that quitSTART may be a favorable tool for young adult smokers with SMI. However, clinical support or coaching may be needed to overcome initial usability issues.