JMIR Publications

ABSTRACT

Background:

Leveraging free smartphone applications (apps) can help expand the availability and use of evidence-based smoking cessation interventions. However, there is a need for more research investigating how use of different features within such apps impacts their effectiveness.

Objective:

We used observational data collected from an experiment of a publicly available smoking cessation app to develop supervised machine learning (SML) algorithms intended to distinguish the app features that promote successful smoking cessation. We then assessed the extent to which patterns of app feature use accounted for variance in cessation that could not be explained by other known predictors of cessation (e.g., tobacco use behaviors).

Methods:

Data came from an experiment (clinicaltrials.gov identifier: NCT04623736) testing the impacts of incentivizing ecological momentary assessments within the National Cancer Institute’s quitSTART app. Participants’ (n=133) app activity, including every action they took within the app and its corresponding timestamp, was recorded. Demographic and baseline tobacco use characteristics were measured at the start of the experiment, and short-term smoking cessation (7-day point prevalence abstinence) was measured at 4-weeks post-baseline. Logistic regression SML modeling was used to estimate participants’ probability of cessation from 28 variables reflecting participants’ use of different app features, assigned experimental condition, and phone type (iPhone or Android). The SML model was first fit in a training set (n=100) and then its accuracy was assessed in a held-aside test set (n=33). Within the test set, a likelihood ratio test assessed whether adding individuals’ SML-predicted probabilities of cessation to a logistic regression model that included demographic and tobacco use (e.g., poly-use) variables explained additional variance in 4-week cessation.

Results:

The SML model’s sensitivity (.67) and specificity (.67) in the held-aside test set indicated that individuals’ patterns of using different app features predicted cessation with reasonable accuracy. The likelihood ratio test showed that the logistic regression that included the SML model-predicted probabilities was statistically equivalent to the model that only included the demographic and tobacco use variables (p=.16).

Conclusions:

Harnessing user data through SML could help determine which features of smoking cessation apps are most useful. This methodological approach could be applied in future research focusing on smoking cessation app features to inform the development and improvement of smoking cessation apps.