JMIR Publications

ABSTRACT

Background:

Missing data are common in mobile health (mHealth) research. There has been little systematic investigation of how missingness is handled statistically in mHealth randomized controlled trials (RCTs). While some missing data patterns (i.e., missing at random; MAR¬¬¬¬¬¬¬) may be adequately addressed using modern missing data methods like multiple imputation and maximum likelihood techniques, these methods will not address bias when data are missing not at random (MNAR). It is typically not possible to know whether missing data are MAR. However, higher attrition in active versus passive conditions in mHealth RCTs raise a strong likelihood of MNAR, such as if active participants who benefit less from the intervention are more likely to drop out.

Objective:

We sought to systematically evaluate differential attrition and methods used for handling missingness in a sample of mHealth RCTs comparing active and passive control conditions. We also aimed to illustrate a modern model-based sensitivity analysis and a simpler fixed value replacement approach that could be used to evaluate the influence of MNAR.

Methods:

We re-analyzed attrition rates and predictors of differential attrition in a sample of 36 mHealth RCTs drawn from a recent meta-analysis of smartphone-based mental health interventions. We systematically evaluated design features related to missingness and its handling. We consider two examples of modern methods applicable to MNAR (selection models, pattern-mixture models) as well as a simple alternative approach that can help researchers understand the potential consequences of MNAR on bias. Data from a recent mHealth RCT are used to illustrate two sensitivity analysis approaches.

Results:

Attrition in active conditions was on average roughly twice that in passive controls. Differential attrition was higher in larger studies and was associated with the use of MAR-based multiple imputation or maximum likelihood methods. Half of the studies (50.00%) used these modern missing data techniques. None of the 36 mHealth RCTs reviewed conducted a sensitivity analysis to evaluate the possible consequences of MNAR. Selection models, pattern-mixture models, and a fixed value replacement sensitivity analysis approach are reviewed. Results from a recent mHealth RCT were shown to be robust to missing data reflecting worse outcomes in missing versus non-missing scores in some but not all scenarios. A review of such scenarios helps qualify observations of significant treatment effects.

Conclusions:

MNAR data due to differential attrition are likely in mHealth RCTs using passive controls. Sensitivity analyses are recommended to allow researchers to assess the potential impact of MNAR on trial results.