JMIR Publications

ABSTRACT

Background:

Many mental health conditions (e.g., substance use or panic disorders) involve long-term patient assessment and treatment. Growing evidence suggests that the progression and presentation of these conditions may be highly individualized. Digital sensing and predictive modeling can augment scarce clinician resources to expand and personalize patient care. This manuscript discusses techniques to process patient data into risk predictions, for instance the lapse risk for a patient with alcohol use disorder (AUD). Of particular interest are idiographic approaches that fit personalized models to each patient.

Objective:

This manuscript bridges two active research areas in mental health: risk prediction and time-series idiographic modeling. Existing work in risk prediction has focused on machine learning (ML) classifier approaches, typically trained at the population level. In contrast, psychological explanatory modeling has relied on idiographic time-series techniques. The authors propose state space models (SSMs), an idiographic time-series modeling framework, as an alternative to ML classifiers for patient risk prediction.

Methods:

The authors used a 3-month observational study of participants (N=148) in early recovery from AUD. Using once-daily ecological momentary assessments (EMA), the authors trained idiographic SSMs and compared their predictive performance to logistic regression and gradient-boosted ML classifiers. Performance was evaluated using the area under the receiver operating characteristic curve (auROC) for three prediction tasks: same-day lapse, lapse within 3 days, and lapse within 7 days. To mimic real-world use, the authors evaluated changes in auROC when models were given access to increasing amounts of a participant’s EMA data (15, 30, 45, 60, and 75 days). The authors used Bayesian hierarchical modeling to compare the SSMs to the benchmark ML techniques, specifically analyzing posterior estimates of mean model auROC.

Results:

Posterior estimates strongly suggested that SSMs had the best mean auROC performance on all three prediction tasks with 30+ days of participant EMA data. With 15 days of data, results varied by task. Posterior probabilities that SSMs had the best performance with 30+ days of participant data, given as (first quartile, median, third quartile), for the three prediction tasks were (.877, .997, .999), (.992, .999, .999), and (.995, .998, .999). With 15 days of data, these posterior probabilities were .732, <.001, and <.001.

Conclusions:

This study suggests that SSMs may be a compelling alternative to traditional ML approaches for risk prediction. SSMs support idiographic model fitting, even for rare outcomes, and can offer better predictive performance than existing ML approaches. Further, SSMs estimate a model for a patient’s time-series behavior, making them ideal for stepping beyond risk prediction to frameworks for optimal treatment selection (e.g., administered using a digital therapeutics platform). While AUD is used as a case study, this SSM framework can be readily applied to risk prediction tasks in other mental health conditions.