JMIR Publications

ABSTRACT

Background:

Previous efforts to apply machine learning (ML) based natural language processing (NLP) to longitudinally collected social media data have shown promise to predict suicide risk.

Objective:

Our objective was to externally validate our previous ML algorithm, the suicide artificial intelligence heuristic (SAIPH), against external survey data in two independent cohorts and to assess the ability of SAIPH to act as an indicator of changing suicidal ideation (SI) trajectory both alone and in the context of responses to suicidal mentions online.

Methods:

SAIPH was applied to individual Twitter timeline data from respondents to a Student Survey Cohort (N=159) and a national COVID-19 Survey Cohort (N=307, N=193 with longitudinal follow up) and compared to SI derived from the Beck Depression Inventory and Quick Inventory of Depression Self Report (QIDS-SR), respectively. Responses received within 72 hours to N=471 suicidal mentions were gathered from Twitter between 2019-2022 to evaluate their influence on SI trajectory.

Results:

Mean SAIPH scores derived from fourteen days of Twitter data prior to survey completion predicted the highest level of suicidal ideation (SI) in each cohort with AUCs of 0.71 (95% CI: 0.47-0.96) and 0.87 (95% CI: 0.74-0.99), respectively. A novel ensemble method was generated to predict future SAIPH scores from an individual’s historical data, which successfully predicted SI across cohorts. The slope of average daily SAIPH scores was associated with the change in SI scores within longitudinally followed individuals when evaluating periods of two weeks or less (Rho =0.27, p=0.038). SAIPH scores were associated with perceived stress scores (PSS) but not anxiety or depression scores. Using SAIPH as an indicator of changing SI, we evaluated SI trajectory in two cohorts of suicidal mentioners and identified that those with responses within 72 hours exhibit a significant negative association of SAIPH score with time in the three weeks following suicidal mention (Rho=-0.52, p= 0.014).

Conclusions:

Taken together, our results not only validate the efficacy of SAIPH to function as an indicator of SI and perceived stress, but generate novel methods to forecast these indicators forward in time and provide us with a tool to evaluate the effects of social media interactions on changing suicidal trajectory.