JMIR Publications

ABSTRACT

Background:

Over 100 million Americans lack affordable access to behavioral health care, and among these, military veterans are an especially vulnerable population. Military veterans require unique behavioral health services that can address military experiences in addition to transition difficulties to the civilian sector. Real world programs to help veterans successfully transition to civilian life must build a sense of community, have the ability to scale, and be able to reach the many veterans who cannot or will not access care. Digitally based behavioral health initiatives have emerged within the past few years to improve this access to care. Our novel behavioral health intervention teaches mindfulness-based CBT and narrative therapy using peer support groups as guides, with human facilitated asynchronous online discussions. Historically, however, the standard of assessing efficacy of a therapeutic intervention has been subjective symptom measures pre and post treatment. Our study sought to use natural language processing (NLP) analytics to assess effectiveness of our on-line intervention, because NLP may provide insights and detect nuances of personal change and growth that are not currently captured by subjective symptom measures.

Objective:

To study the value of natural language processing (NLP) analytics in assessing progress and outcomes among combat veterans and military sexual assault survivors participating in novel online interventions for post traumatic growth.

Methods:

IBM Watson and Linguistic Inquiry Word Count tools were applied to the narrative writings of combat veterans and survivors of military sexual trauma who participated in novel online peer supported group therapies for post traumatic growth. Participants watched videos; practiced skills such as mindfulness meditation; told their stories through narrative writing; and participated in asynchronous, facilitated online discussions with peers. The writings, including online postings, by the 16 participants who completed the program were analyzed after completion of the program.

Results:

NLP demonstrated efficacy of the intervention with large effect sizes (d > 0.8, p < 0.05) across emotional tone measures. Emotional tone analysis demonstrated significant decreases in fear/anxiety, sadness, and disgust, as well as increases in joy. Significant effects were found for personal values such as needing/desiring closeness and helping others, and for personality traits of openness, conscientiousness, extroversion, agreeableness and neuroticism (aka emotional range). Participants also demonstrated increases in authenticity and clout (confidence) of expression. NLP results were generally supported by qualitative observations and analysis, structured data and course feedback.

Conclusions:

The aggregate of results in our study suggest the effectiveness of our behavioral health intervention and that NLP can provide valuable insights on shifts in personality traits, personal values and needs, as well as measure changes in emotional tone. NLP’s sensitivity to changes in emotional tone, values and personality strengths suggest efficacy of NLP as a leading indicator of treatment progress.