JMIR Publications

ABSTRACT

Background:

The stress caused by multiple aspects of veteran's transition from military to civilian like, termed Transition Stress, represents a unique source of psychological impact that is under-researched due to its qualitative nature. The assessment of this complex psychological phenomena, has thus relied on laborious interviews designed to extract quantitative information from qualitative narratives of the transition to Civilian life. We sought to determine if large language models (LLMs) could be used as valid measurement tools to extract relevant information from open-ended narratives.

Objective:

This study sought to develop and validate a Generative Artificial Intelligence (GenAI) approach to automate the quantification and subsequent thematic analysis of veteran transition stress.

Methods:

Utilizing transcripts from interviews of a sample of US military veterans we developed a Large Language Model (LLM) to rate transition stress severity, and examined the model’s reliability in relation to human coders, and its validity in relation to a set of related questionnaire measures. Next, we used the LLM scores to quantitatively define high- and low-transition stress groups, enabling a targeted, automated thematic analysis of differences in the narratives from the two groups.

Results:

The LLM ratings correlated very highly with transition stress ratings from human experts and showed significant, theoretically congruent correlations with measures of clinical symptoms, reintegration difficulties, and veterans' self-ratings of transition difficulty. Critically, the AI-derived thematic analyses of high- and low-transition stress veterans revealed distinct narrative structures of resilience and chronic stress characterized by themes of deployment, loss/trauma, material/emotional support and strategies.

Conclusions:

These findings suggest that GenAI offers a robust, scalable, and valid method for multidimensional analysis of complex, narrative-based psychological constructs.