JMIR Publications

ABSTRACT

Background:

Language use and social interactions have demonstrated a close relationship with cognitive measures. It is important to improve the understanding of language use and behavioral indicators from social context to study the early prediction of cognitive decline among healthy populations of older adults.

Objective:

This study aims at predicting an important cognitive ability, working memory, of n=98 healthy older adults participating in a four days-long naturalistic observation study. We used linguistic measures, part-of-speech (POS) tags and social context information extracted from 7450 real-life audio recordings of their everyday conversations.

Methods:

The methods in this study comprise 1) the generation of linguistic measures (representing idea density, vocabulary richness, and grammatical complexity) and POS-tags with natural language processing (NLP) from the transcripts of real-life conversations, and 2) the training of machine learning models to predict working memory using linguistic measures, POS-tags and social context information. We measured working memory using the 1) “Keep Track” test, 2) “Consonant Updating” test, and 3) a composite score of “Keep Track” and “Consonant Updating.” We trained machine learning models using random forests (RF), implementing repeated cross-validation with different numbers of folds and repeats and recursive feature elimination to avoid overfitting.

Results:

For all three prediction routines, models comprising linguistic measures, POS-tags and social coded information improved the baseline performance on the validation folds and on the whole dataset. The best model for the “Keep Track” prediction routine comprises linguistic measures, POS-tags and social context variables, with R^2=0.75. The best models for “Consonant Updating” and the composite working memory score comprise POS-tags and linguistic measures, with R^2=0.40 and R^2=0.45 respectively. The performance of the best models of all three prediction routines is in line with (or it improves) the one of benchmarks in the literature on the modeling of cognitive abilities with behavioral indicators.

Conclusions:

The results suggest that machine learning and NLP may support the prediction of working memory using, in particular, linguistic measures and social context information extracted from the everyday conversations of healthy older adults. Our findings may support the design of an early warning system to be used in longitudinal studies that collects cognitive ability scores and records real-life conversations unobtrusively. This system may support the timely detection of early cognitive decline. In particular, the use of a privacy-sensitive passive monitoring technology would allow designing a program of interventions to enable strategies and treatments to decrease or avoid early cognitive decline.