JMIR Publications

ABSTRACT

Background:

Mild cognitive impairment and early dementia (MCI-ED) are a growing public health challenge, yet more than half of affected patients remain undiagnosed, particularly in home healthcare (HHC). Traditional biomarkers are costly or invasive, and existing federally mandated assessments such as the Outcome and Assessment Information Set (OASIS) lack sensitivity to subtle cognitive decline. Spoken language and interactional cues are among the earliest indicators of cognitive impairment and, when integrated with routinely collected electronic health record (EHR) data, may enable earlier and more equitable detection.

Objective:

This protocol describes the development and evaluation of a multimodal screening algorithm for timely identification of MCI-ED in HHC. The approach leverages three complementary aims: (1) modeling speech and interactional dynamics from patient–nurse encounters, (2) adapting large language models (LLMs) to extract MCI-ED–related information from HHC notes and transcripts, and (3) developing a multimodal screening algorithm that integrates speech, LLM-extracted constructs, and OASIS data.

Methods:

We conduct a cross-sectional case–control study in collaboration with VNS Health among patients aged ≥65 years. Each participant contributes three to four audio-recorded encounters with nurses, yielding approximately 120 minutes of spontaneous speech, which are linked to OASIS assessments and EHR notes. • Aim 1: Speech and interaction modeling. Acoustic, linguistic, emotion, and interactional features are extracted using automated pipelines. Measures include phonetic motor planning, lexical richness, semantic fluency, syntactic organization, emotional expression, and patient–nurse interactional dynamics (e.g., turns, timing, interactivity). • Aim 2: LLM-based extraction. We define an ontology for clinical symptoms, lifestyle risk factors, and communication deficits, and adapt Llama-3.1 models in a secure environment. Hybrid strategies (prompted extraction and parameter-efficient fine-tuning) extract and normalize concepts from HHC notes and transcripts. Outputs are aggregated at the patient level. • Aim 3: Multimodal algorithm. Features from Aims 1–2 are fused with OASIS data. Multiple classifiers, including support vector machines, logistic regression, ensemble trees, and deep neural models, are evaluated using stratified nested cross-validation. Model performance is assessed using AUROC, AUPRC, calibration, and fairness metrics (equality of opportunity/odds) across sex, race, and age groups. The target AUROC is >0.85 across subgroups.

Results:

We enroll 47 HHC patients (51.1% female; 55.3% Black, 29.8% White, 8.5% Asian, 6.4% Hispanic). Encounters average 19 minutes, with nurses contributing more words than patients (median 842 vs. 589). Preliminary analyses show that integrating speech with EHR/OASIS outperforms single-source models. Recruitment and validation are ongoing.

Conclusions:

This protocol demonstrates the feasibility of embedding multimodal MCI-ED screening in HHC. Early results highlight the potential of integrating spontaneous speech, interactional dynamics, and routinely collected clinical data to improve timely identification of cognitive decline. Definitive evaluation of algorithm performance and fairness follows full accrual.