JMIR Publications

ABSTRACT

Background:

Stroke is a leading cause of disability, often accompanied by Unilateral Spatial Neglect (USN), which severely impairs recovery. Traditional assessments like paper-pencil tests provide limited insights into behaviors and eye-hand coordination deficits. Advances in hand pose estimation and eye-tracking in combination with augmented reality (AR) offer potential for data-driven assessments of naturalistic interactions.

Objective:

This study aims to validate an AR-based framework designed to measure eye-hand coordination and attentional biases in stroke patients with USN by capturing head, gaze, hand, and body movements in real-time.

Methods:

We developed an AR-based assessment system using Microsoft HoloLens 2 and an external body-tracking camera to capture real-time gaze, hand, and body movements in an interactive environment. In a preliminary validation study, seven right-brain-lesion patients with mild to moderate USN and eight healthy controls participated. Each performed a designed reaching task, stamping virtual sheets of paper that appeared randomly on a table. We analyzed participants' search behavior patterns to assess attentional biases and examined gaze anchoring timing during targeted reaching motions to explore potential eye-hand coordination deficits.

Results:

In contrast to the control group, USN patients displayed significant ipsilesional biases in gaze direction during visual exploration (Median 7.5 deg, p < .05), and took longer to find contralesional targets (Median difference 1.08 s, p = .02), displaying typical USN behavior. Additionally, they exhibited lateral differences in gaze anchoring during targeted reaching motions, with earlier fixation on contralesional targets (Median difference: 112 ms, p = .02), highlighting potential eye-hand coordination deficits not captured by traditional assessments.

Conclusions:

The proposed AR framework provides a novel, comprehensive data-driven method for capturing interaction behavior in a controlled, yet naturalistic environment. Preliminary validation demonstrated its effectiveness in measuring established USN symptoms, including gaze and head biases, and highlighted its potential to complement traditional assessments by offering deep insights into torso rotation and eye-hand coordination with a high resolution and accuracy. This data-driven approach shows promise for enhancing current USN assessment practices and gaining new insights into patients’ behaviors. Clinical Trial: Trial Registration: ID 2017-02195 with the Ethics Committee of the Canton of Lucerne, Switzerland.