JMIR Publications

ABSTRACT

Background:

Hepato-pancreato-biliary (HPB) oncology presents significant cognitive, spatial, and communicative challenges. While augmented reality (AR) technologies are emerging as valuable tools in surgical planning, most existing systems are developed in a technologically deterministic manner, often without consideration of patients’ lived experiences or the collaborative needs of multidisciplinary clinical teams. This limits their integration into complex workflows and their potential for meaningful clinical impact.

Objective:

This study aimed to co-design and evaluate an AR system that supports spatial understanding, emotional sensitivity, and collaborative decision-making in HPB surgical oncology. Specifically, the research sought to develop a human-centred AR platform that could reduce cognitive load, facilitate emotionally calibrated visualisations, and enhance communication across diverse user roles, including clinicians and patients.

Methods:

A four-phase human-centred design methodology was employed. In Phase 1, co-design workshops were conducted with 14 oncology professionals and 10 cancer patients to identify key user needs. Phase 2 involved synthesis of these insights into system design principles. In Phase 3, an AR prototype was developed using Unity and HoloLens 2, featuring interactive 3D models of liver anatomy based on contrast-enhanced CT imaging. In Phase 4, the system was evaluated in simulated multidisciplinary tumour board planning scenarios. Usability was assessed, and qualitative feedback was collected through observation and interviews.

Results:

The co-design process led to the identification of several intersecting spatial, cognitive, and communicative challenges in HPB surgical oncology. Through iterative development, an AR prototype was created that addressed these challenges by serving as a shared cognitive and communicative space. The system enabled collaborative manipulation of 3D anatomical models, emotionally calibrated visualisations, and seamless integration with existing planning workflows. In simulated multidisciplinary tumour board scenarios, clinicians reported improved clarity in spatial discussions, reduced cognitive load, and enhanced interprofessional coordination. Patients valued the ability to engage with their own anatomy in an emotionally manageable way and reported the need of a greater sense of understanding and agency. The prototype demonstrated the potential to dynamically adapt to different user roles, clinicians, patients, and support staff, without compromising interpretive clarity. Overall, the findings suggest that AR systems designed with human-centred methods can function not only as visualisation tools but also as boundary objects that bridge communication gaps and support distributed cognition in complex cancer care settings.

Conclusions:

A human-centred co-design process can guide the development of AR systems that extend beyond technical novelty to become emotionally attuned, cognitively supportive, and workflow-compatible tools in surgical oncology. By foregrounding the needs of both patients and professionals, AR can function as a boundary object that facilitates distributed cognition, empathetic dialogue, and collaborative decision-making. This study offers actionable design for developing AR technologies that are meaningfully integrated into complex clinical contexts.