JMIR Publications

ABSTRACT

Background:

Patients require timely, understandable, consistent, and actionable perioperative information. In routine care, however, this information is often distributed across clinicians, written materials, institutional documents, and verbal instructions, making consistent, tailored guidance difficult. Conversational agents (CAs) may help provide scalable perioperative education, but implementation in safety-sensitive pathways requires understanding how stakeholders interpret usefulness, trust, role boundaries, and implementation requirements after interacting with a functioning system.

Objective:

The aim was to evaluate how stakeholders judged the acceptability and appropriateness of the Digital Teammate, after hands-on use, and to identify the requirements for its use in routine perioperative care. Digital Teammate is a bilingual English–French, multimodal, retrieval-augmented conversational agent designed to support perioperative patient education.

Methods:

We conducted an explanatory sequential mixed-methods pilot study at a Canadian tertiary care hospital. Quantitative user acceptance testing (UAT) data were collected first and then expanded through open-ended UAT responses and four role- or perspective-homogeneous focus groups. The analytic UAT sample included 58 participants: 35 healthcare workers and 23 patient participants. A purposively selected subsample of 24 UAT participants took part in four focus groups with physicians, registered nurses, respiratory therapists, and anesthesia assistants, operational and support staff, and patient participants. Quantitative data were analyzed descriptively. Qualitative data were analyzed using structuring qualitative content analysis. Integration was conducted through a mixed-methods joint display to examine how qualitative findings explained, expanded, or qualified the quantitative UAT findings.

Results:

Participants evaluated the Digital Teammate positively across all four UAT domains. Domain-level mean scores were 3.71 (SD 0.79) for usability, 3.95 (SD 0.77) for interaction quality, 3.76 (SD 0.85) for perceived usefulness, and 3.61 (SD 0.96) for acceptance and intention to use. Patient participants rated the system more favorably than healthcare workers across domains and rated standardized perioperative responses favorably more often (87.5% vs 79.3%). Qualitative findings generated six themes: a bounded complementary role, usability and interaction reliability, language clarity versus information consistency, workflow and workload impact, conditional trust, and conditions for safe implementation. Mixed-methods integration generated four overarching meta-inferences: usability was strong but did not by itself establish appropriateness for routine use, the clearest initial use case was routine perioperative preparation and orientation, clinical complexity and safety relevance defined the boundary of stand-alone use, and acceptance was positive but conditional on complementarity, trust, governance, and workflow integration.

Conclusions:

The Digital Teammate was usable, showed favorable initial acceptance, and was perceived as valuable for routine perioperative preparation and orientation. However, safe progression toward patient-facing feasibility testing requires a staged pathway. Structured clinician-led validation and correction of content, interaction, and escalation issues should precede prospective feasibility testing in routine care. Complex, individualized, safety-critical, or emotionally distressing questions should be handled through bounded responses, transparent uncertainty acknowledgement, and escalation to human clinicians when appropriate.