JMIR Publications

ABSTRACT

Background:

Family caregivers of children with chronic health conditions experience significant physical and mental health burdens, including burnout, anxiety, depression, fatigue, and sleep disturbances. Despite this growing need, validated digital mental health tools tailored specifically to this population remain limited. Conversational agents powered by artificial intelligence (AI) offer a promising avenue for delivering on-demand, personalized mental health support, yet evidence-based development and evaluation of such tools for family caregivers is lacking.

Objective:

This study aimed to systematically describe the iterative development of COCO (Caring of Caregivers Online), a conversational agent designed to deliver Problem-Solving Therapy (PST) integrated with Motivational Interviewing (MI) principles, and to evaluate its usability and preliminary effect on the emotional well-being of family caregivers of children with chronic health conditions.

Methods:

COCO was developed and refined across four phases. Therapeutic dialogues were grounded in PST and MI principles and informed by evidence-based caregiver personas. The Wizard-of-Oz (WOZ) method was used across phases to iteratively collect naturalistic dialogues and refine COCO's conversational design. Usability was assessed using the System Usability Scale (SUS) and the Post-Study System Usability Questionnaire (PSSUQ). Caregiver emotions were measured pre- and post-session using six subscales of the Positive and Negative Affect Schedule - Expanded Scale (PANAS-X). In the final phase, a large language model (LLM)-powered version of COCO was developed using GPT-4 with few-shot learning and evaluated using persona-based methods.

Results:

COCO achieved a mean SUS score of 75.6%, reflecting acceptable usability. Participants demonstrated significant improvements in negative affect, sadness, guilt, fatigue, and serenity following PST sessions (p ≤ 0.03). Analysis of MI techniques across all phases revealed progressive refinement in conversational quality, with the LLM-powered COCO achieving the highest density of MI techniques per turn (2.56) and greater balance across MI strategy types, particularly in seeking collaboration and reflection.

Conclusions:

COCO is a feasible, usable, and preliminarily efficacious conversational agent for supporting the mental health of family caregivers of children with chronic conditions. The iterative, human-in-the-loop development approach was instrumental in producing empathetic, therapeutically grounded responses. The systematic development and evaluation process described here can serve as a guide for similar conversational agent intervention studies. Future work will explore multi-agent architectures and retrieval-augmented generation (RAG) to further enhance personalization, controllability, and scalability toward clinical deployment.