JMIR Publications

ABSTRACT

Background:

Generative artificial intelligence (AI) tools, including large language model (LLM)-based agents, are increasingly explored for medical and nursing education. However, longitudinal empirical studies examining how individual educators collaborate with AI agents to develop digital teaching materials remain scarce. Most prior work has focused on short-term usability evaluations or performance benchmarks of completed systems, leaving the iterative development process itself under-examined.

Objective:

This study aimed to (1) characterize the collaborative workflow between a single nurse educator and an AI agent (Claude Code) across 19 months of continuous digital material development, (2) quantify changes in the educator's prompt-engineering behavior over time, and (3) derive reproducible conditions and a phase model that may guide other educators seeking to adopt AI agents for curriculum development.

Methods:

A hybrid Design-Based Research (DBR) and Design Science Research (DSR) methodology was employed. The practitioner-researcher (a nurse educator) collaborated with Claude AI (web/desktop, November 2024-May 2025) and Claude Code (terminal-based AI agent, May 2025 onward) over 19 months (November 2024-May 2026), accumulating approximately 1,000 hours, 41,914 messages across 609 sessions, and producing 14 or more digital artifacts spanning six technical layers. Dialogue data (n=8,286 utterances from 20 sampled sessions) were classified into six speech-act categories using Berelson content analysis; inter-rater reliability was assessed by two independent coders (Cohen kappa=0.565, moderate agreement). Specificity density, a novel metric defined as the ratio of domain-specific directives to total educator utterances, was tracked longitudinally across nine iterative design cycles.

Results:

A five-phase collaborative workflow model emerged: (1) natural-language trial-and-error, (2) structured instruction via CLAUDE.md norm files, (3) domain-specific prompt maturation, (4) multi-agent orchestration through Model Context Protocol (MCP) connectivity, and (5) autonomous agent delegation. Specificity density rose from 0.21 (Phase 1) to 0.91 (Phase 5), indicating progressive externalization of the educator's tacit clinical and pedagogical knowledge. Domain success rates varied substantially (web/HTML artifacts: 57%; 3D lip-sync systems: 10%), reflecting differential technical complexity. Five reproduction conditions were identified: (C1) a norm-file ecosystem, (C2) iterative prompt maturation, (C3) failure-log accumulation, (C4) MCP connectivity, and (C5) sustained engagement exceeding 500 hours.

Conclusions:

This 19-month longitudinal case study demonstrates that sustained educator-AI agent collaboration can yield a rich portfolio of digital nursing education materials when supported by structured norm files, iterative prompt refinement, and failure-log-driven learning. The five-phase model and five reproduction conditions offer an empirically grounded framework for educators and institutions considering the integration of AI coding agents into health professions education. Limitations include the single-practitioner design (n=1), reliance on one AI platform family, and the absence of direct learner-outcome evaluation. Future multi-site studies with controlled comparisons and student performance metrics are warranted.