JMIR Publications

ABSTRACT

Background:

Understanding patients’ experiences is essential for advancing patient-centered care, especially in chronic diseases that require ongoing communication. However, qualitative thematic analysis, the primary approach for exploring these experiences, remains labor-intensive, subjective, and difficult to scale.

Objective:

The objective of our study was to develop a multi-agent large language model framework for automating qualitative thematic analysis and examine its performance and role in AI-human collaboration.

Methods:

In this study, we developed a multi-agent large language model framework that automates qualitative thematic analysis through three agents (Instructor, Thematizer, CodebookGenerator), named Collaborative Theme Identification Agent (CoTI). We applied CoTI to 12 heart failure patient interviews to analyze their perceptions of medication intensity. The outputs (key phrases, themes, and codebook) were compared with those generated by a senior investigator. We also implemented CoTI into a user-facing application to enable AI–human interaction in qualitative analysis.

Results:

CoTI identified key phrases, themes, and codebook that were more similar to those of the senior investigator than both junior investigators and baseline NLP models. However, collaboration between CoTI and junior investigators provided only marginal gains.

Conclusions:

CoTI can automate qualitative thematic analysis with performance comparable to expert qualitative analysis. Limited gains from AI–human collaboration suggest junior investigators may over-rely on CoTI and limit their independent critical thinking.