JMIR Publications

ABSTRACT

Background:

Virtual clinical triage is central to many digital front-door strategies, yet how interaction mode - a structured, closed-ended artificial intelligence (AI) questionnaire versus a Large Language Model (LLM)-enabled conversational dialogue - affects information capture, engagement, and alignment with care recommendations is unclear.

Objective:

To compare demographics, elicited clinical content, engagement, and alignment with recommended levels of care between a traditional structured-questionnaire interface (Traditional Triage; TT) and an LLM-enabled conversational interface (Conversational Triage; CT) that share the same validated Bayesian reasoning engine.

Methods:

This retrospective observational comparative study analyzed 116,890 finished encounters over 28 weeks (January - August 2025) completed on a publicly available virtual triage website. Users self-selected TT or CT. Primary endpoints included structured capture of demographic and clinical characteristics, triage recommendation adherence, and user engagement patterns. Analyses used post-stratification weighting by age and sex. Group differences were assessed with the Chi-square and Mann–Whitney U tests. For CT, opening and closing user sentiment (positive/neutral/negative) were labeled using a predefined Gemini 2.5 Flash prompt.

Results:

Of 116,890 encounters, 100,533 (86.0%) used TT and 16,357 (14.0%) used CT. Female users were the majority in both groups but less common in CT (64% vs 71%); TT skewed younger (18–29 >50%), whereas CT was more evenly distributed with higher shares at 12–17, 30–44, and ≥45 years old. Median session duration was longer with CT than TT (8min 21s vs 4min 25s respectively). CT elicited more total clinical findings (median 36 vs 32; P<.05) and surfaced more mental-health evidence (e.g., anxiety 15.5% vs 13.5%; depressive symptoms 8.5% vs 6.2%; all P<.05). Post-triage intent survey completion was higher with CT (31.2% vs 5.9%) underscoring enhanced engagement. Overall adherence to the recommended level of care was higher with CT (34.3% vs 29.2%; P<.05), especially at the extremes of acuity: self-care (85.4% vs 61.9%), emergency room (23.7% vs 10.5%), and ambulance (11.5% vs 4.9%; all P<.05). Among CT encounters, positive sentiment towards interactions with the tool increased from 1.3% at opening to 18.7% by close, while negative rose from 1.7% to 7.7%.

Conclusions:

The LLM-enabled Conversational Triage captured richer clinical context and elicited greater engagement, with higher adherence to recommendations - particularly for both low-acuity self-care and high-acuity emergency scenarios where reassurance or urgent escalation are critical. It also reached a more demographically diverse user base, with a more balanced gender profile and broader age distribution than the traditional questionnaire-based triage. These findings highlight the potential of hybrid LLM tools, which integrate validated clinical logic, to enhance digital front-door accessibility, patient trust, and care-navigation efficiency. Clinical Trial: N/A