JMIR Publications

ABSTRACT

Background:

Sexual and reproductive health (SRH) chatbots are built and evaluated as information systems. Less is known about what users of a deployed SRH chatbot expect it to do, or whether expectations extend beyond information to assessment, connection to services, and continuity of support.

Objective:

This study aimed to characterize what young adults expect an SRH chatbot to do, immediately after using one, to inform design and evaluation: (1) how capability demand is distributed across 4 care-pathway functions (informing, assessing, connecting, and sustaining), (2) the proportion wanting information seeking alone, (3) the structure of capability evaluations, and (4) where gaps between valued and deficient capabilities are widest.

Methods:

We conducted a cross-sectional web-based survey of 1200 adults aged 18 to 32 years in the United States, recruited through a commercial research panel. Each participant completed a 5- to 10-minute guided interaction with Roo, Planned Parenthood Federation of America's free, nongenerative guided-flow SRH chatbot, then ranked 10 features, rated 11 capabilities on a 7-point importance scale, and completed checklists of desired improvements, additions, and anticipated uses. Analyses included a Plackett-Luce ranking model, composite demand indicators, exploratory and split-half confirmatory factor analysis, and importance-by-unmet-demand analysis.

Results:

Demand was present at every care-pathway function. Accurate information ranked first of 10 features and was rated most important (mean 5.92, SD 1.53), yet 54% of participants named accuracy as needing improvement; a symptom checker was the most requested addition (62.7%), and 46.4% wanted video consultation with professionals. In total, 87.8% (1054/1200) wanted a service-connection capability or a use beyond information seeking (median 2 such endorsements), and 8.2% (98/1200) wanted information seeking alone. Exploratory factor analysis supported 2 components (63.5% of variance): a core health-utility component combining accuracy and confidentiality (loadings 0.90) with educational resources, personalized advice, a symptom checker, and real-time expert access (0.70-0.76), and a lower-rated engagement and customization component. In a held-out half, a confirmatory model separating trust attributes from consultative capabilities fit better than a 2-factor model (comparative fit index 0.93 vs 0.85); trust and consultative capability correlated 0.84, each less with engagement (0.33 and 0.65). The widest gaps between importance and perceived deficiency concerned accurate information, real-time expert access, and the symptom checker rather than content breadth.

Conclusions:

Young adults evaluated the chatbot as a first point of consultation that should interpret their situation and connect them to care, with accuracy and confidentiality as preconditions. The capabilities they most wanted, symptom assessment and real-time access to a professional, are those SRH professionals have considered least appropriate for chatbots, placing users' demand where clinical caution is greatest. Services should treat accuracy and confidentiality as gating requirements, prioritize assessment and connection over engagement features, build chatbot-mediated escalation to human channels, and add role-performance outcomes to accuracy-based evaluation.