JMIR Publications

ABSTRACT

Background:

Generative artificial intelligence (AI), particularly large language models (LLMs), are increasingly studied for clinical reasoning and risk assessment tasks. However, AI outputs can be sensitive to prompt variations in the (1) contextual information provided and (2) prompt framing. The impacts of these variations remain insufficiently characterized in complex and inherently ambiguous domains such as psychiatry.

Objective:

We propose an approach for sensitivity analyses in downstream AI tasks by structuring evaluation around two sources of prompt variation -- prompt context and framing -- by (1) including clinically peripheral information (e.g. favorite color) and (2) changing the rhetorical prompt framing. We measure this sensitivity through the task of LLM-generated psychiatric hospitalization risk scores.

Methods:

We generated 50 structured synthetic patient profiles, each consisting of 15 clinically relevant baseline features (e.g. symptom severity) and up to 50 clinically peripheral (e.g., favorite color) features, where clinically peripheral is defined as features with limited established clinical relevance to hospitalization risk. Output differences based on both sources of variation were evaluated for all profiles using four LLM families across four rhetorical prompt framings (neutral, logical, human impact, and clinical judgment).

Results:

Our findings highlight the need for ongoing sensitivity assessments to ensure the reliability of LLM-based clinical decision-support systems. Adding clinically low-relevance features was associated with statistically significant changes in output stability in 8 of 16 model-prompt conditions suggesting that model sensitivity to clinically low-relevance information varied across models and prompt framings. Prompt framing also affected the trajectory of output stability, with effects varying across models.

Conclusions:

Across model families, AI model outputs were differingly sensitive to these sources of variation, demonstrating the benefit of adding sensitivity analysis as evaluation criteria for clinically-facing AI systems, especially in complex or ambiguous domains like psychiatry.