JMIR Publications

ABSTRACT

Background:

Current large language models often fall short of authentic counseling expertise and adaptive interaction capabilities in empathic dialogue, resulting in inadequate emotional support.

Objective:

This study aimed to construct a large-scale emotional support dataset, enhance LLMs’ capacity to provide more professional emotional support, and establish an evaluation framework for LLMs’ empathic dialogue.

Methods:

To address this, we develop a comprehensive empathy-enhancement pipeline consisting of the ECD-3 dataset and the E-URIAL framework. In particular, ECD-3 is a dataset that features 104,897 counselor-response turns using fine-grained psychological annotations. Leveraging this foundation, we develop YuanChat, a family of empathy-enhanced large language models with finetuning. To further align model outputs with more professional standards, we design E-URIAL, a dynamic, retrieval-augmented in-context learning framework. E-URIAL integrates specific exemplars with strategic guidance, ensuring responses are both contextually appropriate and empathically resonant.

Results:

Experiments demonstrate that YuanChat variants outperform their foundation models, improving empathic language cues (increasing from 0.57 to 0.68) and human-rated empathy (increasing from 0.76 to 0.86). Notably, the E-URIAL framework demonstrates consistent performance boosts with a small, diverse set of exemplars ( = 3), suggesting its efficiency in resource-constrained environments.

Conclusions:

Overall, this study presents a scalable and theoretically grounded framework for deploying empathic artificial intelligence in practical mental health applications. It also holds potential for mental healthcare contexts by improving the accessibility and consistency of empathic, psychologically informed support.