JMIR Publications

ABSTRACT

Background:

Insomnia is a prevalent sleep disorder closely linked to negative thought patterns and cognitive distortions. Cognitive Behavioral Therapy for Insomnia (CBT-I), a well-established treatment for this disorder, remains underutilized due to limited accessibility. Digital CBT-I (dCBT-I) chatbots have shown potential in addressing these limitations; however, existing chatbots primarily offer unidirectional information delivery, making personalized interventions challenging.

Objective:

This study proposes a novel dCBT-I chatbot integrating Socratic Questioning, Self-Decision, and Psychoeducation techniques. Utilizing a large language model (LLM), the chatbot aims to help users deeply explore their sleep issues and provide tailored, individualized guidance.

Methods:

The chatbot intervention comprised four phases: Psychoeducation, multi-turn Socratic Questioning dialogues, Self-Decision, and method-specific Advice. Language generation utilized a fine-tuned LLaMA 3.1 model (with LoRA tuning), while CBT-I method recommendations were supported by a RoBERTa-based intent classifier. Forty-five university and graduate students (mean age = 23.1 years, SD = 2.5) participated in a single-arm, single-session evaluation. Quantitative assessment was conducted via a 35-item questionnaire using a 5-point Likert scale, and qualitative feedback was gathered through open-ended questions.

Results:

Quantitative analysis showed that 32 out of 35 items achieved mean scores above 3.0. The highest score was obtained for the item "The information provided by the chatbot is trustworthy" (mean = 4.00, SD = 0.71), followed by "Ease of use" (mean = 3.87, SD = 0.94). However, the item "The chatbot seemed empathetic to my concerns" had a lower average score (mean = 3.1), indicating limitations in emotional empathy. The model evaluation demonstrated improvements in CBT-I method classification and natural language generation compared to previous models. Qualitative feedback confirmed the chatbot’s effectiveness in facilitating exploration of sleep issues, though users suggested improvements in empathy and user interface (UI) design.

Conclusions:

The chatbot in this study demonstrated the potential of digital therapeutic tools that combine Socratic Questioning and Self-Decision techniques to enable personalized and proactive interventions. This approach can effectively address the limitations of existing dCBT-I chatbots and represents a promising strategy for user-centered intervention and sustainable sleep management. Clinical Trial: Not applicable.