Accepted for/Published in: Journal of Medical Internet Research
Date Submitted: Jun 23, 2020
Date Accepted: Oct 28, 2020
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Content-based Recommender Support System for Counselors in a Suicide Prevention Chat Helpline: Design and Evaluation Study
ABSTRACT
Background:
The working environment of a suicide prevention helpline requires high emotional and cognitive awareness from chat counselors. A shared opinion among counselors is that as the chat conversation turns more difficult, it takes longer, with more effort, to compose a response, which, in turn, can lead to writer’s block.
Objective:
This study evaluates and then designs supportive technology to see if a support system that provides inspiration can help counselors resolve their writer’s block when they encounter a difficult situation in a chat with a help-seeker.
Methods:
A content-based recommender system with sentence embedding was used to search a chat corpus for similar chat situations. The system showed a counselor the most similar parts of former chat conversations so that the counselor can use previous approaches taken by their colleagues as inspiration. In a within-subject experiment, counselors’ chat replies when confronted with a difficult situation were analyzed to see if experts could see a noticeable difference in these chat replies that were obtained in three conditions: with the help of the support system, written advice from a senior counselor, or when receiving no help. In addition, the system’s utility and usability were measured, and the validity of the algorithm was examined.
Results:
A total of 24 counselors used a prototype of the support system; the results showed that by reading chat replies experts could significantly predict if counselors had received help from the support system or from a senior counselor (p = 0.0035). Counselors scored the information they received from a senior counselor (M = 1.46, SD = 1.91) as significantly more helpful than the information received from the support system or when no help was given at all (M = -0.21, SD = 2.26). Finally, compared with randomly selected former chat conversations, counselors rated the ones identified by the content-based recommendation system as significantly more similar to their current chat (β = 0.30, p < 0.001).
Conclusions:
Support given to counselors influenced how they responded in a difficult conversation. However, the higher utility scores given for the advice from senior counselors seem to indicate that specific actionable instructions are preferred. We expect that these findings will be beneficial for developing a system that can use similar chat situations to generate advice in a descriptive style, hence helping counselors through writer’s block.
Citation
