STUDY OBJECTIVE: Suicide crisis chat counsellors work in an environment which de- mands high emotional and cognitive awareness. A shared opinion among counsellors is that as the chat conversation turns more difficult it takes longer and more effort to come up with a response. Supportive technology might resolve this ”writer’s block” by giving inspiration in the form of responses other counsellors gave to similar situations. 113 Suicide Prevention has an extensive chat corpus of previous conversation between help-seekers and counsellors that can be leveraged to find these situations. This thesis focuses on the design, development and experimental evaluation of a support system that aims to solve the problem of writer’s block for counsellors in suicide crisis chats.
METHODS: A context-based retrieval system was build. SIF sentence embedding was used to find similar partial chats using cosine similarity. The chats in the corpus were split up and embedded using a sliding window approach, to retrieve only the relevant parts in the chat and reducing reading time for the counsellor. A within-subject experimental design (n=24) with three conditions (no support, support system, expert advice) was used to measure the system’s usefulness and to test the hypothesis that the system has a noticeable difference on counsellor responses, by using experts to blindly label the responses. Furthermore, another within-subject experimental design was used to test the hypothesis that the counsellors can distinguish be- tween partial chats provided by the retrieval system and partial chats randomly selected from the corpus. For this part of the evaluation, counsellors blindly rated suggestions on how much they related to the context of a given chat.
RESULTS: Counsellors rated the support system with a 71.04 on the SUS questionnaire, cor- responding to a good adjective rating. There was variability between chats for counsellor rating for utility of the information provided by the support system. Experts noticed a significant difference between the support and expert advice conditions. Lastly, the suggestions based on the context positively affected counsellor rating of relatedness to context of the chat. CONCLUSION: Technology using NLP techniques can provide useful information for only chat counsellors to help with writer’s block. Improving the quality of the recommendations, is expected to help improve the usefulness of the system.