JMIR Publications

ABSTRACT

Conversational user interfaces, or chatbots, are becoming more popular in the realm of digital health and wellbeing. Whilst many studies focus on measuring the cause of effect of a digital intervention on people's health and wellbeing (outcomes), there is a need to understand how users really engage and use a digital intervention in the real world. In this study we examine the user logs of a mental wellbeing chatbot called ChatPal, which was developed by a collaboration between universities and mental health charities based on the concept of positive psychology. ChatPal log data relating to 579 users revealed that 60% were recruited specifically for the trial with the majority of users being female (67%). User interactions were recorded throughout the day with peak times being around Breakfast and Lunchtime and early evening. Overall usage. user retention over the period was better than average for applications of a similar nature. A number of user characteristics such as user tenure, unique days, mood logs recorded, conversations accessed, and total number of interactions were used with k-means clustering to identify user archetypes. This identified two main groups: Abandoning users and Invested users before being refined to a three-cluster solution resulting in Abandoning users (n=473), Sporadic users (n=93) and Frequent Transient users (n=13) each with distinct usage characteristics. The principal feature of the application, the ability to participate in several conversations with the chatbot was examined to discover usage patterns. While all conversations were accessed at least once, several were more popular with the ‘Treat yourself as a friend’ conversation being the most popular, which was accessed by 29% of users although only 11.7% of users repeated the exercise more than once. This trend was evident for all conversations though ‘Thoughts Diary’ had the lowest drop in repeated usage going from 23.5% (used at least once) to 18.8% (used at least twice). Analysis of transitions between conversations revealed strong links between ‘Treat yourself as a friend', ‘Soothing Touch’ and ‘Thoughts Diary’ among others. Association Rule Mining confirmed these three conversations as having the strongest linkages and suggested other associations between the co-use of chatbot features. This study has provided insight into the types of people using the ChatPal chatbot, patterns of use and associations between the usage of the application’s features which can be used to further develop the application taking into account the features most accessed by users.