JMIR Publications

ABSTRACT

Background:

Behavioural eHealth and mHealth interventions have been moderately successful in increasing physical activity, although opportunity for further improvement remains. Chatbots equipped with natural language processing can interact and engage with users. Chatbots can also help continuously self-monitor physical activity levels using data from wearable sensors and smartphones. However, there is lack of studies evaluating effectiveness of chatbot interventions on physical activity.

Objective:

To investigate the feasibility, usability and effectiveness of an interactive machine learning based physical activity chatbot.

Methods:

A quasi-experimental design without a control group was conducted with outcomes evaluated at baseline and six weeks after participants started to use the chatbot. Recruitment was conducted using an existing e-mail list. Participants were asked to wear a Fitbit Flex 1 and connect to the chatbot via Messenger app. The chatbot was able to 1) provide daily updates on the physical activity level; 2) send out daily motivational messages in relation to goal achievement; and 3) automatically adjust the daily goals based on physical activity level in the last 7 days. The chatbot also 1) provided sources of information on the benefits of physical activity; 2) send general motivational messages to encourage participants become more active; and 3) checked participant’s activity history (i.e., the step counts or minutes that were achieved on any day) when triggered by the participants. Information about usability and acceptability were self-reported. Main outcomes were daily step counts recorded by the Fitbit and self-reported physical activity.

Results:

Most participants (n = 116) were female (81.9%), in a relationship (73.3%), Caucasian (87.1%), and full-time workers (70.7%). Their average age were 49.1 years with an average BMI of 32.5. Most experienced technical issues due to an unexpected Facebook policy change (82.3%). The majority of the participants scored the usability of the chatbot (74,3%) and the Fitbit (65.2%) “below average”. About one-third (35.3%) would continue to use the chatbot in the future, 53.1% agreed that the chatbot helped them become more active. On average, 6.7 messages/week were sent to the chatbot and 5.1 minutes/day were spent using the chatbot. At follow up, participants recorded significantly more steps (increase of 627 steps/day, 95%CI = 219, 1035) and total physical activity (increase of 154.2 min/week; 3.58 times higher at follow-up (95%CI = 2.28, 5.63)). Participants were also more likely to meet the physical activity guideline (OR = 6.37, 95%CI = 3.31, 12.27) at follow-up.

Conclusions:

The machine learning based physical activity chatbot was able to significantly increase participants’ physical activity and was moderately accepted by the participants. However, the Facebook policy change undermined the chatbot functionality and indicates the need to use independent platforms for chatbot deployment to ensure successful delivery of this type of intervention.