JMIR Publications

ABSTRACT

Background:

Body image dissatisfaction, disordered eating and eating disorders represent significant public health concerns, however, many affected individuals never access evidence-based support. We co-designed and developed a rule-based chatbot, JEM, which conducted conversations addressing evidence-based psychoeducation and psychotherapeutic microinterventions. We previously demonstrated feasibility, acceptability and preliminary satisfaction for JEM chatbot in a research setting. However, broader satisfaction, experiences and impacts on users in real-world settings had not been investigated.

Objective:

To conduct a real-world evaluation of JEM chatbot in Australia and Canada as the two countries who have hosted a deployment of the chatbot to date. Specifically, we aimed to explore user satisfaction and experiences with the chatbot and changes in user mood and body image satisfaction when completing the chatbot’s microinterventions.

Methods:

Participants were users of JEM chatbot aged 13-64 years who self-selected to complete a web-based overall evaluation survey, N=230 (n=122 in Australia and n=108 Canada), over a 6-month period. This evaluation survey included user demographic characteristics, satisfaction measures and the system usability scale. Participants were also chatbot users over the same 6-month period who chose to complete brief web-based surveys immediately before and after completing one of the chatbot’s microinterventions, N=75 to 276 (n=34 to 146 in Australia and n=39 to 130 in Canada). These surveys included validated visual analogue scales measuring mood (anxiety, depression, happy, confident) and body image satisfaction (body size satisfaction, body shape satisfaction, physically attractive).

Results:

Demographic characteristics showed that participants were commonly young adult cisgender women or non-binary individuals across Australia and Canada. Participant satisfaction with the chatbot was high in both countries (Australia: M=76.1, SD=22.7; Canada: M=78.8, SD=14.3) and usability of the chatbot was rated as “excellent” in both (Australia: M=86.5, SD=16.9; Canada: M=89.5, SD=11.6) according to the system usability scale, with no significant differences between countries (all Ps>.05). In general, each microintervention yielded significant reductions in anxiety (P<.001 to .032, Cohen d=0.37 to 0.53) and depression (P<.001 to .033, Cohen d=0.25 to 0.79) immediately after completion with small to large effect sizes in both countries. Furthermore, increases in body size satisfaction were generally found after microintervention completion with small to moderate effect sizes (P=.019 to .027, Cohen d=0.20 to 0.51). There were no significant differences between countries for participant responses to the microinterventions (all Ps>.05).

Conclusions:

JEM chatbot achieved high satisfaction and usability ratings and yielded improvements in mood and body image satisfaction following microinterventions in real-world settings across two continents. Effects were consistent across contexts, suggesting that the intervention content translated effectively across diverse populations. Although these findings reflect a subset of users who self-selected to provide feedback, they provide some evidence for the feasibility and potential impact of JEM, indicating the value of further evaluation and broader deployment. Clinical Trial: N/A