JMIR Publications

ABSTRACT

Background:

Online peer-support forums require oversight to ensure they remain safe and therapeutic. As online communities grow, they place a greater burden on their human moderators, which increases the likelihood that people at risk may be overlooked. This paper evaluates the potential for machine learning to assist online peer support by directing moderators’ attention where it is most needed.

Objective:

To evaluate the accuracy of an automated triage system, and the extent to which it influences moderator behaviour.

Methods:

A machine learning classifier was trained to prioritize forum messages as green, amber, red or crisis depending on how urgently they require attention from a moderator. This was then launched as a set of widgets injected into a popular online peer-support forum hosted by ReachOut.com. The accuracy of the system was evaluated using a hold-out test set of manually prioritised messages. The impact on moderator behaviour was measured as response ratio and response latency, i.e. the proportion of messages that receive at least one reply from a moderator, and how long it took for these replies to be made. These measures were compared across three periods: prior to launch, after an informal launch, and after a formal launch accompanied by training.

Results:

The algorithm achieved 84% f-measure in identifying content that required a moderator response. Between pre-launch and post-training periods, response ratios increased by 0.9, 4.4, and 10.5 percentage points for messages labelled as crisis, red and green respectively, but decreased by 5.0 percentage points for amber messages. Logistic regression indicated that the triage system was a significant contributor to response ratios for green, amber and red messages, but not for crisis messages. Between the same periods, response latency was significantly reduced (p’s<0.001) by factors of 80%, 80%, 77% and 12% for crisis, red, amber and green messages respectively. Regression analysis indicated that the triage system made a significant and unique contribution to reducing the time taken to respond to green, amber and red messages, but not to crisis messages, after accounting for moderator and community activity.

Conclusions:

The triage system was generally accurate, and moderators were largely in agreement with how messages were prioritized. It had a modest effect on response ratios, primarily because moderators were already more likely to respond to high priority content prior to the introduction of triage. However, it significantly and substantially reduced the time taken for moderators to respond to prioritized content. Further evaluations are needed to assess the impact of mistakes made by triage algorithm, and how changes to moderator responsiveness impact the wellbeing of forum members.