JMIR Publications

ABSTRACT

Background:

Conversational chatbots are an emerging digital intervention for smoking cessation. No studies have reported on a user-centered design development process for creating cessation chatbots.

Objective:

To describe the user-centered design development process for a novel quit smoking conversational chatbot called “QuitBot.”

Methods:

The four years of formative research for developing QuitBot followed an eleven-step process: (1) specifying a conceptual model, (2) conducting content analysis of existing interventions, (3) assessing user needs, (4) developing the chat’s persona (“personality”), (5) prototyping content and persona, (6) developing full functionality, (7) programming the QuitBot, (8) conducting a diary study, (9) conducting a pilot randomized trial, (10) reviewing results of the trial, and (11) adding a free-form question and answer (QnA) function, based on user feedback from pilot trial

Results:

A quit smoking program spanning 42 days of 2 to 3-minute conversations over two phases: (1) pre-quitting phase (14 days) content covering topics ranging from motivations to quit, setting a quit date, choosing FDA-approved cessation medications; and (2) post-quit phase (28 days) content on coping with triggers and recovering from lapses/relapses. In a pilot randomized trial with 96% three-month outcome data retention, QuitBot demonstrated high user engagement and promising cessation rates compared to the National Cancer Institute’s SmokefreeTXT (SFT) text messaging program—particularly among those who viewed all 42 days of program content: 30-day complete-case, point prevalence abstinence (PPA) rates at three-month follow-up were 63% (39/62) for QuitBot vs. 38% (45/117) for SFT (OR = 2.58; 95% CI: 1.34, 4.99; P =.005). However, Facebook Messenger (FM) intermittently blocked participants’ access to QuitBot so we transitioned from FM to a standalone smartphone app as the communication channel. Participants’ frustration with QuitBot’s inability to answer their open-ended questions lead to us develop a core conversational feature enabling users to ask open-ended questions about quitting cigarette smoking and for the QuitBot to respond with accurate and professional answers. To support this functionality, we developed a library of 11,000 QnA pairs on topics associated with quitting cigarette smoking. Model testing results showed that Microsoft's Azure-based QnA maker effectively handled questions that matched our library of 11,000 QnA pairs. A fine-tuned, contextualized GPT3.5 responds to questions that are not within our library of QnA pairs.

Conclusions:

The development process yielded the first documented case of a comprehensive quit smoking program delivered as a conversational chatbot. Iterative testing led to significant enhancements, including improvements to the delivery channel. A pivotal addition was the inclusion of a core LLM-supported conversational feature allowing users to ask open-ended questions. Clinical Trial: ClinicalTrials.gov Identifier, NCT03585231