JMIR Publications

ABSTRACT

Background:

To motivate people to adopt medical chatbots, the establishment of a specialized medical knowledge database which fits their personal interests is of great importance in developing a medical chatbot for perinatal care, particularly with the help of health professionals.

Objective:

The objectives of this study are to (1) develop a user-friendly Q&A knowledge database-based chatbot (Dr. Joy) for perinatal women’s and their partners’ obstetric and mental health care by applying a text-mining technique and (2) implement a contextual usability testing (UT) by using quantitative and qualitative methods, thus determining whether this medical chatbot built on mobile instant messenger (KakaoTalk) can provide its male and female users with good user experience.

Methods:

Two men (mean 39.00, SD 1.41) and 13 women (mean 34.31, SD 3.95) in pregnancy preparation or different pregnancy stages were enrolled. All participants completed the 7-day long UT during which they were given the tasks of (1) asking Dr. Joy at least 3 questions per day at any time and place and giving the chatbot either positive or negative feedback with emoji, (2) using at least one feature of the obstetrics chatbot, and then (3) sending a facilitator all screenshots for the history of one-day use via KakaoTalk before midnight. One day after the UT completion, all were asked to fill out a questionnaire on the evaluation of usability, perceived benefits and risks, and intention to seek and share health information on the chatbot, as well as demographic characteristics.

Results:

Despite the relatively higher score of ease of learning (EOL), the results of the Spearman correlation indicated that EOL was not significantly associated with usefulness (ρ=0.26, P=.36), ease of use (ρ=0.19, P=.51), satisfaction (ρ=0.21, P=.46), and total usability scores (ρ=0.32, P=.24). Unlike EOL, all the 3 sub-factors and the total usability had significant positive associations each other (all ρ>0.80, P<.001). Furthermore, perceived risks exhibited no significant negative associations with perceived benefits (PB; ρ=-0.29, P=.30) and intention to seek (SEE; ρ=-0.28, P=.32) and share (SHA; ρ=-0.24, P=.40) health information on the chatbot using via KakaoTalk, whereas PB exhibited significant positive associations with both SEE and SHA. PB was more strongly associated with SEE (ρ=0.94, P<.001) than with SHA (ρ=0.70, P=.004).

Conclusions:

This study provides the potential for the uptake of this newly developed Q&A knowledge database-based KakaoTalk chatbot for obstetric and mental health care. As Dr. Joy had quality contents with both utilitarian and hedonic value, its male and female users could be encouraged to use medical chatbots in a convenient, easy-to-use, and enjoyable manner. To boost their continued usage intention for Dr. Joy, its Q&A sets need to be periodically updated to satisfy user intent by monitoring both male and female user utterance.