Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.
Who will be affected?
Readers: No access to all 28 journals. We recommend accessing our articles via PubMed Central
Authors: No access to the submission form or your user account.
Reviewers: No access to your user account. Please download manuscripts you are reviewing for offline reading before Wednesday, July 01, 2020 at 7:00 PM.
Editors: No access to your user account to assign reviewers or make decisions.
Copyeditors: No access to user account. Please download manuscripts you are copyediting before Wednesday, July 01, 2020 at 7:00 PM.
Leung YW, Park B, Heo R, Adikari A, Chackochan S, Wong J, Alie E, Gancarz M, Kacala M, Hirst G, de Silva D, French L, Bender J, Mishna F, Gratzer D, Alahakoon D, Esplen MJ
Providing Care Beyond Therapy Sessions With a Natural Language Processing–Based Recommender System That Identifies Cancer Patients Who Experience Psychosocial Challenges and Provides Self-care Support: Pilot Study
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Providing care beyond the therapy session — a natural language processing–based recommender system that identifies cancer patients who experience psychosocial challenges and provides self-care support
Yvonne W Leung;
Bomi Park;
Rachel Heo;
Achini Adikari;
Suja Chackochan;
Jiahui Wong;
Elyse Alie;
Matthew Gancarz;
Martyna Kacala;
Greame Hirst;
Daswin de Silva;
Leon French;
Jackie Bender;
Faye Mishna;
David Gratzer;
Damminda Alahakoon;
Mary Jane Esplen
ABSTRACT
Background:
The negative psychosocial impacts of cancer diagnoses and treatments are well documented. Virtual care has become an essential mode of care delivery during the COVID-19 pandemic and online support groups (OSGs) are shown to improve accessibility to psychosocial and supportive care. The de Souza Institute offers CancerChatCanada, a therapist-led OSG service where sessions are monitored by an artificial intelligence-based co-facilitator (AICF). AICF is equipped with a recommender system that uses natural language processing to tailor online resources to patients according to their psychosocial needs.
Objective:
To outline the development protocol and to evaluate AICF on its precision and recall in recommending resources to cancer OSG members.
Methods:
Human input informed the design and evaluation on its ability to 1) appropriately identify key words indicating a psychosocial concern and 2) recommend the most appropriate online resource to the OSG member expressing each concern. Three rounds of human evaluation and algorithm improvement were performed iteratively.
Results:
We evaluated 7,190 outputs and achieved .797 precision, .981 recall, and an F1 score of .880 by the third round of evaluation. Resources were recommended to 48 patients and 25 (52.1%) accessed at least one resource. Of those who accessed the resources, 75.4% found them useful.
Conclusions:
The preliminary findings suggest that AICF can help provide tailored support for cancer OSG members with high precision, recall, and satisfaction. AICF has undergone rigorous human evaluation and the results provide much-needed evidence, while outlining potential strengths and weaknesses for future applications in supportive care.
Citation
Please cite as:
Leung YW, Park B, Heo R, Adikari A, Chackochan S, Wong J, Alie E, Gancarz M, Kacala M, Hirst G, de Silva D, French L, Bender J, Mishna F, Gratzer D, Alahakoon D, Esplen MJ
Providing Care Beyond Therapy Sessions With a Natural Language Processing–Based Recommender System That Identifies Cancer Patients Who Experience Psychosocial Challenges and Provides Self-care Support: Pilot Study