Accepted for/Published in: JMIR mHealth and uHealth
Date Submitted: Oct 31, 2019
Date Accepted: Jun 14, 2020
Date Submitted to PubMed: Jun 16, 2020
Can Natural Language Processing and Sentiment Analysis Augment Traditional User Centered Design?
ABSTRACT
Background:
Sarcopenia, defined as the age-associated loss of muscle mass and strength, can be effectively mitigated through resistance-based physical activity. With compliance at ~40% for home-based exercise prescriptions, implementing a remote-sensing system would help patients and clinicians understand treatment progress and increase compliance. Inclusion of end users in the development of mobile applications for remote sensing systems can ensure that they are both user-friendly and facilitate compliance.
Objective:
The objective of our study was to develop a mobile application for a novel device through a user-centered design process with both older adults and clinicians.
Methods:
Through a user-centered design process, we conducted semi-structured interviews during the development of a geriatric-friendly Bluetooth-connected Resistance Exercise Band App. We interviewed patients and clinicians at weeks 0, 5, and 10 of the development of the app. Each semi-structured interview consisted of heuristic evaluations, cognitive walkthroughs, and observations of a geriatric-friendly Bluetooth-connected resistance exercise-band application. We used the Bing sentiment library for sentiment analysis on each transcript, then applied natural-language processing (NLP) based Latent Dirichlet Allocation (LDA) topic modeling to identify differences and similarities in patient and clinician participant interviews. To assess utility, we used post-interview quantitative assessments – System Usability Scale (SUS) and Usefulness, Satisfaction & Ease of use (USE). Finally, we used multivariate linear models – adjusting for age, sex, subject group (clinician vs. patient), and development – to explore the association between sentiment analysis and SUS and USE outcomes.
Results:
The mean age of the 22 participants was 68 years old, 17 (77%) of which were female. The overall mean SUS score was 66.4 (SD: 13.6) and mean USE score was 41.3 (SD: 15.2). We found that both patients and clinicians provided valuable insights into the needs of older adults when designing and building an application. The mean positive-negative sentiment for patients was 16 and 6 for clinicians. We found a positive association with positive sentiment in an interview and SUS score ( = 1.38 [95%CI = 0.37-2.39], P-value = .01). There was no significant association between sentiment and USE. Latent Dirichlet Allocation (LDA) analysis found no overlap between patients and clinicians in the eight topics identified.
Conclusions:
Involving patients and clinicians allowed us to design and build an application that is user-friendly for older adults while supporting compliance. This is the first analysis that used NLP and usability questionnaires in the quantification of the user-centered design of technology for older adults.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.