Accepted for/Published in: JMIR Human Factors
Date Submitted: Oct 31, 2022
Open Peer Review Period: Oct 31, 2022 - Dec 26, 2022
Date Accepted: Jan 29, 2023
(closed for review but you can still tweet)
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Usability of an Online Reference Database with Natural Language Searching Capabilities: Evaluation of DynaMed and Micromedex with Watson
ABSTRACT
Background:
Evidence-based point-of-care information (POCI) tools can facilitate patient safety and care by helping clinicians answer disease state and drug-information questions with less time and effort. However, these tools may also be visually challenging to navigate or lack the comprehensiveness needed to sufficiently address a medical issue.
Objective:
This study aimed to collect clinicians’ feedback and directly observe their use of the combined POCI tool DynaMed and Micromedex with Watson, now known as DynaMedexTM. EBSCO partnered with IBM Watson Health, now known as Merative, to develop the combined tool as a resource for clinicians. We aimed to identify areas for refinement based on participant feedback and examine participant perceptions to inform further development.
Methods:
Participants (n=43) within varying clinical roles and specialties were recruited from Brigham and Women’s Hospital and Massachusetts General Hospital in Boston, MA between August and December 2021 to take part in usability sessions aimed at evaluating the efficiency, effectiveness, and satisfaction with the DynaMed and Micromedex with Watson tool. Usability testing methods, including think aloud and observations of user behavior, were used to identify challenges regarding the combined tool. Data collection included measurements of time on task, task ease, satisfaction with the answer, post-test feedback on likes, dislikes, perceived reliability of the tool, and interest in recommending the tool to a colleague.
Results:
On a seven-point Likert scale, pharmacists rated ease (5.98) and satisfaction (6.31) with the combined POCI tool higher than physicians, nurse practitioners, and physician assistants (5.57; 5.82); pharmacists spent longer (02m:26s) on average finding an answer to their question in comparison to the other roles (01m:40s).
Conclusions:
Overall, the tool performed well, but this usability evaluation identified multiple opportunities for improvement which would help inexperienced users.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.