JMIR Publications

ABSTRACT

Background:

Web based tools developed to facilitate a Shared Decision Making (SDM) process may facilitate implementation of lung cancer screening (LCS), an evidence-based intervention to improve cancer outcomes. Veterans have specific risk factors and shared experiences that impact the benefit and potential harms of LCS, so may value a Veteran centric LCS Decision Support Tool (LCSDecTool).

Objective:

To conduct usability testing of a LCSDecTool designed for Veterans receiving care at a Veteran Affairs Medical Center (VAMC).

Methods:

Usability testing of the LCSDecTool was conducted in a Prototype version (Phase 1) and a High-Fidelity version (Phase 2). Eighteen Veterans and 8 clinicians participated in Phase 1. Forty-three (43) Veterans participated in Phase 2. Quantitative outcomes from the users included the System Usability Scale (SUS) and the End-User Computing Satisfaction (EUCS) in Phase 1 and the SUS, EUCS, and Patient Engagement (PE) scale in Phase 2. Qualitative data were obtained from observation of the user sessions and brief interviews. Results of Phase 1 informed modifications of the Prototype for the High-Fidelity version. Phase 2 usability testing took place in the context of a pilot Type 1 Hybrid Efficacy Implementation Trial.

Results:

In the Phase I Prototype usability testing, the SUS score (potential range: 0 to 100) was (mean, SD): 81.90 [9.80], corresponding to an excellent level of usability. The EUCS score (potential range: 1 to 5) was (mean, SD): 4.30 [0.71]. In the Phase 2 High Fidelity usability testing, the SUS score was (mean, SD): 65.76 [15.23] corresponding to a good level of usability. The EUCS score was (mean, SD): 3.91 [0.95]; and the patient engagement (PE) score (potential range 1[low] to 5 [high]) was (mean, SD): 4.62 [0.67]. Time to completion in minutes (median, IQR) was 13, 10-16. A thematic analysis of user statements documented during Phase 2 High Fidelity usability testing identified the following themes: 1) A low level of awareness and knowledge about LCS increased with use of the LCSDecTool, 2) Users sought more detailed descriptions about the LCS process, 3) The LCSDecTool was generally easy to use but specific navigation challenges remained, 4) Some users noted difficulty understanding medical terms used in the LCSDecTool, and 5) Use of the tool evokes Veterans’ struggles with prior attempts at smoking cessation

Conclusions:

The High-Fidelity version of the LCSDecTool demonstrated a good level of usability among Veterans when testing in the context of clinical care. Further modifications of the tool, including optimizing the navigation and simplifying language are needed prior to implementation of the LCSDecTool. Clinical Trial: ClinicalTrials.gov Identifier: NCT02899754