JMIR Publications

ABSTRACT

Background:

Heuristic evaluations, while commonly used, may inadequately capture the severity of identified usability issues. Incorporating diverse expertise, especially in healthcare contexts, can help assess and address potential negative impacts on patient safety.

Objective:

To assess a multi-stage method for integrating diverse expertise (i.e., clinical, patient/care partner, and IT) with Human Factors Engineering (HFE) expertise in the usability evaluation of an emergency department (ED) discharge summary document.

Methods:

We conducted a three-staged heuristic evaluation of an ED discharge summary using heuristics developed for use in evaluating patient-facing documentation. In Stage 1, HFE experts reviewed the discharge summary to identify usability issues. In Stage 2, six experts of varying expertise (i.e., emergency medicine physicians, ED nurse, geriatrician, transitional care nurse, and patient/care partner) rated each previously identified usability issue on its potential impact on patient comprehension and patient safety. Finally, in Stage 3, an IT expert reviewed each usability issue to identify the likelihood of successfully addressing the issue.

Results:

In Stage 1, we identified 60 usability issues that violated at total of 108 heuristics. In Stage 2, our experts identified 18 additional usability issues that violated 27 heuristics. Impact ratings ranged from all experts rating the issue as “No impact” to 5 out of 6 experts rating the issue as having a “Large negative impact.” On average, the patient/care partner representative rated usability issues as being more significant more of the time. In Stage 3, 31 usability issues were rated by an IT professional as “Impossible to address,” 21 as “Maybe,” and 24 as “Can be addressed.”

Conclusions:

Integrating diverse expertise when evaluating usability is important when patient safety is at stake. The non-HFE experts, included in Stage 2 of our evaluation, identified 23% (18/78) of all the usability issues and, depending on their expertise, rated those issues as having differing impacts on patient comprehension and safety. Combining those findings with ratings from an IT expert, usability issues can be strategically addressed through redesign. Thus, a three-staged heuristic evaluation method offers a framework for integrating context-sensitive expertise efficiently, while providing practical insights to guide human-centered design.