JMIR Publications

ABSTRACT

Background:

The assessment of usability is a complex process that involves several steps and procedures. It is important to standardize the evaluation and reporting of usability procedures across studies to guide researchers, facilitate comparisons across studies and promote high-quality usability studies A first step to standardizing is to have an overview of how usability study procedures are reported across the literature.

Objective:

To synthesize the procedures used or reported for the different steps of the process of conducting a user-centered usability assessment of digital solutions relevant for older adults. A secondary aim was to identify any principles guiding this assessment.

Methods:

This is a scoping review of reviews. A five-stage scoping review methodology was used to identify and describe relevant literature published between 2009 and 2020 as follows: (1) identify the research question, (2) identify relevant studies, (3) select studies for review, (4) chart data from selected literature, and (5) summarize and report results. The research was conducted on five electronic databases: PubMed, ACM Digital Library, IEEE, Scopus, and Web of Science. The articles that met the inclusion criteria were identified, and data extracted for further analysis regarding study evaluator, study participants, methods and techniques, tasks, and test environment.

Results:

A total of 3958 articles were identified. After a detailed screening, 20 reviews matched the eligibility criteria. The characteristics of study evaluators and participants and task procedures were only briefly and differently reported. The methods and techniques used for the assessment of usability are the topics most commonly and comprehensively reported in the reviews, while the test environment was seldom and poorly characterized.

Conclusions:

A lack of a detailed description of several steps of the process of assessing the usability and no evidence on good practices of performing it suggests the need for a consensus framework on the assessment of user-centered usability evaluation. Such a consensus would inform researchers and allow standardization of procedures. Findings also highlight the need to investigate whether different ways of assessing usability are more sensitive than others. Clinical Trial: Not applicable.