JMIR Publications

ABSTRACT

Background:

In the past years a mHealth app, called the Dutch Talking Touch Screen Questionnaire (DTTSQ) was developed in The Netherlands. The aim of development was to enable Dutch physical therapy patients to autonomously complete a health related questionnaire regardless of their level of education or digital literacy.

Objective:

The aim of this study was to evaluate the usability, defined as the effectiveness, efficiency and satisfaction, of the prototype of the DTTSQ for Dutch physical therapy patients with diverse levels of education and experience in using mobile technology.

Methods:

The qualitative Three-Step Test-Interview method, including both think-aloud and retrospective probing techniques, was carried out to get insight into the usability of the DTTSQ. Twenty-four physical therapy patients were included. The interview data were analyzed using a thematic content analysis approach aimed at analyzing the accuracy and completeness with which participants completed the questionnaire (effectiveness), the time it took the participants to complete the questionnaire (efficiency) and the extent to which the participants were satisfied with the ease of use of the questionnaire (satisfaction). The errors encountered by the participants in this study were given a severity rating which was used to provide a rough estimate of the need for additional usability efforts.

Results:

All participants within this study were very satisfied with the ease of use of the DTTSQ. Nine participants stated that the usability of the application exceeded their expectations. The group of four average/high experienced participants encountered only one error in total, while the eleven little experienced participants encountered an average of 2 errors per person and the nine non-experienced participants an average of 3 errors per person. Thirteen different kind of errors were found during this study. Three of these errors need to be addressed before the DTTSQ will be released because they have the potential to negatively influence future usage of the tool. Ten errors were less likely to influence future usage of the tool substantially.

Conclusions:

The usability of the DTTSQ needs to be improved before it can be released. No problems were found with satisfaction or efficiency during the usability-test. The effectivity needs to be improved by 1. increasing the visibility of activated answers, 2. making it easier to navigate through screens and 3. not forcing respondents to answer questions that do not apply to their situation. This study shows the importance of including less skilled participants in a usability study when striving for inclusive design. Further research is necessary to gain more insight into the needs, preferences, capacities, values, and goals in relation to mHealth technology of people with low literacy skills, low educational levels and no or little experience with using mobile technology.