JMIR Publications

ABSTRACT

Background:

Mobile data collection systems are often difficult to use, for non-technical or novice users. This can be attributed often to the fact that, developers of such tools do not adequately involve end-users in the design and development of product features and functions. This has led to products which seem obvious to the developers in their functions but not to the general users, thus creating a big usability gap.

Objective:

The main objective of this study was to assess the guidelines for form design using high fidelity prototypes developed based on end user preferences. We also sought to investigate the association between the results from the System Usability Scale (SUS) and those from the Study Tailored Evaluation Questionnaire (STEQ) after the evaluation. In addition we sought to recommend some practical guidelines for the implementation of the group testing approach particularly in low resource settings during mobile form design.

Methods:

We developed an online high fidelity prototype based on design guidelines for data collection forms between January and February 2018, using Axure RP8, and the prototype which did not have any backend functionality. Thirty four (n=34) research assistants who were all involved in data collection on a maternal and child health project in Northern Uganda evaluated the high fidelity prototype in March 2018. The group was given a set of tasks to complete during one common session. Upon completion, a study tailored evaluation questionnaire (STEQ) comprising of 13 affirmative statements, coupled with the commonly used and validated System Usability Scale (SUS) were administered. The purpose was to evaluate the usability and user experience after interaction with the prototype. The STEQ evaluation was summarized using frequencies in an excel sheet where the evaluation statement with majority agreeing to was taken as the most preferred option. The SUS scores were calculated based on whether the statement was positive (user selection minus 1) or negative (5 minus user selection). These were summed up and the score contributions multiplied by 2.5 to give the overall form usability from each participant.

Results:

More than 24 (70%) of the research assistants appreciated the form progress indication, found the form navigation easy and were satisfied with the error messages. The same number found it easy to recover in case of a mistake and overall found the form easy to use and would like to use it more frequently. The results gave an SUS average score of 70.4 and SD of 11.7, which is above the recommended average SUS score of 68, meaning that the usability of the prototype was above average. The scores from the Study Tailored Evaluation Questionnaire (STEQ) on the other hand indicated a 70% level of agreement with the affirmative evaluation statements. The results from the 2 instruments indicated a fair level of user satisfaction with the forms and a strong positive association as shown by the Pearson correlation value of 0.623, p<0.005.

Conclusions:

A high fidelity prototype was used to give the users experience with a product they would likely use in their work. The design features in the high-fidelity prototype were evaluated to inform development of mobile data collection forms. Two evaluation instruments were adopted during user testing for purposes of comparing and complementing findings. The group testing was due to scarcity of resources like costs and time involved especially in low income countries. . If embraced, the approach used in this study, could help assess user needs of the diverse user groups. With proper preparation and the right infrastructure at an affordable cost, this type of usability testing could lead to the development of highly usable forms. The study thus makes recommendations on the practical guidelines for the implementation of the group testing approach particularly in low resource settings during mobile form design.