JMIR Publications

ABSTRACT

Background:

In response to the COVID-19 pandemic, the COVID Radar app was developed as a population-based surveillance instrument to identify at-risk populations and regions. The app boasts over 8.2 million completed questionnaires with over 180,000 unique users nationwide. Although the COVID Radar app is a valid tool for population-level surveillance, high user engagement of citizens is critical to the success of the COVID Radar to maintain internal and external validity.

Objective:

To identify optimization targets of the COVID Radar app in order to improve its acceptability, adherence, and inclusiveness.

Methods:

The main component of the COVID Radar app is a self-reporting questionnaire assessing COVID-19 symptoms and social distancing behaviours. Three qualitative sub-studies were conducted. First, three semi-structured focus group interviews with end-users (N=14) of the app were held, gathering in-depth information on user experiences. The output was transcribed and thematically coded using the Framework method. Second, a similar qualitative thematic analysis was conducted of 1,080 end-user emails sent to the project team. Third, usability testing was conducted in one-on-one sessions with four individuals with low literacy levels.

Results:

All three sub-studies identified optimization targets in terms of design and content. The results of sub-study 1 showed that participants generally evaluated the app positively. They reported the app to be user friendly and they reported being satisfied with its design and functionalities. Participants’ main motivation to use the app was contributing to science by helping predict future COVID-19 flare-ups. Participants suggested adding motivational tools, such as gamification elements and personalization options within the app, to stimulate user engagement. A larger national publicity campaign for the app was considered potentially helpful in increasing the user population. In-app updates informing users about the project and its outputs motivated users to keep using the app. Feedback on the self-report questionnaire, stemming from sub-study 1 and 2, mostly concerned the content and phrasing of questions. Furthermore, the section of the app allowing users to compare their symptoms and behaviours to those of their peers was found to be suboptimal due to difficulties in interpretation of the presented figures in the app. Finally, the output of sub-study 3 resulted in recommendations primarily related to simplification of the text to render it more accessible and comprehensible for individuals with low literacy levels.

Conclusions:

Convenience of app-use, enabling personal adjustments of the app-experience, and taking into account motivational factors for continued app use (i.e. altruism and collectivism) were found crucial to procuring and maintaining a population of active users of the COVID Radar. Further, there seems a need to increase the accessibility of public health tools like the COVID Radar for individuals with low literacy levels. These results can be used to improve the current COVID Radar app and ultimately improve the representativeness of its user population and user engagement, to ultimately increase the internal and external validity of the app.