JMIR Publications

ABSTRACT

Background:

Mobile exposure notification applications (ENAs) can play a significant role in future pandemics. Identifying lessons learned and areas for improvement from such applications —particularly within specific social and cultural contexts—is therefore crucial.

Objective:

We evaluated the Finnish ENA “Koronavilkku”, according to chosen key performance indicators to assess the ENA’s use and performance during the COVID-19 pandemic in Finland, and to identify strengths and improvement areas.

Methods:

Using available data on Finland’s ENA we defined metrics for four key performance indicators: ENA usage extent, enablers and barriers to usage, the ENA’s influence on user behavior, and overall acceptability. We performed an evaluation study combining system data and data from a survey of 4 061 respondents conducted in April 2022. A sentiment analysis was performed on open-ended responses.

Results:

Koronavilkku ENA peak coverage reached approximately 53% of smartphone users shortly after launch. Usage declined over time, particularly among individuals diagnosed with COVID-19. User engagement remained high - among ENA users that received a token, the proportion that entered their token into the ENA was over 50% during the whole study period. Males, individuals over 34, and those living alone were less likely to use the ENA, while those with higher education, and households with children were more likely to be ENA users. Almost 70% of SARS-CoV-2-positive users reported not receiving a token from health authorities. Users adhered at least part-time to pandemic guidance more often than non-users with prevalence ratios ranging from 1.24 (95%CI: 1.19 – 1.29) to 1.65 (95%CI: 1.45 – 1.91). Exposure notifications were particularly effective in reinforcing adherence among partially compliant users but had limited impact on non-compliant individuals. Overall acceptability was relatively high, with 68% of respondents using it at least once. However, 49% of those users eventually discontinued use. Our sentiment analysis of open-ended feedback suggests that users expressed more positive attitudes toward the ENA compared to non-users. However, a significant portion of the feedback reflected neutral or negative sentiments.

Conclusions:

Based on observed ENA usage trends we recommend maintaining long-term public interest and engagement in ENAs during prolonged emergencies. Tailored communication should be developed encouraging uptake among demographic groups with lower adoption rates. Initial studies during app rollout could inform these strategies. A major barrier to the ENA’s effectiveness was token issuance, therefore earlier and more systematic automation of token distribution in Finland would likely have enhanced effectiveness. Future ENAs should define evaluation criteria during development, with periodic assessments measuring effectiveness and informing improvements. A deeper analysis of open-ended feedback provided for the Koronavilkku ENA using advanced language models could provide additional insights into user perceptions and concerns. By addressing these areas, future digital contact tracing tools can be more effective, widely accepted, and better integrated into pandemic response efforts.