JMIR Publications

ABSTRACT

Background:

The current public health crises we face, ranging from communicable disease pandemics such as Coronavirus disease (COVID-19) to endemic chronic diseases, require cohesive, collective, and deliberate societal efforts to address inherent decision-making gaps in our health systems. Digital health platforms that leverage big data ethically from citizens can transform health systems by enabling real-time data collection, communication, as well as precision prediction and health system rapid responses. However, the lack of standardized and evidence-based methods to develop and implement digital health platforms currently limits their application.

Objective:

This study aimed to evaluate the development of a novel rapid response COVID-19 digital health platform by engaging with the development team which includes computer programmers and data scientists, as well as the research team consisting of interdisciplinary researchers (i.e., key stakeholders).

Methods:

Using a developmental evaluation approach, this evaluation included two key components: 1) A qualitative survey assessing digital health platform objectives, modifications, and challenges administered to five key members of the software development team; and 2) A role-play pilot with key stakeholders to simulate real-world conditions, followed by a self-report survey, to evaluate the utility of the digital health platform for each of its objectives. Survey data were analyzed using an inductive thematic analysis approach. Post-pilot test survey data were aggregated and synthesized by participant role.

Results:

The final digital health platform met original objectives, and was expanded to accommodate evolving needs of potential users and COVID-19 regulations. Key challenges noted by the development team included navigating changing government policies and restrictions, and supporting the data sovereignty of platform users. Strong team cohesion, communication, and problem solving were all quintessential in the overall success of program development. Pilot test participants reported positive experiences interacting with the platform and found its features relatively easy to use. Users in the community member role felt that the platform accurately reflected their risk of contracting COVID-19, but reported challenges interacting with the interface, particularly when submitting citizen reports and food status photos. Those in the decision-maker role found the data visualizations intuitive in helping them to understand the information. Both participant groups highlighted the utility of a tutorial for future users as there were some questions regarding some of the features.

Conclusions:

Evaluation of the digital health platform development process informed our decisions to integrate the research team more cohesively with the development team, resulting in a data scientist being part of both teams going forward. Another key development process decision was to integrate more interdisciplinarity into the research process by providing health system training to computer programmers – a key factor in human-centered artificial intelligence development. The developmental evaluation changed development sprint processes, which paved the way to shorter sprints with quick internal evaluation of ongoing progress.