JMIR Publications

ABSTRACT

Background:

The eight Dutch University Hospitals (UHs) are at the forefront of contributing meaningfully to a sustainable and future-proof health care system. To stimulate nationwide collaboration and knowledge sharing, the Dutch UHs joined forces from 2016 till 2019 within the Citrien fund (CF) - program eHealth. In this program various evidence based eHealth projects, with various subjects and themes, were selected, supported and evaluated. To determine the accomplishment of the deliverables of CF - program eHealth and to contribute to the insights in and practice of formative evaluation of eHealth in general, a comprehensive evaluation was deemed essential.

Objective:

The first objective was to evaluate if - and to what extent - the 11 deliverables of CF - program eHealth were accomplished and to describe the main outcomes of the 29 eHealth projects. The second objective was to evaluate eHealth project progress indicators in relation to the 29 eHealth projects.

Methods:

Two study designs were used to evaluate the two objectives. To analyze the first objective, an evaluation study utilizing an adapted version of the Commonwealth Scientific and Industrial Research Organization (CSIRO) framework was carried out. A mixed-methods study consisting of a two-parted questionnaire and semi-structured interview was carried out to analyze the second objective.

Results:

The 11 deliverables of the CF - program eHealth were successfully achieved. The program yielded, besides 22 tangible eHealth solutions, significant knowledge about the development and use of eHealth solutions. In such, we have learned that: the patient is enthusiastic when it comes to accessing and downloading their own medical data online, but physicians are more cautious; it was not always possible to implement the Dutch set of standards for interoperability due to lack of IT capacities; and fair attention must be paid to patients with low eHealth skills, education in such is important. The eHealth project progress indicators ‘planning’, ‘IT-services’ and ‘legal’ played an important role in successful progress of the 29 projects. The in-depth interviews illustrated that a novel eHealth solution should fulfil a need, partners that already have the knowledge and means to accelerate development should be involved, clear communication towards IT-developers and other stakeholders is key, and a dedicated project leader with sufficient time is of utmost importance for the success of a project.

Conclusions:

The eight Dutch University Hospitals were able to successfully collaborate and stimulate through a bottom-up approach nationwide eHealth development and knowledge sharing. Twenty-two tangible eHealth solutions were developed and significant knowledge was established. The aspects ‘planning’, ‘IT-services’ and ‘legal’ played an important role in successful progress of the projects and should therefore be closely monitored while developing novel eHealth solutions.