JMIR Publications

ABSTRACT

Background:

Profound scientific evaluation of novel Digital Health Technologies (DHT) is key to enhance successful development and implementation. In such, we developed the eHealth evaluation cycle in previous research. The eHealth evaluation cycle contains five consecutive study phases: conceptual, development, feasibility, effectiveness, and implementation.

Objective:

The objective of this study was to create a visual overview of DHTs’ evaluation approaches used in original research and to determine to what extent the study phases of the eHealth evaluation cycle have been utilised.

Methods:

We conducted a systematic literature search in PubMed including the MeSH term ‘telemedicine’ in combination with a wide variety of evaluation approaches. Original studies from 2019 (pre-COVID-19 cohort) were included. Data on the following variables were extracted and systematically analysed: journal, country, publication date, medical specialty, primary user, functional classification, evaluation study phases, and evaluation approach.

Results:

824 studies were included after 1583 titles and abstracts were screened. The majority of the evaluation studies focused on the effectiveness (impact) (36.9%) study phase, whereas uptake (implementation) (8.5%) was the least. The RCT (19.0%) was the most commonly used DHT evaluation method. Within the effectiveness (impact) study phase, the RCT was used in half of the studies. In the conceptual and planning phases, survey research (34.6%) and interview studies (34.6%) were most frequently used. The United States published the largest amount of DHT evaluation studies (36.9%). Psychiatry / mental health (10.6%) and cardiology (8.9%) published the majority of the studies within the field.

Conclusions:

We found that the study phases of the eHealth evaluation cycle are in equally studied. Also, the majority of the studies in the effectiveness study phase still uses a RCT design. However, in order to successful develop and implement novel DHTs, stimulating equal evaluation of the sequential study phases of DHTs and selecting the right evaluation approach that fits to the iterative nature of technology, might be of utmost importance.