JMIR Publications

ABSTRACT

Background:

The synthesis of evidence in healthcare is essential for informed decision-making and policy development. This study aims to validate The Umbrella Collaboration® (TU®), an innovative, semi-automatic tertiary evidence synthesis methodology, by comparing it with Traditional Umbrella Reviews (TUR), which are currently the gold standard.

Objective:

The primary objective of the present study is to assess whether a software-driven AI-assisted system of evidence synthesis, TU®, can match the effectiveness of traditional methods of tertiary synthesis, providing a potentially more timely, efficient, and comprehensive approach while remaining open to findings that could demonstrate superior performance. To support the primary objective of evaluating the effectiveness of TU® compared to traditional methodologies, this study also aims to assess the accessibility and comprehensibility of TU®'s outputs as a secondary objective

Methods:

This protocol outlines a comparative study divided into two main parts. The first part involves a quantitative comparison of results obtained using TU® and TURs in geriatrics. We will evaluate the identification, size effect, direction, statistical significance, and certainty of outcomes, as well as the time and resources required for each methodology. Data for TURs will be sourced from Medline (via PubMed), while TU® will use AI-assisted informatics to replicate the research questions of the selected URTs. The second part of the study assesses the ease of use and comprehension of TU® through an online survey directed at health professionals, utilizing interactive features and detailed data access.

Results:

Expected results include the assessment of concordance in identifying outcomes, the size effect, direction and significance of these outcomes, and the certainty of evidence. Additionally, we will measure the operational efficiency of each methodology by evaluating the time taken to complete projects. User perceptions of the ease of use and comprehension of TU® will be gathered through detailed surveys.

Conclusions:

If TU® proves as effective as TURs but more time-efficient, accessible and easily updatable, it could significantly enhance the process of evidence synthesis, facilitating informed decision-making and improving healthcare. This study represents a step towards integrating innovative technologies into routine evidence synthesis practice, potentially transforming health research.