Accepted for/Published in: Journal of Medical Internet Research
Date Submitted: Oct 7, 2022
Date Accepted: May 5, 2023
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Comparison of published protocols and publications in umbrella reviews showed inconsistent reporting and low transparency
ABSTRACT
Background:
Umbrella reviews (URs) are becoming increasingly influential in biomedical literature. Inconsistencies between protocols and publications, which may not only lead to outcome reporting bias, but may also produce false positive results, were found in non-UR studies. What's more, changes between protocols and publications were disclosed with an explanation for the changes, which is a significant dimension to consider in assessing the transparency of evidence.
Objective:
This study aimed to investigate the inconsistency and the transparency of protocols and publications in URs.
Methods:
We searched electronic databases from their inception to January 1, 2022. We investigated inconsistencies between protocols and publications and transparencies in the field of search strategy, inclusion criteria, methods of screening and data extraction, methodological quality assessment and statistical analysis.
Results:
Thirty-one protocols and 35 publications were included. In the field of the search strategy, 39 inconsistencies were found in 26 URs (73.3%), 16 of them were indicated and explained. There were 81 inconsistencies regarding the field of the inclusion criteria in 31 URs (88.6%), and 26 of inconsistencies were indicated and explained. We found 13 URs (40.6%) occurred inconsistencies in the field of the screening methods, 14 URs (46.7%) in the data extraction methods and 11 URs (34.4%) in the methodological quality assessment methods, nine of them were illustrated. Six publications (17.1%) of URs deviated from their protocols in the tools for quality assessment, half of them indicated and explained the deviations. Compared with the protocols, 31 publications (88.6%) of URs showed 61 inconsistencies in the field of the statistical analysis, and 16 of them were indicated and explained.
Conclusions:
There was a high prevalence of inconsistencies between protocols and publications of URs, and more than half of URs didn’t indicate and explain inconsistencies, so how to promote the transparency of URs will be a major part of future work. Clinical Trial: No
Citation
