Accepted for/Published in: Journal of Medical Internet Research
Date Submitted: Jan 15, 2021
Date Accepted: Jul 13, 2021
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
The CAPOCI-tool: development of a tool to assess the trustworthiness of evidence-based point-of-care information for health care professionals’
ABSTRACT
Background:
User-friendly information at the point-of-care for health care professionals should be well structured, rapidly accessible, comprehensive and trustworthy. The reliability of information and the associated methodological process must be clear. There is no standard tool to evaluate the trustworthiness of such point-of-care information.
Objective:
We aim to develop and validate a new tool for assessment of trustworthiness of evidence-based point-of-care-resources, to enhance the quality of point-of-care-resources and to facilitate evidence-based practice (EBP)
Methods:
We designed the CAPOCI-tool (CAPOCI stands for Critical Appraisal of Point of Care Information, a new tool based on the different criteria important for assessment of trustworthiness of point-of-care (POC) information, reported in a previously published review. A group of health care professionals and methodologists (the authors of this paper) defined criteria for the CAPOCI-tool in an iterative process of discussion and pilot testing until consensus was reached. In a next step, all criteria were subject to content validation with a Delphi study. We invited an international panel of 10 experts to rate their agreement with the relevance and wording of the different criteria and to give feedback. Consensus was reached when 70% of the experts agreed. When no consensus was reached, we reformulated the criteria based on the experts’ comments for a next round of the Delphi study. This process was repeated until consensus was reached for each criterion. In a last step, the inter-rater reliability of the CAPOCI-tool was calculated with a two-tailed Kendall’s tau correlation coefficient to quantify the agreement between two users who piloted the CAPOCI-tool on five POC-resources. Two scoring systems were tested: a 3-point ordinal scale and a 7-point-Likert scale.
Results:
After validation, the CAPOCI-tool was designed with 11 criteria that focuses on methodological quality, and author related information. The criteria assess authorship, literature search, use of pre-appraised evidence, critical appraisal of evidence, expert opinions, peer review, timeliness and updating, conflict of interest, and commercial support. Inter-rater agreement showed substantial agreement between users for scoring with the 3-point ordinal scale (τ = .621, P<.01), as well as for scoring with the 7-point likers scale (τ = .677, P<.01).
Conclusions:
The CAPOCI-tool may support validation teams in the assessment of trustworthiness of point-of-care-resources. It may also be a guidance for producers of POC-resources.
Citation
