JMIR Publications

ABSTRACT

Background:

Concern has arisen over the reliability and inaccuracy of the health-related information provided by online newspaper articles. Many criteria and items have been proposed and published regarding the quality assessment of online information, but there is no standard quality assessment tool available for online newspapers.

Objective:

The purpose of this study was to develop a quality assessment tool for health-related information in online newspaper articles.

Methods:

We reviewed the previous health information quality assessments tools and relative studies, and accordingly developed and customized the new criteria. The inter-rater agreement for the new assessment tool was assessed for three newspaper articles using Fleiss’ kappa (K), Krippendorff’s alpha, and Gwet’s agreement coefficient (AC).

Results:

Overall, the quality assessment tool for health information in newspaper articles comprised five domains: reliability, usefulness, understandability, sufficiency, and transparency, and had a total of 10 items. The inter-rater agreement among the reviewers for the article on colorectal cancer was in the moderate to substantial range (Fleiss' K = 0.48 (s.e.= 0.11); Kripendorff's Alpha = 0.50 (0.11); Gwet's AC = 0.74 (0.13)), while that for the article introducing obesity genetic testing was in the substantial range with values of 0.63 (0.28), 0.64 (0.28), and 0.86 (0.10) for the three measures, respectively. There was relatively low agreement for the third article on hypertension diagnostic criteria at 0.20 (0.10), 0.21 (0.10), and 0.75 (0.13), respectively.

Conclusions:

We developed a new assessment tool to evaluate the quality of health information in online newspaper articles, as well as in helping consumers discern accurate sources of health information. This tool can help increase the accuracy and quality of online health information in Korea.