Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Accepted for/Published in: JMIR Human Factors

Date Submitted: May 26, 2020
Date Accepted: Mar 2, 2021

The final, peer-reviewed published version of this preprint can be found here:

Applying Website Rankings to Digital Health Centers in the United States to Assess Public Engagement: Website Usability Study

Calvano JD, Fundingsland EL Jr, Lai D, Silacci S, Raja AS, He S

Applying Website Rankings to Digital Health Centers in the United States to Assess Public Engagement: Website Usability Study

JMIR Hum Factors 2021;8(1):e20721

DOI: 10.2196/20721

PMID: 33779564

PMCID: 8088849

Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.

Website Rankings for Digital Health Centers in the USA: Applying Usability Testing for Public Engagement.

  • Joshua David Calvano; 
  • Edwin Lauritz Fundingsland Jr; 
  • Deborah Lai; 
  • Sara Silacci; 
  • Ali S Raja; 
  • Shuhan He

ABSTRACT

Background:

As the public increasingly looks to the internet for resources and information regarding health and medicine, it is important that healthcare organizations provide adequate web resources. Usability refers to the ease of user experience on a website. In this study, we focus on usability analyses for digital health center websites. Our objectives were to develop adapt pre-existing usability scoring systems for United States digital health center websites, then apply this system to a sample for the purposes of testing this system and deriving insights from the results on potential areas of improvement for this sample.

Objective:

The primary aims of the study were to: 1) Adapt a pre-existing usability scoring methodology to digital health centers; 2) apply and test this adapted usability scoring methodology on a sample set of digital health center website and; 3) derive recommendations from these results on potential areas of improvements for our sample of digital health center websites.

Methods:

All website usability testing took place from 1 March 2020 to 15 March 2020. We adapted a methodology and scoring system developed from previous literature and applied it to digital health center websites. Our sample included 67 digital health centers affiliated with U.S. universities or hospitals systems. Usability was split into four broad categories: Accessibility (ability of those with low levels of computer literacy to access and navigate hospital’s Web presence), Marketing (ability to be found through search engines, examining the relevance of descriptions to the links provided), Content Quality (grammar, frequency of info updates, material relevancy, and readability), and Technology (download speed, quality of the programming code, and website infrastructure). Using these usability tools, we scored each website in each of the four categories. The composite of key factors in each of the four categories contributed to an overall “General Usability” score for each website. An overall score was then calculated by applying a weighted percentage across all factors and used for the final ranking system.

Results:

The category with the overall highest average score was Content Quality, with a 6.3. Content Quality also had the highest standard deviation at +/- 2.18, with a standard error of 0.27. The lowest performing category was Technology, with an average of 0.9. Technology also had the smallest standard deviation at +/- 0.07, with a standard error of 0.01.

Conclusions:

Data suggests that Content Quality on average was the highest scored variable amongst digital health center websites. Because content is crucial to the knowledge of digital health it is justified that digital health centers invest more in creating quality content. The overall lowest scored variable was Technology. Potential reasons for this finding are related to designated funding for servers, a lack of regulatory framework around social media presence and liability, or infrequent website audits. An easy way to improve this variable is by increasing speed. Accessibility is another area in which these organizations have potential for improvement. In the effort to improve the dissemination of reliable information, this is arguably the most important variable for digital health centers to address. Our recommendation is that these organizations perform periodic audits of their web presence utilizing these tools.


 Citation

Please cite as:

Calvano JD, Fundingsland EL Jr, Lai D, Silacci S, Raja AS, He S

Applying Website Rankings to Digital Health Centers in the United States to Assess Public Engagement: Website Usability Study

JMIR Hum Factors 2021;8(1):e20721

DOI: 10.2196/20721

PMID: 33779564

PMCID: 8088849

Download PDF


Request queued. Please wait while the file is being generated. It may take some time.

© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.