JMIR Publications

ABSTRACT

Background:

As technology continues to shape the landscape of health research, the utilization of online surveys for collecting sexual health information among adolescents and young adults (AYAs) has become increasingly prevalent. However, this shift towards digital platforms brings forth a new set of challenges, particularly the infiltration of automated bots that can compromise data integrity and the reliability of survey results.

Objective:

To outline the data verification process that was used in our study design employing survey programming and data cleaning protocols.

Methods:

A 26-item survey was developed and programmed with several data integrity functions including CAPTCHA, reCAPTCHA, RelevantID fraud and duplicate score, verification of IP addresses, and honeypot questions. Participants aged 15-24 years were recruited with social media advertisements over seven weeks and received a $15 incentive after survey completion. Data verification occurred through a two-part cleaning process, which removed responses that were incomplete, did not meet inclusion criteria, were flagged as spam by Qualtrics, or were from duplicate IP addresses. Final comparisons of reported age with date of birth and reported state with state inclusion criteria were performed. Participants who completed the study survey were linked to a second survey to receive their incentive. Responses without first and last names and full addresses were removed, as were those with duplicate IP addresses and/or the same longitude and latitude coordinates. Finally, IP addresses that were used to complete both surveys were compared, and consistent responses were eligible for an incentive.

Results:

Over seven weeks, online advertisements for an internet-based survey reached 1.4 million social media users. Of the 20,585 survey responses received, 4,589 (22.3%) were verified. Incentives were sent to 462 participants; of these, 14 were duplicates and three contained discrepancies, resulting in the inclusion of 445 respondents in the final sample.

Conclusions:

Confidential online surveys are an appealing method for reaching populations—particularly AYAs, who may be reluctant to disclose sensitive information to family, friends, and clinical providers. Despite the challenge of fraudulent responses, online surveys are a useful tool for researchers targeting hard-to-reach populations due to the difficulty in obtaining a representative sample. Researchers face the ongoing threat of bots and fraudulent responses in a technology-driven world, necessitating the adoption of evolving bot detection software and tailored protocols for data collection in unique contexts.