JMIR Publications

Background:

There is increasing use of online surveys to improve data quality and timeliness and reduce costs. While there have been numerous cross-sectional studies comparing responses to online or paper surveys, there is little research from a longitudinal perspective.

Objective:

In the context of the well-established Australian Longitudinal Study on Women’s Health, we examined the patterns of responses to online or paper surveys across the first two waves of the study in which both modes were offered. We compared the following: differences between women born between 1946 and 1951 and between 1973 and 1978; types of device used for online completion; sociodemographic, behavioral, and health characteristics of women who responded online or using mailed paper surveys; and associations between mode of completion in the first survey and participation and mode of completion in the second survey.

Methods:

Participants in this study, who had responded to regular mailed surveys since 1996, were offered a choice of completing surveys using paper questionnaires or Web-based electronic questionnaires starting in 2012. Two groups of women were involved: an older cohort born between 1946 and 1951 aged in their 60s and a younger cohort born between 1973 and 1978 aged in their 30s when the online surveys were first introduced. We compared women who responded online on both occasions, women who responded online at the first survey and used the paper version of the second survey, women who changed from paper to online, and those who used paper for both surveys.

Results:

Of the 9663 women in their 60s who responded to one or both surveys, more than 50% preferred paper surveys (5290/9663, 54.74%, on the first survey and 5373/8621, 62.32%, on the second survey). If they chose the online version, most used computers. In contrast, of the 8628 women in their 30s, 56.04% (4835/8628) chose the online version at the first survey. While most favored computers to phones or tablets, many did try these alternatives on the subsequent survey. Many women who completed the survey online the first time preferred the paper version on the subsequent survey. In fact, for women in their 60s, the number who went from online to paper (1151/3851, 29.89%) exceeded the number who went from paper to online (734/5290, 13.88%). The online option was more likely to be chosen by better educated and healthier women. In both cohorts, women who completed paper surveys were more likely than online completers to become nonrespondents on the next survey. Due to the large sample size, almost all differences were statistically significant, with P<.001.

Conclusions:

Despite the cost-saving advantages of online compared to paper surveys, paper surveys are likely to appeal to a different population of potential respondents with different sociodemographic, behavioral, and health characteristics and greater likelihood of attrition from the study. Not offering a paper version is therefore likely to induce bias in the distribution of responses unless weighting for respondent characteristics (relative to the target population) is employed. Therefore, if mixed mode (paper or online) options are feasible, they are highly likely to produce more representative results than if only the less costly online option is offered.