JMIR Publications

ABSTRACT

Background:

Probability-based surveys remain essential to epidemiology by providing representative estimates and capturing key health determinants. Growing reliance on web-based self-administered surveys, driven by cost efficiency and scalability, has accelerated the use of mixed-mode designs combining online and face-to-face approaches. However, social interviewer-led interviews, web-based self-administered surveys, and nurse-led interviews may introduce mode effects that compromise comparability, including those related to social desirability bias, interviewer and professional characteristics, and primacy or recency effects.

Objective:

This study aimed to assess mode effects by comparing self-administered measures of mental health and functioning across web-based self-completion and computer-assisted personal interviewing (CAPI) administered by social interviewers and nurses, as well as doctor-diagnosed health conditions collected by interviewers in CAPI and self-completed in the web mode. We further examined whether mode effects varied by sex and age group.

Methods:

The present analysis was based on a randomised mode allocation embedded within Wave 12 (2019) of the UK Household Longitudinal Study Innovation Panel (UKHLS–IP), originally implemented to support evaluation of participant-led biomeasure collection. In IP12, 1,111 households (1,779 participants) were assigned to one of three modes: web-based self-completion, interviewer-administered CAPI, or nurse-administered CAPI. Eligible participants were aged 16 years or older. Mode effects on self-reported mental health, functioning, and diagnosed conditions were estimated using a two-stage residual inclusion instrumental variable approach.

Results:

No significant differences were observed across modes for the General Health Questionnaire-12 or the Short Form-12 mental and physical component scores, and no mode differences were found for doctor-diagnosed conditions in the main analyses. For example, no statistically significant mode effects in psychiatric conditions were identified across the comparisons: nurse relative to social interviewer (OR = 0.72, 95% CI 0.40–1.29, P=.53), web relative to social interviewer (OR = 0.45, 95% CI 0.17–1.18, P=.14), and web relative to nurse (OR = 0.63, 95% CI 0.27–1.49, P=.59). However, stratified analyses showed lower respiratory reporting in web-based responses among adults aged 56 years or older and men, and higher musculoskeletal reporting in nurse-administered interviews among men, with differences exceeding five percentage points.

Conclusions:

Mode effects in report of health status, including mental health conditions, were generally limited. This may support the role of self-administered health scales and condition-specific questions in enhancing comparability across survey modes. Specific differences, including lower respiratory reporting among older adults in web-based surveys and higher musculoskeletal reporting among men in nurse-administered interviews, require further investigation to elucidate the underlying mechanisms. These specific differences may underscore the need for mode-specific design considerations and respondent support strategies, especially for demographic groups that may under-report certain health conditions.