JMIR Publications

ABSTRACT

Background:

Mobile phones have become essential tools for communication, access to services, and socio-economic participation. Yet a significant digital gap persists in many countries by gender, education, and place of residence, with women, less educated people, and rural people less likely to own phones, access the Internet, and leverage phones for economic activity. To measure and understand this gap, survey questions must be both comprehensible to the respondents and contextually appropriate.

Objective:

This study aimed to use cognitive interviews to assess and refine survey questions measuring digital access and use among adult respondents in India, Kenya, and Nigeria.

Methods:

We conducted cognitive interviews in three LMIC settings such as rural Uttar Pradesh, India; rural and peri-urban areas around Nairobi, Kenya; and rural Kano, Nigeria. Interviews were conducted in Hindi (n=101), Kiswahili (n=90) and Hausa (n=90) to test draft survey questions on digital access and use (118 questions in India, 273 in Kenya, and 273 in Nigeria). These questions covered eight domains of our digital access and use framework (connectivity, access, affordability, digital skills, safety and security, agency, social and gender norms, and the relevance of phones to respondents’ lives). Data were analyzed through iterative team debriefs and systematic analysis of the interview transcripts, and inductive and deductive coding of question-functioning problems.

Results:

Cognitive interviews identified cross-cutting and domain-specific challenges affecting question comprehension and response validity. Across all three countries, long or multi-clause questions increased respondent burden and sometimes led participants to focus on only part of the question or answer a different construct. “I-statements” and Likert-type response options were poorly suited to orally administered surveys; respondents often converted scaled responses into binary answers, provided narrative responses, or indicated “I don’t understand” when the difficulty was with the response format rather than the underlying activity. Abstract digital terms such as mobile money, digital device, internet, and media sharing platform were often poorly understood, whereas locally familiar brands and concrete examples improved comprehension. Respondents also struggled to map frequent or routine digital behaviors onto predefined recall and frequency categories. Negatively phrased questions and stem-and-leaf questions generated misleading responses. Domain-specific challenges were identified particularly on agency, digital skills, online safety, and gender norms—often failed to capture intended constructs due to implicit social expectations, shared device use, and local interpretations. These findings informed substantial revisions to question wording, structure, and response options.

Conclusions:

Cognitive testing across Kenya, Nigeria, and India revealed common challenges in question comprehension and response, enabling the development of a standardized set of survey items on digital access and use that can be deployed across LMICs, with modest adaptation and periodic updating as digital environments evolve.