JMIR Publications

ABSTRACT

Background:

Robust measurement of digital inclusion is increasingly recognized as essential for monitoring progress toward equitable digital transformation. However, population-based surveys, including those which aim to measure digital inclusion, often include complex skip patterns, derived variables, and sensitive self-reported behaviors, making them particularly vulnerable to interviewer error, inconsistent data collection, and data falsification. Despite growing investment in digital solutions and increasing recognition of the importance of measuring who these interventions reach, there is limited guidance on scalable quality assurance approaches to ensure the validity and reliability of population-based digital inclusion data in low- and middle-income countries.

Objective:

To design and implement a multi-layered quality assurance and quality control (QA/QC) approach to improve data integrity in a large-scale household survey in a resource constrained setting.

Methods:

The study surveyed 13,568 respondents across 6087 households in 10 randomly selected districts of Bihar. A three-pronged QA/QC approach was deployed: (i) real-time quality control during data collection, (ii) rule-based error flagging to identify logical and range inconsistencies, and (iii) machine learning–based anomaly detection using the Isolation Forest algorithm to detect non-obvious patterns in error combinations. The system generated 17 binary error flags, including an anomaly flag identifying records in the top 5% of anomaly scores. Weekly feedback loops and interactive dashboards supported timely field-level corrective actions.

Results:

Implementation of the QA/QC approach resulted in an over 85% reduction in weekly errors over the 12-week data collection period. The anomaly detection component identified subtle but meaningful inconsistencies, such as irregular patterns in “don’t know” responses to survey questions, enhancing the sensitivity and depth of quality monitoring.

Conclusions:

Integrating rule-based validation with machine learning–based anomaly detection substantially strengthens QA/QC processes in complex survey environments. This scalable and replicable approach improves data quality in large-scale household surveys, particularly where interviewer error and data falsification are potential concerns. As countries increasingly invest in digital technologies and seek to evaluate the effectiveness, equity, and reach of these programs, robust measurement is essential for generating robust evidence to inform policy and programming. Although developed for a digital inclusion survey, the QA/QC approach is broadly applicable to other complex household surveys requiring rigorous quality assurance. With appropriate adaptation, its core components including real-time monitoring, automated validation, anomaly detection, and continuous feedback can also be embedded within routine technology-enabled health and development programs to strengthen data quality, support continuous quality improvement, and enable more timely, evidence-informed decision-making.