JMIR Publications

ABSTRACT

Background:

Sensitivity is a critical measure of lateral-flow antigen test (AT) performance, typically compared to qRT-PCR as the gold standard. For COVID-19 diagnostics, sensitivity reflects the AT’s ability to detect SARS-CoV-2 nucleoprotein. However, estimates of sensitivity can be skewed by differences in target concentration distributions within clinical sample sets, complicating performance comparisons across ATs from different suppliers. Regulatory guidelines generally recommend a balanced representation of low, mid, and high viral loads, yet real-world sample distributions are often variable. Previous studies have largely focused on raw sensitivity without adjusting for variability in viral load distribution (Ct values). While logistic regression has been used to model positive agreement as a function of viral load, no prior method adjusts sensitivity estimates based on a standardized reference distribution.

Objective:

To develop a method for estimating antigen test sensitivity aligned with a standard target concentration distribution using clinical test results from an uncontrolled concentration distribution.

Methods:

Sensitivity is calculated by modeling the probability of positive agreement (PPA) as a function of qRT-PCR cycle thresholds (Cts) through logistic regression on AT results. Raw sensitivity is computed as the ratio of AT positives to total PCR positives. Adjusted sensitivity is then derived by applying the PPA function to a reference concentration distribution, enabling uniform sensitivity comparisons across tests. This approach reduces the impact of sampling variability, as demonstrated using data from a study in Chelsea, Massachusetts, USA.

Results:

Over two years, paired antigen and PCR-positive tests from four AT suppliers were analyzed: A (211 tests), B (156), C (85), and D (43). Significant differences were found in Ct distributions, with suppliers A and D showing more high viral load samples, and supplier C showing more low viral load samples, leading to discrepancies in raw sensitivity. Using the PPA function estimated from each dataset, we calculated adjusted sensitivities for common reference Ct distributions, showing how sample variability affects raw sensitivity. Our method mitigated these discrepancies, enabling more accurate sensitivity comparisons across suppliers.

Conclusions:

This study demonstrates that real-world sensitivity estimates are vulnerable to deviations due to variability in qRT-PCR Ct distributions across studies. We introduce a novel methodology that compensates for this variability by calculating the PPA function from raw data and adjusting sensitivity based on a standardized reference distribution of Cts, ensuring more consistent and accurate sensitivity assessments. Our approach provides a robust mathematical solution for aligning sensitivity estimates with a standardized viral load distribution, enhancing the precision of this key performance metric. By adjusting for sample variability, this method improves quality control and supports regulatory oversight, offering a reliable framework for AT performance evaluation. Clinical Trial: https://clinicaltrials.gov/study/NCT05884515