Estimation of SARS-CoV-2 seroprevalence in central North Carolina: accounting for outcome misclassification in complex sample designs

Abstract

Background:

Population-based seroprevalence studies are crucial to understand community transmission of COVID-19 and guide responses to the pandemic. Seroprevalence is typically measured from diagnostic tests with imperfect sensitivity and specificity. Failing to account for measurement error can lead to biased estimates of seroprevalence. Methods to adjust seroprevalence estimates for the sensitivity and specificity of the diagnostic test have largely focused on estimation in the context of convenience sampling. Many existing methods are inappropriate when data are collected using a complex sample design.

Methods:

We present methods for seroprevalence point estimation and confidence interval construction that account for imperfect test performance for use with complex sample data. We apply these methods to data from the Chatham County COVID-19 Cohort (C4), a longitudinal seroprevalence study conducted in central North Carolina. Using simulations, we evaluate bias and confidence interval coverage for the proposed estimator compared with a standard estimator under a stratified, three-stage cluster sample design.

Results:

We obtained estimates of seroprevalence and corresponding confidence intervals for the C4 study. SARS-CoV-2 seroprevalence increased rapidly from 10.4% in January to 95.6% in July 2021 in Chatham County, NC. In simulation, the proposed estimator demonstrates desirable confidence interval coverage and minimal bias under a wide range of scenarios.

Conclusion:

We propose a straightforward method for producing valid estimates and confidence intervals when data are based on a complex sample design. The method can be applied to estimate the prevalence of other infections when estimates of test sensitivity and specificity are available.

INTRODUCTION

COVID-19, the clinical disease caused by SARS-CoV-2, was declared a pandemic by the WHO on March 11, 2020 following alarming levels of global spread.¹ After regulatory approval of assays for the detection of SARS-CoV-2 antibodies, seroprevalence studies emerged as a key means for surveillance. Accurate seroprevalence estimates are necessary for modeling virus transmission, determining infection fatality rates, guiding public health policies,^2-6 and estimating the duration of detectable antibodies in the population due to infection or vaccination.^{7, 8}

Seroprevalence estimates are based on diagnostic tests that detect whether sampled individuals carry antibodies for a specific infection. While IgG antibodies tend to be highly specific in nature, diagnostic tests typically result in some false positive and false negative results. Misclassification of patient serostatus based on these tests introduces measurement error. Disregarding a test’s sensitivity, or the probability of a positive result when applied to a positive case, and specificity, or the probability of a negative result when applied to a negative case, can lead to biased estimates.^9-11

Methods have been developed to adjust prevalence estimates to account for imperfect diagnostic tests. Rogan and Gladen⁹ discuss a prevalence estimator that corrects for known sensitivity and specificity, though these quantities are often unknown and estimated from assay validation studies.¹² Methods to estimate SARS-CoV-2 seroprevalence that account for both the misclassification and variance associated with the serologic test performance have been developed and evaluated.^{11, 13-15}

Existing methods largely focus on estimation from convenience samples such as those recruited from clinics, social media platforms, and shopping centers.^16-19 These designs offer advantages from the standpoint of time and cost, but introduce the potential for selection bias if factors related to study participation are associated with SARS-CoV-2 infection.³ Alternatively, probability-based sample designs have been used for estimating seroprevalence.^20-24 Two general inferential approaches for the analysis of data from probability-based sample designs are model-based and design-based inference. Model-based inferential methods (e.g., Bayesian and large-sample frequentist methods) use stochastic models to estimate population values, so estimators are only unbiased when the model assumptions hold. With design-based approaches, inference is based on repeated sampling from a finite population. Such approaches require no assumptions about the distribution of the outcome, thus facilitating generalizability of estimates to a specified target population.³ However, design-based analysis must account for complex design features such as stratification, clustering, and unequal probabilities of selection.^{25, 26} Ignoring these features and analyzing complex sample data with conventional statistical methods can lead to biased point estimates and confidence interval undercoverage.²⁷

Many studies implementing probability-based sample designs have not accounted for imperfect sensitivity and specificity of the serologic test.^{21, 28-30} Bootstrap methods are commonly applied to data from convenience samples to account for test sensitivity and specificity,^{15, 31-35} and have been used with probability-based sample designs.^{23, 36-38} However, the standard bootstrap method proposed by Efron³⁹ is inappropriate for use with complex sample data as it assumes data are independent and does not account for correlation within sample clusters or accommodate sample weights or strata.⁴⁰ Design-consistent bootstrap methods have been developed,⁴¹ but not yet evaluated empirically or applied for adjustments of sensitivity and specificity under complex sample designs.

We evaluate and demonstrate an adaptation of the Rao–Wu rescaling bootstrap^{42, 43} that incorporates methods from Rogan and Gladen to estimate seroprevalence from complex sample designs. This method accounts for complex sample design features, including sample weights, stratification, and clustering,⁴¹ and imperfect sensitivity and specificity while avoiding reliance on parametric assumptions. As a motivating example, we estimate the seroprevalence of SARS-CoV-2 antibodies in the target population of the Chatham Country COVID-19 Cohort (C4) study, which was selected using a complex sample design. To date, results of seroprevalence studies in North Carolina are limited to nonprobability samples, such as those from remnant serum samples and convenience samples drawn from health-seeking populations.^{44, 45} Other samples consider special populations, including healthcare and industrial livestock workers.^{46, 47} We present the first seroprevalence estimates from a probability-based cohort in North Carolina.

We estimate seroprevalence using a standard estimator as well as our proposed corrected estimator. In C4, an imperfect diagnostic test was used to detect antibodies in serum. In simulation experiments, we explore the bias of both estimators and corresponding 95% confidence interval coverage and half-width for several scenarios. We vary population prevalence, sensitivity and specificity of the diagnostic test, and average sample size to explore how the performance of the two estimators varies by these factors.

PART 1. EXAMPLE

Methods

C4 was a longitudinal, prospective, population-based study designed to estimate the prevalence of SARS-CoV-2 infection in central North Carolina. The details of the study have been previously published.²⁴ Study protocols were approved by the University of North Carolina at Chapel Hill Institutional Review Board. Briefly, study participants were Chatham County residents age 18 or older selected through a stratified, three-stage cluster sample design targeting a total enrollment of 300 participants. The sample was conducted in two phases, with Phase 1 participants recruited from the existing and ongoing Chatham County Community Cohort and Phase 2 participants recruited from additional census blocks sampled under the same methodology as the original cohort. Phase 2 households were selected using address-based sampling methods with supplemental addresses to improve sampling frame coverage in rural areas.^{48, 49} Among the 1535 addresses sampled for the study, 140 individuals (9%) enrolled and completed at least one study visit as of July 31, 2021. Monthly clinic visits were conducted between August 2020 and September 2022 where serum samples were collected from participants via venous blood draws. Alternatively, participants could elect to use Tasso serum self-collection devices at home. Research has demonstrated strong agreement in anti-SARS-CoV-2 IgG results based on Tasso and venous blood samples.⁵⁰ Our goal is to estimate seroprevalence of SARS-CoV-2 antibodies in adult residents of Chatham County, NC from January through July, 2021.

Preliminaries

Assume that the true population seroprevalence ($p$) will be estimated based on the results of a diagnostic test conducted on a sample of $n$ individuals from a stratified, three-stage cluster sample. In this design, $m_h$ primary sampling units (PSUs) are selected from $M_h$ population clusters in stratum $h$, with $h=1,\dots ,H$ using probability proportional to size with replacement sampling and $U_{hj}$, the number of secondary sampling units (SSUs) in each PSU $j$ in stratum $h$, as the measure of size. A simple random sample (without replacement) of $u_{hj}$ SSUs is then selected from the members of each selected PSU. Finally, a simple random sample (without replacement) of $n_{hjk}$ individuals is selected from the $N_{hjk}$ individuals in each selected SSU, yielding a total of $n={\sum }_{h=1}^H{\sum }_{j=1}^{m_h}{\sum }_{k=1}^{u_{hj}}{n}_{hjk}$ sample members. A selected individual $l$ in stratum $h$, PSU $j$, and SSU $k$ then has sampling weight $w_{hjkl}=(m_h{\Psi }_{hj}{)}^{-1}(U_{hj}∕u_{hj})(N_{hjk}∕n_{hjk})$, where ${\Psi }_{hj}=U_{hj}∕({\sum }_{q=1}^{M_h}{U}_{hq})$.

In C4, census blocks were stratified into three strata using income tertiles. In the first stage of sampling, census blocks, serving as PSUs, were selected using probability proportional to size with replacement sampling, with the estimated number of occupied households in each block serving as the measure of size. Next, in the second stage of sampling, households served as SSUs. An average of six households per census block were sampled and were recruited by mail, telephone, and via in-person visits. Finally, one adult from each selected household was eligible to participate in the study. Ideally, this individual would be randomly selected, but in C4, any one adult in a selected household could elect to participate.

Let the observed binary serologic result for each individual be denoted $y_{hjkl}$, where $y_{hjkl}=1$ indicates that the individual tested positive for antibodies and $y_{hjkl}=0$ otherwise. The true sensitivity and specificity of the serologic test are denoted by $S_e$ and $S_p$, respectively. In C4, serum samples were tested with enzyme-linked immunosorbent assay (ELISA), using the recombinant spike protein antigen to detect total SARS-CoV-2 Ig in plasma with ${\widehat{S}}_e=0.897$ and ${\widehat{S}}_p=0.993$.^{51, 52} This assay detects seropositivity due to either prior SARS-CoV-2 infection or vaccination.

The features of complex sample designs can be leveraged to better achieve study objectives, but require special handling during analyses. Stratified designs allow the sampling frame to be divided into mutually exclusive and exhaustive categories, such as high- or low-risk groups and facilitate oversampling from certain strata relative to others. Oversampling could be used to give high-risk individuals a greater probability of selection compared to low-risk individuals, increasing the number of observed seropositives in the sample for improved precision. In C4, stratification by socioeconomic status (SES) allowed for oversampling of low income census blocks to improve representation in these regions. Cluster sampling is often used with in-person recruitment to reduce costs by minimizing the spread in geographic locations. Sample weights allow each sample member to represent the appropriate number of population members, compensating for any oversampling in the design. Accounting for clustering, sample weights, and stratification facilitates unbiased variance estimation, as each affects the variance of an estimator in a complex sample design. For example, clustering induces correlation between sample members within the same PSUs, which tends to inflate variances compared to simple random sampling.

Standard Estimator

A commonly used design-based estimator of population prevalence is the ratio estimator:²⁵

$${\widehat{p}}_s=\frac{\sum _{h=1}^H\sum _{j=1}^{m_h}\sum _{k=1}^{u_{hj}}\sum _{l=1}^{n_{hjk}}{w}_{hjkl}{y}_{hjkl}}{\sum _{h=1}^H\sum _{j=1}^{m_h}\sum _{k=1}^{u_{hj}}\sum _{l=1}^{n_{hjk}}{w}_{hjkl}}.$$ (1)

The standard Wald confidence interval is constructed as ${\widehat{p}}_s\pm t_{df,1-\alpha ∕2}\widehat{SE}({\widehat{p}}_s)$, where $t_{df,1-\alpha ∕2}$ represents the $1-\alpha ∕2$ percentile of the t-distribution with $df$ degrees of freedom, $df={\sum }_{h=1}^H{m}_h-H$, $\alpha$ is the desired Type I error rate, and $\widehat{SE}({\widehat{p}}_s)$ is the estimated standard error of ${\widehat{p}}_s$. Note $\widehat{SE}({\widehat{p}}_s)$ is estimated using design-based estimation methods such as Taylor Series Linearization, Balanced Repeated Replication, or Jackknife replication using software such as SAS (SAS Institute Inc, Cary, NC) ‘survey’ procedures, the R ‘survey’ package, or SUDAAN (RTI International, Research Triangle Park, NC).²⁵ Each of these methods appropriately accounts for complex sample design features. Because standard Wald-typed confidence intervals have been shown to under-cover the true population proportion, particularly when prevalence is close to zero, alternatives such as the logit transformation have been proposed.^53-55 The lower and upper logit confidence limits, $P_L$ and $P_U$ respectively, are computed as $P_L=\text{exp}(Y_L)∕\{(1+\text{exp}(Y_L)\}$ and $P_U=\text{exp}(Y_U)∕\{(1+\text{exp}(Y_U)\}$ where $Y_L$ and $Y_U$ are the lower and upper endpoints of $\log \{{\widehat{p}}_s∕(1-{\widehat{p}}_s)\}\pm [t_{df,1-\alpha ∕2}\widehat{SE}({\widehat{p}}_s)∕\{{\widehat{p}}_s(1-{\widehat{p}}_s)\}]$, respectively.^{25, 54, 56} This method ensures that resulting confidence intervals will be contained within the bounds of the parameter space and demonstrate better coverage than Wald intervals when the prevalence is close to the boundary of the parameter space.^{25, 57}

When nonresponse or sampling frame undercoverage is nonnegligible, sample weights are typically adjusted or calibrated to mitigate potential bias.⁵⁸ In this setting, the sample weight in (1) is replaced with the adjusted or calibrated weight when estimating seroprevalence, and sums are restricted to include only responding individuals. However, even if calibration methods are appropriately applied to reduce bias from nonresponse or sampling frame undercoverage, they do not correct for misclassification error from the serologic test.

Corrected Estimator

The corrected estimator adjusts the standard estimator for the estimated sensitivity and specificity of the serologic test using a weighted version of the method discussed by Rogan and Gladen,⁹ such that

$${\widehat{p}}_c=\frac{{\widehat{p}}_s+{\widehat{S}}_p-1}{{\widehat{S}}_e+{\widehat{S}}_p-1}$$ (2)

where ${\widehat{S}}_e$ and ${\widehat{S}}_p$ represent the estimated sensitivity and specificity of the diagnostic test, respectively. Note that if ${\widehat{p}}_s$ is less than or equal to $1-{\widehat{S}}_p$, then ${\widehat{p}}_c$ will be negative, which is outside the parameter space of $p$.

The variance of ${\widehat{p}}_c$ depends on the variances of ${\widehat{p}}_s$, ${\widehat{S}}_e$, and ${\widehat{S}}_p$. For confidence interval construction, consider the following adaptation of the non-parametric Rao–Wu rescaling bootstrap method,^{25, 42, 43} which accounts for variation both in selection of the sample (incorporating complex sample features) and in estimation of $S_e$ and $S_p$.

1. Select $B$ replicate samples from the primary sample by selecting $m_h-1$ PSUs from each stratum $h$, with replacement. Within each replicate sample $b=1,2,\dots B$, construct bootstrap weights $w_{hjkl}^b$ for each individual $l$ in SSU $k$, PSU $j$, and stratum $h$ as

   $$w_{hjkl}^b=w_{hjkl}{r}_{hj}^b\frac{m_h}{(m_h-1)}$$ (3)

   where $r_{hj}^b$ is the number of times PSU $j$ in stratum $h$ was selected in bootstrap sample $b$.

2. Estimate the proportion of the population with a positive serologic result based on each replicate sample $b$ as

$${\widehat{p}}_s^b=\frac{\sum _{h=1}^H\sum _{j=1}^{m_h-1}\sum _{k=1}^{u_{hj}}\sum _{l=1}^{n_{hjk}}{w}_{hjkl}^b{y}_{hjkl}}{\sum _{h=1}^H\sum _{j=1}^{m_h-1}\sum _{k=1}^{u_{hj}}\sum _{l=1}^{n_{hjk}}{w}_{hjkl}^b}.$$ (4)

1. To account for variation in the estimation of sensitivity and specificity, generate estimates of sensitivity and specificity for each bootstrap sample $b$. Assume that sensitivity and specificity were estimated based on lab tests of independent samples of sizes $t_1$ and $t_2$, respectively. Take random draws from two independent binomial distributions to estimate sensitivity and specificity for each bootstrap sample. That is, let $X_1^b\sim Binomial(t_1,{\widehat{S}}_e)$ with realized value $x_1^b$ and $X_2^b\sim Binomial(t_2,{\widehat{S}}_p)$ with realized value $x_2^b$. Estimate the sensitivity for bootstrap sample $b$ as ${\widehat{S}}_e^b=x_1^b∕t_1$ and the specificity as ${\widehat{S}}_p^b=x_2^b∕t_2$. If validation data are available rather than just summary data, an alternative approach is to take resamples directly from the validation datasets and compute ${\widehat{S}}_e^b$ and ${\widehat{S}}_p^b$ within each resampled dataset.

2. Correct each bootstrap prevalence estimate ${\widehat{p}}_s^b$ for the bootstrap-specific estimates of sensitivity and specificity by computing

$${\widehat{p}}_c^b=\frac{{\widehat{p}}_s^b+{\widehat{S}}_p^b-1}{{\widehat{S}}_e^b+{\widehat{S}}_p^b-1}.$$ (5)

Steps (A)-(D) result in $B$ estimates of ${\widehat{p}}_c^b$, one for each bootstrap sample. Construct the $1-\alpha$ percentile-based confidence interval for $p$ by taking the $\alpha ∕2$ and $1-\alpha ∕2$ percentiles of the distribution of estimates as the confidence interval endpoints.

As with the standard estimator, the corrected estimator can be adjusted to account for nonresponse or sampling frame undercoverage. When sample weights are adjusted or calibrated, the adjusted or calibrated weights are used to compute ${\widehat{p}}_s$ in (2) for point estimation, and the same weight adjustment procedures are applied to $w_{hjkl}^b$ within each bootstrap sample prior to computing (4).⁴¹

In C4, we calibrated sample weights to 2020 Census population counts for adults in Chatham County by sex and age group (18-49 versus 50+) using Generalized Exponential Models with the WTADJUST procedure in SUDAAN version 11 (RTI International, Research Triangle Park, NC).⁵⁹ In some months, due to high seroprevalence, we truncated the upper confidence limits when the corrected estimator exceeded 1.

Results

We included a total of 134 participants in our study sample (Table 1). Among the participants, 83 (61.9%) were female, 108 (80.6%) were white, and 113 (84.3%) were not Hispanic or Latino. Most participants resided in a rural area (75.4%). Most participants (78.6%) reported completing a college degree or higher, about half (51.1%) reported a yearly income of at least $75,000. The majority of the cohort (87.3%) self-reported full-vaccination with a primary vaccine series.

Table 1:

Demographic characteristics of C4 study participants at enrollment (n=134).

Characteristic	Participants n (%)
Age group
18-49	33 (25)
50+	101 (75)
Biological sex
Female	83 (62)
Male	51 (38)
Race
White	108 (86)
Black/African American	8 (6)
Other	9 (7)
Missing or Unknown	9
Ethnicity
Hispanic or Latino	6 (5)
Not Hispanic or Latino	113 (88)
Other	10 (8)
Missing or Unknown	5
Education
<=Some college	28 (21)
College degree+	103 (79)
Missing or Unknown	3
Income
<$50,000	25 (20)
$50,000-$74,999	33 (26)
$75,000+	68 (54)
Missing or Unknown	8
Residence
Rural	101 (75)
Semi-urban	33 (25)
Primary vaccine series receiveda
Moderna	61 (46)
Pfizer	51 (38)
Janssen	4 (3)
Unknown	1 (1)
Not vaccinated	17 (13)

^aCollected in follow-up surveys after vaccine rollout.

The resulting standard and corrected seroprevalence estimates and corresponding confidence intervals are shown in Figure 1. SARS-CoV-2 seroprevalence due to infection and/or vaccination increased dramatically over this period, with corrected estimates of 10.4% in January and 95.6% in July, 2021. During this period, vaccines were made widely available to adults, so a large increase in seroprevalence was expected. The estimated seroprevalence decreased slightly between May and June, 2021. Accounting for the misclassification of the assay produced corrected estimates that are greater than the standard estimates due to the assay’s very high specificity but lower sensitivity, which has the tendency to produce more false negative than false positive results. Bootstrap confidence intervals for the corrected method are wider than the logit confidence intervals for the standard method. Both sets of confidence intervals are wide, reflecting lower than anticipated sample sizes during this period (sample sizes ranged from $n=103$ in July to $n=113$ in April).

Figure 1:

Estimated SARS-CoV-2 seroprevalence in Chatham County, NC among persons 18 years and older from January through July, 2021. Note the upper confidence limits for the corrected estimator were truncated to 1 for April through July.

PART 2. SIMULATION STUDY

Methods

We conducted a simulation study to assess the performance of the standard and corrected estimators. We modeled simulations after C4 as described above. We compared empirical bias, 95% confidence interval coverage, and confidence interval half-width of the corrected and standard estimators under a range of scenarios. We evaluated sixteen scenarios, where sensitivity ($S_e\in \{0.8,0.9\}$), population seroprevalence ($p\in \{0.01,0.025,0.1,0.5\}$), and sample size (average $n\in \{162,324\}$) were varied. We held the specificity, $S_p$, at 99% for primary scenarios, as serologic tests with emergency use authorization have very high specificity (mean=98.7%, range: 94.8% to 99.3%).¹² In secondary scenarios, we let both sensitivity and specificity be 80%. We conducted all simulations in SAS Studio 3.8 (SAS Institute Inc, Cary, NC) with 1000 iterations per scenario.

We used Census 2010 estimates for the number of occupied households in each census block in Chatham County to simulate the household-level target population. As in C4, we stratified census blocks into three strata using income tertiles. We simulated risk contributions for each census block (PSU) $j$ as $c_j\sim Uniform(-0.005,0.005)$. We simulated risk contributions for each household (SSU) $k$ as $d_k\sim Uniform(-0.01,0.01)$ for the scenario with $p=0.01$, and as $d_k\sim Uniform(-0.05,0.05)$ for all other scenarios. Then, we defined the probability of infection for an individual in each household $k$ in census block $j$, and stratum $h$ in the target population as $p_{hjk}=p_h+c_j+d_k$, where $p_h\in \{0.004,0.009,0.014\}$ for scenarios with average $p=0.01$, $p_h\in \{0.009,0.024,0.039\}$ for scenarios with average $p=0.025$, $p_h\in \{0.06,0.11,0.16\}$ for scenarios with average $p=0.1$, and $p_h\in \{0.46,0.51,0.56\}$ for scenarios with average $p=0.5$. To ensure the probability of infection was positive, we truncated negative values of $p_{hjk}$ to 0.0001.

For each household, we simulated the number of adults from a Poisson distribution with mean 1.94, the average number of adults per household in Chatham County based on 2020 Census data. Then, for each individual $l$ in household $k$ in the population, we simulated a true infection status as $x_{hjkl}\sim Bernoulli(p_{hjk})$. Thus, the true population prevalence for each scenario was equal to

$$p=\frac{\sum _{h=1}^H\sum _{j=1}^{M_h}\sum _{k=1}^{U_{hj}}\sum _{l=1}^{N_{hjk}}{x}_{hjkl}}{\sum _{h=1}^H\sum _{j=1}^{M_h}\sum _{k=1}^{U_{hj}}{N}_{hjk}}.$$

The serologic test status for each individual in the target population if that individual were to be tested for SARS-CoV-2 antibodies was assigned as $y_{hjkl}=I(x_{hjkl}=1)T_1+I(x_{hjkl}=0)T_2$, where $T_1\sim Bernoulli(S_e)$ and $T_2\sim Bernoulli(1-S_p)$ and $I(\ast )$ represents an indicator function that equals 1 if * is true and 0 otherwise.

For each iteration of a simulation scenario, we selected a sample of PSUs from each stratum ($m_1=51$, $m_2=51$, $m_3=60$) using probability proportional to size with replacement sampling with probability of selection proportional to the number of occupied households in each census block (PSU). Within each sampled PSU, we randomly selected $u_{hj}=min(d,U_{hj})$ households without replacement, with $d\in \{1,2\}$ for target sample sizes $n=162$ and $n=324$, respectively. In the third stage of sampling, one individual from each selected household was randomly selected. Based on the sample of individuals, we computed (1) and (2) and constructed 95% confidence intervals to correspond with each estimator. For the standard estimator, we constructed a logit confidence interval for the proportion of individuals testing positive for SARS-CoV-2, with $\widehat{SE}({\widehat{p}}_s)$ estimated using the Taylor Series Linearization method in SAS’ SURVEYMEANS procedure (SAS Institute Inc, Cary, NC).²⁵ For the corrected estimator, we implemented the bootstrap method described above with 1000 bootstrap samples, and the sizes of the validation samples in the C4 study for sensitivity ($t_1=145$) and specificity ($t_2=274$).⁵² We truncated negative values for the corrected prevalence and bootstrap estimates to 0. We report mean empirical bias, estimated by $R^{-1}\sum _{r=1}^R({\widehat{p}}_e-p)$ with $e\in \{s,c\}$. R denotes the number of simulations. For both estimators, we assessed confidence interval coverage as the proportion of simulations where the resulting confidence interval contained the true population prevalence, $p$, and confidence interval half-width was calculated as half the difference of the confidence interval endpoints.

Results

The results of the simulation study are presented in Figures 2-3 and Table 2, which contains detailed results for all sixteen scenarios.

Figure 2:

Results of the simulation study by method and scenario based on 1000 simulated samples. Mean proportion empirical bias and 95% confidence interval coverage calculated for population seroprevalence. Note: p= the true population seroprevalence and S_e=sensitivity.

Figure 3:

Histograms of estimated seroprevalence across the 1000 simulated samples by scenario for the standard estimator and corrected estimator.

Table 2:

Detailed results of the primary simulation scenarios, 1000 simulations. Note: CI=confidence interval, S_e=sensitivity.

	Average Prevalence	Estimator	Mean Empirical Bias	95% CI Coverage (%)	Mean 95% CI Half-width
Se = 0.8, Average n = 324
1a	0.01	Standard	0.008	72	0.018
		Corrected	0.001	94	0.017
2	0.025	Standard	0.004	92	0.022
		Corrected	−0.001	95	0.028
3	0.10	Standard	−0.010	91	0.036
		Corrected	0.000	95	0.049
4	0.50	Standard	−0.100	17	0.063
		Corrected	−0.001	97	0.092
Se = 0.8, Average n = 162
5a	0.01	Standard	0.009	76	0.028
		Corrected	0.003	87	0.022
6a	0.025	Standard	0.004	94	0.032
		Corrected	0.000	92	0.034
7	0.10	Standard	−0.011	93	0.050
		Corrected	0.000	94	0.066
8	0.50	Standard	−0.096	44	0.088
		Corrected	−0.001	96	0.123
Se = 0.9, Average n = 324
9	0.01	Standard	0.009	71	0.019
		Corrected	0.000	94	0.017
10	0.025	Standard	0.007	90	0.023
		Corrected	−0.001	96	0.026
11	0.10	Standard	−0.001	94	0.037
		Corrected	0.000	96	0.045
12	0.50	Standard	−0.043	75	0.064
		Corrected	0.002	97	0.079
Se = 0.9, Average n = 162
13a	0.01	Standard	0.010	76	0.029
		Corrected	0.002	86	0.021
14a	0.025	Standard	0.007	89	0.034
		Corrected	0.000	90	0.033
15	0.10	Standard	−0.001	95	0.053
		Corrected	0.000	94	0.061
16	0.50	Standard	−0.043	86	0.089
		Corrected	0.002	95	0.108

Mean empirical bias, 95% confidence interval coverage, and 95% confidence interval half-widths are presented for population seroprevalence.

^aIn certain scenarios with p ∈ {0.01, 0.025}, some iterations produced no positive test results i.e., (y_hjkl = 0 for all individuals), thus the number of simulations was slightly less than 1000, ranging from 938-996.

Average empirical bias was low (0.003 or less) for the corrected estimator for all scenarios considered (Figure 2, Table 2), with estimates across the 1000 simulations clustered close to the true population prevalence (Figure 3). The standard estimator demonstrated considerable bias across the scenarios, with absolute mean empirical bias up to approximately 0.1. For all scenarios considered, bias was larger for the standard estimator compared to the corrected estimator. For the standard estimator, bias was largest when $p=0.5$ (Figure 2, Table 2). The standard estimator tended to underestimate seroprevalence more severely for the $p\in \{0.1,0.5\}$ scenarios (Figure 3), due to the very high specificity and lower sensitivity values considered. Bias was similar for the two sample sizes considered.

Confidence intervals were generally narrower for the standard method compared to the corrected method, except for all scenarios with $p=0.01$ and one scenario with $p=0.025$ (Table 2), but this came at the expense of coverage. Confidence intervals for the standard method demonstrated close to nominal coverage (at least 93%) in only four scenarios, and demonstrated considerable under-coverage in remaining scenarios with empirical coverage ranging from 17% to 92%. In contrast, the bootstrap method demonstrated close to nominal coverage (94% to 97%) for most scenarios, except where $p\in \{0.01,0.025\}$ and average $n=162$, where empirical coverage ranged from 86% to 92% (Figure 2, Table 2).

Detailed results for secondary scenarios with $S_p=0.08$ are presented in Table 3. The corrected estimator achieved nominal 95% confidence interval coverage in all scenarios, and low bias except when prevalence was extreme. Because of greater misclassification error, the standard estimator demonstrated larger bias than in the primary scenarios, and 95% confidence interval coverage fell to 0 when $p\in \{0.01,0.025,0.1\}$.

Table 3:

Secondary simulation scenarios where sensitivity and specificity are 80%, average n = 324, 1000 simulations. Note: CI=confidence interval.

Average Prevalence	Estimator	Mean Empirical Bias	95% CI Coverage (%)	Mean 95% CI Half-width
0.01	Standard	0.197	0	0.052
	Corrected	0.014	99	0.063
0.025	Standard	0.191	0	0.053
	Corrected	0.009	98	0.070
0.10	Standard	0.153	0	0.056
	Corrected	0.002	98	0.111
0.5	Standard	−0.007	95	0.064
	Corrected	0.001	97	0.013

Overall, these simulations demonstrate minimal bias and close to nominal confidence interval coverage for the corrected estimator with bootstrap confidence intervals for almost all scenarios considered. They also show how failing to correct for the sensitivity and specificity of the serologic test with use of standard design-based estimation methods can result in considerable bias and confidence interval under-coverage.

DISCUSSION

The COVID-19 pandemic has emphasized the need for accurate seroprevalence estimators to monitor virus transmission, inform policy, and track the status of antibodies over time. Complex sample designs allow for generalizability to a specified target population, and features such as clustering and unequal sampling probabilities can reduce data collection costs and allow for oversampling of population subgroups. To yield valid inference, design-based seroprevalence estimators applied to complex data should account for design features and misclassification error introduced by diagnostic tests.

In this article, we discuss and illustrate the importance of accounting for misclassification of diagnostic tests when estimating seroprevalence, and we present the first seroprevalence estimates from a probability-based cohort study in North Carolina. The weighted Rogan–Gladen estimator, paired with a confidence interval generated through the proposed design-based bootstrap method, allows for seroprevalence estimation from complex sample designs. In simulations, the proposed estimator was empirically unbiased and confidence intervals demonstrated close to nominal coverage across a range of scenarios with variation in sample sizes, test characteristics, and population prevalence. When applied to data from C4, the methods accounted for the serologic test’s lower sensitivity and higher specificity.

There are limitations associated with the proposed methods. Confidence interval coverage based on the percentile bootstrap method has been shown to fall below the nominal level when seroprevalence is close to the boundary of the parameter space.¹³ In simulation studies, when prevalence was very low (0.01 or 0.025), the bootstrap method demonstrated close to or exceeded nominal coverage for the larger sample size considered but fell below nominal coverage for the smaller sample size. Given these findings and findings in other settings, caution should be used when applying these methods where extreme seroprevalence values are expected, particularly with small samples. Additionally, the Rogan–Gladen estimator may yield negative estimates in situations with low prevalence and low test specificity.⁶⁰ The methods require estimates of diagnostic test sensitivity and specificity as well as corresponding validation study sample sizes and assume sensitivity and specificity of the serologic test are the same in validation and study samples. We only consider repeated cross-sectional seroprevalence estimation; future work could extend these methods to allow for longitudinal approaches.

There are limitations associated with our motivating example as well. Given limitations in available study recruitment approaches coupled with starting the study during the first months of the pandemic, C4 experienced low response rates where participation was associated with demographic characteristics. While calibration ameliorates some of the bias associated with differential nonresponse, bias due to unmeasured factors or factors not accounted for in the calibration weighting can lead to results that do not reflect population prevalence. Further, while ideally we would randomly select one adult from each household to participate, in C4, individuals residing in a sampled household selected a household member for inclusion. Additionally, the seroprevalence in the county may be overestimated in our sample. Our cohort had a high vaccination rate (weighted = 85%, unweighted = 87%). As of September 2022, only 67% of adults 18 years and older in Chatham County had received a complete primary vaccine series.⁶¹ Despite differences between our study sample and target population, our results illustrate how the standard estimator leads to biased estimates of population seroprevalence when an imperfect diagnostic test is used.

Furthermore, the example demonstrates trends in SARS-CoV-2 seroprevalence in Chatham County. In C4, SARS-CoV-2 seroprevalence increased dramatically from January to July, 2021. It is likely that most of this increase occurred due to vaccination rather than natural infection, as only 12 participants tested positive for COVID-19 during this time. These cases occurred in January (2), February (6) and March (4). The small decline in seroprevalence after May, 2021 could be explained by seroreversions (individuals who were seropositive reverting to seronegative) and that different subsets of study participants contributed serum samples each month.

Even with these limitations, the proposed methods provide a promising approach towards confidence interval construction for seroprevalence estimation from complex sample designs. This approach can be easily implemented, and we provide sample code for those who wish to utilize the method. While the focus of this paper is on SARS-CoV-2, these methods extend directly to prevalence estimation of other infections based on complex sample designs when validation data are available for the diagnostic test characteristics. These methods can also be applied to complex sample designs with a different number of stages, further extending the utility of the methods.

Data availability:

Deidentified individual data that supports the results will be shared beginning 9 to 36 months following publication upon request to the corresponding author, provided the investigator who proposes to use the data has approval from an Institutional Review Board (IRB), Independent Ethics Committee (IEC), or Research Ethics Board (REB), as applicable, and executes a data use/sharing agreement with UNC. Example application code is available on the author’s GitHub.