A Capture–Recapture-based Ascertainment Probability Weighting Method for Effect Estimation With Under-ascertained Outcomes

Abstract

Outcome under-ascertainment, characterized by the incomplete identification or reporting of cases, poses a substantial challenge in epidemiologic research. While capture–recapture methods can estimate unknown case numbers, their role in estimating exposure effects in observational studies is not well established. This paper presents an ascertainment probability weighting framework that integrates capture–recapture and propensity score weighting. We propose a nonparametric estimator of effects on binary outcomes that combines exposure propensity scores with data from two conditionally independent outcome measurements to simultaneously adjust for confounding and under-ascertainment. Demonstrating its practical application, we apply the method to estimate the relationship between health care work and coronavirus disease 2019 testing in a Swedish region. We find that ascertainment probability weighting greatly influences the estimated association compared to conventional inverse probability weighting, underscoring the importance of accounting for under-ascertainment in studies with limited outcome data coverage. We conclude with practical guidelines for the method’s implementation, discussing its strengths, limitations, and suitable scenarios for application.

Administrative population registers play a vital role in epidemiology, providing data on disease incidence and exposure across more or less entire populations.¹ However, despite their richness in data, these databases can have shortcomings such as incomplete coverage, data inaccuracies, and absent records. For instance, it is often unclear if the data source(s) used to measure outcomes can capture all relevant outcome events. Unlike studies with original data collection, where researchers are typically aware of which individuals have missing data, unrecorded events in register-based studies lead to a problem called “under-ascertainment.”² This issue arises from our inability to differentiate between individuals with missing outcomes and those without, which means that conventional missing data imputation cannot be employed.

Under-ascertainment leads to an underestimation of disease prevalence and incidence and can therefore hamper disease surveillance efforts. When outcome ascertainment depends on individual characteristics, it can also result in biased exposure (or treatment) effect estimates by biasing the observed outcome numbers downward in one group more than the other, compromising the validity of causal estimates in a way similar to confounding.^1,3 Despite these serious implications, under-ascertainment often goes unaddressed in empirical studies.⁴

Researchers often conduct register validation studies to assess the accuracy of disease classifiers by comparing them with gold standard reference data, which are presumed to be complete.^1,4 In situations where such reference data are unavailable, capture–recapture methods can be employed. These methods use imperfect case data from multiple sources to estimate the true prevalence of underreported diseases.^5–10 For example, techniques have been applied to uncover “hidden” public health challenges such as drug use,¹¹ modern slavery,¹² homelessness,¹³ and more.¹⁴ However, existing capture–recapture methods have not been developed to estimate exposure effects, limiting their value for epidemiologic research.

In this paper, we introduce an ascertainment probability weighting (APW) framework that combines capture–recapture with exposure propensity scores to simultaneously address under-ascertainment and confounding. Unlike existing empirical approaches for handling outcome misclassification, which require a single, perfect validation dataset, our approach exploits outcome data from two imperfect sources.^15–25 We begin by introducing our framework and estimator, highlighting the assumptions required for valid estimation. We then demonstrate the approach through a hypothetical example and an empirical case study on under-ascertained coronavirus disease 2019 (COVID-19) tests in Sweden, while providing practical implementation guidance, discussing strengths and limitations, and highlighting its appropriate usage scenarios.

METHODS

Preliminaries and Notation

This section introduces the notation used throughout the paper. Let $Y_i$ denote a binary “target outcome” of interest, such as disease outcomes (e.g., lung cancer), health behaviors (smoking), or other characteristics (being homeless). $X_i$ represents a predictor of interest (e.g., an exposure or treatment variable), and $Z_i$ represents covariate(s).

To be concise, we often employ shorthand expressions when presenting conditional probabilities. For instance, instead of writing out the full expression $P(X_i=x|Z_i=z)$, we write $P(x|z)$.

Under-ascertainment

Our framework addresses under-ascertainment, a type of one-sided outcome misclassification. Under-ascertainment occurs when individuals with $Y_i=1$ may either be correctly classified as $Y_i^{\ast }=1$ or wrongly classified as $Y_i^{\ast }=0$, but all individuals with $Y_i=0$ are always classified correctly as $Y_i^{\ast }=0$ (i.e., there are no false positives). In the machine learning literature, this is often referred to as a positive-unlabeled classification problem.²⁶

Causal Framework and Assumptions

To relate the realized $Y_i$ to causal estimands, we apply the potential outcomes framework.²⁷ This framework defines a potential outcome $Y_i\left(x\right)$ as the outcome that would occur if the exposure (or treatment), perhaps contrary to fact, were set to x. We assume standard causal assumptions: consistency ($Y_i=Y_i\left(x\right)$ whenever $X_i=x$ is observed), exchangeability of $X_i$ and the potential outcomes conditional on observed covariates $Z_i$ ($Y_i\left(x\right)\mathrm{\perp }$$X_i$|$Z_i$ for all $x$), and exposure positivity ($P\left(x|z\right)>0$ for all $x$ and $z$). We also assume that all variables except $Y_i$ are measured without error, and that the study sample is representative of the (super-)population of interest.

Ascertainment Probability Weighting

Our objective is to estimate $P(Y_i\left(x\right)=1)$, representing the potential outcome probability under exposure level x, which can be used to compute various marginal causal effects, such as risk differences (RDs) or risk ratios (RRs).

In Equation (A4) in the Appendix, we establish a relationship between $P(Y_i\left(x\right)=1)$ and the under-ascertained data according to the following expression:

$${\displaystyle \begin{array}{l}{\displaystyle P(Y_i\left(x\right)=1)={\displaystyle \sum _z}\frac{P(Y_i^{\ast }=1,x|z)P\left(z\right)}{P(Y_i^{\ast }=1|Y_i=1,x,z)P\left(x|z\right)}.}\end{array}}$$  (1)

Here, $P(Y_i^{\ast }=1|Y_i=1,x,z)$ is the ascertainment probability conditional on $x$ and $z$. The rest of the expression follows the principles of conventional inverse probability weighting (IPW), including adjustment for confounding by inclusion of the exposure propensity score, $P\left(x|z\right)$, in the denominator, which adjusts for observable confounders ($Z_i$) by balancing their distribution between exposure groups.²⁸ For continuous $Z_i$, the summation can be replaced by integration.

Although Equation (1) cannot be computed directly due to the unobserved nature of $Y_i$, it can be estimated using capture–recapture. In our data setup, we observe under-ascertained outcome data from two data sources, where $Y_{ji}^{\ast }=1$ indicates the outcome’s ascertainment in source $j\in \{1,2\}$. In turn, $Y_i^{\ast }=1$ means that the outcome has been ascertained in at least one source, that is, $Y_i^{\ast }=1$ if $Y_{1i}^{\ast }=1$ or $Y_{2i}^{\ast }=1$.

We make two core assumptions beyond those mentioned in the previous section.

1. Assumption 1 (conditional source independence). $Y_{1i}^{\ast }\mathrm{\perp }{Y}_{2i}^{\ast }|(Y_i=1,X_i,Z_i).$

   Assumption 1 means that we assume conditional independence of the probability of ascertainment between the two data sources, given $Y_i=1$ and observed characteristics $Z_i$ and $X_i$. This relaxation of the basic capture–recapture method’s marginal independence assumption, which has also been exploited by others (e.g., Das et al.,⁷ Alho,⁸ and Tilling and Sterne⁹), is akin to the conditional exchangeability assumption with respect to exposure, used for causal estimation in observational studies when adjusting for confounding by observed covariates. In practice, the covariates involved should correspond to factors that are correlated with ascertainment in both sources, such as geographical factors and health behaviors.

2. Assumption 2 (ascertainment overlap). $P(Y_{1i}^{\ast }=1,Y_{2i}^{\ast }=1|Y_i^{\ast }=1,x,z)>0$ for all $(x,z).$

   Assumption 2 means we assume a positive probability for ascertained cases to appear simultaneously in both data sources within all strata of $X_i$ and $Z_i$. This condition is analogous to the exposure positivity assumption and can be verified by examining whether at least some ascertained cases are observed in both sources within all covariate strata.

If Assumptions 1 and 2 hold, Proposition 1 from Das et al.⁷ shows that

$${\displaystyle \begin{array}{l}{\displaystyle \begin{array}{lll}& {\displaystyle P(Y_i^{\ast }=1|Y_i=1,x,z)=}& {\displaystyle \frac{P(Y_{1i}^{\ast }=1,Y_{2i}^{\ast }=1|Y_i^{\ast }=1,x,z)}{P(Y_{1i}^{\ast }=1|Y_i^{\ast }=1,x,z)P(Y_{2i}^{\ast }=1|Y_i^{\ast }=1,x,z)}.}\end{array}}\end{array}}$$  (2)

The right-hand side of Equation (2), which reflects capture–recapture-derived ascertainment probabilities, depends solely on probabilities within the subpopulation with $Y_i^{\ast }=1$ (see Appendix Equations (A1–A3)). Consequently, it can be estimated from observable data. Substituting the right-hand side of Equation (2) into Equation (1), we have that

$${\displaystyle \begin{array}{l}{\displaystyle \begin{array}{lll}& {\displaystyle P(Y_i\left(x\right)=1)=}& {\displaystyle {\displaystyle \sum _z}\frac{P(Y_i^{\ast }=1,x|z)P\left(z\right)}{\frac{P(Y_{1i}^{\ast }=1,Y_{2i}^{\ast }=1|Y_i^{\ast }=1,x,z)}{P(Y_{1i}^{\ast }=1|Y_i^{\ast }=1,x,z)P(Y_{2i}^{\ast }=1|Y_i^{\ast }=1,x,z)}P\left(x|z\right)}.}\end{array}}\end{array}}$$  (3)

This formulation allows us to apply APW using capture–recapture methods, providing a practical solution for handling under-ascertainment while estimating causal effects.

Estimation

Equation (3) provides an avenue for estimation using a plug-in estimator. In the Appendix (Equation (A5)), we also derive the following estimator to simplify estimation with individual-level data:

$${\displaystyle \begin{array}{l}{\displaystyle \begin{array}{lll}& {\displaystyle {\displaystyle \widehat{P}}(Y_i\left(x\right)=1)=}& {\displaystyle \frac{1}{N}{\displaystyle \underset{i=1}{{\displaystyle \sum ^N}}}\frac{I(Y_i^{\ast }=1,X_i=x)}{\frac{{\displaystyle \widehat{P}}(Y_{1i}^{\ast }=1,Y_{2i}^{\ast }=1|Y_i^{\ast }=1,x_i,z_i)}{{\displaystyle \widehat{P}}(Y_{1i}^{\ast }=1|Y_i^{\ast }=1,x_i,z_i){\displaystyle \widehat{P}}(Y_{2i}^{\ast }=1|Y_i^{\ast }=1,x_i,z_i)}{\displaystyle \widehat{P}}\left(x|z_i\right)}.}\end{array}}\end{array}}$$  (4)

This estimation approach simplifies the process by avoiding the need to parametrically estimate the numerator. In Equation (4), $N$ represents the size of the study sample, and $I(Y_i^{\ast }=1,X_i=x)$ is an indicator function coded as 1 for individuals with observed $Y_i^{\ast }=1$ and whose observed exposure $X_i$ is equal to x, and 0 otherwise. The remaining terms represent predicted probabilities for each individual i, which can be obtained either through nonparametric methods or parametric techniques, such as logistic regressions. Naturally, the consistency of the estimator hinges on these models being correctly specified.

eAppendix 1; http://links.lww.com/EDE/C109 contains a simulation validating the estimator’s performance when all assumptions are met. The simulation also verifies that percentile-based bootstrap confidence intervals, derived from resampling the full size, $N$ study sample with replacement, can be used to quantify the uncertainty of APW estimates (eAppendix 1; http://links.lww.com/EDE/C109).

Numerical Example

To aid intuition about the method, we first present a hypothetical example focusing on the association between seatbelt use and injury risk in car crashes. In eAppendix 2; http://links.lww.com/EDE/C109, we also provide code to generate similar data and show how the method can be applied with individual-level observations.

Our hypothetical data follows the causal model in the Figure. The observed outcome data, $Y_i^{\ast }$, are gathered from two sources: police records, $Y_{1i}^{\ast }$, and hospital admissions, $Y_{2i}^{\ast }$. The unit of observation is car crash events, where injury outcomes are imperfectly ascertained. In our data-generating process, confounding bias arises from drunk driving ($Z_i$), which is a common cause affecting both seatbelt use ($X_i$) and the true injury outcome ($Y_i$). Under-ascertainment bias occurs because both $X_i$ and $Z_i$ impact ascertainment in the two data sources. Specifically, seatbelt use affects injury likelihood and severity, which, in turn, influence ascertainment in both police and hospital records. Additionally, drunk driving may affect the likelihood of police ascertainment of injury outcomes for all involved parties, as well as influence injury and crash severity. Consequently, under-ascertainment varies across the four strata defined by $X_i$ and $Z_i$.

FIGURE.

Directed acyclic graph depicting the causal model used to generate the hypothetical example data. In our hypothetical example, Y stands for car crash injury, X is seatbelt use, and Z is drunk driving. Our exposure of interest is X and Z is a confounding variable (a common cause of X and Y). The true Y is unobserved; instead, we observe under-ascertained outcomes from two sources, represented by $Y_1^{\ast }$ (police records) and $Y_2^{\ast }$ (hospital admissions data). Seatbelt use and drunk driving (X and Z) each affect the probability of ascertainment in both sources, as depicted by the arrows from these variables into $Y_1^{\ast }$ and $Y_2^{\ast }$.

Table 1 contains conditional probabilities based on a scenario such as the one outlined above. Setting $P(Z_i=1)$ to 0.5 for simplicity, the potential outcome probabilities $P(Y\left(1\right)=1)$ and $P(Y\left(0\right)=1)$ in the population are 0.379 and 0.621, respectively. The true causal RD is therefore −0.242, and the causal RR is 0.61. Estimating these effects using the observed $Y_i^{\ast }$, we instead get −0.099 for the RD and 0.76 for RR, which are both clearly biased. These estimates are already adjusted for confounding by $Z_i$, so the bias is solely related to differential ascertainment.

TABLE 1.

Conditional Probabilities From a Hypothetical Register-based Study Suffering From Outcome Under-ascertainment With Imperfect Outcome Data From Two Sources

	Variable Values
Conditional Probability	X = 1, Z = 1	X = 1, Z = 0	X = 0, Z = 1	X = 0, Z = 0
Outcomes
$P(Y_i\left(x\right)=1x,z)$a,b	0.449	0.309	0.691	0.550
$P(Y_i^{\ast }=1x,z)$	0.411	0.215	0.552	0.272
$P(Y_i^{\ast }=1,xz)$	0.256	0.082	0.208	0.169
Exposure propensities
$P(xz)$	0.623	0.379	0.377	0.621
Ascertainment
$P(Y_i^{\ast }=1Y_i=1,x,z)$a,c	0.917	0.698	0.797	0.494
$P(Y_{1i}^{\ast }=1Y_i^{\ast }=1,x,z)$	0.751	0.648	0.688	0.627
$P(Y_{2i}^{\ast }=1Y_i^{\ast }=1,x,z)$	0.799	0.643	0.689	0.539
$P(Y_{1i}^{\ast }=1,Y_{2i}^{\ast }=1Y_i^{\ast }=1,x,z)$	0.550	0.291	0.378	0.167

^aThese conditional probabilities (and their sample analogues) are presumed to be unobservable in our data setup but are displayed here for reference.

^b$P(Y\left(x\right)=1)=\sum _{z}P(Y_i\left(x\right)=1x,z)P\left(z\right)$ gives the marginal potential outcome probabilities. We apply this formula to compute the true RD and RR and use the same method on $P(Y_i^{\ast }=1x,z)$ to obtain the biased RD and RR presented in the text.

^cTrue ascertainment probabilities. These are unobservable but estimable from observable data by capture–recapture methods using Equation (2). For instance, plugging the three observable ascertainment probabilities from the left-most column into the equation gives us 0.550/(0.751 × 0.799) ≈ 0.917, which is equivalent to the true ascertainment probability.

To address the remaining bias, we first apply Equation (3) to estimate $P(Y_i\left(1\right)=1)$:

$${\displaystyle \begin{array}{l}{\displaystyle \begin{array}{llll}& {\displaystyle P(Y_i\left(1\right)=1)=}& {\displaystyle {\displaystyle \sum _z}\frac{P(Y_i^{\ast }=1,X_i=1|z)P\left(z\right)}{\frac{P(Y_{1i}^{\ast }=1,Y_{2i}^{\ast }=1|Y_i^{\ast }=1,X_i=1,z)}{P(Y_{1i}^{\ast }=1|Y_i^{\ast }=1,X_i=1,z)P(Y_{2i}^{\ast }=1|Y_i^{\ast }=1,X_i=1,z)}P(X_i=1|z)}}& {\displaystyle =\frac{0.256\times 0.5}{\frac{0.550}{0.751\times 0.799}\times 0.623}+\frac{0.082\times (1-0.5)}{\frac{0.291}{0.648\times 0.643}\times 0.379}\approx \mathrm{0.379.}}\end{array}}\end{array}}$$ (5)

The above expressions demonstrate how the observable data in Table 1 can be used to estimate the potential outcome probabilities. Applying the same steps for $P(Y_i\left(0\right)=1)$, we get $P(Y_i\left(0\right)=1)\approx \mathrm{0.621.}$ From these estimates, we obtain an RD of −0.242 and an RR of 0.61, which correspond to the true effects.

Empirical Example

In this section, we apply our approach to real-world data from the Swedish COVID-19 Investigation for Future Insights, a Population Epidemiology Approach using Register Linkage (SCIFI-PEARL) project.²⁹ The SCIFI-PEARL project is a comprehensive register-based study covering COVID-19-related outcomes, disease histories, and sociodemographic characteristics of the entire Swedish population. The project benefits from robust national register infrastructures and the ability to link data between registers using personal identification numbers with almost perfect accuracy.^30,31 In this example, we focus on investigating variations in the propensity to get tested for the severe acute respiratory syndrome coronavirus 2 virus across different population groups. Such variations can have implications for control strategies (e.g., contact tracing) as well as for interpreting endpoints that rely on individual’s willingness to get tested, such as confirmed cases.³²

The SCIFI-PEARL project was approved by the Swedish Ethical Review Authority (diary number 2020-01800).

Data Sources and Factors Assumed to Cause Under-ascertainment of Tests

Estimating testing propensities in Sweden is a challenge due to the lack of a nationally complete register on tests actually conducted that are linkable to population registers. In our example, we instead rely on data from two imperfect sources linked to the SCIFI-PEARL database: a complete register of all positive polymerase chain reaction (PCR) tests reported to the national infectious disease reporting system, SmiNet,³³ and tests recorded in a database at Sahlgrenska University Hospital, the largest hospital in Sweden’s second most populated region (Västra Götaland).

Laboratories where PCR tests are analyzed are required by Swedish law to report all positive PCR tests to SmiNet. However, the database lacks information on negative and invalid tests and, therefore, provides an incomplete account of all tests that were conducted in the country. This means that ascertainment of tests in this data source is driven entirely by factors that influence test-positivity (i.e., the probability that a performed test is positive).

Our second data source encompasses all PCR tests, both negative and invalid, from Sahlgrenska University Hospital in Gothenburg, Sweden’s second-largest city. The inclusion of tests in this database hinges on the test’s analysis location, largely determined by the health care provider’s type and location. Sweden’s health care, funded by taxes and divided into 21 regions, allows both regional and contracted private providers to offer services. In Västra Götaland, about half of the primary health care centers are privately operated. The Sahlgrenska database logs tests from the university hospital, two neighboring hospitals in Kungälv and Alingsås, and an external lab serving regional outpatient care facilities. However, it lacks comprehensive data on tests from private providers, which often use a nonincluded lab. We estimate that the Sahlgrenska database holds roughly half of the region’s PCR tests, as about half of the positive tests we observe in SmiNet in the Västra Götaland population are also present in the Sahlgrenska database.

We argue that the observed tests from these two sources fit our definition of under-ascertainment for two reasons: (1) observing a PCR test in either source implies that the individual was indeed tested for COVID-19 ($Y_i^{\ast }=1$ implies $Y_i=1$), and (2) the absence of tests in either source does not necessarily mean that the individual was not tested, as these sources do not encompass all tests conducted in the region ($Y_i^{\ast }=0$ does not necessarily imply $Y_i=0$).

To ensure the validity of the capture–recapture-based ascertainment adjustments, we must consider factors affecting ascertainment of tests in both data sources. Translating Assumption 1 into our context implies that once a test is conducted ($Y_i=1$), it must be as good as random whether a positive test ends up in the Sahlgrenska database or not, conditional on the exposure and observed covariates. In our case, this means that we must consider factors influencing both test-positivity and an individual’s choice of testing location (particularly, their choice of private or public health care providers). Existing research supports the hypothesis that proximity to the health care center more strongly influences provider choice in Sweden than health care quality indicators.³⁴ Hence, one might assume that area of residence or work is the primary driver of this choice. There were also notable spatial variations in infection rates,³⁵ which may affect test-positivity rates. We therefore suspect that geographic factors may be important to adjust for, because they may influence test ascertainment in both data sources. More broadly, sociodemographic factors may also impact provider choice, infection risk, and individual vulnerability, so we will want to adjust for those as well.

Application of Ascertainment Probability Weighting

We perform the APW analysis following the steps outlined in the Box. Our target outcome, $Y_i$, is defined as “tested at least once between 1 July and 26 December 2020.” We selected this period because it reflects a time when community testing was available, but largely before COVID-19 antigen self-test kits were commonly used and prior to the Swedish vaccination campaign (which started 27 December 2020). During this time, PCR testing therefore played an essential role in the public’s response to the pandemic. In our dataset, we code any individual with an observed test in one of our two data sources during this period as $Y_i^{\ast }=1$, and $Y_i^{\ast }=0$ otherwise.

Our example compares testing propensities between two exposure groups, health care workers ($X_i=1$) and other essential occupations ($X_i=0$; a composite group including teachers, social care workers, service sector workers, postal/delivery, transport services workers, police/security, and cleaners). Our occupation data come from the Longitudinal Integrated Database for Health Insurance and Labor Market Studies (LISA), and we used four-digit codes from the Swedish Standard Occupational Classification (SSYK2012) to classify occupations (see eTable1; http://links.lww.com/EDE/C109 for details). Our study population comprises the entire working-age population (20–64 years) working in essential occupations and residing in Västra Götaland at the end of 2019 (N = 261,314).

We use logistic regression to estimate the exposure propensity score and to compute the three “source ascertainment models” required to compute Equation (4) (see Box). To adjust for confounding and help satisfy the conditional source independence assumption (Assumption 1), we include a set of covariates ($Z_i$) in each model. Based on the rationale in the previous section, we include adjustment for age (5-year groups), sex, health care worker (yes/no), disposable income in national quartiles, country of birth (Sweden or abroad; where abroad is classified into low income, lower-middle income, middle-high income, and high income countries based on the World Bank’s classification in 2020), educational attainment (primary, secondary, or tertiary), marital status (married, unmarried), number of children living at home (0, 1, 2, 3, 4+), household type (living alone, single parent, or living with partner), if the person scored one or higher on the Charlson Comorbidity Index before the pandemic, and municipality of residence fixed effects (49 in the region). Each variable is based on data from Swedish registers linked to the SCIFI-PEARL database; details and data sources are provided in eTable 2; http://links.lww.com/EDE/C109.

RESULTS

Compared to ignoring the under-ascertainment of tests in our data, our APW-based estimates show substantially higher testing propensities in health care workers and other essential occupations (Table 2).

TABLE 2.

Estimates of the Association Between Health Care Work and the Propensity to Get Tested for COVID-19 in the Working-age Population (20–64 Years) of Västra Götaland, Sweden (N = 261,314) Working in Essential Occupations, With and Without APW Using Testing Data From Two Incomplete Data Sources

	Testing Propensity, %a		Exposure Effect Measure
Estimate	Health Care Workers (n = 76,033)	Other Essential Workers (n = 185,281)	Difference	Ratio
Unadjusted	46.1 (45.7, 46.4)	23.2 (23.0, 23.4)	22.9 (22.5, 23.3)	1.99 (1.96, 2.01)
IPWb	45.8 (45.4, 46.2)	23.8 (23.6, 24.0)	22.0 (21.6, 22.5)	1.92 (1.90, 1.95)
APWb	60.7 (59.9, 62.3)	38.5 (37.9, 39.6)	22.2 (21.0, 23.6)	1.58 (1.54, 1.62)

Confidence intervals (95% CIs), which are presented in parentheses, are based on a nonparametric percentile bootstrap with 1000 resamples (resampling the entire study sample with replacement).

^aBinary outcome reflecting if the individual underwent PCR testing for COVID-19 at least once between 1 July and 26 December 2020.

^bConfounding adjustment includes sociodemographic factors (see text for details). Under-ascertainment adjustment includes the same factors and the exposure variable. IPW estimate includes only confounding adjustment, while APW includes confounding and under-ascertainment adjustment.

While absolute differences in testing propensities between the two occupational groups are relatively unaffected in this case, applying the APW estimator substantially impacts the estimated relative differences, that is, testing propensity ratios (Table 2). Specifically, the unadjusted testing propensity ratio estimate between health care workers and other essential occupations changes from 1.99 to 1.92 with conventional IPW adjustment for confounding, and further reduces to 1.58 with APW adjustment for under-ascertainment. Additionally, the 95% confidence intervals for the IPW and APW estimates do not overlap.

These findings emphasize the importance of considering potential under-ascertainment in register-based studies, as it may impact differences on an absolute, or as in this example, relative scale. Ultimately, all estimates, including the APW-adjusted estimate, suggest that health care workers were more likely to be tested for COVID-19 than workers in other essential occupations.

DISCUSSION

We have introduced a novel framework to adjust for both under-ascertainment and confounding, addressing a common challenge, especially for studies investigating sensitive outcomes, such as drug use, and studies relying on administrative data from resource-limited settings. Our contribution relates to misclassification,^3,36 which is a well-established problem in epidemiology that has gained renewed attention in recent years.^15–25 Importantly, we extend IPW methods for causal inference to account for incomplete ascertainment of outcomes through capture–recapture estimation, which to our knowledge has not previously been explored.

Our APW estimator corrects for under-ascertainment using two imperfect data sources. While existing methods can be used to account for more general classes of misclassification,^15–25 including settings with false positives, they require perfect validation data, which is not always available. Unlike sensitivity or simulation-based methods dependent on external data or assumptions,^37,38 methods that rely on internal data, including APW, ensure that adjustments are relevant to the study sample.^15,19 Hence, this method provides a contextually relevant way to adjust for under-ascertainment bias without requiring perfect validation data.

Every analytical method, including APW, is subject to limitations and requires a thorough evaluation of its assumptions. The method’s exposure propensity aspect is subject to the same limitations as traditional propensity score weighting, including that all confounding must be adjustable by observable factors and the need for sufficient overlap in covariate distributions between exposure groups (see, e.g., Austin and Stuart²⁸).

The additional considerations required are instead closely related to considerations for capture–recapture estimation. Like other capture–recapture techniques, APW assumes a minimal rate of false positives, as their presence would lead to overestimated outcome probabilities.³⁹ The extent of this bias depends on the false positive rate and the difference in this rate across exposure groups. Register validation studies often highlight under-ascertainment as a primary concern in administrative data compared to false positives. Systematic reviews suggest that specificities usually range from 90% to 100% in administrative registers,^40–46 suggesting that the false positive rate may be reasonably low in many practical contexts. Nonetheless, it is still important to carefully evaluate this assumption in each application. In our empirical example, for instance, there is no complex diagnostic process involved. We therefore deem it is safe to assume that individuals observed with a positive test in SmiNet or with a test in the Sahlgrenska database were indeed tested (unless there are mistakes in data linkage or recording).

On the other hand, if a subset of true positives systematically evades detection in both data sources, capture–recapture methods may underestimate the true outcome probabilities. For instance, if both data sources exclusively include hospital admissions data, generalizing findings to nonhospitalized cases would be inappropriate. However, in such settings, inferences can still be drawn about ascertainable cases, though caution is required when making conclusions about subpopulations or types of outcomes not ascertained in any data source.

The conditional source independence assumption (Assumption 1) should be critically evaluated in every application. Similar to confounding adjustment, this can be done using contextual, empirical, and theoretical knowledge, possibly incorporated into causal diagrams.⁴⁷ To see how this can be done, note that Assumption 1 implies that adjustment variables should be selected so that they block any open path between $Y_1^{\ast }$ and $Y_2^{\ast }$, after conditioning on $Y_i=1$. For example, Figure suggests that $X_i$ and $Z_i$ must be included in the models predicting ascertainment probabilities to block all open paths and enable valid estimation. For reference, Brenner⁴⁸ also provides general guidance on the effects of violations of Assumption 1, namely that positive ascertainment covariance leads to an underestimation of outcome probabilities, whereas negative ascertainment covariance leads to overestimation. Whether the estimated exposure effects are also biased, depends on the direction and magnitude of these covariances in each exposure group and the degree to which they differ. A similar concern arises when one source’s ascertainment influences the other (e.g., depicted by $Y_1^{\ast }\to Y_2^{\ast }$ in a causal diagram), which can be difficult to adjust away.⁴⁹ In our example, all laboratories, regardless of their affiliation with the Sahlgrenska database, must report positive test results to SmiNet. This arrangement minimizes additional ascertainment pathways, which should only occur through test-positivity. However, Assumption 1 could still be compromised if systematic reporting biases exist, for example, if Sahlgrenska database labs are more prone to report positive tests to SmiNet.

Finally, our paper leaves room for several possible refinements to the APW framework, such as accommodating misclassified exposures, integrating with design-based sampling techniques to handle false positives,⁵⁰ and expanding it to multisource capture–recapture estimation, which may offer additional possibilities to account for source dependence.¹⁰ Further analysis of the statistical properties of alternative estimators for the method, including finite sample performance under different scenarios, would also be of interest.

PROPOSED STEP-BY-STEP PROCESS TO COMPUTE THE CAPTURE–RECAPTURE-BASED ASCERTAINMENT PROBABILITY WEIGHTING ESTIMATOR.

Step 1. Define the target outcome ($Y_i$) and its under-ascertained counterpart ($Y_i^{\ast }$).

Step 2. Define the exposure of interest ($X_i$) and estimate a model for the propensity score for exposure, $P(x|z)$, controlling for observed confounders ($Z_i$). Predict the propensity score for exposure level $x$ (e.g., $x=1$) for all individuals in the data.

Step 3. Using data only from the subset of individuals with $Y_i^{\ast }=1$, estimate one model each for the following conditional probabilities, controlling for the exposure ($X_i$) and covariates ($Z_i$) assumed to influence ascertainment in both data sources:

1. the probability of observing the outcome in source 1 ($Y_{1i}^{\ast })$,

2. the probability of observing the outcome in source 2 ($Y_{2i}^{\ast })$,

3. the probability of observing the outcome in both sources ($Y_{1i}^{\ast }$ and $Y_{2i}^{\ast }$).

Step 4. Use the predicted probabilities from the models in Step 3 to estimate ascertainment probabilities for all individuals in the data (including those with $Y_i^{\ast }=0$) using Equation (2).

Step 5. Use Equation (4) to estimate the potential outcome probability for exposure level x ($P(Y\left(x\right)=1$), for example, $x=1$ (the “exposed group”).

Step 6. Repeat Step 5 for exposure level $x^{\prime }$, for example, $x=0$ (the “unexposed group”).

Step 7. Use the estimated potential outcome probabilities to compute effect estimates on a desired scale (e.g., RD, RR).

Step 8. Repeat Steps 2–7 on M (e.g., 1,000) bootstrap resamples of the data (resample the entire study sample with replacement) to compute percentile-bootstrap confidence intervals.

Supplementary Material

ede-35-340-s001.pdf ^{(331KB, pdf)}

APPENDIX

First, we give the proof of Equation (2). For any source j, including the product of multiple sources, we have that

$${\displaystyle \begin{array}{l}{\displaystyle P(Y_{ji}^{\ast }=1|Y_i=1,x,z)}\\ {\displaystyle =P(Y_{ji}^{\ast }=1,Y_i^{\ast }=1|Y_i=1,x,z)}\\ {\displaystyle =P(Y_{ji}^{\ast }=1|Y_i^{\ast }=1,Y_i=1,x,z)}\\ {\displaystyle \times P(Y_i^{\ast }=1|Y_i=1,x,z)}\\ {\displaystyle =P(Y_{ji}^{\ast }=1|Y_i^{\ast }=1,x,z)P(Y_i^{\ast }=1|Y_i=1,x,z),}\end{array}}$$ (A1)

where the first equality follows from the fact that $Y_{ji}^{\ast }=1$ implies $Y_i^{\ast }=1$, the second from the product rule, and the third because $Y_i^{\ast }=1$ implies $Y_i=1$.

By Assumption 1,

$${\displaystyle \begin{array}{l}{\displaystyle P(Y_{1i}^{\ast }=1,Y_{2i}^{\ast }=1|Y_i=1,x,z)=}\\ {\displaystyle P(Y_{1i}^{\ast }=1|Y_i=1,x,z)P(Y_{2i}^{\ast }=1|Y_i=1,x,z).}\end{array}}$$ (A2)

It follows from these expressions that

$${\displaystyle \begin{array}{l}{\displaystyle P(Y_{1i}^{\ast }=1,Y_{2i}^{\ast }=1|Y_i^{\ast }=1,x,z)}\\ {\displaystyle \times P(Y_i^{\ast }=1|Y_i=1,x,z)}\\ {\displaystyle =P(Y_{1i}^{\ast }=1,Y_{2i}^{\ast }=1|Y_i^{\ast }=1,Y_i=1,x,z)}\\ {\displaystyle \times P(Y_i^{\ast }=1|Y_i=1,x,z)}\\ {\displaystyle =P(Y_{1i}^{\ast }=1,Y_{2i}^{\ast }=1,Y_i^{\ast }=1|Y_i=1,x,z)}\\ {\displaystyle =P(Y_{1i}^{\ast }=1,Y_{2i}^{\ast }=1|Y_i=1,x,z)}\\ {\displaystyle =P(Y_{1i}^{\ast }=1|Y_i=1,x,z)}\\ {\displaystyle \times P(Y_{2i}^{\ast }=1|Y_i=1,x,z)}\\ {\displaystyle =\left\{P(Y_{1i}^{\ast }=1|Y_i^{\ast }=1,x,z)P(Y_i^{\ast }=1|Y_i=1,x,z)\right\}}\\ {\displaystyle \times \left\{P(Y_{2i}^{\ast }=1|Y_i^{\ast }=1,x,z)P(Y_i^{\ast }=1|Y_i=1,x,z)\right\}}\\ {\displaystyle =P(Y_{1i}^{\ast }=1|Y_i^{\ast }=1,x,z)P(Y_{2i}^{\ast }=1|Y_i^{\ast }=1,x,z)}\\ {\displaystyle \times P{(Y_i^{\ast }=1|Y_i=1,x,z)}^2.}\end{array}}$$ (A3)

Canceling common factors by dividing both sides of (A3) by $P(Y_i^{\ast }=1|Y_i=1,x,z)$ and then solving for the remaining $P(Y_i^{\ast }=1|Y_i=1,x,z)$, which requires Assumption 2, gives Equation (2).

Next, we give the proof of Equations (1) and (3). For discrete $Z_i$, we have that

$${\displaystyle \begin{array}{l}{\displaystyle P(Y_i\left(x\right)=1)}\\ {\displaystyle ={\displaystyle \sum _z}P(Y_i\left(x\right)=1|z)P\left(z\right)}\\ {\displaystyle ={\displaystyle \sum _z}P(Y_i\left(x\right)=1|x,z)P\left(z\right)}\\ {\displaystyle ={\displaystyle \sum _z}P(Y_i=1|x,z)P\left(z\right)}\end{array}}$$

$${\displaystyle \begin{array}{l}{\displaystyle ={\displaystyle \sum _z}\frac{P(Y_i^{\ast }=1|Y_i=1,x,z)P(Y_i=1|x,z)P\left(x|z\right)P\left(z\right)}{P(Y_i^{\ast }=1|Y_i=1,x,z)P\left(x|z\right)}}\\ {\displaystyle ={\displaystyle \sum _z}\frac{P(Y_i^{\ast }=1,Y_i=1,x|z)P\left(z\right)}{P(Y_i^{\ast }=1|Y_i=1,x,z)P\left(x|z\right)}}\end{array}}$$

$${\displaystyle \begin{array}{l}{\displaystyle ={\displaystyle \sum _z}\frac{P(Y_i^{\ast }=1,x|z)P\left(z\right)}{P(Y_i^{\ast }=1|Y_i=1,x,z)P\left(x|z\right)}}\\ {\displaystyle ={\displaystyle \sum _z}\frac{P(Y_i^{\ast }=1,x|z)P\left(z\right)}{\frac{P(Y_{1i}^{\ast }=1,Y_{2i}^{\ast }=1|Y_i^{\ast }=1,x,z)}{P(Y_{1i}^{\ast }=1|Y_i^{\ast }=1,x,z)P(Y_{2i}^{\ast }=1|Y_i^{\ast }=1,x,z)}P\left(x|z\right)}}\end{array}}$$

$${\displaystyle \begin{array}{l}{\displaystyle \equiv {\displaystyle \sum _z}\frac{P(Y_i^{\ast }=1,x|z)P\left(z\right)}{a(x,z)\times e_x\left(z\right)},}\end{array}}$$ (A4)

where the first equality follows from the law of total probability; the second from conditional independence of the exposure and potential outcomes; the third from counterfactual consistency; the fourth by introducing common terms to the numerator and denominator, which requires exposure and ascertainment positivity; the fifth by the chain rule; the sixth because $Y_i^{\ast }=1$ implies $Y_i=1$ by the definition of $Y_i^{\ast }$; and the seventh by Equation (2). The final expression redefines terms to simplify notation in the derivations below. For continuous $Z_i$, the summation in (A4) can be replaced with integration.

While the final expression in (A4) could in principle be estimated using a plug-in estimator, we rewrite the estimator to avoid having to estimate the numerator parametrically:

$${\displaystyle \begin{array}{l}{\displaystyle {\displaystyle \widehat{P}}(Y_i\left(x\right)=1)}\\ {\displaystyle ={\displaystyle \sum _z}\frac{{\displaystyle \widehat{P}}(Y_i^{\ast }=1,x|z){\displaystyle \widehat{P}}\left(z\right)}{{\displaystyle \widehat{a}}(x,z)\times {{\displaystyle \widehat{e}}}_x\left(z\right)}}\\ {\displaystyle ={\displaystyle \sum _z}\frac{{\displaystyle \widehat{P}}(Y_i^{\ast }=1,x|z)N_z/N}{{\displaystyle \widehat{a}}(x,z)\times {{\displaystyle \widehat{e}}}_x\left(z\right)}}\end{array}}$$

$${\displaystyle \begin{array}{l}{\displaystyle =\frac{1}{N}{\displaystyle \sum _z}\frac{{\displaystyle \widehat{P}}(Y_i^{\ast }=1,x|z)N_z}{{\displaystyle \widehat{a}}(x,z)\times {{\displaystyle \widehat{e}}}_x\left(z\right)}}\\ {\displaystyle =\frac{1}{N}{\displaystyle \sum _z}\frac{{\displaystyle \sum _{i:z_i=z}}I(Y_i^{\ast }=1,X_i=x)}{{\displaystyle \widehat{a}}(x,z)\times {{\displaystyle \widehat{e}}}_x\left(z\right)}}\\ {\displaystyle =\frac{1}{N}{\displaystyle \sum _z}{\displaystyle \sum _{i:z_i=z}}\frac{I(Y_i^{\ast }=1,X_i=x)}{{\displaystyle \widehat{a}}(x,z)\times {{\displaystyle \widehat{e}}}_x\left(z\right)}}\end{array}}$$

$${\displaystyle \begin{array}{l}{\displaystyle =\frac{1}{N}{\displaystyle \underset{i=1}{{\displaystyle \sum ^N}}}\frac{I(Y_i^{\ast }=1,X_i=x)}{{\displaystyle \widehat{a}}(x,z_i)\times {{\displaystyle \widehat{e}}}_x\left(z_i\right)}=\frac{1}{N}{\displaystyle \underset{i=1}{{\displaystyle \sum ^N}}}\frac{I(Y_i^{\ast }=1,X_i=x)}{{\displaystyle \widehat{a}}(x_i,z_i)\times {{\displaystyle \widehat{e}}}_x\left(z_i\right)},}\end{array}}$$  (A5)

where $N_z$ is the z-specific stratum size. In the final expression, we replace $\widehat{a}(x,z_i)$ with $\widehat{a}(x_i,z_i)$. While our target quantity is defined for a fixed $x$, the observed $x_i$ can be used with equivalent results when estimating $a(x,z)$ with the expressions in (A5) that include $I(Y_i^{\ast }=1,X_i=x)$, because the terms will always evaluate to zero for individuals with $x_i\ne x$. Thus, one can either predict $\widehat{a}(x,z_i)$ for each $x$ separately or, more conveniently, use $\widehat{a}(x_i,z_i)$ for all $x$.