Adjusted logistic propensity weighting methods for population inference using nonprobability volunteer-based epidemiologic cohorts

Abstract

Many epidemiologic studies forgo probability sampling and turn to nonprobability volunteer-based samples because of cost, response burden, and invasiveness of biological samples. However, finite population (FP) inference is difficult to make from the nonprobability sample due to the lack of population representativeness. Aiming for making inferences at the population level using nonprobability samples, various inverse propensity score weighting methods have been studied with the propensity defined by the participation rate of population units in the nonprobability sample. In this article, we propose an adjusted logistic propensity weighting (ALP) method to estimate the participation rates for nonprobability sample units. The proposed ALP method is easy to implement by ready-to-use software while producing approximately unbiased estimators for population quantities regardless of the nonprobability sample rate. The efficiency of the ALP estimator can be further improved by scaling the survey sample weights in propensity estimation. Taylor linearization variance estimators are proposed for ALP estimators of FP means that account for all sources of variability. The proposed ALP methods are evaluated numerically via simulation studies and empirically using the naïve unweighted National Health and Nutrition Examination Survey III sample, while taking the 1997 National Health Interview Survey as the reference, to estimate the 15-year mortality rates.

1 |. INTRODUCTION

In the big data era, assembling volunteer-based epidemiologic cohorts within integrated healthcare systems that have electronic health records and a large preexisting base of volunteers are increasingly popular due to their cost-and-time efficiency, such as the UK Biobank in the UK National Health Service.¹ However, samples of volunteer-based cohorts are not randomly selected from the underlying finite target population, and therefore cannot well represent the target population. As a result, the naïve sample estimates obtained from the cohort can be biased for the finite population (FP) quantities. For example, the estimated all-cause mortality rate in the UK Biobank was only half that of the UK population,² and the Biobank is not representative of the UK population with regard to many sociodemographic, physical, lifestyle, and health-related characteristics.

Aiming for making inferences at the population level using nonprobability samples, various propensity-score weighting, and matching methods have been proposed to improve the population representativeness of nonprobability samples, by using probability-based survey samples as external references in survey research.^3–6

Inverse propensity score weighting (IPSW) methods have been studied with the propensity defined by the participation rate of population units in the nonprobability sample. We review two methods—both assume that the units in the nonprobability sample are observed according to some random, but unknown, mechanism. Because that mechanism is unknown, the inclusion probability of each unit must be estimated. As described in Section 2, all methods are based on estimating a pseudo log-likelihood, although the methods differ in their details. Valliant and Dever⁷ estimated participation rates by fitting a logistic regression model to the combined nonprobability sample and a reference, probability sample. Sample weights for the probability sample were scaled by a constant so that the scaled probability sample was assumed to represent the complement of the nonprobability sample. Each unit in the nonprobability sample was assigned a weight of one. This results in the sum of the scaled weights in the combined probability plus nonprobability sample being an estimate of the population size. This method will be referred to as the rescaled design weight (RDW) method. The participation rate for each nonprobability sample unit was estimated by the inverse of the estimated inclusion (or participation) probability.

The RDW estimator is biased especially when the participation rate of the nonprobability sample is large, as noted by Chen et al.⁴ As a remedy, Chen et al⁴ estimated the participation rate by manipulating the log-likelihood estimating equation in a somewhat different way. The resulting estimator, denoted by CLW, is consistent and approximately unbiased regardless of the magnitude of participation rates. Compared with the CLW method, which requires special programming, the RDW method has the advantage of easy implementation by ready-to-use software such as R, Stata, or SAS. Survey practitioners can simply fit a logistic regression model with scaled survey weights in the probability sample to obtain the estimated participation rates.

In this article, we propose an adjusted logistic propensity weighting (ALP) method to estimate the participation rates for nonprobability sample units. Like the CLW, the proposed ALP method relaxes the assumptions required by the RDW method,^7,8 by formulating the method in an innovative way. As in the RDW method, the proposed ALP method retains the advantage of easy implementation by fitting a propensity model with survey weights in ready-to-use software. Taylor linearization (TL) variance estimators are proposed for ALP estimates that account for variability due to differential pseudo-weights in the nonprobability sample, complex survey design of the reference probability survey, as well as the estimation of the propensity scores. The variance of the proposed estimator has the order of the inverse of the nonprobability sample size (as shown in Appendix C). Moreover, under the logistic propensity model, the ALP method can flexibly scale the probability sample weights for propensity estimation to further improve efficiency. In summary, the contributions of the proposed ALP method include (1) easy implementation with ready-to-use software, (2) high efficiency, as well as (3) the justification of a set of pseudo-estimating equation (14) that underly the straightforward implementation in survey software.

2 |. METHODS

2.1 |. Basic setting

Let FP = {1, ⋯ , N} represent the FP with size N. We are interested in estimating the FP mean $\mu =N^{-1}{\sum }_{i\in \text{FP}}{y}_i$. Suppose a volunteer-based nonprobability sample s_c of size n_c is selected from FP by a self-selection mechanism, with ${\delta }_i^{(c)}$ (= 1 if i ∈ s_c; 0 otherwise) denoting the indicator of s_c inclusion. The underlying participation rate of nonprobability sample for a FP unit is defined as

$${\pi }_i^{(c)}=P\left(i\in s_c\mid \text{FP}\right)=E_c\left\{{\delta }_i^{(c)}\mid y_i,{\mathit{x}}_i\right\},i\in \text{FP},$$

Where the expectation E_c is with respect to the nonprobability sample selection, and x_i is a vector of self-selection variables, that is, covariates related to the probability of inclusion in s_c. The corresponding implicit nonprobability sample weight is $w_i=1/{\pi }_i^{(c)}$ for i ∈ FP.

We consider the following assumptions for the nonprobability sample self-selection.

A1. The nonprobability sample selection is uncorrelated with the variable of interest given the covariates, that is, ${\pi }_i^{(c)}=E_c\left\{{\delta }_i^{(c)}\mid y_i,{\mathit{x}}_i\right\}=E_c\left\{{\delta }_i^{(c)}\mid {\mathit{x}}_i\right\}$ for i ∈ FP.

A2. All FP units have a positive participation rate, that is, ${\pi }_i^{(c)}>0$ for i ∈ FP.

A3. The indicators of participation in the nonprobability cohort are uncorrelated with each other given the self-selection variables, that is, $\mathrm{cov}\left({\delta }_i^{(c)},{\delta }_j^{(c)}\mid {\mathit{x}}_i,{\mathit{x}}_j\right)=0$ for i ≠ j.

An independent reference probability-based survey sample s_p of size n_p is randomly selected from FP. The sample inclusion indicator, selection probability, and the corresponding sample weights are defined by ${\delta }_i^{(p)}$(=1 if i ∈ s_p; 0 otherwise), ${\pi }_i^{(p)}=E_p({\delta }_i^{(p)}\mid {\mathit{x}}_i)$, and $d_i=1/{\pi }_i^{(p)}$, respectively, where E_p is with respect to the survey sample selection.

2.2 |. Existing logistic propensity weighting method

In this section, we first briefly introduce the existing RDW and CLW methods and discuss their pros and cons.

2.2.1 |. RDW method

Valliant and Dever^6,7 assumed a logistic regression model for the participation rates ${\pi }_i^{(c)}(\mathit{\gamma })$

$$\log \left\{\frac{{\pi }_i^{(c)}(\mathit{\gamma })}{1-{\pi }_i^{(c)}(\mathit{\gamma })}\right\}={\mathit{\gamma }}^T{\mathit{x}}_i,\text{for}i\in \text{FP},$$ (1)

where γ is a vector of unknown parameters, and x_i is a vector of covariates for i ∈ FP. To simplify the notation, we use ${\pi }_i^{(c)}$ below. They considered (implicitly) the population likelihood function of ${\pi }_i^{(c)}$ as

$$L(\mathit{\gamma })=\prod _{i\in \text{FP}}{\left\{{\pi }_i^{(c)}\right\}}^{{\delta }_i^{(c)}}{\left\{1-{\pi }_i^{(c)}\right\}}^{1-{\delta }_i^{(c)}}.$$ (2)

Then, the log-likelihood function can be written as

$$l(\mathit{\gamma })=\sum _{i\in \text{FP}}[{\delta }_i^{(\text{c})}\log {\pi }_i^{(\text{c})}+\{1-{\delta }_i^{(\text{c})}\}\log \{1-{\pi }_i^{(\text{c})}\}]=\sum _{i\in S_c}\log {\pi }_i^{(\text{c})}+\sum _{i\in \text{FP}-s_c}\log \{1-{\pi }_i^{(\text{c})}\},$$ (3)

where the set FP − s_c represents the FP units that are not self-selected into the nonprobability sample. Since FP − s_c is not available in practice, the pseudo-loglikelihood function was constructed to estimate l(γ) by

$${\tilde{l}}^{\text{RDW}}(\mathit{\gamma })=\sum _{i\in S_c}{w}_i^{*}\log {\pi }_i^{(c)}+\sum _{i\in s_p}{w}_i^{*}\log \{1-{\pi }_i^{(c)}\},$$ (4)

where $w_i^{*}=\{\begin{array}{ll}1,\hfill & \text{for}i\in s_c\hfill \\ d_i\left({\widehat{N}}_p-n_c\right)/{\widehat{N}}_p,\hfill & \text{for}i\in s_p\hfill \end{array}$, with ${\widehat{N}}_p={\sum }_{i\in s_p}{d}_i$ being the survey estimate of the target FP size N. This leads to the total of the scaled weights across the probability sample units being ${\sum }_{i\in s_p}{w}_i^{*}={\widehat{N}}_p-n_c$. The rationale for rescaling is to weight the survey sample to represent the complement of s_c in the FP, that is, the set FP − s_c. Under the logistic regression model, the nonprobability sample participation rate ${\pi }_i^{(c)}$ for i ∈ s_c can be estimated by fitting Model (1) to the combined sample of s_c and scaled-weighted s_p with scaled weights $w_i^{*}$, leading to the RDW estimates.

The RDW method has been shown to effectively reduce the bias of the naïve nonprobability sample estimates. However, the summand ${\sum }_{i\in \text{FP}-S_c}\log \left\{1-{\pi }_i^{(c)}\right\}$ in (3) is not a fixed FP total because units in the nonprobability sample s_c are treated as being randomly observed. This leads to a bias as shown below.

Comparing the expectation of the population log-likelihood function l(γ) in (3) and the expectation of the pseudo log-likelihood ${\tilde{l}}^{\text{RDW}}(\mathit{\gamma })$ in (4), and letting E(·) = E_cE_p(·) we have

$$\begin{gathered} E\{l(\mathit{\gamma })\}=\sum _{i\in FP}{\pi }_i^{(c)}\log {\pi }_i^{(c)}+\sum _{i\in FP}\{1-{\pi }_i^{(c)}\}\log \{1-{\pi }_i^{(c)}\},\text{and} \\ E\{{\tilde{l}}^{\text{RDW}}(\mathit{\gamma })\}=E_c{E}_p\{{\tilde{l}}^{\text{RDW}}(\mathit{\gamma })\}=E_c\left[\sum _{i\in FP}{\delta }_i^{(c)}\log {\pi }_i^{(c)}\right]+E_p\left[\sum _{i\in FP}{\delta }_i^{(p)}\cdot \frac{{\widehat{N}}_p-n_c}{{\widehat{N}}_p}{d}_i\log \{1-{\pi }_i^{(c)}\}\right] \\ \doteq \sum _{i\in FP}{\pi }_i^{(c)}\log {\pi }_i^{(c)}+\sum _{i\in FP}\{1-\frac{n_c}{N}\}\log \{1-{\pi }_i^{(c)}\} \end{gathered}$$

by assuming $E_p\left({\widehat{N}}_p\right)=N$. The difference of the two expectations, denoted by Δ_RDW, can be written as

$${\Delta }_{\text{RDW}}=E\{{\tilde{l}}^{\text{RDW}}(\mathit{\gamma })\}-E\{l(\mathit{\gamma })\}=\sum _{i\in \text{FP}}\left\{\frac{n_{\text{c}}}{N}-{\pi }_i^{(\text{c})}\right\}\log \{1-{\pi }_i^{(\text{c})}\}$$

which, in general, is nonzero. Accordingly, the nonprobability sample participation rates estimated by solving for γ in $\partial {\tilde{l}}^{\text{RDW}}(\mathit{\gamma })/\partial \mathit{\gamma }=0$ under Model (1) can be biased, unless either (i) the nonprobability sample units have small participation rates, that is, both n_c∕N and ${\pi }_i^{(c)}$ are close to 0 for all i ∈ FP, in which case $\log \left\{1-{\pi }_i^{(c)}\right\}\approx 0$, or (ii) all population units are equally likely to participate in the nonprobability sample, that is, ${\pi }_i^{(c)}\equiv n_c/N$. In many practical applications, (i) will hold. For example, suppose that n_c = 1000 and the US population age 18 and over is the target population. The population size is approximately 210 million, so that n_c∕N ≐ 5 × 10⁻⁶. If, instead, the population is for a small state like Wyoming where the 18+ population size is about 365000, then n_c∕N ≐ 0.0027. In both examples, with such small sampling fractions, all $\left\{{\pi }_i^{(c)},i\in s_c\right\}$ should be near zero also.

2.2.2 |. CLW method

Chen et al⁴ proposed another IPSW method using the same likelihood function L(γ) in (2), but rewriting the population log-likelihood as

$$l(\mathit{\gamma })=\sum _{i\in S_c}\log \frac{{\pi }_i^{(c)}}{1-{\pi }_i^{(c)}}+\sum _{i\in FP}\log \{1-{\pi }_i^{(c)}\}.$$ (5)

In contrast to the RDW method, CLW estimated the population total of $\log \left\{1-{\pi }_i^{(c)}\right\}$ by a weighted reference sample total and constructed the pseudo log-likelihood as

$${\tilde{l}}^{\text{CLW}}(\mathit{\gamma })=\sum _{i\in S_c}\log \frac{{\pi }_i^{(c)}}{1-{\pi }_i^{(c)}}+\sum _{i\in s_p}{d}_i\log \{1-{\pi }_i^{(c)}\}.$$ (6)

Under the same logistic regression model (1), the participation rate ${\pi }_i^{(c)}$ was estimated by solving the pseudo estimation equation

$$\tilde{S}(\mathit{\gamma })=\frac{1}{N}\left\{\sum _{i\in s_c}{\mathit{x}}_i-\sum _{i\in s_p}{d}_i{\pi }_i^{(c)}{\mathit{x}}_i\right\}=0,$$ (7)

derived from the pseudo log-likelihood (6). The resulting CLW weights are calculated as $\left\{w_i^{\text{CLW}}=1+{\exp }^{-1}\left({\widehat{\mathit{\gamma }}}^T{\mathit{x}}_i\right),i\in s_c\right\}$. Chen et al⁴ proved that the CLW estimator of the FP mean, ${\widehat{\mu }}^{\text{CLW}}={\left({\sum }_{i\in s_c}{w}_i^{\text{CLW}}\right)}^{-1}{\sum }_{i\in s_c}{w}_i^{\text{CLW}}{y}_i$, was design consistent when model (1) for the participation rates was correct.

In contrast to the RDW method, CLW does not require condition (i) or (ii) in the RDW method for unbiased estimation of participation rates $\left\{{\pi }_i^{(c)},i\in S_c\right\}$. In the following section, we propose an ALP method, which corrects the bias in the RDW method. The proposed ALP method provides consistent estimators of FP means and is as easy to implement as the RDW method.

2.3 |. ALP method

The ALP method also aims to estimate the cohort sample participation rates $\left\{{\pi }_i^{(c)},i\in S_c\right\}$ and use the inverse of estimated ${\pi }_i^{(c)}$ as the pseudo-weight for i ∈ s_c. As a computational device, we construct a pseudo-population of $s_c^{*}\cup FP$, where $s_c^{*}$ is a copy of s_c that has the same joint distributions of covariates x and outcome y with the original s_c. The number of units in $s_c^{*}\cup FP$ is n_c + N. In the union of $s_c^{*}\cup FP$, $s_c^{*}$ and s_c are treated as two different sets. We use R_i to indicate the membership of $s_c^{*}$ in $s_c^{*}\cup FP$(=1 if $i\in s_c^{*}$; 0 if i ∈ FP), and $p_i=P\left(R_i=1\right)=P\left(i\in s_c^{*}\mid s_c^{*}\cup \text{FP}\right)$. Instead of directly modeling ${\pi }_i^{(c)}$ as in the RDW and CLW methods, we model p_i as a function of ${\pi }_i^{(c)}$:

$$p_i=\frac{{\pi }_i^{(c)}}{1+{\pi }_i^{(c)}},\text{or}\text{equivalently,}{\pi }_i^{(c)}=\frac{p_i}{1-p_i}.$$ (8)

The relationship between p_i and ${\pi }_i^{(c)}$ follows because

$$\frac{p_i}{1-p_i}=\frac{P\left(i\in s_c^{*}\mid s_c^{*}\cup \text{FP}\right)}{P\left(i\in \text{FP}\mid s_c^{*}\cup \text{FP}\right)}=\frac{P\left(i\in s_c\right)}{P(i\in \text{FP})}=P\left(i\in s_c\mid \text{FP}\right)={\pi }_i^{(c)}$$ (9)

since $s_c^{*}$ is a copy of s_c and $P\left(i\in s_c^{*}\mid s_c^{*}\cup \text{FP}\right)=P\left(i\in s_c\mid s_c^{*}\cup \text{FP}\right)$. Notice that, derived from Formula (8),

$$p_i\le \frac{1}{2}\text{,}$$

since ${\pi }_i^{(c)}\le 1$ and the equality holds only if ${\pi }_i^{(c)}=1$, that is, the FP unit i participates in the cohort with certainty. As illustrated by the examples at the end of section 2.2.1, requiring p_i ≤ 1∕2 is not unrealistic in typical applications because ${\pi }_i^{(c)}$ is generally quite small.

Suppose that p_i can be modeled parametrically by p_i = p(x_i;β) = expit(β^Tx_i), where β is a vector of unknown model parameters. That is,

$$\log \left\{\frac{p_i}{1-p_i}\right\}={\mathit{\beta }}^T{\mathit{x}}_i,\text{for}i\in s_c^{*}\cup \text{FP}$$ (10)

Notice that β, the coefficients in Model (10), differ from the coefficients γ in Model (1) because the two logistic regression models have different dependent variables. Based on (8), expression (10) implies that ${\pi }_i^{(c)}$ is being modeled as exp(β^Tx_i), which differs from the RDW/CLW model in (1) where ${\pi }_i^{(c)}=\exp \left({\mathit{\gamma }}^T{\mathit{x}}_i\right)/\left\{1+\exp \left({\mathit{\gamma }}^T{\mathit{x}}_i\right)\right\}$. The corresponding “likelihood” function can be written as

$$L^{*}(\beta )=\prod _{i\in s_c^{*}\cup FP}{p}_i^{R_i}{\left(1-p_i\right)}^{\left(1-R_i\right)},$$ (11)

where R_i indicates the membership of $s_c^{*}$ in $s_c^{*}\cup \text{FP}\left(=1\text{if}i\in s_c^{*};0\text{if}i\in \text{FP}\right)$. We put “likelihood” in quotes because L*(β) varies depending on which set of units is selected for $s_c^{*}$. This contrasts with the population likelihood in (2) which applies regardless of which sample is selected. Note that L*(β) in (11) is written as if the units are independent when they are not. This is a standard procedure in pseudo-MLE estimation, and the resulting parameter estimators remain design-consistent even when some units may be correlated due to, for example, clustering.^9,10 The quantity L*(β) should be viewed as motivation for developing the estimating equations given below in (13). The log-likelihood generated from L*(β) is

$$l^{*}(\mathit{\beta })=\sum _{i\in S_c^{*}\cup FP}\left\{R_i\cdot \log p_i+\left(1-R_i\right)\log \left(1-p_i\right)\right\}=\sum _{i\in \text{FP}}{\delta }_i^{(c)}\log p_i+\sum _{i\in \text{FP}}\log \left(1-p_i\right).$$ (12)

Notice that the randomness of L*(β) and l*(β) comes from the cohort selection, that is, ${\delta }_i^{(c)}$ in the first summand in the last line of (12). In reality, since the unit level information of FP is unknown, we replace the second summand in (12) by a survey sample estimate, ${\sum }_{i\in s_p}{d}_i\log \left(1-p_i\right)$, and obtain the maximum pseudo-likelihood estimator $\widehat{\mathit{\beta }}$ by solving the pseudo-estimating equation

$${\tilde{S}}^{*}(\mathit{\beta })=\frac{1}{N+n_{\text{c}}}\left\{\sum _{i\in {\mathbf{s}}_{\text{c}}}\left(1-p_i\right){\mathit{x}}_i-\sum _{i\in s_p}{d}_i{p}_i{\mathit{x}}_i\right\}=\mathbf{0}.$$ (13)

Assuming that p_i is bounded by 0 and 1/2 implies that ${\pi }_i^{(c)}$ is automatically bounded by 0 and 1. Note that (13) falls in a general class of estimating equations that ensure unique solutions of parameters^4,11,12 (eg, equation (6) in CLW⁴ if their function h(x_i, θ) is set equal to ${\mathit{x}}_i{\left\{1+{\pi }_i^{(c)}\right\}}^{-1}$).

The ALP estimator of μ is

$${\widehat{\mu }}^{\text{ALP}}=\frac{{\sum }_{i\in S_c}{w}_i^{\text{ALP}}{y}_i}{{\sum }_{i\in S_c}{w}_i^{\text{ALP}}},$$ (14)

where $w_i^{\text{ALP}}=1/{\pi }_i^{(c)}(\widehat{\mathit{\beta }})$ for i ∈ s_c. Although L*(β) is not a standard likelihood, ${\widehat{\mu }}^{\text{ALP}}$ is a consistent estimator of the population mean as shown in the theorem below.

We consider the following limiting process for the theoretical development.^4,13 Suppose there is a sequence of FPs FP_k of size N_k, for k = 1, 2, …. Cohort s_c,k of size n_c,k and survey sample s_p,k of size n_p,k are sampled from FP_k. The sequences of the FP, the cohort and the survey sample have their sizes satisfy ${\lim }_{k\to \infty }{n}_{t,k}/N_k\to f_t$, where t = c or p and 0 < f_t ≤ 1 (regularity condition C1 in Appendix A). In the following the index k is suppressed for simplicity.

Theorem.

Consistency of ALP estimator of FP mean (see Appendix B).

Under the regularity conditions A1 to A3, and C1 to C5 in Appendix A, and assuming logistic regression model (10) for p_i, the ALP estimate ${\widehat{\mu }}^{\text{ALP}}$ is design consistent for μ, in particular ${\widehat{\mu }}^{\text{ALP}}-\mu =O_p\left(n_c^{-1/2}\right)$, with the FP variance

$$\mathrm{Var}\left({\widehat{\mu }}^{\text{ALP}}\right)\doteq N^{-2}\sum _{i\in \text{FP}}{p}_i\left(1-2p_i\right){\left\{\frac{\left(y_i-\mu \right)}{p_i}-{\mathit{b}}^T{\mathit{x}}_i\right\}}^2+{\mathit{b}}^T\mathit{D}\mathit{b},$$ (15)

where $p_i=\mathrm{expit}\left({\mathit{\beta }}^T{\mathit{x}}_i\right),{\mathit{b}}^T=\left\{{\sum }_{i\in \text{FP}}\left(y_i-\mu \right){\mathit{x}}_i^T\right\}{\left\{{\sum }_{i\in \text{FP}}{p}_i{\mathit{x}}_i{\mathit{x}}_i^T\right\}}^{-1}$, and $\mathit{D}=N^{-2}{V}_p\left({\sum }_{i\in s_p}{d}_i{p}_i{\mathit{x}}_i\right)$ is the design-based variance-covariance matrix under the probability sampling design for s_p.

In practice, the ALP estimator of a FP mean can be obtained by three steps:

Step 1 Search for covariates x available in both the cohort (s_c) and the reference survey sample (s_p) and combine the two samples. Assign R_i = 1 for i ∈ s_c and R_i = 0 for i ∈ s_p in the combined sample.

Step 2 Fit a logistic regression model for p_i = P(R_i = 1) in the combined s_c and weighted s_p, with the survey sample weights {d_i, i ∈ s_p}, and obtain the estimate ${\widehat{p}}_i$ for i ∈ s_c.

Step 3 Estimate the FP mean by Formula (14) with the ALP pseudo weight $w_i^{\text{ALP}}={\widehat{p}}_i/\left(1-{\widehat{p}}_i\right)$ for i ∈ s_c.

Notice that Step 2 can be accomplished by any existing survey software, such as svyglm in survey package of R, svy:logit in Stata, and PROC SURVEYLOGISTIC in SAS. In addition to being easy to implement, the ALP estimator from (14) does not require conditions (i) or (ii), unlike RDW. Moreover, we prove that in large samples, $\mathrm{Var}\left({\widehat{\mu }}^{\text{ALP}}\right)=O\left(n_c^{-1}\right)$ is as or more efficient compared with $\mathrm{Var}\left({\widehat{\mu }}^{\text{CLW}}\right)=O\left\{\min {\left(n_p,n_c\right)}^{-1}\right\}$ under their correct propensity model, respectively, which depends on both the nonprobability and probability sample sizes (see Appendix C).

An alternative method would be to omit the odds transformation, which uses p_i to approximate the participation rate ${\pi }_i^{(c)}$. Denote this method by FDW for full design weight, which contrasts to the scaling of the survey sample weights in the RDW method. Comparing the expectation of the population log-likelihood function l(γ) in (3) and the expectation of the pseudo log-likelihood ${\tilde{l}}^{*}(\beta )$ in (12) with ${\pi }_i^{(c)}$ replacing p_i by the FDW method, that is, ${\tilde{l}}^{*}(\mathit{\gamma })={\sum }_{i\in s_c}\log {\pi }_i^{(c)}+{\sum }_{i\in s_p}{d}_i\log (1-{\pi }_i^{(c)})$, we have their difference, denoted by Δ_FDW, written as

$$\begin{gathered} {\Delta }_{\text{FDW}}=E\{{\tilde{l}}^{*}(\mathit{\gamma })\}-E\{l(\mathit{\gamma })\}=\sum _{i\in FP}{\pi }_i^{(c)}\log {\pi }_i^{(c)}+\sum _{i\in FP}\log \{1-{\pi }_i^{(c)}\} \\ -\sum _{i\in FP}{\pi }_i^{(c)}\log {\pi }_i^{(c)}-\sum _{i\in FP}\left(1-{\pi }_i^{(c)}\right)\log \{1-{\pi }_i^{(c)}\}=\sum _{i\in FP}{\pi }_i^{(c)}\log \{1-{\pi }_i^{(c)}\}. \end{gathered}$$

The bias is zero only if ${\pi }_i^{(c)}$ for i ∈ FP are all close to zero. Thus, the odds transformation step in ALP could be skipped if all nonprobability participation rates are extremely small; but, in general, that step is essential for unbiased estimation.

2.4 |. Variance estimation

Using the FP variance formula (15), the first summand can be consistently estimated by

$${\left\{{\widehat{N}}^{(c)}\right\}}^{-2}\sum _{i\in s_c}\left(1-{\widehat{p}}_i\right)\left(1-2{\widehat{p}}_i\right){\left\{\frac{\left(y_i-{\widehat{\mu }}^{\text{ALP}}\right)}{{\widehat{p}}_i}-{\widehat{\mathit{b}}}^T{\mathit{x}}_i\right\}}^2,$$ (16)

where ${\widehat{p}}_i$ is the prediction for i ∈ s_c, ${\widehat{N}}^{(c)}={\sum }_{i\in S_c}{w}_i^{\text{ALP}}$, and ${\widehat{\mathit{b}}}^T=\left\{{\sum }_{i\in {\text{s}}_{\text{c}}}\left(y_i-{\widehat{\mu }}^{\text{ALP}}\right){\mathit{x}}_i^T\right\}{\left\{{\sum }_{i\in s_c}{\widehat{p}}_i{\mathit{x}}_i{\mathit{x}}_i^T\right\}}^{-1}$. The second summand b^TDb is estimated by ${\widehat{\mathit{b}}}^T\widehat{D}\widehat{\mathit{b}}$, where $\widehat{\mathit{D}}$ is the survey design consistent variance estimator of D. For example, under stratified multistage cluster sampling with H strata and a_h primary sampling units (PSUs) in stratum h selected with replacement,

$$\widehat{\mathit{D}}={\left\{{\widehat{N}}^{(p)}\right\}}^{-2}\cdot \sum _{h=1}^H\frac{a_h}{a_h-1}\sum _{l=1}^{a_h}\left(z_l-\overline{z}\right){\left(z_l-\overline{z}\right)}^T,$$ (17)

where ${\widehat{N}}^{(p)}={\sum }_{i\in s_p}{d}_i$, ${\mathbf{z}}_l={\sum }_{i\in s_{p(hl)}}{d}_i{\widehat{p}}_i{\mathit{x}}_i$ is the weighted PSU total for cluster l in stratum h, s_p(hl) is the set of sample elements stratum h and cluster l, and $\overline{z}=a_h^{-1}{\sum }_l^{a_h}{z}_l$ is the mean of the PSU totals in stratum h.

2.5 |. Scaling survey weights in the likelihood for the ALP method

The proposed ALP can flexibly scale the survey weights in estimating equation (13) to improve efficiency. For case-control studies, Scott and Wild¹⁴ and Li et al¹⁵ previously used the technique we propose below to reduce variances of estimates of relative risks when weights for cases and controls are substantially different. We multiply the second summand in ${\tilde{S}}^{*}(\mathit{\beta })$ by a constant λ, say $\lambda =n_c/\left({\sum }_{i\in s_p}{d}_i\right)$, so that the sum of the scaled survey weights (λd_i) is n_c. Accordingly, the score function becomes

$${\tilde{S}}_{\lambda }^{*}(\mathit{\beta })=\sum _{i\in s_c}\left(1-p_i\right){\mathit{x}}_i-\lambda \sum _{i\in s_p}{d}_i{p}_i{\mathit{x}}_i.$$ (18)

Solving ${\tilde{S}}_{\lambda }^{*}(\mathit{\beta })=0$ for β, and the resulting vector of estimates is denoted by ${\widehat{\mathit{\beta }}}_{\lambda }=\left({\widehat{\beta }}_{0,\lambda },{\widehat{\mathit{\beta }}}_{1,\lambda }\right)$, where ${\widehat{\beta }}_{0,\lambda }$ is estimate of the intercept. Similar derivations to those in Chambers and Skinner¹⁰ and Beaumont¹¹ can be used to prove that ${\widehat{\mathit{\beta }}}_{1,\lambda }$, is design-consistent with various efficiency gains, depending on the variability of survey weights vs the nonprobability sample weights (with implicit common value of 1). However, the estimate of the intercept ${\widehat{\beta }}_{0,\lambda }$ can be badly biased with scaled weights. As a result, the estimate of participation rate $\exp \left({\widehat{\mathit{\beta }}}_{\lambda }^T{\mathit{x}}_i\right)$ including ${\widehat{\beta }}_{0,\lambda }$ would also be biased. The bias of ${\widehat{\beta }}_{0,\lambda }$, however, would not affect the estimate of population mean because the scaled ALP-weighted mean, ${\widehat{\mu }}^{\text{ALP}.S}$,

$${\widehat{\mu }}^{\text{ALP}.S}=\frac{{\sum }_{i\in s_c}{w}_i^{\text{ALP}.S}{y}_i}{{\sum }_{i\in s_c}{w}_i^{\text{ALP}.S}}=\frac{{\sum }_{i\in s_c}{\exp }^{-1}\left({\widehat{\mathit{\beta }}}_{1,\lambda }^T{\mathit{x}}_i\right)y_i}{{\sum }_{i\in s_c}{\exp }^{-1}\left({\widehat{\mathit{\beta }}}_{1,\lambda }^T{\mathit{x}}_i\right)},$$

depends on ${\widehat{\mathit{\beta }}}_{1,\lambda }$, but not ${\widehat{\beta }}_{0,\lambda }$, where $w_i^{\text{ALP}.S}={\exp }^{-1}\left({\widehat{\mathit{\beta }}}_{1,\lambda }^T{\mathit{x}}_i\right)$ is the scaled ALP pseudo weight.

It can be proved that ${\widehat{\mu }}^{\text{ALP}.S}$ is a consistent estimator of the FP mean, μ. The TL variance estimator of ${\widehat{\mu }}^{\text{ALP}.S}$ can be obtained by substituting $w_i^{\text{ALP}}$, ${\widehat{\mu }}^{\text{ALP}}$, $\widehat{\mathit{\beta }}$, ${\widehat{p}}_i$, and d_i by $w_i^{\text{ALP}.S}$, ${\widehat{\mu }}^{\text{ALP}.S}$, ${\widehat{\mathit{\beta }}}_{\lambda }$, ${\widehat{p}}_{i,\lambda }=\exp \left({\widehat{\mathit{\beta }}}_{\lambda }^T{\mathit{x}}_i\right)$, and λd_i, respectively, in Formulae (16) and (17). Details on the variance and the consistency of ALP.S are discussed in the dissertation by Wang.¹⁶

3 |. SIMULATIONS

3.1 |. FP generation and sample selection

We applied simulation setups similar to those in Chen et al.⁴ In the FP of size N = 500 000, a vector of covariates x_i = (x_1i, x_2i, x_3i, x_4i)^T was generated for i ∈ FP where x_1i = v_1i, x_2i = v_2i + 0.3×_1i, x_3i = v_3i + 0.2(x_1i + x_2i), x_4i = v_4i + 0.1 (x_1i + x_2i + x_3i), with v_1i ~ Bernoulli(0.5), v_2i ~ Uniform(0, 2), v_3i ~ Exponential(1), and v_4i ~ χ²(4). The variable of interest y_i ~ Normal(μ_i, 1), where μ_i = −x_1i − x_2i + x_3i + x_4i for i ∈ FP. The parameter of interest was the FP mean $\mu =N^{-1}{\sum }_{i\in \text{FP}}{y}_i=3.97$.

The probability-based survey sample s_p with the target sample size n_p = 12 500 (sampling fraction f_p = 2.5%) was selected by Poisson sampling, with inclusion probability ${\pi }_i^{(p)}=\left(n_p\cdot q_i\right)/{\sum }_{i\in \text{FP}}{q}_i$ for i ∈ FP, where q_i = const + x_3i + 0.03y_i to control for the variation of the survey weights, $d_i=1/{\pi }_i^{(p)}$. We set const = −0.26 so that max q_i/ min q_i = 20.

As noted in Section 2, the ALP and CLW methods do assume somewhat different models for the participation rate. Thus, it is interesting to check their performances both when their underlying models are correct and when the assumed participation rate models fail. The volunteer-based nonprobability sample s_c (with a target sample size n_c) was also selected by Poisson sampling but with different inclusion probabilities ${\pi }_i^{(c)}$ for i ∈ FP. We considered two scenarios with different functional forms of ${\pi }_i^{(c)}$ so that the ALP (and FDW) or the CLW method had the true linear logistic regression propensity model in one scenario but not in the other. In Scenario 1, ${\pi }_i^{(c)}=\exp \left({\beta }_0+{\mathit{\beta }}^T{\mathit{x}}_i\right)$ was the specified participation rate for the ith population unit to be included into the nonprobability sample. The underlying true propensity model for ALP (and FDW) methods, shown in (10), was $\mathrm{logit}\left(p_i\right)=\mathrm{logit}\left\{{\pi }_i^{(c)}\right\}={\beta }_0+{\mathit{\beta }}^T{\mathit{x}}_i$, which implies $\mathrm{logit}\left\{{\pi }_i^{(c)}\right\}={\beta }_0+{\mathit{\beta }}^T{\mathit{x}}_i-\log \left\{1-{\pi }_i^{(c)}\right\}$. This model differs from the underlying linear model (1) assumed by the CLW method by the addition of the term $\log \left\{1-{\pi }_i^{(c)}\right\}$. In Scenario 2, ${\pi }_i^{(c)}=\mathrm{expit}\left({\mathit{\gamma }}_0+{\mathit{\gamma }}^T{\mathit{x}}_i\right)$ was specified so that $\mathrm{logit}\left\{{\pi }_i^{(c)}\right\}={\mathit{\gamma }}_0+{\mathit{\gamma }}^T{\mathit{x}}_i$, which was the model (1) assumed by the CLW method. This model, however, implied that $\mathrm{logit}\left(p_i\right)={\mathit{\gamma }}_0+{\mathit{\gamma }}^T{\mathit{x}}_i-\log \left\{1-{\pi }_i^{(c)}\right\}$, which was different from the model assumed by the ALP and the FDW method (by the extra term $\log \left\{1-{\pi }_i^{(c)}\right\}$). Hence, ALP and CLW estimates of the population mean are expected to be unbiased in one scenario but not the other since both methods assume a linear logistic propensity model. The biases of the FDW and RDW estimates, as measured by Δ_FDW and Δ_RDW, depend on ${\pi }_i^{(c)}$, and go to 0 as ${\pi }_i^{(c)}$ approaches 0. The biases become larger as ${\pi }_i^{(c)}$ increases in either scenario.

In both scenarios, the coefficients were set to be β = γ = (0.18, 0.18, −0.27, −0.27)^T. The parameters were chosen so that $0<{\pi }_i^{(c)}<1$ for all units i ∈ FP. The intercepts β₀ and γ₀ were also controlled so that the expected number of nonprobability sample units E_c(n_c) = $E_c\left(n_c\right)={\sum }_{\text{FP}}{\pi }_i^{(c)}$ was varied from 1250, 2500, 5000, to 10000 with the corresponding overall participation rate f_c = E_c(n_c)∕N being 0.5%, 5%, 10%, or 20%.

3.2 |. Evaluation criteria

We examined the performance of five IPSW estimators of FP mean μ: (1) to (2) ${\widehat{\mu }}^{\text{ALP}}$ and ${\widehat{\mu }}^{\text{ALP}.S}$ described in Sections 2.2 to 2.5; (3) ${\widehat{\mu }}^{\text{FDW}}$ using weights from the ALP method omitting the odds transformation; (4) ${\widehat{\mu }}^{\text{CLW}}$ proposed by Chen et al⁴; and (5) ${\widehat{\mu }}^{\text{RDW}}$ proposed by Wang et al,⁶ compared with the naïve nonprobability sample mean $\left({\widehat{\mu }}^{\text{Naive}}\right)$ that did not use weights, and the weighted nonprobability sample mean, ${\widehat{\mu }}^{\text{TW}}$, with weights equal to the inverse of the true nonprobability sample inclusion probabilities. Note that ${\widehat{\mu }}^{\text{TW}}$ is unavailable in practice because the true nonprobability sample inclusion probabilities are unknown. Relative bias (%RB), empirical variance (V), mean squared error (MSE) of the point estimates were used to evaluate the performance of the four IPSW point estimates, calculated by

$$\%\text{RB}=\frac{1}{B}\sum _{b=1}^B\frac{{\widehat{\mu }}^{(b)}-\mu }{\mu }\times 100,V=\frac{1}{B-1}\sum _{b=1}^B{\left\{{\widehat{\mu }}^{(b)}-\frac{1}{B}\sum _{b=1}^B{\widehat{\mu }}^{(b)}\right\}}^2,\text{MSE}=\frac{1}{B}\sum _{b=1}^B{\left\{{\widehat{\mu }}^{(b)}-\mu \right\}}^2,$$

where B = 4000 is the number of simulation runs, ${\widehat{\mu }}^{(b)}$ is one of the point estimates obtained from the bth simulated sample, and μ is the true FP mean.

We also evaluated the variance estimates using the variance ratio (VR) and 95% confidence interval coverage probability (CP), which were calculated as

$$\text{VR}=\frac{\frac{1}{B}{\sum }_{b=1}^B{\widehat{v}}^{(b)}}{V}\times 100,\text{and}\text{CP}=\frac{1}{B}\sum _{b=1}^{B}I\left(\mu \in {\text{CI}}^{(b)}\right),$$

where ${\widehat{v}}^{(b)}$ is the proposed analytical variance estimate in simulated sample b, and ${\text{CI}}^{(b)}=({\widehat{\mu }}^{(b)}-1.96\sqrt{{\widehat{v}}^{(b)}},{\widehat{\mu }}^{(b)}+1.96\sqrt{{\widehat{v}}^{(b)}})$ is the 95% confidence interval from the bth simulated sample.

3.3 |. Results

Table 1 presents simulation results for the seven nonprobability sample estimators of the FP mean. The naïve estimator ${\widehat{\mu }}^{\text{Naive}}$ that ignored the underlying sampling scheme had relative biases ranging from −36.5% to −42.8% while the true weighted nonprobability sample estimator, ${\widehat{\mu }}^{\text{TW}}$, was approximately unbiased in all scenarios. The variance of ${\widehat{\mu }}^{\text{Naive}}$ was much smaller than that of the other estimators, but its bias caused the MSE to be extremely high (not reported).

TABLE 1.

Results from 4000 simulated survey samples and nonprobability samples with low to high participation rates under various propensity score models

	Scenario 1 True propensity model for ALP					Scenario 2 True propensity model for CLW
	%RB	V (×105)	VR	MSE (×105)	CP5	%RB	V (×105)	VR	MSE (×105)	CP
fc = 0.5%
${\widehat{\mu }}^{\text{Naive}}$	−42.76	0.22	0.99			−42.61	0.22	1.00
${\widehat{\mu }}^{\text{TW}}$	−0.13	4.38	0.93	4.39	0.90	−0.12	4.38	0.93	4.38	0.90
${\widehat{\mu }}^{\text{RDW}}$	−0.29	3.73	0.93	3.75	0.87	−0.40	3.63	0.93	3.66	0.87
${\widehat{\mu }}^{\text{FDW}}$	−0.28	3.73	0.93	3.75	0.87	−0.40	3.63	0.93	3.66	0.87
${\widehat{\mu }}^{\text{ALP}}$	−0.07	3.70	0.93	3.77	0.88	−0.19	3.66	0.93	3.67	0.88
${\widehat{\mu }}^{\text{CLW}}$	0.05	3.87	0.93	3.87	0.89	−0.07	3.76	0.93	3.76	0.88
${\widehat{\mu }}^{\text{ALP}.S}$	−0.11	3.54	0.92	3.54	0.87	−0.21	3.45	0.92	3.45	0.87
fc = 5%
${\widehat{\mu }}^{\text{Naive}}$	−42.74	0.02	0.99			−41.21	0.02	1.01
${\widehat{\mu }}^{\text{TW}}$	−0.04	0.50	0.98	0.50	0.92	−0.02	0.46	1.00	0.47	0.93
${\widehat{\mu }}^{\text{RDW}}$	−2.15	0.56	1.00	1.29	0.66	−3.05	0.43	1.01	1.89	0.45
${\widehat{\mu }}^{\text{FDW}}$	−2.05	0.57	1.00	1.23	0.68	−2.95	0.43	1.01	1.81	0.47
${\widehat{\mu }}^{\text{ALP}}$	−0.01	0.62	1.00	0.62	0.94	−1.03	0.47	1.01	0.64	0.85
${\widehat{\mu }}^{\text{CLW}}$	1.29	0.84	1.00	1.10	0.95	0.01	0.61	1.01	0.61	0.94
${\widehat{\mu }}^{\text{ALP}.S}$	−0.05	0.45	1.00	0.45	0.92	−0.63	0.35	1.02	0.41	0.86
fc = 10%
${\widehat{\mu }}^{\text{Naive}}$	−42.74	0.01	1.11			−39.65	0.01	1.11
${\widehat{\mu }}^{\text{TW}}$	−0.01	0.25	1.02	0.25	0.94	−0.01	0.22	1.01	0.22	0.94
${\widehat{\mu }}^{\text{RDW}}$	−4.25	0.34	1.00	3.20	0.17	−5.62	0.22	0.99	5.20	0.02
${\widehat{\mu }}^{\text{FDW}}$	−3.87	0.35	1.00	2.71	0.24	−5.28	0.22	0.99	4.62	0.03
${\widehat{\mu }}^{\text{ALP}}$	0.01	0.42	1.00	0.42	0.95	−1.81	0.27	0.99	0.79	0.65
${\widehat{\mu }}^{\text{CLW}}$	2.94	0.80	1.00	2.16	0.76	0.03	0.42	0.99	0.42	0.95
${\widehat{\mu }}^{\text{ALP}.S}$	−0.03	0.27	1.02	0.27	0.94	−0.86	0.18	1.01	0.29	0.81
fc = 20%
${\widehat{\mu }}^{\text{Naive}}$	−42.75	0.00	1.26			−36.50	0.01	1.21
${\widehat{\mu }}^{\text{TW}}$	−0.02	0.15	0.93	0.15	0.93	−0.02	0.11	0.96	0.11	0.93
${\widehat{\mu }}^{\text{RDW}}$	−8.58	0.21	0.95	11.83	0.00	−9.59	0.10	0.97	14.60	0.00
${\widehat{\mu }}^{\text{FDW}}$	−7.15	0.23	0.96	8.29	0.01	−8.51	0.11	0.98	11.53	0.00
${\widehat{\mu }}^{\text{ALP}}$	0.00	0.32	0.96	0.32	0.95	−2.85	0.16	0.98	1.44	0.19
${\widehat{\mu }}^{\text{CLW}}$	7.80	1.67	0.92	11.27	0.06	0.01	0.33	0.98	0.33	0.95
${\widehat{\mu }}^{\text{ALP}.S}$	−0.03	0.19	0.96	0.19	0.94	−1.02	0.10	1.00	0.27	0.73

Abbreviations: ALP, adjusted logistic propensity; CP, coverage probability; MSE, mean squared error; VR, variance ratio.

Consistent with the bias theory in Section 2, the RDW point estimator ${\widehat{\mu }}^{\text{RDW}}$ and the FDW point estimator ${\widehat{\mu }}^{\text{FDW}}$ were approximately unbiased when ${\pi }_i^{(c)}$ was small for all i ∈ FP and the overall participation rate $f_c=N^{-1}{\sum }_{i\in \text{FP}}{\pi }_i^{(c)}$ was low, but more biased as f_c increased. The CPs decreased correspondingly.

As expected, the ALP estimators ${\widehat{\mu }}^{\text{ALP}}$ and ${\widehat{\mu }}^{\text{ALP}.S}$(or the CLW estimator ${\widehat{\mu }}^{\text{CLW}}$) consistently provided unbiased point estimators in the scenarios where they were expected to be unbiased, that is, scenario 1 for ${\widehat{\mu }}^{\text{ALP}}$ and ${\widehat{\mu }}^{\text{ALP}.S}$, and scenario 2 for ${\widehat{\mu }}^{\text{CLW}}$. When the underlying model was incorrect for an estimator, biases occurred. For example, the relative biases of ${\widehat{\mu }}^{\text{CLW}}$ in scenario 1 were 0.05%, 1.29%, 2.94%, and 7.80% as f_c increased from 0.5%, 5%, 10%, to 20%, respectively. In scenario 2, the corresponding relative biases for ${\widehat{\mu }}^{\text{ALP}}$ are −0.19%, −1.03%, −1.81%, and −2.85%.

Consistent with the theory in Section 2, the ALP estimator ${\widehat{\mu }}^{\text{ALP}}$ was more efficient than ${\widehat{\mu }}^{\text{CLW}}$ with consistently smaller empirical variances in all scenarios, especially when the nonprobability cohort size was much larger than the probability sample size. Among all considered methods, ${\widehat{\mu }}^{\text{ALP}.S}$ was approximately unbiased with the smallest variance under Scenario 1 of the correct model. Under Scenario 2 of a misspecified model, ${\widehat{\mu }}^{\text{ALP}.S}$ was biased but most efficient, and therefore achieved smallest MSE.

The variance estimators for ${\widehat{\mu }}^{\text{ALP}}$, ${\widehat{\mu }}^{\text{ALP}.S}$, and ${\widehat{\mu }}^{\text{CLW}}$ performed very well (with VR’s near 1), providing CPs close to the nominal level under the correct propensity models when f_c was large. The lower coverage of the nominal level (about 88%) when f_c = 0.5% was due to the small sample bias with skewed distributions of underlying sampling weights in the selected nonprobability sample.

4 |. REAL DATA EXAMPLE

We use the same data example as Wang et al⁶ for illustration purposes. We estimated prospective 15-year all-cause, all-cancer, and heart disease mortality rates for adults in the US using the adult household interview part of The Third U.S. National Health and Nutrition Examination Survey (NHANES III) III conducted in 1988 to 1994, with sample size n_c = 20 050. We ignored all complex design features of NHANES III and treated it as a nonprobability sample. The coefficient of variation of sample weights is 125%, indicating highly variable selection probabilities, and thus low representativeness of the unweighted sample. For estimating mortality rates, we approximated that the entire sample of NHANES III was randomly selected in 1991 (the midpoint of the data collection time period).

For the reference survey, we used 1994 U.S. National Health Interview Survey (NHIS) respondents to the supplement for monitoring achievement of the Healthy People Year 2000 objectives. Adults aged 18 and older are included (sample size n_p = 19 738). The 1994 NHIS used a multistage stratified cluster sample design with 125 strata and 248 pseudo-PSUs.^17,18 We collapsed strata with only one PSU with the next nearest stratum for variance estimation purposes.¹⁹ Both samples of NHANES III and NHIS were linked to National Death Index (NDI) for mortality, allowing us to quantify the relative bias of unweighted NHANES estimates, assuming the NHIS estimates as the gold standard. Notice that the mortality information was obtained by statistical linkage between the survey sample and NDI,²⁰ but not responses from the questionnaires. The all-cancer and heart-disease mortality were classified according to National Center for Health Statistics death code.^21,22

The usage of NHANES III as the “nonprobability cohort” has several advantages for illuminating the performance of the propensity weighting methods. The “nonprobability sample” and the reference survey sample have approximately the same target population, data collection mode, and similar questionnaires. This ensures that the pseudo-weighted “nonprobability sample” could potentially represent the target population, and thus enables us to characterize the performance of the propensity weighting methods in real data.

The distributions of selected common covariates and variables of interests in the two samples are presented in Table 2. As expected, the variables in the weighted samples of NHANES and 1994 NHIS have very close distributions because both weighted samples represent approximately the same FP. By contrast, covariates distribute quite differently in the unweighted NHANES from the weighted samples, especially for design variables such as age, race/ethnicity, poverty, and region, which leads to large biases in mortality rates estimated from the unweighted NHANES.

TABLE 2.

Distribution of selected common variables in NIH-AARP and NHIS

		NHIS 1994		NHANES III
		np = 19738	${\widehat{N}}_p=189608549$	nc = 20050	${\widehat{N}}_c=187647206$
Variable	Total count	%	Weighted %	%	Weighted %
Age Group	18–24 years	10.5	13.3	15.8	15.8
	25–44 years	42.9	43.7	35.4	43.7
	45–64 years	26.1	26.6	22.6	24.6
	65 years and older	20.5	16.4	26.2	16.0
Race	NH-White	76.1	75.9	42.3	76.0
	NH-Black	12.6	11.2	27.4	11.2
	Hispanic	8.0	9.0	28.9	9.3
	NH-other	3.3	4.0	1.5	3.5
Region	Northeast	20.7	20.5	14.6	20.8
	Midwest	26.1	25.1	19.2	24.1
	South	31.5	32.5	42.7	34.3
	West	21.6	21.9	23.5	20.9
Poverty	No	79.1	82.3	67.9	80.3
	Yes	13.1	10.6	21.4	12.1
	Unknown	7.8	7.0	10.7	7.6
Education	Lower than high school	20.1	19.1	42.5	26.6
	High school/Some college	58.7	59.6	45.9	54.1
	College or higher	21.2	21.3	11.6	19.3
Health status	Excellent/Very good	60.5	62.0	39.0	51.6
	Good	25.7	25.7	35.9	32.7
	Fair/Poor	13.8	12.3	25.1	15.7
Mortality	All-cause	20.8	17.6	26.7	17.1
	Heart-disease	9.43	5.69	4.95	4.04
	All-cancer	5.57	4.11	5.10	4.47

The propensity model included main effects of common demographic characteristics (age, sex race/ethnicity, region, and marital status), socioeconomic status (education level, poverty, and household income), tobacco usage (smoking status, and chewing tobacco), health variables (body mass index and self-reported health status), and a quadratic term for age. Appendix D shows the final propensity models for the five considered methods.

To evaluate the performance of the five PS-based methods, we used relative difference from the NHIS estimate $\%\text{RD}=\left(\widehat{\mu }-{\widehat{\mu }}^{\text{NHIS}}\right)/{\widehat{\mu }}^{\text{NHIS}}\times 100$, TL variance estimate (V), and estimated $\text{MSE}={\left(\widehat{\mu }-{\widehat{\mu }}^{\text{NHIS}}\right)}^2+V$, which treated the NHIS estimate as truth. Table 3 shows that the naïve NHANES III estimate of overall mortality was ~52% biased from the NHIS estimate because older people who have higher mortalities were oversampled (Table 2). All five IPSW methods substantially reduced the bias from the naïve estimate. Consistent with the simulation results, the ALP, FDW, RDW, and CLW method yielded close estimates when the sample fraction of the nonprobability sample was small ( ${\widehat{f}}_c=n_c/{\widehat{N}}_p=1.06\times {10}^{-4}$ calculated from Table 2). The ALP.S method, by scaling the NHIS sample weights in propensity estimation, reduced more bias than the other methods, and was more efficient. Therefore, the ALP.S estimate had the smallest MSE. The results for inference of all-cancer mortality had the similar pattern as the results for all-cause mortality. All pseudo-weighting methods removed most bias of the naïve NHANES estimate (with %RD = −3.21%~2.07% reduced from 24.68%). By contrast, for heart-disease mortality, all pseudo-weighting methods were substantially less biased than the naïve estimate, with %RD = 42.58%~57.78% reduced from 133.66%, but the alternative estimators still had undesirably large biases themselves. The bias reduction is not as much as that for all-cancer or all-cause mortality, and this may be due to the omission of important predictors of having heart disease and of being observed in the nonprobability sample in the propensity model.

TABLE 3.

Relative difference (%RD) of all-cause 15-year mortality estimates from the NHIS estimate with estimated variance (V) and mean squared error (MSE)

Mortality	Method	Estimate (%)	%RD	V (×105)	MSE (×105)
All cause	NHIS	17.6
	Naïve	26.7	52.16
	ALP	18.6	6.08	1.87	13.27
	FDW	18.6	6.08	1.87	13.28
	RDW	18.6	6.08	1.87	13.28
	CLW	18.6	6.07	1.87	13.24
	ALP.S	17.2	−2.05	1.08	2.37
All cancer	NHIS	4.5
	Naïve	5.6	24.68
	ALP	4.6	2.07	0.38	0.46
	FDW	4.6	2.07	0.38	0.46
	RDW	4.6	2.07	0.38	0.46
	CLW	4.6	2.06	0.37	0.46
	ALP.S	4.3	−3.21	0.32	0.53
Heart disease	NHIS	4.0
	Naïve	9.4	133.66
	ALP	6.4	57.78	0.50	54.88
	FDW	6.4	57.78	0.50	54.90
	RDW	6.4	57.78	0.50	54.90
	CLW	6.4	57.77	0.50	54.87
	ALP.S	5.8	42.58	0.33	29.86

Abbreviations: ALP, adjusted logistic propensity; RDW, rescaled design weight.

5 |. DISCUSSION

This article proposed ALP weighting methods for population inference using nonprobability samples. The proposed ALP method corrects the bias in the RDW method⁷ by formulating the problem in an innovative way. As does the RDW method, the proposed ALP method retains the advantage of easy implementation by fitting a propensity model with survey weights in ready-to-use software. The proposed ALP estimators are design consistent if the assumed model for participation rate is correct. TL variance estimators for ALP estimates are derived. Consistency of the ALP FP mean estimators was proved theoretically and evaluated numerically.

A primary competitor to ALP is the CLW estimator developed by Chen et al.⁴ If the nonprobability cohort is a small fraction of the population, ALP and CLW are very similar, although ALP does have computational advantages regardless of the size of the sampling fraction. As the sampling fraction increases, ALP and CLW become more distinct.

Both ALP and CLW methods fit a propensity model to the combined nonprobability sample and a weighted survey sample. Highly variable weights in the combined sample can lead to low efficiency of the estimated propensity model coefficients. Therefore, the variances of the ALP and the CLW estimators of the FP means can be large in some applications. However, the proposed ALP is proved analytically and numerically to have a variance that is less than or equal to that of the CLW method regardless of whether the propensity model underlying ALP is correct. It worth noting that ALP and CLW methods assume different logistic regression models for propensity score estimation. Propensity is defined as $p_i=P\left(i\in s_c^{*}\mid s_c^{*}\cup \text{FP}\right)$ by ALP in (9) and π_i = P(i ∈ s_c|FP) by CLW in (2). Model diagnostics should be developed to select which propensity model is more appropriate for a given dataset and will be the focus of our future research.

An alternative ALP with scaled survey weights in the logistic regression propensity model produces consistent propensity estimates and further improves efficiency as shown in the simulation and the real data example. The scaled ALP had the smallest MSE in every scenario in our simulation study regardless of underlying model and had the smallest MSE for two of the three mortality causes in our real data application. The CLW estimator with the scaled survey weights, albeit more efficient than the unscaled CLW, is biased (simulation results not shown). The extension of scaling technique to the CLW method and other binary regression propensity models requires further investigation.

The theory for the ALP method implies that p_i > 1/2 since p_i defines the probability of the nonprobability sample inclusion among the combined FP units and the nonprobability sample, that is, $P\left(i\in s_c^{*}\mid s_c^{*}\cup \text{FP}\right)$. In estimation, however, ${\widehat{p}}_i>1/2$ can happen, especially in the unusual case where the nonprobability sample $s_c^{*}$ is a large proportion of the population. Fortunately, this will not be a concern for the estimation of population mean or regression coefficients. By scaling survey weights in the propensity model for the ALP method, we can control for all ALP weights $w_i^{\text{ALP}.S}=1/{\widehat{\pi }}_i^{\text{ALP}.S}$ to be greater than one. As proved in Section 2.5, using scaled survey weights, $w_i^{\text{ALP}.S}=\lambda \cdot {\exp }^{-1}\left({\widehat{\mathit{\beta }}}_{\lambda }^T{\mathit{x}}_i\right)={\exp }^{-1}\left({\widehat{\mathit{\beta }}}_{0\lambda }^{*}+{\widehat{\mathit{\beta }}}_{1\lambda }^T{\mathit{x}}_i\right)$, would not bias the mean estimates. For population total estimation, an option is to estimate the population mean first and then multiply by a known or estimated population size from an independent source. Another approach to avoid estimated p_i > 1/2 is to solve the pseudo-estimating equations in (14) using a constrained optimization algorithm that requires ${\widehat{p}}_i\le 1/2$ for all units. This would, of course, negate the computation advantage of the ALP estimator.

Both ALP and CLW are inverse-propensity-score-weighting methods that directly use (functions of) the propensity score to estimate the cohort participation rate. They can be sensitive to propensity model misspecification (eg, missing interaction terms in the fitted propensity model) due to inaccurate estimates of participation rates. Furthermore, extreme pseudo-weights can occur if the estimate of the participation rate is close to 0. By contrast, propensity-score-based matching methods (not included in this study) may be more robust to the model misspecification and less likely to produce extreme pseudo-weights, because they use propensity scores to measure the similarity between survey and cohort sample units and distribute survey sample weights to the cohort based on their similarity. Examples of matching methods are propensity-score adjustment by subclassification,²³ propensity-score-based kernel weighting methods,^5,6,16 and River’s matching method.²⁴

There are a number of shortcomings associated with the estimation of propensity scores using logistic regression. First, the logistic model is susceptible to model misspecification, requiring assumptions regarding correct variable selection and functional form, including the choice of polynomial terms and multiple-way interactions. If any of these assumptions are incorrect, propensity score estimates can be biased, and balance may not be achieved when conditioning on the estimated PS. Second, implementing a search routine for model specification, such as repeatedly fitting logistic regression models while in/excluding predictor variables, interactions, or transformations of variables can be computationally infeasible or suboptimal. In this context, parametric regression can be limiting in terms of possible model structures that can be searched over, particularly when many potential predictors are present (high-dimensional data). Various machine learning methods for estimating the propensity score that incorporate survey weights will also be our future research interest.

DATA AVAILABILITY STATEMENT

The data that support the findings of this study are available on request from the corresponding author.