Secondary outcome analysis for data from an outcome-dependent sampling design

Abstract

Outcome-dependent sampling (ODS) scheme is a cost-effective way to conduct a study. For a study with continuous primary outcome, an ODS scheme can be implemented where the expensive exposure is only measured on a simple random sample and supplemental samples selected from 2 tails of the primary outcome variable. With the tremendous cost invested in collecting the primary exposure information, investigators often would like to use the available data to study the relationship between a secondary outcome and the obtained exposure variable. This is referred as secondary analysis. Secondary analysis in ODS designs can be tricky, as the ODS sample is not a random sample from the general population. In this article, we use the inverse probability weighted and augmented inverse probability weighted estimating equations to analyze the secondary outcome for data obtained from the ODS design. We do not make any parametric assumptions on the primary and secondary outcome and only specify the form of the regression mean models, thus allow an arbitrary error distribution. Our approach is robust to second- and higher-order moment misspecification. It also leads to more precise estimates of the parameters by effectively using all the available participants. Through simulation studies, we show that the proposed estimator is consistent and asymptotically normal. Data from the Collaborative Perinatal Project are analyzed to illustrate our method.

1 | INTRODUCTION

In many epidemiology studies, the primary outcome variable is easy to obtain, while some exposure variables are expensive or difficult to measure. This motivates statisticians to develop outcome-dependent sampling (ODS) designs, in which the selection probability depends on the primary outcome variable. The main idea of such ODS designs is to concentrate resources on those participants that are more informative in explaining the outcome/exposure relationship. The case-control design has been widely used for studies with a binary primary outcome.¹ Prentice² proposed a case-cohort study design for failure time regression analysis. Zhou et al³ considers an ODS design for data with a continuous primary outcome: In their design, in addition to a simple random sample (SRS) from the full cohort, 2 supplemental SRSs are drawn from 2 tails of the outcome distribution. The initial SRS from the entire cohort provides information about the overall population, and supplemental samples allow investigators to oversample those participants that are more informative about the exposure-response relationship. One example of such ODS design is the Collaborative Perinatal Project (CPP).^3,4 The main purpose of CPP is to study the relationship between in utero exposure to polychlorinated biphenyls (PCBs) and multiple neurological outcomes, including children’s IQ performance. As PCB level is expensive to ascertain, an ODS scheme is adopted: An SRS is taken, and 2 supplemental samples are chosen from 2 tails of the IQ distribution. Related works on ODS to evaluate the association between expensive exposure and the primary outcome variable include Zhou et al,^3,10,11 Weaver and Zhou,⁵ Wang and Zhou,^6,8 Song et al,⁷ and Qin and Zhou.⁹

In any real studies, it is typical that there are more than 1 endpoint of interest. As such, investigators would like to reuse the ODS data to study the association between a secondary outcome and the obtained exposure variable. For example, in the CPP data, investigators are also interested in examining the relationship between PCB level and children’s birth weight. Many prior studies have tried to assess the association between these 2 measures, and yet so far have failed to reach a consistent conclusion.^12–18 With CPP data collected in the first place using an ODS design to evaluate children’s IQ and PCB level, we are interested in adding some evidence to this research problem by developing a valid and precise method for secondary analysis under ODS designs.

In this paper, we develop a method for conducting secondary analysis under continuous outcome ODS design described by Zhou et al.³ As the data obtained from ODS design is not a random sample of the overall population, performing secondary analysis is not straightforward. Ignoring the biased sampling nature of the data could yield an invalid estimate of the true parameters in the general population. The analysis restricted to the participants in the SRS portion is clearly inefficient as it underuses the available data. A significant amount of work was done on secondary analysis in case-control data. This includes the likelihood-based methods,^19–21 inverse probability weighting (IPW),^22,23 and estimating equation.^24,25 However, to the best of our knowledge, there has been no research conducted on the secondary regression analysis in the continuous outcome ODS design framework.

We propose estimating equation approaches to analyze a secondary outcome for data obtained from an ODS design with a continuous primary outcome. The advantage of our approach is that no additional model assumptions are specified. The augmented estimating equation utilizes the available information in the full cohort, and hence increases estimation precision. In addition, our method is computationally stable and fast. The organization of the paper is as follows. In Section 2, we present some notations, data structure and our model under ODS designs. In Section 3, we propose two estimating equations, IPW estimating equation and augmented IPW estimating equation. We give the corresponding asymptotic properties in Section 4. In Section 5, we present the simulation results that compare our proposed estimator to other competing estimators. In Section 6, we apply our methods to CPP data to study the relationship between children’s birth weight and maternal PCB level. We conclude this paper by a brief discussion in Section 7.

2 | DATA STRUCTURE AND MODEL

To fix notation, let Y₁ be the primary continuous outcome variable that the ODS sampling scheme is based on. Let X be the expensive exposure, which is only observed for some participants, and Z be the vector of other covariates that are easy to obtain. Furthermore, let Y₂ denote a continuous secondary response. Our interest lies in inference of the secondary response Y₂ with respect to X adjusting for other covariates Z for data obtained from continuous outcome ODS design.

We partition the domain of Y₁ into a union of 3 mutually exclusive intervals: A₁∪A₂∪A₃ = (−∞, a]∪(a, b]∪(b, +∞). We assume that the underlying data {(Y₁, Y₂, X, Z), i = 1, …, N} are independent and identically distributed random vectors, with N be the size of the full cohort. The ODS design proposed by Zhou et al³ can be regarded as a 2-phase design: In the first phase, information on primary outcome, secondary outcome, and inexpensive covariates are observed for each member of the full cohort. That is, we observe {(Y_1i, Y_2i, Z_i), i = 1, …, N}. In the second phase, the expensive exposure X is measured on an SRS of size n₀ from the full cohort and 2 supplemental SRSs drawn from 2 tails of the distribution of Y₁, ie, supplemental sample of size n₁ from {Y₁∈A₁} and supplemental sample of size n₃ from {Y₁∈A₃}. Let V₀, V₁, V₃ be the index set of SRS, supplemental sample taken from {Y₁ ≤ a}, and supplemental sample taken from {Y₁ > b}, respectively. That is to say, we observe {X_i, i ∈ V₀∪V₁∪V₃} in the second phase. Here, the sample sizes n₀, n₁, and n₃ are fixed by design. Note that we use fixed-size sampling (sampling without replacement) for both the initial SRS and the supplemental samples. When stratum sizes (ie, number of participants in A₁ and A₃) are very large, it is equivalent to independent Bernoulli sampling, as the stratum specific sampling probabilities are effectively fixed.

Let V = V₀∪V₁∪V₃, and let n_V be the size of V. Then, n_V = n₀+ n₁ + n₃. Using terminology from measurement error literature, these n_V observations are called validation sample. In addition, we let $n_{\overline{V}}=N-n_V$. We refer to the $n_{\overline{V}}$ observations as the nonvalidation sample because expensive exposure X is not measured for these individuals. Let $\overline{V}$ represent the index set of the nonvalidation sample, and r_i be the indicator variable of observing X for participant i, then V={i : r=1} and $\overline{V}=\left\{i:r_i=0\right\}$.

The data structure for the ODS design can be summarized as the following:

$$\begin{array}{llll}\text{First phase :}\hfill & \hfill & \left\{Y_{1i},Y_{2i},Z_i\right\},\hfill & i=1,\cdots ,N;\hfill \\ \text{Second phase}:\hfill & \left\{\text{SRS}\right\}\hfill & \left\{X_i\right\},\hfill & i\in V_0;\hfill \\ \hfill & \left\{\text{supplemental sample}1\right\}\hfill & \left\{X_i|Y_{1i}\in A_1\right\},\hfill & i\in V_1;\hfill \\ \hfill & \left\{\text{supplemental sample}2\right\}\hfill & \left\{X_i|Y_{1i}\in A_3\right\},\hfill & i\in V_3.\hfill \end{array}$$ (1)

Let μ_1i = E(Y_1i | X_i, Z_i) and μ_2i = E(Y_2i | X_i, Z_i) denote the conditional expectation of Y_1i and Y_2i given the covariates, respectively. In most problems, we are interested in estimating the regression coefficients (β, γ) from the following 2 models:

$${\mu }_i=\left(\genfrac{}{}{0ex}{}{{\mu }_{1i}}{{\mu }_{2i}}\right)=\left(\genfrac{}{}{0ex}{}{\mathrm{E}\left(Y_{1i}|X_i,Z_i\right)}{\mathrm{E}\left(Y_{2i}|X_i,Z_i\right)}\right)=\left(\genfrac{}{}{0ex}{}{g_1^{-1}\left({\beta }_0+{\beta }_1{X}_i+{\beta }_2{Z}_i\right)}{g_2^{-1}\left({\gamma }_0+{\gamma }_1{X}_i+{\gamma }_2{Z}_i\right)}\right),$$ (2)

where g₁(·) and g₂(·) are specified link functions, such as g(x) = x for linear regression. Without loss of generality, we use the identity link g₁(x) = g₂(x) = x to illustrate our ideas throughout the paper. It is also worth mentioning that no distributional assumptions are made about y_1i and y_2i. Since analysis on secondary outcome is our primary goal, we focus on developing an inference procedure for (γ₀, γ₁, γ₂).

3 | ESTIMATING EQUATION APPROACH

3.1 | Inverse probability weighted estimating equation

Let ξ = (β, γ). Since we do not make any parametric assumptions about Y₁ and Y₂, no likelihood-based approaches are available. Let e_i = (e_1i, e_2i)′ = (Y_1i − μ_1i, Y_2i − β₂)′. Following the ideas from Horvitz and Thompson, Liang and Zeger, and Zhao et al,^26–28 we first propose an IPW estimating equation that uses the validation sample only:

$$S_1\left(\xi ,Q,\pi \right)={\displaystyle \sum _{i\in V}{s}_{1i}\left(\xi ,Q,\pi \right)={\displaystyle \sum _{i\in V}\frac{1}{{\pi }_i}{D}_i^T{Q}^{-1}{e}_i={\displaystyle \sum _{i=1}^N\frac{r_i}{{\pi }_i}{D}_i^T{Q}^{-1}{e}_i=0},}}$$ (3)

where Q is the covariance matrix of (Y₁, Y₂), ie, Q = Cov(Y₁, Y₂), π_i is the probability of being selected into the validation sample for each participant i, and

$$D_i=\frac{\partial {\mu }_i}{\partial {\left(\beta ,\gamma \right)}^T}=\left(\begin{array}{llllll}1\hfill & X_i\hfill & Z_i\hfill & 0\hfill & 0\hfill & 0\hfill \\ 0\hfill & 0\hfill & 0\hfill & 1\hfill & X_i\hfill & Z_i\hfill \end{array}\right).$$

The selection probability π_i is a function of the observed outcome value Y_1i. Let V_0,k be the index set of the observations in the SRS that belongs to the kth stratum A_k. That is, V₀ = V_0,1 ∪ V_0,2 ∪ V_0,3. Then, π_i can be expressed as follows:

$${\pi }_i=Pr\left(r_i=1|Y_{1i}\right)=\{\begin{array}{ll}Pr\left(i\text{in}{V}_{0,1}\text{or}{V}_1\right)\hfill & \text{if}{Y}_{1i}\le a,\hfill \\ Pr\left(i\text{in}{V}_{0,2}\right)\hfill & \text{if}a<Y_{1i}\le b,\hfill \\ Pr\left(i\text{in}{V}_{0,3}\text{or}{V}_3\right)\hfill & \text{if}{Y}_{1i}>b.\hfill \end{array}$$

For complex sampling designs, such as the ODS design described in this paper, it is difficult to express π_i in explicit forms. We cannot directly solve Equation 3 as the covariance matrix Q, and the selection probability π is unknown. Hence, the general idea is to plug in consistent estimators of Q and π into S₁(ξ, Q, π) to get ${\widehat{S}}_1\left(\xi \right)=S_1\left(\xi ,\widehat{Q},\widehat{\pi }\right)$ and then solve the equation Ŝ₁(ξ) = 0.

Because Y₁ and Y₂ are observed for each member of the full cohort, a consistent estimator of the covariance matrix Q is the sample covariance derived from the full cohort. That is,

$$\widehat{Q}=\left(\begin{array}{cc}\frac{1}{N-1}{\displaystyle {\sum }_{i=1}^N{\left(Y_{1i}-{\overline{Y}}_1\right)}^2}& \frac{1}{N-1}{\displaystyle {\sum }_{i=1}^N\left(Y_{1i}-{\overline{Y}}_1\right)\left(Y_{2i}-{\overline{Y}}_2\right)}\\ \frac{1}{N-1}{\displaystyle {\sum }_{i=1}^N\left(Y_{1i}-{\overline{Y}}_1\right)\left(Y_{2i}-{\overline{Y}}_2\right)}& \frac{1}{N-1}{\displaystyle {\sum }_{i=1}^N\left(Y_{2i}-{\overline{Y}}_2\right)}\end{array}\right),$$

where ${\overline{Y}}_1$ and ${\overline{Y}}_2$ are sample means for Y₁ and Y₂, respectively.

Let N_k, k = 1, 2, 3 be the number of observations in the full cohort that belong to the kth stratum A_k. Similarly, let n_0,k, k = 1, 2, 3 be the number of observations in the SRS that belong to stratum A_k. That is, N = N₁+N₂+N₃, n₀ = n_0,1+n_0,2+n_0,3. Then for each participant, the observed probability of being sampled within its respective strata A_k can be written as

$${\widehat{\pi }}_i=\{\begin{array}{ll}\left(n_{0,1}+n_1\right)/N_1\hfill & \text{if}{Y}_{1i}\le a,\hfill \\ n_{0,2}/N_2\hfill & \text{if}a<Y_{1i}\le b,\hfill \\ \left(n_{0,3}+n_3\right)/N_3\hfill & \text{if}{Y}_{1i}>b.\hfill \end{array}$$

It is straightforward to show that ${\widehat{\pi }}_i$ is a consistent estimator for π_i. Hence, our first proposed estimator ${\widehat{\xi }}_{\mathit{IPW}}$ satisfies the following estimating Equation 4 and can be obtained using Newton-Raphson algorithm.

$${\widehat{S}}_1\left(\xi \right)={\displaystyle \sum _{i\in V}{\widehat{s}}_{1i}\left(\xi \right)={\displaystyle \sum _{i\in V}\frac{1}{{\widehat{\pi }}_i}{D}_i^T{\widehat{Q}}^{-1}{e}_i={\displaystyle \sum _{i=1}^N\frac{r_i}{{\widehat{\pi }}_i}{D}_i^T{\widehat{Q}}^{-1}{e}_i=0.}}}$$ (4)

3.2 | Augmented inverse probability weighted estimating equation

The weighted estimating equation 4 described above may not be precise as it uses only the information contained in the validation sample where the expensive exposure X is observed. Following the ideas from Robins et al,²⁹ an augmented estimating equation can be used. Let u_i be any kernel function, and let h(y_1i, y_2i, z_i) be any function, then

$${\displaystyle \sum _{i=1}^N\left[\frac{r_i}{{\pi }_i}{u}_i+\left(1-\frac{r_i}{{\pi }_i}\right)h\left(y_{1i},y_{2i},z_i\right)\right]}=0$$

is an augmented estimating equation. Any choice of h(·) would lead to a consistent estimate of the parameters. This comes from the fact that

$$E\left[\left(1-\frac{r_i}{{\pi }_i}\right)h\left(y_{1i},y_{2i},z_i\right)\right]=E_{y_{1i},y_{2i},z_i}\left[h\left(y_{1i},y_{2i},z_i\right)E_{r_i|y_{1i},y_{2i},z_i}\left(1-\frac{r_i}{{\pi }_i}\right)\right]=0.$$

However, an optimal choice of h(·) would improve the estimation precision. From Robins et al,²⁹ it is shown that the optimal h(·) should be the conditional expectation of the kernel function given the observed data, ie, h(y_1i, y_2i, z_i) = E(u_i|y_1i, y_2i, z_i).

Following this line of reasoning, we propose the following augmented IPW (AIPW) estimating equation based on estimating equation 3:

$$S_2\left(\xi ,Q,\pi \right)={\displaystyle \sum _{i=1}^N{s}_{2i}\left(\xi ,Q,\pi \right)={\displaystyle \sum _{i=1}^N\left\{\frac{r_i}{{\pi }_i}{D}_i^T{Q}^{-1}{e}_i+\left(1-\frac{r_i}{{\pi }_i}\right)E_{X_i|Y_{1i},Y_{2i},Z_i}\left[D_i^T{Q}^{-1}{e}_i\right]\right\}}}=0.$$ (5)

Notice that the augmented estimating equation 5 incorporates all the information available in the full cohort, including those observations in the nonvalidation sample. To use estimating equation 5, one needs to assume a form of the conditional moments, ie, E(X_i | Y_1i, Y_2i, Z_i) and $\mathrm{E}\left(X_i^2|Y_{1i},Y_{2i},Z_i\right)$. As the expensive exposure variable is often on a continuous scale, it is reasonable to assume that

$$\mathrm{E}\left(X_i|Y_{1i},Y_{2i},Z_i\right)={\varphi }_0+{\varphi }_1{Y}_{1i}+{\varphi }_2{Y}_{2i}+{\varphi }_3{Z}_i$$

and

$$\text{Var}\left(X_i|Y_{1i},Y_{2i},Z_i\right)={\sigma }^2.$$

Then, the second-order moment can be expressed through the conditional mean and conditional variance as $\mathrm{E}\left(X_i^2|Y_{1i},Y_{2i},Z_i\right)=\text{Var}\left(X_i|Y_{1i},Y_{2i},Z_i\right)+{\left[\mathrm{E}\left(X_i|Y_{1i},Y_{2i},Z_i\right)\right]}^2$.

Let ϕ = (ϕ₀, ϕ₁, ϕ₂, ϕ₃). We notice that S₂(ξ, Q, π, ϕ, σ²) has several components, in which ξ is the parameter of interest and (Q, π, ϕ, σ²) are nuisance parameters. The following summarizes the steps on how to conduct the analysis:

1. Fit a linear regression to obtain parameter estimates for (ϕ, σ²) based on the SRS portion of the data.

2. As outlined in Section 3.1, obtain the consistent estimator $\widehat{Q}$, $\widehat{\pi }$ for Q and π.

3. Plug $\left(\widehat{Q},\widehat{\pi },\widehat{\varphi },{\widehat{\sigma }}^2\right)$ into S₂ to obtain ${\widehat{S}}_2\left(\xi \right)=S_2\left(\xi ,\widehat{Q},\widehat{\pi },\widehat{\varphi },{\widehat{\sigma }}^2\right)$. Our second proposed estimator ${\widehat{\xi }}_{\mathit{AIPW}}$ is the solution to the following augmented IPW estimating equation:

   $${\widehat{S}}_2\left(\xi \right)={\displaystyle \sum _{i=1}^N{\widehat{s}}_{2i}\left(\xi \right)={\displaystyle \sum _{i=1}^N\left\{\frac{r_i}{{\widehat{\pi }}_i}{D}_i^T{\widehat{Q}}^{-1}{e}_i+\left(1-\frac{r_i}{{\widehat{\pi }}_i}\right){\widehat{E}}_{X_i|Y_{1i},Y_{2i},Z_i}\left[D_i^T{\widehat{Q}}^{-1}{e}_i\right]\right\}}}=0.$$ (6)

Note that how to estimate (Q, π, ϕ, σ²) does not influence the asymptotic distribution of ${\widehat{\xi }}_{\mathit{AIPW}}$ as long as the nuisance parameter estimates are root-N consistent. In Appendix A, we use a lemma from Yuan and Jennrich³⁰ to show why this is the case.

4 | ASYMPTOTIC RESULTS

In this section, we will present theorems regarding the consistency and asymptotic normality for our proposed estimators ${\widehat{\xi }}_{\mathit{IPW}}$ and ${\widehat{\xi }}_{\mathit{AIPW}}$. Let ξ_* be the true values of the parameters of interest, and let (Q_*, π_*, ϕ_*, ${\sigma }_{\ast }^2$) denote the true values of the nuisance parameters (Q, π, ϕ, σ²). In addition, we let E_k denote the conditional expectation given Y₁ ∈ A_k. That is, for any function f(·), E_k[f(Y₁, Y₂, X, Z)] = E[f(Y₁, Y₂, X, Z) | Y₁∈A_k]. Under regularity conditions outlined in Appendix A, assuming that n₀/n_V→ρ₀>0 and n_k/n_V→ρ_k≥0 for k = 1, 3, the following theorems hold for ${\widehat{\xi }}_{\mathit{IPW}}$ and ${\widehat{\xi }}_{\mathit{AIPW}}$:

Theorem 1

${\widehat{\xi }}_{\mathit{IPW}}$ and ${\widehat{\xi }}_{\mathit{AIPW}}$ converge in probability to ξ_*.

Theorem 2

Let θ denote the nuisance parameters (Q, π), then ${\widehat{\xi }}_{\mathit{IPW}}$ has the following asymptotic distributional properties:

$$\sqrt{n_V}\left({\widehat{\xi }}_{\mathit{IPW}}-{\xi }_{\ast }\right)\stackrel{D}{\to }N\left(0,I_1^{-1}\left({\xi }_{\ast },{\theta }_{\ast }\right){\sum }_1\left({\xi }_{\ast },{\theta }_{\ast }\right)I_1^{-1}\left({\xi }_{\ast },{\theta }_{\ast }\right)\right),$$ (7)

where

$$I_1\left(\xi ,\theta \right)=-{\rho }_{0}E\left[\frac{\partial s_1\left(\xi ,\theta \right)}{\partial {\xi }^T}\right]-{\rho }_1{E}_1\left[\frac{\partial s_1\left(\xi ,\theta \right)}{\partial {\xi }^T}\right]-{\rho }_3{E}_3\left[\frac{\partial s_1\left(\xi ,\theta \right)}{\partial {\xi }^T}\right]$$

and

$${\sum }_1\left(\xi ,\theta \right)={\rho }_{0}E\left[s_1\left(\xi ,\theta \right)s_1{\left(\xi ,\theta \right)}^T\right]+{\rho }_1{E}_1\left[s_1\left(\xi ,\theta \right)s_1{\left(\xi ,\theta \right)}^T\right]+{\rho }_3{E}_3\left[s_1\left(\xi ,\theta \right)s_1{\left(\xi ,\theta \right)}^T\right].$$

Replacing the population quantities with the sample quantities, a consistent estimator for the asymptotic variance-covariance matrix can be obtained as

$${\widehat{I}}_1^{-1}\left({\widehat{\xi }}_{\mathit{IPW}},\widehat{\theta }\right){\widehat{\sum }}_1\left({\widehat{\xi }}_{\mathit{IPW}},\widehat{\theta }\right){\widehat{I}}_1^{-1}\left({\widehat{\xi }}_{\mathit{IPW}},\widehat{\theta }\right),$$

where ${\widehat{I}}_1\left(\xi ,\theta \right)=-\frac{1}{n_V}{\displaystyle {\sum }_{i\in V}\frac{\partial s_{1i}\left(\xi ,\theta \right)}{\partial {\xi }^T}}$ and ${\widehat{\sum }}_1\left(\xi ,\theta \right)=\frac{1}{n_V}{\displaystyle {\sum }_{i\in V}{s}_{1i}\left(\xi ,\theta \right)s_{1i}{\left(\xi ,\theta \right)}^T}$.

Theorem 3

Let n = (Q, π, ϕ, σ²) denote all the nuisance parameters. ${\widehat{\xi }}_{\mathit{AIPW}}$ has the following asymptotic distributional properties:

$$\sqrt{N}\left({\widehat{\xi }}_{\mathit{AIPW}}-{\xi }_{\ast }\right)\stackrel{D}{\to }N\left(0,I_2^{-1}\left({\xi }_{\ast },{\eta }_{\ast }\right){\sum }_2\left({\xi }_{\ast },{\eta }_{\ast }\right)I_2^{-1}\left({\xi }_{\ast },{\eta }_{\ast }\right)\right),$$ (8)

where $I_2\left(\xi ,\eta \right)=-E\left[\frac{\partial }{\partial {\xi }^T}{s}_2\left(\xi ,\eta \right)\right]$, and ${\sum }_2\left(\xi ,\eta \right)=E\left[s_2\left(\xi ,\eta \right)s_2{\left(\xi ,\eta \right)}^T\right]$.

Replacing the population quantities with the sample quantities, a consistent estimator for the asymptotic variance-covariance matrix can be obtained as:

$${\widehat{I}}_2^{-1}\left({\widehat{\xi }}_{\mathit{AIPW}},\widehat{\eta }\right){\widehat{\sum }}_2\left({\widehat{\xi }}_{\mathit{AIPW}},\widehat{\eta }\right){\widehat{I}}_2^{-1}\left({\widehat{\xi }}_{\mathit{AIPW}},\widehat{\eta }\right),$$

where ${\widehat{I}}_2\left(\xi ,\eta \right)=-\frac{1}{N}{\displaystyle {\sum }_{i=1}^N\frac{\partial }{\partial {\xi }^T}{s}_{2i}\left(\xi ,\eta \right)}$ and ${\widehat{\sum }}_2\left(\xi ,\eta \right)=\frac{1}{N}{\displaystyle {\sum }_{i=1}^N{s}_{2i}\left(\xi ,\eta \right)s_{2i}{\left(\xi ,\eta \right)}^T}$.

Outline of the proofs are in Appendix A. We will apply a result of Foutz³¹ to prove the consistency of our estimator and use Taylor expansion together with Slutsky theorem to prove asymptotic normality.

5 | SIMULATION STUDIES

In this section, we conduct extensive simulation studies to evaluate the finite sample performance of our proposed estimators. There are 5 competing estimators: (1) ${\widehat{\xi }}_{\mathit{SRS}}$ denotes the regression estimator based on SRS portion of validation sample. (2) ${\widehat{\xi }}_R$ denotes the regression estimator from an SRS of the same size as the validation sample. Notice that this estimator is not available in practice, because when the existing data are obtained from an ODS design, it is impossible to obtain a SRS of the same size of the ODS sample. We include ${\widehat{\xi }}_R$ in the table for comparison purpose only. (3) ${\widehat{\xi }}_{\mathit{IPW}}$ denotes the estimate from our inverse probability weighted estimating equation proposed in Section 3.1. (4) ${\widehat{\xi }}_{\mathit{AIPW}}$ denotes the estimate from our proposed augmented IPW estimating equation in Section 3.2. (5) ${\widehat{\xi }}_{\mathit{SPML}}$ denotes a semiparametric maximum likelihood estimator similar to Jiang et al.²⁰ In deriving ${\widehat{\xi }}_{\mathit{SPML}}$, we assume that (Y₁, Y₂) is bivariate normal and use estimated likelihood technique to deal with the nuisance functions. The details are shown in Appendix B.

The data are generated from the following models:

$$\begin{gathered} Y_1={\beta }_0+{\beta }_{1}X+{\beta }_{2}Z+e, \\ {Y}_2={\gamma }_0+{\gamma }_{1}X+{\gamma }_{2}Z+\epsilon , \end{gathered}$$ (9)

where X ~ N(0, 1), Z ~ Bernoulli(0.45) and (e, ε) follows a bivariate normal distribution with $\mathit{var}\left(e\right)={\sigma }_1^2$, $\mathit{var}\left(\epsilon \right)={\sigma }_2^2$, cov(e,ε) = ρσ₁σ₂. The true parameter values are β₀ = 1, β₂ = − 0.5, γ₀ = 1, γ₂ = − 0.5, σ₁ = σ₂ = 1, and ρ = 0.8. We allow β₁ and γ₁ to take value 0 or 0.5.

Let ${\mu }_{Y_1}$ and ${\sigma }_{Y_1}$ be the sample mean and standard deviation for the primary outcome Y₁ observed in the full cohort. In continuous outcome ODS design, we first select an SRS of size n₀, then a supplemental sample of size n₁ is chosen from $\left\{Y_1\le {\mu }_{Y_1}-a{\sigma }_{Y_1}\right\}$ and a supplemental sample of size n₃ chosen from $\left\{Y_1\ge {\mu }_{Y_1}+a{\sigma }_{Y_1}\right\}$. The validation sample has size n_V = n₀ + n₁ + n₃. We consider the following 2 settings: (1) n₀ = 200, n₁ = n₃ = 100. (2) n₀ = 300, n₁ = n₃ = 50. In addition, we also vary the cutoff points of the strata by considering different values of a, ie, a = 1 or 1.5.

Tables 1 and 2 show the simulation results based on 1000 independent replications. The full cohort size is N = 3000. In Table 1, (n₀, n₁, n₃) = (200, 100, 100). In Table 2, (n₀, n₁, n₃) = (300, 50, 50). The average of parameter estimates (Mean), empirical variance of parameter estimates across all simulations (VAR), average of the variance estimator $\left(\widehat{\text{VAR}}\right)$, and 95% CI coverage are reported. In addition, we show the sample relative efficiency of all estimators relative to ${\widehat{\xi }}_{\mathit{IPW}}$ in terms of estimating γ₁. The sample relative efficiency is defined as the ratio of empirical variance, ie, $SR{E}_{\mathit{AIPE}:\mathit{IPW}}=\mathit{var}\left({\widehat{\xi }}_{\mathit{IPW}}\right)/\mathit{var}\left({\widehat{\xi }}_{\mathit{AIPW}}\right)$.

TABLE 1.

Simulation results based on 1000 simulations with n₀ = 200, n₁ = n₃ = 100, the validation sample size is n_V = 400, the full cohort size is N = 3000

α	β1	γ1	Methods	${\widehat{\beta }}_1$				${\widehat{\gamma }}_1$
				Mean	VAR	$\widehat{\text{VAR}}$	CI	Mean	VAR	$\widehat{\text{VAR}}$	CI	SRE
1.0	0	0	ξSRS	−0.002	0.0049	0.0050	0.948	−0.001	0.0046	0.0050	0.966	0.58
			ξR	0.000	0.0024	0.0025	0.958	0.001	0.0026	0.0025	0.950	1.04
			ξIPW	−0.001	0.0020	0.0021	0.947	0.002	0.0027	0.0027	0.947	1.00
			ξAIPW	0.000	0.0021	0.0021	0.946	0.002	0.0027	0.0027	0.945	1.00
			ξSPML	0.001	0.0015	0.0014	0.954	0.001	0.0019	0.0018	0.940	1.42
		0.5	ξSRS	0.002	0.0048	0.0050	0.956	0.502	0.0049	0.0050	0.955	0.53
			ξR	0.001	0.0026	0.0025	0.949	0.501	0.0026	0.0025	0.950	1.00
			ξIPW	0.003	0.0020	0.0020	0.952	0.502	0.0026	0.0026	0.947	1.00
			ξAIPW	0.003	0.0014	0.0014	0.956	0.504	0.0016	0.0016	0.938	1.63
			ξSPML	0.002	0.0010	0.0010	0.950	0.500	0.0013	0.0013	0.955	2.00
	0.5	0	ξSRS	0.501	0.0053	0.0050	0.944	0.001	0.0049	0.0050	0.952	0.55
			ξR	0.501	0.0026	0.0025	0.943	0.001	0.0026	0.0025	0.939	1.04
			ξIPW	0.503	0.0023	0.0025	0.958	0.003	0.0027	0.0028	0.947	1.00
			ξAIPW	0.503	0.0018	0.0018	0.936	0.000	0.0018	0.0018	0.942	1.50
			ξSPML	0.499	0.0017	0.0017	0.942	0.002	0.0014	0.0014	0.944	1.93
		0.5	ξSRS	0.499	0.0052	0.0050	0.949	0.499	0.0050	0.0050	0.948	0.56
			ξR	0.500	0.0025	0.0025	0.947	0.498	0.0026	0.0025	0.950	1.08
			ξIPW	0.501	0.0022	0.0025	0.960	0.500	0.0028	0.0028	0.952	1.00
			ξAIPW	0.502	0.0020	0.0020	0.951	0.502	0.0025	0.0023	0.943	1.12
			ξSPML	0.500	0.0016	0.0017	0.958	0.500	0.0017	0.0018	0.955	1.65
1.5	0.5	0	ξSRS	0.500	0.0048	0.0052	0.968	0.002	0.0046	0.0052	0.960	0.67
			ξR	0.498	0.0024	0.0025	0.956	−0.001	0.0024	0.0025	0.940	1.29
			ξIPW	0.504	0.0027	0.0029	0.955	0.004	0.0031	0.0033	0.953	1.00
			ξAIPW	0.505	0.0022	0.0022	0.947	0.001	0.0022	0.0022	0.949	1.41
			ξSPML	0.504	0.0017	0.0017	0.948	0.006	0.0014	0.0014	0.947	2.21
		0.5	ξSRS	0.496	0.0050	0.0050	0.947	0.500	0.0049	0.0050	0.956	0.61
			ξR	0.502	0.0025	0.0025	0.953	0.502	0.0025	0.0025	0.950	1.20
			ξIPW	0.501	0.0028	0.0029	0.954	0.504	0.0030	0.0032	0.955	1.00
			ξAIPW	0.503	0.0025	0.0025	0.944	0.505	0.0027	0.0027	0.947	1.11
			ξSPML	0.501	0.0017	0.0018	0.950	0.501	0.0018	0.0018	0.953	1.67

Abbreviation: SRE, sample relative efficiency. Results are based on the model Y₁=β₀+β₁X+β₂Z+e, Y₂=Y₀+Y₁X+Y₂Z+ε, where X~N(0, 1), Z~Bernoulli(0.45) and (e, ε) follow a bivariate normal distribution with $\mathit{var}\left(e\right)={\sigma }_1^2$, $\mathit{var}\left(\epsilon \right)={\sigma }_2^2$, cov(e, ε)=ρσ₁σ₂; the true parameter values are β₀=1, β₂=−0.5, γ₀=1, γ₂=−0.5, σ₁=σ₂=1, ρ=0.8. The cutoff points for the outcome-dependent sampling design are ${\mu }_{Y_1}-a{\sigma }_{Y_1}$ and ${\mu }_{Y_1}+a{\sigma }_{Y_1}$. ξ_SRS denotes the regression estimator based on simple random sample portion of the validation sample. ξ_R denotes the regression estimator from a simple random sample of the same size as the validation sample. ξ_IPW denotes the estimate from our inverse probability weighted (IPW) estimating equation. ξaipw denotes the estimate from augmented IPW (AIPW) estimating equation. ξ_SPWL is a semiparametric maximum likelihood (SPML) estimator similar to Jiang et al,²⁰ which models (Y₁, Y₂) parametrically using a bivariate normal distribution.

TABLE 2.

Simulation results based on 1000 simulations with n₀ = 300, n₁ = n₃ = 50, the validation sample size is n_V = 400, the full cohort size is N=3000

α	β1	γ1	Methods	${\widehat{\beta }}_1$			${\widehat{\gamma }}_1$
				Mean	VAR	$\widehat{\text{VAR}}$	Mean	VAR	$\widehat{\text{VAR}}$	SRE
1.0	0	0	ξSRS	0.000	0.0034	0.0034	−0.001	0.0036	0.0034	0.69
			ξR	0.000	0.0024	0.0025	0.000	0.0025	0.0025	1.00
			ξIPW	0.001	0.0021	0.0020	0.000	0.0025	0.0023	1.00
			ξAIPW	0.001	0.0021	0.0020	0.000	0.0025	0.0023	1.00
			ξSPML	0.001	0.0018	0.0018	0.000	0.0021	0.0021	1.19
		0.5	ξSRS	−0.004	0.0034	0.0034	0.496	0.0034	0.0034	0.65
			ξR	−0.001	0.0023	0.0025	0.500	0.0024	0.0025	0.92
			ξIPW	−0.001	0.0020	0.0020	0.498	0.0022	0.0023	1.00
			ξAIPW	−0.001	0.0013	0.0014	0.500	0.0014	0.0014	1.57
			ξSPML	0.001	0.0012	0.0012	0.498	0.0013	0.0014	1.69
	0.5	0	ξSRS	0.501	0.0032	0.0034	0.001	0.0034	0.0034	0.62
			ξR	0.499	0.0024	0.0025	0.000	0.0025	0.0025	0.84
			ξIPW	0.502	0.0018	0.0022	0.001	0.0021	0.0024	1.00
			ξAIPW	0.502	0.0014	0.0015	0.000	0.0016	0.0015	1.31
			ξSPML	0.500	0.0014	0.0015	0.001	0.0014	0.0014	1.50
		0.5	ξSRS	0.500	0.0032	0.0034	0.498	0.0032	0.0033	0.66
			ξR	0.499	0.0025	0.0025	0.499	0.0025	0.0025	0.84
			ξIPW	0.502	0.0018	0.0022	0.501	0.0021	0.0024	1.00
			ξAIPW	0.503	0.0017	0.0017	0.502	0.0019	0.0018	1.11
			ξSPML	0.500	0.0016	0.0016	0.499	0.0016	0.0018	1.31
1.5	0.5	0	ξSRS	0.500	0.0036	0.0034	0.001	0.0035	0.0034	0.74
			ξR	0.498	0.0025	0.0025	−0.001	0.0026	0.0025	1.00
			ξIPW	0.501	0.0023	0.0022	0.000	0.0026	0.0024	1.00
			ξAIPW	0.502	0.0018	0.0016	0.000	0.0018	0.0017	1.44
			ξSPML	0.500	0.0014	0.0014	0.002	0.0014	0.0014	1.86
		0.5	ξSRS	0.499	0.0035	0.0034	0.499	0.0034	0.0034	0.68
			ξR	0.502	0.0026	0.0025	0.501	0.0027	0.0025	0.85
			ξIPW	0.500	0.0021	0.0022	0.500	0.0023	0.0024	1.00
			ξAIPW	0.501	0.0018	0.0018	0.501	0.0019	0.0019	1.21
			ξSPML	0.499	0.0014	0.0015	0.499	0.0017	0.0017	1.35

Abbreviation: SRE, sample relative efficiency. Results are based on the model Y₁=β₀+β₁X+β₂Z+e, Y₂=γ₀+γ₁X+γ₂Z+ε, where X~N(0, 1), Z~Bernoulli(0.45) and (e, ε) follow a bivariate normal distribution with $\mathit{var}\left(e\right)={\sigma }_1^2$, $\mathit{var}\left(\epsilon \right)={\sigma }_2^2$, cov(e,ε)=ρσ₁σ₂; the true parameter values are β₀=1, β₂=−0.5, γ₀=1, γ₂=−0.5, σ₁=σ₂=1, ρ=0.8. The cutoff points for the outcome-dependent sampling design are ${\mu }_{Y_1}-a{\sigma }_{Y_1}$ and ${\mu }_{Y_1}+a{\sigma }_{Y_1}$ ξ_SRS denotes the regression estimator based on simple random sample (SRS) portion of the validation sample. ξ_R denotes the regression estimator from an SRS of the same size as the validation sample. ξ_IPW denotes the estimate from our inverse probability weighted (IPW) estimating equation. ξ_AIPW denotes the estimate from augmented IPW (AIPW) estimating equation. ξ_SPMI is a semiparametric maximum likelihood (SPML) estimator similar to Jiang et al,²⁰ which models (Y₁, Y₂) parametrically using a bivariate normal distribution.

From Tables 1 and 2, we have the following observations: (1) All 5 estimators yield virtually unbiased estimates. (2) ξ_SPML has the most precise estimate. However, as later shown in the simulation studies, the validity of ${\widehat{\xi }}_{\mathit{SPML}}$ depends heavily on the correctness of bivariate normal assumption and is hence not robust. (3) Among ${\widehat{\xi }}_{\mathit{SRS}}$, ${\widehat{\xi }}_{\mathit{IPW}}$, and ${\widehat{\xi }}_{\mathit{AIPW}}$, the augmented estimating equation estimator ${\widehat{\xi }}_{\mathit{AIPW}}$ is the most precise in all settings except for when β₁ = y₁ = 0, where the performance of ${\widehat{\xi }}_{\mathit{IPW}}$ and ${\widehat{\xi }}_{\mathit{AIPW}}$ are similar. For example, when (n₀, n₁, n₃) = (200, 100, 100), a = 1, β₁ = 0, γ₁ = 0.5, the empirical variance estimating γ₁ is 0.0016 for ${\widehat{\xi }}_{\mathit{AIPW}}$, which is smaller than 0.0026 for ${\widehat{\xi }}_{\mathit{IPW}}$ and 0.0049 for ${\widehat{\xi }}_{\mathit{SRS}}$. The precision gain comes from the fact that ${\widehat{\xi }}_{\mathit{IPW}}$ and ${\widehat{\xi }}_{\mathit{AIPW}}$ use more participants than ${\widehat{\xi }}_{\mathit{SRS}}$. (4) For all estimators, averages of the variance estimator is very close to the empirical variance (ie, $\widehat{\text{VAR}}$ is close to VAR). (5) The 95% CI coverage is close to 0.95, which implies that the asymptotic normal approximation works well in these finite sample size settings. (6) When the cutoff points are further out (ie, a = 1 versus a = 1.5), the precision gains of ${\widehat{\xi }}_{\mathit{IPW}}$ and ${\widehat{\xi }}_{\mathit{AIPW}}$ over ${\widehat{\xi }}_{\mathit{SRS}}$ are slightly lower (Table 1). The ODS sample is more enriched with a=1.5. However, this more enriched study design is offset by the highly variable IPW weighting distribution that results in precision loss. (7) Comparing the results across Tables 1 and 2 in terms of estimating γ₁, we find that, for a given validation sample size (n_V = 400), when SRS sample size is larger (ie, n₀ = 200 versus n₀ = 300), the variance of ${\widehat{\xi }}_{\mathit{IPW}}$ and ${\widehat{\xi }}_{\mathit{AIPW}}$ decreases. However, the variance of ${\widehat{\xi }}_{\mathit{SRS}}$ has a faster decreasing rate. That is, the precision gains of ${\widehat{\xi }}_{\mathit{IPW}}$ and ${\widehat{\xi }}_{\mathit{AIPW}}$ over ${\widehat{\xi }}_{\mathit{SRS}}$ is smaller when the SRS sample size is larger.

We investigate the scenario where the error term ε is not normally distributed. The simulation set up is the same as Table 1 except for the error term. We assume that e ~ N(0, 1), ε is a gamma distribution with shape parameter 2, rate parameter 1, then normalized to have mean 0 and variance 1. This error term is right skewed. From Table 3, we see that our proposed estimators are more robust to model misspecification than ${\widehat{\xi }}_{\mathit{SPML}}$. When true γ₁ = 0.5, the 95% CI coverage rate is poor for ${\widehat{\xi }}_{\mathit{SPML}}$. ${\widehat{\xi }}_{\mathit{SPML}}$ also has larger empirical variance than ${\widehat{\xi }}_{\mathit{IPW}}$ and ${\widehat{\xi }}_{\mathit{AIPW}}$ as it misspecified the distribution of Y₂. Another limitation of ${\widehat{\xi }}_{\mathit{SPML}}$ is that it is subject to “curse of dimensionality” as nonparametric method is used to estimate the nuisance function. Therefore, the method cannot be directly applied when the dimension of Z is relatively high, ie, > 3. In addition, the method does not have a natural extension when Z has both discrete and continuous components.

TABLE 3.

Simulation results when ε is not normal. The full cohort size is N=3000, (n₀, n₁, n₃)=(200, 100, 100)

α	β1	γ1	Methods	${\widehat{\gamma }}_1$
				Mean	VAR	$\widehat{\text{VAR}}$	CI
1.0	0	0	ξIPW	0.003	0.0038	0.0036	0.941
			ξAIPW	0.003	0.0039	0.0036	0.926
			ξSPML	0.001	0.0027	0.0026	0.960
		0.5	ξIPW	0.499	0.0038	0.0037	0.946
			ξAIPW	0.502	0.0037	0.0035	0.932
			ξSPML	0.493	0.0055	0.0041	0.871
	0.5	0	ξIPW	0.002	0.0036	0.0034	0.933
			ξAIPW	0.001	0.0032	0.0030	0.930
			ξSPML	−0.001	0.0021	0.0021	0.953
		0.5	ξIPW	0.501	0.0031	0.0032	0.954
			ξAIPW	0.505	0.0026	0.0027	0.940
			ξSPML	0.494	0.0038	0.0029	0.899

Results are based on the model Y₁=β₀+β₁X+β₂Z+e, Y₂=γ₀+γ₁X+γ₁Z+ε, where X~N(0, 1), Z~Bernoulli(0.45), and e~N(0, 1), ε is a gamma distribution with shape parameter 2, rate parameter 1, normalized to have mean 0 and variance 1. The true parameter values are β₀=1, β₂=−0.5, γ₀=1, γ₂=−0.5. The cutoff points for the outcome-dependent sampling design are ${\mu }_{Y_1}-a{\sigma }_{Y_1}$ and ${\mu }_{Y_1}+a{\sigma }_{Y_1}$. ξ_IPW denotes the estimate from our inverse probability weighted (IPW) estimating equation; ξaipw denotes the estimate from augmented IPW (AIPW) estimating equation; ξ_SPML is a semiparametric maximum likelihood (SPML) estimator similar to Jiang et al,²⁰ which models (Y₁, Y₂) parametrically using a bivariate normal distribution.

We further investigate the performance of our proposed estimators in estimating γ₁ under different combinations of the SRS sample and supplemental samples. The simulation set up is as follows: a = 1.0,β₀ = 1,β₁ = 0.5,β₂ = −0.5,γ₀ =1,γ₁ = 0.5,γ₂ = −0.5,σ₁ = σ₂ = 1,ρ = 0.8. The full cohort has size N=3000. We fix the validation sample to have size n_V =400 and vary (n₀, n₁, n₃). Figure 1 shows the sample relative efficiency of ${\widehat{\xi }}_{\mathit{AIPW}}$ and ${\widehat{\xi }}_{\mathit{SRS}}$ relative to ${\widehat{\xi }}_{\mathit{IPW}}$ over a wide range of proportion of the SRS sample in the validation sample (n₀/n_V). We confirm that when SRS size is larger, there is larger precision gain of ${\widehat{\xi }}_{\mathit{AIPW}}$ over ${\widehat{\xi }}_{\mathit{IPW}}$, while the precision gain of ${\widehat{\xi }}_{\mathit{IPW}}$ and ${\widehat{\xi }}_{\mathit{AIPW}}$ over ${\widehat{\xi }}_{\mathit{SRS}}$ is smaller.

FIGURE 1.

Sample relative efficiencies (SREs) comparing ${\widehat{\xi }}_{\mathit{AIPW}}$ and ${\widehat{\xi }}_{\mathit{SRS}}$ to ${\widehat{\xi }}_{\mathit{IPW}}$ in terms of estimating γ₁, under various combinations of simple random sample (SRS) and supplemental samples. The SRE is defined as $SR{E}_{\mathit{AIPW}:\mathit{IPW}}=\mathit{var}\left({\widehat{\xi }}_{\mathit{IPW}}\right)/\mathit{var}\left({\widehat{\xi }}_{\mathit{AIPW}}\right)$. The X-axis is the fraction of SRS in the validation sample: n₀/n_V

We also evaluate the precision gain of our proposed estimators for different values of ρ, where ρ is the correlation coefficient between 2 error terms e and e in (9). Figure 2 shows the corresponding sample relative efficiency. When correlation changes from 0 to 1, the precision gain of ${\widehat{\xi }}_{\mathit{IPW}}$ over ${\widehat{\xi }}_{\mathit{SRS}}$ is relatively stable for different ρ values. On the other hand, the precision gain of ${\widehat{\xi }}_{\mathit{AIPW}}$ over ${\widehat{\xi }}_{\mathit{IPW}}$ is decreasing when γ₁ >0, and increasing when γ₁<0.

FIGURE 2.

Sample relative efficiencies (SREs) comparing ${\widehat{\xi }}_{\mathit{AIPW}}$ and ${\widehat{\xi }}_{\mathit{SRS}}$ to ${\widehat{\xi }}_{\mathit{IPW}}$ in terms of estimating γ₁, under different values of ρ. The sample relative efficiency is defined as $SR{E}_{\mathit{AIPW}:\mathit{IPW}}=\mathit{var}\left({\widehat{\xi }}_{\mathit{IPW}}\right)/\mathit{var}\left({\widehat{\xi }}_{\mathit{AIPW}}\right)$. AIPW, augmented IPW; IPW, inverse probability weighted

6 | COLLABORATIVE PERINATAL PROJECT DATA

In this section, we applied our method to analyze the CPP data set. The CPP is originally conducted as a cohort study to evaluate the risk factors for birth defects and other neurological disorders of childhood.³² The study involved 12 hospitals/universities located across the United States. In all, 55 908 pregnancies were registered, representing the experience about 44 000 women. The children born during the study were followed up until 8 years old. One hypothesis is that maternal PCB levels are related to children’s IQ performance at 7 years of age.⁴ Because the exposure variable PCB levels are very expensive to measure, ODS design is conducted on the basis of CPP data. An SRS of 849 individuals is selected, and then 2 supplemental samples are selected based on the children’s IQ score. One supplemental sample of size 81 is selected from the lower tail of the IQ distribution, defined by 1 standard deviation below the mean IQ score in the CPP population. The other supplemental sample of size 108 is chosen from 1 standard deviation above the mean IQ score.

Many researchers have made efforts to assess whether there is association between PCB level and child’s birth weight, but the findings from these studies are not consistent. Some indicate an inverse association,^12,16,18 while other indicates a positive association¹³ or no association at all.^14,15,17 We use our available CPP data to perform the secondary outcome analysis. In our analysis, we use the Weschler Intelligence Scale for children at 7 years old (IQ) as the primary outcome, and child’s birth weight (in grams) to be the secondary outcome. Other confounding variables include parent’s education level (EDU), social economic status of the child’s family (SES), race ethnicity of the child (RACE), and gender of the child (GENDER).

Table 4 shows the parameter estimates, standard error, and 95% CIs for the secondary outcome model, which regresses birth weight over PCB level adjusting for other covariates. Simple random sample denotes the regression analysis based on the SRS portion of the data. Inverse probability weighted denotes the inverse probability weighted estimating equation we proposed in Section 3.1, which uses the validation sample only. Augmented IPW denotes the augmented inverse probability weighted estimating equation proposed in Section 3.2. The semiparametric maximum likelihood estimator is excluded from the analysis, as it does not have a natural extension when the covariates contain both discrete and continuous components.

TABLE 4.

Analysis for a secondary outcome: child’s birth weight in CPP study

Variables		SRS	IPW	AIPW
Int	Estimate	3208.14	3209.44	3089.24
	SE	86.54	79.49	21.97
	95% CI	(3038.52, 3377.76)	(3053.64, 3365.24)	(3046.18, 3132.30)
PCB	Estimate	−0.67	−5.44	−5.49
	SE	9.20	8.41	8.46
	95% CI	(−18.70, 17.36)	(−21.92, 11.04)	(−22.07, 11.09)
EDU	Estimate	−8.97	−8.81	1.70
	SE	9.10	8.39	1.36
	95% CI	(−26.81, 8.87)	(−25.25, 7.63)	(−0.97, 4.37)
SES	Estimate	13.94	16.06	13.58
	SE	10.77	10.33	2.16
	95% CI	(−7.17, 35.05)	(−4.19, 36.31)	(9.35, 17.81)
RACE	Estimate	204.10	206.16	189.84
(WHITE = 1)	SE	38.13	37.93	6.29
	95% CI	(129.37, 278.83)	(131.82, 280.50)	(177.51, 202.17)
GENDER	Estimate	−147.69	−145.65	−119.25
(FEMALE = 1)	SE	34.72	31.96	5.33
	95% CI	(−215.74, −79.64)	(−208.29, −83.01)	(−129.70, −108.80)

The response variable is child’s birth weight in grams. The expensive exposure is mother’s polychlorinated biphenyl (PCB) level. Other confounding variables include: parent’s education level (EDU), social economic status of the child’s family (SES), race ethnicity of the child (RACE), and gender of the child (GENDER). The results under simple random sample (SRS) are the regression analysis using the SRS portion of the outcome-dependent sampling sample. Inverse probability weighted (IPW) is the inverse probability weighted estimating equation, and augmented IPW (AIPW) is the augmented inverse probability weighted estimating equation we proposed. SE, standard error.

All 3 analyses confirm that maternal PCB concentration is not significantly associated with child’s birth weight. However, the proposed IPW and AIPW estimator provide more precise estimates of the effects, evidenced by the reduced standard error and narrower CI. For example, the standard error for the PCB effect is 8.41 for IPW and 8.46 for AIPW, which is smaller than 9.20 for the standard regression analysis based on the SRS portion of the data. All 3 analyses confirm that being White has a positive impact on child’s birth weight, while AIPW analysis shows that social economic status has a positive impact on birth weight. In addition, girls have lower birth weight compared with boys as we expected. Note that there seems to be some difference between AIPW and IPW estimator for the EDU effect. We conducted additional simulation studies to show that such difference is merely due to sampling variability, and the observed difference is not that large if we take into account of the magnitude of the standard error. The details are shown in the Supporting Information.

7 | DISCUSSION

Investigators would all like to use the availability of expensive exposure that is already measured in a previous study. Most studies have multiple endpoints beside the primary outcome. This means that we often need to reuse the already collected data to analyze a secondary outcome in relation to the expensive exposure. However, when the original data are collected via an outcome-dependent fashion, secondary outcome analysis can be challenging as the ODS sample is no longer an SRS of the general population. As more studies are conducted using ODS designs, there is ever increasing needs for performing secondary analysis correctly and precisely for data from these studies. Our research is intended to fill these gaps and is the first attempt to develop precise inference procedure for a secondary outcome under the continuous outcome ODS design. We proposed IPW and AIPW estimating equations, in which only the form of the regressions are specified. Our proposed approach has the advantage of making no parametric distribution assumptions on (Y₁, Y₂) and thus is robust to model misspecification. Yet our proposed estimators are able to improve estimation precision relative to the naive analysis using SRS sample only.

Recall that r is the indicator variable of being selected into the validation sample. The expensive exposure X and r are conditionally independent given Y₁. This means that in the Step 1 of AIPW analysis, we could actually use the whole validation sample (n_V participants) to estimate the nuisance parameters (ϕ, σ²). As we mentioned before, how to estimate the nuisance parameter (ϕ, σ²) does not influence the asymptotic distribution of ${\widehat{\xi }}_{\mathit{AIPW}}$ as long as the nuisance parameter estimates are consistent. However, in small sample scenarios, there could be some difference. For instance, when full cohort has size 3000, (n₀, n₁, n₃) = (50, 175, 175), the proposed estimator ${\widehat{\xi }}_{\mathit{AIPW}}$ could have some bias (mean of parameter estimates is 0.515 while the true value for γ₁ is 0.5), if we only used SRS (50 participants) to estimate the nuisance parameters. On the other hand, the bias is reduced to 0.07, if we use the validation sample (400 participants) to estimate the nuisance parameters and then solve the AIPW estimating equation. When SRS sample size is larger, such as (n₀, n₁, n₃) = (200, 100, 100), the results between these 2 approaches are almost identical.

We found that our proposed estimators have the same asymptotic variance regardless of whether we are using fixed-size sampling (sampling without replacement) or independent Bernoulli sampling in choosing the initial SRS and the supplemental samples. This is because even though (r₁, ⋯, r_N) are correlated under fixed-size sampling, it can be proved that $\mathit{cov}\left(\frac{r_i}{{\pi }_i}{D}_i^T{Q}^{-1}{e}_i,\frac{r_j}{{\pi }_j}{D}_j^T{Q}^{-1}{e}_j\right)=0$ for different participants i and j. For case-cohort design, the difference between 2 subcohort sampling methods have been studied.^33,34 They found that the variance under fixed-size SRS is always smaller than or equal to the variance under Bernoulli sampling. There is equivalence when the corresponding covariance parts equal to 0 (see the comparison between ${\sum }_H^{\ast }$ and Σ_H in Kulich and Lin³³).

In our paper, when we implemented our AIPW estimator, we specified the form of the conditional moments, E(X | Y₁, Y₂, Z) and E(X² | Y₁, Y₂, Z) using a linear regression model. In practice, the expensive exposure X might be discrete, then a generalized linear model could be adopted. We could also use some nonparametric techniques to estimate these conditional moments. Interest rises to see if there exists any difference between parametric and non-parametric methods. In addition, we used the observed selection probability ${\widehat{\pi }}_i$ in both IPW and AIPW estimators. In some ODS designs, the true selection probability is known based on the design structure and can be calculated. It would be interesting to see whether using the true probability or observed probability makes any difference in statistical efficiency. Furthermore, we are looking for a flexible parametric family to jointly model the primary and secondary outcome. Copula seems to be a natural way to achieve the goal. This is another possible area of future research.

APPENDIX A

PROOFS

We provide the outline of the proofs for the augmented IPW estimator ${\widehat{\xi }}_{\mathit{AIPW}}$. The proof for ${\widehat{\xi }}_{\mathit{IPW}}$ follows the similar arguments and thus omitted here. Let η = (Q, π, ϕ, σ²) denote all the nuisance parameters. We make the following regularity conditions:

1. The parameter space of ξ, Ξ, is a compact subspace of R^p, and that the true underlying value ξ_* lies in the interior of the parameter space; the covariate space, $\mathcal{X}$ is a compact subset of R; and the covariate space, $\mathcal{Z}$, is a compact subset of R^q for some q≥ 1.

2. For all (Y₁, Y₂, X, Z), s₂(ξ, η) is continuous for all ξ ∈ Ξ; the partial derivatives $\frac{\partial s_2\left(\xi ,\eta \right)}{\partial {\xi }_i}$, for i = 1, …, p, exist and are continuous for all ξ ∈ Ξ.

3. Interchanges of differentiation and integration are valid for s₂(ξ, η) and its first-order partial derivatives with respect to ξ.

4. The following expected value matrix is finite and negative definite at (ξ_*, η_*):

   $$E\left[\frac{\partial }{\partial {\xi }^T}{s}_2\left(\xi ,\eta \right)\right].$$

5. The supremum of $\frac{\partial s_2\left(\xi ,\eta \right)}{\partial {\xi }^T}$ in the neighborhood of ξ_* is bounded by a function g that has finite expectation. (Outline of the proof for consistency):

By law of larger numbers, it is straightforward to show that

$$\frac{1}{N}{\displaystyle \sum _{i=1}^N{s}_{2i}\left({\xi }_{\ast },{\eta }_{\ast }\right)\stackrel{p}{\to }E\left[s_2\left({\xi }_{\ast },{\eta }_{\ast }\right)\right]=0\text{as}N\to \mathrm{\infty }.}$$ (A1)

Using the fact that $\widehat{\eta }=\left(\widehat{Q},\widehat{\pi },\widehat{\varphi },{\widehat{\sigma }}^2\right)$ is a consistent estimator of η_*, and ${\widehat{s}}_{2i}\left(\xi \right)=s_{2i}\left(\xi ,\widehat{\eta }\right)$, it can be shown that

$$\frac{1}{N}{\displaystyle \sum _{i=1}^N{\widehat{s}}_{2i}\left({\xi }_{\ast }\right)-\frac{1}{N}{\displaystyle \sum _{i=1}^N{s}_{2i}\left({\xi }_{\ast },{\eta }_{\ast }\right)\stackrel{p}{\to }0\text{as}N\to \mathrm{\infty }.}}$$ (A2)

Combining (A1) and (A2), we know that

$$\frac{1}{N}{\widehat{S}}_2\left({\xi }_{\ast }\right)=\frac{1}{N}{\displaystyle \sum _{i=1}^N{\widehat{s}}_{2i}\left({\xi }_{\ast }\right)\stackrel{p}{\to }0\text{as}N\to \mathrm{\infty }.}$$

Furthermore, using Assumptions 1-3 and consistency of $\widehat{\eta }$, we can show that

$$\frac{1}{N}\left[\frac{\partial {\widehat{S}}_2\left(\xi \right)}{\partial {\xi }^T}-\frac{\partial S_2\left(\xi ,{\eta }_{\ast }\right)}{\partial {\xi }^T}\right]\stackrel{p}{\to }0\text{as}N\to \mathrm{\infty }.$$ (A3)

holds uniformly for ξ in parameter space. Also, by law of large numbers,

$$\frac{1}{N}\frac{\partial S_2\left(\xi ,{\eta }_{\ast }\right)}{\partial {\xi }^T}=\frac{1}{N}{\displaystyle \sum _{i=1}^N{s}_{2i}\left(\xi ,{\eta }_{\ast }\right)\stackrel{p}{\to }E\left[\frac{\partial }{\partial {\xi }^T}{s}_2\left(\xi ,{\eta }_{\ast }\right)\right]\text{as}N\to \mathrm{\infty }.}$$ (A4)

Combining (A3) and (A4), we have

$$\frac{1}{N}\frac{\partial {\widehat{S}}_2\left(\xi \right)}{\partial {\xi }^T}\stackrel{p}{\to }E\left[\frac{\partial }{\partial {\xi }^T}{s}_2\left(\xi ,{\eta }_{\ast }\right)\right]\text{as}N\to \mathrm{\infty }.$$

uniformly for ξ ∈ Ξ. From Assumption 4, we know that $E\left[\frac{\partial }{\partial {\xi }^T}{s}_2\left(\xi ,\eta \right)\right]$ is negative definite and hence invertible at (ξ_*, η_*). Hence, we can apply a result of Foutz³¹ that uses the inverse function theorem to prove that our proposed estimator is a consistent and unique solution to the estimating equations.

(outline of the proof for asymptotic normality):

Let us denote

$$A_N=\frac{1}{N}{\displaystyle \sum _{i=1}^N\frac{\partial }{\partial {\xi }^T}{s}_{2i}\left({\xi }_{\ast },{\eta }_{\ast }\right),B_N=\frac{1}{N}{\displaystyle \sum _{i=1}^N\frac{\partial }{\partial {\eta }^T}{s}_{2i}\left({\xi }_{\ast ,}{\eta }_{\ast }\right)}.}$$

Using lemma 2 from Yuan and Jennrich,³⁰ we have

$$\sqrt{N}\left[\frac{1}{N}{S}_2\left(\widehat{\xi },\widehat{\eta }\right)-\frac{1}{N}{S}_2\left({\xi }_{\ast },{\eta }_{\ast }\right)\right]=A_N\sqrt{N}\left(\widehat{\xi }-{\xi }_{\ast }\right)+B_N\sqrt{N}\left(\widehat{\eta }-{\eta }_{\ast }\right).$$ (A5)

Let A and B be the limit of A_N and B_N, respectively. By law of large numbers, we know that

$$\begin{gathered} A_N=\frac{1}{N}{\displaystyle \sum _{i=1}^N\frac{\partial }{\partial {\xi }^T}{s}_{2i}\left({\xi }_{\ast },{\eta }_{\ast }\right)\stackrel{p}{\to }E\left[\frac{\partial s_2\left({\xi }_{\ast },{\eta }_{\ast }\right)}{\partial {\xi }^T}\right]=-I_2\left({\xi }_{\ast },{\eta }_{\ast }\right)} \\ {B}_N=\frac{1}{N}{\displaystyle \sum _{i=1}^N\frac{\partial }{\partial {\eta }^T}{s}_{2i}\left({\xi }_{\ast }{\eta }_{\ast }\right)\stackrel{p}{\to }E\left[\frac{\partial s_2\left({\xi }_{\ast }{\eta }_{\ast }\right)}{\partial {\eta }^T}\right]=0.} \end{gathered}$$

That is, A = −I₂(ξ_*, η_*) and B = 0. Rearrange (A5) and using the fact that $S_2\left(\widehat{\xi },\widehat{\eta }\right)=0$, we have

$$-A_N\sqrt{N}\left(\widehat{\xi }-{\xi }_{\ast }\right)=\frac{1}{\sqrt{N}}{S}_2\left({\xi }_{\ast },{\eta }_{\ast }\right)+B\sqrt{N}\left(\widehat{\eta }-{\eta }_{\ast }\right)+\left(B_N-B\right)\sqrt{N}\left(\widehat{\eta }-{\eta }_{\ast }\right).$$

B = 0, $\sqrt{N}\left(\widehat{\eta }-{\eta }_{\ast }\right)$ is bounded in probability. Hence, we know that

$$-A_N\sqrt{N}\left(\widehat{\xi }-{\xi }_{\ast }\right)=\frac{1}{\sqrt{N}}{S}_2\left({\xi }_{\ast },{\eta }_{\ast }\right)+o_p\left(1\right).$$

By the proposition in appendix 1 of Kulich and Lin,³³

$$-A_N\sqrt{N}\left(\widehat{\xi }-{\xi }_{\ast }\right)\stackrel{D}{\to }N\left(0,{\sum }_2\left({\xi }_{\ast },{\eta }_{\ast }\right)\right),$$

where Σ₂(ξ, η) = E[s₂(ξ, η)s₂(ξ, η)^T. Then, using Slutsky theorem, we know that

$$\sqrt{N}\left(\widehat{\xi }-{\xi }_{\ast }\right)\stackrel{D}{\to }N\left(0,I_2^{-1}\left({\xi }_{\ast },{\eta }_{\ast }\right){\sum }_2\left({\xi }_{\ast },{\eta }_{\ast }\right)I_2^{-1}\left({\xi }_{\ast },{\eta }_{\ast }\right)\right).$$

Notice that B=0 implies that the asymptotic distribution of $\widehat{\eta }$ does not influence the asymptotic distribution of $\widehat{\xi }$ as long as $\widehat{\eta }$ is root-N consistent.

APPENDIX B: BRIEF DESCRIPTION OF THE ESTIMATED LIKELIHOOD APPROACH

To develop the semiparametric maximum likelihood estimator ${\widehat{\xi }}_{\mathit{SPML}}$, we need to assume that (Y₁, Y₂) is bivariate normal. That is,

$$\begin{gathered} Y_{1i}={\beta }_0+{\beta }_1{X}_i+{\beta }_2{Z}_i+e_i, \\ {Y}_{2i}={\gamma }_0+{\gamma }_1{X}_i+{\gamma }_2{Z}_i+{\epsilon }_i, \end{gathered}$$

where (e_i, ε_i) follows a bivariate normal distribution with $\mathit{var}\left(e_i\right)={\sigma }_1^2$, $\mathit{var}\left({\epsilon }_i\right)={\sigma }_2^2$, cov(e_i, ε_i) = ρσ₁σ₂.

For a participant in V(validation sample), the contribution to the likelihood is (Y₁, Y₂, X, Z). For a participant in $\overline{V}$ (nonvalidation sample), the contribution is (Y₁, Y₂, Z). Hence, the likelihood corresponding to (1) is proportional to

$$L\left(\xi \right)={\displaystyle \prod _{i\in V}{f}_{\xi }\left(Y_{1i},Y_{2i}|X_i,Z_i\right)\times {\displaystyle \prod _{j\in \overline{V}}{\displaystyle {\int }_x{f}_{\xi }\left(Y_{1j},Y_{2j}|x,Z_j\right)d{G}_{X|Z}\left(x|Z_j\right)}}},$$

where f_ξ(Y₁, Y₂ | X, Z) is the density function of a bivariate normal distribution and G_X|Z(·|·) represents the conditional distribution function of X given Z. The log-likelihood is

$$l\left(\xi \right)={\displaystyle \sum _{i\in V}\log f_{\xi }\left(Y_{1i},Y_{2i}|X_i,Z_i\right)+{\displaystyle \sum _{j\in \overline{V}}\log \left\{{\displaystyle {\int }_x{f}_{\xi }\left(Y_{1j},Y_{2j}|x,Z_j\right)d{G}_{X|Z}\left(x|Z_j\right)}\right\}}.}$$

Notice that G_X|Z is a nuisance function. Using ideas from Weaver and Zhou,⁵ we propose to work with the following estimated log-likelihood function:

$$\widehat{l}\left(\xi \right)={\displaystyle \sum _{i\in V}\log f_{\xi }\left(Y_{1i},Y_{2i}|X_i,Z_i\right)+{\displaystyle \sum _{j\in \overline{V}}\log \left\{{\displaystyle {\int }_x{f}_{\xi }\left(Y_{1j},Y_{2j}|x,Z_j\right)d{\widehat{G}}_{X|Z}\left(x|Z_j\right)}\right\}},}$$

where we nonparametrically estimate G_X|Z using the SRS sample. For discrete Z, let

$${\widehat{G}}_{X|Y}\left(x|z\right)={\displaystyle \sum _{i\in V_0}I\left(X_i\le x,Z_i=z\right)/{\displaystyle \sum _{i\in V_0}I\left(Z_i=z\right)}.}$$

For continuous Z, we use the kernel method, ie,

$${\widehat{G}}_{X|Z}\left(x|z\right)={\displaystyle \sum _{i\in V_0}I\left(X_i\le x\right)K_H\left(Z_i-z\right)/{\displaystyle \sum _{i\in V_0}{K}_H\left(Z_i-z\right),}}$$

where K_H(·) = |H|^−1/2K(H^−1/2·) is a kernel with a bandwidth matrix H. Then, the proposed semiparametric maximum likelihood estimator ${\widehat{\xi }}_{\mathit{SPML}}$ is the solution to the following estimating equation:

$$\frac{1}{N}\frac{\partial \widehat{l}\left(\xi \right)}{\partial \xi }=0.$$