Unified Analysis of Secondary Traits in Case-Control Association Studies

Abstract

It has been repeatedly shown that in case-control association studies, analysis of a secondary trait which ignores the original sampling scheme can produce highly biased risk estimates. Although a number of approaches have been proposed to properly analyze secondary traits, most approaches fail to reproduce the marginal logistic model assumed for the original case-control trait and/or do not allow for interaction between secondary trait and genotype marker on primary disease risk. In addition, the flexible handling of covariates remains challenging. We present a general retrospective likelihood framework to perform association testing for both binary and continuous secondary traits which respects marginal models and incorporates the interaction term. We provide a computational algorithm, based on a reparameterized approximate profile likelihood, for obtaining the maximum likelihood (ML) estimate and its standard error for the genetic effect on secondary trait, in presence of covariates. For completeness we also present an alternative pseudo-likelihood method for handling covariates. We describe extensive simulations to evaluate the performance of the ML estimator in comparison with the pseudo-likelihood and other competing methods.

2 Introduction

The retrospective case-control design is one of the most important tools for epidemiology, and for rare diseases/traits may offer tremendous savings in time and expense compared to a prospective design. Even so, case-control designs remain costly, and efficiency is further maximized by gathering additional clinical phenotypes/traits for the sampled individuals. We refer to the dichotomous case/control variable as the primary trait (or phenotype), and other traits, which may be discrete or continuous, as secondary. Methods for analyzing such data have received considerable recent attention due to the availability of genome-wide association (GWA) datasets, which often follow a case-control design and include numerous secondary traits, which may be correlated with the primary trait (Frayling et al., 2007; Weedon et al., 2008; Spitz et al., 2008). In such studies the risk variables of main interest are genotypes of single nucleotide polymorphisms (SNPs), with possible additional covariate effects. The approaches described here apply generally to secondary analysis of case-control data, but the notation and examples are applicable to genetic association studies.

It is widely understood that, for case-control designs, a (prospective) analysis which ignores the sampling scheme yields consistent estimates of the odds ratio for disease risk (e.g., using logistic regression models (Prentice and Pyke, 1979)). For the analysis of secondary traits within a case-control design, a number of approaches have been described (Jiang et al., 2006; Lin and Zeng, 2009; He et al., 2012; Wang and Shete, 2011, 2012). The existing methods can be broadly divided as follows: a) the naïve method of analyzing the combined sample of cases and controls, without accounting for the case-control ascertainment; b) performing analysis within case and control groups; c) combining estimates from the case and control groups either by meta-analysis with inverse variances as weights or by using case/control status as a covariate; d) correcting bias using weighting schemes originally developed for sample surveys; and d) explicitly accounting for the case/control sampling scheme via a retrospective likelihood.

The naïve method can lead to severely biased estimates of the risk effects for the secondary traits, except under restrictive conditions (Nagelkerke et al., 1995; Jiang et al., 2006; Lin and Zeng, 2009; Monsees et al., 2009). Simple methods which account for the biased sampling include adjusting for the disease status in a regression model or restricting the analysis to cases or controls only. For rare diseases, the controls-only analysis is approximately unbiased, but may be highly inefficient. If the association between the primary and the secondary traits does not depend on the SNP genotype, then the cases also give a valid risk effect estimate and can be combined with the controls-only estimate via a inverse-variance meta-analytic procedure to provide a weighted estimate with improved efficiency. Li et al. (2010) describe an adaptively weighted method for rare diseases with binary secondary trait and genotype, that further combines the controls-only and the weighted estimates. Recently, they proposed another adaptive procedure to analyze secondary phenotypes for data from a case-control study of a primary disease that is not rare (Li and Gail, 2012).

The survey approach uses weights inversely proportional to the sampling fractions (Jiang et al., 2006; Scott and Wild, 2002). Richardson et al. (2007) and Monsees et al. (2009) use this standard survey-weighted approach, applied to the analysis of binary secondary traits, termed the inverse-probability-of-sampling-weighted (IPW) regression by Monsees et al. (2009). Technically this approach requires knowledge of the case-control sampling fractions, but the disease prevalence is often available externally and may be used as in Wang and Shete (2011).

In general, most of the approaches described above result in inconsistent risk estimates, although the bias may be low under some specific assumptions. The efficiency of the procedures also varies widely (Jiang et al., 2006). An alternative approach to account for the sampling mechanism is to use the retrospective likelihood, explicitly conditioning on the sampling scheme (Jiang et al., 2006; Scott and Wild, 1997b, 2001, 1991; Lee et al., 1998; Lin and Zeng, 2009; He et al., 2012). The retrospective likelihood models the joint distribution for the primary and secondary traits as a function of the genotype and other covariates. There are a number of attractive features to this approach. If the model is correct, then it will provide large-sample optimality for the maximum likelihood (ML) estimates. Provided the model is sufficiently rich, it enables partitioning of the correlations between primary and secondary phenotypes into the portion due to the risk variable (genotype), as well as a residual portion that may be due to other genes or environment. In addition, standard semi-parametric maximum likelihood (SPML) approaches are available to handle covariates in a flexible manner, with minimal assumptions for the potentially complex interplay of covariates with other data. For this reason the retrospective approaches are collectively termed “SPML” in the taxonomy of Jiang et al. (2006).

Although the motivation for the retrospective likelihood is clear, specific implementations vary. One approach is to factor the joint distribution into the marginal for the secondary trait and the conditional for the primary trait given secondary (SPML2 in Jiang et al., 2006). For example, Lin and Zeng (2009) handle both binary and continuous secondary traits by modeling the disease status given the secondary trait as a logistic regression. Wang and Shete (2011) use this joint model and apply a method of moments approach to produce bias-corrected odds ratio estimates for binary secondary traits using prevalence estimates for the primary and secondary traits from the literature. Recently they have shown that their method is robust even when there is an interactive effect of the SNP and secondary phenotype on the primary disease risk (Wang and Shete, 2012). He et al. (2012) have proposed a Gaussian copula-based approach that models the joint distribution in terms of the marginals for the primary and secondary phenotypes (SPML3 in Jiang et al., 2006) and uses the multivarite normal distribution to build in correlation between the phenotypes. Their method can handle multiple correlated secondary phenotypes.

Despite all of these efforts, a number of deficiencies remain. Most retrospective likelihood approaches do not incorporate interaction between the genetic variant and the secondary trait on primary disease risk. Other than the Gaussian copula approach (He et al., 2012), retrospective methods do not generally preserve the marginal logistic model for the disease trait, creating a contradiction between the primary trait risk estimates obtained from the marginal model and from the joint primary-secondary analysis. Additionally, the nonparametric handling of continuous covariates remains challenging; when there are multiple covariates, including one or more continuous ones, direct maximization of the likelihood is infeasible.

We propose an approach that specifies the joint distribution of primary and secondary traits in terms of the marginals for the two traits, with terms to govern their association (SPML3 in Jiang et al., 2006). Our framework enables association testing for both binary and continuous secondary traits, while respecting the desired logistic model for the primary trait and standard marginal models for the secondary trait. We demonstrate how this approach can incorporate covariates, and easily allow for interaction between the genetic variant and the secondary trait on primary disease risk. To handle the computational complexity introduced by the covariates, we reparameterize the profile likelihood, and derive a closed form expression which provides ML estimates of risk effects. For completeness, we briey describe a pseudo-likelihood approach that bypasses the need to involve the potentially high-dimensional covariate distribution. We perform extensive simulations to evaluate the performance of our profile likelihood method and compare it with the pseudo-likelihood and other competing methods.

The remainder of the paper is organized as follows. In Section 3 we lay down the details for our proposed profile likelihood-based method. In particular, in Subsection 3.1 we describe our joint model for the primary and secondary traits and in the following subsection we provide details of the estimation procedure. In Section 4 we demonstrate the performance of the proposed method as compared to other competing methods via a real data example and simulations. Section 5 concludes with some comments on related work and future directions. In Section 7 we expand on the theoretical details.

3 Methods

Let D denote the disease status or the primary trait (0=control, 1=case), Y the secondary trait, G the genotype at a biallelic locus, and Z the vector of covariates. Under additive model, G represents the number of minor alleles (0, 1, or 2) at the locus; under dominant (recessive) model, G denotes whether the individual carries at least one minor allele (two minor alleles). We randomly sample n₀ and n₁ individuals from the controls (D = 0) and the cases (D = 1) in the population respectively, and observe their Y-, G-, and Z-values. The log-likelihood for our case-control sampled data (d_u, y_u, g_u, z_u), u = 1, 2, …, n, takes the retrospective form ${\sum }_{u=1}^n\text{log}P(y_u,g_u,{\mathbf{z}}_u|d_u)$ where

$$\text{log}P(y_u,g_u,{\mathbf{z}}_u|d_u)=\text{log}P(d_u,y_u|g_u,{\mathbf{z}}_u)+\text{log}P(g_u,{\mathbf{z}}_u)-\text{log}P(d_u).$$

We model P(d, y|g, z) parametrically, the regression P(y|g, z) being of primary interest. Note that P(g, z), the joint distribution for G and Z in the population, cannot be ignored, since

$$P(d)=\sum _{y,g,\mathbf{z}}P(d,y,|g,\mathbf{z})P(g,\mathbf{z}).$$

We deal with P(g, z) nonparametrically, thereby taking a semi-parametric approach to modeling the likelihood. In the following subsections we describe the joint model for the bivariate response (D, Y) and the estimation procedure.

3.1 Joint modeling of the primary and secondary traits

In modeling the joint distribution for the traits we try to preserve the marginal logistic model typically assumed for the original case-control trait D. We specify the bivariate distribution, P(D, Y|g, z), by parametrically modeling the marginals for the primary and secondary traits and also building a parametric model for their association given the genotype and the covariates. The natural choice for the marginal distribution for disease status is logistic, and that for the secondary trait is logistic or normal depending on whether the trait is binary or continuous respectively. For binary Y, the Palmgren model (Palmgren, 1989), the Bahadur model (Bahadur, 1959), and models based on copula theory (Meester and MacKay, 1994) have been used previously to specify joint distributions for bivariate binary responses. But for continuous Y, there is no standard bivariate distribution that yields logistic and normal marginals for D and Y respectively.

3.1.1 Binary secondary trait

The bivariate logistic model, considered by Palmgren (1989), has been used previously to model the joint distribution for correlated binary data (Jiang et al., 2006; Lee et al., 1998) and is conceptually very simple. It is based on the fact that the joint distribution of two binary variables can be specified in terms of their marginal probabilities and their odds ratio. Thus, for a randomly sampled individual in the population we specify the joint distribution of D and Y given g and z as

$$\mathit{\text{logit}}P(D=1|g,\mathbf{z})={\alpha }_1+{\beta }_{1}g+{\mathit{\gamma }}_{\mathbf{1}}\prime \mathbf{z},\mathit{\text{logit}}P(Y=1|g,\mathbf{z})={\alpha }_2+{\beta }_{2}g+{\mathit{\gamma }}_{\mathbf{2}}\prime \mathbf{z},$$

and $\text{log}\mathit{\text{OR}}(D,Y|g,\mathbf{z})=\text{log}\frac{P(D=1,Y=1|g,\mathbf{z})P(D=0,Y=0|g,\mathbf{z})}{P(D=1,Y=0|g,\mathbf{z})P(D=0,Y=1|g,\mathbf{z})}={\alpha }_3+{\beta }_{3}g$.

We are interested in inferring about β₂, the risk effect for the secondary trait.

3.1.2 Continuous secondary trait

For continuous Y, we consider joint models for (D, Y |g, z) such that the marginal distributions for D and Y given g and z correspond to the standard logistic and linear regressions respectively, that is,

$$\mathit{\text{logit}}P(D=1|g,\mathbf{z})={\alpha }_1+{\beta }_{1}g+{\mathit{\gamma }}_{\mathbf{1}}\prime \mathbf{z}\text{and}Y|g,\mathbf{z}~N({\alpha }_2+{\beta }_{2}g+{\mathit{\gamma }}_{\mathbf{2}}\prime \mathbf{z},{\sigma }_2^2).$$

To come up with a joint model that complies with the above marginals, we start with the following bivariate normal distribution,

$$\left(\begin{array}{c}\hfill V\hfill \\ \hfill Y\hfill \end{array}\right)|g,\mathbf{z}~N(\left(\begin{array}{c}\hfill {\mu }_1\hfill \\ \hfill {\mu }_2\hfill \end{array}\right),\left(\begin{array}{cc}\hfill 1\hfill & \hfill \rho {\sigma }_2\hfill \\ \hfill \rho {\sigma }_2\hfill & \hfill {\sigma }_2^2\hfill \end{array}\right)),$$

where ${\mu }_1={\alpha }_1+{\beta }_{1}g+{\mathit{\gamma }}_{\mathbf{1}}\prime \mathbf{z},{\mu }_2={\alpha }_2+{\beta }_{2}g+{\mathit{\gamma }}_{\mathbf{2}}\prime \mathbf{z}$, and $\text{log}\frac{1+\rho }{1-\rho }={\alpha }_3+{\beta }_{3}g$. We then introduce another latent variable to produce the logistic marginal. To transform the normal variate V, we define $U={\mu }_1+\text{log}\frac{\mathrm{\Phi }(V-{\mu }_1)}{1-\mathrm{\Phi }(V-{\mu }_1)}$. It follows that the density of U given g and z is logistic with location parameter μ₁ and scale parameter 1,

$$p_{U|G,\mathbf{Z}}(u|g,\mathbf{z})=\frac{\text{exp}(-(u-{\mu }_1))}{{(1+\text{exp}(-(u-{\mu }_1)))}^2},-\infty <u<\infty .$$

We then threshold U at 0 to derive D. That is, D, defined as

$$D=\{\begin{array}{cc}1,\hfill & U\ge 0\hfill \\ 0,\hfill & U<0\hfill \end{array},$$

follows

$$\mathit{\text{logit}}P(D=1|g,\mathbf{z})={\alpha }_1+{\beta }_{1}g+{\mathit{\gamma }}_{\mathbf{1}}\prime \mathbf{z}.$$

As in the binary case, β₂ is our parameter of interest.

3.2 Estimation of β₂

3.2.1 ML estimate in absence of covariates

Let us first consider the situation without any covariates. The retrospective log-likelihood is

$$l=\sum _{u=1}^n\text{log}P(y_u,g_u|d_u)=\sum _{u=1}^n\text{log}P(d_u,y_u|g_u)+\sum _{u=1}^n\text{log}P(g_u)-\sum _{i=0}^1{n}_i\text{log}P(D=i).$$ (1)

We assume Hardy-Weinberg equilibrium and parameterize the genotype distribution in terms of the minor allele frequency, δ. We use θ to denote the set of parameters describing the joint distribution for D and Y given g. We write θ as (θ₁, θ₂)′, where θ₁ = (α₁, β₁, γ₁)′ parameterizes the disease model. Then the log-likelihood in (1) can be written as

$$l=\sum _{u=1}^n\text{log}P(d_u,y_u|g_u;\mathit{\theta })+\sum _{u=1}^n\text{log}P(g_u;\delta )-\sum _{i=0}^1{n}_i\text{log}\sum _{k=0}^{1}P(D=i|G=k;{\mathit{\theta }}_1)P(G=k;\delta ).$$ (2)

We maximize (2) with respect to η = (θ, δ)′ to derive η̂, the ML estimate for η. We can use the standard Newton-Raphson method to obtain the ML estimate iteratively, or other optimization tools such as quasi-Newton methods, Nelder-Mead simplex algorithm for derivative-free maximization, simulated annealing. Specifically, using Newton-Raphson method, we update the parameter in the (k + 1)^th iteration by

$${\mathit{\eta ̂}}^{(k+1)}={\mathit{\eta ̂}}^{(k)}-{\{\frac{{\partial }^2}{\partial \mathit{\eta }\partial \mathit{\eta }\prime }l({\mathit{\eta ̂}}^{(k)})\}}^{-1}\frac{\partial }{\partial \mathit{\eta }}l({\mathit{\eta ̂}}^{(k)}).$$

The inverse of the observed information matrix, ${\{-\frac{{\partial }^2}{\partial \mathit{\eta }\partial \mathit{\eta }\prime }l(\mathit{\eta ̂})\}}^{-1}$, gives standard error estimate for η̂.

3.2.2 ML estimate via profiling in presence of covariates

Let us now consider the situation where the joint model for the disease and the secondary trait involves covariates. The retrospective log-likelihood is

$$l=\sum _{u=1}^n\text{log}P(d_u,y_u|g_u,{\mathbf{z}}_u;\mathit{\theta })+\sum _{u=1}^n\text{log}P(g_u,{\mathbf{z}}_u)-\sum _{i=0}^1{n}_i\text{log}\sum _{g,\mathbf{z}}P(D=i|g,\mathbf{z};{\mathit{\theta }}_1)P(g,\mathbf{z}).$$ (3)

We can reasonably assume that G and Z are independently distributed in the underlying population, but must somehow account for the distribution of Z. Since the covariate structure is usually far too complicated to model parametrically (considering that it may be correlated with the primary trait), we want to make no assumptions about the form of the covariate distribution.. For a single binary covariate parameterized by ψ, we can easily derive the ML estimate by maximizing the retrospective likelihood with respect to (η, ψ). However, this approach is infeasible for a continuous covariate as the ML estimate involves maximization with respect to a high-dimensional nuisance parameter. In the following section we describe a computational technique to derive a closed-form expression, which may be thought of as an approximate profile likelihood up to one nuisance parameter, that upon maximization provides the ML estimate for the parameters of interest.

Let {z₁, z₂, …, z_L} represent unique values of Z in the case-control sample. We parameterize the distribution function for Z in terms of the probability masses {ψ₁, ψ₂, …, ψ_L} that we assign to {z₁, z₂, …, z_L}. The retrospective log-likelihood in (3) can now be written as

$$l=\sum _{u=1}^n\text{log}P(d_u,y_u|g_u,{\mathbf{z}}_u;\mathit{\theta })+\sum _{u=1}^n\text{log}P(g_u;\delta )+\sum _{u=1}^n\text{log}P({\mathbf{z}}_u;\mathit{\psi })-\sum _{i=0}^1{n}_i\text{log}\sum _{k=0}^1\sum _{l=1}^{L}P(D=i|G=k,{\mathbf{z}}_l;{\mathit{\theta }}_1)P(G=k;\delta ){\psi }_l.$$ (4)

When ψ is high-dimensional, maximizing the log-likelihood with respect to (η, ψ) can be a daunting task. We take the approximate profile likelihood approach described by Chatterjee and Carroll (2005) to handle the nuisance parameter ψ (hereafter dropping the “approximate” qualifier). To obtain the overall ML estimate for η, we maximize the profile log-likelihood, l(η) = sup_ψl(η, ψ) with respect to η. Following Scott and Wild (1997a, 2001) and Chatterjee and Carroll (2005), we show in the Appendix (Section 7.1) that the profile log-likelihood l(η) can be equivalently expressed as

$$l^{*}(\mathit{\eta },\kappa )=\sum _{u=1}^n\text{log}{P}_{d_u,y_u,g_u}^{*}({\mathbf{z}}_u;\mathit{\eta },\kappa )$$

where

$$P_{d_u,y_u,g_u}^{*}({\mathbf{z}}_u;\mathit{\eta },\kappa )=\frac{P(d_u,y_u|g_u,{\mathbf{z}}_u;\mathit{\theta })P(g_u;\delta ){\kappa }^{d_u}}{{\sum }_{i=0}^{1}P(D=i|{\mathbf{z}}_u;{\mathit{\theta }}_1){\kappa }^i}.$$

The parameter κ represents the ratio of the sampling fractions,

$$\kappa =\frac{n_1/P(D=1)}{n_0/P(D=0)},$$

and satisfies the equation $\frac{\partial l^{*}(\mathit{\eta },\kappa )}{\partial \kappa }=0$. We thus have effectively reduced the number of parameters from (p + L − 1) to (p + 1), with p used to denote the dimension of η. We provide a closed form expression, l*(ϕ), that on maximization with respect to ϕ = (η, κ)′ gives η̂, thereby bypassing the need to numerically maximize the likelihood with respect to a high-dimensional nuisance parameter.

Although l*(ϕ) is not a true log-likelihood the relevant asymptotic theory for ϕ̂ can be obtained by working with $S^{*}(\mathit{\varphi })=\frac{\partial }{\partial \mathit{\varphi }}{l}^{*}(\mathit{\varphi })$, referred to as the “pseudo” score-equations by Scott and Wild (2001). Under the assumption that n goes to infinity with n₁/n and n₂/n remaining fixed, we show in the Appendix (Section 7.2) that $\sqrt{n}(\mathit{\varphi ̂}-\mathit{\varphi })$ converges in distribution to a normal random vector with mean zero and covariance matrix

$$n\mathit{\text{Cov}}(\mathit{\varphi ̂})=n(J^{*}{(\mathit{\varphi })}^{-1}-J^{*}{(\mathit{\varphi })}^{-1}\mathrm{\Gamma }(\mathit{\varphi })J^{*}{(\mathit{\varphi })}^{-1}),$$

where

$$J^{*}(\mathit{\varphi })=-E(\frac{\partial }{\partial \mathit{\varphi }\prime }{S}^{*}(\mathit{\varphi }))$$

and

$$\mathrm{\Gamma }(\mathit{\varphi })=(\frac{1}{n_0}+\frac{1}{n_1})E_{\mathbf{z}}(f(\mathbf{z})\frac{\partial }{\partial \mathit{\varphi }}{P}_1^{*}(\mathbf{z}))E_{\mathbf{z}}(f(\mathbf{z})\frac{\partial }{\partial \mathit{\varphi }\prime }{P}_1^{*}(\mathbf{z}))$$

with $P_1^{*}(\mathbf{z})=\frac{P(D=1|\mathbf{z})\kappa }{{\sum }_i{\kappa }^{i}P(D=i|\mathbf{z})}$ and $f(\mathbf{z})=(\frac{n_1}{\kappa }+n_0){\sum }_i{\kappa }^{i}P(D=i|\mathbf{z})$. In practice, we can use

$$J^{*}{(\mathit{\varphi ̂})}^{-1}=-\frac{{\partial }^2}{\partial \mathit{\varphi }\partial \mathit{\varphi }\prime }{l}^{*}(\mathit{\varphi }){|}_{\mathit{\varphi ̂}}^{-1},$$

the inverse of the observed information matrix based on l*, to estimate Cov(ϕ̂). Since Γ(ϕ) ≥ 0, J*(ϕ̂)⁻¹ provides a conservative estimate for Cov(ϕ̂). Note that if the disease prevalence is known or well-estimated, we fix κ at its true value and work with l*(η) to obtain η̂ and its standard error.

3.2.3 Pseudo-likelihood estimate

Motivated by the fact that β₂, the parameter of interest, appears only in the first term of the expression for the log-likelihood in (3), one might attempt to handle the covariates via a previously described pseudo-likelihood approach (Gong and Samaniego, 1981). For the sake of completeness, we lay down the framework for this approach to estimate β₂ and its standard error, and evaluate its performance via simulations. The retrospective log-likelihood in (4) can be viewed as

$$l=l_1({\mathit{\theta }}_1,{\mathit{\theta }}_2)+l_2({\mathit{\theta }}_1,\delta ,\mathit{\psi }),$$ (5)

where

$$l_1({\mathit{\theta }}_1,{\mathit{\theta }}_2)=\sum _{u=1}^n\text{log}P(d_u,y_u|g_u,{\mathbf{z}}_u;{\mathit{\theta }}_1,{\mathit{\theta }}_2),$$ (6)

and

$$l_2({\mathit{\theta }}_1,\delta ,\mathit{\psi })=\sum _{u=1}^n\text{log}P(g_u;\delta )+\sum _{u=1}^n\text{log}P({\mathbf{z}}_u;\mathit{\psi })-\sum _{i=0}^1{n}_i\text{log}\sum _{k=0}^1\sum _{l=1}^{L}P(D=i|G=k,{\mathbf{z}}_l;{\mathit{\theta }}_1)P(G=k;\delta ){\psi }_l.$$

As β₂ is of primary interest, we treat θ₁ as nuisance parameter. Gong and Samaniego proposed first obtaining an estimate for θ₁. When disease prevalence is known, we derive an estimate for θ₁, say θ̃₁, from l₂(θ₁, δ, ψ) (in general different from the ML estimate θ̂₁ derived from l). We then plug this estimate into l₁(θ₁, θ₂) to obtain the pseudo log-likelihood

$${\mathrm{l̃}}_1({\mathit{\theta }}_2)=\sum _{u=1}^n\text{log}P(d_u,y_u|g_u{\mathbf{z}}_u;{\mathit{\theta ̃}}_1,{\mathit{\theta }}_2).$$ (7)

The pseudo-likelihood estimate, θ̃₂, is obtained by maximizing l̃₁ with respect to θ₂. We discuss the asymptotic properties of the pseudo-likelihood estimate and provide formulae for its variance estimation in the Appendix (Section 7.3).

4 Results

We present a data example and simulation results to demonstrate the performance of the profile likelihood-based ML estimate in comparison with the pseudo-likelihood estimate and other competing methods: the naïve method ignoring the sampling mechanism, the cases-only and controls-only methods, the weighted method combining the cases-only and controls-only estimators via inverse-variance, the adaptively weighted method, proposed by Li et al. (2010), combining the controls-only and weighted estimators, the adjusted method including the disease status as a covariate in the regression for the secondary trait, and finally the survey approach using weights inversely proportional to the sampling fractions. All analyses were performed in R v.2.15.0. In the table and figures we use ‘Wtd’, ‘Awtd’, and ‘Adj’ to denote the weighted, the adaptively weighted, and the adjusted estimates respectively.

4.1 Data example

We reanalyze the data on colorectal cancer, smoking, and N-acetyltransferase 2 (NAT2) presented in Li et al. (2010). The authors compare different analysis methods for a binary secondary trait using case-control data for colorectal adenoma described in Moslehi et al. (2006). Moslehi et al. explore how variants in NAT1 and NAT2 genes affect the smoking-colorectal cancer relationship using cases with colorectal adenoma and controls selected from the screening arm of the Prostate, Lung, Colorectal and Ovarian (PLCO) cancer screening trial. Of 42,037 participants who provided a blood sample, 4,834 were excluded. 772 cases were randomly selected from 1234 participants who had at least one advanced colorectal adenoma detected at baseline screen and 777 gender and age-matched controls were sampled from 26,651 participants with a negative baseline sigmoidoscopy screening. Li et al. use this case-control data for colorectal adenoma to analyze the effect of NAT2 gene on smoking (secondary trait). In Table 1 we display the data analyzed in Li et al. (2010) and in Table 2 we illustrate our re-analysis of the data.

Table 1.

Data for studying effect of NAT2 on smoking from a case-control study for colorectal adenoma

Cases of colorectal adenoma				Matched controls
	Smoking status		Total		Smoking status		Total
	Never/Former	Current			Never/Former	Current
NAT2=0	199	380	579	NAT2=0	255	317	572
NAT2=1	18	13	31	NAT2=1	10	23	33
Total	217	393	610	Total	265	340	605

Table 2.

Estimates of effect of NAT2 on smoking from a case-control study for colorectal adenoma

Method	log(OR)	SE
Naïve	−0.18	0.26
Cases	−0.97	0.37
Controls	0.62	0.39
Adj	−0.17	0.26
Wtd	−0.21	0.27
Awtd	0.57	0.40
ML/Survey	0.54	0.37

There are no covariates here, so the ML estimate based on the Palmgren model is easily derivable from the retrospective likelihood, without requiring additional profiling or pseudo-likelihood. We used a prevalence of 0.04 (=1234/(1234+26651)) based on Moslehi et al. (2006). Under known prevalence, the ML based on the Palmgren model and the survey approach are the same. Li et al., in their paper, treat the robust controls-only estimate as the gold standard and show that the adaptively weighted estimate is similar to the controls-only estimate. The authors point out that the non-zero interaction between NAT2 and smoking on colorectal adenoma risk results in high bias for the commonly used methods, apparent in the table for the naïve, the cases-only, the weighted, and the adjusted estimates. In contrast, our results show that the ML/survey estimate is similar to the adaptively weighted and controls-only estimates, with the ML/survey estimate enjoying smaller standard error.

4.2 Simulations

We compare performances of the different methods for both binary and continuous secondary traits, for biologically plausible scenarios, by generating data from the corresponding bivariate models (see Methods). We consider a biallelic SNP with an additive mode of inheritance and a minor allele frequency of 0.25. For demonstration purposes we include two covariates, one following a Bernoulli distribution with 0.45 success probability and the other a standard normal variate. For our simulations we consider the SNP genotype and the covariates as independent in the population. We fix the disease prevalence at 5% and the mean or prevalence of the secondary trait at 20%. Rather than specifying α₁ and α₂ directly, it is more interpretable to solve for them for specified values of the prevalences and other parameters. We fix β₁ at 0.25 (odds ratio = 1.28). We consider β₂ = 0 to reect the null and compare Type I error for different methods. To simulate data under alternative hypotheses and examine power, we fix β₂ at either −0.25 or 0.25 for the binary secondary trait; for the continuous setup we use −0.15 and 0.15. We assign the values (−0.94, −0.37)′ and (−1.82, 0.30)′, each generated independently from the standard normal, to the parameters γ₁ and γ₂, respectively. We fix α₃ at 1.0 and examine β₃ ranging from −2.0 to 2.0. We use σ₂ = 1.0 for the continuous scenario. For fixed prevalences, α₁ and α₂ are calculated according to the following equations:

$$P(D=1)={\int }_{z_2}\sum _{g,z_1}\frac{\mathit{\text{exp}}({\alpha }_1+{\beta }_{1}g+{\mathit{\gamma }}_{\mathbf{1}}\prime \mathbf{z})}{(1+\mathit{\text{exp}}({\alpha }_1+{\beta }_{1}g+{\mathit{\gamma }}_{\mathbf{1}}\prime \mathbf{z}))}\left(\begin{array}{c}\hfill 2\hfill \\ \hfill g\hfill \end{array}\right){\delta }^g{(1-\delta )}^{(2-g)}{0.45}^{z_1}{(1-0.45)}^{(1-z_1)}\varphi (z_2)d{z}_2,P(Y=1)={\int }_{z_2}\sum _{g,z_1}\frac{\mathit{\text{exp}}({\alpha }_2+{\beta }_{2}g+{\mathit{\gamma }}_{\mathbf{2}}\prime \mathbf{z})}{(1+\mathit{\text{exp}}({\alpha }_2+{\beta }_{2}g+{\mathit{\gamma }}_{\mathbf{2}}\prime \mathbf{z}))}\left(\begin{array}{c}\hfill 2\hfill \\ \hfill g\hfill \end{array}\right){\delta }^g{(1-\delta )}^{(2-g)}{0.45}^{z_1}{(1-0.45)}^{(1-z_1)}\varphi (z_2)d{z}_2,$$

and

$$E(Y)=E_{G,\mathbf{Z}}E(Y|g,\mathbf{z})={\alpha }_2+{\beta }_2\delta .$$

For each combination of parameter values, we ran 10,000 simulations, each consisting of 1500 cases and 1500 controls. For each simulated dataset, we derive the ML, pseudo-likelihood, naïve, cases-only, controls-only, weighted, adaptively weighted, adjusted and survey-weighted estimates of the risk effect for the secondary trait and the corresponding standard error estimates. We used the “nmk” function within the “dfoptim” package in R to implement the robust Nelder-Mead algorithm for derivative-free optimization of the likelihood and the “hessian” function from the “numDeriv” package to derive Hessian matrices at parameter estimates. We judge the different estimators based on the amount of bias they incur in estimating the risk effect and their mean squared error (MSE). Using the standard error estimates we construct 95% Wald-type confidence intervals for the risk effect and study their coverage probabilities. Also, we examine Type I error and power for corresponding Wald tests conducted at an intended 5% significance level.

Figures 1 and 2 present the key simulation results for the binary and continuous secondary traits, respectively. The upper row plots the bias, MSE, and coverage probability estimates for the different estimators. In the lower row the estimated Type I error and power are displayed for the corresponding tests under the null and the alternative scenarios. In Figures 1 and 2 we demonstrate the performance of the likelihood-based methods in comparison with only three other methods: the controls-only, adaptively weighted, and survey approaches. The remaining methods (naïve, cases-only, weighted and adjusted), are hugely biased, exhibit extremely large MSEs, and suffer from poor coverage (Figure 4 in Appendix).

Figure 1.

Binary secondary trait

Figure 2.

Continuous secondary trait

We observe the same performance patterns for binary (Figure 1) and continuous (Figure 2) secondary traits. The leftmost plot in the upper row displays the bias incurred by the different estimators. The controls-only and the adaptively weighted methods are considerably biased for non-zero values of β₃. The likelihood-based and survey-weighted estimators are essentially unbiased. The corresponding MSEs for the estimators are shown in the middle plot of the upper row. Both the ML and pseudo-likelihood estimators, as well as the survey-weighted estimator, have smaller mean squared errors than the controls-only and the adaptively weighted estimators for most β₃ values examined. In particular, the two likelihood-based estimators have practically the same MSEs and perform slightly better than the survey-weighted estimator, especially for the continuous trait. The rightmost plot in the upper row presents the estimated coverage probabilities. The coverage for the controls-only and the adaptively weighted methods drops considerably for large β₃. For the confidence intervals corresponding to the likelihood-based and survey methods, the coverage is maintained at the nominal level.

Only the likelihood and the survey approaches maintain the correct Type I error throughout. The power for the Wald tests corresponding to the likelihood and survey methods ranges between 62% to 75% for the binary setup, and varies roughly from 71% to 95% for the continuous setup, whereas the controls-only and the adaptive methods have power as low as 6% (21%) to as high as 100% (95%) for the continuous (binary) trait. But their apparently superior power for large values of β₃ is largely illusory, as they suffer from substantially inflated Type I error for those β₃ values. The key message from the simulations is that the relative advantage of the likelihood and the survey methods over the controls-only and the adaptively weighted methods is preserved across the simulation setups. Furthermore, the likelihood methods can have substantial power advantage over the survey method, evident in the plots for the continuous secondary trait. The mean squared error estimates for the continuous trait suggest that the likelihood estimators can be almost one and a half times more efficient than the survey-weighted estimator. The performances of the ML and pseudo-likelihood estimators are comparable.

4.2.1 Evaluation of robustness

We illustrate the robustness of the proposed ML method by generating data for the binary secondary trait under an alternative model. Earlier, we had simulated data from the Palmgren model. Here we generate data from the following bivariate model, proposed by Lin and Zeng (2009),

$$\mathit{\text{logit}}P(Y=1|g,\mathbf{z})={\alpha }_2+{\beta }_{2}g+{\mathbf{\gamma }}_{\mathbf{2}}\prime \mathbf{z}\mathit{\text{logit}}P(D=1|g,\mathbf{z},y)={\alpha }_1+{\beta }_{1}g+{\mathit{\gamma }}_{\mathbf{1}}\prime \mathbf{z}+\tau y.$$

We note that the disease model above does not preserve the standard logistic marginal distribution for primary disease risk and does not allow for interaction between the genetic variant and the secondary trait on disease risk. The simulation setup is as before. We vary the correlation between the primary and secondary phenotypes, denoted by τ in the disease model, between −0.7 and 0.7. We note that estimates from all the methods are practically unbiased and that Type I error is maintained; the adaptively weighted method is slightly conservative.

5 Discussion

The recent attention to the analysis of secondary traits in case/control genome scans has spurred a variety of methodological efforts. However, it is not clear that the genetics literature has utilized the full range of longstanding methods, including survey-based weighted estimators (Scott and Wild, 2002; Binder, 1983), the Palmgren model (Palmgren, 1989) and the work of Lee et al. (1998). The paper by Jiang et al. (2006) provided a relatively complete taxonomy of various approaches to the problem, but key choices in the modeling of primary and secondary effects had remained open. The more recent work of Lin and Zeng (2009); Li et al. (2010); Li and Gail (2012); He et al. (2012); Wang and Shete (2011, 2012) have provided specific solutions, relevant for the genomic scan context, but with constraints such that it is not clear that a comprehensive approach has been available.

Our likelihood approach is designed to encompass the data types typically encountered for genome scans. The proposed framework can handle continuous secondary traits, as well as the binary traits which have received more attention. In addition, our models allow for interaction between the genetic variant and the secondary trait on primary disease risk. We do not require the rare-disease assumption, i.e., our method can be used to to analyze secondary phenotypes for data from case-control studies of both rare and common primary diseases. We have developed our approach in the context of genome scans, but it is relevant to any standard case/control design. However, our joint model may be particularly useful in a genetic context, for which the parameters are interpretable quantities elucidating genetic effects on risk for both traits, as well as additional trait-trait correlation unexplained by the SNP.

Although demonstrated for a single SNP in this paper, our method can, in principle, be applied to genome-wide association data. However, it will be computationally challenging, as will also be any other retrospective method handling covariates nonparametrically. Derivative-based numerical optimization techniques, as well as C routines for numerical optimization, are likely to provide a faster solution. To reduce computational burden of a genome scan, we can employ a two-stage approach whereby we first screen SNPs using one of the faster but reasonably accurate methods, such as the survey approach, to identify SNPs that might be potentially significant and then follow-up on the interesting SNPs using the ML method.

Our formulation of the joint model for the two traits arises from viewing the pair as a bivariate response, and differs from the usual models based on conditional factorization of the joint distribution. In particular, specifying a logistic model for the conditional distribution of disease status given the secondary trait distorts the natural logistic marginal distribution for primary disease risk, resulting in incompatible models for the primary trait when performing primary vs. secondary analysis. Our parameterization of the joint distribution respects the conventional logistic choice for the marginal distribution of the disease trait, whether the secondary trait is binary or continuous. For binary secondary traits, this property is achieved using a well-known bivariate (Palmgren) logistic model. For continuous secondary traits we achieve the intended marginal model via a two-stage latent variable approach. It turns out that our model for the continuous trait is a special case of the Gaussian copula model (He et al., 2012) for a single normally distributed secondary phenotype.

We emphasize that we provide algorithms that allow us to deal with covariates nonparametrically. In our approximate profile likelihood approach, we reparameterize the profile likelihood to provide a closed form expression eliminating the need for explicit maximization over high-dimensional nuisance parameters, thus moving a step forward from theoretically discussing profile likelihood approaches to handling covariates nonparametrically. We present the pseudo-likelihood approach as another attractive way of dealing with the covariate distribution. We note that both the likelihood approaches yield almost identical results. The closed form expression for the reparameterized profile likelihood can be treated just like a likelihood to obtain the ML estimate and its standard error. The pseudo-likelihood approach, on the other hand, is intuitively very appealing and easily provides estimates; but variance estimation adds to the computational complexity. Yet another option would be to use Bayesian techniques. As one reviewer pointed out, we can use a Dirichlet prior on the probability masses ψ = {ψ₁, ψ₂, …, ψ_L} for Z and then integrate out ψ as in Zhang and Liu (2007). However, in absence of prior knowledge on covariate distribution, the likelihood and Bayesian approaches will provide very similar results.

We have assumed the genetic and environmental exposures to be independently distributed in the underlying population. We emphasize that the independence assumption applies to the general population, so that after selection by case-control status, they may be dependent (due to their common dependence on D). Thus the assumption is not as restrictive as it may seem. Moreover, for specific genetic applications, the assumption may be reasonable. For random environmental exposures, for instance, it may be entirely reasonable to assume that the genotype does not cause the exposure. The reciprocal assumption is also very standard, i.e. the exposure does not ”cause” genotype, which is fixed from conception. However, if gene-environment independence assumption seems biologically implausible, we can easily relax this assumption. Earlier, we used Z to denote covariates and G the SNP genotype. Instead, Z will now contain all genetic and environmental factors together, including gene-environment interactions, and we model P(z) nonparametrically as before. On the other hand, if it is reasonable to assume independence of genetic and environmental exposures in the population, our framework allows us to exploit this and still leave the distribution of the environmental exposures to be nonparametric.

Our setup lends itself to performing corrections for secondary trait risk estimates which are subject to significance bias, commonly known as the “winner’s curse”. Such bias correction is necessary if the SNPs have been initially selected based on association with the primary trait, and approaches based on likelihood models (Ghosh et al., 2008) can be extended to encompass our joint likelihood.

Several proposed methods for secondary analysis require knowledge of the disease prevalence. The survey approach uses the ratio of sampling fractions, a function of disease prevalence, for appropriate weighting of cases and the controls. The method of moments approach proposed by Wang and Shete (2011) involves knowing the prevalence for the secondary trait in addition to the disease prevalence. In principle, likelihood-based methods do not require specification of the disease prevalence. However, when disease prevalence is assumed unknown, the estimation of intercept parameter α₁ can lead to numerically unstable solutions, as has been noted by others (Lin and Zeng, 2009; Chatterjee and Carroll, 2005). In practice, a sensitivity analysis is warranted to determine the degree to which inference is affected by the assumed prevalence, and robust approaches to estimate the prevalence in the context of our models is an important area for further exploration.

Figure 3.

Performance under alternative model

Appendix

7.1 The profile log-likelihood

Let n_+++l denote the number of observations in the sample with Z = z_l, l = 1, …, L. The retrospective log-likelihood in (4) can then be expressed as

$$l=\sum _{u=1}^n\text{log}P(d_u,y_u|g_u,{\mathbf{z}}_u;\mathit{\theta })+\sum _{u=1}^n\text{log}P(g_u;\delta )+\sum _{l=1}^L{n}_{+++l}\text{log}{\psi }_l-\sum _{i=0}^1{n}_i\text{log}\sum _{l=1}^{L}P(D=i|{\mathbf{z}}_l;{\mathit{\theta }}_1){\psi }_l.$$ (8)

Using a Lagrange multiplier λ to account for the constraint ${\sum }_{l=1}^L{\psi }_l=1$ and then maximizing the retrospective log-likelihood with respect to ψ_l, we obtain

$$\frac{n_{+++l}}{{\psi }_l}-\sum _i{n}_i\frac{P(D=i|{\mathbf{z}}_l;{\mathit{\theta }}_1)}{P(D=i)}-\lambda =0.$$ (9)

Multiplying (9) by ψ_l and summing over l gives us λ = 0 which on substitution results in

$${\psi }_l=\frac{n_{+++l}}{{\sum }_i{\mu }_{i}P(D=i|{\mathbf{z}}_l;{\mathit{\theta }}_1)},$$ (10)

where ${\mu }_i=\frac{n_i}{P(D=i)}$. We note that $\frac{n_0}{{\mu }_0}+\frac{n_1}{{\mu }_1}=1$. Substituting ψ_l in the expression for μ_i we have

$$n_i=\sum _l{n}_{+++l}\frac{P(D=i|{\mathbf{z}}_l;{\mathit{\theta }}_1){\mu }_i}{{\sum }_{i}P(D=i|{\mathbf{z}}_l;{\mathit{\theta }}_1){\mu }_i},i=0,1.$$ (11)

We substitute (10) in (8) to obtain the following expression that is equivalent to the profile log-likelihood up to two additional nuisance parameters, μ_i, i = 0, 1,

$$l^{*}(\mathit{\eta },\mathit{\mu }(\mathit{\eta }))=\sum _{u=1}^n\text{log}P(d_u,y_u|g_u,{\mathbf{z}}_u;\mathit{\theta })+\sum _{u=1}^n\text{log}P(g_u;\delta )-\sum _{l=1}^L{n}_{+++l}\text{log}\sum _i{\mu }_{i}P(D=i|{\mathbf{z}}_l;{\mathit{\theta }}_1)+\sum _{i=0}^1{n}_i\text{log}{\mu }_i=\sum _{u=1}^n\text{log}{P}_{d_u,y_u,g_u}^{*}({\mathbf{z}}_u;\mathit{\eta },\mathit{\mu }).$$

where μ_i’s satisfy (11) and $P_{d_u,y_u,g_u}^{*}({\mathbf{z}}_u;\mathit{\eta },\mathit{\mu })$ is defined as

$$P_{d_u,y_u,g_u}^{*}({\mathbf{z}}_u;\mathit{\eta },\mathit{\mu })=\frac{P(d_u,y_u|g_u,{\mathbf{z}}_u;\mathit{\theta })P(g_u;\delta ){\mu }_u}{{\sum }_{i=0}^{1}P(D=i|{\mathbf{z}}_u;{\mathit{\theta }}_1){\mu }_i}.$$

We note that

$$\frac{\partial l^{*}(\mathit{\eta },\mathit{\mu })}{\partial {\mu }_i}=\frac{n_i-{\sum }_l{n}_{+++l}\frac{P(D=i|{\mathbf{z}}_l;{\mathit{\theta }}_1){\mu }_i}{{\sum }_{i}P(D=i|{\mathbf{z}}_l;{\mathit{\theta }}_1){\mu }_i}}{{\mu }_i}.$$ (12)

Comparing (12) with (11), we realize that (11) can be replaced by $\frac{\partial l*(\mathit{\eta },\mathit{\mu })}{\partial {\mu }_i}=\mathbf{0},i=0,1$. Thus, we can treat l*(η, μ) as a log-likelihood, although it is not an actual log-likelihood, and maximize it with respect to (η, μ) to obtain η̂. We further note that l*(η, μ) can be expressed in terms of η and $\kappa =\frac{{\mu }_1}{{\mu }_0}$ as follows:

$$l^{*}(\mathit{\eta },\mathit{\mu })=\sum _{u=1}^n\text{log}{P}_{d_u,y_u,g_u}^{*}({\mathbf{z}}_u;\mathit{\eta },\kappa )=l^{*}(\mathit{\eta },\kappa ),$$ (13)

with

$$P_{d_u,y_u,g_u}^{*}({\mathbf{z}}_u;\mathit{\eta },\kappa )=\frac{P(d_u,y_u|g_u,{\mathbf{z}}_u;\mathit{\theta })P(g_u;\delta ){\kappa }^{d_u}}{{\sum }_{i=0}^{1}P(D=i|{\mathbf{z}}_u;\mathit{\theta }){\kappa }^i}.$$

7.2 Asymptotic theory for ML estimate

We first show that E(S*(ϕ)) = 0. The proof is presented in the following subsection. We then assume that n goes to infinity with n₁/n and n₂/n remaining fixed. Under this assumption it can be shown, by expanding S*(ϕ̂) about the true value ϕ and then applying standard procedures, that $\sqrt{n}(\mathit{\varphi ̂}-\mathit{\varphi })$ converges in distribution to a normal random variable with mean zero and covariance matrix

$$n\mathit{\text{Cov}}(\mathit{\varphi ̂})=n{J}^{*}{(\mathit{\varphi })}^{-1}\mathit{\text{Cov}}(S^{*}(\mathit{\varphi }))J^{*}{(\mathit{\varphi })}^{-1}.$$

We then show in Subsection 7.2.2 that Cov(S*(ϕ)) = J*(ϕ) − Γ(ϕ). This leads to

$$\mathit{\text{Cov}}(\mathit{\varphi ̂})=J^{*}{(\mathit{\varphi })}^{-1}-J^{*}{(\mathit{\varphi })}^{-1}\mathrm{\Gamma }(\mathit{\varphi })J^{*}{(\mathit{\varphi })}^{-1}.$$

7.2.1 E(S*(ϕ)) = 0

$$E(S^{*}(\mathit{\varphi }))=\sum _u{E}_{d_u}(\frac{\partial }{\partial \mathit{\varphi }}\text{log}{P}_{d_u,y_u,g_u}^{*}({\mathbf{z}}_u;\mathit{\varphi })),\text{where}{E}_{d_u}(․)=E(․|D=d_u)=\sum _i{n}_i{E}_i(\frac{\partial \text{log}{P}_{iyg}^{*}(\mathbf{z};\mathit{\varphi })}{\partial \mathit{\varphi }})=\sum _i{n}_i\sum _{y,g,\mathbf{z}}\frac{\partial \text{log}{P}_{iyg}^{*}(\mathbf{z};\mathit{\varphi })}{\partial \mathit{\varphi }}P(y,g,\mathbf{z}|D=i)=\sum _i{n}_i\sum _{y,g,\mathbf{z}}\frac{1}{P_{iyg}^{*}(\mathbf{z})}\frac{\partial P_{iyg}^{*}(\mathbf{z};\mathit{\varphi })}{\partial \mathit{\varphi }}\frac{P(D=i,y|g,\mathbf{z})P(g)}{P(D=i)}P(\mathbf{z})=\sum _i\sum _{y,g,\mathbf{z}}\frac{\partial P_{iyg}^{*}(\mathbf{z};\mathit{\varphi })}{\partial \mathit{\varphi }}f(\mathbf{z})P(\mathbf{z}),\text{where}f(\mathbf{z})=(\frac{n1}{\kappa }+n_0)\sum _i{\kappa }^{i}P(D=i|\mathbf{z})=\sum _{\mathbf{z}}\sum _{i,y,g}\frac{\partial P_{iyg}^{*}(\mathbf{z};\mathit{\varphi })}{\partial \mathit{\varphi }}f(\mathbf{z})P(\mathbf{z})=0,\text{since}\sum _{i,y,g}{P}_{iyg}^{*}(\mathbf{z};\mathit{\varphi })=1.$$

7.2.2 Cov(S*(ϕ)) = J*(ϕ) − Γ(ϕ)

$$\mathit{\text{Cov}}(S^{*}(\mathit{\varphi }))=\sum _u{E}_{d_u}(\frac{\partial }{\partial \mathit{\varphi }}\text{log}{P}_{d_u,y_u,g_u}^{*}({\mathbf{z}}_u;\mathit{\varphi })\frac{\partial }{\partial \mathit{\varphi }\prime }\text{log}{P}_{d_u,y_u,g_u}^{*}({\mathbf{z}}_u;\mathit{\varphi }))-\sum _u{E}_{d_u}(\frac{\partial }{\partial \mathit{\varphi }}\text{log}{P}_{d_u,y_u,g_u}^{*}({\mathbf{z}}_u;\mathit{\varphi }))E_{d_u}(\frac{\partial }{\partial \mathit{\varphi }\prime }\text{log}{P}_{d_u,y_u,g_u}^{*}({\mathbf{z}}_u;\mathit{\varphi }))=\sum _i{n}_i{E}_i(\frac{\partial }{\partial \mathit{\varphi }}\text{log}{P}_{iyg}^{*}(\mathbf{z};\mathit{\varphi })\frac{\partial }{\partial \mathit{\varphi }\prime }\text{log}{P}_{iyg}^{*}(\mathbf{z};\mathit{\varphi }))-\sum _i{n}_i{E}_i(\frac{\partial }{\partial \mathit{\varphi }}\text{log}{P}_{iyg}^{*}(\mathbf{z};\mathit{\varphi }))E_i(\frac{\partial }{\partial \mathit{\varphi }\prime }\text{log}{P}_{iyg}^{*}(\mathbf{z};\mathit{\varphi })).$$

Using the same argument as before it can be shown that

$$\sum _i{n}_i{E}_i(\frac{1}{P_{iyg}^{*}(\mathbf{z})}\frac{{\partial }^2}{\partial \mathit{\varphi }\partial \mathit{\varphi }\prime }{P}_{iyg}^{*}(\mathbf{z}))=0$$

which implies that

$$\sum _i{n}_i{E}_i(\frac{{\partial }^2}{\partial \mathit{\varphi }\partial \mathit{\varphi }\prime }\text{log}{P}_{iyg}^{*}(\mathbf{z}))=-\sum _i{n}_i{E}_i(\frac{\partial }{\partial \mathit{\varphi }}\text{log}{P}_{iyg}^{*}(\mathbf{z})\frac{\partial }{\partial \mathit{\varphi }\prime }\text{log}{P}_{iyg}^{*}(\mathbf{z})).$$

We have also shown before that

$$E_i(\frac{\partial }{\partial \mathit{\varphi }}\text{log}{P}_{iyg}^{*}(\mathbf{z};\mathit{\varphi }))=\frac{1}{n_i}\sum _{y,g,\mathbf{z}}\frac{\partial }{\partial \mathit{\varphi }}{P}_{iyg}^{*}(\mathbf{z};\mathit{\varphi })f(\mathbf{z})P(\mathbf{z})=\frac{1}{n_i}E(f(\mathbf{z})\frac{\partial }{\partial \mathit{\varphi }}{P}_i^{*}(\mathbf{z};\mathit{\varphi })),$$

where $P_i^{*}(\mathbf{z})=\frac{P(D=i|\mathbf{z}){\kappa }^i}{{\sum }_i{\kappa }^{i}P(D=i|\mathbf{z})}$. Then

$$\sum _i{n}_i{E}_i(\frac{\partial }{\partial \mathit{\varphi }}\text{log}{P}_{iyg}^{*}(\mathbf{z};\mathit{\varphi }))E_i(\frac{\partial }{\partial \mathit{\varphi }\prime }\text{log}{P}_{iyg}^{*}(\mathbf{z};\mathit{\varphi }))=\sum _{i}E(f(\mathbf{z})\frac{\partial }{\partial \mathit{\varphi }}{P}_i^{*}(\mathbf{z};\mathit{\varphi }))\frac{1}{n_i}E(f(\mathbf{z})\frac{\partial }{\partial \mathit{\varphi }\prime }{P}_i^{*}(\mathbf{z};\mathit{\varphi }))=\frac{1}{n_0}E(f(\mathbf{z})\frac{\partial }{\partial \mathit{\varphi }}{P}_0^{*}(\mathbf{z};\mathit{\varphi }))E(f(\mathbf{z})\frac{\partial }{\partial \mathit{\varphi }\prime }{P}_0^{*}(\mathbf{z};\mathit{\varphi }))+\frac{1}{n_1}E(f(\mathbf{z})\frac{\partial }{\partial \mathit{\varphi }}{P}_1^{*}(\mathbf{z};\mathit{\varphi }))E(f(\mathbf{z})\frac{\partial }{\partial \mathit{\varphi }\prime }{P}_1^{*}(\mathbf{z};\mathit{\varphi }))=\frac{1}{n_0}E(-f(\mathbf{z})\frac{\partial }{\partial \mathit{\varphi }}{P}_1^{*}(\mathbf{z};\mathit{\varphi }))E(-f(\mathbf{z})\frac{\partial }{\partial \mathit{\varphi }\prime }{P}_1^{*}(\mathbf{z};\mathit{\varphi }))+\frac{1}{n_1}E(f(\mathbf{z})\frac{\partial }{\partial \mathit{\varphi }}{P}_1^{*}(\mathbf{z};\mathit{\varphi }))E(f(\mathbf{z})\frac{\partial }{\partial \mathit{\varphi }\prime }{P}_1^{*}(\mathbf{z};\mathit{\varphi }))=(\frac{1}{n_0}+\frac{1}{n_1})E(f(\mathbf{z})\frac{\partial }{\partial \mathit{\varphi }}{P}_1^{*}(\mathbf{z};\mathit{\varphi }))E(f(\mathbf{z})\frac{\partial }{\partial \mathit{\varphi }\prime }{P}_1^{*}(\mathbf{z};\mathit{\varphi })).$$

∴

$$\mathit{\text{Cov}}(S^{*}(\mathit{\varphi }))=-\sum _i{n}_i{E}_i(\frac{{\partial }^2}{\partial \mathit{\varphi }\partial \mathit{\varphi }\prime }\text{log}{P}_{iyg}^{*}(\mathbf{z};\mathit{\varphi }))-(\frac{1}{n_0}+\frac{1}{n_1})E(f(\mathbf{z})\frac{\partial }{\partial \mathit{\varphi }}{P}_1^{*}(\mathbf{z};\mathit{\varphi }))E(f(\mathbf{z})\frac{\partial }{\partial \mathit{\varphi }\prime }{P}_1^{*}(\mathbf{z};\mathit{\varphi }))=-E(\frac{\partial }{\partial \mathit{\varphi }\prime }{S}^{*}(\mathit{\varphi }))-\mathrm{\Gamma }(\mathit{\varphi })=J^{*}(\mathit{\varphi })-\mathrm{\Gamma }(\mathit{\varphi }).$$

7.3 Asymptotic theory for pseudo-likelihood estimate

Gong and Samaniego (1981), under some regularity conditions, showed that θ̃₂ is consistent when θ̃₁ is consistent. Also, suppose that

$$\sqrt{n}\left(\begin{array}{c}\hfill {\mathit{\theta ̃}}_1-{\mathit{\theta }}_1^0\hfill \\ \hfill \frac{\partial l_1({\mathit{\theta }}_1^0,{\mathit{\theta }}_2^0)/\partial {\mathit{\theta }}_2}{n}\hfill \end{array}\right){\to }_{d}N(\mathbf{0},(\begin{array}{cc}\hfill {\mathrm{\Sigma }}_{11}\hfill & \hfill {\mathrm{\Sigma }}_{12}\hfill \\ \hfill {\mathrm{\Sigma }}_{12}^{\prime }\hfill & \hfill I_{22}\hfill \end{array}\left)\right),$$

where $({\mathit{\theta }}_1^0,{\mathit{\theta }}_2^0)\prime$ is the true value of (θ₁, θ₂)′ and

$$I_{22}=\underset{n\to \infty }{\text{lim}}\left[E\right\{\frac{-{\partial }^2{l}_1({\mathit{\theta }}_1^0,{\mathit{\theta }}_2^0)/\partial {\mathit{\theta }}_2^2}{n};{\mathit{\theta }}_1^0,{\mathit{\theta }}_2^0\left\}\right].$$

Then, under some regularity conditions, discussed in Gong and Samaniego,

$$\sqrt{n}({\mathit{\theta ̃}}_2-{\mathit{\theta }}_2^0){\to }_{d}N(\mathbf{0},{\mathrm{\Sigma }}_{22}),$$

with

$${\mathrm{\Sigma }}_{22}=I_{22}^{-1}(I_{22}-2I_{12}^{\prime }{\mathrm{\Sigma }}_{12}+I_{12}^{\prime }{\mathrm{\Sigma }}_{11}{I}_{12})I_{22}^{-1}=I_{22}^{-1}{I}_{22}^{*}{I}_{22}^{-1},$$ (14)

where

$$I_{12}=\underset{n\to \infty }{\text{lim}}\left[E\right\{\frac{-{\partial }^2{l}_1({\mathit{\theta }}_1^0,{\mathit{\theta }}_2^0)/\partial {\mathit{\theta }}_1\partial {\mathit{\theta }}_2}{n};{\mathit{\theta }}_1^0,{\mathit{\theta }}_2^0\left\}\right].$$

I₂₂ is equal to the variance-covariance matrix of the score function for θ₂ evaluated at $({\mathit{\theta }}_1^0,{\mathit{\theta }}_2^0)$ and $I_{22}^{*}$ is the limiting variance-covariance matrix of $\sqrt{n}\partial {\mathrm{l̃}}_1({\mathit{\theta }}_2)/\partial {\mathit{\theta }}_2$.

7.3.1 Variance Estimation

We use

$${Î}_{22}={-\frac{1}{n}\frac{{\partial }^2{l}_1({\mathit{\theta }}_1,{\mathit{\theta }}_2)}{\partial {\mathit{\theta }}_2^2}|}_{({\mathit{\theta ̃}}_1,{\mathit{\theta ̃}}_2)}={-\frac{1}{n}\frac{{\partial }^2{\mathrm{l̃}}_1({\mathit{\theta }}_2)}{\partial {\mathit{\theta }}_2^2}|}_{{\mathit{\theta ̃}}_2}$$

and

$${Î}_{12}={-\frac{1}{n}\frac{{\partial }^2{l}_1({\mathit{\theta }}_1,{\mathit{\theta }}_2)}{\partial {\mathit{\theta }}_1\partial {\mathit{\theta }}_2}|}_{({\mathit{\theta ̃}}_1,{\mathit{\theta ̃}}_2)}.$$

We note that

$${\mathrm{\Sigma }}_{12}=\mathit{\text{cov}}({\mathit{\theta ̃}}_1-{\mathit{\theta }}_1^0,\frac{\partial }{\partial {\mathit{\theta }}_2}{l}_1({\mathit{\theta }}_1^0,{\mathit{\theta }}_2^0))$$

If θ̃₁ is the ML estimate obtained by maximizing g(θ₁), by Taylor expansion, ${\mathit{\theta ̃}}_1-{\mathit{\theta }}_1^0$ is asymptotically equivalent to

$$\mathit{\text{Var}}({\mathit{\theta ̃}}_1;{\mathit{\theta }}_1^0)\frac{\partial }{\partial {\mathit{\theta }}_1}g({\mathit{\theta }}_1^0)$$

which leads to

$${\mathrm{\Sigma }}_{12}=\mathit{\text{Var}}({\mathit{\theta ̃}}_1;{\mathit{\theta }}_1^0)\mathit{\text{cov}}(\frac{\partial }{\partial {\mathit{\theta }}_1}g({\mathit{\theta }}_1^0),\frac{\partial }{\partial {\mathit{\theta }}_2}{l}_1({\mathit{\theta }}_1^0,{\mathit{\theta }}_2^0)).$$

We can easily estimate the asymptotic variance of θ̃₁ by

$$\widehat{\text{Var}}({\mathit{\theta ̃}}_1)={(-\frac{{\partial }^2}{\partial {{\mathit{\theta }}_1}^2}g({\mathit{\theta ̃}}_1))}^{-1}.$$

Now we have only to estimate $\mathit{\text{cov}}(\frac{\partial }{\partial {\mathit{\theta }}_1}g({\mathit{\theta }}_1^0),\frac{\partial }{\partial {\mathit{\theta }}_2}{l}_1({\mathit{\theta }}_1^0,{\mathit{\theta }}_2^0))$. If we can write

$$\frac{\partial }{\partial {\mathit{\theta }}_1}g({\mathit{\theta }}_1^0)=\sum _u\frac{\partial }{\partial {\mathit{\theta }}_1}{g}_u({\mathit{\theta }}_1^0)$$

and

$$\frac{\partial }{\partial {\mathit{\theta }}_2}{l}_1({\mathit{\theta }}_1^0,{\mathit{\theta }}_2^0)=\sum _u\frac{\partial }{\partial {\mathit{\theta }}_2}{l}_{1u}({\mathit{\theta }}_1^0,{\mathit{\theta }}_2^0),$$

then we can use the following to estimate the covariance,

$$\widehat{\mathit{\text{Cov}}}(\frac{\partial }{\partial {\mathit{\theta }}_1}g({\mathit{\theta }}_1^0),\frac{\partial }{\partial {\mathit{\theta }}_2}{l}_1({\mathit{\theta }}_1^0,{\mathit{\theta }}_2^0))=\sum _u\frac{\partial }{\partial {\mathit{\theta }}_1}{g}_u({\mathit{\theta ̃}}_1),\frac{\partial }{\partial {\mathit{\theta }}_2}{l}_{1u}({\mathit{\theta ̃}}_1,{\mathit{\theta ̃}}_2)\prime =\sum _u\frac{\partial }{\partial {\mathit{\theta }}_1}{g}_u({\mathrm{\theta ̃}}_1)\frac{\partial }{\partial {\mathit{\theta }}_2}{\mathrm{l̃}}_{1u}({\mathit{\theta ̃}}_2)\prime$$

Thus,

$${\mathrm{\Sigma ̂}}_{12}={(-\frac{{\partial }^2}{\partial {{\mathit{\theta }}_1}^2}g({\mathit{\theta ̃}}_1))}^{-1}\sum _u\frac{\partial }{\partial {\mathit{\theta }}_1}{g}_u({\mathit{\theta ̃}}_1)\frac{\partial }{\partial {\mathit{\theta }}_2}{\mathrm{l̃}}_{1u}({\mathit{\theta ̃}}_2)\prime .$$

Also,

$${\mathrm{\Sigma ̂}}_{11}=n{(-\frac{{\partial }^2}{\partial {{\mathit{\theta }}_1}^2}g({\mathit{\theta ̃}}_1))}^{-1}$$

Plugging in all the corresponding estimates in (14) gives

$${\mathrm{\Sigma ̂}}_{22}={Î}_{22}^{-1}({Î}_{22}-2{Î}_{12}^{\prime }{\mathrm{\Sigma ̂}}_{12}+{Î}_{12}^{\prime }{\mathrm{\Sigma ̂}}_{11}{Î}_{22}){Î}_{22}^{-1}.$$ (15)