PLEMT: A NOVEL PSEUDOLIKELIHOOD BASED EM TEST FOR HOMOGENEITY IN GENERALIZED EXPONENTIAL TILT MIXTURE MODELS

Abstract

Motivated by analyses of DNA methylation data, we propose a semiparametric mixture model, namely the generalized exponential tilt mixture model, to account for heterogeneity between differentially methylated and non-differentially methylated subjects in the cancer group, and capture the differences in higher order moments (e.g. mean and variance) between subjects in cancer and normal groups. A pairwise pseudolikelihood is constructed to eliminate the unknown nuisance function. To circumvent boundary and non-identifiability problems as in parametric mixture models, we modify the pseudolikelihood by adding a penalty function. In addition, the test with simple asymptotic distribution has computational advantages compared with permutation-based test for high-dimensional genetic or epigenetic data. We propose a pseudolikelihood based expectation–maximization test, and show the proposed test follows a simple chi-squared limiting distribution. Simulation studies show that the proposed test controls Type I errors well and has better power compared to several current tests. In particular, the proposed test outperforms the commonly used tests under all simulation settings considered, especially when there are variance differences between two groups. The proposed test is applied to a real data set to identify differentially methylated sites between ovarian cancer subjects and normal subjects.

1. INTRODUCTION

DNA methylation plays an important role in the development of many types of cancer. To identify differentially methylated Cytosine-Phosphate-Guanine (CpG) sites between cancer and normal subjects is one of the central tasks to understand contributions of the DNA methylation process on cancer development. Usually, cancer subjects are more heterogeneous in terms of DNA methylation distribution as cancer subjects may have different subtypes of cancer, different stages of cancer, and different history of treatment (Mikeska et al., 2010). Thus, DNA methylation levels of some cancer subjects may follow one distribution and are differentially methylated compared to normal subjects, while the rest of cancer subjects may follow a similar distribution as that of normal subjects and are not differentially methylated. The epigenetic heterogeneity in cancer has gained tremendous interest lately (Brocks et al., 2014; Oakes et al., 2014; Easwaran et al., 2014). CpG sites with high variability among cancer samples can potentially be used as epigenetic biomarkers for determining the stage of cancer progression and designing personalized treatment. Most of the existing methods for DNA methylation data focus on testing for differences in means between the cancer and normal groups, which does not fully capture the differences in variances in DNA methylation data. There is evidence that there are not only differences in DNA methylation means but also differences in DNA methylation variations between the cancer and normal groups (Hansen et al., 2011; Gervin et al., 2011). A recently proposed method DiffVar (Phipson and Oshlack, 2014) tests the equality of variances in two groups by performing a t-test on the absolute or squared deviations of the methylation levels from the group mean. It is, however, restricted in comparing the variances and cannot detect mean differences in two groups. More importantly, most of the existing methods for DNA methylation data are distribution based methods, including the logit-normal mixture model (Siegmund et al., 2004), the beta mixture model (Houseman et al., 2008), the uniform–truncated–normal–uniform mixture model (Wang, 2011), and a GLM based method (Ahn and Wang, 2013). However, due to the heterogeneity in distributions of DNA methylation across loci (Huang et al., 2013), it is insufficient to assume a parametric distribution for all loci. Fitting site-specific parametric models may not be feasible for a large number of loci, and also leads to difficulties in model interpretations.

To relax parametric model assumptions, exponential tilt mixture models (ETMM) have been considered (Qin, 1999; Zou et al., 2002; Tan, 2009). Specifically, subjects under one condition are sampled from a population with the baseline density function f(x), and subjects under the other condition are sampled from a mixture population with the density function h(x). The density h(·) and the relationship between the densities f(·) and g(·) can be formulated as follows,

$$h(x)=(1-\lambda )f(x)+\lambda g(x);\text{log}\{g(x)/f(x)\}=\alpha +\beta x,$$ (1)

where g(·) is defined as the density function of the methylation levels for the subpopulation in the case group that are differentially methylated, λ is an unknown mixture proportion parameter, β is an unknown parameter and $\alpha =-\text{log}\{{\int }_{-\infty }^{\infty }\text{exp}(\beta x)f(x)\mathit{\text{dx}}\}$ is a normalizing constant for the density function g(x). Note that β = 0 implies α = 0. Model (1) contains many parametric models as special cases, such as the mixture of normal distributions with different means but equal variances, and the mixture of gamma distributions with different shape parameters but equal scale parameters. Under the ETMM assumption, testing for homogeneity between cancer and normal groups i.e., f(·) = h(·), is equivalent to testing λ = 0 or β = 0.

It has been long recognized that testing for homogeneity in mixture models is a non-regular problem because the mixture proportion parameter λ lies on the boundary of its parameter space [0, 1] and the parameters α and β are not identifiable when λ = 0. Thus, the asymptotic distributions of tests for homogeneity are usually rather complicated and possibly dependent on the parametric distributions assumed (Davies, 1977, 1987). Recently, Qin and Liang (2011) derived a original score test under model (1) with a simple limiting chi-squared distribution. More recently, Liu et al., (2012) proposed a novel modified empirical likelihood ratio test under model (1) and developed an efficient and intuitive expectation–maximization (EM) algorithm for computing the test statistic. Despite the current success on the test of homogeneity in ETMM, the aforementioned methods only allow a scalar parameter β, which excludes important parametric distributions such as normal distributions with unequal means and unequal variances, gamma distributions with different shape and scale parameters, and beta distributions. As recent studies have observed that cancer tissues and some complex disease cases can also be characterized by an increased variability in DNA methylation patterns (Hansen et al., 2011; Issa, 2011; Teschendorff et al., 2012; Xu et al., 2013), tests that ignore this feature may lead to a substantial loss of power. We therefore extended both the score test by Qin and Liang (2011) and the modified empirical likelihood ratio test by Liu et al., (2012) by generalizing x to k(x) = (x, x²) in equation (1) to account for differences in both means and variances. However, as the simulation results summarized in Section 2 of online supplementary materials suggested, both the extended score test and the extended modified empirical likelihood ratio test have inflated Type I errors. This suggests that an alternative approach should be considered.

In this paper, we generalize the one-parameter ETMM to a multi-parameter ETMM, namely the generalized exponential tilt mixture model (GETMM), which aims to capture the differences in higher order moments between two distributions. Specifically, the right handside of equation (1) is extended to a general form of α + β^Tk (x), so that the multi-parameter ETMM includes many parametric models, such as the normal mixture model with unequal variances, the gamma mixture model, and the beta mixture model. Rather than estimating the baseline density function f(·) with the empirical likelihood procedure as in Qin and Liang (2011) and Liu et al., (2012), we construct a novel pseudolikelihood based on a conditioning procedure, which eliminates the baseline density function f(·) and avoids its estimation. To handle the non-regularity problems (i.e. boundary and non-identifiability problems), we construct a penalized pseudolikelihood where the impacts of the tuning parameter are studied. Finally, we propose an EM algorithm based test for computational efficiency and stability, which can be shown to follow a simple chi-squared limiting distribution.

The contributions of this work are three-fold. First, we develop a semiparametric model that captures the differences in higher moments between distributions. Second, we construct a novel penalized pseudolikelihood, where the unknown baseline density function f(·) is eliminated and the non-regularity problems are circumvented. Third, we propose an EM algorithm based test with a simple chi-squared limiting distribution, which is computationally efficient and stable. The pseudolikelihood EM algorithm has been proposed for handling spatial data (Varin et al., 2005), hidden Markov model (Gao and Song, 2011), and family data with multistage sampling (Choi and Briollais, 2011). The convergence property of the EM algorithm is established by Gao and Song (2011). Unlike these existing results, the estimated parameters at each iteration of EM algorithm rather than the estimated stationary point are used to construct the proposed test.

This paper is organized as follows. Section 2 describes the penalized pseudolikelihood based EM test (hereafter referred to as the PLEMT test). The asymptotic null distribution and the local asymptotic power for the PLEMT test are provided in Section 3. Simulation studies comparing Type I errors and power of the PLEMT test with existing tests are summarized in Section 4. A real data application to DNA methylation data for ovarian cancer is given in Section 5 followed by a brief discussion in Section 6. Proofs are relegated to Appendices in online supplementary materials.

2. STATISTICAL METHODOLOGY

We propose the following two-group generalized exponential tilt mixture model (GETMM). We present our model in the setting of modeling DNA methylation data for the concreteness of interpretation, while noting that it can be generally applied to any two-group testing problem for homogeneity. At the ℓ^th CpG site, let u_ℓ1, …, u_ℓn0 be independent, identically distributed (i.i.d.) DNA methylation levels in the normal group with distribution f_ℓ(u), where n₀ is the number of normal subjects. It is believed that in the cancer group, only a proportion of subjects are methylated differentially compared to those in the normal group, known as non-homogeneity or heterogeneity in methylation among cancer subjects (Kalari, 2010). Moreover, the effect of differential methylation may appear as changes in variation, in addition to potential a shift in means (Hansen et al., 2011; Gervin et al., 2011; Teschendorff et al., 2012; Xu et al., 2013). To account for such features of DNA methylation data, we assume that the i.i.d subjects v_ℓ1, …, v_ℓn1 in the cancer group follow a mixture distribution with the density h_ℓ(v) as follows

$$h_{\mathrm{\ell }}(v)=(1-{\lambda }_{\mathrm{\ell }})f_{\mathrm{\ell }}(v)+{\lambda }_{\mathrm{\ell }}{g}_{\mathrm{\ell }}(v),$$

where n₁ is the number of cancer subjects, λ_ℓ is an unknown mixture proportion parameter (0 ≤ λ_ℓ ≤ 1), and the density functions f_ℓ(v) and g_ℓ(v) are related through a multi-parameter exponential tilt

$$\text{log}\{g_{\mathrm{\ell }}(v)/f_{\mathrm{\ell }}(v)\}={\alpha }_{\mathrm{\ell }}+{{\mathit{\beta }}_{\mathrm{\ell }}}^T\mathit{k}(v).$$ (2)

Here β_ℓ = (β_ℓ1, …, β_ℓd)^T is a d-dimensional vector of unknown parameters, k(v) = {k₁(v), …, k_d(v)}^T is a vector of pre-specified functions of v, and ${\alpha }_{\mathrm{\ell }}=-\text{log}[\int \text{exp}\{{{\mathit{\beta }}_{\mathrm{\ell }}}^T\mathit{k}(v)\}{f}_{\mathrm{\ell }}(v)\mathit{\text{dv}}]$ is a normalizing constant. It is easy to see that β_ℓ = 0 implies α_ℓ = 0. For simplicity of notation, we hereafter suppress the site index ℓ. We acknowledge that the baseline density function f_ℓ(·) can be site-specific and is left completely unspecified. The parameter λ_ℓ can also be site-specific. Note that the GETMM includes many parametric mixture models. When k(v) = v, the GETMM reduces to the one-parameter ETMM described in equation (1); when k(v) = (v, v²), the GETMM includes the normal mixture model with unequal variances; when k(v) = {log(v), log(1 − v)}, the GETMM includes the beta mixture model. Both parametric models have been used to model DNA methylation data (Siegmund et al., 2004; Houseman et al., 2008).

Since the majority of differences between the cancer and normal groups may be contained in means and variances of methylation levels, we consider the GETMM with two parameters for model parsimony. While the GETMM with more than two parameters may better capture the differences in higher moments such as skewness and kurtosis, the corresponding tests may be underpowered due to the larger degree of freedom. In addition, the theoretical development of GETMM with more than two parameters is similar. Specifically, we let k(v) = (v, v²) and β = (β₁, β₂). Under this model, testing for homogeneity between two groups is equivalent to testing

$$H_0:\lambda =0\text{ or }{\beta }_1={\beta }_2=0.$$

.

Maximizing the likelihood function generally involves estimating the baseline density function f(·), typically by an empirical likelihood procedure (Owen, 1988). Here, we construct a pairwise pseudolikelihood, which eliminates f(·) by conditioning on order statistics. Specifically, the advantage of the conditioning procedure is to avoid the estimation of f(·). We consider a pair of observations from the two groups, i.e., u_i from the normal group and v_j from the cancer group. The conditional density of (u_i, v _j) given their order statistics t⁽¹⁾ = min(u_i, v_j) and t⁽²⁾ = max(u_i, v_j) can be calculated as,

$$\text{pr}(u_i,v_j|t^{(1)},t^{(2)})={\{1+R(u_i,v_j;\lambda ,\alpha ,{\beta }_1,{\beta }_2)\}}^{-1},$$ (3)

where

$$R(u_i,v_j;\lambda ,\alpha ,{\beta }_1,{\beta }_2)=\frac{(1-\lambda )+\lambda \text{ exp}(\alpha +{\beta }_1{u}_i+{\beta }_2{u}_i^2)}{(1-\lambda )+\lambda \text{ exp}(\alpha +{\beta }_1{v}_j+{\beta }_2{v}_j^2)}.$$

The derivation of equation (3) is provided in Appendix A of online supplementary materials. The baseline density function f(·) is eliminated through this conditioning procedure. This idea of conditioning was originally proposed by Kalbfleisch (1978) for rank tests and permutation tests in regression problems, and later revitalized by Liang and Qin (2000) in regression analyses under biased sampling.

For each pair of observations (u_i, v_j), we can calculate the pairwise conditional density. We then multiply all these densities together and obtain the following log pseudolikelihood function for all observations,

$${\mathbf{ℒ}}_p(\lambda ,\alpha ,{\beta }_1,{\beta }_2)=\frac{2}{n}\sum _{i=1}^{n_0}\sum _{j=1}^{n_1}-\text{log }\{1+R(u_i,v_j;\lambda ,\alpha ,{\beta }_1,{\beta }_2)\},$$

where n = n₀ + n₁. We note that the sum of log conditional densities is standardized by the total number of individuals rather than the number of summands. This is owing to the projection theory in U-statistics (Lehmann and D’Abrera, 1975). As we will show later, such modification of the loglikelihood is necessary so that the proposed test has a simple χ² limiting distribution with 2 degrees of freedom.

Under the null hypothesis, λ = 0 lies at the boundary of its parameter space, which leads to a boundary problem (Self and Liang, 1987; Chen and Liang, 2010). Furthermore, the null hypothesis holds for λ = 0 regardless of the values of α, β₁ and β₂, and holds for β₁ = β₂ = 0 regardless of the value of λ. This implies that the parameter (λ, α, β₁, β₂) is not identifiable under H₀, which results in complicated asymptotic properties of the pseudolikelihood ratio function (Chen and Chen, 2001; Zhu and Zhang, 2004). To deal with the non-identifiability problem, Qin and Liang (2011) fixed the value of λ at 1, so that the other parameters (α and β) are identifiable and can be estimated by maximizing the empirical likelihood. The score test statistic was then constructed. However, the choice of the value for the fixed λ is arbitrary, and the performance of the score test depends on the choice of λ. Alternatively, Liu et al., (2012) proposed an empirical likelihood function by adding penalty on λ. Rather than using the empirical likelihood, we propose the following penalized pseudolikelihood function to avoid the boundary and identifiability problems

$${\mathbf{ℒ}}_{\mathit{\text{pp}}}(\lambda ,\alpha ,{\beta }_1,{\beta }_2)={\mathbf{ℒ}}_p(\lambda ,\alpha ,{\beta }_1,{\beta }_2)+C\text{ log}(\lambda ),$$

where C is a positive number. The penalty is heavy when λ is close to 0 and less so when λ approaches 1. The parameter C is a multiplicative factor of such penalty and is often termed as the tuning parameter. By using the penalized pseudolikelihood, the parameter λ is bounded away from 0, and the null hypothesis is then reduced to β₁ = β₂ = 0. That is, β₁ and β₂ in the penalized pseudolikelihood function is asymptotically identifiable. When k(x) = x, it has been recommended in the literature that C can be taken as 1, and the existing tests based on penalizing λ are not sensitive to the choice of C (Chen and Chen, 2001; Fu et al., 2006a). The GETMM we considered involves both mean and variance, e.g., k(x) = (x, x²), and the recommendation on the choice of C may be different. In our simulation studies, we investigate the impact of C on the performance of the proposed test. Di and Liang (2011) have investigated the impacts of tuning parameter C on type I errors and power. They suggested to use the smallest C that still provides the correct type I error rate. However, the admixture model considered in Di and Liang (2011) is different from the model considered in this paper. Sensitivity analyses on the choice of C have been conducted and summarized in the Supplementary Materials. In general, C = 20 is recommended based on various scenarios considered in our sensitivity analyses.

In general, the parameter λ and the parameters (α, β₁, β₂) are highly intertwined with each other as in many mixture models (Bandeen-Roche et al., 1997). This can lead to numerical problems such as multiple local maxima when maximizing the pseudolikelihood. Our simulation results summarized in Table 7 of the Supplementary Materials show that the pseudolikelihood method that simultaneously maximizes over all parameters faces the problems of unstable results and inflated Type I errors. In our proposed PLEMT test, we use a set of different initial values of λ to avoid being trapped at a local maximum.

The key idea of the PLEMT procedure is that maximizing the penalized pseudolikelihood with respect to (α, β₁, β₂) for a fixed λ is more stable than maximizing over all parameters simultaneously. In DNA methylation data, consider DNA methylation levels v₁, . . . , v_n1 in the cancer group are sampled from a mixture of f(·) and g(·). That is, v₁, . . . , v_n1 are i.i.d. with a density function h(v) = (1 − λ) f(v) + λg(v). The information on whether the DNA methylation level of a particular subject in the cancer group is from the subpopulation f(·) or g(·) is considered as missing data.

Here we describe the algorithm to calculate the PLEMT test statistic. We first choose a finite set of Λ = {λ₁, …, λ_Z}, where Z is the number of points in the grid (e.g., Λ = {0.1, …1}). We then choose the number of iterations S of the EM algorithm. Although the PLEMT test is motivated by the EM algorithm, the calculation of the test statistic does not require the convergence of the EM algorithm, which has been shown in Appendix D of online supplementary materials. Generally, only a few steps of iterations is needed. This feature offers great computational advantages in analyses of high dimensional data. Sensitivity analyses in the Supplementary Materials suggest that the performance of the proposed PLEMT test is not sensitive to the choices of S and Z. Here we choose Z = 10, and S = 3. At the z^th grid value λ_z, we set the initial value ${\lambda }_z^{(1)}={\lambda }_z$, calculate the initial values of $({\alpha }_z^{(1)},{\beta }_{z1}^{(1)},{\beta }_{z2}^{(1)})$ by maximizing ${\mathbf{ℒ}}_{\mathit{\text{pp}}}({\lambda }_z^{(1)},\alpha ,{\beta }_1,{\beta }_2)$. We carry out the following EM algorithm for S − 1 times.

At the E step of the EM algorithm, we calculate the posterior probability of the j^th cancer subject being differentially methylated given v_j and $({\lambda }_z^{(s)},{\alpha }_z^{(s)},{\beta }_{z1}^{(s)},{\beta }_{z2}^{(s)})$ as,

$${\omega }_{\mathit{\text{jz}}}^{(s)}=\frac{{\lambda }_Z^{(s)}g(v_j)}{(1-{\lambda }_Z^{(s)})f(v_j)+{\lambda }_Z^{(s)}g(v_j)}=\frac{{\lambda }_Z^{(s)}\text{exp}({\alpha }_Z^{(s)}+{\beta }_{z1}^{(s)}{v}_j+{\beta }_{z2}^{(s)}{v}_j^2)}{1-{\lambda }_Z^{(s)}+{\lambda }_Z^{(s)}\text{exp}({\alpha }_Z^{(s)}+{\beta }_{z1}^{(s)}{v}_j+{\beta }_{z2}^{(s)}{v}_j^2)}.$$

We then calculate the expected complete likelihood given the data and the current parameter estimates, which only involves the parameter λ

$$\sum _{j=1}^{n_1}(1-{\omega }_{\mathit{\text{jz}}}^{(s)})\text{ log}(1-\lambda )+\sum _{j=1}^{n_1}{\omega }_{\mathit{\text{jz}}}^{(s)}\text{ log}(\lambda )+C\text{ log}(\lambda ).$$

At the M step, we update λ and other parameters (α, β₁, and β₂). Specifically,

$${\lambda }_Z^{(s+1)}=\underset{\lambda }{\text{argmax}}\sum _{j=1}^{n_1}(1-{\omega }_{\mathit{\text{jz}}}^{(s)})\text{ log}(1-\lambda )+\sum _{j=1}^{n_1}{\omega }_{\mathit{\text{jz}}}^{(s)}\text{ log}(\lambda )+C\text{ log}(\lambda )=\frac{{\sum }_{j=1}^{n_1}{\omega }_{\mathit{\text{jz}}}^{(s)}+C}{n_1+C},$$ (4)

$$({\alpha }_Z^{(s+1)},{\beta }_{z1}^{(s+1)},{\beta }_{z2}^{(s+1)})=\underset{\alpha ,{\beta }_1,{\beta }_2}{\text{argmax }}{\mathbf{ℒ}}_{\mathit{\text{pp}}}({\lambda }_Z^{(s+1)},\alpha ,{\beta }_1,{\beta }_2).$$

Equation (4) suggests an intuitive explanation of the penalty term C log(λ) in ℒ_pp(λ, α, β₁, β₂). Specifically, the proportion of the subgroup with differential methylation among cases, ${\lambda }_z^{(s+1)}$, is calculated as the average of posterior probabilities of being in this subgroup, ${\omega }_{jz}^{(s)}$, plus C pseudo observations (known to be in this subgroup). This idea of pseudo-observation adjustment was originally proposed by Jiahua Chen and his colleagues. The pseudolikelihood method that maximizing over all parameters simultaneously faces the non-convergence and computational problems. Our simulation result summarized in the Supplementary Materials shows that the parameter estimates are sensitive to the choice of initial values when maximizing over all parameters simultaneously, leading to unstable results. By fixing λ at this step, we can get more stable results than maximizing over all parameters simultaneously. We use the general-purpose optimization procedure in R (R Development Core Team, 2009) to obtain $({\alpha }_z^{(s+1)},{\beta }_{z1}^{(s+1)},{\beta }_{z2}^{(s+1)})$. At the z^th grid value, we define $M_n({\lambda }_z^{(S)}=2\{{\mathbf{ℒ}}_{\mathit{\text{pp}}}({\lambda }_z^{(S)},{\alpha }_z^{(S)},{\beta }_{z1}^{(S)},{\beta }_{z2}^{(S)})-{\mathbf{ℒ}}_{\mathit{\text{pp}}}(1,0,0,0)\}$. We then define our PLEMT test statistic as follows,

$$\text{PLEMT}=\text{max }\{M_n({\lambda }_Z^{(S)}),z=1,\dots ,Z\}.$$

Details of the derivation of the PLEMT test statistic are provided in Appendix D of online supplementary materials.

3. ASYMPTOTIC RESULTS

This section provides asymptotic results for the PLEMT test statistic under the null H₀ and under the local alternatives.

We assume the following regularity conditions.

(C1). The parameter sets Ω_α and Ω_β for α and β are compact.

(C2). The distributions of u_i and v_j have common support and are not degenerate to a point measure.

(C3). The ratio n₁/n → ρ, as n → ∞, where 0 < ρ < 1. The variance σ² = var(u_i) < ∞ and for some t > M, $\int u^2\text{ exp}(t|u|)f(u)\mathit{\text{du}}<\infty$, where M is a small positive constant. The compactness of the parameter spaces in assumption (C1) is commonly adopted in the statistical literature. Such an assumption may be relaxed by imposing the uniform boundedness assumption on the baseline density; for example, see Li et al., (2009). The assumption (C2) is to guarantee the GETMM is identifiable. The assumption (C3) is a reasonable technical condition for applying the uniform law of large numbers.

Theorem 1

Under the regularity conditions C1–C3, and the null hypothesis H₀, the PLEMT test statistic converges weakly to ${\chi }_2^2$ as n → ∞. The proof of Theorem 1 is given in Appendix D of online supplementary materials.

Evaluation of test statistics under alternative hypotheses is crucial for sample size calculation and experimental design. In practice, the most interesting situations for alternatives are those close to the null hypothesis. For mathematical convenience, statisticians typically focus on the local asymptotic power of test statistics. Here we provide a useful result about the asymptotic power of the PLEMT test under a sequence of local alternatives. Specifically, for any 0 < λ₀ < 1, density function f₀(·) and a fixed τ₀, consider a sequence of alternatives:

$$H_a:\lambda ={\lambda }_0,f(·)=f_0(·),\mathit{\beta }=n^{-1/2}{\mathit{\tau }}_0,$$

where τ₀ is a vector. Under H_a, we do not explicitly specify the local alternative for α, because α is a function of β and f₀(·). With LeCam’s third lemma (Van der Vaart, 2000), we can establish the following results.

Theorem 2

Under the alternatives H_a, the limiting distribution of the PLEMT test statistic is ${\chi }_2^2\{{\lambda }_0^2(1-\rho )\rho {\mathit{\tau }}_0^T\sum {\mathit{\tau }}_0\}$, where ${\chi }_2^2(c)$ denotes the noncentral chi-squared distribution with 2 degrees of freedom and non-centrality parameter c.

The proof is given in Appendix E of online supplementary materials. An important observation is, given a total number of subjects n, the asymptotic power is maximized when the design is balanced (i.e., ρ = 0.5 or n₀ = n₁).

4. SIMULATION STUDIES

We conduct simulation studies to evaluate the finite sample performance of the proposed PLEMT test comparing with seven existing tests, namely the score test based on the one-parameter ETMM, the modified empirical likelihood ratio test, the Wald test based on logistic regression, the t-test, the Wilcoxon test, the F test of equality of variances and the Kolmogorov-Smirnov test.

We consider a variety of parametric models. Specifically, for each simulation setting, the data are generated from the mixture model (2) with one of the following choices of density functions f(·) and g(·).

Model A (Normal model)

Let f(·) and g(·) be the density functions of Normal $({\mu }_1,{\sigma }_1^2)$ and Normal $({\mu }_2,{\sigma }_2^2)$, respectively. Then

$$\text{log}\{g(x)/f(x)\}=\frac{1}{2}(\text{log }{\sigma }_1^2-\text{log }{\sigma }_2^2)+\frac{{\sigma }_2^2{\mu }_1^2-{\sigma }_1^2{\mu }_2^2}{2{\sigma }_1^2{\sigma }_2^2}+\frac{{\sigma }_1^2{\mu }_2-{\sigma }_2^2{\mu }_1}{{\sigma }_1^2{\sigma }_2^2}x+\frac{{\sigma }_2^2-{\sigma }_1^2}{{\sigma }_1^2{\sigma }_2^2}{x}^2.$$

Model B (Beta model)

Let f(·) and g(·) be the density functions of two beta distributions with shape parameters (a₁, b₁) and (a₂, b₂), respectively. Then

$$\text{log}\{g(x)/f(x)\}=\text{log }\left\{\frac{B(a_1,b_1)}{B(a_2,b_2)}\right\}+(a_2-a_1)\text{ log }x+(b_2-b_1)\text{ log}(1-x),$$

where B(·, ·) is the beta function.

Model C (Gamma model)

Let f(·) and g(·) be the density functions of Gamma (m₁, θ₁) and Gamma (m₂, θ₂) with shape parameters m₁, m₂ and scale parameters θ₁, θ₂ > 0. Then

$$\text{log}\frac{g(x)}{f(x)}=\text{log }\left\{\frac{\mathrm{\Gamma }(m_1)}{\mathrm{\Gamma }(m_2)}\right\}+m_1\text{ log}({\theta }_1)-m_2\text{ log}({\theta }_2)+(m_2-m_1)\text{ log }x+(\frac{1}{{\theta }_1}-\frac{1}{{\theta }_2})x.$$

Note that Models A–C belong to the GETMM. In equation (2), Models A–C have k(x) = (x, x²), {log(x), log(1 − x)}, and {log(x), x}, respectively. To evaluate the robustness of the proposed test to model misspecifications, we consider two additional models (Models D–E) when the exponential tilt model assumption is not satisfied. It is expected that misspecified models may lead to incorrect Type I errors for tests based on the GETMM assumption.

Model D (Negative binomial model)

Let f(·) and g(·) be the density functions of the Negative Binomial (NB) distribution (r₁, p₁) and NB (r₂, p₂), where r₁ and r₂ are the numbers of failures until the experiment is stopped, and p₁ and ps are success probabilities in each experiment. Then

$$\text{log}\frac{g(x)}{f(x)}=r_2\text{ log}(\frac{p_2}{1-p_2})-r_1\text{ log}(\frac{p_1}{1-p_1})+x\text{ log}(\frac{1-p_2}{1-p_1})+\text{log }\left(\begin{array}{c}x-1\\ r_2-1\end{array}\right)-\text{log }\left(\begin{array}{c}x-1\\ r_1-1\end{array}\right).$$

Model E (t distribution)

Let f(·) and g(·) be the density functions of the t distributions (ncp₁, d f₁) and (ncp₂, d f₂), where ncp₁ and ncp₂ are the noncentrality parameters, and d f₁ and d f₂ are degrees of freedom.

We compare Type I errors and power of the tests. For power comparisons, we conduct simulation studies under Scenario I where f(·) and g(·) are different in means only, Scenario II where f(·) and g(·) are different in variances only, and Scenario III where both means and variances of f(·) and g(·) are different. For each of the power scenarios, we consider settings where λ takes different values from 0 to 1. The rejection rates based on 5000 simulations are used to estimate Type I errors and the rejection rates based on 1000 simulations are used to estimate the power. We consider a sample size setting with 100 subjects in each group.

Table 1 summarizes the Type I errors of the eight tests under comparison for five models (A–E). Under both non-misspecification scenario (Models A–C) and misspecification scenario (Models D–E), the Type I errors of the PLEMT test, the EST test, the MELRT test, the t-test, the Wilcoxon test and the Logistic regression test are relatively close to the corresponding nominal levels given the moderate sample size (n = 200). The proposed test has slightly inflated type I errors under model misspecifications compared to t-test, Wilcoxon test and logistic regression test. It is not surprising to see that the Type I errors of the F test are inflated under the Gamma, Negative binomial and the t models, because the F test is known to be sensitive to non-normality. The Kolmogorov-Smirnov test yields conservative Type I errors under all settings, especially in the negative binomial model.

Table 1.

Type I error (%) comparisons of the PLEMT test, the score test based on empirical likelihood (EST), the modified empirical likelihood ratio test (MELRT), the t-test, the Wilcoxon test, the Logistic regression test, the F test of equality of variances and the Kolmogorov-Smirnov (KS) test, at 0.05 and 0.1 significant levels for Normal, Beta, Gamma, Negative binomial and t models (Models A–E).

Model	level (%)	PLEMT	EST	MELRT	t-test	Wilcoxon	Logistic	F	KS
Non-misspecified models
A: Normal	5.0	5.2	5.7	5.8	4.9	4.8	4.8	4.8	3.3
	10.0	10.1	10.9	11.3	10.2	9.9	10.0	9.6	7.7
B: Beta	5.0	5.0	5.1	5.1	4.7	4.7	4.3	5.3	3.7
	10.0	10.3	9.6	9.5	9.1	8.9	8.9	10.3	7.5
C: Gamma	5.0	5.3	6.2	6.3	5.5	5.4	5.2	10.7	3.8
	10.0	10.5	11.5	11.7	10.6	10.1	10.2	17.5	8.0
Misspecified models
D: Negative binomial	5.0	5.5	5.7	5.7	5.0	4.8	4.8	7.9	1.5
	10.0	11.8	11.1	11.3	10.4	10.6	10.0	14.6	3.3
E: t	5.0	5.6	6.5	6.3	5.4	4.8	4.8	27.9	3.8
	10.0	11.0	11.2	12.1	9.8	10.1	9.3	36.0	7.9

Since the Type I errors of the F test are inflated, we only compare the power of the remaining seven tests. Figure 1 plots the power curves of the seven tests under GETMMs (Models A–C). As λ increases, the power of all tests increases. For power Scenario I with mean differences only, the PLEMT test is slightly more powerful than the Kolmogorov-Smirnov test, and has slightly lower but comparable power than the rest five tests. This is because the PLEMT test has two degrees of freedom, while the other five tests have only one degree of freedom. The slight loss of power for the PLEMT test is due to the extra one degree freedom when there are only mean differences. For the power Scenario II with variance differences only, the PLEMT test is much more powerful than all the other tests as expected since they do not fully account for variance differences and the mixture structure of the data. More specifically, the MELRT test has about 17% – 46% less power than that of the PLEMT test, the Kolmogorov-Smirnov test has about 40% – 90% less power than that of the PLEMT test, whereas the other four tests have essentially no power beyond Type I errors. This is consistent with the simulation results in Liu et al., (2012) that the MELRT test is more powerful than the EST test. When both mean and variance are different, the PLEMT test remains to be the most powerful one. The degree of the power loss of the other six tests depends on the proportion of the variance difference in the overall mean and variance differences. Under the setting we considered, the MELRT test is the second most powerful test with a power about 5% – 25% lower than that of the PLEMT test. The other four tests are grossly underpowered.

Figure 1.

Power of the PLEMT test, the score test based on empirical likelihood (EST), the modified empirical likelihood ratio test (MELRT), the t-test, the Wilcoxon test, the Logistic regression test, and the Kolmogorov-Smirnov (KS) test for Normal, Beta, and Gamma models as a function of mixture proportion λ when the numbers of observations in two groups are n₀ = n₁ = 100.

Similar patterns are observed in Figure 2 for misspecified models (Models D and E). The PLEMT test has a slightly lower power than the other tests under power Scenario I, while it is the most powerful test under power Scenarios II and III. One interesting phenomena is that there is a power gain for the Wilcoxon test under Scenario II and III for the misspecified models comparing with that under GETMMs, while the Wilcoxon test has essentially no power beyond Type I errors under GETMMs. This is because the Wilcoxon test may capture some of the variance differences when the distributions are heavily skewed, but cannot capture these differences when the distributions are symmetric.

Figure 2.

Power of the PLEMT test, the score test based on empirical likelihood (EST), the modified empirical likelihood ratio test (MELRT), the t-test, the Wilcoxon test, the Logistic regression test, and the Kolmogorov-Smirnov (KS) test for Negative binomial and T models as a function of mixture proportion λ when the numbers of observations in two groups are n₀ = n₁ = 100.

In summary, our simulation studies suggest performance from the proposed PLEMT test compared to the existing ones. The proposed PLEMT test has well controlled Type I errors and substantial power gain when the variance differences need to be taken into account. The proposed PLEMT test also shows some degree of robustness under model misspecifications. The PLEMT test is implemented as an R software package robustETM, which is attached as the Supplementary Materials.

5. APPLICATION TO DNA METHYLATION DATA OF OVARIAN CANCER

We apply the PLEMT test to the data from the United Kingdom Ovarian Cancer Population Study to select differentially methylated sites between ovarian cancer cases and age-matched healthy controls using the Illumina Infinium Human Methylation27 Beadchip (Teschendorff et al., 2010). The original data have 266 ovarian cancer cases with 131 pre-treatment cases and 135 post-treatment cases, and 274 age-matched healthy controls. Since age and having received treatment or not when blood samples are taken are factors known to affect DNA methylation levels, we choose to use the 131 ovarian cancer cases who gave their blood at the time of their diagnosis prior to treatment and with age-matched controls. We refer readers to Wang (2011) for the detailed quality control steps. We end up with 96 cancer subjects and 136 normal subjects with DNA methylation levels at 22951 sites. Because our simulation results suggest that the MELRT test is the second most powerful test accounting for the variance differences, we focus on the comparisons among the PLEMT test, the commonly used t-test, and the MELRT test in this real data application. We also only focus on the original DNA methylation levels instead of the logit transformed ones. Due to the presence of multiple hypothesis testing, the number of false positives may rapidly increase such that the scientific discoveries may become unreliable. To address this problem, we adapt the procedure in Storey (2002) to control the false discovery rate (FDR), which is defined as the number of false positives divided by the number of total discoveries. In particular, for each hypothesis test, we calculate a number called q-value, which is the minimum FDR that can be attained when the test is significant. It can be regarded as a hypothesis testing error measure for each test with respect to FDR (Storey, 2002). Here, we use the “qvalue” package in R to calculate the q-value for each hypothesis test.

The proposed PLEMT test, the t-test and the MELRT test are then applied to the methylation data at these 22951 sites. Of the sites tested, 3112 sites have q-values < 0.05 using the PLEMT test, 2699 sites have q-values < 0.05 using the t-test, and 2881 sites have q-values < 0.05 using the MELRT test. These numbers are cross-tabulated in Table 2. There are 2418 overlapping sites that have q-values < 0.05 using both the PLEMT test and the t-test, and 694 sites that are identified by the PLEMT test but not by the t-test. There are 2543 overlapping sites that have q-values < 0.05 using both the PLEMT test and the MELRT test, and 569 sites that were identified by the PLEMT test but not by the MERLT test.

Table 2.

2 × 2 tables for the number of sites identified by the PLEMT test vs. t-test, and the PLEMT test vs. the MELRT test.

		PLEMT				PLEMT
		q-value < 0.05	≥ 0.05			q-value < 0.05	≥ 0.05
t-test	q-value < 0.05	2418	281	MELRT	< 0.05	2543	338
	q-value ≥ 0.05	694	19558		≥ 0.05	569	19501

We denote Δ = (m₁ − m₂)/sd₁ as the standardized mean difference between cancer and normal subjects, and r₂₁ = sd₂/sd₁ as the ratio of standard deviations between cancer and normal subjects, where m₁ and m₂ are the means of the normal subjects and cancer subjects, respectively, and sd₁ and sd₂ are the standard deviations of the normal subjects and cancer subjects, respectively. We further examine the distribution of Δ and r₂₁ for the sites tested. The upper three panels of Figure 3 displays the distribution of Δ and r₂₁ for the 694 sites that are identified by the PLEMT test but not the t-test, for the 281 sites that are identified by the t-test but not the PLEMT test, and for the 2418 overlapping sites that are identified by both the PLEMT test and the t-test, respectively. The lower three panels of Figure 3 displays the distributions of Δ and r₂₁ for the 569 sites that are identified by the PLEMT test but not the MELRT test, for the 338 sites that are identified by the MELRT test but not the PLEMT test, and for the 2542 sites that are identified by both the PLEMT test and the MELRT test, respectively.

Figure 3.

Upper panels: distributions of the standardized mean difference Δ and the ratio of standard deviations r₂₁ for the 694, 281, 2418 sites that are identified by the PLEMT test but not the t-test, by the t-test but not the PLEMT test, by both the PLEMT test and the t-test, respectively; Lower panels: distributions of Δ and r₂₁ for the 569, 338, 2543 sites that are identified by the PLEMT test but not the MELRT test, by the MELRT test but not the PLEMT test, by both the PLEMT test and the MELRT test. The 9 CpG sites between the clouds in the lower middle panel are highlighted in red.

It is clear that the sites identified by the PLEMT test only but not the t-test have more significant variance differences than mean differences between the cancer and normal groups, which is the scenario the proposed test is designed for. In contrast, those sites that are identified by the t-test only have more significant mean differences than variance differences between the cancer and normal groups in general. For the overlapping 2418 sites that are identified by both the PLEMT test and the t-test, the majority have much larger differences in means than the sites identified by only one method. Thus these sites are relatively easier to be identified as all methods look for mean differences. Moreover, for those sites that have relatively small mean differences (but still have larger mean differences than sites that are identified by the PLEMT test only), they have large differences in variance in general. Thus the PLEMT test is able to identify them, although simulation studies suggested a slightly lower power for the PLEMT test in such scenarios than the t-test. Similar patterns can be found when comparing the PLEMT test and the MELRT test. An interesting phenomenon is that for the 338 sites identified by the MELRT test only but not the PLEMT test (lower middle panel of Figure 3), there are 9 points between the clouds (highlighted in red), where the differences in both means and variances are very small. As shown later, our further examination suggests that those sites identified by the MELRT test may be false positive.

We examine the top 50 sites with most significant results from the PLEMT test among the 694 sites that are identified by the PLEMT test but not the t-test. By estimating the proportion of differentially methylated subjects in the cancer group compared to those in the normal group, we find the mixture feature at 16 out of the 50 sites (i.e., close to one-third) based on the estimated λ. For example, at site cg26457013, the proportion of differentially methylated subjects in the cancer group compared to those in the normal group is 22%, with estimated (β₁, β₂) = (−86.79, −5.52). At site cg11905589, the proportion of differentially methylated subjects in the cancer group compared to those in the normal group is 78%, with estimated (β₁, β₂) = (−10.87, −71.68). At both sites, there are sizable differences in means and variances. Specifically, we have (Δ, r₂₁) = (0.28, 0.76) at site cg26457013, and (Δ, r₂₁) = (0.11, 0.70) at site cg11905589. We further examine the 9 CpG sites between the clouds in the lower middle panel of Figure 3 with small differences in both means and variances that are identified by the MELRT test but not the PLEMT test, where we apply all eight tests investigated in the simulation studies on these 9 sites. The results suggest that some of the identified sites might have been false positive since all other six existing tests generate large p-values (results are included in Section 3 of online supplementary materials).

We compare the predictive power of significant CpG sites detected by PLEMT, MELRT and t-test. The top ranked CpG sites (by statistical significance) detected by the three methods are largely overlapped, because most of them show differences in average methylation levels. To emphasize the distinctions of different methods, we ignore the common ones and pick top 30 CpG sites uniquely identified by these three tests, and then use them as predictors to classify cancer and normal samples. We use Random Forest (Breiman, 2001) as the classification method, and compare the receiver operating characteristics (ROC) curves generated from 100 runs of 3-fold cross validation. As shown in Figure 4, the area under the ROC curve of PLEMT is greater than that of MELRT and t-test, (AUC: 0.76, 0.66 and 0.64 for PLEMT, MELRT and t-test, respectively). This result indicates that compared with the other two methods, PLEMT identifies CpG sites that can better distinguish cancer patients from normal people.

Figure 4.

Receiver operating characteristics (ROC) curves for the PLEMT test, the MELRT test and t-test using Random Forest as the classification method.

We further investigate the 15 genes that the top 15 CpG sites reside among those CpG sites that were uniquely identified by the proposed PLEMT method only but not by the t-test or by the MELRT method. Out of the 15 genes, 9 of them have been reported to be associated with cancer. These include genes which are reported to be related with breast cancer: PMP22 (Winslow et al., 2013), AIM2 (Liu et al., 2015), colorectal cancer: CNGA3 (Shaikh et al., 2015), ovarian cancer HOXB8 (Stavnes et al., 2013), liver cancer HNF4A (Ning et al., 2010), pancreatic cancer PCDHB2 (Carter et al., 2010), and cancer progression STRN4 (Wong et al., 2014), CHCHD4 (Yang et al., 2012), and VHL (Kim and Kaelin, 2004), where the number in the parenthesis is the rank of p-value among the 15 genes.

In summary, the proposed PLEMT test has identified novel sites of potential interest that are missed by the commonly used t-test and the MELRT test, and has better predictive powers. Therefore, it can serve as a useful complement to the standard tests.

6. DISCUSSION

In this paper, we proposed a novel pseudolikelihood based EM test to identify differentially methylated loci. Specifically, we developed a semiparametric model to account for heterogeneity between differentially methylated subjects and non-differentially methylated subjects in the cancer group, and capture the differences in higher order moments (e.g. mean and variance) between subjects in the cancer and normal groups. We constructed a novel penalized pseudolikelihood to eliminate the unknown baseline density function and circumvent the non-regularity problems. We also proposed an EM algorithm based test for computational efficiency and stability, which follows a simple chi-squared limiting distribution. Through simulation studies we demonstrated the feasibility and power of the proposed test. The proposed test outperformed the existing tests especially when there is variance difference between two groups. We have also conducted sensitivity analyses to empirically show that the results are not sensitive to the tuning parameters C, S and Z. Cross-validation procedures can be used here to obtain an optimal choice of tuning parameters. However, these procedures are usually computational expensive. Instead, C can be simply set at 20 for reasonable type I errors and power, as suggested from our sensitivity analyses.

The proposed PLEMT test with pairwise conditioning procedure has the advantage of eliminating the nuisance baseline density function f(·). However, the baseline density function may be assumed known in other scenarios. For example, an accurate estimate of the baseline density function could be obtained when the data of large size of normal subjects are available. In this case, there is no need to use the proposed conditioning procedure to eliminate the nuisance baseline function. Instead, a penalized likelihood ratio test for admixture model can be used to test for homogeneity (Fu et al., 2006b; Di and Liang, 2011). As suggested by a referee, we have conducted simulation studies to compare the performance of the proposed PLEMT test ( f(·) unknown) with the penalized likelihood ratio test for admixture model (f(·) known); see the additional simulation results in the Supplementary Materials. We found that the penalized likelihood ratio test for admixture model has more power compared with the proposed PLEMT test, especially when the proportion parameter is relatively small.

In this paper, the sample sizes in normal and cancer groups are set to be equal. In some cancer data sets, there may be more cancer samples than normal samples, or more normal samples than cancer samples for relatively rare/under-studied cancer. We have conducted additional simulation studies when the two groups are unbalanced. The results for type I errors and power comparisons are summarized in Table 7 in the Supplementary Materials. The balance of two groups has some impacts on type I errors and permutation methods may be needed to better control type I errors if two groups are highly unbalanced.

We compare the type I errors of the proposed PLEMT test using χ² distribution with the type I errors of the permutation-based PLEMT test. The type I errors of the proposed PLEMT test and the permutation-based PLEMT test are similar. Due to the time cost of the permutation-based test, we suggest the use of the proposed PLEMT test with a simple χ² asymptotic distribution when the sample size is sufficiently large.

In this paper, the pre-specified kernel function k(v) = (v, v²) is considered for illustration. The misspecification of the kernel function does not lead to inflated type I errors. However, better choice of the kernel function may yield higher power. We are currently working on incorporating the Box-Cox transformation into the generalized exponential tilt model for more model flexibilities.

The proposed method aims to detect methylation loci that are marginally different between the case and control groups, which plays the same role as the sure independent screening method for high dimensional feature selection (Fan and Lv, 2008). However, this marginal approach may neglect the methylation loci which are jointly associated with cancer development but are marginally uncorrelated. To address this problem in the framework of linear models and generalized linear models, Fan and Lv (2008) and Fan et al., (2009) proposed an iterative sure independent screening method, which iteratively adds a new feature into the current variables and then perform the sure screening step. Different from these existing approaches, our case group may contain misclassification, such that the cancer status cannot be directly modeled by a logistic regression. It is of interest to develop an iterative screening method to handle our methylation data.

For analysis of DNA methylation data in cancer research, age has been considered as a strong demographic risk factor (Christensen et al., 2009). Chen et al., (2013) proposed an age-adjusted nonparametric method to detect differentially methylated loci. Specifically, the rank-based Kruskal-Wallis test was conducted separately in different age groups, then a combined p-value was reported. This method focuses on the differences in medians, and is expected to be underpowered when there are differences in variances. Alternatively, Huang et al., (2013) proposed an age-adjusted nonparametric method to capture the differences in both means and variances, where the Neuhaeuser’s one-sided test (Neuhäuser, 2003) was conducted within each age group and a combined p-value was reported. Permutation was used to obtain the age-specific p-values. However, both methods are not easy to extend to more than one confounders and do not account for the heterogeneity in DNA methylation among cases. The proposed PLEMT test can be extended to regression models where multivariate covariates are simultaneously adjusted and heterogeneity in cases can be accounted for. Such an extension is currently under investigation and will be reported in the future.