KERNEL-PENALIZED REGRESSION FOR ANALYSIS OF MICROBIOME DATA

Abstract

The analysis of human microbiome data is often based on dimension-reduced graphical displays and clusterings derived from vectors of microbial abundances in each sample. Common to these ordination methods is the use of biologically motivated definitions of similarity. Principal coordinate analysis, in particular, is often performed using ecologically defined distances, allowing analyses to incorporate context-dependent, non-Euclidean structure. In this paper, we go beyond dimension-reduced ordination methods and describe a framework of high-dimensional regression models that extends these distance-based methods. In particular, we use kernel-based methods to show how to incorporate a variety of extrinsic information, such as phylogeny, into penalized regression models that estimate taxonspecific associations with a phenotype or clinical outcome. Further, we show how this regression framework can be used to address the compositional nature of multivariate predictors comprised of relative abundances; that is, vectors whose entries sum to a constant. We illustrate this approach with several simulations using data from two recent studies on gut and vaginal microbiomes. We conclude with an application to our own data, where we also incorporate a significance test for the estimated coefficients that represent associations between microbial abundance and a percent fat.

1. Introduction

A common tool in the analysis of data from microbiome studies is a scatterplot of dimension-reduced microbial abundance vectors. This is a display of the samples’ beta diversity which, in ecology, refers to differences among various habitats. When applied to human studies, beta diversity describes the variation in microbial community structure across sampling units (e.g., human subjects): a beta diversity plot displays the n sampling units with respect to the principal coordinates of their microbial abundance vectors, each consisting of measures on the p taxa (bacterial types) observed in the study; see, e.g., Claesson et al. (2012); Koren et al. (2013); Kuczynski et al. (2010); Goodrich et al. (2014). This principal coordinates analysis (PCoA; or multidimensional scaling, MDS) begins with an n×n matrix of pairwise dissimilarities between vectors of taxon abundances. The choice of dissimilarity measure may greatly influence the biological interpretation (Lozupone et al., 2007; Fukuyama et al., 2012). Euclidean distance is rarely used.

A common assay of microbial content is based on counting sequences observed from the 16S rRNA gene, a marker used to identify bacterial species or other taxonomic categories. We will generically refer to “taxa” rather than specifying a category, such as genus or species. A single taxon may be placed within the context of a phylogenetic tree in order to provide evolutionary relationships among taxa. Dissimilarity measures that account for these phylogenetic relationships are assumed to enhance statistical analyses — for instance, to improve the power of statistical tests — because they incorporate the degree of divergence between sequences (Chen et al., 2012) and do not ignore “the correlation between evolutionary and ecological similarity” (Hamady and Knight, 2009). The UniFrac distance (Lozupone and Knight, 2005), in particular, is based on the premise that taxa which share a large fraction of the phylogenetic tree should be viewed as more similar than those sharing a small fraction of the tree. In the unweighted version of UniFrac, each taxon is quantified merely by its presence or absence; the distance between a pair of samples is based on the number of branches in the tree shared by both. Figure 1(a) is a beta diversity plot of n = 100 human microbial abundance vectors with p = 149 taxa based on data from Yatsunenko et al. (2012). Each sample is represented by 2-dimensional coordinates with respect to the unweighted UniFrac distance, and the size of each point is proportional to log(age) of the subject.

Fig 1. PCoA plots of data from Yatsunenko et al. (2012).

(a): PCoA plot with respect to unweighted UniFrac distance where dot size is proportional to log(age) of the subject. (b): PCoA plot with respect to unweighted UniFrac distance, dot size is proportional to y_True from the model in Eq. (3.2) with ε = 0.

Dissimilarity measures in microbiome studies are many and varied, with a rich collection that, like UniFrac, exploit the phylogentic structure: Chen et al. (2012) generalize UniFrac by reweighting rare and abundant lineages; double principal coordinate analysis (DPCoA) (Pavoine, Dufour and Chessel, 2004), as shown by Purdom (2011), generalizes PCA by incorporating the covariance that would arise if the data was created by a process modeled by the tree; the edge PCA method of Matsen and Evans (2013) incorporates taxon abundance information at all nodes in a phylogenetic tree, rather than just the leaves of the tree, and Evans and Matsen (2012) formalize the mathematical interpretation of UniFrac as just one example within a large family of Wasserstein (or earth mover’s) metrics. A wide variety of nonphylogenetic dissimilarities are also in common use, such as Bray-Curtis (The Human Microbiome Project Consortium, 2012) and Jenson-Shannon (Koren et al., 2013), among others.

While PCoA plots provide valuable graphical insight into the relationships among microbial profiles and an outcome or phenotype, they do not quantify this association. More importantly, the (sets of) taxa associated with the outcome — and the magnitude or statistical significance of such associations — are not ascertained from a PCoA plot; once a matrix of (dis)similarities between samples is formed, it is not clear how to identify individual taxa that are associated with an outcome. Specifically, given a PCoA plot as in Figure 1(a), with structure imposed by the chosen dissimilarity matrix (e.g., unweighted UniFrac) and with associations implied by a class label or continuous outcome (e.g., age), how does one estimate which taxa or subcommunities are associated with this outcome? We address this question by formulating multivariate regression models that are constrained by the structure of the (dis)similarity matrix. This is made possible by exploiting an equivalence between a taxon-based (primal space) and sample-based (dual space) formulation of our penalized regression models. While exploiting such an equivalence is straightforward in the special case of ridge regression (with purely Euclidean structure), it becomes complicated when more general distance measures are used. To this end, we show how a little-used regularization scheme by Franklin (1978) provides a dual-space regression coefficient estimate that naturally connects to primal-space coefficients. Because a dissimilarity matrix can be used to construct a similarity matrix (as commonly done in classical MDS (Mardia, Kent and Bibby, 1980)), we work with kernels, rather than distances, and allow for general kernels, including those constructed from a nonlinear feature map.

In addition to complications stemming from more general distances, the analysis of microbiome data is also complicated by the compositional nature of the data itself. More specifically, taxon measures typically represent relative, rather than absolute, abundances. The p-variate relative abundance vectors are thus compositional in that they are constrained to a simplex within ℝ^p; such data do not reside in a Euclidean vector space (Aitchison, 2003a). Consequently, spurious correlations arise and standard multiple regression models fail. Our proposed KPR framework, however, addresses this: the centered log (CLR) transform of the relative abundance vectors first removes the vectors from the simplex, then the estimation process is constrained using a penalization term defined by Aitchison’s variation matrix. This approach takes a different perspective from the recent proposal of Li (2015) which forces the estimated coefficient vector to reside in the simplex. Given that the CLR transforms the compositional vectors to Euclidean space and that the units of the Aitchison variation matrix are the same as the CLR transformed data (Egozcue and Pawlowsky-Glahn, 2011), our constraint seems more suitable for the geometry of the problem.

In summary, we describe a family of high-dimensional regression problems in Section 2, which are designed to incorporate the assumptions that are tacitly implied by various exploratory and graphically-focused PCoA plots common in microbiome studies. We show how phylogenetic and other structure can be incorporated via kernel penalized regression in either the primal (p-dimensional) feature space or the dual (n-dimensional) samples space; see Sections 2.2 and 2.3. Finally, our proposed framework leads to an approach, described in Section 2.4, for addressing well-known problems that arise from applying standard (Euclidean-based) statistical models to compositional data. Section 3 illustrates the proposed framework with simulations based on publicly available data, while Section 4 presents an application to our recent microbiome study of premenopausal women. In this analysis, we obtain estimates of associations between microbial species and percent fat measured in premenopausal women, and also provide inference for these estimates by applying a recent significance test (Zhao and Shojaie, 2016) in our kernel-penalized regression (KPR) framework.

2. Kernel Penalized Regression for Microbiome Data

We describe a family of multiple regression problems aimed at incorporating assumptions that are implicit in principal coordinate analysis (PCoA) plots common in microbiome studies. We begin in Section 2.1 by establishing notation and concepts from existing dimension-reduction (ordination) methods with the goal of extending them to non-truncated (penalized) regression models. Section 2.2 extends PCoA and principal component regression (PCR) to penalized regression models in the primal space in a manner that incorporates structures implicit in recent microbiome analyses. Section 2.3 extends kernel ridge regression to general (non-L²) structure and the use of two kernels. This extension exploits a dual-space regularization scheme of Franklin (Franklin, 1978). Section 2.4 describes how our proposed framework can be applied to formulate a penalized regression model that accounts for the compositional nature of relative abundance data.

We denote by y_i, i = 1, .., n, a real-valued quantified trait and by x_i = [x_i1, …, x_ip]′ a p-dimensional vector of microbial abundance values measured for each of n subjects. Denote by X the n × p sample-by-taxon matrix whose ith row is $x_i^{\prime }$. We assume throughout that the columns of X are mean centered. For now, we assume that the abundance values are appropriately normalized/transformed and postpone the treatment of compositional data to Section 2.4. The transpose of a matrix A is denoted by A′ and the Frobenius norm is denoted as ||A||_F. I ≡ I_p denotes the identity matrix on ℝ^p and the Euclidean norm of a vector x ∈ ℝ^p is denoted ||x||_ℝ^p or simply ||x||.

2.1. Background for PCoA and principal component regression

Consider first the Euclidean PCoA, which is obtained from the eigenvectors of the kernel matrix K_I := XX′ of inner products K_ij = 〈x_i, x_j〉 between samples. Let 𝒥 be the centering matrix, $\mathcal{J}=I-\frac{1}{n}{\mathbf{11}}^{\prime }$, where 1 is the n×1 vector of ones. Then, it can be seen that $X{X}^{\prime }=-\frac{1}{2}\mathcal{J}{\mathrm{\Delta }}^E\mathcal{J}$, where Δ^E is the n×n matrix of squared Euclidean distances between samples: ${\mathrm{\Delta }}_{i,j}^E={\Vert x_i-x_j\Vert }_{{ℝ}^p}^2$. The relationship between a kernel and a distance matrix Δ is more general. In particular, if Δ is any n × n symmetric matrix of squared dissimilarities between vectors in ℝ^p then $H=-\frac{1}{2}\mathcal{J}\mathrm{\Delta }\mathcal{J}$ serves as a kernel matrix summarizing similarities; see, e.g., Gower (1966); Pekalska, Paclik and Duin (2002). A particular case involves a p×p symmetric, positive definite matrix Q that defines an inner product ${\langle x_i,x_j\rangle }_Q=x_i^{\prime }Q{x}_j$ on ℝ^p. If Δ^Q denotes the matrix of squared distances, ${\mathrm{\Delta }}_{i,j}^Q={\Vert x_i-x_j\Vert }_Q^2={\langle x_i-x_j,x_i-x_j\rangle }_Q$, defined with respect to this inner product, then $XQ{X}^{\prime }=-\frac{1}{2}\mathcal{J}{\mathrm{\Delta }}^Q\mathcal{J}$ is also a similarity kernel for the n samples. We will denote this kernel by K_Q = XQX′. Similarly, one may start with a matrix Δ^U of squared distances defined by a tree-based UniFrac dissimilarity (Lozupone and Knight, 2005), and define a similarity kernel by $H=-\frac{1}{2}\mathcal{J}{\mathrm{\Delta }}^U\mathcal{J}$.

In graphical displays, two or three coordinates are typically used to explore the relationship between samples. Let K = US²U′ be the eigendecomposition of any similarity kernel, K, where U is the matrix whose columns are eigenvectors and $S^2=\text{diag}\{{\sigma }_j^2\}$ is the diagonal matrix of eigenvalues. The two-dimensional PCoA plot is then the collection of points ${\{{\eta }_{i1},{\eta }_{i2}\}}_{i=1}^n:={\{({\sigma }_1{U}_{i1},{\sigma }_2{U}_{i2})\}}_{i=1}^n$; i.e., a plot of the points represented by the first two columns of the matrix US. These points are often colored according to a grouping label or continuous values, ${\{y_i\}}_{i=1}^n$, to graphically explore the existence of an association between the outcome y and the sample profiles summarized by the first few columns of US. So, a PCoA plot may be viewed as a graphical depiction of a two-component regression model of association:

$$y_i={\gamma }_1{\eta }_{i1}+{\gamma }_2{\eta }_{i2}+\epsilon ,i=1,\dots ,n,$$ (2.1)

where η₁ and η₂ are the first two PCoA axes. Ordinary principal component regression corresponds to the case that η₁ and η₂ come from the Euclidean kernel K_I = XX′. On the other hand, the configuration of points in Figure 1(b) correspond to the first two eigenvectors of the kernel defined by an unweighted UniFrac distance matrix Δ^U, and the size of individual points correspond to the values of y from eq. (2.1) with ε = 0.

Let A_(k) denote the first k columns of a matrix A, or its first k rows and columns if A is diagonal. Then, using the singular value decomposition (SVD) X = USV′, if we express the dimension-reduced approximation of X as $\stackrel{⌣}{X}:=U_{(2)}{S}_{(2)}{V}_{(2)}^{\prime }$, then eq. (2.1) can be written as

$$\begin{gathered} y={\gamma }_1{\eta }_1+{\gamma }_2{\eta }_2+\epsilon \\ =U_{(2)}{S}_{(2)}\gamma +\epsilon \\ =\stackrel{⌣}{X}{V}_{(2)}\gamma +\epsilon , \end{gathered}$$ (2.2)

where γ = [γ₁ γ₂]′. Here, X̆V ₍₂₎ = U₍₂₎S₍₂₎ and Range(X̆′) = Range(V₍₂₎). Consequently, the model y = X̆V₍₂₎ γ + ε can be written as y = X̆β + ε, where β is some vector of the form β = X̆′γ. So inherent in a Euclidean PCoA plot is an implicit coefficient vector, β, which models a linear association between y and X̆. Using the SVD of X in (2.2), the PCR estimate of β ∈ ℝ^p is expressed as

$${\widehat{\beta }}_{\text{PCR}}={({\stackrel{⌣}{X}}^{\prime }\stackrel{⌣}{X})}^{†}{\stackrel{⌣}{X}}^{\prime }y=V_{(2)}{S}_{(2)}^{-1}{U}_{(2)}^{\prime }y=\sum _{k=1}^2\frac{1}{{\sigma }_k}{u}_k^{\prime }y{v}_k,$$ (2.3)

where † denotes the Moore-Penrose inverse.

2.2. Penalized regression and DPCoA

An alternative to a Euclidean PCR is the ordinary ridge regression (Hoerl and Kennard, 1970),

$${\widehat{\beta }}_{\text{ridge}}={(X^{\prime }X+\lambda I)}^{-1}{X}^{\prime }y=\sum _{k=1}^n\left(\frac{{\sigma }_k^2}{{\sigma }_k^2+{\lambda }^2}\right)\frac{1}{{\sigma }_k}{u}_k^{\prime }y{v}_k,$$ (2.4)

in which the terms are re-weighted, instead of being truncated as in β̂_PCR. The estimate in (2.4) is the solution of the penalized least squares regression problem, β̂_ridge = argmin_β{||y − Xβ||² + λ||β||²}, where here and throughtout λ is a tuning parameter that controls the amount of shrinkage or size of β in the penalty term. Here the penalty is simply the Euclidean (or ℓ²) norm on ℝ^p, but a wide range of penalty terms have been proposed to replace or extend this particular form of regularization; see Bühlmann, Kalisch and Meier (2014) for a review of the most established methods. These methods, such as the lasso, elastic net or SCAD do not incorporate any extrinsic information, but a variety of other penalization methods have been proposed which aim to do this. For instance, Tanaseichuk, Borneman and Jiang (2014) uses a tree-guided penalty (Kim and Xing, 2010) to incorporate such structure into a penalized logistic regression framework to encourage similar coefficients among taxa according to their relationships in the phylogenetic tree. Tibshirani and Taylor (2011) study the solution path for computing a “generalized lasso” estimate in which an ℓ² penalty is replaced with an ℓ¹ penalty applied to a linear transformation of the features, λ||Lβ||₁. Within the context of genetic networks, Li and Li (2008) accounted for network structure by augmenting the ℓ¹ penalty with a second penalty of the form ${\lambda }_2{\Vert \beta \Vert }_{\mathcal{L}}^2={\beta }^{\prime }\mathcal{L}\beta$, where ℒ denotes the graph Laplacian matrix corresponding to pre-defined connections between genes in a pathway.

For now, we consider a positive definite p×p matrix Q with a Cholesky decomposition Q = LL′, and a penalty term of the form ${\Vert L^{-1}\beta \Vert }^2={\Vert \beta \Vert }_{Q^{-1}}^2={\beta }^{\prime }{Q}^{-1}\beta$. The generalized ridge (or Tikhonov regularization (Golub and van Loan, 2012)) estimate with respect to Q is then defined as

$$\begin{gathered} {\widehat{\beta }}_Q=\underset{\beta }{argmin}\{{\Vert y-X\beta \Vert }^2+\lambda {\Vert \beta \Vert }_{Q^{-1}}^2\}={(X^{\prime }X+\lambda Q^{-1})}^{-1}{X}^{\prime }y \\ =\sum _{k=1}^n\left(\frac{{\sigma }_k^2}{{\sigma }_k^2+\lambda {\mu }_k^2}\right)\frac{1}{{\sigma }_k}{u}_k^{\prime }y{v}_k. \end{gathered}$$ (2.5)

This estimate takes the same form as (2.4) but now the vectors u_k and v_k arise from the SVD of XL = USV′. Regarding the last equality, note that if A denotes any matrix with p columns, the structure of an estimate β̂_A from a penalty term of the form ${\Vert A\beta \Vert }_2^2$ is determined by the joint eigenstructure of the pair (X, A) via the generalized singular value decomposition.¹ In particular, the basis expansion of β̂_Q in (2.5) is given in terms of the generalized singular vectors of (X, L⁻¹). Although the ridge estimate (with Q = I_p) is biased, an informed choice of penalty term can reduce the bias (Randolph, Harezlak and Feng, 2012).

Now consider the context of phylogentic information and let δ represent the matrix of squared patristic distances between pairs of taxa — i.e., the sum of branch lengths between each pair of taxa on the leaves of a phylogenetic tree. Set $Q=-\frac{1}{2}\mathcal{J}\delta \mathcal{J}$, a matrix of similarities between taxa. Double principal coordinate analysis (DPCoA) was proposed by Pavoine, Dufour and Chessel (2004) to provide an alternative to ordinary PCoA that incorporates structure among samples as well as structure implied by the taxa’s distribution among subcommunities, as summarized by Q. Purdom (2011) clarified the original multi-step DPCoA procedure and showed how it can be more simply understood as a generalized PCA (gPCA) in which one obtains the new coordinates from the eigenvectors of K_Q = XQX′. Note that when Q = I_p, DPCoA reduces to PCA/MDS. As emphasized in Purdom (2011), the use of a non-identity Q matrix incorporates structure from known relationships between the p taxa by exploiting a matrix representation of phylogenetic relationships, thus providing a model for covariance structure.

If we let Q = LL′ be a Cholesky decomposition of Q and set Z := XL, then the kernel K_Q = XQX′ has an eigendecomposition of the form US²U′ with respect to the SVD of Z = USV′. This leads to a two-dimensional regression estimate that takes the same the form as β̂_PCR in (2.3). Indeed, we can recover a primal space estimate in terms of singular vectors as

$${\widehat{\beta }}_{\text{DPCR}}:=V_{(2)}{S}_{(2)}^{-1}{U}_{(2)}^{\prime }y=\sum _{k=1}^2\frac{1}{{\sigma }_k}{u}_k^{\prime }y{v}_k.$$ (2.6)

That is, implicit in a DPCoA plot is a coefficient vector β̂_DPCR which models a two-dimensional linear association between y and Z = XL in the same way that β̂_PCR represents a two-dimensional linear association between y and X. Further, XQX′ = (XL)(XL)′ and so U, S and V in β̂_DPCR of (2.6) are the same as those in the penalized (non-truncated) estimate, β̂_Q, in (2.5). When Q = I, these two estimates reduce to β̂_PCR and β̂_ridge, respectively.

2.3. Kernel-based regression with two kernels

In addition to similarities among taxa, as in Q, it is often of interest to incorporate similarities among samples as derived, for instance, from UniFrac distances: $H=-\frac{1}{2}\mathcal{J}{\mathrm{\Delta }}^U\mathcal{J}$. The symmetric positive definite n×n kernel H defines a new inner product on ℝⁿ given by 〈u, w〉_H = u′Hw, with the corresponding norm ${\Vert u\Vert }_H^2={\langle u,u\rangle }_H$. If we consider both a general kernel, H, and a DPCoA kernel K_Q = XQX′, the generalized ridge estimate β̂_Q in (2.5) can be extended to

$$\begin{gathered} {\widehat{\beta }}_{Q,H}:=\underset{\beta \in {ℝ}^p}{argmin}\left\{{\Vert y-X\beta \Vert }_H^2+\lambda {\Vert \beta \Vert }_{Q^{-1}}^2\right\} \\ ={(X^{\prime }HX+\lambda Q^{-1})}^{-1}{X}^{\prime }Hy. \end{gathered}$$ (2.7)

In this section, we show that the estimate in (2.7) is directly defined based on the generalized eigenvectors of the two kernels K_Q and H. Before proceeding to the general case, let us examine the special case of ridge regression for which H = I_n and Q = I_p. A ridge estimate can be obtained by solving an equivalent optimization problem in the dual space ℝⁿ, known as kernel ridge regression (Schölkopf and Smola, 2002). Specifically, taking K_I = XX′, the ridge estimate in (2.4) can be obtained as β̂_ridge = X′γ̂_{kernel ridge}, where

$$\begin{gathered} {\widehat{\gamma }}_{\text{kernel}\text{ridge}}={(K_I+\lambda I)}^{-1}y={(K_I^2+\lambda K_I)}^{-1}{K}_{I}y \\ =\underset{\gamma \in {ℝ}^n}{argmin}\{{\Vert y-K_I\gamma \Vert }^2+\lambda {\Vert \gamma \Vert }_{K_I}^2\}. \end{gathered}$$ (2.8)

In the case of ridge, the connection between the dual- and primal-space estimates, γ̂_{kernel ridge} and β̂_ridge, relies on the form K_I = XX′. Unfortunately, it is less clear how to extend this connection to a general kernel (e.g., UniFrac or polynomial). One way to incorporate a more general kernel K and a second kernel H in (2.8) is to define the penalty in terms of H as

$${\widehat{\gamma }}_{\ast }={(K^2+\lambda H^{-1})}^{-1}Ky=\underset{\gamma }{argmin}\{{\Vert y-K\gamma \Vert }^2+\lambda {\Vert \gamma \Vert }_{H^{-1}}^2\},$$ (2.9)

which is exactly Tikhonov regularization, but in the dual space; compare eq. (2.5). However, γ̂_* ∈ ℝⁿ has no obvious connection to a penalized estimate of β ∈ ℝ^p and cannot be used to obtain a penalized regression estimate in the primal space, even if K = K_I = XX′.

To bridge this gap, we instead apply the Franklin regularization scheme (Franklin, 1978), a little-used alternative to Tikhonov regularization. More specifically, for any kernels K and H, we define the dual estimate

$${\widehat{\gamma }}_{H,K}:={(K+\lambda H^{-1})}^{-1}y=\underset{\gamma \in {ℝ}^n}{argmin}\{{\Vert y-K\gamma \Vert }_{K^{-1}}^2+\lambda {\Vert \gamma \Vert }_{H^{-1}}^2\},$$ (2.10)

where the justification for the second equality is given in the supplementary material (Randolph et al., 2017).

Comparing (2.9) and (2.10), one sees that the analytic form of (2.10) involves just K rather than K² = K′K. As shown in Proposition 2.2, this subtle difference is a key for relating a dual-space estimate γ̂_H,KQ and its primal-space counterpart, β̂_Q,H = QX′γ̂_H,KQ. Before presenting the main result of this section, we provide several equivalent forms of γ̂_H,K.

$${\widehat{\gamma }}_{H,K}={(K+\lambda H^{-1})}^{-1}y=\underset{\gamma \in {ℝ}^n}{argmin}\{{\Vert y-K\gamma \Vert }_{K^{-1}}^2+\lambda {\Vert \gamma \Vert }_{H^{-1}}^2\}=\underset{\gamma \in {ℝ}^n}{argmin}\{{\Vert y-K\gamma \Vert }_H^2+\lambda {\Vert \gamma \Vert }_K^2\}={(HK+\lambda I)}^{-1}Hy=H{(KH+\lambda I)}^{-1}y.$$ (2.11)

In Proposition 2.2, we also refer to the special case corresponding to the DPCoA ordination. As before, let Z = XL so that K_Q = XQX′ = XLL′X′ = ZZ′. Taking H = I, the dual-space estimate in (2.10) is γ̂_I,KQ = (K_Q + λI)⁻¹y, and so the corresponding primal-space estimate is β̂ ≡ Z′γ̂_I,KQ. Since this estimate arises from the DPCoA kernel, we make the following definition.

Definition 2.1

A primal space DPCoA estimate is of the form β̂_DPCoA = Z′γ̂_I,KQ = L′X′(XQX′ + λI)⁻¹y.

The next proposition collects several properties that emphasize the roles of H and K in our penalized regression framework. In particular, we show that the primal space estimate β̂_Q,H can be recovered in terms of two kernels, H and K_Q.

Proposition 2.2

Let H and K be any two kernels constructed using the rows of X in the regression model y = Xβ + ε. Then,

1. γ̂_H,K is a linear combination of the eigenvectors of the matrix product HK.

2. For any kernel H and DPCoA kernel K_Q = XQX′, then the primal- and dual-space estimates in (2.7) and (2.10), respectively, are related as: β̂_Q,H = QX′γ̂_H,KQ.

3. For H = I and Q = LL′, the generalized ridge and DPCoA estimates are related as β̂_Q = QX′(K_Q + λI_n)⁻¹y = Lβ̂_DPCoA.

The proof, given in the supplementary material (Randolph et al., 2017), makes use of some linear algebraic identities which show, in particular, that

$${\widehat{\beta }}_{Q,H}=Q{X}^{\prime }{(XQ{X}^{\prime }+\lambda H^{-1})}^{-1}y=Q{X}^{\prime }{\widehat{\gamma }}_{H,K_Q}.$$ (2.12)

Remarks

1. Types of similarity kernels. In general, a sufficient condition for a matrix K to be a similarity kernel is that it is induced by a feature map ϕ: ℝ^p → 𝒦. More specifically, the i, j entry of K is defined as the inner product of the observations x_i ∈ ℝ^p with respect to their transformed versions K_ij = 〈ϕ(x_i), ϕ(x_j)〉 in the new inner product space, (𝒦, 〈·, ·〉). Examples include K_I = XX′ or K_Q = XQX′, where 𝒦 is ℝ^p with inner product 〈·, ·〉_Q (as in DPCoA). It is this quadratic form that we require for K_Q in Proposition 2.2(2)–(3); see Freytag et al. (2014) for genomic applications of this form. On the other hand, H can be any symmetric positive semi-definite matrix. Here, we are more interested in biologically-motivated kernels, such as UniFrac or DPCoA, than mathematically-derived ones, such as those constructed from polynomials or radial basis functions (Schölkopf and Smola, 2002).

2. Co-informative kernels and the HSIC. Any kernels K and H may be used in (2.10) and (2.11), but to be useful in this framework, we assume that they are “co-informative” in the sense that they exhibit a shared eigenstructure; for instance, both should be informative for classifying samples. This concept is illustrated in the simulation of Section 3.3 and Figure 4. The co-informativeness can be made precise using the Hilbert-Schmidt information criteria (HSIC) (Gretton et al., 2005) or its relatives—the distance covariance (Székely and Rizzo, 2009) and the RV statistic (Robert and Escoufier, 1976). Josse and Holmes (2016) provide a nice review of these and related kernel-based tests. The HSIC provides a test for the statistical dependence of two data sets, X₁ (n×p) and X₂ (n×q), and is based on the eigen-spectrum of covariance operators defined by kernels created from X₁ and X₂. For two kernels K and H, the empirical HSIC is simply trace(HK). The HSIC is thus of particular interest in item (1) of Proposition 2.2, which shows how two co-informative kernels may be used to obtain a penalized estimate β̂_Q,H.

3. Linear mixed models and KPR. As an alternative to the regularization framework presented here, one may consider a kernel as a generalized covariance among either the p variables (using Q) or n subjects (using H) (Purdom, 2011; Schaid, 2010). This alternative representation can be made precise using the linear mixed model (LMM) framework (Ruppert, Wand and Carroll, 2003). Specifically, recall from equations (2.7) and (2.11) that

   $$\begin{gathered} {\widehat{\beta }}_{Q,H}=\underset{\beta \in {ℝ}^p}{argmin}\{{\Vert y-X\beta \Vert }_H^2+\lambda {\Vert \beta \Vert }_{Q^{-1}}^2\} \\ =Q{X}^{\prime }{(K_Q+\lambda H^{-1})}^{-1}y \\ =Q{X}^{\prime }\underset{\gamma \in {ℝ}^n}{argmin}\{{\Vert y-K_Q\gamma \Vert }_H^2+\lambda {\Vert \gamma \Vert }_{K_Q}^2\}=Q{X}^{\prime }{\widehat{\gamma }}_{H,K_Q}. \end{gathered}$$

   These regression estimates are compatible with $\beta ~N(0,{\sigma }_b^{2}Q),\epsilon ~N(0,{\sigma }_e^2{H}^{-1})$ and var(y) = (τK_Q + λH⁻¹)⁻¹. And the estimate γ̂H,K_Q is compatible with $\gamma ~N(0,{\sigma }_a^2{K}_Q^{-1})$ and $\epsilon ~N(0,{\sigma }_e^2{H}^{-1})$. With regard to the latter, a genetic similarity between subjects (e.g., kinship) is often used for grouping subjects and several authors have proposed this form of kernel for testing the (global) genetic association with a trait or phenotype, y; see, e.g., Schifano et al. (2012). In particular, these methods use the LMM framework to motivate and define a “kernel association test”. The variance score statistic for testing the null hypothesis of no association between y and X (H₀ : β = 0) is, using our notation above, $\mathcal{T}:={\Vert y\Vert }_{H^{1/2}{K}_Q{H}^{1/2}}^2$. The kernel association testing framework has been applied to microbiome data using a single kernel at a time derived from UniFrac (Zhao et al., 2015), but this is a test for whether β ≠ 0 and, unlike our KPR framework, provides no insight about which taxa, as represented by coordinates of β, are associated with y.

Fig 4. Analysis of bacterial vaginosis data from Srinivasan et al. (2012).

(a): representation of the samples in the space of the first two PC’s of the edge-matrix kernel H = EE′. The color corresponds to the pH of the sample; (b): heatmap of edge-matrix kernel used to generate the plot in (a); (c): two-dimensional PCA plot based on the genus-level relative abundances, colored according to pH; (d): heatmap of the genus-abundance kernel K = XX′ used to create the plot in (c). In (b) and (d), subjects are ordered by the pH values.

2.4. Regression with compositional data

Data from 16S rRNA gene sequencing methods are random counts of the molecules in each sample. The number of sequence reads assigned to a taxon contains no information about the actual number of molecules in the sample; the total number of reads observed in two samples can vary by several orders of magnitude. Hence, only relative amounts can be investigated. Common approaches for normalizing these data include converting them to proportions (relative percent) or subsampling the sequences to create equal library sizes for each sample (rarefying). These data are “compositional” in the sense that the microbial abundances represent a proportion of a constant total. It is known, however, that compositional measures can result in spurious correlations among taxa (Pearson, 1896; Aitchison, 2003a; Friedman and Alm, 2012), an effect that can be quite extreme when there are a few dominant taxa.

Compositional data reside on the simplex 𝕊^p−1 of unit-sum vectors in ℝ^p and so standard multivariate methods do not apply (Aitchison, 2003b; Egozcue and Pawlowsky-Glahn, 2011; Li, 2015; Lovell et al., 2015). In particular, because these data do not naturally reside in a Euclidean vector space, standard regression models based on Euclidean covariance measures are inappropriate. However, ordinary least-squares and ridge regression estimates are of the form β̂ = (X′X + λI)⁻¹X′y (with λ = 0 and λ > 0, respectively). Thus, these estimates depend on the empirical covariance structure, X′X, among taxa, which may include spurious correlations. Similarly, Li (2015) points out that a naïve application of lasso regression is not expected to perform well due to the compositional nature of the covariates. He addresses this issue by applying a lasso regression model to the log-ratio abundances and imposing an additional constant-sum constraint on the coefficient vector, β.

We next show that the generality of KPR for handling non-Euclidean structures can be used to address the compositional nature of microbiome data. In particular, we propose an approach that uses the centered log-ratio transformation of the compositional vectors and an estimate of covariance among the log taxa counts that is obtained via Aitchison’s variance matrix (Aitchison, 2003b; Egozcue and Pawlowsky-Glahn, 2011).

Let X be the n×p sample-by-taxon matrix whose rows are relative percent (compositional) vectors ${\{x_i\}}_{i=1}^n\subset {\mathbb{S}}^{p-1}$. The columns of X will be denoted by x^k, corresponding to k = 1, ..., p taxa. Let $g(z)={({\mathrm{\Pi }}_{k=1}^p{z}^k)}^{1/p}$ be the geometric mean of a row vector, z, and denote the centered log-ratio (CLR) transform of x_i by ${\stackrel{\sim }{x}}_i=\text{clr}(x_i):=\left[log\frac{x_i^1}{g(x_i)},\dots ,log\frac{x_i^p}{g(x_i)}\right]$. In what follows we denote the matrix of CLR vectors by X̃, and use the normalized variation matrix T, of X, as defined by Aitchison (1982): $T_{k,\ell }=var\left(\frac{1}{\sqrt{2}}log\frac{x^k}{x^{\ell }}\right)$. T is a symmetric dissimilarity matrix with zeros on the diagonal and entries that have squared Aitchison distance units: the Aitchison norm of a vector x ∈ 𝕊^p−1 is defined as ${\Vert x\Vert }_a^2=\frac{1}{2p}{\sum }_{k,\ell }{\left(log\frac{x^k}{x^{\ell }}\right)}^2$. In fact, ${\Vert x\Vert }_a^2={\Vert \text{clr}(x)\Vert }^2$. One can show that T is related to the covariance matrix, C, of the log of the true unobserved taxa counts via T = v1′+1v′−2C (Li, 2015). Consequently, $C=-\frac{1}{2}{\mathcal{J}}^T\mathcal{J}$, and we can use C in place of Q in eq. (2.5) to obtain

$${\stackrel{\sim }{\beta }}_C=\underset{\beta }{argmin}\left\{{\Vert y-\stackrel{\sim }{X}\beta \Vert }_{{ℝ}^n}^2+\lambda {\Vert \beta \Vert }_{C^{-1}}^2\right\}.$$ (2.13)

As a comparison, we observe that Li (2015) proposed a constrained regression

$$\mathrm{E}(y_i)={\beta }_{1}log{x}_i^1+\cdots +{\beta }_{p}log{x}_i^p\text{subject}\text{to}\sum _{j=1}^p{\beta }_j=0,$$ (2.14)

augmented with a lasso penalty to obtain an estimate of the form

$$\underset{\beta }{argmin}\left\{\frac{1}{2n}{\Vert y-\sum _{j}log(x^j){\beta }_j\Vert }_{{ℝ}^n}^2+\lambda \sum _j\mid {\beta }_j\mid \right\}\text{subject}\text{to}\sum _{j=1}^p{\beta }_j=0.$$

The zero-sum constraint on β was emphasized for interpretability advantages over the standard lasso estimate. Temporarily denoting ${\beta }_p=-{\sum }_{j=1}^{p-1}{\beta }_j$, we see that (2.14) is equivalent to

$$\begin{gathered} \mathrm{E}(y_i)={\beta }_{1}log\frac{x_i^1}{x_i^p}+{\beta }_{2}log\frac{x_i^2}{x_i^p}+\cdots +{\beta }_{p-1}log\frac{x_i^{p-1}}{x_i^p} \\ ={\beta }_{1}log{x}_i^1+{\beta }_{2}log{x}_i^2+\cdots +{\beta }_{p-1}log{x}_i^{p-1}-\sum _{j=1}^{p-1}{\beta }_j·log{x}_i^p. \end{gathered}$$

Since ${\sum }_{j=1}^p{\beta }_j=0$, this can be rewritten as

$$\begin{gathered} \mathrm{E}(y_i)={\beta }_{1}log{x}_i^1+\cdots +{\beta }_{p}log{x}_i^p-(\sum _{j=1}^p{\beta }_j)logg(x_i) \\ ={\beta }_{1}log\frac{x_i^1}{g(x_i)}+\cdots +{\beta }_{p}log\frac{x_i^p}{g(x_i)}\text{subject}\text{to}\sum _{j=1}^p{\beta }_j=0. \end{gathered}$$

Therefore, Li’s proposal of regression on log-ratio abundances is equivalent to regression on the CLR-transformed data X̃ provided a zero-sum constraint is imposed on β. In contrast, however, our formulation does not explicitly impose a constant-sum constraint. In fact, this constraint is not needed because the CLR transform removes the analysis from the simplex to allow an analysis in Euclidean vector space algebra (Egozcue and Pawlowsky-Glahn, 2011). Our model instead incorporates the appropriate covariance structure for the CLR transformation, C.

As a final observation, a positive-definite C in (2.13), or more generally Q in (2.5), can be decomposed as a sum Q = I+Q̃ of the identity plus a positive semi-definite singular matrix Q̃. The identity term constrains ${\sum }_{j=1}^p{\beta }_j^2$ to be small while, overall, Q̃ encourages extrinsic structure (e.g., smoothness). One may also control the size of ${\sum }_{j=1}^p{\beta }_j^2$ by adding or subtracting values in the diagonal entries of Q. This idea is similar to that of “Grace-ridge” in Zhao and Shojaie (2016) where, in addition to the penalty induced by Q, the authors propose to further impose a ridge-type penalty in the objective. We apply the significance testing framework of Zhao and Shojaie (2016) in Section 4.

3. Numerical Experiments

To illustrate the proposed framework, we perform several data-driven simulations using publicly available micro-biome data. We consider three scenarios from the literature that exploit extrinsic structure from a phylogenetic tree, including DPCoA, UniFrac and edge PCA. To achieve realistic simulations, we simulate “true” signals of the type implied by each of these methods in order to create benchmarks for performance evaluation. Our emphasis is on formalizing the role that such structure plays in penalized regression when modeling associations between the multivariate data, X, and a response variable, y. Since y is directly simulated from X in these settings, the compositional nature of the data discussed in Section 2.4 does not affect the simulation results. We will return to this topic when analyzing the relative abundance data in Section 4.

The numerical experiments in this section are motivated by the relationship between the PCoA plots and PCR described in Section 2.1 and Figure 1(b). This connection can be generalized to a number of other commonly-used graphical representations in the microbiome literature. For instance, any two-dimensional DPCoA plot involves an implicit coefficient vector, β, of associations between y and X.

Throughout this section, we compare the performance of KPR with ridge regression and lasso. Ridge regression provides a direct extension of ordinary least squares and thus is a natural benchmark for comparing various KPR estimates. Lasso, which gives sparse estimates, is used as a benchmark in settings where the true β is sparsely non-zero. The choice of competing methods is limited by our emphasis on estimating β, rather than predicting the outcome y. Indeed, most kernel methods focus on prediction which renders them inappropriate for comparison.

In all simulation experiments, the tuning parameters for KPR, ridge and lasso are chosen using 10-fold cross-validation. Specifically, to compare the prediction performance of KPR, ridge and lasso, we choose the tuning parameters that minimize squared test error in held-out cross validation samples (CV min). On the other hand, the task of estimation usually requires more smoothing than prediction (Cai and Hall, 2006). Therefore, when examining the estimation performances of KPR, ridge and lasso, we use the largest tuning parameters such that the squared test errors are within one standard error of the minimum squared test error (CV 1se), as suggested in Hastie, Tibshirani and Friedman (2009). For comparison, we also consider the tuning parameters corresponding to the minimum squared test error for ridge and lasso.

3.1. Regression and DPCoA

In our first example, we compare the estimation and prediction performances of KPR, ridge and lasso using the data depicted in Figure 1. The rows of X represent relative abundances of p = 149 taxa from n = 100 subjects in a study by Yatsunenko et al. (2012). The outcome y is log-transformed age of each subject. For KPR, we use K_Q = XQX′ and H = I, where $Q=-\frac{1}{2}\mathcal{J}\delta \mathcal{J}$ is a matrix of similarities between taxa obtained from the matrix of squared patristic distances, δ. Motivated by DPCoA plots, we assume the underlying “true” response y_True is generated from the first two eigenvectors of K_Q. Let L be the Cholesky factor of Q, i.e., Q = LL′, and let XL = U^LS^L(V^L)′. Recall that A_(k) denotes the first k columns of matrix A, or its first k rows and columns if A is diagonal. Motivated by (2.6), we let

$${\beta }_{\text{True}}=s\left(V_{(2)}^L{(S_{(2)}^L)}^{-1}{(U_{(2)}^L)}^{\prime }y,\tau \right),$$ (3.1)

where, s(·, τ) is the hard-thresholding operator, i.e., s(x, τ) = x · 1(|x| > τ). The threshold τ ≥ 0 is set to achieve various levels of sparsity: ||β_True||₀ ∈ {⌊0.2p⌋, ⌊0.6p⌋, p}. After generating β_True, we simulate

$$y_{\text{True}}=U_{(2)}^L{S}_{(2)}^L{(V_{(2)}^L)}^{\prime }{\beta }_{\text{True}}.$$

The simulation is repeated 500 times, each with a different $\epsilon ~N_n(0,{\sigma }_{\epsilon }^2{I}_n)$ in y_obs = y_True + ε. Further, ${\sigma }_{\epsilon }^2$ is set to achieve $R^2=var(y_{\text{True}})/(var(y_{\text{True}})+{\sigma }_{\epsilon }^2)\in \{0.1,0.2,\dots ,0.9\}$. In each repetition, we estimate β̂_DPCoA from y_obs according to definition 2.1. To make the simulation more realistic, we add error to the matrix Q used to simulate β_True and y_True. I.e., we use Q_obs, obtained by adding random Gaussian noise to Q, to estimate β̂_DPCoA. The eigenvalues of Q_obs are adjusted to be equal to the eigenvalues of Q. The amount of Gaussian noise added to the entries of Q_obs is empirically determined to achieve ||Q − Q_obs||_F/||Q||_F ∈ {0, 0.25, 0.5}. As a comparison, we estimate β̂_Ridge and β̂_Lasso using only X and y_obs, without incorporating Q. From the estimated coefficients, we compute ŷ_DPCoA = XL_obsβ̂_DPCoA, ŷ_Ridge = Xβ̂_Ridge and ŷ_Lasso = Xβ̂_Lasso. The performance metrics are the prediction sum of squared error (PSSE) from y_True and estimation sum squared error (ESSE) from β_True.

Figure 2 shows the estimation and prediction performance of KPR, ridge and lasso. KPR significantly outperforms both ridge regression and lasso for both prediction and estimation in all settings. As expected, the performance of ridge and lasso for estimation improve when using a larger tuning parameter. On the other hand, neither mis-specification of Q nor sparsity of β_True seems to substantially impact the relative performance of the three methods. This may be due to the fact that KPR estimates the correct target β_True, even with mis-specified Q, whereas ridge regression and lasso estimate the wrong target.

Fig 2.

Estimation sum squared error (ESSE: left panels) and prediction sum squared errors (PSSE: right panels) of KPR, ridge regression and lasso, and their 95% confidence bands. Standard errors for ESSE and PSSE are estimated based on 500 simulation runs, and are roughly 0.5%–2% (ESSE) and 2%–5% (PSSE) for KPR. We consider three sparsity settings for β_True, based on (3.1): ||β_True||₀ = p in top panels; ||β_True||₀ = ⌊0.6p⌋ in center panels, and ||β_True||₀ = ⌊0.2p⌋ in bottom panels. For ridge and lasso, tuning parameters that produce the smallest cross-validated squared test error (CV min), and the largest tuning parameters such that the cross-validated squared test errors are within one standard error of the minimum cross-validated squared test error (CV 1se) are considered. For KPR, we consider ||Q − Q_obs||_F/||Q||_F = 0 (no Q error), 0.25 (small Q error) and 0.5 (large Q error).

3.2. Regression and PCoA with respect to a UniFrac kernel

In the case of PCoA with respect to a UniFrac matrix Δ^U of squared dissimilarities, the graphical displays are based on the eigen-decomposition of $H=-\frac{1}{2}\mathcal{J}{\mathrm{\Delta }}^U\mathcal{J}$. That is, for $H=U^H{(S^H)}^2{(U^H)}^{\prime }\approx U_{(2)}^H{(S_{(2)}^H)}^2{(U_{(2)}^H)}^{\prime }$, the n samples are represented in two dimensions by the columns of $U_{(2)}^H{S}_{(2)}^H$; this results in points ${\{{\eta }_{i1},{\eta }_{i2}\}}_{i=1}^n:={\{({\sigma }_1{U}_{i1}^H,{\sigma }_2{U}_{i2}^H)\}}_{i=1}^n$, as plotted in Figure 1. When the points are colored according to a response variable, ${\{y_i\}}_{i=1}^n$, the implied regression model is

$$\begin{gathered} y={\gamma }_1{\eta }_1+{\gamma }_2{\eta }_2+\epsilon \\ =U_{(2)}^H{S}_{(2)}^H\gamma +\epsilon . \end{gathered}$$ (3.2)

However, in contrast to PCR in eq. (2.2), where US = XV, it is not obvious how to connect γ directly to the p-coordinates corresponding to the p columns of X. Here, we exploit the joint eigenstructure of kernels K_I and H by proceeding as in (2.11) to obtain the estimate β̂_H = X′ γ̂ as in (2.12), with Q = I.

In this example, we use the same data as in Section 3.1. For KPR, we use K = XX′ and obtain $H=-\frac{1}{2}\mathcal{J}{\mathrm{\Delta }}^U\mathcal{J}$ using the UniFrac distance matrix. We simulate γ_True and y_True from the first two eigenvectors of H, as in (3.2):

$$\begin{gathered} {\gamma }_{\text{True}}={\left({(U_{(2)}^H)}^{\prime }(U_{(2)}^H)\right)}^{-1}{S}_{(2)}^H{(U_{(2)}^H)}^{\prime }y \\ {y}_{\text{True}}=U_{(2)}^H{S}_{(2)}^H{\gamma }_{\text{True}}. \end{gathered}$$ (3.3)

This bivariate ordinary least squares regression is illustrated in Figure 1(b).

The simulation is repeated 500 times, each with a different $\epsilon ~N_n(0,{\sigma }_{\epsilon }^2{I}_n)$ to produce various values of R² ∈ {0.1, 0.2, . . . , 0.9}. We compute ŷ_KPR = K γ̂_KPR, where γ̂_KPR is estimated using (2.10). Similar to the last example, we do not assume we always observe the H matrix that is used to generate γ_True and y_True; rather, we use a noisy version, H_obs, of H in KPR with ||H − H_obs||_F/||H||_F ∈ {0, 0.25, 0.5}.

Although we estimate β here as in (2.7) (with Q = I), there is no obvious way to simulate a β_True using UniFrac and so we do not compare the methods based on their estimation performances, and only consider prediction. For all three methods, we find the tuning parameters that minimize the cross-validated H_obs-weighted squared test error. While the use of H in tuning ridge and lasso penalties deviates from the common practice, it results in improved performances, given the important role of H in this simulation. The H matrix also defines the valid distance in this example. Thus, to evaluate the prediction performances of various methods, we use the H-weighted prediction sum of squared error (HPSSE), ${\Vert \widehat{y}-y_{\text{True}}\Vert }_H^2$.

Figure 3 shows that KPR consistently outperforms ridge regression and lasso in prediction, even with a reasonable amount of misspecification of H. This may be due to the fact that, with the incorporation of the H matrix, KPR estimates the correct target whereas ridge and lasso do not.

Fig 3.

H-weighted prediction sum of squared error (HPSSE) of KPR, ridge and lasso, with 95% confidence bands. Standard errors for HPSSE are estimated based on 500 simulation runs, and are roughly 1%–4% for KPR. For KPR, we consider ||H − H_obs||_F/||H||_F = 0 (no H Error), 0.25 (small H Error) and 0.5 (large H Error).

3.3. Regression and PCoA using an edge-matrix kernel

In this section, simulations are based on data from a study of bacterial vaginosis (BV) by Srinivasan et al. (2012) in which 16S rRNA gene samples were collected using vaginal swabs from n = 220 women with and without BV. Here, the outcome y represents pH measured from vaginal fluid of each subject and we consider the association of y with genus-level taxa. In this example, we use the p = 62 genera that exhibit non-zero sequence counts in at least 20% of the subjects. So here, X represents 220 × 62 abundances in a sample-by-genus matrix, and we use a kernel K = XX′. Additionally, however, we define a second kernel H = EE′ based on the “edge mass difference matrix”, E, originally introduced by Matsen and Evans (2013). If the full phylogenetic tree has q edges, each sample can be represented by a vector indexed by all q edges, the e^th coordinate of which quantifies the difference between the fraction of sequence reads on either side of the edge; i.e., the fraction of reads observed on the root side of the tree minus the fraction of reads on the non-root side. We refer to Matsen and Evans (2013) for details and a discussion of “edge PCA”, which refers to PCA applied to the n × q matrix E. Note, in particular, that abundances from every taxon level in the tree contribute to a similarity between subjects as opposed to abundances at a single taxon level, which is used in UniFrac or DPCoA.

In summary, X represents p = 62 genus-level abundances while E is based on all q = 1770 edges in the original phylogenetic tree. Figure 4(a) shows a PCA plot of the 220 subjects in which their similarity is defined using the edge kernel H = EE′; the color of each dot represents the subject’s pH. Figure 4(b) is a heatmap of the kernel H used to create Figure 4(a). The columns and rows of H represent similarities between samples based on the edge mass difference matrix, ordered by subjects’ pH measurements. Similarly, Figure 4(c) is a (Euclidean) PCA plot based on similarities defined using the genus-level abundance kernel, K = XX′. Figure 4(d) is a heatmap of the kernel K used to create Figure 4(c), and subjects are again ordered by pH. These figures illustrate how two different measures of similarity (two separate kernels) may be co-informative in the sense that they both provide information about grouping of subjects’ microbiota in relation to their pH. It is thus natural to expect that incorporating information from both H and K within the KPR framework may result in improved estimates of association between y = pH and the microbial abundances.

For the simulation, we define a “true” association between pH and the genus-level taxa in X using the 2-dimensional PCR model in eq. (2.2) and (2.3). Specifically, we use the apparent association between y = pH and genus-level abundances in Figure 4(c) to construct a “true” coefficient vector β_True as follows. Using the SVD of X = USV′, and proceeding as in (3.3), define

$$\begin{gathered} {\gamma }_{\text{True}}={[{(U_{(2)}{S}_{(2)})}^{\prime }(U_{(2)}{S}_{(2)})]}^{-1}{(U_{(2)}{S}_{(2)})}^{\prime }y \\ {y}_{\text{True}}=U_{(2)}{S}_{(2)}{\gamma }_{\text{True}}, \end{gathered}$$

We then project y_True onto the space spanned by the first two singular vectors of X to define a true coefficient vector as

$${\beta }_{\text{True}}=V_{(2)}{S}_{(2)}^{-1}{U}_{(2)}^{\prime }{y}_{\text{True}}.$$

We now consider how the contribution of H = EE′ can aid in both the prediction of y_True and the estimation of β_True even though, by construction, neither are informed by E.

Taking H = EE′ in a KPR model of the form (2.10), we compare the resulting estimate of β with ridge and lasso estimates. The simulation is repeated 500 times, each with a different $\epsilon ~N_n(0,{\sigma }_{\epsilon }^2{I}_n)$ to produce various values of R² ∈ {0.1, 0.2, ..., 0.9}. The performance metrics are the estimation sum squared error (ESSE) the H-weighted prediction sum squared error (HPSSE) as in the previous section. In this numerical example, we do not assume we always observe the true H matrix; rather, we use a noisy version, H_obs, of H in KPR with ||H − H_obs||_F/||H||_F ∈ {0, 0.25, 0.5}. For all three methods, tuning parameter values are chosen to minimize the sum of squared test error weighted by H_obs. As in the simulation for DPCoA, we also allow for using the largest tuning parameters such that the squared test error weighted by H is within one standard error of the minimum squared test error.

Figure 5 shows that KPR significantly outperforms ridge and lasso in both prediction and estimation. Even though H is not used to simulate the true association, using the edge kernel in KPR enhances the performance of both estimation and prediction, as long as H is not severely misspecified. Once again, the performance of ridge and lasso estimates improve when using a larger tuning parameter (CV 1se).

Fig 5.

In silico evaluation of using tree-based edge information in regression models. Estimation sum squared error (ESSE) and H-weighted prediction sum squared error (HPSSE) of KPR, ridge regression and lasso, with the 95% confidence bands. Standard errors for ESSE and HPSSE are estimated based on 500 simulation runs, and are roughly 2%–5% (ESSE) and 3%–5% (hPSSE) for KPR. For KPR, we consider ||H − H_obs||_F/||H||_F = 0 (no H error), 0.25 (small H error) and 0.5 (large H error).

4. Application to an observational study

We apply our kernel-penalized regression framework to data from 16S rRNA gene collected in a study of premenopausal women (Hullar et al., 2015). This study investigated aspects of gut microbial communities in stool samples from premenopausal women using 454 pyrosequencing of the 16S rRNA gene. The abundances of 127 species were zero for more than 90% of the subjects and were removed from our analysis. The data set we consider consists of p = 128 species sampled from n = 102 women.

To make the measurements comparable between subjects, the species abundances were scaled by the total number of sequences measured in each sample. This scaling produces compositional data (the relative abundances in each sample sum to 1) which introduces analytical complications. In particular, regression analysis using compositional covariates must somehow account for their unit sum constraint (Kurtz et al., 2015; Li, 2015). For this reason, we apply the CLR transformation to the relative abundance values and use this transformed data X̃ as the matrix of predictors in the KPR model. Additionally, using Aitchison’s variation matrix (Aitchison, 1982), T, we obtain the covariance matrix, C, as described prior to eq. (2.13). As C provides more accurate information on the covariance among the true abundances than does the empirical covariance matrix from relative abundances, X, or their CLR transform, X̃, we use C in place of Q in (2.5).

In this example, we examine the effect of using the CLR transformed data X̃ and covariance C as in (2.13) and fit penalized regression models with the goal of estimating β̃_C in (2.13) for the purpose of identifying specific species that may be associated with percent fat in the cohort described above. To this end, we apply a recently developed significance testing procedure to three high-dimensional models in order to identify species exhibiting evidence of association with subjects’ adiposity. This significance test for graph-constrained estimation, called Grace (Zhao and Shojaie, 2016), provides a means to assign significance to estimates from penalized regression models that incorporate structure of the type provided by Q in (2.5) (or C in (2.13)). The method asymptotically controls the type-I error rate regardless of the choice of Q. The special case with Q = I provides a sig-nificance test for ordinary ridge regression. In each application of the Grace test, tuning parameters are selected based on the smallest squared test error using 10-fold cross validation. Following Zhao and Shojaie (2016), the assumed sparsity parameter is set to be ξ = 0.05. The tuning parameter for the initial estimator is set to be ${\lambda }_{\mathit{init}}=4{\widehat{\sigma }}_{\epsilon }\sqrt{3logp/n}$, where σ̂_ε is the estimated standard deviation of the random error ε, using the scaled Lasso (Sun and Zhang, 2012). To assess significance for the sparse models using lasso, we apply the recently proposed significance test for lasso regressions based on low-dimensional projection estimator (LDPE) (Zhang and Zhang, 2014; Van de Geer et al., 2014), which provides an asymptotically valid test for lasso-penalized regression estimates.

We report on five regression estimation methods for which the significance of regression coefficients can be evaluated using existing high-dimensional testing methods. Two are obtained using the relative abundances, X, with respect to: (i) an ordinary ridge penalty and (ii) a lasso penalty. Three are obtained using the CLR transformed abundances, X̃, with respect to: (iii) an ordinary ridge penalty, (iv) a lasso penalty, and (v) the KPR estimate in (2.13). None of these methods results in any species associated with the outcome of percent fat when controlled for false discovery rate (FDR) at 0.1 using the Benjamini-Yekutieli procedure (Benjamini and Yekutieli, 2001). However, when using a cut-off of p = 0.01, the KPR estimate (2.13) results in ten species. With a cut-off of p = 0.005, KPR results in four species. Ordinary ridge regressions using the CLR-transformed vectors find no associations at a cut-off of p = 0.01, whereas using the relative abundances, ridge finds two species at the p = 0.01 cut-off and none at p = 0.005. Lasso regression with the CLR-transformed vectors identifies one specie at the p = 0.01 cut-off and none at p = 0.005 cut-off. When using the relative abundances, lasso identifies two species as significant at the p = 0.01 cut-off and one at the p = 0.005 cutoff. See Table 1 for the list of identified species.

Table 1.

Species found to be associated with percent fat (in increasing order of p-values) at different significant levels using: KPR with centered log-ratio transformed abundances (CLR) ; ridge and lasso regression with centered log-ratio transformed abundances; and ridge and lasso regression with untransformed relative abundances (rel%).

	p < 0.01	p < 0.005	FDR < 0.1
KPR + CLR	Bacteroides, Anaerovorax, Acidaminococcus, Blautia, Dethiosulfatibacter, Asaccharobacter, Turicibacter, Lebetimonas, Streptobacillus, Anoxynatronum	Bacteroides, Turicibacter, Acidaminococcus, Dethiosulfatibacter	(none)
Ridge + CLR	(none)	(none)	(none)
Ridge + rel%	Catonella, Dethiosulfatibacter	(none)	(none)
Lasso + CLR	Roseburia	(none)	(none)
Lasso + rel%	Dethiosulfatibacter, Micropruina	Dethiosulfatibacter	(none)

5. Discussion

We have formulated a family of regression models that naturally extends the dimension-reduced graphical explorations common to microbiome studies. In this sense, we have simply re-focused the role of the eigen-structures used in ordination methods toward exploiting this structure in penalized regression models. The large family of models developed here provides a supervised statistical learning counterpart to the unsupervised methods of principal coordinate analysis (PCoA).

A primary motivation for PCoA graphical displays is the ability to incorporate biologically-inclined measures of (dis)similarity. The popular use of UniFrac, for instance, is motivated by the desire to impose phylogeny into the analysis. These dissimilarities have also been used for rigorous statistical testing in the context of Anderson’s nonparametric MANOVA (Anderson, 2006) or the closely-related kernel machine regression score test (Chen et al., 2012; Pan, 2011; Zhao et al., 2015) for global association of a multivariate predictor with an outcome. However, the use of UniFrac and other non-Euclidean distances make it difficult to identify specific associations between the microbial abundance profiles and a phenotype; indeed, none of these analyses proceed to estimate the individual associations. In addition to ordination displays and global tests for associations, a variety of machine learning approaches have emphasized on models that predict a response. In contrast, we focus on estimating the coefficient vector, which is a key aspect of any approach used to draw scientific conclusions based on the association of microbial communities with an outcome or phenotype.

An interesting feature of the proposed kernel-penalized regression framework is its ability to sidestep some of the problems inherent in compositional data analysis. Indeed, as emphasized by Li (2015) regression analysis with compositional covariates must somehow acknowledge their unit-sum constraint and spurious correlations. Our approach, which differs somewhat from that of Li (2015), may also be viewed as a penalized version of the low-dimensional linear model for compositions by Tolosana-Delgado and Van Den Boogart (2011), who use the isometric log-ratio (ILR) coordinates. We note that ILR coordinates arise from the SVD of mean-centered CLR-transformed data, X̃ (see Egozcue and Pawlowsky-Glahn (2011)), which is also used in our model. However, to estimate β ∈ ℝ^p, we used instead a regularization framework; our penalty in Section 2.4 arises from Aithison’s total variation matrix whose singular values are the total variances of ILR components. Moreover, the proposed framework also allows us to use existing inference frameworks for high-dimensional regression, and in particular the Grace test (Zhao and Shojaie, 2016), to assess the significance of estimated regression coefficients.