Hierarchical Spatial Process Models for Multiple Traits in Large Genetic Trials

Abstract

This article expands upon recent interest in Bayesian hierarchical models in quantitative genetics by developing spatial process models for inference on additive and dominance genetic variance within the context of large spatially referenced trial datasets of multiple traits of interest. Direct application of such multivariate models to large spatial datasets is often computationally infeasible because of cubic order matrix algorithms involved in estimation. The situation is even worse in Markov chain Monte Carlo (MCMC) contexts where such computations are performed for several thousand iterations. Here, we discuss approaches that help obviate these hurdles without sacrificing the richness in modeling. For genetic effects, we demonstrate how an initial spectral decomposition of the relationship matrices negates the expensive matrix inversions required in previously proposed MCMC methods. For spatial effects we discuss a multivariate predictive process that reduces the computational burden by projecting the original process onto a subspace generated by realizations of the original process at a specified set of locations (or knots). We illustrate the proposed methods using a synthetic dataset with multivariate additive and dominant genetic effects and anisotropic spatial residuals, and a large dataset from a scots pine (Pinus sylvestris L.) progeny study conducted in northern Sweden. Our approaches enable us to provide a comprehensive analysis of this large trial which amply demonstrates that, in addition to violating basic assumptions of the linear model, ignoring spatial effects can result in downwardly biased measures of heritability.

1. INTRODUCTION

Increasing international carbon credit markets and demand for wood fiber to supply burgeoning bioeconomies has spurred interest in improving forest ecosystem services. Much of this demand will be met by large plantation forests and, as a result, foresters seek methods to improve the productivity of these plantations. Tree breeding is one of the most important methods to improve productivity. Here, the focus is on increasing the genetic gain in future generations of certain traits by identifying and mating superior individuals that have high breeding values. In plantation settings, foresters are typically interested in improving traits which increase wood production (e.g., increasing both stem height and diameter). Depending on the anticipated use of the wood, there are other traits considered such as those that determine stem quality (e.g., the number and form of knots produced by branches from the stem). The principal objective is to reveal whether interrelationship among these various traits of primary or secondary interest are favorable or unfavorable for future selection (i.e., evaluating if traits are positively or negatively correlated depending on the objectives of the study). Henderson and Quaas (1976) introduced the multitrait animal (or tree or individual) model in quantitative genetics. Efficient breeding requires accurate estimates of the genetic variance and covariance parameters within the multitrait animal model and individual breeding values (Lynch and Walsh 1998).

Quantitative genetics studies the inheritance of polygenic traits, focusing upon estimation of additive genetic variance and heritability often estimated as the proportion of additiuve variance out of the total genetic and unexplained variation. A high heritability should result in a larger selection response, that is, a higher probability for genetic gain in future generations. A feature common to genetic trials in forestry and agriculture is the presence of systematic heterogeneity among observational units. If this spatial heterogeneity is neglected in the model, estimates of the genetic parameters might be biased (Dutkowski et al. 2002; Cappa and Cantet 2007). For spatially arranged observational units, heterogeneity typically results from small-scale environmental conditions (e.g., soil characteristics, microclimates, light availability, etc.) that might cause correlation among units as a function of distance and/or direction. While traditional randomized designs sought to negate any confounding spatial correlation, recent interest has shifted to models accounting for spatial correlation that, most often, incorporate spatial correlation through differencing or by constructing covariates using residuals from neighboring units (see, e.g., Zimmerman and Harville 1991; Cullis et al. 1998).

Hierarchical random effects models implemented through Markov chain Monte Carlo (MCMC) methods have recently gained popularity within quantitative genetics (see, e.g., Sorensen and Gianola 2002; Waldmann and Ericsson 2006; and references therein). Both single-component and blocked Gibbs samplers have been proposed. However, the former suffer from slow mixing and the latter often involve matrix decompositions whose complexity increases as O(n³) in the number of observations, n, at every iteration of the MCMC algorithm. These computationally intensive matrix calculations limit inference to small datasets insufficient for breeding purposes. Recently, Finley et al. (2009a) proposed a class of hierarchical models for a single outcome variable with genetic and spatial random effects that are suited for large datasets. A regular spectral decomposition of genetic covariance matrices together with a reduced-rank spatial process enable full Bayesian inference using MCMC methods.

The current article extends the work of Finley et al. (2009a) to a multivariate spatial setting, where georeferenced trees with known pedigrees have been observed. Multiple traits of breeding interest are measured on each tree. We posit that the variation within and between these traits can be explained by heredity, spatial variation (e.g., soil characteristics or small scale topographic attributes), and some unstructured measurement error. Here we focus upon three traits that are vitally important in the production of trees for lumber and biomass: height (H), diameter at 1.4 meters from the ground (D), and branch angle (B). The first two traits describe the amount of wood produced by a tree (i.e., the wood volume) and the third describes both the wood quality (i.e., due to knots resulting from branches connecting to the tree bole) and structural integrity of the tree (i.e., susceptibility to breakage due to heavy snow loadings). By estimating the variability of each trait’s additive and, to some extent, nonadditive effects while adjusting for their associations within a location, along with the breeding values, plantation tree nurseries are better able to choose among breeding stock to produce seed with the greatest genetic potential for the desired traits.

One approach builds hierarchical multivariate spatial models by exploiting conditional relationships (spatial regressions) between the variables (Royle and Berliner 1999). This can accrue some computational benefits (see, e.g., Banerjee, Carlin, and Gelfand 2004, section 7.2.2) and can be useful in situations where there is a natural chronology, or causality, for the conditioning. For genetic traits, however, such chronology is often absent or, at best, controversial and one seeks joint multivariate models (e.g., Gelfand et al. 2004; Jin, Banerjee, and Carlin 2007).

For the genetic effects we make use of separable association structures and derive a multivariate eigen-decomposition that separates heredity from association among the traits and produces significant computational benefits. For multivariate spatial dependence, a computationally feasible choice is “separable” or “intrinsic” association structures (see, e.g., Banerjee, Carlin, and Gelfand 2004, chapter 7). Here each trait is assigned the same spatial correlation function. It is unlikely that the strength and nature of association will be the same for the different traits and we seek more flexible correlation structures that accommodate a different spatial correlation function for each trait.

A “linear model of coregionalization” (Gelfand et al. 2004) yields arbitrarily rich multivariate correlation models, but its estimation for large datasets is onerous. A reduced-rank multivariate “kriging” model can help achieve dimension reduction. One possibility is to use low-rank smoothing splines that have been fairly conspicuous in univariate settings (Kamman and Wand 2003; Ruppert, Wand, and Carroll 2003; Crainiceanu, Diggle, and Rowlingson 2008). With multiple traits we require multivariate splines whose specifications may be awkward. A reduced-rank multivariate spatial process is arguably more natural and at least as flexible here with the nice interpretability of correlation functions (see Banerjee et al. 2008; Cressie and Johannesson 2008).

A key point here is to address the issue of systematic biases in the estimates of variance components that arise in low-rank kriging or spline models. Since most existing low-rank methods focus upon prediction and smoothing, this has remained largely unaddressed in the aforementioned literature. Here we offer further insight into this systematic bias and improve the predictive process model with a bias-adjustment that easily applies to multivariate applications. Again, the spatial process approach that we adopt here facilitates in understanding and rectifying these biases that, arguably, would have been more difficult to resolve with multivariate splines.

In Section 2, we develop the multivariate spatial models for the data. Section 3 discusses the dimension-reducing predictive process models, systematic biases in estimating variance components from such models and their remedy. Section 4 outlines the implementation details for the genetic and spatial effects. A synthetic data analysis and the scots pine analyses are presented in Section 5. Finally, Section 6 concludes the article with a brief discussion.

2. MULTIVARIATE SPATIAL MODELS FOR PROGENY TRIALS

For our subsequent development we consider a finite collection of spatial locations, say = {s₁, …, s_n}, referenced by Easting–Northing, where m phenotypic traits of a tree have been observed. Let Y_j(s_i) denote the outcome variable corresponding to the jth trait at location s_i and Y(s_i) = (Y₁(s_i), …, Y_m(s_i))′ is the collection of the outcome measures in site s_i. Models for trait outcome measures, that is, the Y_j(s_i)’s, should account for inherent association between the traits within each location and between observations across locations.

2.1 Multivariate Trait Model With Structured Random Effects

Finley et al. (2009a) (also see references therein) discussed structured random effect models for a single trait. Here we extend their work to multivariate traits over spatial domains. The following multivariate spatially referenced trait model

$$\mathbf{Y}({\mathbf{s}}_i)=\mathbf{X}{({\mathbf{s}}_i)}^{\prime }\mathit{\beta }+{\mathbf{a}}_i+{\mathbf{d}}_i+\mathbf{w}({\mathbf{s}}_i)+\mathit{\epsilon }({\mathbf{s}}_i);i=1,\dots ,n.$$ (1)

Here X(s_i)′ is an m × p block-diagonal matrix ( $p={\sum }_{l=1}^m{p}_l$), with lth diagonal element being the 1 × p_l vector x_l(s_i)′ and $\mathit{\beta }={({\mathit{\beta }}_1^{\prime },\dots ,{\mathit{\beta }}_p^{\prime })}^{\prime }$ is a p × 1 vector of regression coefficients, with each β_l a p_l × 1 vector of regression coefficients corresponding to x_l(s_i)′. This specifies the mean structure that accounts for large-scale variation in the outcome. Finer-scale variation in the dependent variable at each location s_i is modeled using three structured random effects. The first is an m × 1 vector of additive effects a_i = (a_i1, …, a_im)′, the second is an m× 1 vector of dominance effects d_i, defined analogously to a_i, and the third is an m × 1 vector of spatial processes w(s_i). Finally, we account for the unstructured residuals ε(s_i) that follow a zero-centered multivariate normal distribution with zero mean and an m × m diagonal matrix, denoted $\mathbf{\Psi }=\text{diag}\{{\tau }_j^2\}$, as its dispersion matrix. Therefore, each outcome variable accommodates its respective variability in measurement—a phenomenon not uncommon in progeny trial datasets.

While a_i and d_i play the role of correlated random effects to capture extraneous variation in the outcome attributable to additive and dominant effects, the spatial effects are assumed to arise from realizations of a spatial stochastic process. Conceptually, w(s) is an unknown function of space and exists at every every location s residing in our spatial domain . The realizations of this process over a finite set of locations, such as , yields spatially correlated random effects, but w(s), being an unknown, perhaps arbitrarily complex, function of space can also act as a surrogate for missing spatially informative predictors in the model. A challenge is to specify computationally feasible multivariate spatial processes. We discuss this in Section 2.3.

In modeling these multivariate effects, we need to model associations between the outcome variables within a site as well as association across sites. The m × 1 additive and dominance vector effects, a_i and d_i, are assumed to have the m × m dispersion matrices Σ_a and Σ_d to model within-site association between the different types. These associations are assumed to remain invariant across the locations. One approach is to “separate” the correlation between the different trees from the association between the traits within a location. For example, we can specify cov{a_i, a_j} = ã_ijΣ_a, where Σ_a is a m × m dispersion matrix capturing the association attributable to additive effects between the traits. This specification produces valid probability models whenever Σ_a is positive definite and the A = {ã_ij} is a positive-definite matrix. The dominance effects d_i is modeled analogously as cov{d_i, d_j} = d̃_ijΣ_d where Σ_d is the dispersion matrix modeling association between the traits attributable to dominance effects and D = {d̃_ij} is positive definite. Lynch and Walsh (1998) detail calculation of A and D from pedigree information.

Estimating (1) from large datasets can be a computationally onerous task as it involves the inverses of the dispersion matrices of $\mathbf{a}={({\mathbf{a}}_1^{\prime },\dots ,{\mathbf{a}}_n^{\prime })}^{\prime }$, d defined analogously and w = (w(s₁)′, …, w(s_n)′)′. For the genetic effects, separability simplifies computations through the Kronecker products var{a} = A ⊗ Σ_a and var{d} = D ⊗ Σ_d. Modeling the multivariate spatial effects flexibly in a computationally feasible manner is, however, more challenging and we explore this in the subsequent sections.

2.2 Spatial Cross-Covariance Functions for Multivariate Processes

Spatially structured dependence is introduced in (1) through a multivariate (m × 1) spatial process w(s) ~ GP(0, C_w(·, ·; θ)) (e.g., Cressie 1993), where the cross-covariance function C_w(s₁, s₂) is defined to be the m × m matrix with (i, j)th entry cov{w_i(s₁), w_j(s₂)}. The cross-covariance function completely determines the joint dispersion structure implied by the spatial process. Specifically, for any n and any arbitrary collection of sites = {s₁, …, s_n} the nm × 1 vector of realizations w = (w(s₁)′, …, w(s_n)′)′ will have the variance–covariance matrix given by Σ_w, an nm × nm block matrix whose (i, j)th block is the cross-covariance matrix C_w(s_i, s_j). Since Σ_w must be symmetric and positive definite, it is immediate that the cross-covariance function must satisfy (i) C_w(s_i, s_j) = C_w(s_j, s_i)′ for any pair of locations and (ii) ${\sum }_{i=1}^n{\sum }_{j=1}^n{\mathbf{x}}_i^{\prime }{\mathbf{C}}_{\mathbf{w}}({\mathbf{s}}_i,{\mathbf{s}}_j){\mathbf{x}}_j>0$ for all x_i, x_j ∈ ℜ^m\{0}.

The first condition asserts that Σ_w be symmetric, although the cross-covariance function itself need not be. The second ensures the positive definiteness of Σ_w and is in fact quite stringent: this condition must hold for any arbitrary set of distinct locations. The above definition implies that C_w(s, s) is symmetric and positive definite. In fact, it is precisely the variance–covariance matrix for the elements of w(s) within site s. We say that w(s) is stationary if C_w(s_i, s_j) = C_w(h_ij), where h_ij = s_i − s_j, while w(s) is isotropic if C_w(s_i, s_j) = C_w(||h_ij||), that is, the cross-covariance function depends only upon the distance between the sites. For stationary processes, condition (i) above implies that C_w(−h) = C_w(h)′, so that symmetric cross-covariance functions are even, that is, C_w(−h) = C_w(h). For isotropic processes, the cross-covariance function will be even and the matrix is necessarily symmetric.

One possibility for C_w(s₁, s₂) is a separable specification, ρ(s₁, s₂; φ)Σ_w, where Σ_w is an m × m covariance matrix between the traits and ρ(s₁, s₂; φ) is a spatial correlation function (e.g., Zimmerman and Harville 1991; Cressie 1993; Stein 1999). This implies var(w) = R(φ) ⊗ Σ_w, where R(φ) = {ρ(s_i, s_j; φ)}. Correlation functions must be positive definite, that is, the matrix R(φ) must be positive definite for any finite collection of sites. When this function depends only upon the separation (difference) between the coordinates, we say the function, or the spatial process, is (weakly) stationary. Moreover, if the function depends only upon the distance between the locations, then we say that the process is isotropic. A well-known result due to Bochner (e.g., Cressie 1993; Stein 1999) characterizes all real-valued positive-definite functions as characteristic functions of symmetric random variables. A separable spatial association structure is rather restrictive here, assigning a single set of spatial correlation parameters φ to be shared by all the traits. This can be inappropriate: why should all the traits have, for example, the same strength of spatial association?

More general cross-covariance functions are not routine to specify since they demand that for any number of locations and any choice of these locations the resulting covariance matrix for the associated data be positive definite. Various constructions are possible. One possibility is the moving average approach of Ver Hoef, Cressie, and Barry (2004). The technique is also called kernel convolution and is a well-known approach for creating rich classes of stationary and nonstationary spatial processes (e.g., Higdon 2002). Yet another approach could attempt a multivariate version of local stationarity, extending ideas in Fuentes (2002). The primary characterization theorem for cross-covariance functions (Yaglom 1987) says that real-valued functions, say C_ij(h), will form the elements of a valid cross-covariance matrix ${\mathbf{C}}_{\mathbf{w}}(\mathbf{h})={\{C_{ij}(\mathbf{h})\}}_{i,j=1}^m$ if and only if each C_ij(h) has the cross-spectral representation C_ij(h) = ∫ exp(2πιt′h) d(C_ij(t)), where $\iota =\sqrt{-1}$, with respect to a positive definite measure C(·), that is, the cross-spectral matrix $\mathbf{M}(B)={\{C_{ij}(B)\}}_{i,j=1}^m$ is positive definite for any Borel subset B ⊆ ℜ^d. The cross-spectral representation provides a very general representation for cross-covariance functions. Matters simplify when C_ij(t) is assumed to be square-integrable ensuring that a spectral density function c_ij(t) exists such that d(C_ij(t)) = c_ij(t) dt. Now one simply needs to ensure that ${\{c_{ij}(\mathbf{t})\}}_{i,j=1}^p$ are positive definite for all t ∈ ℜ^d. Corollaries of the above representation lead to the approaches proposed by Gaspari and Cohn (1999) and Majumdar and Gelfand (2007) for constructing valid cross-covariance functions as convolutions of covariance functions of stationary random fields. For isotropic settings the development is very similar; customary notations use C_w(h_ij) where h_ij = ||s_i − s_j|| is the distance between any two sites.

2.3 A Simple Construction of Valid Cross-Covariance Functions

Cross-spectral forms are often computationally infeasible resulting in integral representations with embedded parameters that are difficult to estimate. We motivate an alternative approach as follows. For the separable model, notice that we can write C_w(s₁, s₂; θ) = LΘ(s₁, s₂; φ)L′ where Σ_w = LL′ is a Cholesky factorization and Θ(s₁, s₂; φ) = ρ(s₁, s₂; φ)I_m. More generally, any valid cross-covariance matrix can be expressed as C_w(s₁, s₂; θ) = L(s₁)Θ(s₁, s₂; φ)L(s₂)′. We call Θ(s₁, s₂) the cross-correlation function which must satisfy the two conditions for a cross-covariance function and, in addition, must satisfy Θ(s, s) = I_m. This implies that C_w(s, s) = L(s)L(s)′ and L(s) identifies with a matrix square root (e.g., Cholesky) of C_w(s, s). A flexible and computationally feasible approach generalizes the spatial correlation function Θ(s₁, s₂; φ) as a diagonal matrix with ρ_k(s₁, s₂; φ_k), k = 1, …, m, as its diagonal elements. This incorporates a set of m correlation parameters, or even m different correlation functions, offering an attractive, easily interpretable and flexible approach. This approach resembles the linear model of coregionalization (LMC) as in, for example, Wackernagel (2006), Gelfand et al. (2004), and Zhang (2007). See, also, Reich and Fuentes (2007) for a Bayesian non-parametric adaptation.

For modeling L(s), without loss of generality one can assume that $\mathbf{L}(\mathbf{s})={\mathbf{C}}_{\mathbf{w}}^{1/2}(\mathbf{s},\mathbf{s})$ is a lower-triangular square root; the one-to-one correspondence between the elements of L(s) and C_w(s, s) is well known (see, e.g., Harville 1997, p. 229). Therefore, L(s) determines the association between the elements of w(s) within s. Choices for modeling L(s) include an inverse spatial-Wishart process for L(s)L(s)′ (Gelfand et al. 2004), element-wise modeling with Gaussian and log-Gaussian processes, or using parametric association structures suggested by the design under consideration. When stationarity is assumed, so that C_w(s, s) = C_w(0) and L(s) = L, we could assign a prior, for example, inverse Wishart, to LL′. Alternatively, instead of a lower-triangular Cholesky factor, we could opt for a spectral square root, setting L(s) = P(s) Λ (s)^1/2, where C_w(s, s) = P(s) ΛP(s)′ is the spectral decomposition for C_w(s, s). We can further parameterize the orthogonal m × m orthogonal matrix function P(s) in terms of the p(p − 1)/2 Givens angles θ_ij(s) for i = 1, …, p − 1 and j = i + 1, …, p (Daniels and Kass 1999). Specifically, $\mathbf{P}(\mathbf{s})={\prod }_{i=1}^{p-1}{\prod }_{j=i+1}^p{\mathbf{G}}_{ij}({\theta }_{ij}(\mathbf{s}))$ where i and j are distinct and G_ij(θ_ij(s)) is the p × p identity matrix with the ith and jth diagonal elements replaced by cos(θ_ij(s)), and the (i, j)th and (j, i)th elements replaced by ± sin(θ_ij(s)), respectively. Given P(s) for any s, the θ_ij(s)’s are unique within range (−π/2, π/2). These may be further modeled by means of Gaussian processes on a suitably transformed function, say ${\stackrel{\sim }{\theta }}_{ij}(\mathbf{s})=log(\frac{\pi /2+{\theta }_{ij}(\mathbf{s})}{\pi /2-{\theta }_{ij}(\mathbf{s})})$.

2.4 Choice of Correlation Function

Finally, we turn to the choice of the spatial correlation functions. The nonstationary Matérn correlation function (Paciorek and Schervish 2006),

$$\frac{1}{\mathrm{\Gamma }(\nu )2^{\nu -1}}\mid \mathbf{\sum }({\mathbf{s}}_i){\mid }^{1/4}\mid \mathbf{\sum }({\mathbf{s}}_j){\mid }^{1/4}{\left|\frac{\mathbf{\sum }({\mathbf{s}}_i)+\mathbf{\sum }({\mathbf{s}}_j)}{2}\right|}^{-1/2}\times {(2\sqrt{\nu d({\mathbf{s}}_i,{\mathbf{s}}_j)})}^{\nu }{\kappa }_{\nu }(2\sqrt{\nu d({\mathbf{s}}_i,{\mathbf{s}}_j)}),$$

offers a very flexible choice for ρ(s_i, s_j; φ(s_i, s_j)). Here φ(s_i, s_j) = (ν, Σ(s_i), Σ(s_j)) varies across the domain (nonstationary), Γ(ν) is the Gamma function, κ_ν is a Bessel function of the second kind and $d({\mathbf{s}}_i,{\mathbf{s}}_j)={({\mathbf{s}}_i-{\mathbf{s}}_j)}^T{(\frac{\mathbf{\sum }({\mathbf{s}}_i)+\mathbf{\sum }({\mathbf{s}}_j)}{2})}^{-1}({\mathbf{s}}_i-{\mathbf{s}}_j)$ is the Mahalanobis distance between s_i and s_j. When Σ(s) = Σ is constant over space, we have $\rho ({\mathbf{s}}_i,{\mathbf{s}}_j;\mathit{\phi })=(1/\mathrm{\Gamma }(\nu )2^{\nu -1}){(2\sqrt{\nu d({\mathbf{s}}_i,{\mathbf{s}}_j)})}^{\nu }{\kappa }_{\nu }(2\sqrt{\nu d({\mathbf{s}}_i,{\mathbf{s}}_j)})$, where d(s_i, s_j) = (s_i − s_j)^TΣ⁻¹(s_i − s_j). This yields a stationary but anisotropic process which can be parameterized as Σ = G(ψ) × Λ²G^T(ψ) where G(ψ) is a rotation matrix with angle ψ and Λ is the diagonal matrix with positive diagonal elements, the λ’s. Thus, φ = (ν, ψ, Λ). The parameter ν controls the smoothness while the rate of spatial decay is controlled by the λ²’s. Finally, a stationary and isotropic process results with Σ = I, whence $\rho ({\mathbf{s}}_i,{\mathbf{s}}_j;\mathit{\phi })=\frac{1}{\mathrm{\Gamma }(\nu )2^{\nu -1}}{(\sqrt{\lambda d({\mathbf{s}}_i,{\mathbf{s}}_j)})}^{\nu }{\kappa }_{\nu }(\sqrt{\lambda d({\mathbf{s}}_i,{\mathbf{s}}_j)})$ with Σ⁻¹ = I and where φ = (ν, λ) with λ controlling spatial decay. This is further simplified by fixing ν = 0.5, yielding the familiar exponential correlation ρ(s_i, s_j; λ) = exp(−λd(s_i, s_j)).

Once a valid cross-covariance function is specified for a multivariate Gaussian process, the realizations of w(s) over the set of observed locations are given by N(0, Σ_w(θ)), where Σ_w(θ) is an mn × mn block matrix whose (i, j)th block is the m × m cross-covariance C_w(s_i, s_j; θ), i, j = 1, …, n. Without further specifications, estimating (1) will involve factorizations that yield the inverse and determinant of the mn × mn matrix Σ_w(θ). Such computations invoke linear solvers or Cholesky decompositions of complexity O(n³m³), once every iteration, to produce estimates of θ. With large n, this is computationally infeasible and we resort to a class of dimension-reducing models in the next section.

3. PREDICTIVE PROCESS MODELS FOR LARGE SPATIAL DATASETS

3.1 Approaches for Modeling Large Spatial Datasets: A Brief Review

Modeling large spatial datasets have received much attention in the recent past. Vecchia (1988) proposed approximating the likelihood with a product of appropriate conditional distributions to obtain maximum-likelihood estimates. Stein, Chi, and Welty (2004) adapt this effectively for restricted maximum likelihood estimation. Another possibility is to approximate the likelihood using spectral representations of the spatial process (Fuentes 2007). These likelihood approximations yield a joint distribution, but not a process that facilitate spatial interpolation. Furthermore, these approaches do not seem to extend easily to multivariate settings, especially in the presence of other structured dependencies such as genetic traits. Yet another approach considers compactly supported correlation functions (Furrer, Genton, and Nychka 2006; Kaufman, Schervish, and Nychka 2008; Du, Zhang, and Mandrekar 2009) that yield sparse correlation structures. More efficient sparse solvers can then be employed for kriging and variance estimation, but the tapered structures may limit modeling flexibility. Also, full likelihood-based inference still requires determinant computations that may be problematic.

In recent work Rue, Martino, and Chopin (2009) propose a promising Integrated Nested Laplace Approximation (INLA) algorithm as an alternative to MCMC that utilizes the sparser matrix structures to deliver fast and accurate posterior approximations. This uses conditional independence to achieve sparse spatial precision matrices that considerably accelerate computations, but relaxing this assumption would significantly detract from the computational benefits of the INLA and the process needs to be approximated by a Gaussian Markov Random Field (GMRF) (Rue and Held 2006). Furthermore, the method involves a mode-finding exercise for hyper-parameters that may be problematic when the number of hyperparameters is more than 10. Briefly, its effectiveness is unclear for our multivariate genetic models with different structured random effects and unknown covariance matrices as hyperparameters.

We opt for a class of models that emerge from representations of the spatial process in a lower-dimensional subspace and that easily adapt to multivariate processes. These are often referred to as low-rank or reduced-rank spatial models and have been explored in different contexts (Kamman and Wand 2003; Stein 2007, 2008; Banerjee et al. 2008; Crainiceanu, Diggle, and Rowlingson 2008; Cressie and Johannesson 2008). Many of these methods are variants of the so-called “subset of regressors” methods used in Gaussian process regressions for large datasets in machine learning (e.g., Rasmussen and Williams 2006). The idea here is to consider a smaller set of locations, or “knots,” say ${\mathcal{S}}^{\ast }=\{{\mathbf{s}}_1^{\ast },\dots ,{\mathbf{s}}_{n^{\ast }}^{\ast }\}$, where the number of knots, n^*, is much smaller than the number of observed sites, and to express the spatial process realizations over in terms of its realizations over the smaller set of knots. It is reasonable to assume that the spatial information available from the n sites could be summarized in terms of a smaller, but representative, set of locations—the knots. This is achieved by an optimal projection of w(s) using the information available from the knots.

3.2 Predictive Process Models

A process-based reduced-rank formulation has recently been explored by Banerjee et al. (2008). We call w(s) the parent process and consider its realizations over an arbitrary set of n^* locations ${\mathcal{S}}^{\ast }=\{{\mathbf{s}}_1^{\ast },\dots ,{\mathbf{s}}_{n^{\ast }}^{\ast }\}$ with n^* ≪ n. Letting ${\mathbf{w}}^{\ast }=(\mathbf{w}{({\mathbf{s}}_1^{\ast })}^{\prime },\dots ,\mathbf{w}{({\mathbf{s}}_n^{\ast })}^{\prime })$ be the realizations of the parent process over , we formulate a multivariate predictive process defined as

$$\begin{array}{l}\stackrel{\sim }{\mathbf{w}}(\mathbf{s})=\text{E}[\stackrel{\sim }{\mathbf{w}}(\mathbf{s})\mid {\mathbf{w}}^{\ast }]\\ =cov(\mathbf{w}(\mathbf{s}),{\mathbf{w}}^{\ast }){var}^{-1}({\mathbf{w}}^{\ast }){\mathbf{w}}^{\ast }\\ =\mathcal{C}{(\mathbf{s};\mathit{\theta })}^{\prime }{\mathbf{\sum }}^{\ast }{(\mathit{\theta })}^{-1}{\mathbf{w}}^{\ast },\end{array}$$ (2)

where (s; θ)′ is a 1× n^* block matrix, each block being m× m, C_w(s, s^*; θ) is the ith block, and Σ^*(θ) is the n^*m × n^*m dispersion matrix of w^*, that is, with ${\mathbf{C}}_{\mathbf{w}}({\mathbf{s}}_i^{\ast },{\mathbf{s}}_j^{\ast };\mathit{\theta })$ as its (i, j)th block. Equation (2) reveals w̃(s) to be a zero mean m × 1 spatial process with cross-covariance matrix given by C_w̃(s_i, s_j; θ) = (s_i; θ)′Σ^*(θ)⁻¹(s_j; θ). The appeal here is that every spatial process (parent) model produces a corresponding predictive process version. Thus, the predictive process counterpart of the parent model in (1) is given by

$$\mathbf{Y}({\mathbf{s}}_i)=\mathbf{X}{({\mathbf{s}}_i)}^{\prime }\mathit{\beta }+{\mathbf{a}}_i+{\mathbf{d}}_i+\stackrel{\sim }{\mathbf{w}}({\mathbf{s}}_i)+\mathit{\epsilon }({\mathbf{s}}_i);i=1,\dots ,n.$$ (3)

No new parameters are introduced and the cross-covariance function of the predictive process derives directly from that of the parent process.

Unlike many other knot-based methods, the predictive process model does not introduce additional parameters or in having to project the data onto a grid. Also, some approaches (e.g., Cressie and Johannesson 2008) require obtaining an empirical estimate of the covariance function. This may be challenging for the hierarchical models we envision here as the variability in the “data” is assumed to be a sum of different structured effects. Hence we cannot use variograms on the data to isolate the empirical estimates of the covariance function from the other random effects. These effects are identified through the second-stage specifications of their dispersion structures. In addition, the predictive process enjoys several desirable theoretical properties in terms of approximating the parent process. For instance, w̃(s), being a conditional expectation, can be regarded as an orthogonal projection of w(s) on an appropriate linear subspace (e.g., Stein 1999). Other properties are discussed in Banerjee et al. (2008).

3.3 Biased Estimates and the Rectified Predictive Process

For multivariate spatial processes, we have the following inequality:

$$\begin{array}{l}var\{\mathbf{w}(\mathbf{s})\}-var\{\stackrel{\sim }{\mathbf{w}}(\mathbf{s})\}\\ =C_{\mathbf{w}}(\mathbf{s},\mathbf{s})-\mathcal{C}{(\mathbf{s},\mathit{\theta })}^{\prime }{\mathbf{\sum }}^{\ast }{(\mathit{\theta })}^{-1}\mathcal{C}(\mathbf{s},\mathit{\theta })\\ =var\{\mathbf{w}(\mathbf{s})\mid {\mathbf{w}}^{\ast }\}\succcurlyeq 0,\end{array}$$ (4)

where var{·} denotes the variance–covariance matrix and ≥ 0 indicates nonnegative definiteness. Equality holds only when s ∈ and thus the predictive process coincides with the parent process.

To understand the implications of the inequality in (4), let us momentarily focus upon a univariate predictive process. For the univariate process (i.e., with m = 1), let w(s) follow a spatial process with covariance function parameterized as $C_w({\mathbf{s}}_1,{\mathbf{s}}_2)={\sigma }_w^2\rho ({\mathbf{s}}_1,{\mathbf{s}}_2;\mathit{\phi })$. The corresponding covariance function for w̃(s) is given by $C_{\stackrel{\sim }{w}}({\mathbf{s}}_1,{\mathbf{s}}_2)={\sigma }_w^2\mathbf{r}{({\mathbf{s}}_1;\mathit{\phi })}^{\prime }{\mathbf{R}}^{\ast }{(\mathit{\phi })}^{-1}\mathbf{r}({\mathbf{s}}_2;\mathit{\phi })$, where ${\sigma }_w^2$ now represents the variance of the spatial process, r(s; φ) is the n^* × 1 vector with the spatial correlation between s and ${\mathbf{s}}_i^{\ast }$, that is, $\rho (\mathbf{s};{\mathbf{s}}_i^{\ast })$, as its ith element and R^*(φ) is the n^* × n^* correlation matrix of the realizations of the parent process over . In particular, the variance of the predictive process at any location s is given by ${\sigma }_{\stackrel{\sim }{w}}^2(\mathbf{s})={\sigma }_w^2\mathbf{r}{(\mathbf{s};\mathit{\phi })}^{\prime }{\mathbf{R}}^{\ast }{(\mathit{\phi })}^{-1}\mathbf{r}(\mathbf{s};\mathit{\phi })$. The inequality in (4) implies ${\sigma }_{\stackrel{\sim }{w}}^2(\mathbf{s})\le {\sigma }_w^2$ at every location s.

The above argument reveals the lower variability of the predictive process which leads to oversmoothing by predictive process models. This, in turn, would lead to overestimation by the predictive process model of the residual (unstructured) variance, τ², as it attempts to absorb the remaining variability. Furthermore, note that the predictive process is inherently nonstationary. This calls for care in interpreting the parameter ${\sigma }_w^2$ in predictive process models. In fact, it may be tempting to conclude that the oversmoothing caused by the predictive process would lead to underestimation of ${\sigma }_w^2$. The matter, however, is somewhat more subtle and in practice we tend to see an upward bias in the estimation of both the spatial and unstructured residual variance components (see Section 5.1).

To explain this, consider a single trait Y(s) in a univariate version (i.e., m = 1) of the model in (1). Let = {s₁, …, s_n} denote the set of observed data locations and assume (for convenience) that ∩ is empty, that is, the knots are completely exclusive of the data locations. Then, standard multivariate normal calculations reveal

$$var(\mathbf{w}\mid {\mathbf{w}}^{\ast })={\sigma }_w^2(\mathbf{R}(\mathit{\phi })-\mathcal{R}(\mathit{\phi }){\mathbf{R}}^{\ast }{(\mathit{\phi })}^{-1}\mathcal{R}(\mathit{\phi }))\succ 0,$$

where w = (w(s₁), …, w(s_n))′, $\mathcal{R}{(\mathit{\phi })}^{\prime }=\{\rho ({\mathbf{s}}_i,{\mathbf{s}}_j^{\ast })\}$ is the n × n^* cross-correlation matrix between the realizations over the locations and over the knots, and > 0 denotes positive definiteness. This implies that R(φ) + γ² I_n > (φ)′R^*(φ)⁻¹(φ) + γ²I_n (A > B denotes A − B is positive definite), where γ² = τ²/σ², or the noise–signal ratio. Since A > B implies B⁻¹ > A⁻¹ (Harville 1997, theorem 18.3.4, p. 434), we have ((φ)′R^*(φ)⁻¹(φ) + γ²I_n)⁻¹ > (R(φ) + γ²I_n)⁻¹. Therefore,

$${\mathbf{u}}^{\prime }{(\mathcal{R}{(\mathit{\phi })}^{\prime }{\mathbf{R}}^{\ast }(\mathit{\phi })\mathcal{R}(\mathit{\phi })+{\gamma }^2{I}_n)}^{-1}\mathbf{u}\ge {\mathbf{u}}^{\prime }{(\mathbf{R}(\mathit{\phi })+{\gamma }^2{I}_n)}^{-1}\mathbf{u},$$ (5)

where u = y − Xβ − a − d. Here $\mathbf{y}={\{Y({\mathbf{s}}_i)\}}_{i=1}^n,\mathbf{a}={\{a_i\}}_{i=1}^n$, and $\mathbf{d}={\{d_i\}}_{i=1}^n$ are n × 1 vectors of the observed outcomes, the additive and dominant effects respectively. Conditional upon the genetic effects and the regression parameters, Equation (5) reveals a systematic upward bias in likelihood-based estimates of the spatial variance parameter from the predictive process model as compared to the parent model. An analogous argument holds for estimates of τ².

If ${\widehat{\sigma }}_{w2}^2$ and ${\widehat{\sigma }}_{w1}^2$ denote the likelihood-based estimates from the predictive process model (3) and the parent model (1), respectively, then using standard eigen-analysis (Appendix A) we obtain ${\widehat{\sigma }}_{w1}^2/{\widehat{\sigma }}_{w2}^2\le {\lambda }_{max}((\mathbf{R}(\mathit{\phi })+{\gamma }^2{I}_n){(\mathcal{R}{(\mathit{\phi })}^{\prime }{\mathbf{R}}^{\ast }{(\mathit{\phi })}^{-1}\times \mathcal{R}(\mathit{\phi })+{\gamma }^2{I}_n)}^{-1})$, where λ_max(A) denotes the maximum eigen-value of the matrix A. Arguments analogous to the above can be used to demonstrate biases in variance component estimation in other low-rank spline models as well.

Using equivalence of Gaussian measures and fixed domain asymptotics Zhang (2004) proved that the spatial covariance parameters cannot all be estimated consistently in classical geostatistical settings with the isotropic Matérn correlation function (see Section 2.4), although the function σ²(λ^2ν) is consistently estimable. However, assigning priors to such reparametrizations may not always be straightforward. From a Bayesian perspective, this implies that the effect of the priors on the inference of the variance components may not be easily overwhelmed by the data—even when the datasets are large.

In lieu of the above, the biases we point out are derived holding the parameters in φ as constant. In such settings, our simulation experiments reveal that both the parent model and the predictive process model often tend to estimate the regression coefficients β and γ² = τ²/σ² in a stable manner. However, while γ² is estimated in a stable manner both τ² and σ² are systematically over-estimated. We point out that we regard the predictive process as a model in its own right, specifically suited for large spatial datasets, and not as an approximation to some model. In other words, its parameters should be interpreted with respect to (3) and not (1).

The key point is that the predictive process, univariate or multivariate, introduces nonstationarity in the spatial variance component. The variance component parameters, therefore, have different interpretations for stationary parent process models and their nonstationary predictive process counterparts. To adjust for the nonstationary spatial variance component, we need to introduce a nonstationary unstructured variance while making sure not to incur computational expenses.

Returning to the more general m× 1 multivariate processes, a rectification is obtained by adding a spatially adaptive “nugget” process to the predictive process w̃(s). More precisely, we introduce w̃_ε̃(s) = w̃(s) + ε̃(s), where $\stackrel{\sim }{\mathit{\epsilon }}(\mathbf{s})\stackrel{\text{ind}}{\sim }\text{MVN}\{\mathbf{0},C_{\mathbf{w}}(\mathbf{s},\mathbf{s};\mathit{\theta })-\mathcal{C}(\mathbf{s};\mathit{\theta }){\mathbf{\sum }}^{\ast }{(\mathit{\theta })}^{-1}\mathcal{C}(\mathbf{s};\mathit{\theta })\}$. We call w̃_ε̃(s) the bias-adjusted predictive process and the model in (3) is now modified to

$$\mathbf{Y}({\mathbf{s}}_i)=\mathbf{X}{({\mathbf{s}}_i)}^{\prime }\mathit{\beta }+{\mathbf{a}}_i+{\mathbf{d}}_i+{\stackrel{\sim }{\mathbf{w}}}_{\stackrel{\sim }{\epsilon }}({\mathbf{s}}_i)+\mathit{\epsilon }({\mathbf{s}}_i);i=1\dots ,n.$$ (6)

Notice that var{w̃_ε̃(s)} = C_w(s, s; θ) = var{w(s)} and ε̃(s) also acts as a nonstationary adjustment to the residual process eliminating the systematic bias in variance components. Section 5.1 demonstrates these biases and their rectification.

We provide some brief remarks on knot selection (see Finley et al. 2009b, for details). With a fairly even distribution of data locations, one possibility is to select knots on a uniform grid overlaid on the domain. Alternatively, selection can be achieved through a formal design-based approach based upon minimization of a spatially averaged predictive variance criterion (e.g., Diggle and Lophaven 2006). However, in general, the locations are highly irregular, generating substantial areas of sparse observations where we wish to avoid placing knots, since they would be “wasted” and possibly lead to inflated predictive process variances and slower convergence. Here, more practical space-covering designs (e.g., Royle and Nychka 1998) can yield a representative collection of knots that better cover the domain. Another alternative is to apply popular clustering algorithms such as k-means or more robust median-based partitioning around medoids algorithms (e.g., Kaufman and Rousseeuw 1990). User-friendly implementations of these algorithms are available in R packages such as fields and cluster and have been used in spline-based low-rank kriging models (Ruppert, Wand, and Caroll 2003; Crainiceanu, Diggle, and Rowlingson 2008).

4. HIERARCHICAL MODEL SPECIFICATION AND IMPLEMENTATION

Let Y = (Y(s₁)′, …, Y(s_n)′)′ be mn×1 and X = (X(s₁), …, X(s_n))′ be mn × p. We assume that a and d follow zero-centered multivariate normal distributions with dispersion matrices A ⊗ Σ_a and D ⊗ Σ_d, respectively, where A = {a_ij} and D = {d_ij} and let F(θ) = (θ)′Σ^*(θ)⁻¹, where (θ)′ is the n × n^* block-matrix whose (i, j)th block is the m × m matrix ${\mathbf{C}}_{\mathbf{w}}({\mathbf{s}}_i,{\mathbf{s}}_j^{\ast })$ with i = 1, …, n and j = 1, …, n^*. Also, let Σ_ε̃ denote the n × n block-diagonal matrix with ith block being the m × m matrix C_w(s_i, s_i; θ) − (s_i; θ)′Σ^*(θ)⁻¹(s_i; θ). The hierarchical model corresponding to (6) can be written in terms of the joint density

$$\begin{array}{l}p(\{\mathbf{\Psi }\})p(\mathit{\theta })\text{N}(\mathit{\beta }\mid {\mathit{\mu }}_{\beta },{\mathbf{\sum }}_{\beta })\text{IW}({\mathbf{\sum }}_a\mid r_a,{\mathrm{\Upsilon }}_a)\text{N}(\mathbf{a}\mid \mathbf{0},\mathbf{A}\otimes {\mathbf{\sum }}_a)\times \text{IW}({\mathbf{\sum }}_d\mid r_d,{\mathrm{\Upsilon }}_d)\text{N}(\mathbf{d}\mid \mathbf{0},\mathbf{D}\otimes {\mathbf{\sum }}_d)\times \text{N}({\mathbf{w}}^{\ast }\mid \mathbf{0},{\mathbf{\sum }}^{\ast }(\mathit{\theta }))\times \text{N}({\stackrel{\sim }{\mathbf{w}}}_{\stackrel{\sim }{\epsilon }}\mid \mathbf{F}(\mathit{\theta }){\mathbf{w}}^{\ast },{\mathbf{\sum }}_{\stackrel{\sim }{\epsilon }})\times \prod _{i=1}^n\text{N}\{\mathbf{Y}({\mathbf{s}}_i)\mid \mathbf{X}{({\mathbf{s}}_i)}^{\prime }\mathit{\beta }+{\mathbf{a}}_i+{\mathbf{d}}_i+{\stackrel{\sim }{\mathbf{w}}}_{\stackrel{\sim }{\epsilon }}({\mathbf{s}}_i),\mathbf{\Psi }\}.\end{array}$$ (7)

The matrices A and D are fixed, while Σ_a and Σ_d are assigned inverse-Wishart priors. The prior on β is N(β|μ_β, Σ_β) and is often taken as “flat,” setting ${\mathbf{\sum }}_{\beta }^{-1}$ to be the null matrix. One could either assume an inverse-Wishart prior for Ψ or take $\mathbf{\Psi }=\text{diag}\{{\tau }_i^2\}$ with $p(\{\mathbf{\Psi }\})={\prod }_{j=1}^m\text{IG}({\tau }_j^2\mid r_{\epsilon j},s_{\epsilon j})$, where IG is the inverse-Gamma distribution [mean = s_εj/(r_εj − 1)]. The spatial covariance parameters are collected into θ.

Assuming stationary cross-covariances [i.e., L(s) = L in Section 2.3], we have θ = {L, φ} and the cross-covariance matrix of the predictive process can be written as (I_n ⊗ L)Θ̃(θ)Σ^*(θ)⁻¹Θ̃(θ)′(I_n^* ⊗ L′), where Θ̃(θ) is a block matrix with Θ(s_i, s_j) as the (i, j)th block. We induce a prior on L by assigning p(LL′) = IW(η, S_L), while all the elements in φ = (ν, ψ, Λ) are assigned weakly informative uniform priors (see Section 5 for details). Therefore, we write p(θ) in (7) as $p(\nu )p(\psi ){\prod }_{i=1}^{2}p({\lambda }_i)p(\mathbf{L})$. Using matrix differentials, we can show that the induced prior on L is given by $p(\mathbf{L})=p(\mathbf{L}{\mathbf{L}}^{\prime })2^m{\prod }_{i=1}^m{l}_{ii}^{m-i+1}$, where L = {l_ij}, and $2^m{\prod }_{i=1}^m{l}_{ii}^{m-i+1}$ is the required jacobian of the transformation.

Estimation proceeds using MCMC employing a Gibbs sampler with Metropolis–Hastings (MH) steps (Gelman et al. 2004); see Appendix B for details. Briefly, the full conditional distributions for each of the parameters’ regression coefficients, the additive, dominant and spatial effects are all normal, while those for the matrices Σ_a, Σ_d, and Ψ are conjugate inverse-Wishart. The spatial variance covariance parameters θ = {L, φ} do not have closed-form full conditionals and are updated using MH steps. Observe that the likelihood [i.e., the last term in (7)] does not involve L or any of the spatial covariance parameters in θ and need not be evaluated for the Metropolis steps. These parameters are updated using Metropolis steps, possibly with block-updates (e.g., all the parameters in φ in one block and those in L in another). Typically, a random walk MH algorithm with (multivariate) normal proposals is adopted.

Gibbs updates still requires computing ${[{\mathbf{A}}^{-1}\otimes {\mathbf{\sum }}_a^{-1}+I_n\otimes {\mathbf{\Psi }}^{-1}]}^{-1}$ for updating a and a similar matrix for d, defined analogously, that are both nm × nm. A brute-force approach will not be computationally feasible, so we derive the following multivariate eigen-decomposition (Appendix C) to obtain

$${\mathbf{\sum }}_{a\mid ·}=({\mathbf{P}}_{\mathbf{A}}\otimes I_m)\left[\underset{i=1}{\overset{n}{\oplus }}{\left(\frac{1}{{\lambda }_{\mathbf{A}}^{(i)}}{\mathbf{\sum }}_a^{-1}+{\mathbf{\Psi }}^{-1}\right)}^{-1}\right]({\mathbf{P}}_{\mathbf{A}}^{\prime }\otimes I_m).$$ (8)

Here ${\mathbf{A}}^{-1}={\mathbf{P}}_{\mathbf{A}}{\mathrm{\Lambda }}_{\mathbf{A}}^{-1}{\mathbf{P}}_{\mathbf{A}}^{\prime }$ and ${\mathbf{\sum }}_a^{-1}={\mathbf{P}}_{{\mathbf{\sum }}_a}{\mathrm{\Lambda }}_{{\mathbf{\sum }}_a}^{-1}{\mathbf{P}}_{{\mathbf{\sum }}_a}^{\prime }$ are the spectral decompositions of A⁻¹ and ${\mathbf{\sum }}_a^{-1}$, and ${\lambda }_{\mathbf{A}}^{(i)}$ denotes the eigenvalues of A. Massive computational gains are achieved since the matrix A is fixed, so the spectral decomposition, which delivers the eigenvalues, needs to be evaluated only once, preceding the Gibbs sampler. All the other matrix inversions on the right hand expression in (8) involve the much smaller m × m matrices. The setting for Σ_d|· is exactly analogous.

A useful strategy for estimating spatial hierarchical models is to replace the conditionally independent likelihood in (7) with the marginalized likelihood obtained after integrating out the spatial effects (e.g., Banerjee et al. 2008; Finley et al. 2009b). However, this becomes problematic in the presence of other structured random effects as we are unable to arrive at forms such as (8). Instead, we integrate out the w^* from (7) and design the Gibbs sampler to obtain the realizations of the predictive process w̃_ε̃. More specifically, this leads to replacing N(w^*|0, Σ^*(θ)) × N(w̃_ε̃|F(θ)w^*, Σ_ε̃) in (7) with N(w̃_ε̃|0, Σ_w̃ε̃(θ)), where Σ _w̃ε̃(θ) = Σ_ε̃+ F(θ)Σ^*(θ)F(θ)′.

Updating w̃_ε̃ requires the inverse of Σ_w̃ε̃(θ)⁻¹+ I_n ⊗ Ψ⁻¹, an nm × nm matrix. Substantial computational benefits accrue from using the Sherman–Woodbury–Morrison (SWM) matrix identities (Henderson and Searle 1981). We first write ${\mathbf{\sum }}_{{\stackrel{\sim }{\mathbf{w}}}_{\stackrel{\sim }{\epsilon }}}{(\mathit{\theta })}^{-1}={\mathbf{\sum }}_{\stackrel{\sim }{\epsilon }}^{-1}-\mathbf{V}{(\mathit{\theta })}^{\prime }\mathbf{M}{(\mathit{\theta })}^{-1}\mathbf{V}(\mathit{\theta })$, where $\mathbf{V}(\mathit{\theta })=\mathcal{C}(\mathit{\theta }){\mathbf{\sum }}_{\stackrel{\sim }{\epsilon }}^{-1}$ is n^*m × nm and M(θ) = Σ^*(θ)⁻¹ + V(θ)Σ_ε̃ V(θ)′ is n^*m × n^*m. After some algebra, we obtain

$${\left[{\mathbf{\sum }}_{{\stackrel{\sim }{\mathbf{w}}}_{\stackrel{\sim }{\epsilon }}}{(\mathit{\theta })}^{-1}+I_n\otimes {\mathbf{\Psi }}^{-1}\right]}^{-1}=\mathbf{H}{(\mathit{\theta })}^{-1}+\mathbf{H}{(\mathit{\theta })}^{-1}\mathbf{V}{(\mathit{\theta })}^{\prime }\mathbf{J}{(\mathit{\theta })}^{-1}\mathbf{V}(\mathit{\theta })\mathbf{H}{(\mathit{\theta })}^{-1},$$

where H(θ) is block-diagonal with [Ψ⁻¹ + (C_w(s_i, s_i; θ) − (s_i; θ)′Σ^*(θ)⁻¹(s_i; θ))⁻¹] as its m × m diagonal blocks, and J(θ) = M(θ) − V(θ)H(θ)⁻¹V(θ)′. The determinant of Σ_w̃ε̃(θ) is required for updating the spatial covariance parameters in the MH steps. This can be evaluated avoiding nm × nm matrices using the identity (derived similarly to the SWM identity)

$$det({\mathbf{\sum }}_{{\stackrel{\sim }{\mathbf{w}}}_{\stackrel{\sim }{\epsilon }}}(\mathit{\theta }))=det(\mathbf{M}(\mathit{\theta }))\prod _{i=1}^{n}det\{{\mathbf{C}}_{\mathbf{w}}({\mathbf{s}}_i,{\mathbf{s}}_i)-\mathcal{C}{({\mathbf{s}}_i,\mathit{\theta })}^{\prime }{\mathbf{\sum }}^{\ast }{(\mathit{\theta })}^{-1}\mathcal{C}({\mathbf{s}}_i,\mathit{\theta })\}/det({\mathbf{\sum }}^{\ast }(\mathit{\theta })).$$

This setup still allows us to recover full posterior inference on the two components of w̃_ε̃, viz. w^* and ε̃. Specifically, given each posterior sample ${\stackrel{\sim }{\mathbf{w}}}_{\stackrel{\sim }{\epsilon }}^{(k)}$ and θ^(k), k = 1, …, K, we draw w^*(l) from a multivariate normal distribution with mean ${\mathbf{\sum }}_{{\mathbf{w}}^{\ast }\mid ·}({\mathit{\theta }}^{(k)})\mathbf{F}{({\mathit{\theta }}^{(k)})}^{\prime }{\mathbf{\sum }}_{\stackrel{\sim }{\epsilon }}{({\mathit{\theta }}^{(k)})}^{-1}{\stackrel{\sim }{\mathbf{w}}}_{\stackrel{\sim }{\epsilon }}^{(k)}$ and variance–covariance matrix Σ_w^*|·(θ^(k)) = Σ^*(θ^(k))⁻¹ + F(θ^(k))′Σ_ε̃(θ^(k))⁻¹ × F(θ^(k)))⁻¹. Next, we can obtain w̃^(k) = F(θ^(k))w^*(k). Finally, if desired, ${\stackrel{\sim }{\mathbf{\epsilon }}}^{(k)}={\stackrel{\sim }{\mathbf{w}}}_{\stackrel{\sim }{\epsilon }}^{(k)}-{\stackrel{\sim }{\mathbf{w}}}^{(k)}$ gives the posterior samples of ε̃.

5. ANALYSIS OF DATA

5.1 Synthetic Illustration of Bias-Adjustment

To supplement the discussion on the predictive process bias-adjustment in Section 3.3, we offer a brief analysis of a synthetic univariate dataset. This dataset comprises 500 locations distributed randomly within a unit square domain. The univariate outcome Y(s) is generated over these 500 locations from a zero-centered isotropic Gaussian process, with covariance $cov\{Y(\mathbf{s}),Y({\mathbf{s}}^{\prime })\}={\sigma }_w^{2}exp(-\phi \mid \mid \mathbf{s}-{\mathbf{s}}^{\prime }\mid \mid )+{\tau }^2{I}_{[\mathbf{s}={\mathbf{s}}^{\prime }]}$, where ${\sigma }_w^2=5$, τ² = 5, and φ = 12 [i.e., the spatial correlation drops to 0.05 at a distance of 0.25 units]. To illustrate the relationship between bias and knot intensity over the domain, we fit regular (unadjusted) and bias-adjusted predictive process models to these data based on 11 knot intensities between 0 and 500. For each model, ${\sigma }_w^2$, τ², φ, and the model intercept β₀ were estimated using three MCMC chains of post burn-in 25,000 iterations.

For brevity, we present only the results for the variance parameters of interest. Results for the unadjusted models are presented in the top subfigures in Figure 1 which illustrates the upward bias in ${\sigma }_w^2$ and τ² estimates as a function of knot intensity. Here, we see a large bias in ${\sigma }_w^2$ and τ² estimates when knot intensity is low. Increasing knot intensity does reduce the bias, but only marginally for τ² which fails to recover the true parameter value even as the number of knots approaches the number of observations. In comparison, the bias-adjusted predictive process recovers the true ${\sigma }_w^2$ and τ² values even at low knot intensities, as illustrated in the bottom subfigures in Figure 1. For the analyses offered in Sections 5.2 and 5.3 only the bias-adjusted models are considered.

Figure 1.

The top row shows estimates of univariate model’s spatial variance σ² (left) and residual variance τ² (right) given increasing number of knots. The second row provides corresponding estimates from the bias-adjusted predictive process model.

5.2 Synthetic Data Analysis

One replicate of full-sib families (800 offspring in total) were constructed by crossing 50 unrelated parents according to a partial diallel design (see Kempthorne and Curnow 1961, for design details). The 850 individuals were assigned randomly to intersections of a 34 × 25 unit grid. The values of three traits of interest were generated for each individual. The synthetic population mean of each trait was described with an intercept and single slope coefficient [with corresponding variable drawn from N(0, 1)] and the multitrait random polygenic effects a and d were drawn from MVN(0, A ⊗ Σ_a) and MVN(0, D ⊗ Σ_d), respectively.

We estimate the hierarchical model in (7) using a stationary but anisotropic Matérn correlation function as described in Section 2.4. The mean fixed effects and lower-triangles of Σ_a and Σ_d are provided in the column labeled True within Table 1. In addition to the genetic effects, random spatial effects w were also added. Here, w was drawn from MVN(0, Σ_w), where Σ_w is the multivariate dispersion matrix defined in Section 2. The spatial processes associated with each response variable arise from an anisotropic Matérn correlation function. The estimates of the stationary process cross-covariance matrix (see Section 2.3), LL′ = C_w(0) ≡ C_w, along with the correlation functions’ parameters are provided in Table 2. These correlation parameters define the processes’ primary axis with an effective spatial range of ~21, that are oriented to ±45° and a second axis, perpendicular to the first, with an effective spatial range of ~9.5. Here, effective range is defined as the distance at which the correlation drops to 0.05 [i.e., when ν = 0.5 the effective range is − log(0.05)λ]. Further, the off-diagonal values in C_w define a strong negative correlation between the first and second process and zero correlation between the first two and the third process. The measurement error was drawn from MVN(0, I_n ⊗ Ψ), with Ψ = 5.0I₃. The multitrait phenotypic values, Y, for each of the 800 individuals were then generated by model (1). The top left image in Figure 2 illustrates the planting grid and the interpolated surfaces of the synthetic spatial random effects follow down the left column. The phenotypic values of the 50 unrelated parents are assumed unknown and are therefore treated as missing data in the subsequent analysis.

Table 1.

Parameter credible intervals, 50% (2.5%, 97.5%) for the synthetic dataset nonspatial and spatial predictive process models. Credible intervals that do not include the parameter’s true value are bolded

Parameter	True	Nonspatial	Anisotropic predictive process
			80 knots	125 knots
β1,0	1	7.76 (6.77, 8.87)	2.04 (0.93, 4.69)	1.40 (−0.39, 3.75)
β1,1	5	5.32 (4.72, 5.89)	5.28 (4.78, 5.74)	5.22 (4.83, 5.66)
β2,0	10	3.72 (2.10, 5.09)	10.32 (8.41, 11.90)	13.20 (8.92, 16.60)
β2,1	15	14.99 (14.20, 15.64)	14.99 (14.44, 15.51)	15.01 (14.42, 15.60)
β3,0	20	27.61 (25.86, 29.38)	20.48 (18.34, 22.85)	22.18 (19.69, 30.61)
β3,1	25	25.07 (24.49, 25.64)	24.95 (24.43, 25.49)	24.91 (24.48, 25.39)
Σa;1,1	25	31.26 (22.35, 40.24)	21.58 (15.39, 29.48)	21.20 (13.35, 30.07)
Σa;1,2	25	26.61 (17.98, 35.77)	25.60 (17.85, 35.20)	22.17 (15.31, 30.08)
Σa;1,3	25	27.88 (19.37, 36.34)	25.02 (17.39, 33.95)	23.74 (15.45, 35.87)
Σa;2,2	50	55.65 (41.04, 73.43)	50.37 (33.40, 69.94)	42.03 (30.37, 56.79)
Σa;2,3	50	56.96 (40.23, 70.59)	51.29 (33.84, 63.76)	47.48 (35.00, 62.20)
Σa;3,3	75	75.43 (55.17, 93.81)	77.19 (50.74, 94.29)	84.04 (58.76, 99.72)
Σd;1,1	25	27.54 (19.90, 37.89)	26.70 (20.56, 35.14)	27.29 (20.68, 35.83)
Σd;1,2	25	19.51 (13.84, 28.78)	22.39 (16.04, 28.31)	24.27 (17.76, 31.56)
Σd;1,3	0	−2.39 (−16.28, 8.38)	−2.15 (−5.87, 3.32)	2.87 (−2.23, 6.62)
Σd;2,2	50	25.12 (7.69, 46.12)	50.26 (36.58, 60.02)	51.55 (41.26, 62.59)
Σd;2,3	0	0.59 (−8.71, 9.79)	−1.31 (−5.80, 8.63)	−1.32 (−8.38, 4.53)
Σd;3,3	4	1.79 (0.28, 9.57)	0.93 (0.19, 2.70)	1.31 (0.16, 4.69)

Table 2.

Parameter credible intervals, 50% (2.5%, 97.5%) for the synthetic dataset nonspatial and spatial predictive process models (continuation of Table 1). Credible intervals that do not include the parameter’s true value are bolded

Parameter	True	Nonspatial	Anisotropic predictive process
			80 knots	125 knots
Cw; 1,1	25	–	36.49 (15.51, 54.58)	32.66 (4.89, 89.72)
Cw; 1,2	−25	–	−31.98 (−55.09, −27.19)	−30.19 (−75.42, −16.18)
Cw; 1,3	0	–	−0.57 (−6.85, 1.37)	−1.16 (−12.03, 10.18)
Cw; 2,2	26	–	31.11 (25.44, 34.09)	30.92 (18.71, 40.23)
Cw; 2,3	0	–	0.26 (−4.31, 2.21)	1.42 (−9.23, 6.03)
Cw; 3,3	25	–	55.79 (13.12, 108.21)	38.26 (3.98, 68.04)
Ψ1,1	5	13.58 (3.52, 24.65)	1.46 (0.17, 5.93)	1.54 (0.14, 6.89)
Ψ2,2	5	43.64 (21.27, 67.24)	0.85 (0.15, 7.22)	4.15 (0.80, 11.31)
Ψ3,3	5	31.02 (22.80, 41.83)	1.56 (0.40, 7.07)	5.40 (1.55, 19.22)
ψw1	45°	–	38.39 (22.06, 81.36)	42.23 (8.08, 58.96)
ψw2	45°	–	57.30 (9.13, 84.63)	47.21 (13.52, 86.00)
ψw3	45°	–	33.17 (1.03, 78.04)	48.07 (4.87,76.89)
Λw1,1	50	–	49.35 (47.94, 50.41)	50.31 (49.17, 52.96)
Λw1,2	10	–	9.79 (8.22, 11.03)	10.05 (8.28, 11.72)
Λw2,1	50	–	50.31 (49.00, 51.04)	51.19 (49.67, 52.09)
Λw2,2	10	–	9.67 (8.93, 10.32)	9.81 (7.78, 10.68)
Λw3,1	10	–	10.58 (9.84, 11.46)	10.82 (9.49, 12.79)
Λw3,2	50	–	49.60 (48.60, 50.38)	49.51 (48.27, 53.06)
νw1	0.5	–	0.57 (0.27, 0.92)	0.62 (0.09, 1.00)
νw2	0.5	–	0.68 (0.40, 0.90)	0.60 (0.29, 0.84)
νw3	0.5	–	0.36 (0.15, 0.55)	0.68 (0.39, 1.16)
$h_1^2$	0.45	0.43 (0.32, 0.51)	0.43 (0.31, 0.54)	0.42 (0.27, 0.54)
$h_2^2$	0.48	0.45 (0.35, 0.55)	0.49 (0.36, 0.63)	0.43 (0.33, 0.53)
$h_3^2$	0.9	0.69 (0.55, 0.78)	0.97 (0.88, 0.99)	0.92 (0.76, 0.97)

Figure 2.

The left column shows the interpolated surfaces of the three response variables over the 850 synthetic data points (overlaid on top left surface). The right column shows the associated estimates of the random spatial effects based on 80 knots (overlaid on the top right surface).

The synthetic dataset is used to illustrate the extent to which the proposed methods can recover genetic parameters of interest. Assuming ignorance about the partitioning of the variance in Y, we assign an IW(m + 1, I_m) prior to each genetic dispersion matrix, while each diagonal element of Ψ received an IG(2, ·) with the scale hyperparameters being set to yield a mean comparable with the residual variances from fixed effects only linear model. The maximum distance across the synthetic domain is about 40; hence, for the anisotropic Matérn models, we assign λ_i ~ U(0.5, 20) for i = 1, 2, 3, which corresponds to an effective range between 1.5 to 60 distance units (when ν = 0.5). Each smoothness and rotation parameter in ν and ψ were assigned U(0, 2) and U[π/2, −π/2) priors, respectively.

Four candidate models were considered: the nonspatial model with only genetic effects and three models with genetic effects and predictive process based random spatial effects. For the spatial models, we considered knot intensities of 80, 101, and 125. Knot configuration followed a lattice plus close pairs design (see Diggle and Lophaven 2006). This design consists of a regular k × k lattice of knots intensified by a set of m′ knots placed at random adjacent to the lattice knots—say within a circle having the knot point as center and a radius that is some fraction of the spacing on the lattice. This design is illustrated in the top right panel in Figure 2.

For each model, three MCMC chains were run for 25,000 iterations. The sampler was coded in C++ and leveraged threaded BLAS and LAPACK routines for matrix computations. The code was run on an Altix 350 SGI with two 64-bit 1.6 GHz Intel Itanium processors. Each chain of 25,000 iterations took less than 2 hours to complete. The CODA package in R (http://cran.r-project.org/web/packages/coda/) was used to diagnose convergence by monitoring mixing using Gelman–Rubin diagnostics and autocorrelations (Gelman et al. 2004, section 11.6). Acceptable convergence was diagnosed within 10,000 iterations and therefore 45,000 samples (15,000 × 3) were retained for posterior analysis.

Tables 1 and 2 offer parameter estimates and associated credible intervals for the nonspatial and predictive process models based on 80 and 125 knot intensities. The nonspatial model is not able to estimate correctly three of the six fixed effects (as identified in bold text in Table 1). Interestingly, despite the misspecification of the model (i.e., missing spatial effects), the additive and dominance dispersion matrices are estimated correctly, but for Σ_d;2,2. However, apportioning of spatial variation to pure error causes the 95% credible intervals of two diagonal elements in Ψ to not cover their true values, Table 2. This can cause a downward bias in the measures of heritability. The true heritability for the three traits are Σ_a;1,1/(Σ_a;1,1 + Σ_d;1,1 + Ψ_1,1) = 25/(25 + 25 + 5) = 0.45, Σ_a;2,2/(Σ_a;2,2 + Σ_d;2,2 + Ψ_2,2) = 0.48, and Σ_a;3,3/(Σ_a;3,3 + Σ_d;3,3 + Ψ_3,3) = 0.9, and the nonspatial model estimates with 95% credible intervals are 0.43 (0.32, 0.51), 0.45 (0.35, 0.55), and 0.69 (0.55, 0.78), as provided in the Table 2. A similar downward bias is seen in the dominance ratio when calculated.

The spatial models provide improved estimates of the fixed and genetic random effects as well as estimates of heritability and dominance ratio. For example, at the 80 knot intensity the heritability estimates for the tree traits are 0.43 (0.31, 0.54), 0.49 (0.36, 0.63), and 0.97 (0.88, 0.99). We note that two parameters fall outside of the 95% credible interval in the 80 knot model (i.e., Σ_d;3,3 and C_w;1,2 in Tables 1 and 2, respectively). However, beyond the 80 knot intensity, all parameters are estimated correctly. The discrepancies in estimates for the lowest knot intensity are likely due to excessive smoothing of the random effects surfaces, as seen in the right column of Figure 2. Here, the response-specific anisotropic rotation and range of dependence are clearly matched between the true (left column) and estimated (right column); however, the 80 knot intensity is forced to smooth over some local variation. Again, more knots allow the model to better conform to the residual surface.

5.3 Scots Pine Data Analysis

Field measurements from a 26-year-old scots pine (Pinus sylvestris L.) progeny study in northern Sweden serve as a trial dataset for our proposed methods. The original design established in 1971 (by Skogforsk; trial identification S23F7110264 Vindeln) consisted of 8160 seedlings divided into two spatially disjoint trial sites (i.e., denoted as northern and southern trial site). The analysis presented here focuses on the northern planting site. At planting, this site was subdivided into 105 8.8 m × 22 m blocks each containing 40 seedlings placed on a 2.2 m × 2.2 m grid. Parent stock was crossed according to a partial diallel design of 52 parent trees assumed unrelated and seedlings were planted out on the grids with unrestricteded randomization. In 1997, the 2598 surviving trees were measured for various traits of breeding interest, including diameter (D) (measured at 130 cm above ground), total tree height (H) and branch angle (B). The branch angle was scored such that an average tree was given score 5, and the range 1–9 reflect desirable to undesirable branch angle (i.e., horizontal branches were given low scores and upwards vertical branches high scores). The top panel in Figure 3 shows the location of the surviving trees use in the subsequent analysis.

Figure 3.

First row: Each point represents a scots pine tree location. The left column shows interpolated surface of nonspatial model residuals for stem height (H), diameter (D), and branch angle (A), respectively from top to bottom. The middle and right columns show associated estimated surfaces of spatial random effects for the 78 and 160 knot intensities modeled with the stationary isotropic Matérn correlation function. Note that to reduce clutter, the knots are overlaid on only the 78 knot H surface.

As in Section 5.2, we fit both nonspatial and spatial models with random spatial effects estimated with varying knot intensities. Specifically, we fit a nonspatial model with a single intercept and additive and dominance effects for each trait. Then random spatial effects were added with knot intensities of 78, 124, and 160. An initial exploratory analysis with an omnidirectional semivariogram of the nonspatial additive and dominance model residuals showed no evidence of an anisotropic process for any of the response variables; therefore, we consider only the spatial isotropic models here.

The left column of Figure 3 shows clear spatial association among the residuals, which is further affirmed by the empirical semivariogram for the trait of interest (not shown). Specifically, in the left column of Figure 3, we see that D and H are strongly associated and have a relatively longer spatial range compared to the surface for B. Inspection of the semivariogram supports this observation, showing an effective range of ~50 for D and H, while B’s spatial range is less than 10. This disparity in effective range suggests that a separable model, with a single spatial range parameter, would be inappropriate. Further, the relatively short spatial range of B, will require a knot configuration where the distance between knots is less than the residual spatial range. We again use the lattice plus close pairs design to help estimate small scale variability and local dependence. Using the isotropic Matérn correlation function, the decay parameters were set to φ ~ U(0.03, 3) and smoothness ν fixed at 0.05. This implies an effective spatial range of about 1 to 100 meters. Again, assuming ignorance about the partitioning of the variance in Y, we assign an IW(m + 1, I_m) prior to each dispersion matrix with the exception of Ψ which received IG(2, ·) on the diagonal elements. The respective IG scale hyperparameters were set to the residual variance of the multivariate intercept-only model.

For each model, three MCMC chains were run for 25,000 iterations and 45,000 post burn-in samples (15,000 × 3) were retained for posterior analysis. The parameter estimates for the nonspatial additive and dominance model as well as the spatial models based on 78 and 160 knots are provided in Table 3. Here, overlapping 95% credible intervals suggest agreement among the nonspatial and spatial models’ fixed and genetic dispersion parameters (i.e., β, Σ_a, and Σ_d). However, the 160 knot model does generally estimate greater additive variance for several diagonal and off-diagonal elements in Σ_a.

Table 3.

Parameter credible intervals, 50% (2.5%, 97.5%) for the scots pine dataset nonspatial and spatial predictive process models

Parameter	Nonspatial	Isotropic predicted process
		78 knots	160 knots
βH,0	72.56 (70.45, 74.68)	73.68 (70.70, 76.78)	70.78 (68.39, 73.28)
βD,0	119.64 (117.01, 122.63)	119.19 (112.35, 126.14)	120.31 (113.96, 126.22)
βB,0	4.89 (4.75, 5.05)	4.78 (4.59, 4.95)	4.93 (4.66, 5.22)
Σa;H	29.38 (19.43, 38.19)	21.25 (15.80, 30.79)	33.12 (21.41, 46.57)
Σa;H,D	17.73 (6.76, 29.45)	13.21 (3.79, 30.36)	30.64 (10.81, 58.05)
Σa;H,B	−0.99 (−1.79, −0.50)	−1.04 (−1.57, −0.66)	−1.05 (−2.09, −0.24 )
Σa;D	43.30 (14.85, 67.31)	52.94 (25.47, 119.27)	89.23 (26.80, 129.97)
Σa;D,B	−0.55 (−1.24, 0.12)	−0.18 (−1.14, 0.77)	−0.38 (−2.02, 1.57)
Σa;B	0.22 (0.17, 0.31)	0.23 (0.16, 0.33)	0.21 (0.15, 0.30)
Σd;H	1.22 (0.80, 2.46)	7.91 (1.40, 15.15)	2.56 (0.14, 9.29)
Σd;H,D	0.21 (−1.66, 1.09)	0.37 (−1.35, 4.29)	−2.25 (−4.53, 5.84)
Σd;H,B	−0.08 (−0.36, 0.08)	−0.13 (−0.64, 0.10 )	−0.12 (−0.52, 0.02)
Σd;D	2.50 (1.48, 3.88)	2.76 (1.70, 8.10)	5.27 (2.35, 12.46)
Σd;D,B	0.05 (−0.22, 0.25)	−0.01 (−0.21, 0.38)	0.17 (−0.23, 0.59)
Σd;B	0.19 (0.13, 0.29)	0.18 (0.13, 0.26)	0.17 (0.12, 0.25)
Cw;H	–	28.41 (19.88, 52.32)	22.83 (19.69, 28.32)
Cw;H,D	–	0.56 (−0.12, 1.91)	0.40 (−0.64, 2.24)
Cw;H,B	–	−0.47 (−0.88, −0.01)	−0.28 (−0.80, 0.15)
Cw;D	–	64.29 (49.62, 86.89)	62.39 (43.32, 163.98)
Cw;D,B	–	0.41 (−0.01, 1.05)	0.23 (−0.46, 1.08)
Cw;B	–	0.16 (0.11, 0.19)	0.16 (0.14, 0.24)
ΨH	134.37 (125.55, 142.99)	87.93 (79.68, 96.13)	86.08 (75.62, 94.43)
ΨD	954.38 (901.96, 1011.65)	790.73 (725.24, 854.88)	777.79 (716.86, 847.03)
ΨB	0.41 (0.32, 0.48)	0.26 (0.19, 0.35)	0.28 (0.20, 0.34)
φwH	–	0.05 (0.03, 0.06)	0.07 (0.05, 0.09)
φwD	–	0.06 (0.03, 0.08)	0.06 (0.03, 0.07)
φwB	–	0.59 (0.57, 0.61)	0.60 (0.58, 0.62)
$h_H^2$	0.18 (0.12, 0.22)	0.18 (0.14, 0.26)	0.27 (0.18, 0.37)
$h_D^2$	0.04 (0.02, 0.07)	0.06 (0.03, 0.14)	0.10 (0.03, 0.15)
$h_B^2$	0.27 (0.21, 0.36)	0.34 (0.26, 0.45)	0.32 (0.24, 0.45)

We compare the candidate models using Deviance Information Criterion (DIC) (Spiegelhalter et al. 2002). With Ω denoting the generic set of parameters being estimated for each model, we compute $\overline{D(\mathrm{\Omega })}=E_{\mathrm{\Omega }\mid \mathbf{Y}}\{-2logL(\mathit{Data}\mid \mathrm{\Omega })\}$, the expected posterior deviance, where L(Data|Ω) is the first stage likelihood from the respective model and the effective number of parameters (penalty) is $p_D=\overline{D(\mathrm{\Omega })}-D(\stackrel{\sim }{\mathrm{\Omega }})$, where Ω̄ is the posterior mean of the model parameters. The DIC is then computed easily from the posterior samples as $\overline{D(\mathrm{\Omega })}+p_D$ with lower values indicating better models.

Table 4 provides DIC scores and associated p_D for the non-spatial and predictive process models. Here, we see a large decrease in DIC moving from the nonspatial to the spatial models. As the knot intensity increases the models become more responsive to local variation in the residual surface and, as a result, we see the DIC scores incrementally decrease.

Table 4.

Model comparisons using the DIC criterion for the scots pine dataset

Model	pD	DIC
Non-spatial	1862.1	34,121.5
Spatial bias-adjusted predictive process
78 knots	3574.6	31,682.1
124 knots	3614.8	31,195.7
160 knots	3685.7	31,098.4

From a breeding point of view, it is encouraging that the additive genetic correlations between branch angle (B) and growth traits (H and D) are small or negative, which means that faster growing trees develop more desirable branching characteristics (lower B scores denote more favorable branch angles). For instance the 160 knot model estimates the median and 95% credible intervals ρ_a;H,B = −0.40 (−1.00, −0.06) and ρ_a;D,B = −0.09 (−1.01, 0.25). For all models, we see a significant positive correlation between H and D, for example, the 160 knot model estimates ρ_a;H,D = 0.56 (0.45,0.75). Again, when the nonspatial model is forced to bring the spatial variation into the Ψ we see lower estimates of heritiability result, compared to those of the spatial models, specifically the 160 knot model. The panels in the middle and right columns of Figure 3 depict the estimated spatial effects surfaces for the 78 and 160 knot models. Comparing these columns to the left-most column, we see that global and local trends in the residual surfaces are captured even when the dimensionality of the problem is reduced from 2598 to 78 knots. Finally, the large disparity in the spatial range estimates, given in Table 3, supports the use of the nonseparable model.

Ultimately, interest turns to estimation and subsequent ranking of the breeding values (i.e., additive effects) and specifically disparities in the ranking order given by the different models. Figure 4 offers a trait specific comparison between the ranking of the top 100 breeding values from the nonspatial and 160 knot predictive process models. Here, departures from the 1: 1 line indicate different ranking. These dispersed scatters suggest the addition of the spatial effect substantially influences breeding preference.

Figure 4.

Comparative ranking of the additive genetic effects for scots pine height (left), diameter (middle), and branch angle (right) from the nonspatial and 160 knot predictive process models. The x-axis plots the nonspatial model estimated position of the 100 highest ranked individuals and the y-axis plots the corresponding ranking from the spatial model. The straight line indicates no discrepancy in rank between the two models.

This analysis supports the idea that ignoring spatial variability in these models can lead to biased estimates of genetic variance parameters and ranking of breeding values, which could result in nonoptimal selection response and lower genetic gain (Magnussen 1993). Analysis of the scots pine data reveals that tree height and diameter are highly genetically correlated, have relatively low heritability, and display similar long distance effective spatial range. In contrast, branch angle seems to be more influenced by genetic components, as indicated by a higher heritability and shorter effective spatial range. Comparison between the nonspatial and spatial predictive process models also shows that the nonspatial model estimates lower heritability for all three traits, although the effect is less pronounced for height. Similar results have been found in other spatial genetic studies of trees. Zas (2006) reported that application of an iterative kriging method increased heritability in Pinus pinaster from a 95% confidence interval of 0.04–0.11 to 0.10–0.17 for diameter, and from 0.13–0.18 to 0.17–0.33 for height growth. In Cappa and Cantet (2007), the heritability of diameter at breast height in Eucalyptus globulus increased from 0.08 to 0.27 when a spatial spline component was included in the model. Moreover, both Zas (2006) and Cappa and Cantet (2007) showed a dramatic increase in accuracy of breeding values for spatial models, a trend that is likely also seen in the scots pine analysis presented here.

6. SUMMARY

While modeling of large spatial datasets have received much attention in the literature, estimation of variance components in reduced-rank kriging models have been explored less. Here we proposed computationally feasible multivariate spatial genetic trait models that robustly estimate variance components in datasets of sizes that normally are out of range because of prohibitive matrix computations. The process-based approach laid out here not only helps us identify a bias in the estimates of variance components that plagues most reduced-rank spline models but also leads to a remedy that neither involves additional computational burden, nor introduces any additional parameters. Future work may include extending predictive process models to space-time dynamic models and exploration of variance components in temporal contexts (see, e.g., Pourahmadi 1999; Ghosh et al. 2009).

Our simulated and field analysis reveals the spatial component to demonstrably improve the model’s ability to recapture correct estimates of fixed and genetic random effects as well as estimates of heritability and dominance ratios. The spBayes package in R (http://cran.r-project.org/web/packages/spBayes/) now include functions for these bias-adjusted multivariate predictive process models. Functions specifically geared for spatial quantitative genetics are being planned.

Finally, selecting the “knots” (Section 3) is an integral part of any reduced-rank spatial methodology. Since our field trials are conducted over a fairly regular configuration of plots, fixed knots on a regular grid provided robust inference. An arguably better strategy is to allow the knots to vary in their positions and/or their number. In particular, the knots can learn from the realizations and while they will change with each MCMC iteration, the number of adaptive knots required to deliver robust inference will likely be far smaller. These will form a part of future explorations.

APPENDIX A: BOUND FOR VARIANCE COMPONENT ESTIMATES

That the ratio ${\widehat{\sigma }}_{w1}^2/{\widehat{\sigma }}_{w2}^2$ is bounded can be seen as follows. Let A = ((φ)′R^*(φ)⁻¹(φ) + γ²I_n)⁻¹ and B = (R(φ) + γ²I_n)⁻¹; thus, both are symmetric and positive-definite matrices. Then, there exists a nonsingular matrix C such that B = C′C. Writing Cu = z, where u = y − Xβ − a − d, we have

$$\underset{\mathbf{u}\ne \mathbf{0}}{max}\frac{{\mathbf{u}}^{\prime }\mathbf{Au}}{{\mathbf{u}}^{\prime }\mathbf{Bu}}=\underset{\mathbf{z}\ne \mathbf{0}}{max}\frac{{\mathbf{z}}^{\prime }{{\mathbf{C}}^{-1}}^{\prime }\mathbf{A}{\mathbf{C}}^{-1}\mathbf{z}}{{\mathbf{z}}^{\prime }\mathbf{z}}={\lambda }_{max}({{\mathbf{C}}^{-1}}^{\prime }\mathbf{A}{\mathbf{C}}^{-1}).$$

Note that the eigenvalues of C^−1′AC⁻¹ are all real and are the same as those of C⁻¹C^−1′A = B⁻¹A (applying Theorem 21.10.1, pp. 545–546, Harville 1997) This shows that ${\widehat{\sigma }}_{w1}^2/{\widehat{\sigma }}_{w2}^2\le {\lambda }_{max}((\mathbf{R}(\mathit{\phi })+{\gamma }^2{I}_n){(\mathcal{R}{(\mathit{\phi })}^{\prime }{\mathbf{R}}^{\ast }{(\mathit{\phi })}^{-1}\mathcal{R}(\mathit{\phi })+{\gamma }^2{I}_n)}^{-1})$.

APPENDIX B: FULL CONDITIONAL DISTRIBUTIONS

The full conditional distributions are listed below:

$$\begin{array}{l}\mathit{\beta }\mid ·\sim \text{MVN}({\mu }_{\beta \mid ·},{\mathbf{\sum }}_{\beta \mid ·}),\text{where}\\ {\mathbf{\sum }}_{\beta \mid ·}={[{\mathbf{\sum }}_{\beta }^{-1}+{\mathbf{X}}^{\prime }(I\otimes {\mathbf{\Psi }}^{-1})\mathbf{X}]}^{-1},\\ {\mu }_{\beta \mid ·}={\mathbf{\sum }}_{\beta \mid ·}[{\mathbf{\sum }}_{\beta }^{-1}{\mathit{\mu }}_{\beta }+{\mathbf{X}}^{\prime }(I_n\otimes {\mathbf{\Psi }}^{-1})(\mathbf{Y}-\mathbf{a}-\mathbf{d}-{\stackrel{\sim }{\mathbf{w}}}_{\stackrel{\sim }{\epsilon }})];\\ \mathbf{a}\mid ·\sim \text{MVN}({\mu }_{a\mid ·},{\mathbf{\sum }}_{a\mid ·}),\text{where}\\ {\mathbf{\sum }}_{a\mid ·}={[{\mathbf{A}}^{-1}\otimes {\mathbf{\sum }}_a^{-1}+I_n\otimes {\mathbf{\Psi }}^{-1}]}^{-1},\text{and}\\ {\mu }_{a\mid ·}={\mathbf{\sum }}_{a\mid ·}[({\mathbf{A}}^{-1}\otimes {\mathbf{\sum }}_a^{-1}({\mathit{\mu }}_a+(I_n\otimes {\mathbf{\Psi }}^{-1})(\mathbf{Y}-\mathbf{X}\mathbf{\beta }-\mathbf{d}-{\stackrel{\sim }{\mathbf{w}}}_{\stackrel{\sim }{\epsilon }})];\text{and}\\ \mathbf{d}\mid ·\sim \text{MVN}({\mu }_{d\mid ·},{\mathbf{\sum }}_{d\mid ·}),\text{where}\\ {\mathbf{\sum }}_{d\mid ·}={[{\mathbf{D}}^{-1}\otimes {\mathbf{\sum }}_d^{-1}+I_n\otimes {\mathbf{\Psi }}^{-1}]}^{-1},\text{and}\\ {\mu }_{d\mid ·}={\mathbf{\sum }}_{d\mid ·}[({\mathbf{D}}^{-1}\otimes {\mathbf{\sum }}_d){\mathit{\mu }}_d+(I_n\otimes {\mathbf{\Psi }}^{-1})(\mathbf{Y}-\mathbf{X}\beta -\mathbf{a}-{\stackrel{\sim }{\mathbf{w}}}_{\stackrel{\sim }{\epsilon }})];\\ {\stackrel{\sim }{\mathbf{w}}}_{\stackrel{\sim }{\epsilon }}\mid ·\sim \text{MVN}({\mathit{\mu }}_{{\stackrel{\sim }{\mathbf{w}}}_{\stackrel{\sim }{\epsilon }}\mid ·},{\mathbf{\sum }}_{{\stackrel{\sim }{\mathbf{w}}}_{\stackrel{\sim }{\epsilon }}\mid ·}),\text{where}\\ {\mathbf{\sum }}_{{\stackrel{\sim }{\mathbf{w}}}_{\stackrel{\sim }{\epsilon }}\mid ·}={[{\mathit{\sum }}_{{\stackrel{\sim }{\mathbf{w}}}_{\stackrel{\sim }{\epsilon }}}^{-1}+(I_n\otimes {\mathbf{\Psi }}^{-1})]}^{-1};\text{and}\\ {\mathit{\mu }}_{{\stackrel{\sim }{\mathbf{w}}}_{\stackrel{\sim }{\epsilon }}\mid ·}={\mathbf{\sum }}_{{\stackrel{\sim }{\mathbf{w}}}_{\stackrel{\sim }{\epsilon }}\mid ·}(I_n\otimes {\mathbf{\Psi }}^{-1})(\mathbf{Y}-\mathbf{a}-\mathbf{d}).\end{array}$$

For the full conditional distribution of the dispersion matrices, we have conjugate Wishart distributions. Assuming that Σ_a ~ IW(r_a, ϒ_a) is the Inverse-Wishart prior, that is,

$$p({\mathbf{\sum }}_a)\propto \frac{1}{det{({\mathbf{\sum }}_a)}^{r_a+(m+1)/2}}exp\left\{-\frac{1}{2}tr({\mathrm{\Upsilon }}_a^{-1}{\mathbf{\sum }}_a^{-1})\right\},$$

and Σ_d ~ IW(r_d, ϒ_d), we obtain the full conditionals,

$$\begin{array}{l}{\mathbf{\sum }}_a\mid ·\sim \text{IW}\left(r_a+n,{\left[{\mathrm{\Upsilon }}_a^{-1}+\sum _{i=1}^n\sum _{j=1}^n{\stackrel{\sim }{a}}^{ij}{\mathbf{a}}_j{\mathbf{a}}_i^{\prime }\right]}^{-1}\right),\\ {\mathbf{\sum }}_d\mid ·\sim \text{IW}\left(r_d+n,{\left[{\mathrm{\Upsilon }}_d^{-1}+\sum _{i=1}^n\sum _{j=1}^n{\stackrel{\sim }{d}}^{ij}{\mathbf{d}}_j{\mathbf{d}}_i^{\prime }\right]}^{-1}\right),\end{array}$$

where A⁻¹ = {ã^ij} and D⁻¹ = {d̃^ij}. Finally, with $\mathbf{\Psi }=\text{diag}{\{{\tau }_j^2\}}_{j=1}^m$, with each ${\tau }_j^2\sim \text{IG}(r_{\epsilon j},s_{\epsilon j})$, with mean r_εj/s_εj, we obtain the full conditionals

$${\tau }_j^2\mid ·\text{IG}\left(r_{\epsilon j}+\frac{n}{2},s_{\epsilon j}+\frac{1}{2}\sum _{i=1}^n{(y_j({\mathbf{s}}_i)-{\mathbf{x}}_j{({\mathbf{s}}_i)}^{\prime }\mathit{\beta }-a_{ij}-d_{ij}-{\stackrel{\sim }{w}}_{\stackrel{\sim }{\epsilon }j}({\mathbf{s}}_i))}^2\right)$$

for j = 1, …, m, where a_ij, d_ij, and w̃_ε̃j(s_i) are the jth elements of a_i, d_i, and w̃_ε̃(s_i) respectively.

APPENDIX C: DERIVATION OF (8)

In the above expressions, we can simplify ${\mathbf{\sum }}_{a\mid ·}={[{\mathbf{A}}^{-1}\otimes {\mathbf{\sum }}_a^{-1}+I_n\otimes {\mathbf{\Psi }}^{-1}]}^{-1}$ as

$$\begin{array}{l}{\mathbf{\sum }}_{a\mid ·}={[{\mathbf{A}}^{-1}\otimes {\mathbf{\sum }}_a^{-1}+I_n\otimes {\mathbf{\Psi }}^{-1}]}^{-1}\\ ={[({\mathbf{P}}_{\mathbf{A}}\otimes {\mathbf{P}}_{{\mathbf{\sum }}_a})({\mathrm{\Lambda }}_{\mathbf{A}}^{-1}\otimes {\mathrm{\Lambda }}_{{\mathbf{\sum }}_a}^{-1})({\mathbf{P}}_{\mathbf{A}}^{\prime }\otimes {\mathbf{P}}_{{\mathbf{\sum }}_a}^{\prime })+I_n\otimes {\mathbf{\Psi }}^{-1}]}^{-1}\\ =({\mathbf{P}}_{\mathbf{A}}\otimes {\mathbf{P}}_{{\mathbf{\sum }}_a}){[{\mathrm{\Lambda }}_{\mathbf{A}}^{-1}\otimes {\mathrm{\Lambda }}_{{\mathbf{\sum }}_a}^{-1}+I_n\otimes ({\mathbf{P}}_{{\mathbf{\sum }}_a}^{\prime }{\mathbf{\Psi }}^{-1}{\mathbf{P}}_{{\mathbf{\sum }}_a})]}^{-1}\times ({\mathbf{P}}_{\mathbf{A}}^{\prime }\otimes {\mathbf{P}}_{{\mathbf{\sum }}_a}^{\prime })\\ =({\mathbf{P}}_{\mathbf{A}}\otimes {\mathbf{P}}_{{\mathbf{\sum }}_a})(I_n\otimes {\mathbf{P}}_{{\mathbf{\sum }}_a}^{\prime })\left[\underset{i=1}{\overset{n}{\oplus }}{\left(\frac{1}{{\lambda }_{\mathbf{A}}^{(i)}}{\mathbf{\sum }}_a^{-1}+{\mathbf{\Psi }}^{-1}\right)}^{-1}\right]\times (I_n\otimes {\mathbf{P}}_{{\mathbf{\sum }}_a})({\mathbf{P}}_{\mathbf{A}}^{\prime }\otimes {\mathbf{P}}_{{\mathbf{\sum }}_a}^{\prime })\\ =({\mathbf{P}}_{\mathbf{A}}\otimes I_m)\left[\underset{i=1}{\overset{n}{\oplus }}{\left(\frac{1}{{\lambda }_{\mathbf{A}}^{(i)}}{\mathbf{\sum }}_a^{-1}+{\mathbf{\Psi }}^{-1}\right)}^{-1}\right]({\mathbf{P}}_{\mathbf{A}}^{\prime }\otimes I_m),\end{array}$$

where ${\lambda }_{\mathbf{A}}^{(i)}$ is the ith diagonal element of Λ_A. Note that P_A is the only n × n matrix here but it needs to be computed only once. Instead of having to invert a big mn × mn matrix, we now require n inversions of m × m matrices. Note that both Σ_a and Ψ will need to be updated in each iteration of the Gibbs.