An Analysis of Polynomial Chaos Approximations for Modeling Single-Fluid-Phase Flow in Porous Medium Systems

Abstract

We examine a variety of polynomial-chaos-motivated approximations to a stochastic form of a steady state groundwater flow model. We consider approaches for truncating the infinite dimensional problem and producing decoupled systems. We discuss conditions under which such decoupling is possible and show that to generalize the known decoupling by numerical cubature, it would be necessary to find new multivariate cubature rules. Finally, we use the acceleration of Monte Carlo to compare the quality of polynomial models obtained for all approaches and find that in general the methods considered are more efficient than Monte Carlo for the relatively small domains considered in this work. A curse of dimensionality in the series expansion of the log-normal stochastic random field used to represent hydraulic conductivity provides a significant impediment to efficient approximations for large domains for all methods considered in this work, other than the Monte Carlo method.

1 Introduction

Analysis of groundwater flow in naturally occurring water-saturated porous media is influenced by significant variations in medium properties and source and boundary conditions over relevant length scales. When this variability cannot be characterized deterministically using available means, stochastic analysis of single-fluid-phase (hereafter simply single-phase) flow through porous media is commonly performed [cf. ^{28, 82}]. The objective of such an analysis is to characterize the uncertainty of quantities such as the hydraulic head or velocity as a function of uncertain parameters and conditions in the model. This uncertainty is typically described by low-order moments of the quantities of interest.

While the stochastic nature of subsurface systems has been recognized for more than a quarter of a century [69], the quest continues for development of accurate and efficient methods of stochastic analysis that effectively reduce such analysis to routine practice. An ideal, but perhaps unattainable, solution to the problem would provide expressions functionally incorporating stochastic parameters, associated with the conductivity field and boundary conditions, from which desired statistical properties could be computed. This goal has not been attained, except in special cases, and current practice often aims for somewhat less, such as approximate values of the first few statistical moments. Calculation of such statistics involves evaluation of certain integrals in probability space, but hydrological problems potentially involve many degrees of freedom, and the density functions are often unknown.

Naïve Monte Carlo (MC) methods [e.g. ⁴⁰] were the first to be used and are still often chosen, being easily implemented adaptively and having the advantage of generality but converging sublinearly at a rate independent of the stochastic dimension. Certain pitfalls must be avoided when using MC methods: for example, the pseudorandom variates used for simulation can fail in important ways [53]. These methods are not easily conditionalized [61], and their slow convergence has led to interest in acceleration techniques [13, 30]. Transformation of the underlying probability space using importance sampling was suggested in [49], using a spatially constant random conductivity field with one-dimensional flow, but to the best of our knowledge it has not been considered for nontrivial stochastic flow problems.

One class of standard acceleration techniques involves preferential sampling of the underlying probability space, as in the Latin hypercube method [70]; one might consider the direct attempt to calculate statistical moments from simulations by a numerical cubature as a limiting case of such methods, which has led to interest in sparse cubature schemes, as in Xiu and Hesthaven [78], for example. Chorin [13] has suggested a variance-reduction use of Hermite series to accelerate MC. Other alternatives include spectral methods [46], as well as perturbation series methods and moment equations [66, 82, 83].

Polynomial chaos (PC) methods [25] for approximating solutions of stochastic equations have received recent attention in the water resources [22, 23], engineering [68], and mathematics literature [5]. These methods approximate both stochastic model inputs and output using series expansions to replace stochastic equations by deterministic systems that are then finitely truncated and solved discretely. Variations of the method have been used to study diffusion [3, 45] and fluid dynamics [47, 51]. There is also a growing literature on applications of PC methods to steady state flow in porous media and related elliptic equations [4, 5, 22, 41, 64, 77, 79, 80].

As opposed to MC methods, which require many domain-sized solves, PC methods typically produce large coupled systems. In special cases, it is possible to decouple these large systems into domain-sized problems, restoring one of the benefits of the MC approach. One technique accomplishing this decoupling is the so-called method of double orthogonal variables (DOV), solving a generalized eigenvalue problem to decompose the system. Another technique decomposes an approximation to the system by computing certain expectations numerically. As opposed to MC, none of these methods are adaptive: one decides in advance the approximation to be used.

Applying the PC approach becomes problematic as the number of independent modes of variability of the problem increases: if many variables are required to model the stochastic inputs, the resulting systems can become quite large. The suggestion of Roy and Grilli [66], that moment methods might sometimes be more practical in such cases, led to a multivariate moment method [83]; similar to MC, this method requires only domain-sized solves and can be run until a desired tolerance is reached, at the cost of storing some prior results for reuse.

It is at present unclear which statistical approximation methods, if any, are superior. Some methods, for example, easily produce approximate solutions to the stochastic equation that are polynomial in the random input variables. There are also questions of applicability in complex environments and convenience of use. Method-specific limitations arise, such as the restriction to low-variability systems required by moment methods [16, 67]. A detailed comparison of statistical approximation methods is currently lacking from the literature, and it is not clear if and when such approaches are preferable computationally to MC methods.

The overall goal of this work is to investigate and compare methods to approximate low-order stochastic moments of the single-phase groundwater flow equation. The specific objectives of this work are: (1) to formulate a PC approximation of the groundwater flow equation; (2) to summarize a set of methods that result in decoupled approximations; (3) to establish conditions under which such decoupling is possible; (4) to formulate a MC acceleration and error analysis approach based upon the derived stochastic polynomial models, (5) to compare the approximation methods for computational efficiency; and (6) to assess the limitations and areas of potentially fruitful future study for these methods.

2 Background

As described in Ghanem and Spanos [25], PC methods involve expanding random functions of independent variates in orthogonal polynomials, based on work of Wiener [76] and Cameron and Martin [11], truncating the series and projecting to replace the stochastic systems by a deterministic system. Solution of the deterministic systems then produces approximate polynomial solutions to the original stochastic equation. Using properties of the orthogonal polynomials, the integrals needed in moment calculations can be obtained directly. Related ideas have been used to construct polynomial solutions to study climate [74], to model dose and exposure [36], and to simulate watersheds [34]. PC methods typically produce large coupled systems, which can be tedious to derive and can require special solution techniques.

PC methods seem attractive because the underlying polynomial expansions can converge quickly [5, 21]. These methods can also be used to produce easily conditionalized polynomial models [23]. But when many independent random variables are required to model the stochastic inputs, it may be necessary to ignore some random variates in order to model the others to sufficiently high degree, and the effects of this trade-off are not immediately clear. The advantages might therefore be illusory when the number of degrees of freedom is large. Pellissetti and Ghanem [62] noted that for stochastic systems satisfying a certain linearity condition, the PC approach involves matrices with a sparse block structure and suggested avoiding full matrix assembly. Such systems are obtainable for the groundwater flow equation with lognormal conductivity by rewriting the equation appropriately. For some truncations of such systems, the DOV method, attributed in Keese [38] to Werder et al. [75], will reduce the problem to a number of uncoupled domain-sized problems. This decoupling is accomplished by solving a generalized eigenvalue problem. Using this decoupling, Frauenfelder et al. [21] studied an elliptic equation, with conductivity linear in each of the underlying random variates. The DOV method has also been applied to a linear transport model [43].

Relations between numerical quadrature and pseudospectral methods being well known [9, 73], several authors naturally investigated collocation techniques related to PC systems as an alternative to projection methods [40, 56, 57, 68]. Numerical cubature has been applied to simplify computation of expectations in PC expansions [56] and as a method for estimating PC coefficients without considering the coupled systems that result from projection [1, 36, 74]. Babuska et al. [6] studied the groundwater flow equation with a multilinear conductivity, noted that the eigenvalues obtained by DOV are multivariate Gauss cubature knots, remarked that the DOV decoupling strategy works in the multilinear case, and noted the equivalence of DOV decoupling and numerical cubature in the stochastic dimensions (also called collocation) in the case of a multilinear source and homogeneous Dirichlet boundary. Collocation based on knots associated with sparse grid cubatures has also been studied [38, 78]. It has similarly been remarked that PC systems can be decoupled by certain cubature formulæ: for example, Sandu et al. [68] have remarked that a cubature formula respecting the orthogonality of the polynomials will decouple PC systems associated with certain mechanical problems.

PC methods have been applied to flow and transport problems [22, 23]. Babuska et al. [5] provided a theoretical discussion of the relative merits of PC expansion and MC methods, concentrating on cases for which DOV decoupling is available when the underlying variates are independent and not examining more general truncations. Lognormal conductivities seldom appear in PC investigations of the elliptic steady groundwater flow equation, but Babuska et al. [6] established that lognormal conductivities in the flow equation leads to well-posed problems. Because the coupled systems are large, the question raised by Roy and Grilli [66] remains: is the method really useful for ground-water flow equations? Related nonspectral moment-method techniques [50] have been examined as an alternative.

For hydrological investigations, the fields of interest are likely to involve more modes of variation than can be modeled in full detail, and the effects of neglecting some variability are therefore important. In this context, the relative accuracy of PC and related methods, and the effects of domain size and field variability on the accuracy of the methods, deserves further investigation.

3 Polynomial Chaos and Projection

The standard PC approach to approximate the solution of stochastic differential equations involves: expansion of the uncertain input parameters and unknown dependent variables in series of orthogonal polynomials, computation of certain expectations to produce a system of deterministic equations, and solution of the deterministic system.

Consider the stochastic steady state porous medium flow equation

$$\nabla \cdot K\nabla H=-\theta$$ (1)

with first- and second-kind boundary conditions given by

$$H{\mid }_{{\partial }_D}=H^D$$ (2)

$${\partial }_{n}H{\mid }_{{\partial }_N}=-K^{-1}{Q}^N$$ (3)

where ∂_D and ∂_N respectively refer to complementary portions of the Dirichlet and Neumann boundary. The conductivity K ≡ K(x, ω), source θ ≡ θ (x, ω), head H ≡ H(x, ω), and boundary values H^D(x, ω) and Q^N(x, ω)/K(x, ω) here are all random fields, functions of spatial position x and of a certain finite set of random variates ω that have known joint probability density ρ ≡ ρ(ω). Throughout this article, ω always denotes one of the random variates in ω.

The head, source, and boundary terms are expanded as infinite series

$$H(\mathbf{x},\mathit{\omega })={\sum }_{\alpha }{H}_{\alpha }(\mathbf{x})h_{\alpha }(\mathit{\omega })$$ (4)

$$\theta (\mathbf{x},\mathit{\omega })={\sum }_{\alpha }{\theta }_{\alpha }(\mathbf{x})h_{\alpha }(\mathit{\omega })$$ (5)

$$H{\mid }_{{\partial }_D}={\sum }_{\alpha }{H}_{\alpha }^D(\mathbf{x})h_{\alpha }(\mathit{\omega })$$ (6)

$${\partial }_{n}H{\mid }_{{\partial }_N}=-{\sum }_{\alpha }{K}^{-1}(\mathbf{x})Q_{\alpha }^N(\mathbf{x})h_{\alpha }(\mathit{\omega })$$ (7)

involving a family of multivariate polynomials h_α(ω) orthonormal with respect to ρ, as well as purely spatial functions H_α, θ_α, $H_{\alpha }^D$, and $Q_{\alpha }^N$. To avoid negative diffusion coefficients, that might result from truncation, K is not similarly expanded [cf. ^{4, 64}].

We substitute the expansions given by Eqs. (4) - (7) into Eqs. (1) - (3), multiply each of the resulting equations by various h_β, and compute expectations relative to ρ. Thus from Eq. (1), for example, we obtain

$$E[h_{\beta }\nabla \cdot K\nabla H]=-E[h_{\beta }\theta ]$$ (8)

$${\sum }_{\alpha }\nabla \cdot G_{\beta \alpha }\nabla H_{\alpha }=-{\theta }_{\beta }$$ (9)

where

$$G_{\beta \alpha }\equiv E[K{h}_{\beta }{h}_{\alpha }]=G_{\alpha \beta }$$ (10)

Similarly, Eqs. (2) and (3) produce

$$H_{\beta }{\mid }_{{\partial }_D}=H_{\beta }^D$$ (11)

$${\partial }_n{H}_{\beta }{\mid }_{{\partial }_N}=-{\sum }_{\alpha }{G}_{\beta \alpha }^{-1}{Q}_{\alpha }^N$$ (12)

where

$$G_{\beta \alpha }^{-1}\equiv E[K^{-1}{h}_{\beta }{h}_{\alpha }]$$ (13)

Similarly, rewriting Eq. (1) in the form

$$\nabla \ln \hspace{0.17em}(K)\cdot \nabla H+{\nabla }^{2}H=-\theta /K$$ (14)

multiplying by h_ϕ and computing expectations yields

$${\sum }_{\alpha }\nabla E[\ln (K)h_{\alpha }{h}_{\phi }]\cdot \nabla H_{\alpha }+{\nabla }^2{H}_{\phi }=-{\sum }_{\alpha }{\theta }_{\alpha }{G}_{\alpha \phi }^{-1}$$ (15)

To proceed further, it is necessary to specify the field K as a function of the variables ω and their joint density ρ; specify the polynomials h_α; and compute explicitly certain integrals such as G_βα. It is further necessary to choose a finite truncation of the infinite series in Eqs. (9) or (15) and (12).

3.1 Further Specification

A natural interpretation, of the common view that the hydraulic conductivity K(x, ω) is lognormal, is that the log-conductivity κ ≡ ln(K) is a Gaussian field. Whether conductivities can reasonably be assumed lognormal is a regular topic of investigation: the distribution in a sandstone formation, for example, varied with the part sampled [18], while a soil investigation concluded the lognormal distribution fit the data better than a gamma distribution [59]. Although the validity of using Gaussian fields to describe log-conductivities remains open to question [27], we adopt the hypothesis here because it is convenient for calculation and has commonly been assumed [82]. We thus assume a finite expansion for the log-conductivity of the form

$$\kappa (\mathbf{x},\mathit{\omega })\equiv \overline{\kappa }(\mathbf{x})+{\sum }_{\omega }{\kappa }_{\omega }(\mathbf{x})\omega$$ (16)

involving the mean field κ̄(x) and certain auxiliary fields κ_ω(x) indexed by the random variates ω in the set ω. Expansions, or approximate expansions, of the form (16) can be obtained in various ways, but there is some advantage to using a finitely truncated Karhunen-Loève expansion (KLE). The KLE is determined by the mean κ̄(x) and covariance function c_κ (x, x′) of κ. The KLE terms and variates correspond to certain eigenpairs of an integral operator, whose eigenvalues can be arranged in decreasing order λ_ω1 ≥ λ_ω2 ≥ λ_ω3 ⋯; it is natural to say that the variate ω is “more important” than the variate ω′ if the corresponding eigenvalues satisfy λ_ω > λ_ω′. Series obtained by KLE have optimal mean-square convergence in the following sense: no sum of n terms from any other series expansion of the random field can produce a more accurate approximation of κ than the sum of the first n KLE terms [25]. This optimal convergence property explains the common use of KLE in PC analyses, because it minimizes the number of variables required for a given accuracy and hence limits system size. See Appendix A for details and references.

The assumption that κ is a Gaussian field leads to uncorrelated (hence independent) standard normal variates ω ~ N(0, 1). The normalized univariate probabilists’ Hermite polynomials (univariate h-polynomials) are defined by the three-term relation

$$h_{n+1}(\omega )=[\omega h_n(\omega )-\sqrt{n}{h}_{n-1}(\omega )]/\sqrt{n+1}$$ (17)

with h₀(ω) ≡ 1 and h₁(ω) ≡ ω. These h-polynomials are orthonormal with respect to the standard univariate normal density

$$\rho (\omega )\equiv exp(-{\omega }^2/2)/\sqrt{2\pi }$$ (18)

Since ω is assumed finite, the product density

$$\rho (\mathit{\omega })\equiv {\prod }_{\omega }\rho (\omega )$$ (19)

makes sense, and an orthonormal family of multivariate polynomials for ρ(ω) is obtained by taking products of the univariate h-polynomials.

The remainder of this subsection involves some largely notational issues. To help fix ideas, the reader may consult Appendix B, which contains some sample calculations. It is convenient to forget the random character of the variates ω, when they are used for indexing purposes, regarding them merely as formal variables appearing in certain polynomials.

There is a natural indexing of the multivariate h-polynomials: each monomial α, in the variates ω, determines a vector of integer exponents 〈a_ω〉_ω∊ω defined by

$$\alpha \equiv {\prod }_{\omega }{\omega }^{a_{\omega }}$$ (20)

and we define the corresponding multivariate h-polynomial by

$$h_{\alpha }(\mathit{\omega })\equiv {\prod }_{\omega }{h}_{a_{\omega }}(\omega )$$ (21)

so α is the pure monomial part of the unique highest degree term in h_α. The natural indexing is a notational convenience: for coding purposes, it is useful to maintain a table of the various exponent vectors ⟨a_ω⟩_ω∊ω.

For the univariate h-polynomials, the triple product expectations are

$$X_{bc}^a\equiv E[h_a(\omega )h_b(\omega )h_c(\omega )]$$ (22)

and we define infinite matrices X⁰, X¹, X², ⋯ by

$$X^a\equiv {(X_{bc}^a)}_{0\le b<\infty ,0\le c<\infty }$$ (23)

Appendix C contains explicit expressions for the matrix entries.

Similarly for the multivariate h-polynomials, the triple product expectations

$$X_{\beta \gamma }^{\alpha }\equiv E[h_{\alpha }{h}_{\beta }{h}_{\gamma }]={\prod }_{\omega }{X}_{b_{\omega }{c}_{\omega }}^{a_{\omega }}$$ (24)

define, for each monomial α in the variates ω, an infinite (but row and column finite) matrix

$$X^{\alpha }\equiv (X_{\beta \gamma }^{\alpha })$$ (25)

with entries indexed by the various pairs of monomials in those variates. The matrices X^α describe the multiplication of the h-polynomials:

$$h_{\beta }{h}_{\gamma }={\sum }_{\alpha }{X}_{\beta \gamma }^{\alpha }{h}_{\alpha }$$ (26)

Since each variate ω is a monomial, the definition (24) makes sense for α ≡ ω, and the resulting matrices X^ω provide multivariate analogs of the univariate Jacobi matrix encoding the three-term relation, that in the multivariate case becomes

$$\omega h_{\alpha }=\sqrt{a_{\omega }+1}{h}_{\alpha \omega }+\sqrt{a_{\omega }}{h}_{\alpha /\omega }$$ (27)

In Eq. (27), αω and α/ω respectively denote the results of multiplying or dividing the monomial α by the variate ω, with the convention that the second term is dropped if the variate ω does not actually occur in α.

Because the matrix entries $X_{\beta \gamma }^{\alpha }$ can be computed exactly, it is possible to obtain exact expressions for the entries of the matrices G and G^–1; see Appendix C. The quantities $X_{\beta \gamma }^{\alpha }$ are also required for the computation of higher order statistics, although the mean and covariance function can be expressed easily:

$$\overline{H}(\mathbf{x})=H_1(\mathbf{x})$$ (28)

$$c_H(\mathbf{x},{\mathbf{x}}^{\prime })={\sum }_{\alpha \ne 1}{H}_{\alpha }(\mathbf{x})H_{\alpha }({\mathbf{x}}^{\prime })$$ (29)

Here, α ≠ 1 indicates that the summation extends over multivariate polynomials of positive degree.

3.2 Truncation

For numerical purposes, the infinite series in Eqs. (9) or (15) and (12) must be truncated. We write β≺ α to indicate that the monomial β divides the monomial α or (in other words) b_ω ≤ a_ω for each variate ω. By a truncation set τ of the system, we mean a finite set of monomials, containing the constant unit monomial, and with the following property: whenever α belongs to τ and β ≺ α, then β also belongs to τ. Throughout this article, τ always denotes a truncation set.

The set of variates ω∩τ actually appearing in τ may be a proper subset of ω. So we further impose the following condition: whenever a variate ω belongs to τ, so does every variate “more important” than ω, in the sense of §3.1.

We note, in particular, two special classes of a truncation set. By a total degree truncation set τ = τ(e, d), we mean a truncation set obtained by choosing e of the most important variates and then collecting the monomials α in those variates with total degree ∣α∣ ≡ Σ_ωa_ω ≤ d. Such a truncation set has size

$$N(e,d)\equiv \left(\begin{array}{c}e+d\\ d\end{array}\right)$$ (30)

and is related to a standard orthogonal decomposition of a Gaussian Hilbert space [17, 37]. By a rectangular truncation set τ = τ(μ), we mean a truncation set consisting of all α ≺ μ for some fixed monomial μ = ∏_ωω^m_ω . For such a truncation set, we also use the convenient notation of Frauenfelder et al. [21], which describes a rectangular truncation τ(μ) in terms of the degree-vector of μ

$$\tau (\mu )=[m_{{\omega }_1},m_{{\omega }_2},\cdots ,m_{{\omega }_e}]$$ (31)

with indices ordered by decreasing importance of the variates; the size is ∏_ω(1 + m_ω).

Each truncation set τ naturally induces truncations of matrices and summations, by restricting the indices to τ. We indicate a truncated summation by Σ_α∈τϕ_α; in the case of a rectangular truncation set, τ ≡ τ(μ), we write Σ_α≺μϕ_α. Given a random function Y (x, ω) with PC expansion

$$Y(\mathbf{x},\mathit{\omega })={\sum }_{\alpha }{Y}_{\alpha }(\mathbf{x})h_{\alpha }(\mathit{\omega })$$ (32)

we define the truncated summation corresponding to τ by

$${\stackrel{⌣}{Y}}^{\tau }(\mathbf{x},\mathit{\omega })={\sum }_{\alpha \in \tau }{Y}_{\alpha }(\mathbf{x})h_{\alpha }(\mathit{\omega })$$ (33)

In particular, this has the effect of zeroing the variates not in τ. We similarly indicate by X˘^γ the finite matrix with entries $X_{\alpha \beta }^{\gamma }$ with indices α and β (but not γ) restricted to lie in τ, which will always be clear from the context and hence is not reflected in the notation.

3.3 Number of Variates

Solving the truncated system, consisting of (9) or (15) and (11)–(13), produces a polynomial model from the truncated form of Eq. (4), and the statistics may be obtained as described in §3.1.

The truncation set, however, must be chosen so the truncated system is not too large: the infinite series in Eq. (4) is expected to contain terms of arbitrarily high degree, which are suppressed by truncation, and in general ω ∩ τ will be a proper subset of ω and some of the true variability of K will be lost.

If the conductivity K has a covariance functions decaying with separation, the field values become increasingly decoupled at distance; the field therefore exhibits more degrees of freedom on larger domains. Arranging the eigenvalues from a KLE in decreasing order λ_ω1 ≥ λ_ω2 ≥ λ_ω3 ≥ ⋯, one may ask how quickly λ_ωn decays as a function of n. Frauenfelder et al. [21] obtained estimates for the decay of eigenvalues, showing the decay rate decreases with increasing spatial dimension, and that the decay is faster for analytic covariance functions than for the nonanalytic ones. Some intuition is provided by the one-dimensional case: Huang et al. [35] have shown that the eigenfunctions approach sinusoids as the domain size increases and the expansion approaches a spectral representation; thus, dropping eigenpairs with small eigenvalues from the KLE corresponds to a primitive upscaling of the conductivity field, by ignoring the higher frequency modes of variation of κ.

For our simulations, we adopted a Gaussian covariance that leads, on a two-dimensional rectangular domain, to Kronecker product matrices and to rapid eigenpair decay; the required expansion for the field κ was thus obtained as a sum of products of the discretized one-dimensional eigenfunctions for the separated components of the covariance function. Table I illustrates that many eigenpairs may be required to obtain a random field that approximates the target field well by KLE. This table shows the L₂ error of the approximate covariance function on a unit square obtained from a truncated KLE as a function of domain size in correlation lengths and number of retained variates. The data were obtained by computing the eigenfunctions Ψ_ω of an isotropic Gaussian covariance

Table I.

Approximation of target covariance

Domain Size	Number of Eigenpairs	∥ε∥2/∣Ω∣
1 × 1	5	1.389 × 10−2
—	10	3.809 × 10−4
—	15	2.208 × 10−5
—	20	1.268 × 10−6
2 × 2	5	8.090 × 10−2
—	10	1.885 × 10−2
—	15	4.415 × 10−3
—	20	6.345 × 10−4
3 × 3	5	1.267 × 10−1
—	10	6.366 × 10−2
—	15	1.911 × 10−2
—	20	9.068 × 10−3
4 × 4	5	1.315 × 10−1
—	10	9.237 × 10−2
—	15	4.518 × 10−2
—	20	2.920 × 10−2

$$c_{\mathrm{\Psi }}(\mathbf{x},{\mathbf{x}}^{\prime })\equiv {\sigma }^{2}exp(-{\parallel \mathbf{x}-{\mathbf{x}}^{\prime }\parallel }^2/{\varrho }^2)$$ (34)

with correlation length ϱ on a unit square Ω and computing

$$\epsilon \equiv \parallel c_{\mathrm{\Psi }}\left(\mathbf{x},{\mathbf{x}}_{\mathbf{0}}\right)-{\stackrel{\sim }{c}}_{\mathrm{\Psi }}\left(\mathbf{x},{\mathbf{x}}_0\right)\parallel \mathrm{\Omega }$$ (35)

where x₀ is a corner point of Ω, the norm is defined by spatial integration

$${\parallel \mathrm{\Phi }(\mathbf{x})\parallel }_{\mathrm{\Omega }}^2\equiv {\int }_{\mathrm{\Omega }}{\mathrm{\Phi }}^2(\mathbf{x})d\mathbf{x}$$ (36)

and

$${\stackrel{\sim }{c}}_{\mathrm{\Psi }}(\mathbf{x},{\mathbf{x}}^{\prime })={\sum }_{j=1}^n{\mathrm{\Psi }}_{{\omega }_j}(\mathbf{x}){\mathrm{\Psi }}_{{\omega }_j}({\mathbf{x}}^{\prime })$$ (37)

is an estimate for c_ψ, based on Eq. (A.4) but using only the eigenpairs corresponding to the largest eigenvalues.

In general, the total number of variates required is expected to exceed the number required to reproduce κ accurately, since not all variability in the source term and boundary conditions will arise from modes of fluctuation of the log-conductivity field.

4 Decoupling the Polynomial Chaos System

The size of the coupled systems, produced by projection, grows quickly with the number of normal variates ω and the degrees of the retained h-polynomials h_α, which limits the feasibility of the PC methods. In certain cases, however, the large system given by Eq. (15) can be decoupled: that is, it is possible to solve the large system by solving a number of independent domain-sized problems. Using Eq. (16) to expand ln(K), together with the definition (24) produces

$$\nabla \overline{\kappa }\cdot \nabla H_{\phi }+{\sum }_{\alpha }\nabla ({\sum }_{\omega }{X}_{\alpha \phi }^{\omega }{\kappa }_{\omega })\cdot \nabla H_{\alpha }+{\nabla }^2{H}_{\phi }=-{\sum }_{\alpha }{\theta }_{\alpha }{G}_{\alpha \phi }^{-1}$$ (38)

For each α and ϕ , the summation ${\sum }_{\omega }{X}_{\alpha \phi }^{\omega }{\kappa }_{\omega }$ contains at most one nonzero term. After truncating the system in Eq. (38), it is natural to attempt to decouple the system by simultaneous diagonalization of the truncated matrices X̆^ω. As shown in Appendix D, such a simultaneous diagonalization is possible only for rectangular truncation sets τ = τ(μ). In this case, the diagonalizing matrix U^τ has entries

$$U_{\alpha \zeta }^{\tau }\equiv h_{\alpha }(\zeta )C_{\zeta }$$ (39)

where the ζ are the knots, for the cubature given by the tensor-product of the corresponding univariate Hermite-Gauss (HG) quadratures, while the squares of the C_ζ are the corresponding weights W_ζ . Further, the eigenvalue of X^ω, corresponding to column ζ of U^τ, is simply the ω-component z_ω of the tensor-product cubature knot ζ. Note that retaining only the τ(μ) terms in the PC series for a random function Y and setting each ω = z_ω yields

$${\stackrel{⌣}{Y}}^{\tau }(\mathbf{x},\zeta )=C_{\zeta }^{-1}{\sum }_{\alpha \prec \mu }{Y}_{\alpha }(\mathbf{x})U_{\alpha \zeta }^{\tau }$$ (40)

In the sections that follow, We discuss two related decoupling strategies for rectangular truncation sets: the method of DOV and the collocation by cubature.

4.1 Double Orthogonal Variables

With a rectangular truncation set τ = τ(μ) of the variables, the matrices X̆^ω are simultaneously diagonalized by U^τ:

$${\stackrel{⌣}{X}}_{\alpha \phi }^{\omega }{U}_{\phi \zeta }^{\tau }=U_{\alpha \zeta }^{\tau }{z}_{\omega }$$ (41)

We write

$${\hat{H}}_{\zeta }\equiv {\sum }_{\phi \prec \mu }{H}_{\phi }{U}_{\phi \zeta }^{\tau }\equiv C_{\zeta }{\stackrel{⌣}{H}}^{\tau },$$ (42)

and denote by κ̆^τ(x, ζ) the result of setting ω = z_ω for each ω in τ and zeroing the remaining variates in the expansion of κ. Applying the matrix U^τ to Eq. (38) yields

$$\begin{array}{c}\nabla \overline{\kappa }\cdot \nabla {\hat{H}}_{\zeta }+{\sum }_{\omega \in \tau }{\sum }_{\alpha \prec \mu }{\sum }_{\phi \prec \mu }{\stackrel{⌣}{X}}_{\alpha \phi }^{\omega }{U}_{\phi \zeta }^{\tau }\nabla {\kappa }_{\omega }\cdot \nabla H_{\alpha }\\ \phantom{\nabla \overline{\kappa }\cdot }+{\nabla }^2{\sum }_{\phi \prec \mu }{H}_{\phi }{U}_{\phi \zeta }^{\tau }=-{\sum }_{\alpha \prec \mu }{\sum }_{\phi \prec \mu }{\theta }_{\alpha }{G}_{\alpha \phi }^{-1}{U}_{\phi \zeta }^{\tau }\end{array}$$ (43)

Then

$$\nabla {\stackrel{⌣}{\kappa }}^{\tau }(\mathbf{x},\zeta )\cdot \nabla {\hat{H}}_{\zeta }+{\nabla }^2{\hat{H}}_{\zeta }=-{\sum }_{\alpha \prec \mu }{\sum }_{\phi \prec \mu }{\theta }_{\alpha }{G}_{\alpha \phi }^{-1}{U}_{\phi \zeta }^{\tau }$$ (44)

$$\nabla \cdot exp[{\stackrel{⌣}{\kappa }}^{\tau }(\mathbf{x},\zeta )]\nabla {\hat{H}}_{\zeta }=-exp[{\stackrel{⌣}{\kappa }}^{\tau }(\mathbf{x},\zeta )]{\sum }_{\alpha \prec \mu }{\sum }_{\phi \mu }{\theta }_{\alpha }{G}_{\alpha \phi }^{-1}{U}_{\phi \zeta }^{\tau }$$ (45)

$$\begin{array}{l}\phantom{=-\exp \hspace{0.17em}{\stackrel{⌣}{\kappa }}^{\tau }(\text{x},\zeta )}\nabla \cdot \exp \hspace{0.17em}[{\stackrel{⌣}{\kappa }}^{\tau }(\text{x},\zeta )]\nabla {\stackrel{⌣}{H}}^{\tau }(\text{x},\zeta )\\ =-\exp \hspace{0.17em}[{\stackrel{⌣}{\kappa }}^{\tau }(\text{x},\zeta )]C_{\zeta }^{-1}{\sum }_{\alpha \prec \mu }{\sum }_{\phi \prec \mu }{\theta }_{\alpha }{G}_{\alpha \phi }^{-1}{U}_{\phi \zeta }^{\tau }\end{array}$$ (46)

Boundary conditions are handled similarly, leading to

$${\stackrel{⌣}{H}}^{\tau }(\mathbf{x},\zeta ){\mid }_{{\partial }_D}=C_{\zeta }^{-1}{\sum }_{\phi \prec \mu }{H}_{\phi }^D{U}_{\phi \zeta }^{\tau }={\stackrel{⌣}{H}}^{\tau D}(\mathbf{x},\zeta )$$ (47)

$${\partial }_n{\stackrel{⌣}{H}}^{\tau }(\mathbf{x},\zeta ){\mid }_{{\partial }_N}=C_{\zeta }^{-1}{\sum }_{\alpha \prec \mu }{\sum }_{\phi \prec \mu }{Q}_{\alpha }^N{G}_{\alpha \phi }^{-1}{U}_{\phi \zeta }^{\tau }$$ (48)

This leads to the following polynomial model:

$$H(\mathbf{x},\mathit{\omega })\approx {\sum }_{\zeta }{\sum }_{\phi \prec \mu }{\hat{H}}_{\zeta }\hspace{0.17em}{U}_{\phi \zeta }^{\tau }\hspace{0.17em}{h}_{\phi }(\mathit{\omega })={\sum }_{\zeta }{\stackrel{⌣}{H}}^{\tau }(\mathbf{x},\zeta )\hspace{0.17em}{p}_{\zeta }\hspace{0.17em}(\mathit{\omega })$$ (49)

where

$$p_{\zeta }\equiv {\sum }_{\phi \prec \mu }\hspace{0.17em}{h}_{\phi }(\zeta )h_{\phi }(\mathit{\omega })W_{\zeta }$$ (50)

so that the DOV decoupling corresponds to multivariate polynomial interpolation back from the solutions H̆^τ(x, ζ) of certain auxiliary problems indexed by the multivariate HG knots ζ.

In special cases, the DOV decoupling corresponds to interpolation from certain natural problems: one simply truncates the series for the boundary conditions and evaluating the truncated series at the knots ζ, then performs the interpolation corresponding to the HG-cubature. This holds, in particular, in the multilinear cases isolated by Babuska et al. [6]. In the general case, the auxiliary problems, on which the interpolation is based, seem less natural. The method of DOV has limited applicability: as shown in Appendix D, the matrices X̆^ω commute only for rectangular truncation sets; even in this case, the matrix U^τ will not diagonalize other matrices X̆^α.

4.2 Approximate Collocation

Since the double orthogonal variable decoupling reflects a polynomial interpolation based on tensor-product cubature knots, it is reasonable to consider approximating the various integrals, involved in the stochastic projections, by the corresponding cubature formulæ. Unlike the truncated matrices X̆^γ, the approximations X̃^γ computed by the cubature formula are all diagonalized by the matrix U^τ, so all expressions simplify further.

With a rectangular truncation set τ = τ(μ) of the variables, Eq. (9) becomes

$${\mathrm{\sum }}_{\alpha \prec \mu }\nabla \cdot G_{\alpha \phi }\nabla H_{\alpha }=-{\theta }_{\phi }$$ (51)

One obtains by the corresponding Gaussian cubature

$$\begin{array}{l}{G}_{\alpha \phi }\approx {\stackrel{\sim }{G}}_{\alpha \phi }\equiv {\mathrm{\sum }}_{\zeta }\hspace{0.17em}\exp \hspace{0.17em}[{\stackrel{⌣}{\kappa }}^{\tau }\left(\mathbf{x},\zeta \right)]h_{\alpha }\left(\zeta \right)h_{\phi }\left(\zeta \right)W_{\zeta }\\ \phantom{G_{\alpha \phi }\approx {\stackrel{\sim }{G}}_{\alpha \phi }\equiv {\mathrm{\sum }}_{\zeta }\hspace{0.17em}}={\mathrm{\sum }}_{\zeta }\hspace{0.17em}\exp \hspace{0.17em}[{\stackrel{⌣}{\kappa }}^{\tau }\left(\mathbf{x},\zeta \right)]U_{\alpha \zeta }^{\tau }{U}_{\phi \zeta }^{\tau }\end{array}$$ (52)

and hence

$${\mathrm{\sum }}_{\zeta }\nabla \cdot \exp \hspace{0.17em}[{\stackrel{⌣}{\kappa }}^{\tau }\left(\mathbf{x},\zeta \right)]U_{\phi \zeta }^{\tau }\nabla {\mathrm{\sum }}_{\alpha \prec \mu }{H}_{\alpha }{U}_{\alpha \zeta }^{\tau }\approx -{\theta }_{\phi }$$ (53)

$${\mathrm{\sum }}_{{\zeta }^{\prime }}{\mathrm{\sum }}_{\phi \prec \mu }{U}_{\phi \zeta }^{\tau }{U}_{\phi {\zeta }^{\prime }}^{\tau }\nabla \cdot \exp \hspace{0.17em}\left[{\stackrel{⌣}{\kappa }}^{\tau }\left(\mathbf{x},{\zeta }^{\prime }\right)\right]\nabla {\hat{H}}_{{\zeta }^{\prime }}\approx -{\mathrm{\sum }}_{\phi \prec \mu }{\theta }_{\phi }{U}_{\phi \zeta }^{\tau }$$ (54)

$$\nabla \cdot \exp \hspace{0.17em}\left[{\stackrel{⌣}{\kappa }}^{\tau }\left(\mathbf{x},\zeta \right)\right]\nabla {\hat{H}}_{\zeta }\approx -{\hat{\theta }}_{\zeta }$$ (55)

$$\nabla \cdot \exp \hspace{0.17em}[{\stackrel{⌣}{\kappa }}^{\tau }\left(\mathbf{x},\zeta \right)]\nabla {\stackrel{⌣}{H}}^{\tau }\left(\mathbf{x},\zeta \right)\approx -{\stackrel{⌣}{\theta }}^{\tau }\left(\mathbf{x},\zeta \right)$$ (56)

Here H^ is defined by Eq. (42); Eq. (56) results from a division by C_γ; and we continue to use the definition (40). The Dirichlet boundary condition in Eqs. (47) and (48) is unaffected, while the Neumann condition becomes

$${\partial }_n{\stackrel{⌣}{H}}^{\tau }\left(\mathbf{x},\zeta \right){\mid }_{{\partial }_N}=\exp \hspace{0.17em}\left[-{\stackrel{⌣}{\kappa }}^{\tau }\left(\mathbf{x},\zeta \right)\right]{\stackrel{⌣}{Q}}^{\tau N}\left(\mathbf{x},\zeta \right)$$ (57)

The interpolation scheme (49) remains in effect. For these equations, there seems to be no obvious reason to prefer the truncated expansions θ̆^τ(x, ζ), H̆^τD(x, ζ) and Q̆^τN(x, ζ) to the exact evaluations θ(x, ζ), H^D(x, ζ) and Q^N(x, ζ), use of which may avoid a somewhat tedious expansion.

Thus, in the collocation decoupling, as opposed to the DOV decoupling, the log-conductivity need not be multilinear. Moreover, the problems on which the polynomial interpolation is based are simpler in the case of the collocation decoupling. In fact, given a stochastic equation for an unknown Z ≡ Z(ω) involving known functions Y₁(ω), ⋯ , Y_n(ω) of independent normal variates, rectangular truncation of the PC expansion for Z, with computation of the projections by tensor-product HG cubature, will produce a system neatly decoupled by the matrix U^τ, and the functions Y will be replaced by their values (or the values of their truncated expansions) at the knots ζ.

The calculations in this section will not directly generalize to nonrectangular truncation sets. In Appendix D, it is shown that there is no exact analog for HG cubature for the nonrectangular truncation sets τ. A cubature rule, having the same number of knots as monomials in τ, either fails to respect the orthonormality of the h-polynomials corresponding to monomials in τ or miscalculates entries in some X̆^ω . Appendix D contains a brief discussion of the restrictions imposed by such a decoupling cubature rule, and some indications of the problem that would have to be solved to find such a cubature. To our knowledge, the relations of such cubature problems to polynomial chaos have not been explored.

5 Karhunen-Loève-based Moment Equations

The Karhunen-Loève Moment Equation (KLME) method, introduced in Lu and Zhang [50] and Zhang and Lu [83], offers a polynomial interpolation method with reduced coupling between polynomials: the field H is expanded in a multivariate power series

$$H(\mathbf{x})\equiv {\mathrm{\sum }}_{\alpha }{H}_{\alpha }\alpha$$ (58)

where, as before, the α denote monomials in the random variates ω obtained from κ by KLE. Inserting the expansions of H and the log-conductivity κ into Eq. (14), expanding the modified source θe^-κ in a multivariate power series, and comparing terms, one finds H_α satisfies an equation in which the source-term involves various H_α/ω, so the coefficient fields H_α may be found successively. The expectations E[α] being known, the various covariance functions involving H are directly computable. Calculations can become somewhat more convenient, however, if one transforms the expansion of Eq. (58) into a Hermite series, which is inexpensive as the basis-change coefficients, described in Appendix C, form sparse upper triangular matrices.

KLME seems attractive since, like MC, it requires only domain-sized solves and can be applied adaptively, without initial decisions regarding the truncation set. But even in the univariate case, techniques based on manipulation of ordinary power series may not perform well with the generality of PC expansions. Because the moments E[ωⁿ] grow rapidly, simple series manipulations may not produce sensible results. The following can occur, for example. An ordinary (nonstochastic) function ϕ may admit a Taylor series expansion which converges everywhere in the ordinary sense, and the expectation E[ϕ(ω)] may exist for the Gaussian variate ω, yet an attempt to compute E[ϕ(ω)] by taking the expectation term-by-term from the ordinary Taylor expansion yields a non-convergent series. Use of ordinary series expansions in a stochastic setting may therefore require careful attention. On the other hand, under mild conditions Cameron and Martin [11] establishes a good Hermite series approximation to ϕ, considered as a function of the Gaussian variate ω.

6 Nicolaides Interpolation

To obtain polynomial models, using the decouplings of §4, becomes increasingly expensive with dimension: the cost of the cubatures is exponential in e, and the interpolatory matrix U^τ of Eq. (49) is full. Because these decouplings indicate that the PC attack on a stochastic equation is closely related to polynomial interpolation in stochastic space, further examination of interpolation seems reasonable.

Using various sparse cubature schemes Xiu and Hesthaven [78] have considered a collocation solution of a one-dimensional analog of Eq. (1) with independent uniform variates ω. Although the sparse cubatures considered [38, 78] involve many fewer knots than the full tensor-product HG-cubature, the number of required knots is still a multiple of the number of PC terms sought. Furthermore, the effect of this multiple may be non-negligible, since each function evaluation requires solving a realization of Eq. (1).

We briefly consider, as a simple alternative to the decouplings of §4.2, solving the polynomial interpolation problem with N(e, d) knots by the method of Nicolaides [14]. In this method, interpolatory polynomials are constructed using hyperplanes defined by barycentric coordinates relative to a given simplex. For the standard simplex with vertices defined by the origin and standard basis vectors, one considers vectors ζ ≡ (z_ω1, z_ω2, ⋯ , z_ωe) of non-negative integers satisfying ∣ζ∣ ≤ d and takes as knots the points with spatial coordinates n_ζ ≡ ζ/d. The corresponding interpolatory polynomials are defined by

$$p_{\zeta }(\mathit{\omega })\equiv \left[{\mathrm{\Pi }}_{\omega }\frac{{\mathrm{\Pi }}_{r=o}^{z_{\omega }-1}(d\omega -r)}{z_{\omega }!}\right]\times \left[\frac{{\mathrm{\Pi }}_{r=0}^{d-\mid \zeta \mid -1}[(d-r)-d{\mathrm{\sum }}_{{\omega }^{\prime }}{\omega }^{\prime }]}{(d-\mid \zeta \mid )!}\right]$$ (59)

If instead, one dilates the standard simplex by the factor d, the knots coincide with the vectors ζ and the polynomials become p_ζ(ω/d). Carnicer et al. [12] have obtained error estimates for such interpolations. A simple algorithm thus results by solving Eq. (1) at the Nicolaides knots and using Eq. (59) to obtain a polynomial model; statistics are available from the typically sparse matrix of moments.

7 Monte Carlo Simulation

We next consider evaluation of the quality of a truncated PC expansion by comparison to naïve MC. Although it seems natural to value a PC solution in terms of the number of MC simulations required to obtain the same accuracy, judging quality in this manner is problematic, since the exact problem solution is unknown. Chorin [13] proposed use of univariate Hermite expansions to accelerate MC, and Maltz and Hitzl [52] suggested the extension to multivariate expansions. Here we instead use such ideas to evaluate the accuracy of PC expansions, rather than to improve MC. A PC solution provides a polynomial “easy function” [30] that can be used to accelerate MC by variance reduction. This acceleration provides a natural, easily estimated measure of the accuracy of the approximate PC coefficients.

Approximate statistics of a random function η can be obtained from a sequence η_j of N independent realizations. Direct calculation shows the estimator

$${\overline{\eta }}_N\equiv N^{-1}{\sum }_{j=1}^N{\eta }_j$$ (60)

for the mean η̄ converges subject to the estimate

$$E[\mid {\overline{\eta }}_N-\overline{\eta }\mid ]\le E{[{({\overline{\eta }}_N-\overline{\eta })}^2]}^{1/2}=\sigma /\sqrt{N}$$ (61)

where σ is the standard deviation of η. As the mean η̄ is unknown, convergence may be conveniently judged by examining the fluctuation sizes between successive estimates; the fluctuation of η̄_N with lag k is given by

$$E[{({\overline{\eta }}_N-{\overline{\eta }}_{N+k})}^2]=k{\sigma }^2/[N(N+k)]$$ (62)

where σ² is the variance of η, and the log-fluctuation of the standard deviation is therefore

$$\log \hspace{0.17em}(\sigma \sqrt{k})-\log \hspace{0.17em}[N(N+k)]/2\approx \log \hspace{0.17em}(\sigma \sqrt{k})-\log \hspace{0.17em}(N)$$ (63)

when N ≫ k.

Another estimator for the mean head can be obtained from H̄_N, using a finite approximation

$$H^{\mathit{est}}\equiv {\sum }_{\alpha \in \tau }{H}_{\alpha }^{\mathit{est}}{h}_{\alpha }$$ (64)

to the PC expansion H = Σ_αH_αh_α of the true field H. Each realization κ_j is associated with a collection ω^(j) of specific values for the variates ω, and the H_j, obtained by solving Eq. (1) with log-conductivity κ_j, can be approximated by substituting the ω^(j) into the approximate PC expansion to obtain

$$H_j^{\mathit{est}}\equiv {\sum }_{\alpha \in \tau }{H}_{\alpha }^{\mathit{est}}{h}_{\alpha }({\mathit{\omega }}^{(j)})$$ (65)

so the multivariate h-polynomials, defined by (21) and (17), are required here.

Consider the accelerated estimator

$$H_N^{\mathit{acc}}\equiv {\overline{H}}^{\mathit{est}}+N^{-1}{\sum }_{j=1}^N(H_j-H_j^{\mathit{est}})$$ (66)

where the H̄^est is the estimated mean head obtained from the PC approximation (64). The naïve estimator H̄_N has variance

$${\sigma }_N^2=N^{-1}{\sum }_{\alpha \ne 1}{H}_{\alpha }^2\equiv N^{-1}{\sigma }^2$$ (67)

while the accelerated estimator $H_N^{\mathit{acc}}$ has variance

$${({\sigma }_N^{\mathit{acc}})}^2=N^{-1}{\sum }_{\alpha \ne 1}{(H_{\alpha }-H_{\alpha }^{\mathit{est}})}^2\equiv N^{-1}{({\sigma }^{\mathit{acc}})}^2$$ (68)

where Σ_α≠1 denotes summation over the exponent vectors < α_ω >_ω∈ω corresponding to nonconstant multivariate h-polynomials h_α, with the convention $H_{\alpha }^{est}\equiv 0$ for α not belonging to τ. Convergence of the modified estimator is accelerated relative to the naïve estimator by the factor

$${\phi }_{\mu }=\sigma /{\sigma }^{\mathit{acc}}$$ (69)

in the sense that use of the accelerated estimator $H_N^{\mathit{acc}}$ leads to approximately the accuracy expected from H̄_Nϕμ; true acceleration, of course, only occurs if ϕ_μ > 1. The factor ϕ_μ can be easily be estimated using Eq. (63): in log-log space, regression lines for the fluctuations of H̄_N and $H_N^{\mathit{acc}}$ (both with lag k) should be approximately parallel, with negative unit slope, separated by log(ϕ_μ). The acceleration measures the accuracy of a truncated PC expansion, since 1/ϕ_μ is the relative error associated with the truncation.

To assess the quality of a PC approximation, by comparing the unaccelerated and accelerated estimators as just described, it is necessary to obtain the PC coefficients together with the variate values ω^(j) for each realization κ_j and to evaluate the PC expansion at those values. The acceleration of higher order moments may be gauged similarly, using the analogous variance reduced “easy functions”

$${(H^m)}_N^{acc}\equiv {\overline{H^m}}^{\mathit{est}}+N^{-1}{\sum }_{j=1}^N[H_j^m-{(H_j^{\mathit{est}})}^m]$$ (70)

where H_j and $H_j^{\mathit{est}}$ are again the actual and estimated values of the head for a particular realization and the uncentered moment ${\overline{H^m}}^{\mathit{est}}$ is precalculated from the approximation H^est.

In the discussion above, the acceleration is measured pointwise throughout the domain. Rather than obtaining a domain-averaged version, we produce two estimates, differing in the order in which ratios are computed and domain-averaging conducted. We describe these estimates for the mean, the variance is handled similarly. The first estimate domain-averages the logspace fluctuations of the ordinary and accelerated MC generators and, assuming that true results would be approximately governed by a linear law of the form (63), uses the average separation of the lines of best fit to estimate the reduction of variance. This estimate is thus obtained from the separation of the lines of best fit to log ∥H̄_n − H̄_n+k∥_Ω and $\log \parallel H_n^{\mathit{acc}}-H_{n+k}^{\mathit{acc}}\parallel \mathrm{\Omega }$. The second estimate domain-averages the difference of the logspace fluctuations and, assuming that the results would be approximately constant, uses the average height of the line of best fit to estimate the reduction of variance. The second estimate is obtained from the average height of the line of best fit to

$$\parallel \log \hspace{0.17em}\mid {\overline{H}}_n-{\overline{H}}_{n+k}\mid -\log \hspace{0.17em}\mid H_n^{\mathit{acc}}-H_{n+k}^{\mathit{acc}}\mid \parallel \mathrm{\Omega }$$ (71)

A polynomial model may be compared to MC in the following manner. By the triangle inequality, an MC estimate H̄_N of the mean H̄ is more accurate than the estimate H̄^est provided by a polynomial model once the MC estimate error is less than half the difference between the two estimates. Thus MC results are expected to be more accurate than PC results when

$$\mid {\overline{H}}_N-{\overline{H}}^{\mathit{est}}\mid /2>\mid {\overline{H}}_N-\overline{H}\mid \approx \sigma /\sqrt{N}\approx \sqrt{N}{\mathrm{\Delta }}_{M{C}_N}/\sqrt{k}$$ (72)

where Δ_MCN ≡ ∣H̄_N − H̄_N+k∣ is the MC fluctuation. The left side of the inequality should tend to a constant with increasing N, while the right side continues to decrease. Equivalently, MC wins once

$$\sqrt{k/N}\mid {\overline{H}}_N-{\overline{H}}^{est}\mid /2>{\mathrm{\Delta }}_{M{C}_N}$$ (73)

Higher order moments may be treated similarly.

8 Implementation and Results

In principle, any simulator for the field κ could be used, since the family ω can then be recovered using Eq. (A.3). However, as Xiu and Karniadakis [79] have noted, additional numerical issues would arise if a generator, different from that analyzed by the PC expansion, were used for MC; we therefore simulated κ using the computed KLE. The same generator is reused to compute the values of exp(κ˘^τ) at the cubature points in the solution of the decoupled system. After solving the decoupled systems of §4, the approximate solution of the coupled PC system is recovered by polynomial interpolation back from the multivariate knots. As discussed in §3.2, the variates appearing in τ may be a subset of those in ω: that is, the KL simulator may include variability for which the truncation set τ does not account.

A pseudorandom variable cannot pass all statistical tests. PC analysis requires the nonconstant multivariate probabilists’ h-polynomials (in a standard normal variate) to vanish in expectation, and good correspondence between MC and PC results cannot be expected if the simulated expectations E[h_n] are large. Bontemps and Meddahi [8] have remarked that the vanishing of such polynomials is actually characteristic of the standard normal distribution. This leads to a natural family of conditions E[h_n] ≡ 0 (n > 0) for the pseudorandom generator. We examined the ziggurat, described in Marsaglia and Tsang [54] and references therein, and implemented by Matlab’s randn, as well as Box-Muller transformations of several uniform generators (Matlab’s rand, the Sedgewick generator, L’Ecuyer’s MLCG [44]), and the Mathwork’s mex-file implementation of Twister [58], before choosing the ziggurat for N(0, 1) variate simulation. Estimators described in §7 were obtained using 1000 MC simulations with lag k ≡ 10.

For computational purposes, the indices α of the h-polynomials can be encoded by a table, with the row corresponding to α containing the exponent vector < a_ω >_ω∈ω. For total degree truncation sets, such a table with degree-lexicographic ordering can be built quickly by recursion on the degree.

The operator ∇K∇ was discretized using a central five-point finite difference stencil. The operator ∇_κ · ∇ + ∇², was discretized using central second-order finite differences for the first term and a five point stencil for the Laplacian. Stencils for the coupled system were obtained using the same projection technique described in §3, that is, by multiplying the stencil with various h-polynomials and computing expections. The numerical versions of the computed PC fields then actually provide a truncated PC expansion of the numerical version of H. The symmetric systems resulting from (1) were solved using the conjugate gradient method and the nonsymmetric systems resulting from (14) were solved using the biconjugate gradient method. In both cases, Jacobi preconditioning was used on a 50 × 50 grid; for comparison purposes, some cases were rerun on a 100 × 100 grid.

8.1 Analytic Example

Simple analytic examples can provide some information regarding the accuracy of PC methods. Given independent standard normal variates ω and corresponding functions k_ω (on domain Ω) satisfying

$$k_{\omega }{\mid }_{\partial \mathrm{\Omega }}\equiv 0$$ (74)

define

$$\kappa \equiv {\sum }_{\omega }{k}_{\omega }\omega ,\hspace{0.17em}K\equiv \exp \hspace{0.17em}\left(\kappa \right),\hspace{0.17em}\mathrm{\theta }\equiv {\nabla }^2\kappa$$ (75)

Then equation (1) with boundary conditions

$$H{\mid }_{\partial \mathrm{\Omega }}\equiv 0$$ (76)

has analytic solution

$$H\equiv \exp \left(-\kappa \right)-1$$ (77)

against which MC and PC solutions can be compared. We take

$$k_{{\omega }_{mn}}\equiv c_{mn}\hspace{0.17em}\sin \hspace{0.17em}\left(2\pi mx/L_x\right)\hspace{0.17em}\sin \hspace{0.17em}\left(2\pi ny/L_y\right)$$ (78)

with normalization constants c_mn and certain synthetic eigenvalues λ_mn. Double orthogonal variable decoupling requires computation of the source-terms from Eq. (46)

$$\begin{array}{c}\phantom{=-\exp \hspace{0.17em}\left[\kappa (\mathbf{x},\zeta )\right]C_{\zeta }^{-1}\sum }\nabla \cdot \exp \hspace{0.17em}\left[\kappa \left(\mathbf{x},\zeta \right)\right]\nabla {\stackrel{⌣}{H}}^{\tau }\left(\mathbf{x},\zeta \right)\\ =-\exp \hspace{0.17em}\left[\kappa \left(\mathbf{x},\zeta \right)\right]C_{\zeta }^{-1}{\sum }_{\phi \prec \mu }{\sum }_{\mid \alpha \mid \le 1}\left({\nabla }^2{\kappa }_{\alpha }\right)G_{\alpha \phi }^{-1}{U}_{\phi \zeta }^{\tau }\end{array}$$ (79)

In this fashion, small dimensional examples can be constructed for which MC converges slowly while PC methods provide accurate results; it is also possible to construct such examples for which the decoupled PC methods do not produce acceptable results. Such examples also illustrate the reason for the decision in §3, to work with the conductivity in the form K ≡ exp(κ), rather than using a truncated Hermite series

$${\stackrel{⌣}{K}}^{\tau }\equiv {\sum }_{\alpha \prec \mu }{K}_{\alpha }{h}_{\alpha }$$ (80)

with K_α found, for example, from Eq. (C.9); use of the truncated series can lead to erratic numerical breakdown. Since the statistics vary rapidly in space the two acceleration estimates described in §7 can differ substantially for such examples.

Taking m₁ = n₁ = 1, m₂ = n₂ = 2, λ₁₁ = 0.5, and λ₂₂ = 0.25, using the rectangular truncation set described in the notation of §3.2 as $\tau =\tau \left({\omega }_{11}^9{\omega }_{22}^9\right)$ or τ = [9, 9], and decoupling by DOV, produces mean and variance estimates an order of magnitude more accurate than those provided by 1000 naïve MC simulations. The same example, using the exact collocation source term (rather than the truncated collocation source suggested by the tensor-product HG cubature) in the collocation decoupling actually provides a slightly more accurate variance estimate than provided by the orthogonal variables decoupling (with errors measured in the L_∞ spatial norm). The Nicolaides interpolation based on the dilated simplex is completely unsatisfactory for this example; the analogous interpolation based on the undilated simplex reproduces some basic features of the exact solution (in the degree five case, say) but suffers rapid breakdown with increasing degree and does not appear competitive with the tensor-product interpolation.

8.2 Non-Analytic Investigation

Using a twenty-term KLE to simulate the log-conductivity field, we solved (14) in the source-free case on the unit square with two opposite no-flow and two opposite constant head boundaries

$$\begin{array}{l}H\left[\left(x_1,0\right),\mathit{\omega }\right]\equiv 0\\ H\left[\left(x_1,1\right),\mathit{\omega }\right]\equiv 1\end{array}$$ (81)

for a range of domain sizes (measured in correlation lengths ϱ) and variances of the log-conductivity κ, assuming the Gaussian covariance described by Eq. (34). Polynomial models were evaluated by calculating MC accelerations as described in §7; not all variates used for the field simulator are necessarily included in the analysis leading to the models. To compare polynomial models to MC, we used Eq. (73).

Four different polynomial models were compared: the one obtained from HG cubature, the one obtained by Nicolaides interpolation, the one obtained from the coupled PC system in Eq. (15), and the one obtained by KLME. Table II exhibits results for the HG cubature associated with the DOV decoupling, Table III for Nicolaides interpolation, Table IV for the coupled PC systems, and Table V for the KLME method. The truncations for Table II were chosen to involve truncations sets of approximately the same size as those used in Tables III, IV, and V, so that the number of function evaluations is approximately equivalent for all cases shown. These tables all exhibit MC acceleration factors ϕ_μ and ϕ_σ for head mean μ_H and variance ${\sigma }_H^2$, as functions of domain size ϱ⁻¹ × ϱ⁻¹ and log-conductivity variance ${\sigma }_{\kappa }^2$. In calculation of Table V, solver failures of unknown origin affected one row (marked by an asterisk), which was therefore recalculated independently.

Table II.

Estimated MC acceleration from HG cubature

Degree Vector	${\sigma }_{\kappa }^2$	ϱ−1	ϕμ(est1)	ϕμ(est2)	ϕσ(est1)	ϕσ(est2)
[4442]	0.5	1	5.712	6.009	5.533	5.460
—	1.0	—	5.945	6.360	5.744	6.381
—	1.5	—	4.054	4.141	3.825	3.677
—	0.5	2	2.168	2.160	2.174	2.084
—	1.0	—	2.710	2.900	2.629	2.632
—	1.5	—	2.585	2.827	2.505	2.617
—	0.5	3	1.925	1.904	1.857	1.842
—	1.0	—	1.726	1.794	1.770	1.818
—	1.5	—	1.720	1.682	1.733	1.639
[3333]	0.5	1	5.193	5.817	5.155	5.840
—	1.0	—	5.147	5.659	4.995	5.566
—	1.5	—	3.955	4.108	4.116	4.089
—	0.5	2	2.537	2.569	2.609	2.726
—	1.0	—	2.411	2.458	2.266	2.366
—	1.5	—	2.335	2.469	2.101	2.404
—	0.5	3	1.900	1.883	1.769	1.874
—	1.0	—	1.580	1.519	1.659	1.519
—	1.5	—	1.782	1.804	1.865	1.841
[22222]	0.5	1	9.461	10.640	9.088	10.625
—	1.0	—	7.979	8.865	8.094	9.286
—	1.5	—	9.255	10.889	8.655	9.595
—	0.5	2	3.177	3.532	3.144	3.666
—	1.0	—	3.358	3.634	3.292	3.565
—	1.5	—	3.090	3.394	2.864	3.015
—	0.5	3	2.513	2.528	2.507	2.383
—	1.0	—	2.042	2.145	1.960	2.008
—	1.5	—	2.324	2.225	2.227	2.292
[11111111]	0.5	1	13.206	14.850	12.380	15.913
—	1.0	—	6.208	6.223	8.655	9.689
—	1.5	—	6.294	6.820	5.306	6.557
—	0.5	2	4.910	4.950	4.863	5.128
—	1.0	—	4.652	4.417	4.080	3.974
—	1.5	—	3.892	3.780	3.560	3.650
—	0.5	3	3.357	3.231	3.220	3.351
—	1.0	—	2.980	2.977	2.915	3.197
—	1.5	—	2.853	3.009	2.473	2.669

Table III.

Estimated MC acceleration from Nicolaides interpolation

(e,d)	${\sigma }_{\kappa }^2$	ϱ−1	ϕμ(est1)	ϕμ(est2)	ϕσ(est1)	ϕσ(est2)
(5, 5)	0.5	1	9.918	11.341	10.087	12.735
—	1.0	—	5.394	6.356	7.259	8.160
—	1.5	—	6.440	7.924	8.625	10.521
—	0.5	2	3.646	3.878	3.845	4.093
—	1.0	—	2.940	2.921	3.151	3.377
—	1.5	—	2.136	2.204	2.352	2.453
—	0.5	3	2.379	2.442	2.504	2.358
—	1.0	—	2.058	2.227	2.089	2.226
—	1.5	—	2.091	2.155	2.220	2.211
(6, 4)	0.5	1	11.648	12.829	12.454	14.031
—	1.0	—	6.996	7.762	9.502	10.670
—	1.5	—	5.085	5.519	7.252	8.810
—	0.5	2	3.356	3.813	3.456	3.994
—	1.0	—	3.496	3.639	3.762	4.397
—	1.5	—	2.895	3.378	3.611	4.357
—	0.5	3	3.079	3.187	2.873	3.112
—	1.0	—	2.472	2.438	2.427	2.351
—	1.5	—	2.191	2.330	2.323	2.357
(9, 3)	0.5	1	15.334	15.779	18.390	22.583
—	1.0	—	10.063	11.790	12.979	18.576
—	1.5	—	4.127	4.424	7.441	10.636
—	0.5	2	6.349	6.246	7.089	7.501
—	1.0	—	3.437	3.601	5.366	6.192
—	1.5	—	2.546	2.519	3.914	4.437
—	0.5	3	4.569	4.589	4.489	4.509
—	1.0	—	2.823	2.763	3.345	3.502
—	1.5	—	2.549	2.568	3.141	3.071
(20, 2)	0.5	1	9.396	10.877	15.398	17.854
—	1.0	—	5.654	6.471	10.067	12.613
—	1.5	—	5.711	6.470	9.339	10.701
—	0.5	2	5.586	5.381	8.463	9.218
—	1.0	—	3.016	3.176	5.468	6.475
—	1.5	—	2.939	2.984	3.853	3.860
—	0.5	3	6.146	6.020	8.146	8.149
—	1.0	—	3.031	3.000	4.748	5.174
—	1.5	—	2.572	2.584	3.006	3.206

Table IV.

Estimated MC acceleration from coupled PC system

(e,d)	${\sigma }_{\kappa }^2$	ϱ−1	ϕμ(est1)	ϕμ(est2)	ϕσ(est1)	ϕσ(est2)
(5, 5)	0.5	1	10.929	13.796	10.376	14.065
—	1.0	—	11.911	15.269	11.124	14.223
—	1.5	—	11.938	15.582	12.032	14.968
—	0.5	2	4.429	5.131	4.157	4.652
—	1.0	—	3.638	4.016	3.784	4.408
—	1.5	—	3.705	4.156	3.653	4.045
—	0.5	3	2.339	2.547	2.549	2.356
—	1.0	—	2.352	2.365	2.305	2.291
—	1.5	—	1.991	2.072	2.033	2.106
(6, 4)	0.5	1	11.619	13.554	11.198	13.175
—	1.0	—	13.213	15.758	12.747	14.883
—	1.5	—	11.480	13.609	10.571	11.298
—	0.5	2	4.094	4.578	3.894	4.468
—	1.0	—	3.322	3.660	3.326	3.211
—	1.5	—	3.153	3.102	3.182	3.062
—	0.5	3	2.477	2.506	2.411	2.498
—	1.0	—	2.167	2.173	2.219	2.188
—	1.5	—	2.452	2.634	2.383	2.537
(9, 3)	0.5	1	43.130	48.143	40.242	44.161
—	1.0	—	36.731	41.928	35.687	39.881
—	1.5	—	19.151	20.978	18.501	20.309
—	0.5	2	9.770	10.779	9.233	10.457
—	1.0	—	9.069	9.401	8.881	9.334
—	1.5	—	6.330	6.739	6.306	6.618
—	0.5	3	4.530	4.684	4.298	4.573
—	1.0	—	4.410	4.657	4.219	4.404
—	1.5	—	3.444	3.599	3.269	3.647
(20, 2)	0.5	1	24.366	26.731	25.480	28.972
—	1.0	—	16.794	18.770	16.628	17.416
—	1.5	—	16.468	20.109	15.082	18.191
—	0.5	2	15.337	15.213	15.464	16.374
—	1.0	—	9.164	9.123	9.257	9.628
—	1.5	—	6.360	6.433	6.247	6.241
—	0.5	3	13.682	14.253	13.814	15.227
—	1.0	—	7.009	6.965	6.949	6.543
—	1.5	—	5.222	5.116	5.356	5.348

Table V.

Estimated MC acceleration from KLME

(e,d)	${\sigma }_{\kappa }^2$	ϱ−1	ϕμ(est1)	ϕμ(est2)	ϕσ(est1)	ϕσ(est2)
(5, 5)	0.5	1	14.046	16.701	13.381	15.917
—	1.0	—	11.662	14.425	12.100	14.388
—	1.5	—	10.478	13.062	10.385	13.051
—	0.5	2	4.143	4.740	4.073	4.482
—	1.0	—	3.154	3.228	2.940	2.982
—	1.5	—	3.569	3.757	3.381	3.399
—	0.5	3	2.478	2.568	2.367	2.490
—	1.0	—	2.172	2.339	2.072	2.169
—	1.5	—	2.057	1.948	1.957	1.870
(6, 4)	0.5	1	11.733	13.248	11.851	13.721
—	1.0	—	9.524	11.448	8.834	10.922
—	1.5	—	7.684	9.150	8.477	9.839
—	0.5	2	4.250	4.541	3.873	4.283
—	1.0	—	3.221	3.410	3.098	3.320
*–	1.5	—	3.225	3.286	3.119	3.196
—	0.5	3	2.508	2.529	2.472	2.551
—	1.0	—	2.210	2.402	2.147	2.425
—	1.5	—	2.192	2.307	2.286	2.358
(9, 3)	0.5	1	37.367	43.170	30.932	38.853
—	1.0	—	26.062	30.324	25.603	30.489
—	1.5	—	12.621	14.030	13.405	13.136
—	0.5	2	10.417	11.752	10.140	11.176
—	1.0	—	5.437	5.314	5.257	5.415
—	1.5	—	3.373	3.429	3.248	3.199
—	0.5	3	4.993	5.269	4.657	4.990
—	1.0	—	3.596	3.596	3.508	3.473
—	1.5	—	3.095	3.287	2.963	3.166
(20, 2)	0.5	1	19.629	24.208	19.807	24.280
—	1.0	—	11.826	14.244	10.241	11.471
—	1.5	—	6.615	7.369	6.341	6.276
—	0.5	2	10.391	11.821	8.727	8.977
—	1.0	—	3.809	3.738	3.559	3.332
—	1.5	—	3.710	3.858	3.369	3.323
—	0.5	3	9.185	9.458	8.463	8.862
—	1.0	—	5.098	5.674	3.939	4.098
—	1.5	—	3.020	2.950	2.394	2.267

Several distinct issues contribute to the nonagreement between the numerical acceleration estimates given in each row of the tables. As discussed in §7, the estimates ϕ_μ(est₁) and ϕ_μ(est₂), and similarly the estimates ϕ_σ(est₁) and ϕ_σ(est₂), differ in the order in which ratios and domain-averages were calculated. See Fig. 1 in which the average gap between the lower lines represents the first estimate, while the average height of the upper line represents the second estimate. There is no reason to expect the first and second estimates to agree precisely. Moreover, the final estimates are obtained by exponentiating the results of such linear regressions in log-log space based on MC simulations. But the natural measure of the error is the reciprocal of the acceleration. For example, a table row with acceleration factors ranging from 40–50 would correspond to estimated relative errors of 2.0–2.5%. On balance, the table data appear surprisingly consistent.

Fig. 1.

MC and accelerated MC for truncation τ(9, 3) on a domain 1 × 1 correlation lengths in size, assuming σ² = 1. The lower two lines exhibit convergence of the MC methods; their separation provides the first acceleration estimate. The upper line provides the second estimate.

8.3 Discussion

8.3.1 Acceleration Factors

As discussed in §3.3, the number of variates required to accurately describe the field κ will increase with domain size (measured in correlation lengths). For a fixed truncation, increasing domain size increases the importance of un-simulated variates, and we therefore expect accuracy to decrease with domain size. The loss of accuracy, however, persists for the truncation (e, d) = (20, 2), although the twenty-term KLE we use as a simulator models the effect of all twenty variates to second order in this case.

Acceleration factors reported in the tables estimate the accuracy with which the various methods capture the influence of the variates. Our reported factors are overly optimistic because we simulate, not the full intended Gaussian field, but rather a truncated version that involves only a finite number of variates. Using more terms, from the KL expansion of the field, in these simulations should reduce the acceleration factors.

Because stencils for the PC systems were produced by inserting the KL and PC expansions into the deterministic stencils and projecting, as described in §3, the numerical PC coefficients reflect truncated PC system approximations to the random variates forming the numerical H field. The latter variates change with the discretization, but given sufficiently many PC terms one expects from the Cameron-Martin theorem that the numerical PC coefficients will faithfully model the numerical field, whether or not the numerical field accurately models the intended field. In support of this view, consider Table VI which exhibits the acceleration factors computed using a domain one correlation length in size for the truncations (e, d) = (5, 5) and (e, d) = (9, 3) with various discretizations. For Table IV acceleration factors smaller than about 15 (that is, in all (e, d) = (5, 5) cases), there is a spread of about 3% between the various estimates; when the acceleration factors are near 20 (that is, in the (e, d) = (9, 3) case with ${\sigma }_{\kappa }^2=1.5$), the spread is about 2%; and for acceleration factors larger than about 35 (that is, in the ${\sigma }_{\kappa }^2=0.5$ and ${\sigma }_{\kappa }^2=1.0$ cases of (e, d) = (9, 3)), the spread is only about 1%. Data in the tables indicates acceleration factors for a given truncation decrease with increasing variance ${\sigma }_{\kappa }^2$ and with increasing domain size for all methods considered.

Table VI.

Discretization Effect on Relative Error

Grid	(e,d)	${\sigma }_{\kappa }^2$	ϱ−1	% (μ est1)	% (μ est2)	% (σ est1)	% (σ est2)
25 × 25	(5, 5)	0.5	1	7.78	6.26	8.32	6.60
50 × 50	—	—	—	9.15	7.25	9.31	7.11
75 × 75	—	—	—	8.65	7.61	8.84	7.54
100 × 100	—	—	—	8.82	7.25	9.06	7.49
25 × 25	—	1.0	—	9.17	7.29	9.14	7.40
50 × 50	—	—	—	8.39	6.55	8.99	7.03
75 × 75	—	—	—	8.34	6.76	8.25	6.95
100 × 100	—	—	—	7.32	5.77	7.31	6.15
25 × 25	—	1.5	—	9.04	6.60	9.33	6.86
50 × 50	—	—	—	8.37	6.42	8.31	6.68
75 × 75	—	—	—	8.19	6.21	8.00	6.82
100 × 100	—	—	—	8.76	7.18	8.77	7.37
25 × 25	(9, 3)	0.5	1	2.52	2.11	2.61	2.11
50 × 50	—	—	—	2.32	2.08	2.48	2.26
75 × 75	—	—	—	1.95	1.61	2.07	1.61
100 × 100	—	—	—	2.29	2.08	2.45	2.27
25 × 25	—	1.0	—	2.98	2.72	3.11	2.90
50 × 50	—	—	—	2.72	2.38	2.80	2.51
75 × 75	—	—	—	2.40	2.08	2.39	2.13
100 × 100	—	—	—	2.67	2.36	2.75	2.47
25 × 25	—	1.5	—	3.86	3.36	4.17	3.43
50 × 50	—	—	—	5.22	4.77	5.41	4.92
75 × 75	—	—	—	4.41	3.96	4.58	4.40
100 × 100	—	—	—	5.13	4.70	5.32	4.85

The first effect seems natural: had we not renormalized our variates ω by absorbing their deviations into the eigenfunctions κ_ω^, the leading term of each h-polynomial h_α would include these variances; the rescaling must transfer this contribution to the H_α, which consequently must scale as polynomials in σ_κ; and thus neglected higher-order terms become more important as the variance increases.

The second effect may be more complicated. The slower eigenvalue decay associated with larger domains (cf. §3.3) has several consequences. As domain size increases, later eigenvalues become relatively more important, and neglected higher-order terms become more important as with increasing ${\sigma }_{\kappa }^2$. In particular, any failure to model all variates used by the MC simulator will produce larger errors on larger domains, as unmodeled contributions to the variability of κ increase. There being no ignored variates for our (20, 2)-truncation (because our simulation of the log-conductivity field involved only the 20 most important eigenpairs), the decrease in accuracy with increasing domain size in this case cannot be attributed only to missing variates. That the effect is seen in the case (e, d) = (20, 2), not only for PC but for KLME and the Nicolaides interpolation as well, provides additional evidence that the true H is more complicated and requires a higher-order analysis when the eigenvalues are more nearly equal than when there are relatively few large eigenvalues with the rest small.

For PC, there is a further possible contribution, associated with inaccurate calculation of the coefficient fields H_α, because truncation eliminates the full coupling between the defining equations. This effect is more diffcult to probe, because different truncations ignore different coupling effects. On small domains, where relatively few eigenvalues are important, one might expect the coupling among the first nine variates to be more important the coupling between the first nine and the last eleven, while the latter coupling effects become relatively more significant with increasing domain size. To test this, we replaced H_α computed by PC for the (20, 2)-truncation by the corresponding H_α computed for the (20, 2)-truncation; the replacements were made for the H_α with index α in the set of 55 indices τ(9, 3) ∩ τ(20, 2) common to both truncations. Acceleration factors were computed as before by MC, using the (20, 2)-table with the indicated replacements. Results are exhibited in Table VII, which suggests some improvement only on 1 × 1 and 2 × 2 domains and only in the small variance case and suggests some deterioration of the approximation in the remaining cases on the 1 × 1 domain. In general, it is not clear how to improve the estimated H_α except by using larger coupled systems.

Table VII.

Effect of Patching Coefficients from (9,3) into (20,2)

(e,d)	${\sigma }_{\kappa }^2$	ϱ−1	ϕμ(est1)	ϕμ(est2)	ϕσ(est1)	ϕσ(est2)
(20, 2)	0.5	1	33.920	38.636	30.321	35.543
—	1.0	—	14.060	15.205	14.980	17.008
—	1.5	—	12.804	14.589	12.502	13.388
—	0.5	2	23.278	25.249	21.706	22.947
—	1.0	—	9.298	9.497	9.407	9.598
—	1.5	—	6.594	6.488	6.615	6.725
—	0.5	3	13.056	12.480	12.867	13.089
—	1.0	—	7.920	8.471	8.217	8.965
—	1.5	—	5.316	5.129	5.267	4.894

8.3.2 Polynomial Chaos and Karhunen-Loève Moment Equations

The PC approach and KLME both yield polynomial models without much additional computation. The comparative accuracy of these methods is therefore interesting. Comparing Tables IV and V indicates that KLME may outperform PC when using a small number of variates to approximate a field of low variability. With increasing variability and number of variates, PC typically outperforms KLME.

Use of inequality (73) to compare the accuracy of the polynomial model and MC is illustrated for estimates in Figs. 2, 3, and 4. The straight lines provide the first acceleration estimate, while the irregular line provides the left side of inequality (73). Once the irregular line permanently falls above the upper straight line, representing the MC fluctuations, the MC estimate of the mean has become the more accurate of the two. In Fig. 2, cross-over occurs prior to 100 MC simulations, while N(20, 2) = 231. In Fig. 3, the irregular line does not fall above the upper line, indicating N(5, 5) = 252 iterations of KLME outperform 1000 MC simulations. In Fig. 4, the coupled PC system for the truncation τ = τ(9, 3) apparently outperform 600 but not 1000 MC simulations.

Fig. 2.

Comparison of MC and Nicolaides interpolation for truncation τ(20, 2) on a domain 1 × 1 correlation lengths in size, assuming σ² = 1.5.

Fig. 3.

Comparison of MC and KLME methods for truncation τ(5, 5) on a domain 2 × 2 correlation lengths in size, assuming σ² = 1.5.

Fig. 4.

Comparison of MC and PC methods for truncation τ(9, 3) on a domain 2 × 2 correlation lengths in size, assuming σ² = 1.5.

Comparison of KLME to MC by this method is straightforward, since KLME involves only domain-sized solves. For the given truncations τ = τ (e, d), with few exceptions, KLME wins over 1000 MC simulations, and in the exceptional cases KLME clearly provides better accuracy than would be obtained by MC using N(e, d) solves. Comparison of PC to MC by the same method is less meaningful, since the coupled PC system does not involve domain-sized solves and because a cubature decoupling would result in loss of accuracy, if it even existed. In almost all cases, the results obtained from the coupled PC system are more accurate than those obtained from 1000 MC realizations, and in the exceptional cases at least 600 MC realizations are needed to produce more accurate results. In our unoptimized code, the time needed for 1000 MC solves, together with the inexpensive auxiliary computations needed to compute the variance reduced estimators, is typically about twice that needed for the coupled PC system solve.

Individual matrix-vector multiplies, associated with the coupled systems for the truncation sets in Table IV, are about three orders of magnitude more costly than those associated with domain-sized solves. KLME requires eN(e, d−1) auxiliary matrix multiplications, but this extra cost associated with computing the problems to solve is small in our examples.

Thus, both PC and KLME appear to be competitive with MC on small domains. However, it is important to note that the feasibility of PC in this case depends heavily on the possibility of rewriting the stochastic equation in the form given by Eq. (14); projecting instead from Eq. (1) leads to intractably large systems. For both methods, it is clear that modeling enough variates becomes important as domain size increases, while on smaller domains a significant advantage can be obtained by ignoring some variates in order to use a higher degree model.

8.3.3 Interpolation and Cubature

Although HG cubature can produce good results when the number of required variates is small, our data might suggest that, for similar effort, Nicolaides interpolation outperforms HG cubature. However, the large size of rectangular truncation sets obstructs matching of the number of variates and degree to the problem, and the data illustrate the importance of an appropriate truncation, as is clear from the PC and KLME data: thus, for example, for ${\sigma }_{\kappa }^2=1$ on a 1 × 1 domain, the truncation set τ = τ(9, 3) provided better results for both PC and KLME than any other triangular truncation set considered, while for ${\sigma }_{\kappa }^2=1.5$ on a 3 × 3 domain, the truncation set τ = τ (20, 2) provides the best result.

Compared to PC and KLME, Nicolaides interpolation does not perform particularly well. This is not surprising, because no effort has been devoted to the optimal choice of the knots. Nicolaides interpolation with the undilated standard simplex, in fact, may negatively accelerate MC convergence; we used the dilated simplex.

Because the cost of obtaining a polynomial model from a cubature rule involves full matrices and is therefore is quadratic in the size of the truncation set, it is natural to seek interpolatory methods which involve relatively sparse matrices, as would be the case with Nicolaides interpolation. We do not know if good interpolatory methods, that can be applied adaptively, can be found for non-rectangular truncations. Stable interpolatory methods exist, in principle at least, for any finite set of bounded functions [55] but may not be convenient when the set of functions varies. The sparse cubature methods examined in [78] can be applied adaptively, by choosing nested knots, but have substantial overhead costs. KLME offers an adaptive method, which is competitive in the low-variance case with PC expansions, when not too many terms are included in the expansion. Both cubature rules and interpolatory methods deserve further examination as approaches to stochastic problems; for more information, see [7, 10, 12, 15, 20, 33, 48, 71, 81]).

8.3.4 Remarks on the Non-Gaussian Case

Two-point statistics cannot reproduce important features, such as networks of high-permeability channels [19], of interest to hydrologists. There has therefore been some interest in multipoint statistics, and non-Gaussian distributions are then indicated, because multivariate normal densities are completely characterized by their moments to second order.

We now remark on the effect of eliminating the independent normal variate assumption. For independent variates, of course, calculations still reduce to manipulations of univariate polynomials; Koekoek and Swarttouw [42] is a useful reference. Without assuming independence, one might begin as before, approximating the conductivity as a function K(x, ϖ) ≡ ϕ[ϰ(x, ϖ)] of some auxiliary field expansible in a finite series

$$\varkappa \equiv \overline{\varkappa }+{\sum }_{\varpi }{\varkappa }_{\varpi }\varpi$$ (82)

with uncorrelated (but not necessarily independent) variates ϖ, which may be assumed centered with unit variance. As indicated in D.3.2, one obtains under mild hypotheses a three-term recurrence for an orthonormal collection of multivariate polynomials π_α(ϖ). But beyond the problem of selecting a multivariate density, there is the further diffculty of computing the high-dimensional integrals required for the recurrence coefficients, when the number of variates is large.

Although simple formulæ for the $X_{\beta \gamma }^{\alpha }$ are not expected without special assumptions, convenient calculation may nevertheless sometimes be possible. Replacing each ϖ with the corresponding X^ϖ, in the three-term recurrence for the polynomials π_α(ϖ), produces a valid recurrence for the matrices X^α. A truncation of X^α, corresponding to a given truncation set τ, can therefore be computed by appropriately enlarging the truncation set to some τ′ (depending on both τ and α) and replacing each ϖ in π_α with the τ′-truncation of X^ϖ: the desired τ-truncation of X^α lies embedded within the resulting matrix (which, however, is not generally the τ′-truncation of X^α). As previously, the X^ϖ are simultaneously diagonalizable precisely for rectangular truncations.

Auxiliary calculations depend on certain series and hence not only on the joint density of the ϖ but on the particular function ϕ (ϰ). The case K ≡ ϰ is straightforward when the source term is expansible in the desired orthogonal series, because the matrices X̆^ϖ are sparse; moreover, the DOV decoupling applies for rectangular truncations. Thus Frauenfelder et al. [21] consider K ≡ ϰ involving independent identically-distributed (iid) uniform variates ϖ and apply DOV to precomputed rectangular truncations. Similarly with a linear conductivity assumption and using generalized PC expansions from various distribution functions, Xiu and Karniadakis [79] compare polynomial approximation and MC results.

Enforcing the natural requirement K(x, ϖ) > 0 will limit the possible variates ϖ: it would be inappropriate to use an expansion K ≡ ϰ with normal ϖ (say). Therefore other functional dependencies K ≡ ϕ may be unavoidable. To illustrate that other issues may arise from the interaction of ϕ with the density, consider iid exponential variates ϖ with K ≡ exp(ϰ). The univariate $X_{bc}^a$ are known for Laguerre polynomials [26] so the $X_{\beta \gamma }^{\alpha }$ a can be computed. The various G_αβ ≡ E[exp(ϰ)π_απ_β] of Eq. (9) can be recovered from the generating function for the univariate Laguerre polynomials, analogous to the computations in Appendix C (provided exp(ϰ) and θ are expansible; see [32] for conditions). Rewriting Eq. (1) in the more convenient form of Eq. (14) does not represent a useful option unless θ/K admits a Laguerre series. Unfortunately, it is possible that neither exp(ϰ) nor θ exp(−ϰ) can be developed in Laguerre polynomials. The possibility of expanding exp(ϰ) depends on the point values of the various fields ϰ_ϖ, which are determined by the covariance function of ϰ.

In general, there may also be pragmatic reasons to choose models with non-Gaussian variates. Assuming lognormal K and constant S, for example, the transient flow equation

$$S{\partial }_{t}H-\nabla \cdot K\nabla H=\theta$$ (83)

leads to an analog of Eq. (9)

$$S{\partial }_t{H}_{\alpha }-\sum \nabla \cdot G_{\beta \alpha }\nabla H_{\alpha }={\theta }_{\beta }$$ (84)

in which the full matrix G ≡ (G_βα) remains inconvenient, while choosing instead K ≡ ϰ, with independent variates ϖ governed by gamma distributions (say), would replace the G by a sparse matrix.

9 Conclusions

Using acceleration of MC by Hermite series, and assuming a Gaussian covariance function that decays with separation, we compared the quality of polynomial models for stochastic flow obtained by several methods, including numerical cubature, Nicolaides interpolation, truncated PC expansions, and KLME. Such models seem attractive because when available they might provide fast simulators and can be used to compute covariance functions.

We examined an analytic problem, showing that numerical cubature techniques can provide high quality results. Assessment by Monte Carlo, however, suggests the existing numerical cubatures are potentially useful only when the domains are relatively small (as measured in correlation lengths). The cubatures provide decouplings of certain truncated PC systems, and these decouplings suffer from a dimensional curse: system size grows exponentially with the number of variates required and so the methods rapidly become intractable. The method of DOV requires commuting truncations of the Jacobi matrices, available only for rectangular truncation sets. A collocation by cubature attack for non-rectangular truncation sets will either require more function evaluations than the size of the truncation set, force the approximations to the truncated Jacobi matrices to commute by miscomputing entries, or fail to respect the orthogonality of the h-polynomials associated with the truncation set. Prospects for generalizing the decouplings therefore seem limited.

Nicolaides interpolation does not appear to produce satisfactory results if applied without careful attention to knot location. In general, interpolatory methods could reduce the dimensional problems associated with increasing domain size, provided the knots and weights can be obtained without substantial overhead costs. Unless such difficulties are resolved, we expect interpolatory methods will not be useful for stochastic flow problems.

Truncated PC systems can provide accurate results. Because the relative error for a fixed truncation increases with increasing domain size and variance, computational resources required for a given accuracy similarly increase. There remains, however, the matter of appropriately choosing the truncation set. KLME typically provides somewhat less accurate results than PC. In particular, the relative error associated with a given truncation still increases with domain size and variance. It nevertheless has the advantage that it can be implemented adaptively and requires only solution of modified versions of the original equation. Conditions under which KLME converges seem to be unknown. Our investigations further indicate that on large domains, meaningful acceleration of MC by Hermite series can require a significant number of terms.

We conclude that when the number of required variates is small, numerical cubature is the method of choice. When the number of required variates eliminates the cubature option, PC should be considered as an alternative if an appropriate truncation can be found consistent with available memory constraints. When such a truncation cannot be identified, KLME should be considered, unless higher order nonlinearities are expected. When such nonlinearities exist, it is unclear whether KLME will converge, and then MC may be the only option. For large domains of a scale of several correlation lengths or larger, which will typically require many variates to resolve, or for high variability cases, Monte Carlo methods will remain the method of choice unless better interpolatory techniques become available.

Appendix A Karhunen-Loève expansion of a random field

The KLE of a random field Ψ on domain Ω may be obtained formally by seeking an expression

$$\mathrm{\Psi }(\mathbf{x},\omega )=\overline{\mathrm{\Psi }}(\mathbf{x})+\sum \omega {\mathrm{\Psi }}_{\omega }(\mathbf{x})\omega$$ (A.1)

in spatially orthogonal functions Ψ_ω(x) and uncorrelated centered variates ω; then

$$E\left[\left(\mathrm{\Psi }-\overline{\mathrm{\Psi }}\right)\omega \right]={\mathrm{\Psi }}_{\omega }E\left[{\omega }^2\right]={\mathrm{\Psi }}_{\omega }{\sigma }_{\omega }^2$$ (A.2)

$${\int }_{\mathrm{\Omega }}{\mathrm{\Psi }}_{\omega }\left(\mathrm{\Psi }-\overline{\mathrm{\Psi }}\right)d\mathbf{x}=\omega {\int }_{\mathrm{\Omega }}{\mathrm{\Psi }}_{\omega }^{2}d\mathbf{x}=\omega {\parallel {\mathrm{\Psi }}_{\omega }\parallel }^2$$ (A.3)

$$\begin{array}{c}{C}_{\mathrm{\Psi }}(\mathbf{x},{\mathbf{x}}^{\prime })=E\left\{\left[\mathrm{\Psi }(\mathbf{x},\mathit{\omega })-\overline{\mathrm{\Psi }}(\mathbf{x},\mathit{\omega })\right]\left[\mathrm{\Psi }({\mathbf{x}}^{\prime },\mathit{\omega })-\overline{\mathrm{\Psi }}({\mathbf{x}}^{\prime },\mathit{\omega })\right]\right\}\\ \phantom{C_{\mathrm{\Psi }}(\mathbf{x},{\mathbf{x}}^{\prime })=E\{\left[\mathrm{\Psi }(\mathbf{x},\mathit{\omega })-\overline{\mathrm{\Psi }}(\mathbf{x},\mathit{\omega })\right]}={\sum }_{\omega }{\mathrm{\Psi }}_{\omega }(\mathbf{x}){\mathrm{\Psi }}_{\omega }({\mathbf{x}}^{\prime }){\sigma }_{\omega }^2\end{array}$$ (A.4)

$$\begin{array}{c}{\int }_{\mathrm{\Omega }}{c}_{\mathrm{\Psi }}(\mathbf{x},{\mathbf{x}}^{\prime }){\mathrm{\Psi }}_{\omega }({\mathbf{x}}^{\prime })d{\mathbf{x}}^{\prime }={\sum }_{\omega \prime }{\mathrm{\Psi }}_{\omega \prime }(\mathbf{x}){\sigma }_{\omega \prime }^2{\int }_{\mathrm{\Omega }}{\mathrm{\Psi }}_{\omega \prime }({\mathbf{x}}^{\prime }){\mathrm{\Psi }}_{\omega }({\mathbf{x}}^{\prime })d{\mathbf{x}}^{\prime }\\ \phantom{{\int }_{\mathrm{\Omega }}{c}_{\mathrm{\Psi }}(\mathbf{x},{\mathbf{x}}^{\prime }){\mathrm{\Psi }}_{\omega }({\mathbf{x}}^{\prime })d{\mathbf{x}}^{\prime }={\sum }_{\omega \prime }{\mathrm{\Psi }}_{\omega \prime }(\mathbf{x}){\sigma }_{\omega \prime }^2{\int }_{\mathrm{\Omega }}}={\sigma }_{\omega }^2{\parallel {\mathrm{\Psi }}_{\omega }\parallel }^2{\mathrm{\Psi }}_{\omega }(\mathbf{x})\end{array}$$ (A.5)

so that $\left({\mathrm{\Psi }}_{\omega ,}{\sigma }_{\omega }^2{\parallel {\mathrm{\Psi }}_{\omega }\parallel }^2\right)$ is an eigenpair of the integral operator

$$L\left[\phi \right]\equiv {\int }_{\mathrm{\Omega }}{c}_{\mathrm{\Psi }}(\mathbf{x},{\mathbf{x}}^{\prime })\phi ({\mathbf{x}}^{\prime })d{\mathbf{x}}^{\prime }$$ (A.6)

having as its kernel the covariance function c_Ψ. The relative sizes of the corresponding eigenvalues naturally describe the relative importance of the variates ω. Truncating a KLE produces a KLE of the field defined by the truncation. When Ψ is a Gaussian field, the variates ω are also normal and, renormalizing, we may assume ω ~ N(0, 1), so ∥ Ψ_ω∥² is the eigenvalue associated with the eigenfunction Ψ_ω. Since, the family ω is uncorrelated and jointly normal, it comprises independent variates. The KLE applies equally well to vector processes [29]. For further details on KLE, see Adler [2], Ghanem and Spanos [25], Hernandez [31] and Riesz and Szent-Nagy [§§97–98: 63].

Appendix B Notational Examples

The entry of the univariate X-matrix with a = 1, b = 2, c = 3 can be computed as

$$X_{bc}^a=E\left[h_a(\omega )h_b(\omega )h_c(\omega )\right]=E\left[\omega \times \frac{{\omega }^{2-1}}{\sqrt{2}}\times \frac{{\omega }^3-3\omega }{\sqrt{6}}\right]$$ (B.1)

Such calculations need not be performed explicitly: the results are known and are provided in Appendix C

For ω = {ω₁, ω₂, ω₃, ω₄} and $\alpha \equiv {\omega }_1^3{\omega }_3{\omega }_4^2$, the vector of exponents of α is 〈α_ω1, α_ω2, α_ω3, α_ω4 〉 = 〈3,0,1,2 〉 and

$$h_{\alpha }(\mathit{\omega })\equiv h_3({\omega }_1)h_0({\omega }_2)h_1({\omega }_3)h_2({\omega }_4)=\frac{{\omega }_1^3-3{\omega }_1}{\sqrt{6}}\times 1\times {\omega }_3\times \frac{{\omega }_4^2-1}{\sqrt{2}}$$ (B.2)

with leading term ${\omega }_1^3{\omega }_3{\omega }_4^2$. In practice, such products are needed when using polynomial models, for example to evaluate h_α(ω) for specific numerical values of the variates ω.

Again with the same ω and α, but $\beta \equiv {\omega }_1^2{\omega }_2^3{\omega }_3{\omega }_4^3$ and $\gamma \equiv {\omega }_1{\omega }_2^3{\omega }_3^2{\omega }_4$

$$X_{\beta \gamma }^{\alpha }\equiv X_{b_{{\omega }_1}{c}_{{\omega }_1}}^{a_{{\omega }_1}}{X}_{b_{{\omega }_2}{c}_{{\omega }_2}}^{a_{{\omega }_2}}{X}_{b_{{\omega }_3}{c}_{{\omega }_3}}^{a_{{\omega }_3}}{X}_{b_{{\omega }_4}{c}_{{\omega }_4}}^{a_{{\omega }_4}}=X_{21}^3{X}_{33}^0{X}_{12}^1{X}_{31}^2$$ (B.3)

Again with the same ω and α

$${\omega }_1{h}_{\alpha }(\mathit{\omega })=\sqrt{a_{{\omega }_1}+1}\hspace{0.17em}{h}_{\alpha {\omega }_1}+\sqrt{a_{{\omega }_1}}\hspace{0.17em}{h}_{\alpha /{\omega }_1}=2h_{{\omega }_1^4{\omega }_3{\omega }_4^2}+\sqrt{3}\hspace{0.17em}{h}_{{\omega }_1^2{\omega }_3{\omega }_4^2}$$ (B.4)

while

$${\omega }_2{h}_{\alpha }(\mathit{\omega })=h_{{\omega }_1^3{\omega }_2{\omega }_3{\omega }_4^2}$$ (B.5)

where a_ω2 = 0, because ω₂ does not divide α, and the second term of the three-term relation has therefore been dropped.

Again with the same ω, the truncation set

$$\begin{array}{c}\phantom{\equiv \{1,{\omega }_1,{\omega }_2,{\omega }_3,{\omega }_1^2,{\omega }_1{\omega }_2,{\omega }_1{\omega }_3,{\omega }_2}\tau \left({\omega }_1^2{\omega }_2{\omega }_3\right)=\left[2,1,1,0\right]=\left[2,1,1\right]\\ \equiv \left\{1,{\omega }_1,{\omega }_2,{\omega }_3,{\omega }_1^2,{\omega }_1{\omega }_2,{\omega }_1{\omega }_3,{\omega }_2{\omega }_3,{\omega }_1^2{\omega }_2,{\omega }_1^2{\omega }_3,{\omega }_1{\omega }_2{\omega }_3,{\omega }_1^2{\omega }_2{\omega }_3\right\}\end{array}$$ (B.6)

For ω ≡ {ω₁, ω₂} with the truncation set $\tau (2,2)\equiv \left\{1,{\omega }_1,{\omega }_2,{\omega }_1^2,{\omega }_1{\omega }_2,{\omega }_2^2\right\}$, here is one of the truncated Jacobi matrices:

$$\begin{array}{c}{\stackrel{⌣}{X}}^{{\omega }_1}=\left(\begin{array}{cccccc}{X}_{1,1}^{{\omega }_1}& X_{1,{\omega }_1}^{{\omega }_1}& X_{1,{\omega }_2}^{{\omega }_1}& X_{1,{\omega }_1^2}^{{\omega }_1}& X_{1,{\omega }_1{\omega }_2}^{{\omega }_1}& X_{1,{\omega }_2^2}^{{\omega }_1}\\ X_{{\omega }_1,1}^{{\omega }_1}& X_{{\omega }_1,{\omega }_1}^{{\omega }_1}& X_{{\omega }_1,{\omega }_2}^{{\omega }_1}& X_{{\omega }_1,{\omega }_1^2}^{{\omega }_1}& X_{{\omega }_1,{\omega }_1{\omega }_2}^{{\omega }_1}& X_{{\omega }_1,{\omega }_2^2}^{{\omega }_1}\\ X_{{\omega }_2,1}^{{\omega }_1}& X_{{\omega }_2,{\omega }_1}^{{\omega }_1}& X_{{\omega }_2,{\omega }_2}^{{\omega }_1}& X_{{\omega }_2,{\omega }_1^2}^{{\omega }_1}& X_{{\omega }_2,{\omega }_1,{\omega }_2}^{{\omega }_1}& X_{{\omega }_2,{\omega }_2^2}^{{\omega }_1}\\ X_{{\omega }_1^2,1}^{{\omega }_1}& X_{{\omega }_1^2,{\omega }_1}^{{\omega }_1}& X_{{\omega }_1^2,{\omega }_2}^{{\omega }_1}& X_{{\omega }_1^2,{\omega }_1^2}^{{\omega }_1}& X_{{\omega }_1^2,{\omega }_1{\omega }_2}^{{\omega }_1}& X_{{\omega }_1^2,{\omega }_2^2}^{{\omega }_1}\\ X_{{\omega }_1{\omega }_2,1}^{{\omega }_1}& X_{{\omega }_1{\omega }_2,{\omega }_1}^{{\omega }_1}& X_{{\omega }_1{\omega }_2,{\omega }_2}^{{\omega }_1}& X_{{\omega }_1{\omega }_2,{\omega }_1^2}^{{\omega }_1}& X_{{\omega }_1{\omega }_2,{\omega }_1{\omega }_2}^{{\omega }_1}& X_{{\omega }_1{\omega }_2,{\omega }_2^2}^{{\omega }_1}\\ X_{{\omega }_2^2,1}^{{\omega }_1}& X_{{\omega }_2^2,{\omega }_1}^{{\omega }_1}& X_{{\omega }_2^2,{\omega }_2}^{{\omega }_1}& X_{{\omega }_2^2,{\omega }_1^2}^{{\omega }_1}& X_{{\omega }_2^2,{\omega }_1{\omega }_2}^{{\omega }_1}& X_{{\omega }_2^2,{\omega }_2^2}^{{\omega }_1}\end{array}\right)\\ \phantom{{\stackrel{⌣}{X}}^{{\omega }_1}=}=\left(\begin{array}{cccccc}0& X_{01}^1& 0& 0& 0& 0\\ X_{01}^1& 0& 0& X_{12}^1& 0& 0\\ 0& 0& 0& 0& X_{01}^1{X}_{11}^0& 0\\ 0& X_{21}^1& 0& 0& 0& 0\\ 0& 0& X_{10}^1{X}_{11}^0& 0& 0& 0\\ 0& 0& 0& 0& 0& 0\end{array}\right)=\left(\begin{array}{cccccc}0& 1& 0& 0& 0& 0\\ 1& 0& 0& \sqrt{2}& 0& 0\\ 0& 0& 0& 0& 1& 0\\ 0& \sqrt{2}& 0& 0& 0& 0\\ 0& 0& 1& 0& 0& 0\\ 0& 0& 0& 0& 0& 0\end{array}\right)\end{array}$$ (B.7)

Eq. (38) thus assumes the form

$$\left(\nabla \stackrel{⌣}{\mathit{\kappa }}\cdot \nabla +{\nabla }^2\right)\stackrel{⌣}{\mathbf{H}}=\hspace{0.17em}-{\stackrel{⌣}{\mathbf{G}}}^{\hspace{0.17em}-1}\stackrel{⌣}{\mathit{\theta }}$$ (B.8)

where

$$\check{\mathit{\kappa }}\equiv \overline{\kappa }\mathbf{I}+k_{{\omega }_1}{\stackrel{⌣}{X}}^{{\omega }_1}+k_{{\omega }_2}{\stackrel{⌣}{X}}^{{\omega }_2}$$ (B.9)

I is the identity, H˘ and θ˘ are the truncated vectors with components H_α and θ_α, and G˘⁻¹ is the truncated matrix with components $G_{\alpha \beta }^{-1}$

If Eq.(38) is discretized by finite differences, with domain-sized matrices M(?) discretizing ∇?·∇ and D₂ discretizing ∇², then projecting from the discretized equation produces a system matrix with block decomposition

$$\left(\begin{array}{cccccc}M(\overline{\kappa })+D_2& M({\kappa }_{{\omega }_1})& M({\kappa }_{{\omega }_2})& 0& 0& 0\\ M({\kappa }_{{\omega }_1})& M(\overline{\kappa })+D_2& 0& \sqrt{2}M({\kappa }_{{\omega }_1})& M({\kappa }_{{\omega }_2})& 0\\ M({\kappa }_{{\omega }_2})& 0& M(\overline{\kappa })+D_2& 0& M({\kappa }_{{\omega }_1})& \sqrt{2}M({\kappa }_{{\omega }_2})\\ 0& \sqrt{2}M({\kappa }_{{\omega }_1})& 0& M(\overline{\kappa })+D_2& 0& 0\\ 0& M({\kappa }_{{\omega }_2})& M({\kappa }_{{\omega }_1})& 0& M(\overline{\kappa })+D_2& 0\\ 0& 0& \sqrt{2}M({\kappa }_{{\omega }_2})& 0& 0& M(\overline{\kappa })+D_2\end{array}\right)$$ (B.10)

For total degree truncations τ = τ(e, d), the system matrix involves N(e, d)² blocks, of which only (2e + 1)N(e, d − 1) + N(e − 1, d) are occupied. Thus, for τ = τ(2, 2) only 18 of the 36 blocks are nonzero, while for τ = τ(20, 2) only 1, 071 of the 53, 361 blocks are occupied.

Appendix C Computing Expectations

Note first the rules

$${\partial }_{\omega }{h}_n(\omega )=\sqrt{n}{h}_{n-1}(\omega )$$ (C.1)

$$E[h_n(\omega )f(\omega )]=E[{\partial }_{\omega }^{n}f(\omega )]/\sqrt{n!}$$ (C.2)

when f satisfies certain smoothness and growth conditions [cf. ⁹, ²⁴, ³⁹]), and ∂_ω denotes the differentiation operator.

The univariate triple product expectations $X_{bc}^a$ defined by Eq. (22) can therefore be found explicitly (by Liebniz’ rule, for example): $X_{bc}^a$ vanishes unless t ≡ (a + b + c)/2 is an integer and satisfies t ≥ max(a, b, c), in which case

$$X_{bc}^a=\frac{\sqrt{a!b!c!}}{(t-a)!(t-b)!(t-c)!};$$ (C.3)

This also follows from an exact formula in Szego [72].

The coefficients of the expansion

$$\exp \hspace{0.17em}(c\omega )\equiv {\sum }_{n=0}^{\infty }{\phi }_n{h}_n(\omega )$$ (C.4)

formally satisfy

$$\begin{array}{c}{\phi }_n=E[\exp \hspace{0.17em}(c\omega )h_n(\omega )]=E[{\partial }_{\omega }^n\exp \hspace{0.17em}(c\omega )]/\sqrt{n!}\\ \phantom{{\phi }_n\hspace{0.17em}}=c^{n}E[\exp \hspace{0.17em}(c\omega )]/\sqrt{n!}=c^n\exp \hspace{0.17em}(c^2/2)/\sqrt{n!}\end{array}$$ (C.5)

so

$$\exp \hspace{0.17em}(c\omega )\equiv \exp \hspace{0.17em}(c^2/2){\sum }_{n=0}^{\infty }{c}^n{h}_n(\omega )/\sqrt{n!};$$ (C.6)

this may similarly be found from the Taylor series or from the exponential generating function for the ordinary Hermite polynomials [cf. ⁵¹, ⁶⁵, ⁷²]. Thus

$$E[\exp \hspace{0.17em}(c\omega )h_{\alpha }(\omega )h_b(\omega )]=\exp \hspace{0.17em}(c^2/2){\sum }_{n=0}^{\infty }{c}^n{X}_{ab}^n/\sqrt{n!}$$ (C.7)

A version of Eq. (C.7) can be found in [64].

Using Eq. (C.3), Eq. (C.7) becomes a finite expression

$$\begin{array}{c}\phantom{=\hspace{0.17em}}E[\exp \hspace{0.17em}(c\omega )h_a(\omega )h_b(\omega )]=\exp \hspace{0.17em}(c^2/2){\sum }_{n=0}^{\infty }{c}^n{X}_{ab}^n/\sqrt{n!}\\ =\exp \hspace{0.17em}(c^2/2){\sum }_{t=\max \hspace{0.17em}(a,b)}^{a+b}\frac{\sqrt{a!b!}}{(a+b-t)!(t-a)!(t-b)!}{c}^{2t-a-b}\end{array}$$ (C.8)

The multivariate analog is

$$\begin{array}{c}\phantom{=\exp \hspace{0.17em}(\overline{\kappa }){\mathrm{\Pi }}_{\omega }E[\exp \hspace{0.17em}({\kappa }_{\omega }\omega )}{G}_{\alpha \beta }\equiv E[\exp \hspace{0.17em}(\overline{\kappa }+{\sum }_{\omega }{\kappa }_{\omega }\omega )h_{\alpha }{h}_{\beta }]\\ =\exp \hspace{0.17em}(\overline{\kappa }){\mathrm{\Pi }}_{\omega }E[\exp \hspace{0.17em}({\kappa }_{\omega }\omega )h_{a_{\omega }}(\omega )h_{b_{\omega }}(\omega )]=\exp \hspace{0.17em}(\overline{\kappa }+{\sigma }^2/2)\times \\ \phantom{=\hspace{0.17em}}{\mathrm{\Pi }}_{\omega }{\sum }_{t=\max \hspace{0.17em}(a_{\omega },b_{\omega })}^{a_{\omega }+b_{\omega }}\frac{\sqrt{a_{\omega }!b_{\omega }!}}{(a_{\omega }+b_{\omega }-t)!(t-a_{\omega })!(t-b_{\omega })!}{\kappa }_{\omega }^{2t-a_{\omega }-b_{\omega }}\end{array}$$ (C.9)

where the powers $k_{\omega }^n$ are calculated pointwise and

$${\sigma }^2(\mathbf{x})=c_{\kappa }(\mathbf{x},\mathbf{x})={\sum }_{\omega }{k}_{\omega }^2(\mathbf{x})$$ (C.10)

Expressions for the fields $G_{\alpha \beta }^{-1}$, defined by Eq. (13), can be obtained by replacing the components of κ by those of −κ in expressions for the fields G_αβ.

Similarly, the various E[α] are known from the univariate recurrence

$$E[{\omega }^n]=E[\omega {\omega }^{n-1}]=E[\partial {\omega }^{n-1}]=(n-1)E[{\omega }^{n-2}]$$ (C.11)

obtained with Eq. (C.2) [8]. Thus the basis-change coefficients defined by

$$\begin{array}{c}\beta ={\sum }_{\alpha }{P}_{\alpha \beta }{h}_{\alpha }\\ h_{\beta }={\sum }_{\alpha }{Q}_{\alpha \beta }\alpha \end{array}$$ (C.12)

are also easily computable: P_αβ vanishes unless α ≺ β with β/α a square, in which case

$$P_{\alpha \beta }\equiv {\mathrm{\Pi }}_{\omega }\frac{b_{\omega }!}{(b_{\omega }-a_{\omega })!\sqrt{a_{\omega }!}}E[{\omega }^{b_{\omega }-a_{\omega }}]$$ (C.13)

while

$$Q_{\alpha \beta }\equiv P_{\alpha \beta }{\mathrm{\Pi }}_{\omega }\frac{{(-1)}^{(a_{\omega }+b_{\omega })/2}}{\sqrt{a_{\omega }!b_{\omega }!}}$$ (C.14)

Appendix D Basic Properties of X^α

D.1 The Univariate Case

We summarize the theory of the finite Jacobi matrices X˘¹. The eigenvalues of X˘¹ are the zeros z of h_d+1 with associated eigenvectors

$${\nu }_z\equiv {[h_0(z),h_1(z),h_2(z),\cdots ,h_d(z)]}^T.$$ (D.1)

as can be shown, for example, by modifying an argument in [60]. By the Christoffel-Darboux formula, distinct eigenvalues z, z′ produce orthogonal eigenvectors ν_z, ν_z′, and the norms

$${\parallel {\nu }_z\parallel }^2=1/W_z$$ (D.2)

provide the inverse weights for the HG quadrature with knots z [72]. Thus the orthogonal diagonalizing matrix U for X˘¹ has entries

$$U_{az}\equiv h_a(z)C_z$$ (D.3)

where z are the knots and

$$C_z=\sqrt{W_z}$$ (D.4)

the roots of the corresponding weights for the HG quadrature. The polynomials

$$p_z(\omega )=C_z{\sum }_a{U}_{az}{h}_a(\omega )$$ (D.5)

satisfy p_z(z′) = δ_zz′ and provide a polynomial interpolation scheme

$$f(\omega )\approx {\sum }_{z}f(z)p_z(\omega )$$ (D.6)

If the entries of X˘^c are approximately computed by the corresponding HG quadrature, matrices

$${\stackrel{\sim }{X}}^c\equiv ({\stackrel{\sim }{X}}_{ab}^c)=[{\sum }_z{h}_c(z)U_{az}{U}_{bz}]$$ (D.7)

result with X˜¹ = X˘¹, since the quadrature is exact to degree 2d + 1 [6, 68]; thus U diagonalizes X˜^c, so X˜^c = h_c(X˘¹) and the eigenpairs are known from those of X˘¹.

D.2 The Multivariate Case

For the multivariate case, define infinite matrices X^α as tensor products

$$X_{\beta \gamma }^{\alpha }\equiv E[h_{\alpha }(\mathit{\omega })h_{\beta }(\mathit{\omega })h_{\gamma }(\mathit{\omega })]={\mathrm{\Pi }}_{\omega }{X}_{b_{\omega }{c}_{\omega }}^{a_{\omega }};$$ (D.8)

Given a rectangular truncation set, choose (as in D.1 supra) a diagonalizing matrix U^ω for each stochastic dimension, and define

$$U_{\alpha \beta }^{\tau }\equiv {\mathrm{\Pi }}_{\omega }{U}_{a_{\omega }b\omega }^{\omega }.$$ (D.9)

The multivariate interpolation is formally identical to the univariate version given by (D.5).

D.3 Simultaneous Diagonalizability of the X̆^ω

D.3.1 Rectangular Truncation

In the case of a rectangular truncation set, each X̆^ω is a tensor product of some truncated univariate matrix X̆¹ with identity matrices, so the X̆^ω are simultaneously diagonalizable. We show conversely that the truncated matrices X̆^ω can all be simultaneously diagonalized only for truncation sets τ that are rectangular.

Since the X̆^ω can be simultaneously diagonalized iff they commute, consider distinct variates ω ≠ ω′ and study the commutator. The matrices X^ω and X^ω′ commute, since

$$\begin{array}{r}{(X^{\omega }{X}^{{\omega }^{\prime }})}_{\alpha \beta }={\sum }_{\gamma }E[\omega h_{\alpha }{h}_{\gamma }]E[{\omega }^{\prime }{h}_{\gamma }{h}_{\beta }]=E[\omega h_{\alpha }{\sum }_{\gamma }{h}_{\gamma }E[{\omega }^{\prime }{h}_{\gamma }{h}_{\beta }]]\\ =E[\omega {\omega }^{\prime }{h}_{\alpha }{h}_{\beta }]={(X^{{\omega }^{\prime }}{X}^{\omega })}_{\alpha \beta }\end{array}$$ (D.10)

Therefore, for α ∈ τ and β ∈ τ,

$$0={[X^{\omega },X^{{\omega }^{\prime }}]}_{\alpha \beta }={[{\stackrel{⌣}{X}}^{\omega },{\stackrel{⌣}{X}}^{{\omega }^{\prime }}]}_{\alpha \beta }+{\sum }_{\gamma \notin \tau }(X_{\alpha \gamma }^{\omega }{X}_{\gamma \beta }^{{\omega }^{\prime }}-X_{\alpha \gamma }^{{\omega }^{\prime }}{X}_{\gamma \beta }^{\omega })$$ (D.11)

or

$${[{\stackrel{⌣}{X}}^{\omega },{\stackrel{⌣}{X}}^{{\omega }^{\prime }}]}_{\alpha \beta }={\sum }_{\gamma \notin \tau }(X_{\alpha \gamma }^{{\omega }^{\prime }}{X}_{\gamma \beta }^{\omega }-X_{\alpha \gamma }^{\omega }{X}_{\gamma \beta }^{{\omega }^{\prime }})$$ (D.12)

A term $X_{\alpha \gamma }^{{\omega }^{\prime }}{X}_{\gamma \beta }^{\omega }$ (with α ∈ τ, β ∈ τ, and γ ∉ τ) vanishes except when αω′ = γ = βω; similarly, a term $X_{\alpha {\gamma }^{\prime }}^{\omega }{X}_{{\gamma }^{\prime }\beta }^{{\omega }^{\prime }}$ vanishes unless αω = γ′ = βω′. If both αω′ = βω and αω = βω′, then

$$\beta {\omega }^2=\alpha {\omega }^{\prime }\omega =\alpha \omega {\omega }^{\prime }=\beta {{\omega }^{\prime }}^2$$

which is impossible since ω ≠ ω′. The right side of Eq. (D.12) therefore contains at most one term, so the matrices can commute only if the right side is empty.

It follows that the truncated matrices commute iff the following condition holds for every pair of distinct variates ω ≠ ω′ and every monomial μ: whenever μ, ωμ, and ω′μ all belong to τ, so does ωω′μ.

This implies that α ∨ β, defined by (α ∨ β)_ω ≡ max(a_ω, b_ω), belongs to τ whenever α and β do: for suppose that this is true whenever the total degree ∣α∨β∣ < n and consider α and β in T with ∣α∨β∣ = n; there is nothing to show if α = α ∨ β or β = α ∨ β, so we may assume there are variates ω ≠ ω′ with a_ω > b_ω and a_ω′ < b_ω′; by the induction hypothesis α/ω ∨ β = (α ∨ β)/ω and α∨β/ω′ = (α∨β)/ω′ both belong to τ; since τ is a truncation set, (α∨β)/(ωω′) also belongs to τ; hence, by the condition in the prior paragraph, so does α∨β. Therefore τ is rectangular: pick, for each variate ω an element μ(ω) ∈ τ with largest possible ω-degree and set μ = μ(ω₁) ∨⋯∨ μ (ω_e); then by the previous paragraph, μ belongs to τ; but then by construction, τ contains precisely the α ≺ μ and is therefore rectangular. To our knowledge, such questions have not been considered, except in the case of the total degree truncations.

D.3.2 Generality of the Argument

The preceding argument remains valid with some generality. Suppose, for example, given a finite collection of uncorrelated (but not necessarily independent) variates ϖ. To avoid heavy tails and densities concentrated at a finite number of points, require 0 < E[p(ϖ)²] < ∞ for all polynomials p. After shifting and rescaling, we may assume each variate centered with unit variance. By copying standard univariate arguments, obtain a three-term recurrence defining an orthonormal family of multivariate polynomials π_α(ϖ). After introducing as before infinite matrices

$$X^{\alpha }\equiv (X_{\beta \gamma }^{\alpha })\equiv (E[{\pi }_{\alpha }{\pi }_{\beta }{\pi }_{\gamma }])$$ (D.13)

direct computation shows the infinite matrices X^ϖ commute. Inspecting the above arguments then reveals that the truncations X˘^ϖ commute when and only when the truncation is rectangular.

D.4 Remark on Truncation and Cubature

Here is a consequence of the foregoing remarks.

Suppose a truncation set τ admits a cubature rule

$$E[f]\approx I[f]\equiv {\sum }_{\zeta }f(\zeta )W_{\zeta }$$ (D.14)

satisfying the following conditions: (1) the number of knots ζ is the same as the number of monomials in τ; (2) I[1] = 1 and for every α and β in τ and every ω, $I[\omega h_{\alpha }{h}_{\beta }]=X_{\alpha \beta }^{\omega }$. Then τ is rectangular.

To see this, observe first I[h_αh_β] = δ_αβ for α and β in τ . This is clear if either α or β is linear in any ω or if h_α = h_β = 1. Proceeding inductively, we may assume some ω² ≺ α and apply the three-term relation of Eq. (27) to write

$$\begin{array}{c}I[h_{\alpha }{h}_{\beta }]=I[\omega h_{\alpha /\omega }{h}_{\beta }]/\sqrt{a_{\omega }}+I[h_{\alpha /{\omega }^2}{h}_{\beta }]\sqrt{({\alpha }_{\omega }-1)/{\alpha }_{\omega }}\\ E[\omega h_{\alpha /\omega }{h}_{\beta }]/\sqrt{{\alpha }_{\omega }}+E[h_{\alpha /{\omega }^2}{h}_{\beta }]\sqrt{({\alpha }_{\omega }-1)/{\alpha }_{\omega }}=E[h_{\alpha }{h}_{\beta }]\end{array}$$ (D.15)

Choosing $C_{\zeta }=\pm \sqrt{W_{\zeta }}$ and setting U_αζ ≡ h_α(ζ)C_ζ shows that

$${\stackrel{⌣}{X}}^{\omega }={\stackrel{\sim }{X}}^{\omega }={\sum }_{\zeta }{z}_{\omega }{U}_{az}{U}_{bz}$$ (D.16)

so the X̆^ω are simultaneously diagonalizable and τ is rectangular.

The hypothesis I[1] = 1 in condition (2) is equivalent in that context to the assertion that I[h_αh_β] = δ_αβ for all α and β in the truncation set τ.

To generalize the calculations of §4.2, by providing a cubature rule that preserves the orthogonality of the h-polynomials associated with a truncation set τ and with number of knots the size of the truncation set, one must therefore find a cubature that correctly computes ${\stackrel{\sim }{X}}_{\alpha \beta }^{\omega }={\stackrel{⌣}{X}}_{\alpha \beta }^{\omega }$ exactly except possibly when both ωα, ωβ fall outside of τ. If it were possible to selectively damage these entries of the matrices X̆^ω, so the damaged matrices X̃^ω remained symmetric but commuted, then by simultaneously diagonalizing the X̃^ω, one would obtain the knots and weights for a decoupling cubature. But it is not clear how to accomplish this. Additional complications are introduced if one seeks cubature rules involving more knots than polynomials, producing non-rectangular U.

To our knowledge, such questions have not been considered, except in the case of the total degree truncations.