Estimating Varying Coefficients for Partial Differential Equation Models

Summary

Partial differential equations (PDEs) are used to model complex dynamical systems in multiple dimensions, and their parameters often have important scientific interpretations. In some applications, PDE parameters are not constant but can change depending on the values of covariates, a feature that we call varying coefficients. We propose a parameter cascading method to estimate varying coefficients in PDE models from noisy data. Our estimates of the varying coefficients are shown to be consistent and asymptotically normally distributed. The performance of our method is evaluated by a simulation study and by an empirical study estimating three varying coefficients in a PDE model arising from LIDAR data.

1. Introduction

Partial differential equations (PDEs) are used to describe a wide variety of phenomena such as heat, sound, fluid flow and many others. PDEs model these complicated multidimensional dynamical systems by linking some multivariate functions and their partial derivatives together. PDEs are generally much more complicated than ordinary differential equations (ODEs), as ODEs only deal with functions of a single variable. Most PDEs cannot be solved analytically, and many numerical methods have been developed to solve PDEs numerically, for example, the finite element method and finite difference method (Jost, 2002). Parameters in PDEs have important scientific interpretations, and their values determine the behaviour of the dynamical system. However, the values of these parameters are often unknown. There is a need then to estimate these parameters from the measurements or observations of the multidimensional dynamical systems in the presence of measurement errors.

The methods on estimating ODE parameters in the presence of noise have been developed quickly in recent years. Popular methods include the nonlinear least squares approach (Bard (1974), Biegler et al. (1986), Sauer (2006)), the two-step method (Ramsay and Silverman (2005), Chen and Wu (2008), Brunel (2008)), Bayesian methods (Gelman et al. (1996), Huang et al. (2006)), and the parameter cascading method (Ramsay et al. (2007), Cao et al. (2008), Qi and Zhao (2010)). Gugushvili and Klaassen (2012) provided a rigorous analysis of the two-step method. Chen and Wu (2008), Liang et al. (2010), and Cao et al. (2012) proposed the two-step method, the multi-stage smoothing-based approach, and the parameter cascading method, respectively, for estimating time-varying parameters. Xue et al. (2010) considered both numerical error and measurement error and proposed a sieve method for estimating constant and time-varying coefficients in nonlinear ODEs. Cao et al. (2011) proposed a robust estimating method for estimating ODE parameters. Lu et al. (2011) and Wu et al. (2014) proposed multi-stage methods to estimate parameters for high dimensional ODEs. Miao et al. (2014) proposed a generalized ODE model for discrete data. Vujacic et al. (2016) proposed one unifying framework based on generalized Tikhonov regularization and extremum estimation for some existing ODE estimation methods.

The statistical literature on estimating PDE parameters is relatively sparse. Three main methods have been proposed to estimate PDE parameters from noisy data. The first method uses the framework of nonlinear regression by minimizing the distance of the noisy data to the numerical solutions of PDEs for given parameter values (Vallette et al., 1997). Because PDE solutions are very sensitive to parameter values, the nonlinear regression method often has convergence problems, and is computationally costly. This nonlinear regression method also requires the estimation of initial conditions of the PDEs. Müller and Timmer (2002) proposed an extended multiple shooting method to improve the optimization convergence.

The second method is a two-step method (Bär et al., 1999; Voss et al., 1999; Parlitz and Merkwirth, 2000; Coca and Billings, 2000). In the first step, all the state variables and derivatives are estimated from the noisy data by using the multivariate polynomials or nonparametric regression methods such as smoothing splines (Ramsay and Silverman, 2005) or local polynomial regression (Fan and Gijbels, 1996). In the second step, the PDE parameters are estimated by minimizing the distance between the smooth fit and the PDE equations. Both steps have available computer programs to implement, and the computation is also very fast. However, this method depends heavily on the estimation accuracy of derivatives, but derivatives are more difficult to estimate because there are no direct observed data for them. The two-step method also requires that all state variables in the PDEs are observed or measured, which are often hard to achieve in real applications. Müller and Timmer (2004) compared the two-step method with the extended multiple shooting method in their parameter estimates with respect to different noise levels and data resolutions. They concluded that the extended multiple shooting method obtained more accurate parameter estimates than the two-step method, and the superiority of the former method increased with the complexity of the PDE model. The two-step method failed completely when the noise level is higher than a critical level, due to the difficulty of estimating derivatives. Müller and Timmer (2004) also pointed out that the computational cost of the multiple shooting method is a factor of 1.5 × 10³ higher than the two-step method. Dattner and Klaassen (2015) proposed a direct integral approach for estimating ODE parameters and avoided estimating derivatives.

The third method is parameter cascading, which has been shown to obtain more accurate PDE parameter estimates than the two-stage method (Xun et al., 2013). Xun et al. (2013) also established the asymptotic behaviour of the parameter estimates. Frasso et al. (2015) also proposed frequentist and Bayesian approaches for estimating parameters in linear PDEs. Recently, Chkrebtii et al. (2013) considered the uncertainty coming from discretization of system states defined by ODEs or PDEs, and proposed a Bayesian solution to quantify this uncertainty.

The work referenced above assumes that the PDE parameters are constants. In this article, we consider the estimation of PDE parameters which change as functions of covariates, and hence the parameters are varying coefficients. These varying coefficients are often very flexible and easy to interpret, as we show in Section 6. We consider a multidimensional dynamic process g(x,w), where x = (x₁,…, x_p)^T ∈ R^p is a multi-dimensional argument and w is a covariate determining the varying coefficients. Suppose this dynamic process can be modeled with a varying coefficient PDE model

$$ℱ\{\mathbf{x},g(\mathbf{x},w),\frac{\partial g(\mathbf{x},w)}{\partial x_1},\dots ,\frac{\partial g(\mathbf{x},w)}{\partial x_p},\frac{{\partial }^{2}g(\mathbf{x},w)}{\partial x_1\partial x_1},\frac{{\partial }^{2}g(\mathbf{x},w)}{\partial x_1\partial x_2},\dots ;\mathit{\theta }(w)\}=0,$$ (1)

where θ(w) = {θ₁(w),…, θ_m(w)}^T is the varying coefficient vector depending on the scalar w. In practice, we do not observe g(x, w) but instead its surrogate Y(x, w). We assume that g(x, w) are observed over a meshgrid of size n, with measurement errors, and that w is observed at a = 1,…, A locations. Thus, for i = 1,…, n and a = 1,…, A, we observe the data (Y_ai,x_ai, w_a) satisfying

$$Y_{ai}=g({\mathbf{x}}_{ai},w_a)+{\epsilon }_{ai},$$ (2)

where the ε_ai's are independent and identically distributed measurement errors and ε_ai is assumed here to follow a distribution with mean zero and variance σ². Our goal is to estimate the varying coefficient vector θ(w) in the PDE model (1) from noisy data, and to quantify the uncertainty of the estimates.

We propose a parameter cascading method to estimate the varying coefficients in PDE models from noisy data (2) in two nested levels of optimization. Without knowing the boundary conditions of the PDEs, it is impossible to obtain the numerical solutions to PDEs. To address this problem, we approximate the PDE solutions using flexible nonparametric functions, which are expressed as linear combinations of basis functions. For given values of the varying coefficients in PDEs, we estimate the basis coefficients by minimizing the tradeoff between fitting to the data and fidelity to the PDE models in the inner optimization level. Therefore, the estimated basis coefficients, as well as the estimated PDE solution, can be treated as a function of PDE varying coefficients.

Without strong parametric assumptions, we also represent the varying coefficients using flexible nonparametric functions. In the outer optimization level, we estimate the varying coefficients by minimizing the penalized squared errors of the estimated PDE solution to the noisy data in (2). The smoothness of the varying coefficients is controlled by adding a roughness penalty to the squared fitting errors.

The rest of the article is organized as follows. Section 2 introduces our method for estimating varying coefficients in PDEs. Section 3 demonstrates the limiting distribution of the estimates of the varying coefficients. Smoothing parameter selection is described in Section 4. Simulation studies are presented in Section 5 to evaluate the finite performance of our method. Our method is illustrated using LIDAR data in Section 6. Section 7 provides some concluding remarks. Conditions and proofs for the asymptotic theory are provided in the Supplementary Materials, as are the computer codes for our simulation studies.

2. Method for Estimating Varying Coefficients in PDEs

We estimate the varying coefficient vector θ(w) in the varying-coefficient PDE model (1) using the parameter cascading method (Cao and Ramsay, 2007, 2010) in two nested levels of optimization. In the inner level of optimization, we estimate the (p + 1)-dimensional dynamical process g(x, w) for any given value of the varying coefficient vector θ(w). Hence, the estimated ĝ(x, w) can be viewed as a function of θ(w), which is denoted as ĝ {x, w, θ(w)}. In the outer level of optimization, we estimate the varying coefficient vector θ(w) by plugging in ĝ{x, w, θ(w)}. The inner level of optimization of g(x, w) for any given θ(w) is embedded inside the outer level of optimization of θ(w). The two levels of optimization are implemented iteratively until convergence. The algorithm for the parameter cascading method is also outlined at the end of Section 2.2.

2.1 Inner Level of Optimization: Estimation of g(x, w)

Here we describe the estimation of g(x, w) in (2) for a given value of θ(w). The (p + 1)-dimensional dynamical process, g(x, w), is represented by $g(\mathbf{x},w)={\sum }_{k=1}^K{b}_k(\mathbf{x}){\beta }_k(w)={\mathbf{b}}^{\mathrm{T}}(\mathbf{x})\beta (w)$, where b(x) = {b₁(x),…,b_K(x)}^T is a vector of basis functions depending on x, and β(w) = {β₁(w), …, β_k(w)}^T is the corresponding vector of basis coefficients depending on w. We further estimate each β_k(w) as a linear combination of basis functions

$${\beta }_k(w)={\sum }_{u=1}^{V_k}{\varphi }_{ku}(w){\alpha }_{ku}={\mathit{\varphi }}_k^{\mathrm{T}}(w){\mathit{\alpha }}_k,$$ (3)

where ϕ_k(w) = {ϕ_k1(w),…, ϕ_kVk(w)}^T is an additional vector of basis functions, and α_k = (α_k1,…, α_kVk)^T is the corresponding vector of basis coefficients. Let $\mathit{\alpha }={({\mathit{\alpha }}_1^{\mathrm{T}},\dots ,{\mathit{\alpha }}_K^{\mathrm{T}})}^{\mathrm{T}}$ and $\mathbf{\Phi }(w)=diag\{{\mathit{\varphi }}_1^{\mathrm{T}}(w),\dots ,{\mathit{\varphi }}_K^{\mathrm{T}}(w)\}$, where diag(A₁,…, A_K) means a block-diagonal matrix with diagonal elements being A₁,…, A_K. Then, from (3), β(w) = Φ(w)α and

$$g(\mathbf{x},w)={\mathbf{b}}^{\mathrm{T}}(\mathbf{x})\mathbf{\Phi }(w)\mathit{\alpha }.$$ (4)

For any given varying coefficient vector, θ(w), the vector of all basis coefficients α is estimated by minimizing

$$J\{\mathit{\alpha }|\mathit{\theta }(w)\}=\sum _{a=1}^A\sum _{i=1}^n{\{Y_{ai}-{\mathbf{b}}^{\mathrm{T}}({\mathbf{x}}_{ai})\mathbf{\Phi }(w_a)\mathit{\alpha }\}}^2+\frac{{\lambda }_1}{A}\sum _{a=1}^A\int {[ℱ\{\mathbf{x},g(\mathbf{x},w_a),\frac{\partial g(\mathbf{x},w_a)}{\partial x_1},\dots ,\frac{\partial g(\mathbf{x},w_a)}{\partial x_p},\frac{{\partial }^{2}g(\mathbf{x},w_a)}{\partial x_1\partial x_1},\frac{{\partial }^{2}g(x,w_a)}{\partial x_1\partial x_2},\dots ;\mathit{\theta }(w_a)\left\}\right]}^{2}d\mathbf{x},$$ (5)

where the first term measures the fit of (4) to the data, and the second term measures the fidelity of (4) to the PDE model defined in (1). The smoothing parameter λ₁ controls the trade-off between fitting to the data and fidelity to the PDE model.

The estimated basis coefficients α̂ can be obtained for any given varying coefficient vector θ(w) by minimizing (5), so α̂ can be viewed as a function of θ(w), which is denoted as α̂ {θ(w)}. The (p + 1)-dimensional dynamical process can then be estimated as

$$\widehat{g}\{\mathbf{x},w,\mathit{\theta }(w)\}={\mathbf{b}}^{\mathrm{T}}(\mathbf{x})\mathbf{\Phi }(w)\widehat{\alpha }\{\theta (w)\}.$$ (6)

2.2 Outer Level of Optimization: Estimation of θ(w)

The PDE parameter θ(w) is then estimated in the outer level of optimization. For k = 1, …, m, write each θ_k(w) as a linear combination of basis functions

$${\theta }_k(w)={\sum }_{u=1}^{U_k}{\phi }_{ku}(w)c_{ku}={\mathit{\phi }}_k^{\mathrm{T}}(w){\mathbf{c}}_k,$$ (7)

where ψ_k(w) = {ψ_k1(w), …, ψ_kUk(w)}^T is the vector of basis functions, and c_k = (c_k1,…, c_kUk) is the vector of basis coefficients. Because the estimator α̂ {θ(w)} is already regularized, we propose to estimate θ(w) by minimizing the penalized least squares measure of fit

$$H\{\mathit{\theta }(w)\}=\sum _{a=1}^A\sum _{i=1}^n{[Y_{ai}-\widehat{g}\{{\mathbf{x}}_{ai},w_a,\mathit{\theta }(w_a)\}]}^2+{\lambda }_2\sum _{k=1}^m\int {\left\{\frac{d^2{\theta }_k(w)}{d{w}^2}\right\}}^{2}dw,$$ (8)

where λ₂ is a smoothing parameter, which controls the roughness of the estimated varying coefficients θ(w).

Write $\mathbf{c}={({\mathbf{c}}_1^{\mathrm{T}},\mathrm{\dots },{\mathbf{c}}_m^{\mathrm{T}})}^{\mathrm{T}}$ and $\mathbf{\Psi }(w)=diag\{{\phi }_1^{\mathrm{T}}(w),\dots ,{\phi }_m^{\mathrm{T}}(w)\}$ such that

$$\mathit{\theta }(w)=\mathbf{\Psi }(w)\mathbf{c}.$$ (9)

Let Ω_k be a U_k × U_k penalty matrix with the (u₁,u₂)^th entry

$$\int \frac{d^2{\phi }_{k{u}_1}(w)}{d{w}^2}\frac{d^2{\phi }_{k{u}_2}(w)}{d{w}^2}dw$$

and Ω = diag(Ω₁,…, Ω_m). Following (8), we estimate c by minimizing

$$H(\mathbf{c})={\sum }_{a=1}^A{\sum }_{i=1}^n{[Y_{ai}-\widehat{g}\{{\mathbf{x}}_{ai},w_a,\mathbf{\Psi }(w_a)\mathbf{c}\}]}^2+{\lambda }_2{\mathbf{c}}^{\mathrm{T}}\mathbf{\Omega }\mathbf{c}.$$ (10)

The function H(c) can be optimized by the Newton-Raphson method, and the gradient and Hessian functions can be calculated analytically. Magnus and Neudecker (2007) is an excellent reference book for matrix differential calculus. Since ĝ{x_ai, w_a, Ψ(w_a)c} may have no analytical form, it may be an implicit function of c, in which case the implicit function theorem can be applied to obtain the gradient and Hessian functions analytically. Ramsay et al. (2007) provided some detailed expressions for this optimization procedure for ordinary differential equations, which can be directly extended to partial differential equations. Denote the minimizer of (10) by $\widehat{\mathbf{c}}={({\widehat{\mathbf{c}}}_1^{\mathrm{T}},\dots ,{\widehat{\mathbf{c}}}_m^{\mathrm{T}})}^{\mathrm{T}}$; the estimates of varying coefficients are then given by ${\widehat{\theta }}_k(w)={\mathit{\phi }}_k^{\mathrm{T}}(w){\widehat{\mathbf{c}}}_k$ and θ̂(w) = {θ̂₁(w),…, θ̂_m(w)}^T.

Below is the algorithm for the parameter cascading method.

Algorithm 1 Parameter Cascading Method

Outer Level of Optimization - Estimate θ(w) by minimizing H {θ(w)} Initialize θ(w) = θ(0)(w). Inner Level of Optimization - Estimate g(x, w) by minimizing J {α|θ(w)} Initialize α = α(0). Update α(i+1) = α(i) – {(d2 J) / (dαdαT)}−1(dJ/dα) until convergence. Obtain the estimate ĝ{x, w, θ(w)} = bT(x)Φ(w)α̂{θ(w)}. Update c(j+1) = c(j) – {(d2H)/(dcdcT)}−1(dH/dc) until convergence. Obtain the estimate for the varying coefficients: ${\widehat{\theta }}_k(w)={\mathit{\phi }}_k^{\mathrm{T}}(w){\widehat{\mathbf{c}}}_k$ and θ̂(w) = {θ̂1(w),…, θ̂m(w)}T.

2.3 Specialization to Linear PDE Models

The PDE model (1) can be expressed by substituting (4) into model (1) as

$$ℱ\{\mathbf{x},{\mathbf{b}}^{\mathrm{T}}(\mathbf{x})\mathbf{\Phi }(w)\mathit{\alpha },\{\partial {\mathbf{b}}^{\mathrm{T}}(\mathbf{x})/\partial x_1\}\mathbf{\Phi }(w)\mathit{\alpha },\dots ;\mathit{\theta }(w)\}=0.$$ (11)

When the PDE model (11) is linear in α, it can be expressed as

$$ℱ\{\mathbf{x},{\mathbf{b}}^{\mathrm{T}}(\mathbf{x})\mathbf{\Phi }(w)\mathit{\alpha },\frac{\partial {\mathbf{b}}^{\mathrm{T}}(\mathbf{x})}{\partial x_1}\mathbf{\Phi }(w)\alpha ,\dots ;\mathit{\theta }(w)\}={\mathbf{f}}^{\mathrm{T}}\{\mathbf{x},{\mathbf{b}}^{\mathrm{T}}(\mathbf{x})\mathbf{\Phi }(w),\frac{\partial {\mathbf{b}}^{\mathrm{T}}(\mathbf{x})}{\partial x_1}\mathbf{\Phi }(w),\dots ;\mathit{\theta }(w)\}\mathit{\alpha }=0,$$ (12)

then α̂ {θ(w)} has a closed form and the algorithm can be stated as follows.

By using (12), the inner level of optimization criterion (5) becomes

$$\begin{array}{c}J\{\mathit{\alpha }|\mathit{\theta }(w)\}=\sum _{a=1}^A\sum _{i=1}^n{\{Y_{ai}-{\mathbf{b}}^{\mathrm{T}}({\mathbf{x}}_{ai})\mathbf{\Phi }(w_a)\mathit{\alpha }\}}^2+{\lambda }_1{A}^{-1}\sum _{a=1}^A\int {\mathit{\alpha }}^{\mathrm{T}}\mathbf{f}\{\mathbf{x},{\mathbf{b}}^{\mathrm{T}}(\mathbf{x})\mathbf{\Phi }(w_a),\\ \frac{\partial {\mathbf{b}}^{\mathrm{T}}(\mathbf{x})}{\partial x_1}\mathbf{\Phi }(w_a),\dots ;\mathit{\theta }(w_a)\left\}{\mathbf{f}}^{\mathrm{T}}\right\{\mathbf{x},{\mathbf{b}}^{\mathrm{T}}(\mathbf{x})\mathbf{\Phi }(w_a),\frac{\partial {\mathbf{b}}^{\mathrm{T}}(\mathbf{x})}{\partial x_1}\mathbf{\Phi }(w_a),\dots ;\mathit{\theta }(w_a)\}\mathit{\alpha }d\mathbf{x}.\end{array}$$

Let N = nA, B_a = {b(x_1a), …, b(x_na)}^T for a = 1, …, A, B* = diag(B₁, …, B_A), D = {Φ^T(w₁)…, Φ^T(w_a)}^T, B = B*D being a $N\times {\sum }_{k=1}^K{V}_k$ basis matrix, Y = (Y₁₁, Y₁₂, …, Y_An)^T, and

$$\begin{gathered} \mathbf{R}\{\mathit{\theta }(w),w\}=A^{-1}\sum _{a=1}^A\int \mathbf{f}\{\mathbf{x},{\mathbf{b}}^{\mathrm{T}}(\mathbf{x})\mathbf{\Phi }(w_a),\frac{\partial {\mathbf{b}}^{\mathrm{T}}(\mathbf{x})}{\partial x_1}\mathbf{\Phi }(w_a),\dots ;\mathit{\theta }(w_a)\} \\ \times {\mathbf{f}}^{\mathrm{T}}\{\mathbf{x},{\mathbf{b}}^{\mathrm{T}}(\mathbf{x})\mathbf{\Phi }(w_a),\frac{\partial {\mathbf{b}}^{\mathrm{T}}(\mathbf{x})}{\partial x_1}\mathbf{\Phi }(w_a),\dots ;\mathit{\theta }(w_a)\}d\mathbf{x}. \end{gathered}$$

Then, the inner level of optimization criterion (5) can be expressed in the matrix notation

$$J\{\mathit{\alpha }|\mathit{\theta }(w)\}={(\mathbf{Y}-\mathbf{B}\mathit{\alpha })}^{\mathrm{T}}(\mathbf{Y}-\mathbf{B}\mathit{\alpha })+{\lambda }_1{\mathit{\alpha }}^{\mathrm{T}}\mathbf{R}\{\mathit{\theta }(w),w\}\mathit{\alpha }.$$ (13)

By minimizing (13) for given θ(w), the estimator for α can be obtained in closed form:

$$\widehat{\mathit{\alpha }}\{\mathit{\theta }(w)\}={[{\mathbf{B}}^{\mathrm{T}}\mathbf{B}+{\lambda }_1\mathbf{R}\{\mathit{\theta }(w),w\}]}^{-1}{\mathbf{B}}^{\mathrm{T}}\mathbf{Y}.$$ (14)

In addition, because of (9), we can write α̂{θ(w)} = α̂(c) and R{θ(w), w} = R(c) for simplicity. By substituting into (9), (13) becomes

$$J(\mathit{\alpha }|\mathbf{c})={(\mathbf{Y}-\mathbf{B}\mathit{\alpha })}^{\mathrm{T}}(\mathbf{Y}-\mathbf{B}\mathit{\alpha })+{\lambda }_1{\mathit{\alpha }}^{\mathrm{T}}\mathbf{R}(\mathbf{c})\mathit{\alpha },$$ (15)

and by substituting in (6), (9) and (14), (10) becomes

$$H(\mathbf{c})={[\mathbf{Y}-\mathbf{B}{\{{\mathbf{B}}^{\mathrm{T}}\mathbf{B}+{\lambda }_1\mathbf{R}(\mathbf{c})\}}^{-1}{\mathbf{B}}^{\mathrm{T}}\mathbf{Y}]}^{\mathrm{T}}[\mathbf{Y}-\mathbf{B}{\{{\mathbf{B}}^{\mathrm{T}}\mathbf{B}+{\lambda }_1\mathbf{R}(\mathbf{c})\}}^{-1}{\mathbf{B}}^{\mathrm{T}}\mathbf{Y}]+{\lambda }_2{\mathbf{c}}^{\mathrm{T}}\mathbf{\Omega }\mathbf{c}.$$ (16)

The vector of basis coefficients c can be estimated by minimizing H(c), thus we obtain the estimates for the varying coefficients in the PDE model (1).

3. Limit Distribution of Varying Coefficient Estimator for Linear PDE Models

For simplicity, we denote c₁₁, c₁₂, …, c_mU_m by c₁, c₂,…, c_Q. Let c₀ = (c₀₁, …, c_0Q)^T be the true value of c; see Condition C.2 in the Supplementary Materials for its definition. Let λ̃₁ = λ₁/N, S = N⁻¹B^TB, G(c) = S + λ̃₁R(c), ϒ₁ = E(S), and g = {g(x₁₁, w₁), g(x₁₂, w₁),…, g(x_An, w_A)}^T. For q = 1,…, Q, we define Ṙ_q(c) = ∂R(c)/∂c_q, 𝒱_q(c) = R(c)G⁻¹(c)Ṙ_q(c), and ${\mathcal{W}}_q(\mathbf{c})={\mathcal{V}}_q(\mathbf{c})+{\mathcal{V}}_q^{\mathrm{T}}(\mathbf{c})$. Assume that as N → ∞, λ̃₁R(c) converges to a matrix, denoted by R̃(c). Let ϒ₂ (c) = ϒ₁ + R̃(c), α(c) = N⁻¹G⁻¹(c)B^Tg, and 𝒞(c) = {𝒲₁(c)α(c),…, 𝒲_Q(c)α(c)}^T. Also, let ∧(c) be a Q × Q matrix with the (q₁, q₂)^th element

$${\mathbf{\Lambda }}_{q_1{q}_2}(\mathbf{c})={\mathit{\alpha }}^{\mathrm{T}}(\mathbf{c}){\stackrel{.}{\mathbf{R}}}_{q_1}(\mathbf{c}){\mathbf{\Upsilon }}_2^{-1}(\mathbf{c}){\mathbf{\Upsilon }}_1{\mathbf{\Upsilon }}_2^{-1}(\mathbf{c}){\stackrel{.}{\mathbf{R}}}_{q_2}(\mathbf{c})\mathit{\alpha }(\mathbf{c})+{\mathit{\alpha }}^{\mathrm{T}}(\mathbf{c})\mathbf{R}(\mathbf{c}){\mathbf{\Upsilon }}_2^{-1}(\mathbf{c})\frac{\partial {\stackrel{.}{\mathbf{R}}}_{q_1}(\mathbf{c})}{\partial {\mathbf{c}}_{q_2}}\mathit{\alpha }(c),$$

where $\mathbf{\Delta }={\sigma }^2{\mathbf{\Lambda }}^{-1}({\mathbf{c}}_0)\mathcal{C}({\mathbf{c}}_0){\mathbf{\Upsilon }}_2^{-1}({\mathbf{c}}_0){\mathbf{\Upsilon }}_1{\mathbf{\Upsilon }}_2^{-1}({\mathbf{c}}_0)\mathcal{C}{({\mathbf{c}}_0)}^{\mathrm{T}}{\mathbf{\Lambda }}^{-1}({\mathbf{c}}_0)$.

In the Supplementary Materials, we show that under Conditions (C.1)-(C4)

$$N^{1/2}(\widehat{\mathbf{c}}-{\mathbf{c}}_0)\to \text{Normal}(\mathbf{0},\mathbf{\Delta }),$$ (17)

as N → ∞. Let Π_k be a projection matrix such that c_k = Π_kc. Then, for any w, we have

$$N^{1/2}\{{\widehat{\theta }}_k(w)-{\theta }_k(w)\}\to \text{Normal}\{\mathbf{0},{\mathit{\phi }}_k^{\mathrm{T}}(w){\mathbf{\Pi }}_k\mathbf{\Delta }{\mathbf{\Pi }}_k^{\mathrm{T}}{\mathit{\phi }}_k(w)\},k=1,\dots ,m$$ (18)

as N → ∞. To estimate Δ, we replace c₀ by ĉ and estimate σ² by

$${(N-Q)}^{-1}{\sum }_{a=1}^A{\sum }_{i=1}^n{[Y_{ai}-\widehat{g}\{{\mathbf{x}}_{ai},w_a,\mathbf{\Psi }(w_a)\widehat{\mathrm{c}}\}]}^2,$$

where Q is the dimension of c.

4. Smoothing Parameter Selection

We propose a two-step procedure for the smoothing parameter selection. In the first step we set ${\lambda }_1^{\ast }=N^{2/5}$ and ${\lambda }_2^{\ast }=N^{-4/5}$, both of which satisfy Condition (C.1) in the Supplementary Materials. For any λ₁ and λ₂, we rewrite ĉ as ĉ(λ₁, λ₂) and Δ as Δ(λ₁, λ₂). From (17), we have that as N → ∞, $N^{1/2}\{\widehat{\mathbf{c}}({\lambda }_1^{\ast },{\lambda }_2^{\ast })-{\mathbf{c}}_0\}\to \text{Normal}\{\mathbf{0},\mathbf{\Delta }({\lambda }_1^{\ast },{\lambda }_2^{\ast })\}$.

Let $\widehat{\mathbf{\Delta }}({\lambda }_1^{\ast },{\lambda }_2^{\ast })$ be an estimator of $\mathbf{\Delta }({\lambda }_1^{\ast },{\lambda }_2^{\ast })$. In the second step, we select (λ₁, λ₂) by minimizing

$$\mathcal{D}({\lambda }_1,{\lambda }_2)=N{\{\widehat{\mathbf{c}}({\lambda }_1,{\lambda }_2)-\widehat{\mathbf{c}}({\lambda }_1^{\ast },{\lambda }_2^{\ast })\}}^{\mathrm{T}}{\widehat{\mathbf{\Delta }}}^{-1}({\lambda }_1^{\ast },{\lambda }_2^{\ast })\{\widehat{\mathbf{c}}({\lambda }_1,{\lambda }_2)-\widehat{\mathbf{c}}({\lambda }_1^{\ast },{\lambda }_2^{\ast })\}+N^{-1}{[\mathbf{Y}-\mathbf{B}\widehat{\mathit{\alpha }}\{\widehat{\mathbf{c}}({\lambda }_1,{\lambda }_2)\}]}^{\mathrm{T}}[\mathbf{Y}-\mathbf{B}\widehat{\mathit{\alpha }}\{\widehat{\mathbf{c}}({\lambda }_1,{\lambda }_2)\}].$$ (19)

The first term of 𝒟(λ₁, λ₂) in (19) is to ensure that the resulting estimator of c₀ is √N-consistent.

Let 𝒰 be a candidate set of (λ₁,λ₂), containing $({\lambda }_1^{\ast },{\lambda }_2^{\ast })$. The estimated smoothing parameters are (λ₁,λ₂) = argmin_{(λ1, λ2)∈𝒰}𝒟(λ₁,λ₂). Define 𝒰* be a subset of 𝒰 such that ĉ(λ₁, λ₂) − c₀ = O(N^−1/2) for any (λ₁, λ₂) ∈ 𝒰*. Obviously, 𝒰* is not empty because $({\lambda }_1^{\ast },{\lambda }_2^{\ast })$ belongs to 𝒰*. Assuming $\widehat{\mathbf{\Delta }}({\lambda }_1^{\ast },{\lambda }_2^{\ast })$ converges to a positive definite matrix as N → ∞, we can prove that

$$Pr\{({\widehat{\lambda }}_1,{\widehat{\lambda }}_2)\in {\mathcal{U}}^{\ast }\}\to 1,$$ (20)

as N → ∞. See the Supplementary Materials for the proof of (20). This result means that the estimate ĉ(λ̂₁, λ̂₂) is √N-consistent.

5. Simulations

5.1 Data Generating Mechanism

The following varying coefficient PDE model is used to simulate data

$$\frac{\partial g(t,z,w)}{\partial t}-{\theta }_D(w)\frac{{\partial }^{2}g(t,z,w)}{\partial z^2}-{\theta }_S(w)\frac{\partial g(t,z,w)}{\partial z}-{\theta }_A(w)g(t,z,w)=0$$ (21)

with varying coefficients θ_D(w) = 1 − 0.2(w − 1)², θ_S(w) = 0.1 + 0.02 cos(wπ) and θ_A(w) = 0.1 + 0.02 sin(1.2wπ), where w ∈ {0.1,0.2,…, 2.0}.

Model (21) is a PDE of parabolic type in one space dimension, and is also called a (one-dimensional) linear reaction-convection-diffusion equation. Here θ_D(·) is the diffusion rate, θ_S(·) is the drift rate, and θ_A(·) is the reaction rate.

For each w, the PDE model (21) is solved numerically by setting the boundary condition as g(t,0, w) = 0 and the initial condition as g(0,z,w) = {1 + 0.1 × (20 − z)2}⁻¹ over a meshgrid in the time domain t ∈ [1, 20] and the range domain z ∈ [1, 40].

Thus, for each w, our data is on a 20-by-40 meshgrid in the domain [1, 20] × [1, 40]. The resulting sample size N is 20 × 20 × 40 = 16000. Figure 1 displays the numerical solutions of the PDE (21) when w = 0.3, 1.0 and 1.7.

Figure 1.

3-D plots of the surface of the numerical solution of the PDE (21) when varying w = 0.3, 1, 1.7.

The observed error-prone data (2) is simulated by adding independent and identically distributed Gaussian noise with standard deviation σ to the PDE solutions. We varied σ, as 0.02 and 0.05. These values are the same as the simulation used in Xun et al. (2013).

5.2 Performance of Parameter Cascading

The parameter cascading method was applied to estimate the three varying coefficients in the PDE model (21) from the simulated data. The two smoothing parameters λ₁ and λ₂ were selected by using the method proposed in Section 4. We used B-splines of order 4 to form the basis functions, with 5, 15, and 3 equally spaced knots in t, z and w domains, respectively. The parameter cascading method was also compared with the two-stage method. The two-stage method estimated varying coefficient parameters for the PDE model in two stages. In the first stage, for each w_a, a = 1,…, A, g(x, w_a), and its partial derivatives were estimated by multidimensional penalized signal regression (Marx and Eilers, 2005). In the second stage, similar to (8), we estimated the vector of varying coefficient parameters, θ(w_a), by minimizing

$$\sum _{a=1}^A\int {\left[\mathcal{F}\right\{\mathbf{x},g(\mathbf{x},w_a),\frac{\partial g(\mathbf{x},w_a)}{\partial x_1},\dots ,\frac{\partial g(\mathbf{x},w_a)}{\partial x_p},\frac{{\partial }^{2}g(\mathbf{x},w_a)}{\partial x_1\partial x_1},\frac{{\partial }^{2}g(x,w_a)}{\partial x_1\partial x_2},\dots ;\mathit{\theta }(w_a)\left\}\right]}^{2}d\mathbf{x}+{\lambda }_3\sum _{k=1}^m\int {\left\{\frac{d^2{\theta }_k(w)}{d{w}^2}\right\}}^{2}dw,$$

where θ(w_a) was estimated as a linear combination of basis functions, as defined in (9), and the basis functions for expanding θ(w_a) were chosen as the same as the parameter cascading method. The simulation was repeated 1000 times.

Figures S1 and S2 of the Supplementary Materials shows the estimated varying coefficients θ_D(w), θ_S(w), and θ_A(w) for the PDE model (21) from the simulated data in the first 10 simulation replications in our simulation study using the parameter cascading method and the two-stage method, respectively. It is seen that our parameter cascading method can estimate all three varying coefficients reasonably well, but the two-stage method's estimated varying coefficients θ_D(w) are very different from the true curve. To assess the accuracy of the estimated varying coefficients, we use the squared root of the average squared errors (RASEs), namely

$$\text{RASE}\{\widehat{\theta }(w)\}={[{20}^{-1}{\sum }_{w\in \{0.1,0.2,\dots ,2\}}{\{\widehat{\theta }(w)-\theta (w)\}}^2]}^{1/2}.$$ (22)

Table 1 summarizes the mean, median, and standard deviation of the RASEs for the estimated varying coefficients θ̂_D(w), θ̂_S(w) and θ̂_A(w) in 1000 simulation replicates. It shows that our parameter cascading method outperforms the two-stage method. Particularly noticeable is that the two-stage method yields about ten times larger mean, median, and standard deviation of the RASE than the parameter cascading method does for θ_D(w) and θ_S(w).

Table 1.

Mean, Median, and Standard Deviation (SD) of the square root of the average squared errors (RASEs) defined in (22) for the estimated varying coefficients θ̂_D(w), θ̂_S(w) and θ̂_A(w) in 1000 simulation replicates by using the proposed parameter cascading method (PC) and the two-stage method (TS). Here a is the standard deviation of ε in (2), θ_D(·) is the diffusion rate, θ_S(·) is the drift rate, θ_A(·) is the reaction rate.

		σ = 0.05		σ = 0.02
		PC	TS	PC	TS
RMSE of θ̂D(w)	Mean ×102	1.54	22.76	1.29	15.85
	Median ×102	1.53	22.55	1.28	15.88
	SD ×103	3.53	40.22	1.58	21.29
RMSE of θ̂S(w)	Mean ×103	1.86	14.94	1.09	10.03
	Median ×103	1.85	14.50	1.10	9.90
	SD ×104	6.08	43.58	2.78	21.09
RMSE of θ̂A(w)	Mean ×103	1.88	3.46	1.87	2.22
	Median ×103	1.88	3.38	1.87	2.19
	SD ×104	0.30	7.19	0.11	1.85

For the parameter cascading method, when the standard deviation of Gaussian noises added to simulated data is increased from 0.02 to 0.05, the mean and median of RASEs for θ̂_D(w) and θ̂_S(w) increase around 20% and 71%, respectively, while the mean and median of RASEs for θ̂_A(w) only increase slightly. The standard deviation of RASEs is doubled for θ̂_D(w) and θ̂_S(w) and tripled for θ̂_A(w) when increasing σ from 0.02 to 0.05.

6. Application

6.1 Background

We have access to a small subset of long range infrared light detection and ranging (LIDAR) data. The data set consists of samples collected for 28 aerosol clouds, 14 of them being biological and the other 14 being non-biological. For each sample, there is a transmitted signal that is sent into the aerosol cloud at 19 different laser wavelengths w₁, …, w₁₉, and at T time points. For each wavelength and time point, received LIDAR data were observed at equally spaced ranges z = 1,…, Z. The experiment also included background data which were collected before the aerosol cloud was released. The received LIDAR data from aerosol clouds were then background corrected.

An example of the background-corrected received data for the 13^th aerosol cloud (non-biological) is given in Figure 2. Such data such are well-described by

Figure 2.

Snapshots of the background-corrected received data for the 13^th aerosol cloud (non-biological).

$$\frac{\partial g(t,z,w)}{\partial t}-{\theta }_D(w)\frac{{\partial }^{2}g(t,z,w)}{\partial z^2}-{\theta }_S(w)\frac{\partial g(t,z,w)}{\partial z}-{\theta }_A(w)g(t,z,w)=0,$$ (23)

where the parameters θ_D(w), θ_S(w) and θ_A(w) describe the diffusion rate, the drift rate and the reaction rate, respectively, and they are varying with w. In order to estimate the PDE model (23) from the real data, we take T = 20 time points and Z = 60 range values, so that the sample size N is 20 × 60 × 19 = 22800. We use B-spline basis functions of order 4 constructed with 5 inner knots in the time domain, 17 inner knots in the range domain, and 3 inner knots in the wavelength domain.

6.2 Estimation Results

Figure 3 displays the estimates for the three varying coefficients θ_D(w), θ_S(w) and θ_A(w) for the PDE model (23) from the real data shown in Figure 2. It is obvious that the three varying coefficients are not constant over the range of w. Specifically, the diffusion rate, θ_D(w), slightly decreases with the early laser wavelengths and then levels off. The drift rate, θ_S(w), becomes the smallest around w = 10, and increases with the laser length thereafter. The reaction rate, θ_A(w), decreases throughout most of the wavelengths. We used the theoretical result (18) to obtain the 95% confidence intervals for the three varying coefficients, which are also displayed in Figure 3.

Figure 3.

Estimated curves of θ_D(w) (diffusion rate, top), θ_S(w) (drift rate, center), and θ_A(w) (reaction rate, bottom) for the PDE model (21) and their 95% confidence intervals for the 13^th aerosol cloud. This figure appears in color in the electronic version of this article.

Figures 4 and 5 display the means and variances of the estimated varying coefficients θ_D(w), θ_S(w) and θ_A(w) for biological and non-biological aerosol clouds. It is seen that for θ_D(w) and θ_A(w), the biological clouds have higher mean varying coefficients than the non-biological clouds in the whole range of wavelengths, while for θ_S(w), the former has lower mean varying coefficients than the latter in the whole range of wavelengths. For all three coefficients, the non-biological clouds yields larger variances than the biological clouds in the whole range of wavelengths.

Figure 4.

Mean of the estimated varying coefficients θ̂_D(w) (diffusion rate, left), θ̂_S(w) (drift rate, center), and θ̂_A(w) (reaction rate, right) in the PDE model (21) for 14 biological aerosol clouds (solid lines) and 14 non-biological aerosol clouds (dashed lines). This figure appears in color in the electronic version of this article.

Figure 5.

Sample variance of the estimated varying coefficients θ̂_D(w) (diffusion rate, top left), θ̂_S(w) (drift rate, top right), and θ̂_A(w) (reaction rate, lower left) in the PDE model (21) for 14 biological aerosol clouds (solid lines) and 14 non-biological aerosol clouds (dashed lines). Lower right: p-values to assess the hypothesis of equality of the variances from the two groups of clouds by using the Brown-Forsythe test. Solid line is for estimated curve of θ_D(w) (diffusion rate), dash-dotted line is for estimated curve of θ_S(w) (drift rate), and dashed line is for estimated curve of θ_A(w) (reaction rate). The dotted lines represent p = 0.10 and p = 0.05. This figure appears in color in the electronic version of this article.

We further assessed the equality of the variances of the estimated varying coefficients θ_D(w), θ_S(w) and θ_A(w) for biological and non-biological aerosol clouds by using the Brown-Forsythe test (Brown and Forsythe, 1974), which generally has better control of test level than the Levene's test (Levene, 1960). Figure 5 also shows the pointwise p-values for the Brown-Forsythe test of the equality of the variances of the estimated varying coefficients θ̂_D(w), θ̂_S(w) and θ̂_A(w) for biological and non-biological aerosol clouds. Most p-values are smaller than 0.10 and some of them are even smaller than 0.05. It indicates that the variances of the estimated varying coefficients θ̂_D(w), θ̂_S(w) and θ̂_A(w) for biological and non-biological aerosol clouds are significantly different at most wavelengths. At the significance level of 0.10, the diffusion rate θ(·) shows the significant different variance at the wavelengths in [2,19], while the drift rate θ_S(·) shows the significant different variance at the wavelengths in [2,6] and [13,15]. The reaction rate θ_D(·) shows the significant different variance at wavelengths [1,18].

Next, we employ functional principal components analysis to investigate the major variations of the estimated varying coefficients θ̂_D(w), θ̂_S(w) and θ̂_A(w) for all 28 aerosol clouds. Figure S3 of the Supplementary Materials shows the first two functional principal components for the three estimated varying coefficients for all 28 aerosol clouds. The first functional principal component accounts for 91.5%, 98.9%, and 97.4% of total variation of the estimated θ̂_D(w), θ̂_S(w) and θ̂_A(w) for all 28 aerosol clouds, respectively. The first functional principal components for all three estimated varying coefficients change with the wavelength, but they are positive in the whole range of wavelengths. This may be interpreted as that more than 91% of total variation of these estimated varying coefficients comes from the differences of their weighted means, where the weights are defined by the first functional principal components. The second functional principal components account for 7.3%, 0.7%, and 2.1% of total variation of the estimated θ̂_D(w), θ̂_S(w) and θ̂_A(w) for all 28 aerosol clouds, respectively. For all three varying coefficients, the second functional principal components for θ̂_D(w), θ̂_S(w) and θ̂_A(w) are negative when the wavelength is less than 10, 14, 10, respectively, and is positive with larger wavelengths. This may be interpreted as that the second largest source of variations of the three estimated varying coefficients is the change of the varying coefficients from small wavelengths to large wavelengths.

7. Concluding Remarks

We have proposed a parameter cascading method to estimate varying coefficients in PDE models from noisy data. The method employs two nested levels of optimization. In the inner level of optimization, the PDE solution is approximated by a flexible nonparametric function. For any given value of PDE varying coefficients, the nonparametric function is estimated by minimizing the penalized squared errors of the approximated PDE solution to the noisy data, and the fitting penalty is defined by the fit of the nonparametric function to PDEs. In the outer level of optimization, we estimate the varying coefficients as a flexible nonparametric function and add a roughness penalty to control the smoothness of the estimated varying coefficients.

We show that our estimates for varying coefficients are consistent and asymptotically normally distributed. Our simulation study shows that we can obtain accurate estimates for the varying coefficients in PDEs from finite samples in the presence of noises with various scales. We demonstrate our method by estimating three varying coefficients in a PDE model from the LIDAR data. We find that the biological clouds have higher average diffusion rate and reaction rate than the non-biological clouds, but the biological clouds have lower drift rate through the whole range of wavelengths. On the other hand, the non-biological clouds have larger variances for all three varying coefficients than the biological clouds.

Our method allows us to use any multidimensional basis functions to represent the dynamical systems. In our simulation and application, we use tensor product of B-spline basis functions. For this basis function system, the number of basis functions increases quickly with the dimensions of the dynamical system. We believe that we can increase the computational efficiency of our method by defining more efficient multidimensional basis functions. For example, in a bivariate dynamical system, the fast bivariate smoother method of Xiao et al. (2013) or the spatial spline regression method of Sangalli et al. (2013) are promising in this aspect.