Some theoretical properties of Silverman's method for Smoothed functional principal component analysis

Abstract

Principal component analysis (PCA) is one of the key techniques in functional data analysis. One important feature of functional PCA is that there is a need for smoothing or regularizing of the estimated principal component curves. Silverman's method for smoothed functional principal component analysis is an important approach in situation where the sample curves are fully observed due to its theoretical and practical advantages. However, lack of knowledge about the theoretical properties of this method makes it difficult to generalize it to the situation where the sample curves are only observed at discrete time points. In this paper, we first establish the existence of the solutions of the successive optimization problems in this method. We then provide upper bounds for the bias parts of the estimation errors for both eigenvalues and eigenfunctions. We also prove functional central limit theorems for the variation parts of the estimation errors. As a corollary, we give the convergence rates of the estimations for eigenvalues and eigenfunctions, where these rates depend on both the sample size and the smoothing parameters. Under some conditions on the convergence rates of the smoothing parameters, we can prove the asymptotic normalities of the estimations.

1. Introduction

Principal component analysis (PCA) is one of the key techniques in multivariate analysis and functional data analysis. An important difference between classical PCA and functional PCA is that there is a need for smoothing or regularizing of the estimated principal component curves in functional PCA (see Chapter 9 in Ramsay and Silverman [12]). Many methods have been proposed to estimate the smoothed functional principal components when the sample curves are fully observed. A general overview of these methods and an extensive list of references can be found in Ramsay and Silverman [12]. The reader can find in Ferraty and Vieu [6] more discussions on theoretical aspects and nonparametric methods for functional data analysis. Functional PCA has many important applications. For example, functional principal component regression (see for instance Cardot, Ferraty and Sarda [2]) is a direct application of functional principal coponents analysis.

The approach proposed in Silverman [15] is an important method for smoothing functional PCA (see Chapter 9 in Ramsay and Silverman [12]) due to its theoretical and practical advantages. First, the weak assumptions underlying this method make it applicable to data from many fields. Silverman [15] did not make any assumptions on the mean curves and sample curves. Hence, in addition to data with smooth random curves, this method can be applied to analyze data where the sample curves can be unsmooth or even discontinuous, such as those encountered in financial engineering, survival analysis and other fields. For covariance functions, Silverman [15] only assumed that they have series expansions by their eigenfunctions without imposing smoothing constraint. This is attractive because the covariance functions are continuous but unsmooth in many important models such as stochastic differential equation models in financial engineering and counting process models in survival analysis. Second, Silverman's method controls the smoothness of eigenfunction curves by directly imposing roughness penalties on these functions instead of on sample curves or covariance functions. Furthermore, this approach changes the eigenvalue and eigenfunction problems in the usual L² space to problems in another Hilbert space, the Sobolev space (with a norm different from the usual norm in the Sobolev space). Therefore, many powerful tools from the theory of Hilbert space can be employed to study the properties of this method. Third, this approach incorporates the smoothing step into the step for computing eigenvalues and eigenfunctions. Therefore, this method is computationally efficient with the same computational load as the usual unsmoothed functional PCA. Fourth, the estimates produced by this method are invariant under scale transformations. As pointed out by Huang, Shen and Buja [8], the invariance property under scale transformations should be a guiding principle in introducing roughness penalties to functional PCA.

Despite all these advantages, lack of knowledge about the theoretical properties of this method makes it difficult to generalize it to the situations where the sample curves are only observed at discrete time points. Silverman [15] only proved consistency of the estimations as the sample size goes to infinity and the smoothing parameter goes to zero. Even the existence of the solutions to the successive optimization problems in this method is not established. It is not clear how the estimation errors depend on the sample size and the smoothing parameter. Asymptotic normalities of the estimations also need to be proved. In this paper, we aim to solve these open problems. In Section 2, we give the detailed backgroud, basic notations and our main assumptions. In Section 3, Silverman's method is introduced and the existence theorem for the successive optimization problems is proven. Our main results appear in Section 4. Section 5 contains detailed proofs of our theorems.

2. Notations and main assumptions

We introduce notations and definitions used throughout the paper. Let ℕ denote the collection of all the positive integers. We consider a finite time interval [a, b]. In this paper, we will mainly consider functions in the following two space, the L² space

$$L^2\left([a,b]\right)=\{f:f\text{is a measurable function on}[a,b]\text{and}{\int }_a^b{\left|f\left(t\right)\right|}^2\mathit{dt}<\infty \},$$

and the Sobolev space

$$W_2^2\left([a,b]\right)=\{f:f,f^{\prime }\text{are absolutely continuous on}[a,b]\text{and}{f}^{″}\in L^2\left([a,b]\right)\},$$

where f′ and f″ donate the first and second derivatives of f, respectively. For any f, g ∈ L²([a, b]), define the usual inner product

$$(f,g)={\int }_a^{b}f\left(t\right)g\left(t\right)\mathit{dt},$$

with corresponding squared norm ∥f∥² = (f, f). Given a smoothing parameter α > 0, for any f, $g\in W_2^2\left([a,b]\right)$, define

$$[f,g]={\int }_a^b{f}^{″}\left(t\right)g^{″}\left(t\right)\mathit{dt}$$

and the inner product

$${(f,g)}_{\alpha }=(f,g)+\alpha [f,g]$$

with corresponding squard norm ${\parallel f\parallel }_{\alpha }^2={(f,f)}_{\alpha }$. Note that is α = 0, we return to the L²([a, b]) space. For any bounded operator B from L²([a, b]) to L²([a, b]), define the norm

$$\parallel B\parallel =\sup \{\parallel Bf\parallel :f\in L^2\left([a,b]\right)\text{and}\parallel f\parallel \le 1\}.$$ (2.1)

For any measurable function A(s, t) on [a, b] × [a, b], if

$${\int }_a^b{\int }_a^b{A}^2(s,t)\mathit{dsdt}<\infty ,$$

then $f\to {\int }_a^{b}A(s,t)f\left(t\right)\mathit{dt}$ defines a bounded operator from L²([a, b) to L²([a, b]). To simplify the notation, we just use A to denote this operator, that is

$$Af\left(s\right)={\int }_a^{b}A(s,t)f\left(t\right)\mathit{dt},$$

and we have

$$\parallel A\parallel \le {\left({\int }_a^b{\int }_a^b{A}^2(s,t)\mathit{dsdt}\right)}^{\frac{1}{2}}.$$

Let X(t), a ≤ t ≤ b be a measurable stochastic process on [a, b]. Under Assumption 1 below, X(t) ∈ L²([a, b]) a.s.. Let {X₁(t), X₂(t), ⋯, X_n(t)} be i.i.d. sample curves from the distribution of X(t). Assume that EX(t) = ν(t). Define Γ to be the covariance function

$$\mathrm{\Gamma }(s,t)=E\left[(X\left(s\right)-\nu \left(s\right))(X\left(t\right)-\nu \left(t\right))\right],\forall s,t\in [a,b],$$

and Γ̂_n to be the sample covariance function

$${\widehat{\mathrm{\Gamma }}}_n(s,t)=\frac{1}{n}\underset{p=1}{\overset{n}{\Sigma }}(X_p\left(s\right)-\stackrel{-}{X}\left(s\right))(X_p\left(t\right)-\stackrel{-}{X}\left(s\right)),\forall s,t\in [a,b],$$

where X̄ is the sample mean curve

$$\stackrel{-}{X}\left(t\right)=\frac{1}{n}(X_1\left(t\right)+\cdots +X_n\left(t\right)).$$

We will give our basic assumptions below. Silverman [15] made three assumptions in Section 5.2 in order to prove the consistency result. Our assumptions are stronger than those in Silverman [15].

Assumption 1

$$E\left[{\parallel X\parallel }^4\right]=E\left[{\left({\int }_a^b{\left|X\left(t\right)\right|}^2\mathit{dt}\right)}^2\right]<\infty .$$ (2.2)

Remark

1. This assumption is stronger than the first assumption in Section 5.2 of Silverman [15]. Under condition (2.2), the central limit theorem for sample covariance function holds (see Section 2 in Dauxois, Pousse and Romain [3] and Chapter 10 in Ledoux and Talagrand [10]).

2. Assumption 1 is satisfied by many stochastic processes used in applications. For example, if X(t) is a bounded process, it is obvious that (2.2) is true. Gaussian processes are an important class of stochastic processes which are widely used in statistics and other areas. Suppose that X(t) is a Gaussian process with mean zero. Then

   $$\begin{array}{cc}\hfill & E\left[{\parallel X\parallel }^4\right]=E\left[{\left({\int }_a^b{\left|X\left(t\right)\right|}^2\mathit{dt}\right)}^2\right]={\int }_a^b{\int }_a^{b}E\left[X{\left(t\right)}^{2}X{\left(s\right)}^2\right]\mathit{dtds}\hfill \\ \hfill =& {\int }_a^b{\int }_a^b[\mathrm{\Gamma }(s,s)\mathrm{\Gamma }(t,t)+2\mathrm{\Gamma }{(s,t)}^2]\mathit{dsdt}\le {\int }_a^b{\int }_a^{b}3\mathrm{\Gamma }(s,s)\mathrm{\Gamma }(t,t)\mathit{dsdt}\hfill \\ \hfill =& 3{\left[{\int }_a^b\mathrm{\Gamma }(t,t)\mathit{dt}\right]}^2.\hfill \end{array}$$

   Hence if Γ(t, t) is integrahle in [a, b], which is satisfied by Gaussian processes commonly encountered in applications, (2.2) is true. Now let us consider the standard Brovmian motion, the most widely studied Gaussian process. For the standard Brovmian motion, Γ(t, t) = t, hence Assumption 1 is satisfied. It is well known that its sample paths are continuous and nowhere differentiable almost surely. For non-Gaussian processes, let us consider a Poisson process with rate 1 in [0,1], Its sample paths are step functions only taking integer values and hence discontinuous. It is easy to verify that Assumption 1 is satisfied by Poisson processes.
3. Under condition (2.2), we have

   $$\begin{array}{cc}\hfill & {\int }_a^b{\int }_a^b\mathrm{\Gamma }{(s,t)}^2\mathit{dsdt}={\int }_a^b{\int }_a^b{\left(E\left[(X\left(s\right)-\nu \left(s\right))(X\left(t\right)-\nu \left(t\right))\right]\right)}^2\mathit{dsdt}\hfill \\ \hfill =& {\int }_a^b{\int }_a^b{(\mathit{EX}\left(t\right)X\left(s\right)-\nu \left(s\right)\nu \left(t\right))}^2\mathit{dsdt}\hfill \\ \hfill \le & {\int }_a^b{\int }_a^{b}2{\left(\mathit{EX}\left(t\right)X\left(s\right)\right)}^2+2\nu {\left(s\right)}^2\nu {\left(t\right)}^2\mathit{dsdt}\hfill \\ \hfill \le & {\int }_a^b{\int }_a^{b}2{\mathit{EX}}^2\left(t\right)X^2\left(s\right)+2\nu {\left(s\right)}^2\nu {\left(t\right)}^2\mathit{dsdt}\le 4E\left[{\parallel X\parallel }^4\right]<\infty .\hfill \end{array}$$

   Therefore, the operator Γ is a Hilbert-Schmidt operator, hence it is a compact operator (see Section XI.6 in Dunford and Schwartz [5] or Section 97 in Riesz and Sz.-Nagy [13]). It follows that the set of eigenvalues of this operator are bounded and at most countable with at most one limit point at 0. Because the covariance operator Γ is always nonnegative-definite, all the eigenvalues are nonnegative. Let λ₁ ≥ λ₂ ≥ ⋯ ≥ 0 be the collection of all eigenvalues and the corresponding eigenfunctions are γ₁, γ₂, ⋯. Every eigenfunction has been scaled to have L²-norm, 1. The set of all the eigenfunctions forms an orthonormal basis of L²([a, b]). Furthermore, we have decomposition

   $$\mathrm{\Gamma }(s,t)=\underset{j=1}{\overset{\infty }{\Sigma }}{\lambda }_j{\gamma }_j\left(s\right){\gamma }_j\left(t\right),$$ (2.3)

   the series on the right hand side converges in the L² sense. If Γ is a continuous function, the series on the right hand side absolutely and uniformly converges. Although Silverman [15] did not assume that Γ is square integrahle, he assumed the decomposition form of (2.3).
4. We have

   $$\mathrm{\Gamma }{\gamma }_j={\lambda }_j{\gamma }_j,\forall j=1,2,\cdots .$$
5. By (2.2), X(s) is square integrable a.s.. Hence, the sample covariance functions Γ̂_n satisfies

   $${\int }_a^b{\int }_a^b{\widehat{\mathrm{\Gamma }}}_n{(s,t)}^2\mathit{dsdt}<\infty$$

   a.s.. Then we have that the eigenvalues λ̂₁ ≥ λ̂₂ ≥ ⋯ ≥ 0 since the operator Γ̂ is nonnegative-definite. The corresponding eigenfunctions γ̂_j, j ∈ ℕ satisfying

   $${\widehat{\mathrm{\Gamma }}}_n{\widehat{\gamma }}_j={\widehat{\lambda }}_j{\widehat{\gamma }}_j,\forall j=1,2,\cdots .$$

Suppose that we are interested in estimating the first K eigenvalues and eigenfunctions of Γ.

Assumption 2

Any eigenvalue λ_j, 1 ≥ j ≤ K has multiplicity 1, so that

$${\lambda }_1>{\lambda }_2>\cdots >{\lambda }_K>{\lambda }_{K+1}.$$

Remark

This assumption is just the third assumption in Section 5.2 of Silverman [15]. If an eigenvalue has multiplicity 1, then the corresponding eigen-function is uniquely determined up to a sign. If the multiplicity is larger than 1, the eigenfunctions can not he uniquely determined up to a sign.

Assumption 3

The eigenfunctions λ_j, 1 ≤ j ≤ K belong to $W_2^2\left([a,b]\right)$

Remark

1. This assumption is the second assumption in Section 5.2 of Silverman [15] and is essential in our paper.

2. If the covariance function Γ satisfies some smoothness conditions, then Assumption 3 is true. For example, suppose that Γ(s, t), $\mathrm{\Gamma }(s,t),\frac{\partial \mathrm{\Gamma }(s,t)}{\partial s}$ and $\frac{{\partial }^2\mathrm{\Gamma }(s,t)}{\partial s^2}$ are all continuous on [a, b] × [a, b] (hence they are bounded and square integrable), one can easily verify that

   $${\lambda }_k{\gamma }_k^{″}\left(s\right)={\int }_a^b\frac{{\partial }^2\mathrm{\Gamma }(s,t)}{\partial s^2}{\gamma }_k\left(t\right)\mathit{dt}\forall 1\le k\le K.$$

   Hence, by Cauchy-Schwarz inequality and ∥γ_k∥ = 1, we have

   $${\lambda }_k^2{\int }_a^b{\left({\gamma }_k^{″}\left(s\right)\right)}^2\mathit{ds}\le {\int }_a^b{\int }_a^b{\left(\frac{{\partial }^2\mathrm{\Gamma }(s,t)}{\partial s^2}\right)}^2\mathit{dsdt}<\infty \forall 1\le k\le K.$$
3. There are many important random processes whose covariance matrices are not smooth, but the eigenfunctions corresponding to nonzero eigenvalues belong to $W_2^2\left([a,b]\right)$. The simplest examples are standard Brownian motion and Poisson process with rate 1 in time interval [0, 1]. Their covariance functions are the same and equal to min(s, t), 0 ≤ s, t ≤ 1 (see Page 89 in the book Glasserman [7]). The eigenvalues and eigenfunctions are

   $${\lambda }_j={\left(\frac{2}{(2j-1)\pi }\right)}^2,{\gamma }_j=\sqrt{2}\sin \left(\frac{(2j-1)\pi t}{2}\right),j=1,2,\cdots .$$ (2.4)

   The next example is the famous Black-Scholes Model in finance. Let S_t denote the price of a stock at time t. Then S_t satisfies the following SDE,

   $${\mathit{dS}}_t=\mu S_t\mathit{dt}+\sigma S_t{\mathit{dW}}_t,$$

   where μ, is the instantaneous mean return, σ is the instantaneous return volatility and W_t is a Brownian motion. The covariance function of S_t is smooth except at the points on the diagonal line {(s, t) : s = t}. The same is true for the following example. Consider the counting processes model in survival analysis. Let N_t be the number of the occurrences of the event in [0, t]. Then N_t satisfies

   $${\mathit{dN}}_t=\lambda \left(t\right)\mathit{dt}+{\mathit{dM}}_t,$$

   where λ(t) is a smooth intensity function and M_t is a martingale.

Silverman [15] introduced a “half-smoothing” operator which plays an important role in this paper. We give a strict definition of this operator here. We first define an unbounded operator L in L²([a, b]). The domain of L is 𝒟(L) = {f ∈ L² ([a, b]) : f, f′ are absolutely continous and f″ ∈ L² ([a, b])}, and for any f ∈ 𝒟(L),

$$\mathit{Lf}=f^{″}.$$

Then L is a closed but unbounded operator and 𝒟(L) is dense in L²([a, b]) (for the definition of closed operators, see Chapter VIII of Riesz and Sz.-Nagy [13] or Chapter 13 of Rudin [14]). Let L* be the adjoint operator of L. By the theorem in Section 118 of Riesz and Sz.-Nagy [13] or Theorem 13.13 in Rudin [14], (I + αL*L)⁻¹ is a bounded, positive self-adjoint operator with norm less than or equal to 1, where α ≥ 0 is the smoothing parameter. Now it follows from Theorem 12.33 and 13.31 in Rudin [14] that (I + αL*L)⁻¹ has a unique positive and self-adjoint square root S_α with norm less than or equal to 1 which is the “half-smoothing” operator in Silverman [15]. Therefore,

$$S_{\alpha }^2={(I+\alpha L^{\ast }L)}^{-1},$$ (2.5)

and by Theorem 13.11 (b) in Rudin [14], the inverse $S_{\alpha }^{-1}$ exists and is self-adjoint because (I + αL*L)⁻¹ is invertible.

3. Silverman's approach to smoothed functional PCA

In this section, we always assume that the independent sample curves

$$\{X_1\left(t\right),X_2\left(t\right),\cdots ,X_n\left(t\right):a\le t\le b\}$$

are entirely observed. We first consider the usual population functional principal components. The first population functional principal component is defined as the linear functional ℓ₁(X) of X which maximizes

$$\mathit{Var}\left(\ell \left(X\right)\right)$$

over all nonzero linear functionals ℓ in L²([a, b]) with the norm ∥ℓ∥ = 1. The second population functional principal component is defined as the linear functional ℓ₂(X) of X which maximizes

$$\mathit{Var}\left(\ell \left(X\right)\right)$$

over all linear functional ℓ with the norm ‖ℓ‖ = 1 and uncorrelated with ℓ₁(X). Similarly, we can define all the other population functional principal components, ℓ₃(X), …. Because X takes values in L²([a, b]) which is a real Hilbert space, by the Riesz representation theorem, for any bounded linear functional ℓ, there is a unique γ ∈ L²([a, b]), such that for any f ∈ L²([a, b]),

$$\ell \left(f\right)=(\gamma ,f)\text{and}\parallel \ell \parallel =\parallel \gamma \parallel .$$

Hence there exists γ_j, ∈ L²([a, b]), j ∈ ℕ, with ‖γ_j‖ = 1, such that the population functional principal components ℓ_j(X) = (γ_j, X),j ∈ ℕ. γ_j is called the j-th principal component weight function or j-th principal component curve. Because

$$\mathit{Var}\left({\ell }_j\left(X\right)\right)=\mathit{Var}({\gamma }_j,X)=({\gamma }_j,\mathrm{\Gamma }{\gamma }_j),\forall j\in \mathbb{N},$$

γ₁ is the solution of the following optimization problem,

$$\underset{\parallel \gamma \parallel =1}{\max }\frac{(\gamma ,\mathrm{\Gamma }\gamma )}{{\parallel \gamma \parallel }^2}.$$ (3.1)

The maximum value of (3.1) is just the largest eigenvalue Ai of λ₁ and γ₁ is the corresponding eigenfunction (see Section 2, Chapter 3 in Weinberger [16]). γ₂ is the solution of the optimization problem,

$$\underset{\parallel \gamma \parallel =1,(\gamma ,{\gamma }_1)=0}{\max }\frac{(\gamma ,\mathrm{\Gamma }\gamma )}{{\parallel \gamma \parallel }^2}.$$ (3.2)

The maximum value of (3.2) is just the second eigenvalue λ₂ of Γ and γ₂ is the corresponding eigenfunction. Similarly, γ_j is the eigenfunction corresponding to the eigenvalue λ_j which is also the variance of the j-th principal component.

Because the covariance function Γ is usually unknown, we can not obtain the population principal component weight functions directly. Hence, people use the sample covariance function Γ̂_n to estimate Γ and use the eigenvalues and eigenfunctions of Γ̂_n to estimate the eigenvalues and eigenfunctions of Γ. We call them non-smooth estimators. However, the non-smooth principal component curves can show substantial variability (see Chapter 9 in Ramsay and Silverman [12]). There is a need for smoothing of the estimated principal component weight functions.

Silverman [15] (see also Chapter 9 in Ramsay and Silverman [12]) proposed a method of incorporating smoothing by replacing the usual L² norm with a norm that takes the roughness of the functions into account. Let α be a nonnegative smoothing parameter. Define the estimators $\{({\widehat{\lambda }}_j^{\left[\alpha \right]},{\widehat{\gamma }}_j^{\left[\alpha \right]}):j\in \mathbb{N}\}$ of {(λ_j, γ_j) : j ∈ ℕ} to be the solutions of the following successive optimization problems. First, ${\widehat{\gamma }}_1^{\left[\alpha \right]}$ is the solution of the optimization problem

$$\underset{\parallel \gamma \parallel =1}{\max }\frac{\left(\gamma ,{\widehat{\mathrm{\Gamma }}}_n\gamma \right)}{(\gamma ,\gamma )+\alpha [\gamma ,\gamma ]}=\underset{\parallel \gamma \parallel =1}{\max }\frac{\left(\gamma ,{\widehat{\mathrm{\Gamma }}}_n\gamma \right)}{{\parallel \gamma \parallel }_{\alpha }^2}.$$ (3.3)

Let ${\widehat{\lambda }}_1^{\left[\alpha \right]}$ be the maximum value of (3.3). For any k ∈ ℕ, if we have obtained $\{{\widehat{\gamma }}_j^{\left[\alpha \right]},j=1,2,\cdots ,k-1\}$ and $\{{\widehat{\lambda }}_j^{\left[\alpha \right]},j=1,2,\cdots ,k-1\}$, ${\widehat{\gamma }}_k^{\left[\alpha \right]}$ is the solution of the optimization problem

$$\underset{\begin{array}{c}\hfill \parallel \gamma \parallel =1,{(\gamma ,{\widehat{\gamma }}_j^{\left[\alpha \right]})}_{\alpha }=0,\hfill \\ \hfill j=1,\cdots ,k-1\hfill \end{array}}{\max }\frac{\left(\gamma ,{\widehat{\mathrm{\Gamma }}}_n\gamma \right)}{{\parallel \gamma \parallel }_{\alpha }^2},$$ (3.4)

and ${\widehat{\lambda }}_k^{\left[\alpha \right]}$ is the maximum value of (3.4). Note that $\{({\widehat{\lambda }}_j^{\left[\alpha \right]},{\widehat{\gamma }}_j^{\left[\alpha \right]}):j\in \mathbb{N}\}$ depends on both the sample size n and the smoothing parameter α.

First of all, we need to show that the solutions $\{({\widehat{\lambda }}_j^{\left[\alpha \right]},{\widehat{\gamma }}_j^{\left[\alpha \right]}):j\in \mathbb{N}\}$ of the successive optimization problems (3.3) and (3.4) exist.

Theorem 3.1

Under Assumption 1, the solutions $\{({\widehat{\lambda }}_j^{\left[\alpha \right]},{\widehat{\gamma }}_j^{\left[\alpha \right]}):j\in \mathbb{N}\}$ of the successive optimization problems (3.3) and (3.4) exist for any α ≥ 0 almost surely. Moreover, we have, for any $\gamma \in W_2^2\left([a,b]\right)$ and j ∈ ℕ,

$$({\widehat{\mathrm{\Gamma }}}_n{\widehat{\gamma }}_j^{\left[\alpha \right]},\gamma )={\widehat{\lambda }}_j^{\left[\alpha \right]}{({\widehat{\gamma }}_j^{\left[\alpha \right]},\gamma )}_{\alpha }.$$ (3.5)

Similarly, define $\{({\lambda }_j^{\left[\alpha \right]},{\gamma }_j^{\left[\alpha \right]}):j\in \mathbb{N}\}$ to be the solutions of the successive optimization problems (3.3) and (3.4) with Γ̂_n replaced by Γ Similarly, we have the following equalities for Γ and $\{({\lambda }_j^{\left[\alpha \right]},{\gamma }_j^{\left[\alpha \right]}):j\in \mathbb{N}\}$

$$(\mathrm{\Gamma }{\gamma }_j^{\left[\alpha \right]},\gamma )={\lambda }_j^{\left[\alpha \right]}{({\gamma }_j^{\left[\alpha \right]},\gamma )}_{\alpha },\forall j\in \mathbb{N},\gamma \in W_2^2\left([a,b]\right)$$ (3.6)

Note that

$${\gamma }_j^{\left[0\right]}={\gamma }_j,{\lambda }_j^{\left[0\right]}={\lambda }_j,{\widehat{\gamma }}_j^{\left[0\right]}={\widehat{\gamma }}_j,{\lambda }_j^{\left[0\right]}={\widehat{\lambda }}_j,\forall j\in \mathbb{N}.$$

Theorem 1 in Silverman [15] gives the consistency of the estimators

$$\{({\widehat{\lambda }}_j^{\left[\alpha \right]},{\widehat{\gamma }}_j^{\left[\alpha \right]}):j\in \mathbb{N}\}$$

as α → 0 and n → ∞.

4. Asymptotic theory

Fix a positive integer K. We will assume throughout this section that we want to estimates the first K principal component curves. For any 1 ≤ k ≤ K, define

$$L_k=\underset{1\le j\le k}{\max }\sqrt{[{\gamma }_j,{\gamma }_j]}.$$

Then under Assumption 3, L_k is finite and is a measure of roughness of the first k eigenfunctions of Γ. For standard Brownian motion and Poisson process with rate 1 (see remark (3) after Assumption 3),

$$L_k={\left(\frac{(2k-1)\pi }{2}\right)}^2,k=1,2,\cdots .$$

For any 1 ≤ k ≤ K, we have decompositions

$${\widehat{\lambda }}_k^{\left[\alpha \right]}-{\lambda }_k=({\widehat{\lambda }}_k^{\left[\alpha \right]}-{\lambda }_k^{\left[\alpha \right]})+({\lambda }_k^{\left[\alpha \right]}-{\lambda }_k),$$ (4.1)

$${\widehat{\gamma }}_k^{\left[\alpha \right]}-{\gamma }_k=({\widehat{\gamma }}_k^{\left[\alpha \right]}-{\gamma }_k^{\left[\alpha \right]})+({\gamma }_k^{\left[\alpha \right]}-{\gamma }_k).$$ (4.2)

The last terms ${\lambda }_k^{\left[\alpha \right]}-{\lambda }_k,{\gamma }_k^{\left[\alpha \right]}-{\gamma }_k$ on the right hand sides of both (4.1) and (4.2) are nonrandom. They are the “bias terms” due to the introduction of α. We will give the upper bounds for norms of these terms. The first terms on the right hand sides of both (4.1) and (4.2) are the “variation terms” due to the randomness of the sample curves. We will prove a functional central limit theorem for these terms. In order to avoid any confusion it should be pointed out that (4.1) and (4.2) are not the bias-variance decompositions in the strict sense because ${\lambda }_k^{\left[\alpha \right]}$ and ${\gamma }_k^{\left[\alpha \right]}$ are not the expectations of ${\widehat{\lambda }}_k^{\left[\alpha \right]}$ and ${\widehat{\gamma }}_k^{\left[\alpha \right]}$ respectively. Since it is hard to express or characterize the exact expectations of ${\widehat{\lambda }}_k^{\left[\alpha \right]}$ and ${\widehat{\gamma }}_k^{\left[\alpha \right]}$, the asymptotic properties of the usual bias and variation terms in the strict sense may not be easily studied. Heuristic calculations of the usual bias and variation terms in the strict sense were performed in Section 6 of Silverman [15].

Note that even if the multiplicity of λ_k is one, we can not uniquely determine γ_k because −γ_k is also an eigenfunction. In the following theorem, by “Given γ_k”, we mean that not only γ_k is an eigenfunction, but also the direction of γ_k is given.

Define

$${\alpha }_0=\underset{1\le k\le K}{\min }\left\{\min \left\{\frac{\sqrt{1+\frac{2k{({\lambda }_{k-1}-{\lambda }_k)}^2}{(k-1){\lambda }_k\parallel \mathrm{\Gamma }\parallel }}-1}{2{\mathit{kL}}_k^2},\frac{{\lambda }_k-{\lambda }_{k+1}}{(8\sqrt{k}+16k)L_k^2{\lambda }_k},\frac{({\lambda }_{k-1}-{\lambda }_k){\left\{1+\frac{2\parallel \mathrm{\Gamma }\parallel }{{\lambda }_k-{\lambda }_{k+1}}\right\}}^{-\frac{1}{2}}}{4\sqrt{2k(k-1)}{L}_k^2{\lambda }_k}\right\}\right\}.$$ (4.3)

Theorem 4.1

Under Assumptions 1 – 3, for any 1 ≤ k ≤ K and 0 ≤ α ≤ α₀,

$$\begin{array}{cc}\hfill 0& \le {\lambda }_k-{\lambda }_k^{\left[\alpha \right]}\hfill \\ \hfill & \le \sqrt{2}\sqrt{k}{L}_k^2{\lambda }_k\alpha \left(1+O\left(\frac{\sqrt{k}{L}_k^2{\lambda }_k}{{\lambda }_k-{\lambda }_{k+1}}\alpha +\frac{k(k-1)L_k^4{\lambda }_k^2\parallel \mathrm{\Gamma }\parallel }{{({\lambda }_{k-1}-{\lambda }_k)}^2({\lambda }_k-{\lambda }_{k+1})}{\alpha }^2\right)\right).\hfill \end{array}$$ (4.4)

Given γ_k, 1 ≤ k ≤ K, we can uniquely choose ${\gamma }_k^{\left[\alpha \right]}$ for each α ∈ [0, α₀] such that ${\gamma }_k^{\left[\alpha \right]}$ is a continuous function of α and $({\gamma }_k^{\left[\alpha \right]},{\gamma }_k)>0$ for all 0 ≤ α ≤ α₀, and we have

$$\parallel {\gamma }_k^{\left[\alpha \right]}-{\gamma }_k\parallel \le \sqrt{a}\sqrt{\frac{4\sqrt{2}\sqrt{k}{L}_k^2{\lambda }_k}{{\lambda }_k-{\lambda }_{k+1}}}+\alpha \sqrt{4k(k-1)L_k^4{\left(\frac{{\lambda }_k}{{\lambda }_{k-1}-{\lambda }_k}\right)}^2\left\{1+\frac{2\parallel \mathrm{\Gamma }\parallel }{{\lambda }_k-{\lambda }_{k+1}}\right\}.}$$ (4.5)

Remark

1. If K is fixed or hounded, we have

   $$\begin{array}{cc}\hfill & 0\le {\lambda }_k-{\lambda }_k^{\left[a\right]}\le \sqrt{2}\sqrt{k}{L}_k^2{\lambda }_k\alpha +o\left(\alpha \right),\hfill \\ \hfill & \parallel {\gamma }_k^{\left[\alpha \right]}-{\gamma }_k\parallel \le \sqrt{a}\sqrt{\frac{4\sqrt{2}\sqrt{k}{L}_k^2{\lambda }_k}{{\lambda }_k-{\lambda }_{k+1}}}+o\left(\sqrt{\alpha }\right)\hfill \end{array}$$

   Hence, the convergence rates for eigenvalues and eigenfunctions are different. Eigenvalues have faster convergence rates than eigenfunctions.

2. As K → ∞, we have α₀ →. If we choose α in such a way that 0 ≤ α ≤ α₀ and the right hand sides of (4.4) and (4.5) converges to zero, then ${\lambda }_k^{\left[\alpha \right]}\to {\lambda }_k$ and ${\gamma }_k^{\left[\alpha \right]}\to {\gamma }_k$ for all 1 ≤ k ≤ K.

3. The convergence rates for both eigenvalues and eigenfunctions depend on L_k. If the eigenfunctions are less smooth, that is, L_k is large, then the convergence is slow.

4. (4.4) and (4.5) give the upper bounds. However, the lower bounds are 0 for any k ∈ ℕ. Here is a simple example. Without loss of generality, let k = 2. Suppose [a, b] = [0, 2π],

   $$\mathrm{\Gamma }(s,t)=\frac{2}{\pi }\cos \left(s\right)\cos \left(t\right)+\frac{1}{2\pi }+\frac{1}{2\pi }\sin \left(s\right)\sin \left(t\right)+\frac{1}{\pi }\underset{m=2}{\overset{\infty }{\Sigma }}{\left(\frac{1}{2m}\right)}^3\cos \left(\mathit{ms}\right)\cos \left(\mathit{mt}\right)+\frac{1}{\pi }\underset{m=2}{\overset{\infty }{\Sigma }}{\left(\frac{1}{2m+1}\right)}^3\sin \left(\mathit{ms}\right)\sin \left(\mathit{mt}\right).$$

   Note that the right hand side in the above equality converges both uniformly and in L²([0, 2π] × [0, 2π]) to a strictly positive definite covariance functions. Its first eigenvalue and eigenfunction are 2 and $\frac{1}{\sqrt{\pi }}\cos \left(t\right)$, the second ones are 1 and $\frac{1}{\sqrt{2\pi }}$. It is interesting to note that the eigenfunctions of Γ are the same as the solutions of the successive optimization problems (3.3) and (3.4). The first maximum value of the successive optimization problems (3.3) and (3.4) is $\frac{2}{1+\alpha }$ and the second one is still 1. That is, in this case, we have ${\lambda }_2^{\left[\alpha \right]}={\lambda }_2$ and ${\gamma }_2^{\left[\alpha \right]}={\gamma }_2$ for any α, hence the lower bounds are zeros.

Define C_ℝ[0, α₀] to be the normed space of all continuous real functions in [0, α₀] equipped with norm sup_0≤α≤α0. | · |. Let Π_1≤j≤K Cℝ[0, α₀] denote the product space of K copies of Cℝ[0, α₀] Define C_L²([a,b]) [0, α₀] to be the normed space of all continuous functions in [0, α₀] taking values in L²([a,b]) equipped with norm sup_0≤α≤α0 ‖ · ‖. Similarly, we define Π_1≤j≤K C_L²([a,b])[0, α₀].

For each 1 ≤ k ≤ K and each n, we will view $\sqrt{n}({\widehat{\gamma }}_k^{\left[\alpha \right]}-{\gamma }_k^{\left[\alpha \right]})$ as a stochastic process with index α ∈ [0, α₀] and values in L²[a, b] and view $\sqrt{n}({\widehat{\lambda }}_k^{\left[\alpha \right]}-{\lambda }_k^{\left[\alpha \right]})$ as a stochastic process with index α ∈ [0, α₀] and values in ℝ. However, in the following subset in the probability space,

$${\mathrm{\Omega }}_0=\{\omega :\text{there exists at least one}\alpha \in [0,{\alpha }_0]\text{such that}{\widehat{\lambda }}_1^{\left[\alpha \right]},\cdots ,{\widehat{\lambda }}_K^{\left[\alpha \right]}\text{are not mutually different}\},$$ (4.6)

${\widehat{\gamma }}_1^{\left[\alpha \right]},\cdots ,{\widehat{\gamma }}_K^{\left[\alpha \right]}$ are not uniquely determined up to signs. We will show that Ω₀ is measurable and its probability goes to zero as n → ∞ in the proof of the following theorem. Hence, how to define ${\widehat{\gamma }}_1^{\left[\alpha \right]},\cdots ,{\widehat{\gamma }}_K^{\left[\alpha \right]}$ in Ω₀ does not affect our asymptotic results. In order to make the development of our theory easier, we will use the following definition

$$\mathrm{in}{\mathrm{\Omega }}_0,\text{define}{\widehat{\gamma }}_k^{\left[\alpha \right]}=0,1\le k\le K.$$ (4.7)

Theorem 4.2

Under Assumptions 1 — 3 and the definition (4.7), we can properly choose ${\widehat{\gamma }}_k^{\left[\alpha \right]}$ in ${\Omega }_0^c$ to make the sequence

$${\{\sqrt{n}({\widehat{\gamma }}_k^{\left[\alpha \right]}-{\gamma }_k^{\left[\alpha \right]}),1\le k\le K,0\le \alpha \le {\alpha }_0\}}_n$$ (4.8)

of stochastic processes is measurable and has sample paths in

$$\prod _{k=1}^K{C}_{L^2\left([a,b]\right)}[0,{\alpha }_0]$$

a.s. . Furthermore, the sequence converges in distribution to a Gaussian random, element with values in ${\prod }_{k=1}^K{C}_{L^2\left([a,b]\right)}[0,{\alpha }_0]$ and mean zero. Similarly, the sequence

$${\{\sqrt{n}({\widehat{\lambda }}_k^{\left[\alpha \right]}-{\lambda }_k^{\left[\alpha \right]}),1\le k\le K,0\le \alpha \le {\alpha }_0\}}_n$$ (4.9)

of stochastic processes has sample paths in ${\prod }_{k=1}^K{C}_{ℝ}[0,{\alpha }_0]$ a.s. and converges in distribution to a Gaussian random, element with values in ${\prod }_{k=1}^K{C}_{ℝ}[0,{\alpha }_0]$ and mean zero.

Remark

1. Recall the definition of Guassian random elements in a separable Banach space. Suppose that X is a random element with values in a Banach space B with mean zero. Then X is a Guassian element if for any bounded linear functional f, f(X) is a Guassian random, variable. If X is a Guassian random, element, we can define its covariance operator Q. Q is a bounded operator from the dual space B′ to B such that for any f, g ∈ B′, g(Qf) = E [f(X)g(X)]. Note that the distribution of a Gaussian element with values in a Banach space and mean zero is determined by its covariance operator. For further properties of Guassian random elements in Banach spaces, see Ledoux and Talagrand [10].

2. The covariance operators (4.8) and (4.9) can be characterized by the “half-smoothing” operator S_α defined in (2.5) and the limit distribution of $\sqrt{n}({\widehat{\mathrm{\Gamma }}}_n-\mathrm{\Gamma })$. However, the characterization involves some technical definitions. The reader can find the characterization in the proof of this theorem.

3. The measurabilities and a.s. continuities of the sample paths of the processes (4.8) and (4.9) are not obvious at all.

4. The convergences of (4.8) and (4.9) are weak convergences of probability measures in spaces ${\prod }_{k=1}^K{C}_{ℝ}[0,{\alpha }_0]$ and ${\prod }_{k=1}^K{C}_{L^2\left([a,b]\right)}[0,{\alpha }_0]$, which are stronger than the convergences of only the marginal distributions of (4.8) and (4.9).

Now from Theorem 4.1 and Theorem 4.2, we have the following corollaries.

Corollary 4.1

Under Assumptions 1 – 3, for any 1 ≤ k ≤ K and 0 ≤ α ≤ α₀

$$\begin{array}{cc}\hfill & |{\widehat{\lambda }}_k^{\left[\alpha \right]}-{\lambda }_k|\le |{\widehat{\lambda }}_k^{\left[\alpha \right]}-{\lambda }_k^{\left[\alpha \right]}|+\sqrt{2}\sqrt{k}{L}_k^2{\lambda }_k\alpha +o\left(\alpha \right),\hfill \\ \hfill & \parallel {\widehat{\gamma }}_k^{\left[\alpha \right]}-{\gamma }_k\parallel \le |{\widehat{\gamma }}_k^{\left[\alpha \right]}-{\gamma }_k^{\left[\alpha \right]}|+\sqrt{\alpha }\sqrt{\frac{4\sqrt{2}\sqrt{k}{L}_k^2{\lambda }_k}{{\lambda }_k-{\lambda }_{k+1}}}+o\left(\sqrt{\alpha }\right).\hfill \end{array}$$ (4.10)

where

$$\underset{0\le \alpha \le {\alpha }_0}{\sup }|{\widehat{\lambda }}_k^{\left[\alpha \right]}-{\lambda }_k^{\left[\alpha \right]}|=O_p\left(\frac{1}{\sqrt{n}}\right),\underset{0\le \alpha \le {\alpha }_0}{\sup }|{\widehat{\gamma }}_k^{\left[\alpha \right]}-{\gamma }_k^{\left[\alpha \right]}|=O_p\left(\frac{1}{\sqrt{n}}\right).$$

Remark

From, Corollary 4.1, it seems that smoothing (that is, α > 0) is unnecessary since when α = 0, we get the best order $\frac{1}{\sqrt{n}}$. We clarify this problem by the following remarks.

1. Both Silverman [15] and this paper consider the ideal situation where every sample curve is observed at all points in [a, b] without any noise or measurement error. Although in this situation the estimates are consistent when α = 0, smoothing is advantageous.

   • – First, because the “bias terms” and the “variation terms” are not the bias and the variation in the strict sense, they are correlated. Since the upper bounds on the right hand sides of (4.10) are the sums of the upper bounds for bias terms and variation terms, the upper bounds in (4.10) are actually for the cases in which bias terms and variation terms are positively correlated. They are the worst cases when we introduce smoothing. In some cases such as those in Section 6.3 of Silverman [15], the mean squared errors for some α > 0 are less than those for α = 0. For these cases, it is possible that bias terms and variation terms are negatively correlated and hence the estimate errors should be much less than the upper bounds in (4.10). Section 6.4 of Silverman [15] gave an optimal a with order $O\left(\frac{1}{n}\right)$ for estimates of eigenfunctions. By Corollary 4.1, if we choose the optimal a, we obtain the best asymptotic rates $O\left(\frac{1}{\sqrt{n}}\right)$. Even for the worst cases, if we take $\alpha =O\left(\frac{1}{n}\right)$, we can obtain the rate $O\left(\frac{1}{\sqrt{n}}\right)$.
   • – Second, from a practical viewpoint, it is desirable that the estimates of principal component curves can keep main patterns of the true principal component curves. However, the sample curves of many stochastic process are nonsmooth or even discontinuous, such as examples in Remark (3) after Assumption 3. Hence, their sample covariance functions have many local variations and so do the eigenfunctions of those sample covariance functions. In these cases, the local variations can be removed by using an appropriate amount of smoothing, that is, choosing an appropriate positive α.

2. In practice, people cannot observe the entire sample curves. The observations can only be made at discrete points often with noise or measurement error. The observation points could he dense or sparse. If the sample curves are smooth and the observation points are dense, we can obtain smoothed estiamte of each sample function and perform the usual functional PC A. This method cannot be applied to other situations. However, Silverman's method can be generalized to all these situations (see Qi and Zhao [11]). In our generalization, smoothing is essential and the smoothing parameters must be positive. The theoretical results in this paper has been applied to prove the consistency results in Qi and Zhao [11].

If α goes to 0 fast enough as n → ∞, we have the following asymptotic normalities.

Corollary 4.2

Under Assumptions 1 – 3, for any sequence {α_n, n ≥ 1} with ${\alpha }_n=o_p\left(\frac{1}{\sqrt{n}}\right)$, the joint distributions of

$$\{\sqrt{n}({\widehat{\lambda }}_1^{\left[{\alpha }_n\right]}-{\lambda }_1),\sqrt{n}({\widehat{\lambda }}_2^{\left[{\alpha }_n\right]}-{\lambda }_2),\cdots ,\sqrt{n}({\widehat{\lambda }}_K^{\left[{\alpha }_n\right]}-{\lambda }_K)\}$$

converge to the same Gaussian distribution with mean zero. For any sequence {α_n, n ≥ 1} with ${\alpha }_n=o_p\left(\frac{1}{n}\right)$, the joint distributions of

$$\{\sqrt{n}({\widehat{\gamma }}_1^{\left[{\alpha }_n\right]}-{\gamma }_1),\sqrt{n}({\widehat{\gamma }}_2^{\left[{\alpha }_n\right]}-{\gamma }_2),\cdots ,\sqrt{n}({\widehat{\gamma }}_K^{\left[{\alpha }_n\right]}-{\gamma }_K)\}$$

converge to the same Gaussian distribution with mean zero.

Remark

Dauxois et al. [3] gave the asymptotic normalities of the eigenvalues and eigenfunctions of Γ̂_n and characterized the covariance operators of the limit Gaussian random elements. Those results are special cases of Corollary 4.2 with all α_n equal to zeros. Therefore, by Corollary 4.2, all the limit Guassian distributions in Corollary 4.2 are the same as those in Dauxois et al. [3].

5. Proofs

Proof of Theorem 3.1

By Remark (3) after Assumption 1, ∥Γ̂_n∥ < ∞ a.s.. Fix a sample and α ≥ 0 such that ∥Γ̂_n∥ < ∞. Consider the Hilbert space $W_2^2\left([a,b]\right)$ equipped with the inner product (·,·)_α. For any f, $g\in W_2^2\left([a,b]\right)$, the functional (f, Γ̂_ng) define a bilinear form in $W_2^2\left([a,b]\right)$ and

$$\left|(f,{\widehat{\mathrm{\Gamma }}}_{n}g)\right|\le \parallel {\widehat{\mathrm{\Gamma }}}_n\parallel \parallel f\parallel \parallel g\parallel \le \parallel {\widehat{\mathrm{\Gamma }}}_n\parallel {\parallel f\parallel }_{\alpha }{\parallel g\parallel }_{\alpha }.$$

Hence, there is a unique bounded operator R_a in $W_2^2\left([a,b]\right)$, such that for any f, $g\in W_2^2\left([a,b]\right)$,

$$(f,{\widehat{\mathrm{\Gamma }}}_{n}g)={(f,R_{\alpha }g)}_{\alpha },$$

(see Section 84 in Riesz and Sz.-Nagy [13]). It is easy to see that R_a is symmetric and nonnegative-definite. We want to show that R_a is a compact operator (note that a compact operator is called completely continuous operator in Riesz and Sz.-Nagy [13]). By definition 4 in Section 85 of Riesz and Sz.-Nagy [13], we only need to show that for any bounded sequence $\{f_m\in W_2^2\left([a,b]\right),m\in \mathbb{N}\}$, one can select a subsequence {f_mk} such that

$${(f_{m_k}-f_{m_l},R_{\alpha }(f_{m_k}-f_{m_l}))}_{\alpha }=(f_{m_k}-f_{m_l},{\widehat{\mathrm{\Gamma }}}_n(f_{m_k}-f_{m_l}))\to 0,$$ (5.1)

as k, l → ∞. Because Γ̂_n is a compact operator in L²([a,b]) (see Remark (2) after Assumption 1) and {f_m} is also a bounded sequence in L²([a, b]), one can select a subsequence {f_mk} such that {Γ̂_nfm_mk} converges, then (5.1) is true for {f_mk}. Hence R_a is a compact operator. It has eigenvalues and eigenfunctions $\{({\widehat{\lambda }}_j^{\left[\alpha \right]},{\widehat{\gamma }}_j^{\left[\alpha \right]}):j\in \mathbb{N}\}$ with ${\widehat{\lambda }}_1^{\left[\alpha \right]}\ge {\widehat{\lambda }}_2^{\left[\alpha \right]}\ge \cdots \ge 0$. They are the solutions of the successive optimization problems (3.3) and (3.4) (see Chapter 3 of Weinberger [16]). Now for any $\gamma \in W_2^2\left([a,b]\right)$ and any j ∈ ℕ, because

$$R_{\alpha }{\widehat{\gamma }}_j^{\left[\alpha \right]}={\widehat{\lambda }}_j^{\left[\alpha \right]}{\widehat{\gamma }}_j^{\left[\alpha \right]},$$

we have

$$({\widehat{\mathrm{\Gamma }}}_n{\widehat{\gamma }}_j^{\left[\alpha \right]},\gamma )={(R_{\alpha }{\widehat{\gamma }}_j^{\left[\alpha \right]},\gamma )}_{\alpha }={\widehat{\lambda }}_j^{\left[\alpha \right]}{({\widehat{\gamma }}_j^{\left[\alpha \right]},\gamma )}_{\alpha }.$$

Proof of Theorem 4.1

The proof of the existence and uniqueness of the choices of the signs of ${\gamma }_k^{\left[\alpha \right]}$, 1 ≤ k ≤ K making them continuous functions of α will be postponed to the proof of Theorem 4.2 because we need some technical lemmas in the proof of Theorem 4.2. We will assume that we can choose the signs of ${\gamma }_k^{\left[\alpha \right]}$, 1 ≤ k ≤ K such that they are continuous function of α for all 0 ≤ α ≤ α₀ and ${\gamma }_k^{\left[0\right]}={\gamma }_k$, 1 ≤ k ≤ K.

For any 1 ≤ k ≤ K, let P_k be the orthogonal projection operator in L²([a,b]) onto the space spanned by {γ₁,… ,γ_k} and I be the identity operator in L²([a, b]). Then (I − P_k) is the orthogonal projection operator onto the closed subspace spanned by {γ_j,j ≥ (k + 1)}.

Lemma 1

For any k ∈ ℕ, and α₁ ≥ α₂ ≥ 0

$${\lambda }_k^{\left[{\alpha }_1\right]}\le {\lambda }_k^{\left[{\alpha }_2\right]},{\widehat{\lambda }}_k^{\left[{\alpha }_1\right]}\le {\widehat{\lambda }}_k^{\left[{\alpha }_2\right]},\forall k\in \mathbb{N}.$$

Proof. It follows Theorem 8.1 in Chapter 3 of Weinberger [16].

Lemma 2

For any 1 ≤ k ≤ K and α ≥ 0, we have

$${\parallel P_{k-1}{\gamma }_k^{\left[\alpha \right]}\parallel }^2\le {\alpha }^2{\left(\frac{{\lambda }_k}{{\lambda }_{k-1}-{\lambda }_k}\right)}^2(k-1)L_k^2\left[{\gamma }_k^{\left[\alpha \right]},{\gamma }_k^{\left[\alpha \right]}\right].$$ (5.2)

Proof. For any j < k, by (3.6), we have

$$\begin{array}{cc}\hfill {\lambda }_j\left({\gamma }_k^{\left[\alpha \right]},{\gamma }_j\right)=\left({\gamma }_k^{\left[\alpha \right]},\mathrm{\Gamma }{\gamma }_j\right)& =\left(\mathrm{\Gamma }{\gamma }_k^{\left[\alpha \right]},{\gamma }_j\right)={\lambda }_k^{\left[\alpha \right]}{\left({\gamma }_k^{\left[\alpha \right]},{\gamma }_j\right)}_{\alpha }\hfill \\ \hfill =& {\lambda }_k^{\left[\alpha \right]}\left\{\left({\gamma }_k^{\left[\alpha \right]},{\gamma }_j\right)+\alpha \left[{\gamma }_k^{\left[\alpha \right]},{\gamma }_j\right]\right\}.\hfill \end{array}$$

So

$$\left({\lambda }_j-{\lambda }_k^{\left[\alpha \right]}\right)\left({\gamma }_k^{\left[\alpha \right]},{\gamma }_j\right)={\lambda }_k^{\left[\alpha \right]}\alpha \left[{\gamma }_k^{\left[\alpha \right]},{\gamma }_j\right].$$

By Assumption 2 and Lemma 1, ${\lambda }_j>{\lambda }_k\ge {\lambda }_k^{\left[\alpha \right]}$. Therefore,

$$\left({\gamma }_k^{\left[\alpha \right]},{\gamma }_j\right)=\frac{{\lambda }_k^{\left[\alpha \right]}}{{\lambda }_j-{\lambda }_k^{\left[\alpha \right]}}\alpha \left[{\gamma }_k^{\left[\alpha \right]},{\gamma }_j\right].$$

and we have

$$\begin{array}{cc}\hfill & {\parallel P_{k-1}{\gamma }_k^{\left[\alpha \right]}\parallel }^2=\underset{j=1}{\overset{k-1}{\Sigma }}{\left({\gamma }_k^{\left[\alpha \right]},{\gamma }_j\right)}^2=\underset{j=1}{\overset{k-1}{\Sigma }}{\left(\frac{{\lambda }_k^{\left[\alpha \right]}}{{\lambda }_j-{\lambda }_k^{\left[\alpha \right]}}\right)}^2{\alpha }^2{\left[{\gamma }_k^{\left[\alpha \right]},{\gamma }_j\right]}^2\hfill \\ \hfill \le & {\alpha }^2{\left(\frac{{\lambda }_k}{{\lambda }_{k-1}-{\lambda }_k}\right)}^2\underset{j=1}{\overset{k-1}{\Sigma }}{\left[{\gamma }_k^{\left[\alpha \right]},{\gamma }_j\right]}^2\le {\alpha }^2{\left(\frac{{\lambda }_k}{{\lambda }_{k-1}-{\lambda }_k}\right)}^2\underset{j=1}{\overset{k-1}{\Sigma }}\left[{\gamma }_k^{\left[\alpha \right]},{\gamma }_k^{\left[\alpha \right]}\right][{\gamma }_j,{\gamma }_j]\hfill \\ \hfill \le & {\alpha }^2{\left(\frac{{\lambda }_k}{{\lambda }_{k-1}-{\lambda }_k}\right)}^2(k-1)L_k^2\left[{\gamma }_k^{\left[\alpha \right]},{\gamma }_k^{\left[\alpha \right]}\right],\hfill \end{array}$$

where the last inequality in the second line follows from Cauchy-Schwarz in-equality.

Lemma 3

For any 1 ≤ k ≤ K and any

$$0\le \alpha <\frac{\sqrt{1+\frac{4k{({\lambda }_{k-1}-{\lambda }_k)}^2}{(k-1){\lambda }_k\parallel \mathrm{\Gamma }\parallel }}-1}{2{\mathit{kL}}_k^2},$$

(if k = 1, the right hand side is defined to be infinity), we have

$$\left[{\gamma }_k^{\left[\alpha \right]},{\gamma }_k^{\left[\alpha \right]}\right]\le \frac{{\mathit{kL}}_k^2}{1-\alpha \frac{{\lambda }_k}{{({\lambda }_{k-1}-{\lambda }_k)}^2}(k-1)L_k^2(1+\alpha {\mathit{kL}}_k^2)\parallel \mathrm{\Gamma }\parallel }.$$ (5.3)

Furthermore, if

$$0\le \alpha \le \frac{\sqrt{1+\frac{2k{({\lambda }_{k-1}-{\lambda }_k)}^2}{(k-1){\lambda }_k\parallel \mathrm{\Gamma }\parallel }}-1}{2{\mathit{kL}}_k^2},$$

(if k = 1, the right hand side is defined to he infinity), we have

$$\left[{\gamma }_k^{\left[\alpha \right]},{\gamma }_k^{\left[\alpha \right]}\right]\le 2{\mathit{kL}}_k^2.$$ (5.4)

For any α ≥ 0, we have

$$0\le {\lambda }_k-{\lambda }_k^{\left[\alpha \right]}\le \alpha {\mathit{kL}}_k^2{\lambda }_k.$$ (5.5)

Hence, as α → 0, ${\lambda }_k^{\left[\alpha \right]}\to {\lambda }_k$.

Proof. Let span(γ₁, … , γ_k) denote the linear subspace spanned by

$$\{{\gamma }_1,\cdots ,{\gamma }_k\}.$$

From Theorem 5.1 (Poincare's Principle) in Chapter 3 of Weinberger [16], we have

$$\begin{array}{cc}\hfill & \underset{0\ne \gamma \in \text{span}({\gamma }_1,\cdots ,{\gamma }_k)}{\min }\frac{(\gamma ,\mathrm{\Gamma }\gamma )}{{\parallel \gamma \parallel }^2+\alpha [\gamma ,\gamma ]}\le {\lambda }_k^{\left[\alpha \right]}=\frac{({\gamma }_k^{\left[\alpha \right]},\mathrm{\Gamma }{\gamma }_k^{\left[\alpha \right]})}{{\parallel {\gamma }_k^{\left[\alpha \right]}\parallel }^2+\alpha [{\gamma }_k^{\left[\alpha \right]},{\gamma }_k^{\left[\alpha \right]}]}\hfill \\ \hfill =& \frac{\left(P_{k-1}{\gamma }_k^{\left[\alpha \right]}+(I-P_{k-1}){\gamma }_k^{\left[\alpha \right]},\mathrm{\Gamma }\left(P_{k-1}{\gamma }_k^{\left[\alpha \right]}+(I-P_{k-1}){\gamma }_k^{\left[\alpha \right]}\right)\right)}{{\parallel {\gamma }_k^{\left[\alpha \right]}\parallel }^2+\alpha [{\gamma }_k^{\left[\alpha \right]},{\gamma }_k^{\left[\alpha \right]}]}\hfill \\ \hfill =& \frac{(P_{k-1}{\gamma }_k^{\left[\alpha \right]},\mathrm{\Gamma }{P}_{k-1}{\gamma }_k^{\left[\alpha \right]})+((I-P_{k-1}){\gamma }_k^{\left[\alpha \right]},\mathrm{\Gamma }(I-P_{k-1}){\gamma }_k^{\left[\alpha \right]})}{{\parallel {\gamma }_k^{\left[\alpha \right]}\parallel }^2+\alpha [{\gamma }_k^{\left[\alpha \right]},{\gamma }_k^{\left[\alpha \right]}]}\hfill \\ \hfill \le & \frac{\parallel \mathrm{\Gamma }\parallel \cdot {\parallel P_{k-1}{\gamma }_k^{\left[\alpha \right]}\parallel }^2+{\lambda }_k}{{\parallel {\gamma }_k^{\left[\alpha \right]}\parallel }^2+\alpha [{\gamma }_k^{\left[\alpha \right]},{\gamma }_k^{\left[\alpha \right]}]},\hfill \end{array}$$ (5.6)

where the equality in the third line of (5.6) is true because that (I − P_k−1) is the orthogonal projection operator onto the closed subspace spanned by {γ_j,j ≥ k} which is orthogonal to span(γ₁, … , γ_k-1), and both of them are invariant subspaces of Γ. The last inequality in (5.6) holds because the largest eigenvalue of Γ restricted to the closed subspace spanned by {γ_j,j≥k} is λ_k and the L² norm of $(I-P_{k-1}){\gamma }_k^{\left[\alpha \right]}$ is less than 1. On the other hand, we have

$$\begin{array}{cc}\hfill & \underset{0\ne \gamma \in \text{span}({\gamma }_1,\cdots ,{\gamma }_k)}{\min }\frac{(\gamma ,\mathrm{\Gamma }\gamma )}{{\parallel \gamma \parallel }^2+\alpha [\gamma ,\gamma ]}=\underset{0\ne \gamma \in \text{span}({\gamma }_1,\cdots ,{\gamma }_k)}{\min }\frac{(\gamma ,\mathrm{\Gamma }\gamma )}{{\parallel \gamma \parallel }^2(1+\frac{\alpha [\gamma ,\gamma ]}{{\parallel \gamma \parallel }^2})}\hfill \\ \hfill \ge & \underset{0\ne \gamma \in \text{span}({\gamma }_1,\cdots ,{\gamma }_k)}{\min }\frac{(\gamma ,\mathrm{\Gamma }\gamma )}{{\parallel \gamma \parallel }^2(1+{\max }_{0\ne \beta \in \text{span}({\gamma }_1,\cdots ,{\gamma }_k)}\frac{\alpha [\beta ,\beta ]}{{\parallel \beta \parallel }^2})}\hfill \\ \hfill =& \frac{1}{(1+{\max }_{0\ne \beta \in \text{span}({\gamma }_1,\cdots ,{\gamma }_k)}\frac{\alpha [\beta ,\beta ]}{{\parallel \beta \parallel }^2})}\underset{0\ne \gamma \in \text{span}({\gamma }_1,\cdots ,{\gamma }_k)}{\min }\frac{(\gamma ,\mathrm{\Gamma }\gamma )}{{\parallel \gamma \parallel }^2}\hfill \\ \hfill =& \frac{{\lambda }_k}{(1+{\max }_{0\ne \beta \in \text{span}({\gamma }_1,\cdots ,{\gamma }_k)}\frac{\alpha [\beta ,\beta ]}{{\parallel \beta \parallel }^2})}\ge \frac{{\lambda }_k}{(1+\alpha {\mathit{kL}}_k^2)}\hfill \end{array}$$ (5.7)

The equality in the last line follows from the fact that the smallest eigenvalue of Γ in span(γ₁, … , γ_k) is λ_k. The last inequality holds because that, for any β ∈ span(γ₁, … , γ_k), let $\beta ={\Sigma }_{i=1}^k{c}_i{\gamma }_i$, where c₁, … , c_k are some real numbers, then we have

$$\begin{array}{cc}\hfill & \frac{[\beta ,\beta ]}{{\parallel \beta \parallel }^2}=\frac{[{\Sigma }_{i=1}^k{c}_i{\gamma }_i,{\Sigma }_{i=1}^k{c}_i{\gamma }_i]}{{\Sigma }_{i=1}^k{c}_i^2}=\frac{{\Sigma }_{i=1}^k{c}_i^2[{\gamma }_i,{\gamma }_i]+{\Sigma }_{j\ne l}{c}_j{c}_l[{\gamma }_j,{\gamma }_l]}{{\Sigma }_{i=1}^k{c}_i^2}\hfill \\ \hfill \le & \frac{{\Sigma }_{i=1}^k{c}_i^2{\left(\sqrt{[{\gamma }_i,{\gamma }_i]}\right)}^2+{\Sigma }_{j\ne l}{c}_j{c}_l\sqrt{[{\gamma }_j,{\gamma }_j]}\sqrt{[{\gamma }_l,{\gamma }_l]}}{{\Sigma }_{i=1}^k{c}_i^2}\hfill \\ \hfill =& \frac{{\left({\Sigma }_{i=1}^k{c}_i\sqrt{[{\gamma }_i,{\gamma }_i]}\right)}^2}{{\Sigma }_{i=1}^k{c}_i^2}\le \frac{\left({\Sigma }_{i=1}^k{c}_i^2\right)\left({\Sigma }_{i=1}^k[{\gamma }_i,{\gamma }_i]\right)}{{\Sigma }_{i=1}^k{c}_i^2}\le \underset{i=1}{\overset{k}{\Sigma }}[{\gamma }_i,{\gamma }_i]\le {\mathit{kL}}_k^2,\hfill \end{array}$$

where the inequality in the second line is due to Cauchy-Schwarz inequality. Now from (5.6), (5.7) and Lemma 1, we have

$$\frac{{\lambda }_k}{(1+\alpha {\mathit{kL}}_k^2)}\le {\lambda }_k^{\left[\alpha \right]}\le {\lambda }_k.$$

From these inequalities, it can be derived that

$$0\le {\lambda }_k-{\lambda }_k^{\left[\alpha \right]}\le \alpha {\mathit{kL}}_k^2{\lambda }_k.$$

Therefore, ${\lambda }_k^{\left[\alpha \right]}\to {\lambda }_k$ as α → 0.

Again by (5.6), (5.7), and note that $\parallel {\gamma }_k^{\left[\alpha \right]}\parallel =1$, we have

$$\frac{{\lambda }_k}{(1+\alpha {\mathit{kL}}_k^2)}\le \frac{\parallel \mathrm{\Gamma }\parallel \cdot {\parallel P_{k-1}{\gamma }_k^{\left[\alpha \right]}\parallel }^2+{\lambda }_k}{{\parallel {\gamma }_k^{\left[\alpha \right]}\parallel }^2+\alpha [{\gamma }_k^{\left[\alpha \right]},{\gamma }_k^{\left[\alpha \right]}]}=\frac{\parallel \mathrm{\Gamma }\parallel \cdot {\parallel P_{k-1}{\gamma }_k^{\left[\alpha \right]}\parallel }^2+{\lambda }_k}{1+\alpha [{\gamma }_k^{\left[\alpha \right]},{\gamma }_k^{\left[\alpha \right]}]}.$$

Then

$${\lambda }_k(1+\alpha [{\gamma }_k^{\left[\alpha \right]},{\gamma }_k^{\left[\alpha \right]}])\le \parallel \mathrm{\Gamma }\parallel \cdot {\parallel P_{k-1}{\gamma }_k^{\left[\alpha \right]}\parallel }^2(1+\alpha {\mathit{kL}}_k^2)+{\lambda }_k(1+\alpha {\mathit{kL}}_k^2),$$

hence,

$${\lambda }_k\alpha [{\gamma }_k^{\left[\alpha \right]},{\gamma }_k^{\left[\alpha \right]}]\le \parallel \mathrm{\Gamma }\parallel \cdot {\parallel P_{k-1}{\gamma }_k^{\left[\alpha \right]}\parallel }^2(1+\alpha {\mathit{kL}}_k^2)+{\lambda }_k\alpha {\mathit{kL}}_k^2.$$

.

Now by (5.2), we have

$$[{\gamma }_k^{\left[\alpha \right]},{\gamma }_k^{\left[\alpha \right]}]\le \alpha \frac{{\lambda }_k}{{({\lambda }_{k-1}-{\lambda }_k)}^2}(k-1)L_k^2\left[{\gamma }_k^{\left[\alpha \right]},{\gamma }_k^{\left[\alpha \right]}\right](1+\alpha {\mathit{kL}}_k^2)\parallel \mathrm{\Gamma }\parallel +{\mathit{kL}}_k^2.$$

After rearranging the terms, we then obtain

$$[{\gamma }_k^{\left[\alpha \right]},{\gamma }_k^{\left[\alpha \right]}]\le \left\{1-\alpha \frac{{\lambda }_k}{{({\lambda }_{k-1}-{\lambda }_k)}^2}(k-1)L_k^2(1+\alpha {\mathit{kL}}_k^2)\parallel \mathrm{\Gamma }\parallel \right\}\le {\mathit{kL}}_k^2.$$

When the expression in braces on the left of the above inequality is positive, which is equivalent to

$$\alpha <\frac{\sqrt{1+\frac{4k{({\lambda }_{k-1}-{\lambda }_k)}^2}{(k-1){\lambda }_k\parallel \mathrm{\Gamma }\parallel }}-1}{2{\mathit{kL}}_k^2},$$

(if k = 1, the right hand side is denned to be infinity), we have

$$[{\gamma }_k^{\left[\alpha \right]},{\gamma }_k^{\left[\alpha \right]}]\le \frac{{\mathit{kL}}_k^2}{1-\alpha \frac{{\lambda }_k}{{({\lambda }_{k-1}-{\lambda }_k)}^2}(k-1)L_k^2(1+\alpha {\mathit{kL}}_k^2)\parallel \mathrm{\Gamma }\parallel }.$$ (5.8)

When

$$\alpha \le \frac{\sqrt{1+\frac{2k{({\lambda }_{k-1}-{\lambda }_k)}^2}{(k-1){\lambda }_k\parallel \mathrm{\Gamma }\parallel }}-1}{2{\mathit{kL}}_k^2},$$

(if k = 1, the right hand side is denned to be infinity), it can be shown that

$$1-\alpha \frac{{\lambda }_k}{{({\lambda }_{k-1}-{\lambda }_k)}^2}(k-1)L_k^2(1+\alpha {\mathit{kL}}_k^2)\parallel \mathrm{\Gamma }\parallel \ge \frac{1}{2},$$

and then it follows from (5.8) that

$$[{\gamma }_k^{\left[\alpha \right]},{\gamma }_k^{\left[\alpha \right]}]\le 2{\mathit{kL}}_k^2.$$

Lemma 4

For any 1 ≤ k ≤ K and any

$$0\le \alpha \le \frac{{\lambda }_k-{\lambda }_{k+1}}{2{\mathit{kL}}_k^2{\lambda }_k},$$ (5.9)

we have

$$\begin{array}{cc}\hfill & {\parallel (I-P_k){\gamma }_k^{\left[\alpha \right]}\parallel }^2\hfill \\ \hfill \le \frac{2}{{\lambda }_k-{\lambda }_{k+1}}& \left[\parallel \mathrm{\Gamma }\parallel {\alpha }^2{\left(\frac{{\lambda }_k}{{\lambda }_{k-1}-{\lambda }_k}\right)}^2(k-1)L_k^2\left[{\gamma }_k^{\left[\alpha \right]},{\gamma }_k^{\left[\alpha \right]}\right]+{\lambda }_k\alpha {[{\gamma }_k^{\left[\alpha \right]},{\gamma }_k^{\left[\alpha \right]}]}^{\frac{1}{2}}{L}_k\right]\hfill \end{array}$$ (5.10)

Proof. By the following orthogonal decomposition

$${\gamma }_k^{\left[\alpha \right]}=P_{k-1}{\gamma }_k^{\left[\alpha \right]}+({\gamma }_k^{\left[\alpha \right]},{\gamma }_k){\gamma }_k+(I-P_k){\gamma }_k^{\left[\alpha \right]},$$ (5.11)

we have

$$\begin{array}{cc}\hfill & \left({\gamma }_k^{\left[\alpha \right]},\mathrm{\Gamma }{\gamma }_k^{\left[\alpha \right]}\right)\hfill \\ \hfill =& (P_{k-1}{\gamma }_k^{\left[\alpha \right]},\mathrm{\Gamma }{P}_{k-1}{\gamma }_k^{\left[\alpha \right]})+{({\gamma }_k^{\left[\alpha \right]},{\gamma }_k)}^2({\gamma }_k,\mathrm{\Gamma }{\gamma }_k)+((I-P_k){\gamma }_k^{\left[\alpha \right]},\mathrm{\Gamma }(I-P_k){\gamma }_k^{\left[\alpha \right]})\hfill \\ \hfill \le & \parallel \mathrm{\Gamma }\parallel {\parallel P_{k-1}{\gamma }_k^{\left[\alpha \right]}\parallel }^2+{\lambda }_k{({\gamma }_k^{\left[\alpha \right]},{\gamma }_k)}^2+{\lambda }_{k+1}{\parallel (I-P_k){\gamma }_k^{\left[\alpha \right]}\parallel }^2,\hfill \end{array}$$ (5.12)

where the last inequality follows from the fact that $(I-P_k){\gamma }_k^{\left[\alpha \right]}$ belongs to the closed subspace spanned by {γ_j, j ≥ k + 1} in which the largest eigenvalue of Γ is λ_k+1. On the other hand, by (3.6), we have

$$\begin{array}{cc}\hfill & \left({\gamma }_k^{\left[\alpha \right]},\mathrm{\Gamma }{\gamma }_k^{\left[\alpha \right]}\right)={\gamma }_k^{\left[\alpha \right]}{\left({\gamma }_k^{\left[\alpha \right]},{\gamma }_k^{\left[\alpha \right]}\right)}_{\alpha }={\gamma }_k^{\left[\alpha \right]}{\parallel {\gamma }_k^{\left[\alpha \right]}\parallel }^2+{\alpha \gamma }_k^{\left[\alpha \right]}\left[{\gamma }_k^{\left[\alpha \right]},{\gamma }_k^{\left[\alpha \right]}\right]\hfill \\ \hfill =& {\lambda }_k^{\left[\alpha \right]}{\parallel P_{k-1}{\gamma }_k^{\left[\alpha \right]}\parallel }^2+{\lambda }_k^{\left[\alpha \right]}{({\gamma }_k^{\left[\alpha \right]},{\gamma }_k)}^2+{\lambda }_k^{\left[\alpha \right]}{\parallel (I-P_k){\gamma }_k^{\left[\alpha \right]}\parallel }^2+{\alpha \lambda }_k^{\left[\alpha \right]}\left[{\gamma }_k^{\left[\alpha \right]},{\gamma }_k^{\left[\alpha \right]}\right].\hfill \end{array}$$ (5.13)

From (5.12) and (5.13),

$$\begin{array}{cc}\hfill & {\lambda }_k^{\left[\alpha \right]}{\parallel P_{k-1}{\gamma }_k^{\left[\alpha \right]}\parallel }^2+{\lambda }_k^{\left[\alpha \right]}{({\gamma }_k^{\left[\alpha \right]},{\gamma }_k)}^2+{\lambda }_k^{\left[\alpha \right]}{\parallel (I-P_k){\gamma }_k^{\left[\alpha \right]}\parallel }^2+{\alpha \lambda }_k^{\left[\alpha \right]}\left[{\gamma }_k^{\left[\alpha \right]},{\gamma }_k^{\left[\alpha \right]}\right]\hfill \\ \hfill \le & \parallel \mathrm{\Gamma }\parallel {\parallel P_{k-1}{\gamma }_k^{\left[\alpha \right]}\parallel }^2+{\lambda }_k{({\gamma }_k^{\left[\alpha \right]},{\gamma }_k)}^2+{\lambda }_{k+1}{\parallel (I-P_k){\gamma }_k^{\left[\alpha \right]}\parallel }^2,\hfill \end{array}$$

then

$$\begin{array}{cc}\hfill ({\lambda }_k^{\left[\alpha \right]}-{\lambda }_{k+1}){\parallel (I-P_k){\gamma }_k^{\left[\alpha \right]}\parallel }^2& \hfill \\ \hfill \le (\parallel \mathrm{\Gamma }\parallel -{\lambda }_k^{\left[\alpha \right]}){\parallel P_{k-1}{\gamma }_k^{\left[\alpha \right]}\parallel }^2+({\lambda }_k-{\lambda }_k^{\left[\alpha \right]}){({\gamma }_k^{\left[\alpha \right]},{\gamma }_k)}^2-{\alpha \lambda }_k^{\left[\alpha \right]}\left[{\gamma }_k^{\left[\alpha \right]},{\gamma }_k^{\left[\alpha \right]}\right]& \hfill \\ \hfill \le (\parallel \mathrm{\Gamma }\parallel -{\lambda }_k^{\left[\alpha \right]}){\parallel P_{k-1}{\gamma }_k^{\left[\alpha \right]}\parallel }^2+({\lambda }_k-{\lambda }_k^{\left[\alpha \right]}){({\gamma }_k^{\left[\alpha \right]},{\gamma }_k)}^2.& \hfill \end{array}$$ (5.14)

It follows from (5.9) that $\alpha k{L}_k^2{\lambda }_k\le \frac{1}{2}({\lambda }_k-{\lambda }_{k+1})$. Then by (5.5), we have

$${\lambda }_k-{\lambda }_k^{\left[\alpha \right]}\le \frac{1}{2}({\lambda }_k-{\lambda }_{k+1}),$$

hence,

$${\lambda }_k^{\left[\alpha \right]}-{\lambda }_{k+1}\ge \frac{1}{2}({\lambda }_k-{\lambda }_{k+1}).$$ (5.15)

Because

$$\begin{array}{cc}\hfill {\lambda }_k\left({\gamma }_k^{\left[\alpha \right]},{\gamma }_k\right)& =\left({\gamma }_k^{\left[\alpha \right]},\mathrm{\Gamma }{\gamma }_k\right)=\left(\mathrm{\Gamma }{\gamma }_k^{\left[\alpha \right]},{\gamma }_k\right)={\lambda }_k^{\left[\alpha \right]}{\left({\gamma }_k^{\left[\alpha \right]},{\gamma }_k\right)}_{\alpha }\hfill \\ \hfill & ={\lambda }_k^{\left[\alpha \right]}\left\{\left({\gamma }_k^{\left[\alpha \right]},{\gamma }_k\right)+\alpha \left[{\gamma }_k^{\left[\alpha \right]},{\gamma }_k\right]\right\},\hfill \end{array}$$

we have

$$({\lambda }_k-{\lambda }_k^{\left[\alpha \right]})\left({\gamma }_k^{\left[\alpha \right]},{\gamma }_k\right)={\lambda }_k^{\left[\alpha \right]}\alpha \left[{\gamma }_k^{\left[\alpha \right]},{\gamma }_k\right].$$ (5.16)

From (5.14), (5.15) and (5.16),

$$\begin{array}{cc}\hfill \frac{1}{2}({\lambda }_k-{\lambda }_{k+1}){\parallel (I-P_k){\gamma }_k^{\left[\alpha \right]}\parallel }^2& \le \parallel \mathrm{\Gamma }\parallel {\parallel P_{k-1}{\gamma }_k^{\left[\alpha \right]}\parallel }^2+{\lambda }_k^{\left[\alpha \right]}\alpha [{\gamma }_k^{\left[\alpha \right]},{\gamma }_k]({\gamma }_k^{\left[\alpha \right]},{\gamma }_k)\hfill \\ \hfill & \le \parallel \mathrm{\Gamma }\parallel {\parallel P_{k-1}{\gamma }_k^{\left[\alpha \right]}\parallel }^2+{\lambda }_k^{\left[\alpha \right]}\alpha [{\gamma }_k^{\left[\alpha \right]},{\gamma }_k]\hfill \\ \hfill & \le \parallel \mathrm{\Gamma }\parallel {\parallel P_{k-1}{\gamma }_k^{\left[\alpha \right]}\parallel }^2+{\lambda }_k^{\left[\alpha \right]}\alpha {[{\gamma }_k^{\left[\alpha \right]},{\gamma }_k^{\left[\alpha \right]}]}^{\frac{1}{2}}{[{\gamma }_k,{\gamma }_k]}^{\frac{1}{2}}.\hfill \end{array}$$

Now by Lemma 2,

$$\begin{array}{cc}\hfill & \frac{1}{2}({\lambda }_k-{\lambda }_{k+1}){\parallel (I-P_k){\gamma }_k^{\left[\alpha \right]}\parallel }^2\hfill \\ \hfill \le & \parallel \mathrm{\Gamma }\parallel {\alpha }^2{\left(\frac{{\lambda }_k}{{\lambda }_{k-1}-{\lambda }_k}\right)}^2(k-1)L_k^2\left[{\gamma }_k^{\left[\alpha \right]},{\gamma }_k^{\left[\alpha \right]}\right]+{\lambda }_k\alpha {[{\gamma }_k^{\left[\alpha \right]},{\gamma }_k^{\left[\alpha \right]}]}^{\frac{1}{2}}{L}_k.\hfill \end{array}$$

Now we can prove Theorem 4.1. It follows from the definition (4.3) of α₀ that all the conditions in Lemmas 3 and 4 are satisfied. From the orthogonal decomposition

$${\gamma }_k^{\left[\alpha \right]}=P_{k-1}{\gamma }_k^{\left[\alpha \right]}+({\gamma }_k^{\left[\alpha \right]},{\gamma }_k){\gamma }_k+(I-P_k){\gamma }_k^{\left[\alpha \right]},$$

we have

$$1={\parallel {\gamma }_k^{\left[\alpha \right]}\parallel }^2={\parallel P_{k-1}{\gamma }_k^{\left[\alpha \right]}\parallel }^2+{({\gamma }_k^{\left[\alpha \right]},{\gamma }_k)}^2+{\parallel (I-P_k){\gamma }_k^{\left[\alpha \right]}\parallel }^2.$$

Hence, it follows from Lemma 2, Lemma 4 and (5.4) in Lemma 3 that

$$\begin{array}{cc}\hfill & {({\gamma }_k^{\left[\alpha \right]},{\gamma }_k)}^2=1-{\parallel P_{k-1}{\gamma }_k^{\left[\alpha \right]}\parallel }^2-{\parallel (I-P_k){\gamma }_k^{\left[\alpha \right]}\parallel }^2\hfill \\ \hfill \ge & 1-{\alpha }^2{\left(\frac{{\lambda }_k}{{\lambda }_{k-1}-{\lambda }_k}\right)}^2(k-1)L_k^2\left[{\gamma }_k^{\left[\alpha \right]},{\gamma }_k^{\left[\alpha \right]}\right]\hfill \\ \hfill -& \frac{2}{{\lambda }_k-{\lambda }_{k+1}}\left[\parallel \mathrm{\Gamma }\parallel {\alpha }^2{\left(\frac{{\lambda }_k}{{\lambda }_{k-1}-{\lambda }_k}\right)}^2(k-1)L_k^2\left[{\gamma }_k^{\left[\alpha \right]},{\gamma }_k^{\left[\alpha \right]}\right]+{\lambda }_k\alpha {[{\gamma }_k^{\left[\alpha \right]},{\gamma }_k^{\left[\alpha \right]}]}^{\frac{1}{2}}{L}_k\right]\hfill \\ \hfill \ge & 1-\frac{2\sqrt{2}\sqrt{k}{L}_k^2{\lambda }_k}{{\lambda }_k-{\lambda }_{k+1}}\alpha -2k(k-1)L_k^4{\left(\frac{{\lambda }_k}{{\lambda }_{k-1}-{\lambda }_k}\right)}^2\left\{1+\frac{2\parallel \mathrm{\Gamma }\parallel }{{\lambda }_k-{\lambda }_{k+1}}\right\}{\alpha }^2.\hfill \end{array}$$ (5.17)

Define

$$a=2k(k-1)L_k^4{\left(\frac{{\lambda }_k}{{\lambda }_{k-1}-{\lambda }_k}\right)}^2\left\{1+\frac{2\parallel \mathrm{\Gamma }\parallel }{{\lambda }_k-{\lambda }_{k+1}}\right\},b=\frac{2\sqrt{2}\sqrt{k}{L}_k^2{\lambda }_k}{{\lambda }_k-{\lambda }_{k+1}}.$$ (5.18)

By solving the following inequalities,

$$a{\alpha }^2+b\alpha \le \frac{1}{2},\alpha \ge 0,$$

we obtain $0\le \alpha \le \frac{\sqrt{b^2+2a-b}}{2a}$. Since

$$\frac{\sqrt{b^2+2a}-b}{2a}=\frac{1}{\sqrt{b^2+2a}+b}\ge \frac{1}{2\sqrt{b^2+2a}}\ge \frac{1}{2\sqrt{2\max \{b^2,2a\}}}\ge \frac{1}{2\sqrt{2}}\min \{\frac{1}{b},\frac{1}{\sqrt{2a}}\}$$

By the definition (4.3) of α₀ and (5.18), we have

$${\alpha }_0\le \frac{1}{2\sqrt{2}}\min \{\frac{1}{b},\frac{1}{\sqrt{2a}}\}\le \frac{\sqrt{b^2+2a}-b}{2a}.$$

Hence, for any 0 ≤ α ≤ α₀, we have $a{\alpha }^2+b\alpha \le \frac{1}{2}$. Now it follows from (5.17) that, for any 0 ≤ α ≤ α₀,

$${({\gamma }_k^{\left[\alpha \right]},{\gamma }_k)}^2\ge 1-b\alpha -a{\alpha }^2=\frac{1}{2}+(\frac{1}{2}-b\alpha -a{\alpha }^2)\ge \frac{1}{2}.$$ (5.19)

Because ${\gamma }_k^{\left[\alpha \right]}$ is a continuous function of α, $({\gamma }_k^{\left[\alpha \right]},{\gamma }_k)$ is also a continuous function of α and $({\gamma }_k^{\left[0\right]},{\gamma }_k)=({\gamma }_k,{\gamma }_k)=1$. Hence, it follows from (5.19) that $({\gamma }_k^{\left[\alpha \right]},{\gamma }_k)>0$ for all 0 ≤ α ≤ α₀.

From (5.16), (5.17) and (5.4), we have

$$\begin{array}{cc}\hfill & ({\lambda }_k-{\lambda }_k^{\left[\alpha \right]})=\frac{{\lambda }_k^{\left[\alpha \right]}\alpha \left[{\gamma }_k^{\left[\alpha \right]},{\gamma }_k\right]}{\left({\gamma }_k^{\left[\alpha \right]},{\gamma }_k\right)}\le \frac{{\lambda }_k^{\left[\alpha \right]}\alpha \mid {\gamma }_k^{\left[\alpha \right]},{\gamma }_k\mid }{\sqrt{{\left({\gamma }_k^{\left[\alpha \right]},{\gamma }_k\right)}^2}}\le \frac{{\lambda }_k^{\left[\alpha \right]}\alpha {\left[{\gamma }_k^{\left[\alpha \right]},{\gamma }_k^{\left[\alpha \right]}\right]}^{\frac{1}{2}}{[{\gamma }_k,{\gamma }_k]}^{\frac{1}{2}}}{\sqrt{{\left({\gamma }_k^{\left[\alpha \right]},{\gamma }_k\right)}^2}}\hfill \\ \hfill =& \sqrt{2}\sqrt{k}{L}_k^2{\lambda }_k\alpha \left(1+O\left(\frac{\sqrt{k}{L}_k^2{\lambda }_k}{{\lambda }_k-{\lambda }_{k+1}}\alpha +\frac{k(k-1)L_k^4{\lambda }_k^2\parallel \mathrm{\Gamma }\parallel }{{({\lambda }_{k-1}-{\lambda }_k)}^2({\lambda }_k-{\lambda }_{k+1})}{\alpha }^2\right)\right).\hfill \end{array}$$

By (5.17) and $({\gamma }_k^{\left[\alpha \right]},{\gamma }_k)>0$, we have

$$\begin{array}{cc}\hfill {\parallel {\gamma }_k^{\left[\alpha \right]}-{\gamma }_k\parallel }^2& =2(1-({\gamma }_k^{\left[\alpha \right]},{\gamma }_k))\le 2(1-({\gamma }_k^{\left[\alpha \right]},{\gamma }_k))(1+({\gamma }_k^{\left[\alpha \right]},{\gamma }_k))\hfill \\ \hfill & =2(1-{({\gamma }_k^{\left[\alpha \right]},{\gamma }_k)}^2),\hfill \end{array}$$

and thus

$$\parallel {\gamma }_k^{\left[\alpha \right]}-{\gamma }_k\parallel \le \sqrt{a}\sqrt{\frac{4\sqrt{2}\sqrt{k}{L}_k^2{\lambda }_k}{{\lambda }_k-{\lambda }_{k+1}}}+\alpha \sqrt{4k(k-1)L_k^4{\left(\frac{{\lambda }_k}{{\lambda }_{k-1}-{\lambda }_k}\right)}^2\left\{1+\frac{2\parallel \mathrm{\Gamma }\parallel }{{\lambda }_k-{\lambda }_{k+1}}\right\}}.$$

Proof of Theorem 4.2

We first study the properties of the “half-smoothing” operators S_α. At the end of Section 2, we know that S_α is a bounded linear operator from L²([a,b]) to L²([a,b]) with norm less than or equal to 1. Moreover, S_α is a one to one (injective) map. Hence, its inverse $S_{\alpha }^{-1}$ exists. When α = 0, S₀ is just the identity operator I in L²([a,b]). The following lemma gives the reason why S_α is called “half-smoothing” operators.

Lemma 5

The range of S_α (or the domain of $S_{\alpha }^{-1}$) is $W_2^2\left([a,b]\right)$. Moreover, for any $f\in W_2^2\left([a,b]\right)$,

$${\parallel S_{\alpha }^{-1}f\parallel }^2={\parallel f\parallel }_{\alpha }^2.$$ (5.20)

Proof If α = 0, the results are trivial. Hence, we assume that α > 0. Since the space C^∞[a, b] of smooth functions is dense in space

$$(W_2^2\left([a,b]\right),{\parallel \cdot \parallel }_{\alpha }),$$

for any $f\in W_2^2\left([a,b]\right)$, there exists a sequence {f_m ∈ C^∞[a, b], m ∈ ℕ} such that ∥f_m − f∥_α → 0. One can see that the domain of $S_{\alpha }^{-2}=I+\alpha L^{\ast }L$ contains C^∞[a, b], hence C^∞[a, b] is also in the domain of $S_{\alpha }^{-1}$. Now we compute

$$\begin{array}{cc}\hfill & {\parallel S_{\alpha }^{-1}{f}_l-S_{\alpha }^{-1}{f}_m\parallel }^2=(S_{\alpha }^{-1}{f}_l-S_{\alpha }^{-1}{f}_m,S_{\alpha }^{-1}{f}_l-S_{\alpha }^{-1}{f}_m)\hfill \\ \hfill =& (f_l-f_m,S_{\alpha }^{-2}(f_l-f_m))=(f_l-f_m,(I+\alpha L^{\ast }L)(f_l-f_m))\hfill \\ \hfill =& (f_l-f_m,f_l-f_m)+(f_l-f_m,\alpha L^{\ast }L(f_l-f_m))\hfill \\ \hfill =& (f_l-f_m,f_l-f_m)+\alpha (L(f_l-f_m),L(f_l-f_m))\hfill \\ \hfill =& (f_l-f_m,f_l-f_m)+\alpha [f_l-f_m,f_l-f_m]={\parallel f_l-f_m\parallel }_{\alpha }\to 0,\hfill \end{array}$$ (5.21)

as m, l → ∞. Hence, $\{S_{\alpha }^{-1}{f}_m,m\in \mathbb{N}\}$ is a Cauchy sequence in L²([a, b]). It converges to some function, say g, in L²([a, b]). Since S_α is a bounded operator, $f_m=S_{\alpha }{S}_{\alpha }^{-1}{f}_m$ converges to S_αg in L²-norm. However, f_m converges to f in ∥ · ∥_α norm, it also converges in L²-norm. Therefore, S_αg = f, that is, f is in the range of S_α. Hence, $W_2^2\left([a,b]\right)$ is in the range of S_α. Because for any m ∈ ℕ, from a similar calculation as in (5.21),

$${\parallel S_{\alpha }^{-1}{f}_m\parallel }^2={\parallel f_m\parallel }_{\alpha }^2,$$

and

$$\parallel S_{\alpha }^{-1}{f}_m-S_{\alpha }^{-1}f\parallel \to 0,{\parallel f_m-f\parallel }_{\alpha }\to 0,$$

we have ${\parallel S_{\alpha }^{-1}f\parallel }^2={\parallel f\parallel }_{\alpha }^2$.

Now we show that the range of S_α is equal to $W_2^2\left([a,b]\right)$. Since we have shown that $W_2^2\left([a,b]\right)$ is in the range of S_α and S_α is a one-to-one map, we only need to show that the range of $W_2^2\left([a,b]\right)$ under $S_{\alpha }^{-1}$ is L²([a, b]). By (5.20) and the completeness of $(W_2^2\left([a,b]\right),\parallel \cdot {\parallel }_{\alpha })$, the range of $W_2^2\left([a,b]\right)$ under $S_{\alpha }^{-1}$ is a closed subspace of L²([a, b]). If the range of $W_2^2\left([a,b]\right)$ under $S_{\alpha }^{-1}$ is not L²([a, b]), then we can find 0 ≠ h ∈ L²([a, b]) such that

$$(h,S_{\alpha }^{-1}f)=0,\forall f\in W_2^2\left([a,b]\right).$$

Since one can see that the domain of $S_{\alpha }^{-2}=I+\alpha L^{\ast }L$ is contained in $W_2^2\left([a,b]\right)$, we have

$$(h,S_{\alpha }^{-1}f)=0,\forall f\in \text{domain of}{S}_{\alpha }^{-2}.$$

Then

$$(h,S_{\alpha }^{-1}f)=(S_{\alpha }^{-1}{S}_{\alpha }h,S_{\alpha }^{-1}f)=(S_{\alpha }h,S_{\alpha }^{-2}f)=0,\forall f\in \text{domain of}{S}_{\alpha }^{-2}.$$

However, because the range of $S_{\alpha }^{-2}$ is the whole L²([a, b]), we have S_αh = 0. Hence h = 0 since S_α is a one-to-one map. We get a contradiction. Therefore, the range of S_α is equal to $W_2^2\left([a,b]\right)$.

Lemma 6

$\{({\widehat{\lambda }}_j^{\left[\alpha \right]},S_{\alpha }^{-1}{\widehat{\gamma }}_j^{\left[\alpha \right]}):j\in \mathbb{N}\}$ and $\{({\lambda }_j^{\left[\alpha \right]},S_{\alpha }^{-1}{\gamma }_j^{\left[\alpha \right]}):j\in \mathbb{N}\}$ are eigenvalues and eigenfunctions of the compact operators S_αΓ̂_nS_α and S_αΓ̂S_α in L²([a, b]) respectively. Moreover, there are no other eigenvalues for S_αΓ̂_nS_α and S_αΓ̂S_α.

Note that the L² norms of $S_{\alpha }^{-1}{\widehat{\gamma }}_j^{\left[\alpha \right]}$ and $S_{\alpha }^{-1}{\gamma }_j^{\left[\alpha \right]}$ may not be 1.

Proof. If α = 0, the results are trivial. Hence, we assume that α > 0. Because $\{({\widehat{\lambda }}_j^{\left[\alpha \right]},{\widehat{\gamma }}_j^{\left[\alpha \right]}):j\in \mathbb{N}\}$ are solutions of the successive optimization problems (3.3) and (3.4), then by Lemma 5,

$$\begin{array}{cc}\hfill & \frac{\left(S_{\alpha }^{-1}{\widehat{\gamma }}_1^{\left[\alpha \right]},S_{\alpha }{\widehat{\mathrm{\Gamma }}}_n{S}_{\alpha }{S}_{\alpha }^{-1}{\widehat{\gamma }}_1^{\left[\alpha \right]}\right)}{{\parallel S_{\alpha }^{-1}{\widehat{\gamma }}_1^{\left[\alpha \right]}\parallel }^2}=\frac{\left({\widehat{\gamma }}_1^{\left[\alpha \right]},{\widehat{\mathrm{\Gamma }}}_n{\widehat{\gamma }}_1^{\left[\alpha \right]}\right)}{{\parallel {\widehat{\gamma }}_1^{\left[\alpha \right]}\parallel }_{\alpha }^2}={\widehat{\lambda }}_1^{\left[\alpha \right]}=\underset{0\ne \gamma \in W_2^2\left([a,b]\right)}{\max }\frac{\left(\gamma ,{\widehat{\mathrm{\Gamma }}}_n\gamma \right)}{{\parallel \gamma \parallel }_{\alpha }^2}\hfill \\ \hfill =& \underset{0\ne \gamma \in W_2^2\left([a,b]\right)}{\max }\frac{\left(S_{\alpha }^{-1}\gamma ,S_{\alpha }{\widehat{\mathrm{\Gamma }}}_n{S}_{\alpha }{S}_{\alpha }^{-1}\gamma \right)}{{\parallel S_{\alpha }^{-1}\gamma \parallel }^2}=\underset{0\ne \beta \in L^2\left([a,b]\right)}{\max }\frac{\left(\beta ,S_{\alpha }{\widehat{\mathrm{\Gamma }}}_n{S}_{\alpha }\beta \right)}{{\parallel \beta \parallel }^2}.\hfill \end{array}$$

Hence, $({\widehat{\lambda }}_1^{\left[\alpha \right]},S_{\alpha }^{-1}{\widehat{\gamma }}_1^{\left[\alpha \right]})$ are the first eigenvalue and the corresponding eigenfunction of S_αΓ̂_nS_α. Similarly, we can prove the conclusions for other eigenvalues and eigenfunctions.

Define

$$H=\mathrm{the}\mathrm{Banach}\mathrm{space}\mathrm{of}\mathrm{all}\mathrm{compact}\mathrm{bounded}\mathrm{operators}\mathrm{from}{L}^2\left([a,b]\right)\mathit{to}{L}^2\left([a,b]\right)\mathrm{with}\mathrm{norm}\mathrm{defined}\mathrm{in}\left(2.1\right).$$ (5.22)

For the definition and properties of compact operators in Banach spaces, we refer reader to Chapter 21 in Lax [9]. Define a sequence of stochastic processes

$$\{Z_n\left(\alpha \right)=\sqrt{n}(S_{\alpha }{\widehat{\mathrm{\Gamma }}}_n{S}_{\alpha }-S_{\alpha }\mathrm{\Gamma }{S}_{\alpha }),n\in \mathbb{N},0\le \alpha \le {\alpha }_0\},$$

which is indexed by α and takes values in H because both Γ̂_n and Γ are compact operators and S_α is a bounded operator. Note that $Z_n\left(0\right)=\sqrt{n}({\widehat{\mathrm{\Gamma }}}_n-\mathrm{\Gamma })$. We follow the notations in Dauxois et al. [3]. Let F denote the space of Hilbert-Schmidt operators from L²([a, b]) to L²([a, b]). Then F is a Hilbert space with a inner product denoted by < ·, · >_F. By Assumption 1,

$$E\left[{\parallel X\parallel }^4\right]<\infty .$$

Thus Γ̂_n, Γ ∈ F. It follows from Proposition 5 in Dauxois et al. [3] that {Z_n(0), n ∈ ℕ}, regarded as a sequence of random elements with values in F, converges in distribution to the Gaussian random element in F with mean 0 and covariance operator Q, where

$$Q=E\left[(X\otimes X-\mathrm{\Gamma })\tilde{\otimes }(X\otimes X-\mathrm{\Gamma })\right]=E\left[(X\otimes X)\tilde{\otimes }(X\otimes X)\right]-\mathrm{\Gamma }\tilde{\otimes }\mathrm{\Gamma }.$$ (5.23)

X ⊗ X denotes the bounded operator from L²([a, b]) to L²([a, b]) with (X ⊗ X) (γ) = (γ, X)X for any γ ∈ L²([a, b]). Γ⊗̃Γ denotes the bounded operator from F to F with (Γ⊗̃Γ)(Λ) = 〈Λ,Γ〉_F Γ for any Λ ∈ F. The other terms in (5.23) are denned similarly. Note that according to the definition (5.23), Q is an operator from F to F. However, because F is a Hilbert space, there is an isometry between F and its dual space F′. Hence, Q can be regarded as a bounded operator from F′ to F and then it satisfies the definition of covariance operators in Remark (1) after Theorem 4.2. However, in this paper, we will consider the space H of compact operators which is larger than the space F of Hilbert-Schmidt operators (every Hilbert-Schmidt operator is compact). In the proof of Proposition 6 in Dauxois et al. [3], the authors used the fact that if A is a Hilbert-Schmidt operator, then (A − zI)⁻¹ is also a Hilbert-Schmidt operator, where z is a complex which is not an eigenvalue of A and I is the identity operator. However, this is not true in general. But (A − zI)⁻¹ is a bounded operator. Because the norm (2.1) in H is smaller than the norm in F, the embedding map i : F ↪ H (i maps any Hilbert-Schmidt operator to itself) is a bounded operator. Then we have

Lemma 7

{Z_n(0), n ∈ ℕ}, regarded as a sequence of random, elements with values in H, converges in distribution to a Gaussian random element in H with mean zero and covariance operator iQi*, where i* is the adjoint operator of i and Q is defined in (5.23).

Proof. It follows immediately from the following lemma.

Lemma 8

Suppose that {X_n, n ≥ 1} is a sequence of random, elements with values in a Banach space B. If X_n converges in distribution to a Gaussian random element X with mean zero and covariance operator Λ. Let T be a bounded operator (that is, a continuous linear function) from B to another Banach space C. Then T(X_n) converges in distribution to T(X) which is also a Guassian random element with mean zero and covariance operator TΛT*, where T* is the adjoint operator of T.

Proof. Since T is a continuous map from B to C, by continuous mapping theorem, T(X_n) converges in distribution to T(X). Now we show that T(X) is an Guassian random element. For any bounded linear functional f ∈ C′, fοT ∈ B′. Hence, f(T(X)) = f ο T(X) is a Gaussian random variable since X is Gaussian. Thus T(X) is Guassian and obviously its mean is zero. In order to compute it covariance operator, we intruduce the following notations. For any x ∈ B, y ∈ C and f ∈ B′, g ∈ C′, define 〈x, f〉_B = f(x), 〈y, g〉_C = g(y). By the definition of covariance operators (see Remark (1) after Theorem 4.2) and the definition of adjoint operators, for any g, h ∈ C′,

$$\begin{array}{cc}\hfill E\left[g\left(T\left(X\right)\right)h\left(T\left(X\right)\right)\right]& =E\left[(g\circ T\left(X\right))(h\circ T\left(X\right))\right]=E\left[\left(T^{\ast }\left(g\right)\left(X\right)\right)\left(T^{\ast }\left(f\right)\left(X\right)\right)\right]\hfill \\ \hfill =& {\langle \mathrm{\Lambda }\left(T^{\ast }\left(g\right)\right),T^{\ast }\left(f\right)\rangle }_B={\langle \mathrm{\Lambda }{T}^{\ast }\left(g\right),T^{\ast }\left(f\right)\rangle }_B={\langle T\mathrm{\Lambda }{T}^{\ast }\left(g\right),f\rangle }_C.\hfill \end{array}$$

Therefore, the covariance operator of TX is TΛT*.

Lemma 9

For any finite 0 ≤ α₁ < … < α_k ≤ α₀, the sequence

$$\{(Z_n\left({\alpha }_1\right),\cdots ,Z_n\left({\alpha }_k\right)),n\in \mathbb{N}\}$$

converges in distribution to a Gaussian random element with values in H^k and mean zero, where H^k is the product space of k copies of H.

Proof. This lemma follows from Lemma 8 and the fact that

$$(Z_n\left({\alpha }_1\right),\cdots ,Z_n\left({\alpha }_k\right))=(S_{\alpha 1}{Z}_n\left(0\right)S_{\alpha 1},\cdots ,S_{\alpha k}{Z}_n\left(0\right)S_{\alpha k})$$

is a continuous and linear function of Z_n(0) since S_α1, i = 1,… , k are bounded operators.

Unfortunately, S_α is not continuous as α → 0 under the norm (2.1). For example, let

$$[a,b]=[0,2\pi ],f_n\left(t\right)=\frac{1}{\sqrt{2\pi }}{e}^{\mathit{int}}.$$

By (5.20),

$${\parallel S_{\alpha }^{-1}{f}_n\parallel }^2={\parallel f_n\parallel }_{\alpha }^2={\parallel f_n\parallel }^2+\alpha [f_n,f_n]=1+\alpha n^4.$$

Define $g_n=\frac{1}{\sqrt{1+\alpha n^4}}{S}_{\alpha }^{-1}{f}_n$. Then ∥g_n∥ = 1 and

$$\begin{array}{cc}\hfill & \parallel (S_{\alpha }-I)g_n\parallel =\parallel \frac{1}{\sqrt{1+\alpha n^4}}{f}_n-g_n\parallel \hfill \\ \hfill \ge \parallel g_n\parallel & -\parallel \frac{1}{\sqrt{1+\alpha n^4}}{f}_n\parallel =1-\frac{1}{\sqrt{1+\alpha n^4}}.\hfill \end{array}$$

Therefore, ∥S_α − I∥ ≥ 1 for all α. Note that S₀ = I. However, we have the following results.

Lemma 10

For any f ∈ L²([a, b]), α → S_αf is a continuous map from [0, α₀] to L²([a, b]).

Proof. Let E be the resolution of the identity for the self-adjoint operator S_α0 (for reference, see Chapter 12 of Rudin [14]). Because S_α0 is a positive operator with ∥S_α0∥ ≤ 1, E_{f, f} is a bounded positive Borel measure in [0, 1]. Fix α ∈ [0, α₀].

$$\begin{array}{cc}\hfill & S_{\alpha }={(I+\alpha L^{\ast }L)}^{-\frac{1}{2}}={((1-\frac{\alpha }{{\alpha }_0})I+\frac{\alpha }{{\alpha }_0}(I+{\alpha }_0{L}^{\ast }L))}^{-\frac{1}{2}}\hfill \\ \hfill =& {((1-\frac{\alpha }{{\alpha }_0})I+\frac{\alpha }{{\alpha }_0}{S}_{{\alpha }_0}^{-2})}^{-\frac{1}{2}}=S_{{\alpha }_0}{(\frac{\alpha }{{\alpha }_0}+(1-\frac{\alpha }{{\alpha }_0})S_{{\alpha }_0}^2)}^{-\frac{1}{2}}.\hfill \end{array}$$

Now define a family continuous functions on [0, 1],

$${\phi }_{\alpha }\left(x\right)=\{\begin{array}{cc}\frac{x}{\sqrt{\frac{\alpha }{{\alpha }_0}+(1-\frac{\alpha }{{\alpha }_0})x^2}},\hfill & 0<\alpha \le {\alpha }_0\hfill \\ 1\hfill & \alpha =0,\hfill \end{array}\phantom{\}},$$

then S_α = φ_α(S_α0). Let α′ ∈ [0, α₀] and α′ → α. It follows from Theorem 12.21 and 12.23 in Chapter 12 of Rudin [14] that

$${\parallel (S_{{\alpha }^{\prime }}-S_{\alpha })f\parallel }^2={\int }_0^1{({\phi }_{{\alpha }^{\prime }}\left(x\right)-{\phi }_{\alpha }\left(x\right))}^2{\mathit{dE}}_{f,f}\left(x\right).$$

The integrand on the right hand side is bounded. If α ≠ 0, the integrand converges to 0 at each point in [0, 1] as α′ → α. By the bounded convergence theorem, ∥(S_α′ − S_α)f∥² → 0. If α = 0, the integrand converges to 0 at each point in [0, 1] except 0. If we can show that the measure value E_{f, f}({0}) of E_{f, f} on the set {0} is zero, then by the bounded convergence theorem, we still have ∥(S_α′ − S_α) f∥² → 0. In fact, for any g ∈ L²([a, b]),

$$(g,S_{{\alpha }_0}E\left(\left\{0\right\}\right)f)={\int }_{\left\{0\right\}}{\mathit{xdE}}_{g,f}\left(x\right)=0.$$

Hence, S_α0 E({0})f = 0. Because S_α0 is a one-to-one operator, E({0})f = 0. Therefore,

$$E_{f,f}\left(\left\{0\right\}\right)=(f,E\left(\left\{0\right\}\right)f)=0.$$

Lemma 11

For any compact operator Λ in L²([a, b]), α → S_αΛS_α is a continuous map from [0, α₀] to H.

Proof. By Lemma 11 in Section XI.9 of Dunford and Schwartz [5], there exists a sequence Λ_m of bounded operators having finite-dimensional range, such that ∥Λ_m − Λ∥ → 0. If we can show that for each m, α → S_αΛ_mS_α is a continuous map, then since ∥S_αΛ_mS_α − S_αΛS_α∥ ≥ ∥Λ_m − Λ∥ → 0 uniformly, α → S_αΛS_α is continuous. Now fix m and 0 ≤ α ≤ α₀. Let {e₁, …, e_k} be an orthonormal basis of the range of Λ_m and α′ → α. For any f ∈ L² ([a, b]) with ∥f∥ ≤ 1,

$$\begin{array}{cc}\hfill & \parallel S_{{\alpha }^{\prime }}{\mathrm{\Lambda }}_m{S}_{{\alpha }^{\prime }}f-S_{\alpha }{\mathrm{\Lambda }}_m{S}_{\alpha }f\parallel =\parallel (S_{{\alpha }^{\prime }}-S_{\alpha }){\mathrm{\Lambda }}_m{S}_{{\alpha }^{\prime }}f+S_{\alpha }{\mathrm{\Lambda }}_m(S_{{\alpha }^{\prime }}-S_{\alpha })f\parallel \hfill \\ \hfill \le & \parallel (S_{{\alpha }^{\prime }}-S_{\alpha }){\mathrm{\Lambda }}_m{S}_{{\alpha }^{\prime }}f\parallel +\parallel S_{\alpha }{\mathrm{\Lambda }}_m(S_{{\alpha }^{\prime }}-S_{\alpha })f\parallel \hfill \\ \hfill =& \parallel (S_{{\alpha }^{\prime }}-S_{\alpha }){\mathrm{\Lambda }}_m(S_{{\alpha }^{\prime }}-S_{\alpha })f+(S_{{\alpha }^{\prime }}-S_{\alpha }){\mathrm{\Lambda }}_m{S}_{\alpha }f\parallel +\parallel S_{\alpha }{\mathrm{\Lambda }}_m(S_{{\alpha }^{\prime }}-S_{\alpha })f\parallel \hfill \\ \hfill \le & \parallel (S_{{\alpha }^{\prime }}-S_{\alpha })\parallel \parallel {\mathrm{\Lambda }}_m(S_{{\alpha }^{\prime }}-S_{\alpha })f\parallel +\parallel (S_{{\alpha }^{\prime }}-S_{\alpha }){\mathrm{\Lambda }}_m{S}_{\alpha }f\parallel \hfill \\ \hfill & +\parallel S_{\alpha }\parallel \parallel {\mathrm{\Lambda }}_m(S_{{\alpha }^{\prime }}-S_{\alpha })f\parallel \le 3\parallel {\mathrm{\Lambda }}_m(S_{{\alpha }^{\prime }}-S_{\alpha })f\parallel +\parallel (S_{{\alpha }^{\prime }}-S_{\alpha }){\mathrm{\Lambda }}_m{S}_{\alpha }f\parallel .\hfill \end{array}$$

Because

$${\mathrm{\Lambda }}_m{S}_{\alpha }f=\underset{i=1}{\overset{k}{\Sigma }}({\mathrm{\Lambda }}_m{S}_{\alpha }f,e_i)e_i$$

$$\begin{array}{cc}\hfill \parallel (S_{{\alpha }^{\prime }}-S_{\alpha }){\mathrm{\Lambda }}_m{S}_{\alpha }f\parallel & \le \underset{i=1}{\overset{k}{\Sigma }}\left|({\mathrm{\Lambda }}_m{S}_{\alpha }f,e_i)\right|\parallel (S_{{\alpha }^{\prime }}-S_{\alpha })e_i\parallel \hfill \\ \hfill & \le \underset{i=1}{\overset{k}{\Sigma }}\parallel {\mathrm{\Lambda }}_m\parallel \parallel (S_{{\alpha }^{\prime }}-S_{\alpha })e_i\parallel \hfill \end{array}$$

which converges to 0 uniformly for all f ∈ L²([a, b]) with ∥f∥ ≤ 1 by Lemma 10. Now

$$\begin{array}{cc}\hfill & {\parallel {\mathrm{\Lambda }}_m(S_{{\alpha }^{\prime }}-S_{\alpha })f\parallel }^2=\underset{i=1}{\overset{k}{\Sigma }}{\left|({\mathrm{\Lambda }}_m(S_{{\alpha }^{\prime }}-S_{\alpha })f,e_i)\right|}^2\hfill \\ \hfill =& \underset{i=1}{\overset{k}{\Sigma }}{\left|(f,(S_{{\alpha }^{\prime }}-S_{\alpha }){\mathrm{\Lambda }}_m^{\ast }{e}_i)\right|}^2\le \underset{i=1}{\overset{k}{\Sigma }}{\parallel (S_{{\alpha }^{\prime }}-S_{\alpha }){\mathrm{\Lambda }}_m^{\ast }{e}_i\parallel }^2.\hfill \end{array}$$

which converges to 0 uniformly for all f ∈ L²([a, b]) with ∥f∥ ≤ 1 by Lemma 10, where Λ_m* is the adjoint operator of Λ_m. Hence, ∥S_α′Λ_mS_α′ − S_αΛ_mS_α∥ → 0.

In the next lemma, we assume that all the eigenfunctions have norms 1.

Lemma 12

Suppose that α → Λ(α) is a continuous map from [0, α₀] to the suhspace of positive compact operators in L²([a, b]) in H. Assume that the first K eigenvalues of Λ(α) for any α ∈ [0, α₀] are positive and mutually different, and each of them has multiplicity 1. Then given the first K eigenfunctions $\{e_k^{\left[0\right]},1\le k\le K\}$ of Λ(0), there exist unique choices of the first k eigenfunctions $\{e_k^{\left[\alpha \right]},1\le k\le K\}$ of Λ(α) for any α ∈ (0, α₀] such that $\alpha \to e_k^{\left[\alpha \right]}$ is a continuous map from [0, α₀] to L²([a, b]) for any 1 ≤ k ≤ K.

Note that for each 1 ≤ k ≤ K and 0 ≤ α ≤ α₀, there exist two eigenfunctions with norm 1 of Λ(α) corresponding its k-th eigenvalues and any one of the two eigenfunctions is equal to the other one multiplied by −1.

Proof. Let ${\mu }_1^{\left[\alpha \right]}>\cdots >{\mu }_K^{\left[\alpha \right]}>0$ be the first K eigenvalues of Λ(α). Let E^k(α) be the orthogonal projection onto the space spanned by the $e_k^{\left[\alpha \right]}$, 1 ≤ k ≤ K, 0 ≤ α ≤ α₀. Note E^k(α) does not depend on the sign of $e_k^{\left[\alpha \right]}$.

We first show that for any 1 ≤ k ≤ K, E^k(α) is a continuous function of from [0, α₀] to H. For any fixed α ∈ [0, α₀], we can find a small positive number ε_α such that the K + 1 intervals

$$[{\mu }_1^{\left[\alpha \right]}-{∊}_{\alpha },{\mu }_1^{\left[\alpha \right]}+{∊}_{\alpha }],[{\mu }_2^{\left[\alpha \right]}-{∊}_{\alpha },{\mu }_2^{\left[\alpha \right]}+{∊}_{\alpha }],\cdots ,[{\mu }_{K+1}^{\left[\alpha \right]}-{∊}_{\alpha },{\mu }_{K+1}^{\left[\alpha \right]}+{∊}_{\alpha }]$$

are disjoint. Since Λ(α) is a continuous function, we can choose a neighborhood ℳ_α of α in [0, α₀], such that for any α′ ∈ ℳ_α

$$\underset{1\le k\le K+1}{\max }|{\mu }_k^{\left[{\alpha }^{\prime }\right]}-{\mu }_k^{\left[\alpha \right]}|\le \parallel \mathrm{\Lambda }\left({\alpha }^{\prime }\right)-\mathrm{\Lambda }\left(\alpha \right)\parallel \le \frac{{∊}_{\alpha }}{4}.$$

where the first inequality follows from Corollary 4 in Section XI.9 of Dunford and Schwartz [5]. Now we define K circles on the complex plane ℂ,

$$C_k=\text{the circle with center}{\mu }_k^{\left[\alpha \right]}\text{and radius}{∊}_{\alpha },1\le k\le K.$$

Then one can see that for any α′ ∈ ℳ_α, the disk bounded by the circle C_k only contains the k-th eigenvalues ${\mu }_k^{\left[{\alpha }^{\prime }\right]}$ of Λ(α′). Hence, we have (see Section VII.3 of Dunford and Schwartz [4] or Definition 10.26 in Rudin [14])

$$E^k\left({\alpha }^{\prime }\right)=\frac{1}{2\pi i}{\int }_{C_k}{(\mathit{zI}-\mathrm{\Lambda }\left({\alpha }^{\prime }\right))}^{-1}\mathit{dz},$$

for any α′ ∈ ℳ_α. Since (zI − Λ(α′))⁻¹ is a continuous function of z ∈ C_k C_k is a compact set, we have

$$M=\underset{z\in C_k}{\sup }\parallel {(\mathit{zI}-\mathrm{\Lambda }\left({\alpha }^{\prime }\right))}^{-1}\parallel <\infty .$$ (5.24)

Since Λ(α) is a continuous function of α, for any 0 < δ < 1, we can find a neighborhood 𝒩_α of α such that

$$\parallel \mathrm{\Lambda }\left({\alpha }^{\prime }\right)-\mathrm{\Lambda }\left(\alpha \right)\parallel \le \frac{\delta }{M},\forall {\alpha }^{\prime }\in {\mathcal{N}}_{\alpha }.$$ (5.25)

Now for any α′ ∈ ℳ_α ⋂ 𝒩_α,

$$\begin{array}{cc}\hfill & \parallel E^k\left({\alpha }^{\prime }\right)-E^k\left(\alpha \right)\parallel \le \frac{1}{2\pi }{\int }_{C_k}\parallel {(\mathit{zI}-\mathrm{\Lambda }\left({\alpha }^{\prime }\right))}^{-1}-{(\mathit{zI}-\mathrm{\Lambda }\left(\alpha \right))}^{-1}\parallel \mathit{dz}\hfill \\ \hfill =& \frac{1}{2\pi }{\int }_{C_k}\parallel {(\mathit{zI}-\mathrm{\Lambda }\left(\alpha \right)-(\mathrm{\Lambda }\left({\alpha }^{\prime }\right)-\mathrm{\Lambda }\left(\alpha \right)))}^{-1}-{(\mathit{zI}-\mathrm{\Lambda }\left(\alpha \right))}^{-1}\parallel \mathit{dz}\hfill \\ \hfill =& \frac{1}{2\pi }{\int }_{C_k}\parallel {(\mathit{zI}-\mathrm{\Lambda }\left(\alpha \right))}^{-1}{\left(I-(\mathrm{\Lambda }\left({\alpha }^{\prime }\right)-\mathrm{\Lambda }\left(\alpha \right)){(\mathit{zI}-\mathrm{\Lambda }\left(\alpha \right))}^{-1}\right)}^{-1}-{(\mathit{zI}-\mathrm{\Lambda }\left(\alpha \right))}^{-1}\parallel \mathit{dz}\hfill \\ \hfill \le & \frac{1}{2\pi }{\int }_{C_k}\parallel {(\mathit{zI}-\mathrm{\Lambda }\left(\alpha \right))}^{-1}\parallel \parallel {\left(I-(\mathrm{\Lambda }\left({\alpha }^{\prime }\right)-\mathrm{\Lambda }\left(\alpha \right)){(\mathit{zI}-\mathrm{\Lambda }\left(\alpha \right))}^{-1}\right)}^{-1}-I\parallel \mathit{dz}\hfill \\ \hfill \le & \frac{M}{2\pi }{\int }_{C_k}\parallel \left(I+\underset{k=1}{\overset{\infty }{\Sigma }}{\left[(\mathrm{\Lambda }\left({\alpha }^{\prime }\right)-\mathrm{\Lambda }\left(\alpha \right)){(\mathit{zI}-\mathrm{\Lambda }\left(\alpha \right))}^{-1}\right]}^k\right)-I\parallel \mathit{dz}\hfill \\ \hfill \le & \frac{M}{2\pi }{\int }_{C_k}\underset{k=1}{\overset{\infty }{\Sigma }}{\left[\parallel \mathrm{\Lambda }\left({\alpha }^{\prime }\right)-\mathrm{\Lambda }\left(\alpha \right))\parallel \parallel {(\mathit{zI}-\mathrm{\Lambda }\left(\alpha \right))}^{-1}\parallel \right]}^k\mathit{dz}\hfill \\ \hfill \le & \frac{M}{2\pi }{\int }_{C_k}\mathit{dz}\underset{k=1}{\overset{\infty }{\Sigma }}{\left[\frac{\delta }{M}M\right]}^k\hfill \end{array}$$ (5.26)

by (5.24) and (5.25)

$$=\frac{M}{2\pi }{\int }_{C_k}\mathit{dz}\frac{\delta }{1-\delta }$$

Since δ can be arbitrarily small, E^k(α) is continuous at α.

Now we show that for any given α ∈ [0, α₀], and given $e_k^{\left[\alpha \right]}$, there exists a neighborhood [α¹, α²] of α such that for any α′ ∈ [α¹, α²], we can uniquely choose $e_k^{\left[{\alpha }^{\prime }\right]}$ such that $e_k^{\left[{\alpha }^{\prime }\right]}$ is continuous in this neighborhood. Because E^k(α′) is a continuous function of α′, $\parallel E^k\left({\alpha }^{\prime }\right)e_k^{\left[\alpha \right]}\parallel$ is a continuous function of α′ and its value is 1 at α′ = α. Hence, we can find a neighborhood [α¹, α²] of α such that $\parallel E^k\left({\alpha }^{\prime }\right)e_k^{\left[\alpha \right]}\parallel \ge \frac{1}{2}$ for α′ ∈ [α¹, α²]. Then

$$e_k^{\left[{\alpha }^{\prime }\right]}=\frac{E^k\left({\alpha }^{\prime }\right)e_k^{\left[\alpha \right]}}{\parallel E^k\left({\alpha }^{\prime }\right)e_k^{\left[\alpha \right]}\parallel },$$

are eigenfunctions and continuous in [α¹, α²]. Now we show the uniqueness. Suppose ${\tilde{e}}_k^{\left[\alpha \right]}$, α′ ∈ [α¹, α²] is another choice of the eigenfunctions such that it is continuous and ${\tilde{e}}_k^{\left[\alpha \right]}=e_k^{\left[\alpha \right]}$. If for some ${\alpha }^{″}\in [{\alpha }^1,{\alpha }^2]$, $e_k^{\left[{\alpha }^{″}\right]}\ne {\tilde{e}}_k^{\left[{\alpha }^{″}\right]}$, we have $e_k^{\left[{\alpha }^{″}\right]}=-{\tilde{e}}_k^{\left[{\alpha }^{″}\right]}$. Since both the inner products $(e_k^{\left[\alpha \right]},e_k^{\left[{\alpha }^{\prime }\right]})$ and $(e_k^{\left[\alpha \right]},{\tilde{e}}_k^{\left[{\alpha }^{\prime }\right]})$ are continuous functions for α′ ∈ [α¹, α²]. By the choice of [α¹, α²], $\left|(e_k^{\left[\alpha \right]},e_k^{\left[{\alpha }^{″}\right]})\right|=\left|(e_k^{\left[\alpha \right]},{\tilde{e}}_k^{\left[{\alpha }^{″}\right]})\right|\ge \frac{1}{2}$. Because $(e_k^{\left[\alpha \right]},e_k^{\left[{\alpha }^{″}\right]})=-(e_k^{\left[\alpha \right]},{\tilde{e}}_k^{\left[{\alpha }^{″}\right]})$, one of them must be negative. Without loss of generality, we assume that $(e_k^{\left[\alpha \right]},e_k^{\left[{\alpha }^{″}\right]})<0$. Since $(e_k^{\left[\alpha \right]},e_k^{\left[\alpha \right]})=1>0$, it follows from the intermediate value theorem that there is at least one point α‴ between α and α″ such that $(e_k^{\left[\alpha \right]},e_k^{\left[{\alpha }^{″\prime }\right]})=0$. However, it is impossible because

$$\left|(e_k^{\left[\alpha \right]},e_k^{\left[{\alpha }^{‴}\right]})\right|=\parallel E^k\left({\alpha }^{‴}\right)e_k^{\left[\alpha \right]}\parallel \ge \frac{1}{2}.$$

Hence we have proved the uniqueness.

Fix $e_k^{\left[0\right]}$. Let the set

$$V=\{\alpha \in [0,{\alpha }_0]:\text{we can uniquely choose}{e}_k^{\left[{\alpha }^{\prime }\right]}\text{for}{\alpha }^{\prime }\in [0,{\alpha }_0]\text{such that}{e}_k^{\left[{\alpha }^{\prime }\right]}\text{is continous in}[0,\alpha ]\}.$$

By the arguments in the last paragraph, 𝒱 is nonempty. Now we show that the set 𝒱 is an open set. Suppose that α* is any point in 𝒱. It follows from the last paragraph that there exists a neighborhood [α¹, α²] of α* such that given e^[α*], we can uniquely choose the sign of e^[α] for any α ∈ [α¹, α²] to make e^[α], α ∈ [α¹, α²] a continuous function. We show that [α¹, α²] ⊂ 𝒱. Let α** be any point in [α¹, α²]. It is easy to see that we can choose the signs of e^[α] for all α ∈ [0, α**] such that e^[α] is a continuous function of α in [0, α**]. We only need to show the uniqueness of e^[α]. The uniqueness is obvious if α** ≥ α* since α* ∈ 𝒱. Hence we assume that α** < α*. We will proceed by contradiction. Assume that there are two different continuous functions ${\tilde{e}}^{\left[\alpha \right]}$ and ${\stackrel{\approx }{e}}^{\left[\alpha \right]}$, 0 ≤ α ≤ α**. By the definition of [α¹, α²], we can choose a continuous function ${\widehat{e}}^{\left[\alpha \right]}$, α** ≤ α ≤ α*. Define

$$e^{\prime \left[\alpha \right]}=\{\begin{array}{ccc}\hfill {\tilde{e}}^{\left[\alpha \right]}& \text{if}\hfill & 0\le \alpha \le {\alpha }^{\ast \ast }\hfill \\ \hfill {\widehat{e}}^{\left[\alpha \right]}& \text{if}\hfill & {\alpha }^{\ast \ast }\le \alpha \le {\alpha }^{\ast }\text{and}{\tilde{e}}^{\left[{\alpha }^{\ast \ast }\right]}={\widehat{e}}^{\left[{\alpha }^{\ast \ast }\right]}\hfill \\ \hfill -\widehat{e}\left[\alpha \right]& \text{if}\hfill & {\alpha }^{\ast \ast }\le \alpha \le {\alpha }^{\ast }\text{and}{\tilde{e}}^{\left[{\alpha }^{\ast \ast }\right]}=-{\widehat{e}}^{\left[{\alpha }^{\ast \ast }\right]}\hfill \end{array}\phantom{\}},$$

and

$$e^{″\left[\alpha \right]}=\{\begin{array}{ccc}\hfill {\stackrel{\approx }{e}}^{\left[\alpha \right]}& \text{if}\hfill & 0\le \alpha \le {\alpha }^{\ast \ast }\hfill \\ \hfill {\widehat{e}}^{\left[\alpha \right]}& \text{if}\hfill & {\alpha }^{\ast \ast }\le \alpha \le {\alpha }^{\ast }\text{and}{\stackrel{\approx }{e}}^{\left[{\alpha }^{\ast \ast }\right]}={\widehat{e}}^{\left[{\alpha }^{\ast \ast }\right]}\hfill \\ \hfill -\widehat{e}\left[\alpha \right]& \text{if}\hfill & {\alpha }^{\ast \ast }\le \alpha \le {\alpha }^{\ast }\text{and}{\stackrel{\approx }{e}}^{\left[{\alpha }^{\ast \ast }\right]}=-{\widehat{e}}^{\left[{\alpha }^{\ast \ast }\right]}\hfill \end{array}\phantom{\}}$$

Then ${\tilde{e}}^{\left[\alpha \right]}$ and ${\stackrel{\approx }{e}}^{\left[\alpha \right]}$ are two different continuous functions in [0, α*], which contradicts to α* ∈ 𝒱. Hence, 𝒱 is an open set.

Now if we can prove that 𝒱 is also a closed set, we have 𝒱= [0, α₀]. Let α_m ∈ 𝒱 be a sequence of positive numbers converging to α ∈ [0, α₀]. If for some m, α_m ≥ α it is obvious that α ∈ 𝒱. Hence we assume that α_m < α for all m. Then we can uniquely choose the signs of of $e_k^{\left[{\alpha }^{\prime }\right]}$ such that $e_k^{\left[{\alpha }^{\prime }\right]}$ is continuous in [0, α). Let $e_k^{\left[\alpha \right]}$ be one of the two eigenfunctions with norm 1. Because for any α′ < α

$$|{(e_k^{\left[\alpha \right]},e_k^{\left[{\alpha }^{\prime }\right]})}^2-1|=|(e_k^{\left[\alpha \right]},E^k\left({\alpha }^{\prime }\right)e_k^{\left[\alpha \right]})-(e_k^{\left[\alpha \right]},E^k\left(\alpha \right)e_k^{\left[\alpha \right]})|\le \parallel E^k\left({\alpha }^{\prime }\right)-E^k\left(\alpha \right)\parallel$$

goes to zero as α′ → α, ${(e_k^{\left[\alpha \right]},e_k^{\left[{\alpha }^{\prime }\right]})}^2\to 1$. Since $e_k^{\left[{\alpha }^{\prime }\right]}$ continuous in [0, α). $(e_k^{\left[\alpha \right]},e_k^{\left[{\alpha }^{\prime }\right]})$ converges either to 1 or −1. In the latter case, we change $e_k^{\left[\alpha \right]}$ to $-e_k^{\left[\alpha \right]}$. Hence, without loss of generality, we assume that $(e_k^{\left[\alpha \right]},e_k^{\left[{\alpha }^{\prime }\right]})\to 1$ as α′ → α. Now one can see that $e_k^{\left[{\alpha }^{\prime }\right]}$ is continuous on [0, α] and its uniqueness is obvious. Hence, α ∈ 𝒱. We have proven that 𝒱 is a close set.

Define C_H [0, α₀] to be the space of all the continuous function from [0, α₀] → H (see Chapter 3 of Billingsley [1]). For any {Λ(α) : 0 ≤ α ≤ α₀} ∈ C_H[0, α₀], define a norm

$$\parallel \mathrm{\Lambda }\parallel =\underset{0\le \alpha \le {\alpha }_0}{\sup }\parallel \mathrm{\Lambda }\left(\alpha \right)\parallel .$$ (5.27)

Under the norm (5.27), C_H[0, α₀] is a Banach space. Recall the definition

$$\{Z_n\left(\alpha \right)=\sqrt{n}(S_{\alpha }{\widehat{\mathrm{\Gamma }}}_n{S}_{\alpha }-S_{\alpha }\mathrm{\Gamma }{S}_{\alpha }),n\in \mathbb{N},0\le \alpha \le {\alpha }_0\}.$$

By Lemma 11, we can regard the stochastic processes Z_n in [0, α] as random elements with values in C_H [0, α₀]. Define a linear map Θ: H → C_H[0, α₀] such that for any compact operator U ∈ H,

$$\mathrm{\Theta }\left(U\right)=\{S_{\alpha }{\mathit{US}}_{\alpha },0\le \alpha \le {\alpha }_0\}.$$ (5.28)

Lemma 13

Θ is a bounded operator and the sequence {Z_n, n ∈ ℕ} of stochastic processes with sample paths in C_H[0, α₀] converges in distribution to the Gaussian random, element with mean zero and covariance operator ΘiQi*Θ*.

Proof. Since the norm of S_α is less than or equal to 1, for any V ∈ H,

$$\underset{0\le \alpha \le {\alpha }_0}{\sup }\parallel S_{\alpha }{\mathit{US}}_{\alpha }-S_{\alpha }{\mathit{VS}}_{\alpha }\parallel \le \parallel U-V\parallel .$$

Hence, the map (5.28) is continuous and hence a bounded operator. Since Z_n = Θ(Z_n(0)), the lemma follows from Lemmas 7 and 8.

Now for any 1 ≤ k ≤ K, define

$${\widehat{\eta }}_k^{\left[\alpha \right]}=\frac{S_{\alpha }^{-1}{\widehat{\gamma }}_k^{\left[\alpha \right]}}{\parallel S_{\alpha }^{-1}{\widehat{\gamma }}_k^{\left[\alpha \right]}\parallel },{\eta }_k^{\left[\alpha \right]}=\frac{S_{\alpha }^{-1}{\gamma }_k^{\left[\alpha \right]}}{\parallel S_{\alpha }^{-1}{\gamma }_k^{\left[\alpha \right]}\parallel }.$$ (5.29)

Note that by Lemma 6, ${\widehat{\eta }}_k^{\left[\alpha \right]}$ and ${\eta }_k^{\left[\alpha \right]}$ are the eigenfunctions of S_αΓ̂S_α and S_αΓS_α with norms 1. By (5.29) and because $\parallel {\widehat{\gamma }}_k^{\left[\alpha \right]}\parallel =1$ and $\parallel {\gamma }_k^{\left[\alpha \right]}\parallel =1$, we have

$$\parallel S_{\alpha }{\widehat{\eta }}_k^{\left[\alpha \right]}\parallel =\frac{1}{\parallel S_{\alpha }^{-1}{\widehat{\gamma }}_k^{\left[\alpha \right]}\parallel },\parallel S_a{\eta }_k^{\left[\alpha \right]}\parallel =\frac{1}{\parallel S_{\alpha }^{-1}{\gamma }_k^{\left[\alpha \right]}\parallel },$$ (5.30)

and

$${\widehat{\gamma }}_k^{\left[\alpha \right]}=\frac{S_{\alpha }{\widehat{\eta }}_k^{\left[\alpha \right]}}{\parallel S_{\alpha }{\widehat{\eta }}_k^{\left[\alpha \right]}\parallel },{\gamma }_k^{\left[\alpha \right]}=\frac{S_{\alpha }{\eta }_k^{\left[\alpha \right]}}{\parallel S_{\alpha }{\eta }_k^{\left[\alpha \right]}\parallel }.$$ (5.31)

Define ${\widetilde{∊}}_k=\frac{{\lambda }_k-{\lambda }_{k+1}}{4}$, 1 ≤ k ≤ K, and ε_K = min_{1≤k≤K ε̃k}. Then the K + 1 intervals

$$[{\lambda }_1-{\widetilde{∊}}_1,{\lambda }_1+{\widetilde{∊}}_1],[{\lambda }_2-{\widetilde{∊}}_2,{\lambda }_2+{\widetilde{∊}}_1],\cdots ,[{\lambda }_K-{\widetilde{∊}}_K,{\lambda }_K+{\widetilde{∊}}_{K-1}],[{\lambda }_{K+1}-{\widetilde{∊}}_K,{\lambda }_{K+1}+{\widetilde{∊}}_K],$$ (5.32)

are disjoint. By the definition (4.3) of α₀ and (5.5) in Lemma 3, for any 0 ≥ α ≥ α₀ and 1 ≥ k ≥ K,

$$0\le {\lambda }_k-{\lambda }_k^{\left[\alpha \right]}\le \alpha {\mathit{kL}}_k^2{\lambda }_k\le {\alpha }_0{\mathit{kL}}_k^2{\lambda }_k\le \frac{{\lambda }_k-{\lambda }_{k+1}}{16{\mathit{kL}}_k^2{\lambda }_k}{\mathit{kL}}_k^2{\lambda }_k\le \frac{{\widetilde{∊}}_k}{4}.$$ (5.33)

Hence, ${\lambda }_1^{\left[\alpha \right]},\cdots ,{\lambda }_K^{\left[\alpha \right]}$ are different mutually for all 0 ≥ α ≥ α₀. Now given γ_k, 1 ≤ k ≤ K, by Lemma 11 and Lemma 12, we can uniquely choose the first K eigenfunctions $\{{\eta }_k^{\left[\alpha \right]},1\le k\le K\}$ of S_αΓS_α such that ${\eta }_k^{\left[0\right]}={\gamma }_k$ and ${\eta }_k^{\left[\alpha \right]}$, 1 ≤ k ≤ K, are continuous functions of α. We have proved the claims about the continuity of ${\gamma }_k^{\left[\alpha \right]}$, 1 ≤ k ≤ K at the beginning of the proof of Theorem 4.1.

Now we define K circles in the complex plane ℂ,

$$\begin{array}{cc}\hfill & C_1=\text{the circle with center}{\lambda }_1\text{and radius}{\widetilde{∊}}_1,1\le k\le K,\hfill \\ \hfill & C_k=\text{the circle with center}{\lambda }_k+\frac{{\widetilde{∊}}_{k-1}-{\widetilde{∊}}_k}{2}\text{and radius}\frac{{\widetilde{∊}}_{k-1}+{\widetilde{∊}}_k}{2},1\le k\le K.\hfill \end{array}$$ (5.34)

Note that the K discs bounded by C_k, 1 ≤ k ≤ K are disjoint and the intersections between these discs and the real line in the complex plane are just the first K intervals in (5.32). Let E^k(α) be the orthogonal projection onto the space spanned by the ${\eta }_k^{\left[\alpha \right]}$, 1 ≤ k ≤ K, 0 ≤ α ≤ α₀. Now because it follows from (5.33) that for any 0 ≤ α ≤ a₀, 1 ≤ k ≤ K, the disk bounded by the circle C_k only contains the k-th eigenvalues ${\lambda }_k^{\left[\alpha \right]}$ of S_αΓS_α for any 0 ≤ α ≤ a₀, 1 ≤ k ≤ K, we have

$$E^k\left(\alpha \right)=\frac{1}{2\pi i}{\int }_{C_k}{(\mathit{zI}-S_{\alpha }\mathrm{\Gamma }{S}_{\alpha })}^{-1}\mathit{dz}.$$ (5.35)

By Lemma 11, S_aΓS_α is a continuous function of α. Hence, by a similar calculation as in (5.26), it can be shown that E^k(α) is a continuous function of α.

Recall that we define in (4.6)

$${\mathrm{\Omega }}_0=\{\omega :\text{there exists at least one}\alpha \in [0,{\alpha }_0]\text{such that}{\widehat{\lambda }}_1^{\left[\alpha \right]},\cdots ,{\widehat{\lambda }}_K^{\left[\alpha \right]}\text{are not mutually different}\}.$$

Lemma 14

Ω₀ is a measurable set and P(Ω₀) → 0 as n → ∞.

Proof. Consider the subset

$$\epsilon =\{B:B\text{is a positive compact operator, its first}K\text{eigenvalues are mutually different and each of them has multiplicity}1\}.$$

ε is an open subset of the space of all positive compact operators which is closed in H, hence it is measurable. Let (Ω, ℱ) be the probability space and ([0, α₀], ℬ[0, α₀]) be the Lebesgue space. Since S_αΓ̂S_α has continuous sample paths, it is jointly measurable in (Ω × [0, α₀], ℱ × ℬ[0, α₀]). One can see that ${\Omega }_0^c$ is the projection of the set {(ω, α) : S_αΓ̂_nS_α ∈ ε} to Ω. Therefore, ${\Omega }_0^c$ is measurable, so is Ω₀. By (5.33) and the definition of ε_K (just above (5.32)), we have

$${\mathrm{\Omega }}_0\subset \{\underset{0\le \alpha \le {\alpha }_0}{\sup }\underset{1\le k\le K+1}{\max }|{\widehat{\lambda }}_k^{\left[\alpha \right]}-{\lambda }_k^{\left[\alpha \right]}|>\frac{{∊}_K}{4}\}.$$

By Corollary 4 in Section XI.9 of Dunford and Schwartz [5],

$$\underset{0\le \alpha \le {\alpha }_0}{\sup }\underset{1\le k\le K+1}{\max }|{\widehat{\lambda }}_k^{\left[\alpha \right]}-{\lambda }_k^{\left[\alpha \right]}|\le \underset{0\le \alpha \le {\alpha }_0}{\sup }\parallel S_{\alpha }{\widehat{\mathrm{\Gamma }}}_n{S}_{\alpha }-S_{\alpha }\mathrm{\Gamma }{S}_{\alpha }\parallel \le \parallel {\widehat{\mathrm{\Gamma }}}_n-\mathrm{\Gamma }\parallel .$$ (5.36)

Hence,

$$P\left({\mathrm{\Omega }}_0\right)\le P(\parallel {\widehat{\mathrm{\Gamma }}}_n-\mathrm{\Gamma }\parallel >\frac{{∊}_K}{4})\to 0$$ (5.37)

by the law of large numbers.

For any ω ∈ Ω₀, define ${\widehat{E}}_n^k\left(\alpha \right)$ to be zero. For any ω ∉ Ω₀, define ${\widehat{E}}_n^k\left(\alpha \right)$ to be the orthogonal projection onto the space spanned by the k-th eigenfunction ${\widehat{\eta }}_n^{\left[\alpha \right]}$ of S_αΓ̂_nS_α (note that ${\widehat{E}}_n^k\left(\alpha \right)$ does not depend on the sign of ${\widehat{\eta }}_k^{\left[\alpha \right]}$). By the same argument as in the proof of Lemma 12, we can show that ${\widehat{E}}_n^k\left(\alpha \right)$ is a continuous function of S_αΓ̂_nS_α, so it is measurable and continuous in α. Now let {e_m, m ∈ ℕ} be a set of complete orthonormal basis functions in L²([a, b]), we choose

$$\begin{array}{cc}\hfill {\widehat{\eta }}_k^{\left[0\right]}& =\frac{{\widehat{E}}_n^k\left(0\right){\gamma }_k}{\parallel {\widehat{E}}_n^k\left(0\right){\gamma }_k\parallel }{\mathrm{\chi }}_{\{{\widehat{E}}_n^k\left(0\right){\gamma }_k\ne 0\}}\hfill \\ \hfill & +\underset{m=1}{\overset{\infty }{\Sigma }}\frac{{\widehat{E}}_n^k\left(0\right)e_m}{\parallel {\widehat{E}}_n^k\left(0\right)e_m\parallel }{\mathrm{\chi }}_{\{{\widehat{E}}_n^k\left(0\right){\gamma }_k=0,{\widehat{E}}_n^k\left(0\right)e_j=0,1\le j\le m-1,{\widehat{E}}_n^k\left(0\right)e_m\ne 0\}}\hfill \end{array}$$ (5.38)

in ${\Omega }_0^c$ and 0 in Ω₀, where χ is the indicator function. Then ${\widehat{\eta }}_k^{\left[0\right]}$ is measurable and

$$({\widehat{\eta }}_k^{\left[0\right]},{\eta }_k^{\left[0\right]})\ge 0.$$ (5.39)

Now by Lemma 11, Lemma 12 and the definition of Ω₀, for any ω ∉ Ω₀, we can uniquely choose ${\widehat{\eta }}_k^{\left[\alpha \right]}$, 1 ≤ k ≤ K, such that ${\widehat{\eta }}_k^{\left[\alpha \right]}$, 1 ≤ k ≤ K are continuous functions of α. ${\widehat{\eta }}_k^{\left[\alpha \right]}$ is measurable by the following lemma. By (5.31), ${\widehat{\gamma }}_k^{\left[\alpha \right]}$, 1 ≤ k ≤ K are continuous and measurable with $({\widehat{\gamma }}_k^{\left[0\right]},{\gamma }_k)\ge 0$.

Lemma 15

If for any 1 ≤ k ≤ K, ${\widehat{\eta }}_k^{\left[\alpha \right]}$ is a measurable map to C_L²([a,b][0, α₀].

Proof. In ${\Omega }_0^c$, ${\widehat{E}}_n^k\left(\alpha \right){\widehat{\eta }}_k^{\left[0\right]}$ is a continuous function of α. Since $\parallel {\widehat{E}}_n^k\left(0\right){\widehat{\eta }}_k^{\left[0\right]}\parallel =1$, let ${\widehat{T}}^{\left(1\right)}=\inf \{\alpha ,\parallel {\widehat{E}}_n^k\left(\alpha \right){\widehat{\eta }}_k^{\left[0\right]}\parallel \le \frac{1}{2}\}\mathrm{\Lambda }{\alpha }_0$ in ${\Omega }_0^c$. In Ω₀, define T̂⁽¹⁾ = 0. Then T̂⁽¹⁾ is a nonnegative random variable. By Lemma 12, we have in ${\Omega }_0^c$, if α ≤ T̂⁽¹⁾,

$${\widehat{\eta }}_k^{\left[\alpha \right]}=\frac{{\widehat{E}}_n^k\left(\alpha \right){\widehat{\eta }}_k^{\left[0\right]}}{\parallel {\widehat{E}}_n^k\left(\alpha \right){\widehat{\eta }}_k^{\left[0\right]}\parallel }.$$

Define a random element

$${\zeta }_1=\frac{{\widehat{E}}_n^k\left({\widehat{T}}^{\left(1\right)}\right){\widehat{\eta }}_k^{\left[0\right]}}{\parallel {\widehat{E}}_n^k\left({\widehat{T}}^{\left(1\right)}\right){\widehat{\eta }}_k^{\left[0\right]}\parallel }$$

in ${\Omega }_0^c$ and 0 in Ω₀. Define a random variable ${\widehat{T}}^{\left(2\right)}=\inf \{\alpha \ge {\widehat{T}}^{\left(1\right)},\parallel {\widehat{E}}_n^k\left(\alpha \right){\zeta }_1\parallel \le \frac{1}{2}\}\Lambda {\alpha }_0$ and a random element

$${\zeta }_2=\frac{{\widehat{E}}_n^k\left({\widehat{T}}^{\left(2\right)}\right){\zeta }_1}{\parallel {\widehat{E}}_n^k\left({\widehat{T}}^{\left(1\right)}\right){\zeta }_1\parallel }$$

in ${\Omega }_0^c$ and 0 in Ω₀. Similarly, we can define (T̂⁽³⁾, ζ₃), …. One can show that for any ω ∈ Ω^c, there are only finite T̂^(m)(ω) < α₀, m =0, 1, 2, …, where T̂⁽⁰⁾(ω). Hence in Ω^c, we have

$${\widehat{\eta }}_k^{\left[\alpha \right]}=\underset{m=0}{\overset{\mathrm{\infty }}{\Sigma }}\frac{{\widehat{E}}_n^k\left(\alpha \right){\zeta }_m}{\parallel {\widehat{E}}_n^k\left(\alpha \right){\zeta }_m\parallel }{\mathrm{\chi }}_{[{\widehat{T}}^{\left(m\right)},{\widehat{T}}^{(m+1)})},$$

where ${\zeta }_0={\widehat{\eta }}_k^{\left[0\right]}$ and χ is the indicator function. Hence, ${\widehat{\eta }}_k^{\left[\alpha \right]}$ is measurable.

By (5.33) and (5.36), in the event $\{\parallel {\widehat{\mathrm{\Gamma }}}_n-\mathrm{\Gamma }\parallel \le \frac{{∊}_K}{4}\}\subset {\Omega }_0^c$, for any 0 ≤ α ≤ α₀, 1 ≤ k ≤ K, the disk bounded by the circle C_k only contains the k-th eigenvalues for S_αΓ̂S_α and S_αΓS_α. Hence, in the event $\{\parallel {\widehat{\mathrm{\Gamma }}}_n-\mathrm{\Gamma }\parallel \le \frac{{∊}_K}{4}\}$, for any 0 ≤ α ≤ α₀, 1 ≤ k ≤ K, we have

$$E^k\left(\alpha \right)=\frac{1}{2\pi i}{\int }_{C_k}{(\mathit{zI}-S_{\alpha }\mathrm{\Gamma }{S}_{\alpha })}^{-1}\mathit{dz},{\widehat{E}}_n^k\left(\alpha \right)=\frac{1}{2\pi i}{\int }_{C_k}{(\mathit{zI}-S_{\alpha }{\widehat{\mathrm{\Gamma }}}_n{S}_{\alpha })}^{-1}\mathit{dz}.$$ (5.40)

The proofs of the following Lemma 16 and Lemma 17 follows the ideas of Section 2 in Dauxois et al. [3]. Define linear maps ϕ_k : C_H[0, α₀] → C_H[0, α₀], 1 ≤ k ≤ K such that for any Λ ∈ C_H[0, α₀ and 0 ≤ α ≤ α₀,

$$\left({\varphi }_k\left(\mathrm{\Lambda }\right)\right)\left(\alpha \right)=\frac{1}{2\pi i}{\int }_{C_k}\left[{(\mathit{zI}-S_{\alpha }\mathrm{\Gamma }{S}_{\alpha })}^{-1}\mathrm{\Lambda }\left(\alpha \right){(\mathit{zI}-S_{\alpha }\mathrm{\Gamma }{S}_{\alpha })}^{-1}\right]\mathit{dz},$$ (5.41)

where (ϕ_k(Λ))(α) denotes the value of ϕ_k(Λ) at the point α. Then define ΦK = (ϕ₁, ϕ₂, …, ϕ_K) which is a linear map from C_H[0, α₀] to ${\prod }_{k=1}^K{C}_H[0,{\alpha }_0]$. One can verify that ϕ_k's are continuous. Hence Φ_K is a bounded operator.

Lemma 16

The sequence ${\{\sqrt{n}({\widehat{E}}_n^k-E^k),1\le k\le K\}}_n$ of stochastic processes has sample paths in ${\prod }_{k=1}^K{C}_H[0,{\alpha }_0]$ a.s. and converges in distribution to a Gaussian random, element with mean zero and covariance operator ${\mathrm{\Phi }}_K\mathrm{\Theta }{iQi}^{\ast }{\mathrm{\Theta }}^{\ast }{\mathrm{\Phi }}_K^{\ast }$.

Proof. In the event $\{\parallel {\widehat{\mathrm{\Gamma }}}_n-\mathrm{\Gamma }\parallel \le \frac{{∊}_K}{4}\}$, for each z ∈ C_K,

$$\begin{array}{cc}\hfill & {(\mathit{zI}-S_{\alpha }{\widehat{\mathrm{\Gamma }}}_n{S}_{\alpha })}^{-1}={((\mathit{zI}-S_{\alpha }\mathrm{\Gamma }{S}_{\alpha })-(S_{\alpha }\widehat{\mathrm{\Gamma }}{S}_{\alpha }-S_{\alpha }{\mathrm{\Gamma }}_n{S}_{\alpha }))}^{-1}\hfill \\ \hfill & ={(\mathit{zI}-S_{\alpha }\mathrm{\Gamma }{S}_{\alpha })}^{-1}={\left(I-(S_{\alpha }\widehat{\mathrm{\Gamma }}{S}_{\alpha }-S_{\alpha }{\mathrm{\Gamma }}_n{S}_{\alpha }){(\mathit{zI}-S_{\alpha }\mathrm{\Gamma }{S}_{\alpha })}^{-1}\right)}^{-1}.\hfill \end{array}$$ (5.42)

If

$$\underset{0\le \alpha \le \alpha 0}{\sup }\parallel S_{\alpha }\widehat{\mathrm{\Gamma }}{S}_{\alpha }-S_{\alpha }{\mathrm{\Gamma }}_n{S}_{\alpha }\parallel =\parallel {\widehat{\mathrm{\Gamma }}}_n-\mathrm{\Gamma }\parallel \le \frac{1}{2}\widetilde{∊},$$

where

$$\widetilde{∊}{}^{-1}=\underset{1\le k\le K}{\max }\underset{0\le \alpha \le {\alpha }_0}{\sup }\underset{z\in C_k}{\sup }\parallel {(\mathit{zI}-S_{\alpha }\mathrm{\Gamma }{S}_{\alpha })}^{-1}\parallel <\mathrm{\infty },$$

then by (5.42), we have an absolutely convergent series expansion

$$\begin{array}{cc}\hfill & {(\mathit{zI}-S_{\alpha }{\widehat{\mathrm{\Gamma }}}_n{S}_{\alpha })}^{-1}\hfill \\ \hfill =& {(\mathit{zI}-S_{\alpha }\mathrm{\Gamma }{S}_{\alpha })}^{-1}\underset{m=0}{\overset{\mathrm{\infty }}{\Sigma }}{\left((S_{\alpha }{\mathrm{\Gamma }}_n{S}_{\alpha }-S_{\alpha }{\widehat{\mathrm{\Gamma }}}_n{S}_{\alpha }){(\mathit{zI}-S_{\alpha }\mathrm{\Gamma }{S}_{\alpha })}^{-1}\right)}^m.\hfill \end{array}$$

Hence,

$$\begin{array}{cc}\hfill & {(\mathit{zI}-S_{\alpha }{\widehat{\mathrm{\Gamma }}}_n{S}_{\alpha })}^{-1}-{(\mathit{zI}-S_{\alpha }\mathrm{\Gamma }{S}_{\alpha })}^{-1}\hfill \\ \hfill =& {(\mathit{zI}-S_{\alpha }\mathrm{\Gamma }{S}_{\alpha })}^{-1}(S_{\alpha }\mathrm{\Gamma }{S}_{\alpha }-S_{\alpha }{\widehat{\mathrm{\Gamma }}}_n{S}_{\alpha }){(\mathit{zI}-S_{\alpha }\mathrm{\Gamma }{S}_{\alpha })}^{-1}+{\widehat{U}}_{\alpha }^n\left(z\right)\hfill \end{array}$$ (5.43)

where

$${\widehat{U}}_{\alpha }^n\left(z\right)={(\mathit{zI}-S_{\alpha }\mathrm{\Gamma }{S}_{\alpha })}^{-1}\underset{m=2}{\overset{\mathrm{\infty }}{\Sigma }}{\left((S_{\alpha }\mathrm{\Gamma }{S}_{\alpha }-S_{\alpha }{\widehat{\mathrm{\Gamma }}}_n{S}_{\alpha }){(\mathit{zI}-S_{\alpha }\mathrm{\Gamma }{S}_{\alpha })}^{-1}\right)}^m.$$

Hence, in the event $\{\parallel {\widehat{\mathrm{\Gamma }}}_n-\mathrm{\Gamma }\parallel \le \frac{1}{2}\widetilde{∊}\}$,

$$\parallel {\widehat{U}}_{\alpha }^n\left(z\right)\parallel \le \frac{2}{{\widetilde{∊}}^3}{\parallel {\widehat{\mathrm{\Gamma }}}_n-\mathrm{\Gamma }\parallel }^2.$$ (5.44)

Now in the event $\{\parallel {\widehat{\mathrm{\Gamma }}}_n-\mathrm{\Gamma }\parallel \le \min (\frac{1}{2}\widetilde{∊},\frac{{∊}_K}{4})\}$, by (5.42) and (5.43),

$$\begin{array}{cc}\hfill & \sqrt{n}({\widehat{E}}_n^k\left(\alpha \right)-E_n^k\left(\alpha \right))=\frac{\sqrt{n}}{2\pi i}{\int }_{C_k}\left[{(\mathit{zI}-S_{\alpha }\widehat{\mathrm{\Gamma }}{S}_{\alpha })}^{-1}\mathit{dz}-{(\mathit{zI}-S_{\alpha }{\mathrm{\Gamma }}_n{S}_{\alpha })}^{-1}\right]\mathit{dz}\hfill \\ \hfill =& {\varphi }_k\left(Z_n\right)+\frac{1}{2\pi i}{\int }_{C_k}\sqrt{n}{\widehat{U}}_{\alpha }^n\left(z\right)\mathit{dz}\hfill \end{array}$$ (5.45)

Now we have from (5.44) and (5.45), for any δ > 0,

$$\begin{array}{cc}\hfill & P(\parallel \sqrt{n}({\widehat{E}}_n^k-E_n^k)-{\varphi }_k\left(Z_n\right)\parallel >\delta )\le P(\parallel {\widehat{\mathrm{\Gamma }}}_n-\mathrm{\Gamma }\parallel >\min (\frac{1}{2}\widetilde{∊},\frac{{∊}_K}{4}))\hfill \\ \hfill & +P(\parallel \sqrt{n}({\widehat{E}}_n^k-E_n^k)-{\varphi }_k\left(Z_n\right)\parallel >\delta ,\parallel {\widehat{\mathrm{\Gamma }}}_n-\mathrm{\Gamma }\parallel \le \min (\frac{1}{2}\widetilde{∊},\frac{{∊}_K}{4}))\hfill \\ \hfill \le & P(\parallel {\widehat{\mathrm{\Gamma }}}_n-\mathrm{\Gamma }\parallel >\min (\frac{1}{2}\widetilde{∊},\frac{{∊}_K}{4}))+P(\parallel \frac{\sqrt{n}}{2\pi i}{\int }_{C_k}{\widehat{U}}_n\left(z\right)\mathit{dz}\parallel >\delta )\hfill \\ \hfill \le & P(\parallel {\widehat{\mathrm{\Gamma }}}_n-\mathrm{\Gamma }\parallel >\min (\frac{1}{2}\widetilde{∊},\frac{{∊}_K}{4}))+P(\sqrt{n}{\parallel {\widehat{\mathrm{\Gamma }}}_n-\mathrm{\Gamma }\parallel }^2>\pi \delta {\widetilde{∊}}^3)\to 0,\hfill \end{array}$$ (5.46)

as n → 0. By Lemmas 8 and 13, Φ_K(Z_n) = (ϕ₁(Z_n), ϕ₂(Z_n), … ϕ_K(Z_n)) converges in distribution to the Gaussian element with mean zero and covariance operator ${\mathrm{\Phi }}_K\mathrm{\Theta }{iQi}^{\ast }{\mathrm{\Theta }}^{\ast }{\mathrm{\Phi }}_K^{\ast }$. Now by (5.46), ${\{\sqrt{n}({\widehat{E}}_n^k-E_n^k),1\le k\le K\}}_n$ converges in distribution to the same distribution.

Define linear maps Ψ_k : C_H[0, α₀] → C_L²([a,b) [0, α₀], 1 ≤ k ≤ K such that for any Λ ∈ C_H[0, α₀],

$${\psi }_k\left(\mathrm{\Lambda }\right)=\{(I-E^k\left(\alpha \right))\mathrm{\Lambda }\left(\alpha \right){\eta }_k^{\left[\alpha \right]},0\le \alpha \le {\alpha }_0\}.$$ (5.47)

Then we define a linear map ${\mathrm{\Psi }}_K:{\prod }_{k=1}^K{C}_H[0,{\alpha }_0]\to {\prod }_{k=1}^K{C}_{L^2\left([a,b]\right)}[0,{\alpha }_0]$ such that for any $({\Lambda }_1,\cdots ,{\Lambda }_K)\in {\prod }_{k=1}^K{C}_H[0,{\alpha }_0]$,

$${\Psi }_K({\mathrm{\Lambda }}_1,\cdots ,{\mathrm{\Lambda }}_K)=({\psi }_1\left({\mathrm{\Lambda }}_1\right),\cdots ,{\psi }_K\left({\mathrm{\Lambda }}_K\right)).$$ (5.48)

It is easy to see that ψ_K is a bounded operator.

Lemma 17

The sequence ${\{\sqrt{n}({\widehat{\eta }}_k^{\left[\alpha \right]}-{\eta }_k^{\left[\alpha \right]}),1\le k\le K\}}_n$ of stochastic processes has sample paths in ${\prod }_{k=1}^K{C}_{L^2\left([a,b]\right)}[0,{\alpha }_0]$ a.s. and converges in distribution to a Gaussian random element with mean zero and covariance operator ${\mathrm{\Psi }}_K{\mathrm{\Phi }}_K\mathrm{\Theta }{iQi}^{\ast }{\mathrm{\Theta }}^{\ast }{\mathrm{\Phi }}_K^{\ast }{\mathrm{\Psi }}_K^{\ast }$.

Proof. By the definitions (5.29) of ${\widehat{\eta }}_k^{\left[\alpha \right]},{\eta }_k^{\left[\alpha \right]},1\le k\le K,{({\eta }_k^{\left[\alpha \right]},{\eta }_k^{\left[\alpha \right]})}^2={\parallel {\eta }_k^{\left[\alpha \right]}\parallel }^2=1$. In ${\Omega }_0^c$, we have

$$\begin{array}{cc}\hfill & \underset{0\le \alpha \le {\alpha }_0}{\sup }\sqrt{n}|{({\widehat{\eta }}_k^{\left[\alpha \right]},{\eta }_k^{\left[\alpha \right]})}^2-1|=\underset{0\le \alpha \le {\alpha }_0}{\sup }\sqrt{n}|{({\widehat{\eta }}_k^{\left[\alpha \right]},{\eta }_k^{\left[\alpha \right]})}^2-{({\eta }_k^{\left[\alpha \right]},{\eta }_k^{\left[\alpha \right]})}^2|\hfill \\ \hfill & =\underset{0\le \alpha \le {\alpha }_0}{\sup }\sqrt{n}\mid \left(({\widehat{\eta }}_k^{\left[\alpha \right]},{\eta }_k^{\left[\alpha \right]}){\widehat{\eta }}_k^{\left[\alpha \right]},{\eta }_k^{\left[\alpha \right]}\right)-\left(({\eta }_k^{\left[\alpha \right]},{\eta }_k^{\left[\alpha \right]}){\eta }_k^{\left[\alpha \right]},{\eta }_k^{\left[\alpha \right]}\right)\mid \hfill \\ \hfill & =\underset{0\le \alpha \le {\alpha }_0}{\sup }\sqrt{n}\left|({\eta }_k^{\left[\alpha \right]},({\widehat{E}}_n^k\left(\alpha \right)-E^k\left(\alpha \right)){\eta }_k^{\left[\alpha \right]})\right|\hfill \end{array}$$

By (5.46), $\sqrt{n}({\widehat{E}}_n^k\left(\alpha \right)-E^k\left(\alpha \right))$ and ϕ_k(Z_n) have the same limit distribution. Because for any Λ ∈ C_H[0, α₀],

$$\begin{array}{cc}\hfill & ({\eta }_k^{\left[\alpha \right]},{\varphi }_k\left(\mathrm{\Lambda }\right){\eta }_k^{\left[\alpha \right]})\hfill \\ \hfill =& ({\eta }_k^{\left[\alpha \right]},{\int }_{C_k}\left[{(\mathit{zI}-S_{\alpha }\mathrm{\Gamma }{S}_{\alpha })}^{-1}\mathrm{\Lambda }\left(\alpha \right){(\mathit{zI}-S_{\alpha }\mathrm{\Gamma }{S}_{\alpha })}^{-1}\right]dz{\eta }_k^{\left[\alpha \right]})\hfill \\ \hfill =& {\int }_{C_k}({\eta }_k^{\left[\alpha \right]},{(\mathit{zI}-S_{\alpha }\mathrm{\Gamma }{S}_{\alpha })}^{-1}\mathrm{\Lambda }\left(\alpha \right){(\mathit{zI}-S_{\alpha }\mathrm{\Gamma }{S}_{\alpha })}^{-1}{\eta }_k^{\left[\alpha \right]})\mathit{dz}\hfill \\ \hfill =& {\int }_{C_k}{(z-{\lambda }_k^{\left[\alpha \right]})}^{-2}\mathit{dz}({\eta }_k^{\left[\alpha \right]},\mathrm{\Lambda }\left(\alpha \right){\eta }_k^{\left[\alpha \right]})=0\hfill \end{array}$$ (5.49)

where we use the facts that

$${(\mathit{zI}-S_{\alpha }\mathrm{\Gamma }{S}_{\alpha })}^{-1}{\eta }_k^{\left[\alpha \right]}={(z-{\lambda }_k^{\left[\alpha \right]})}^{-1}{\eta }_k^{\left[\alpha \right]},{\int }_{C_k}{(z-{\lambda }_k^{\left[\alpha \right]})}^{-2}\mathit{dz}=0,$$

So we have

$$\underset{0\le \alpha \le {\alpha }_0}{\sup }\sqrt{n}|{({\widehat{\eta }}_k^{\left[\alpha \right]},{\eta }_k^{\left[\alpha \right]})}^2-1|\to 0$$

in probability. By (5.39) and the continuities of ${\widehat{\eta }}_k^{\left[a\right]}$ and ${\eta }_k^{\left[a\right]}$, we have

$$\underset{0\le \alpha \le {\alpha }_0}{\sup }\sqrt{n}|({\widehat{\eta }}_k^{\left[\alpha \right]},{\eta }_k^{\left[\alpha \right]})-1|\to 0,$$ (5.50)

in probability. Now

$$\begin{array}{cc}\hfill \sqrt{n}({\widehat{\eta }}_k^{\left[\alpha \right]}-{\eta }_k^{\left[\alpha \right]})=\sqrt{n}{E}^k\left(\alpha \right)({\widehat{\eta }}_k^{\left[\alpha \right]}-{\eta }_k^{\left[\alpha \right]})+\sqrt{n}(I-E^k\left(\alpha \right))({\widehat{\eta }}_k^{\left[\alpha \right]}-{\eta }_k^{\left[\alpha \right]})& \hfill \\ \hfill =\sqrt{n}(({\widehat{\eta }}_k^{\left[\alpha \right]},{\eta }_k^{\left[\alpha \right]})-1){\eta }_k^{\left[\alpha \right]}+\sqrt{n}(I-E^k\left(\alpha \right)){\widehat{\eta }}_k^{\left[\alpha \right]}& \hfill \\ \hfill =\sqrt{n}(({\widehat{\eta }}_k^{\left[\alpha \right]},{\eta }_k^{\left[\alpha \right]})-1){\eta }_k^{\left[\alpha \right]}+\sqrt{n}(I-E^k\left(\alpha \right))\frac{{\widehat{E}}_n^k\left(\alpha \right){\eta }_k^{\left[\alpha \right]}}{({\widehat{\eta }}_k^{\left[\alpha \right]},{\eta }_k^{\left[\alpha \right]})}& \hfill \\ \hfill =\sqrt{n}(({\widehat{\eta }}_k^{\left[\alpha \right]},{\eta }_k^{\left[\alpha \right]})-1){\eta }_k^{\left[\alpha \right]}+\frac{1}{({\widehat{\eta }}_k^{\left[\alpha \right]},{\eta }_k^{\left[\alpha \right]})}(I-E^k\left(\alpha \right))\sqrt{n}({\widehat{E}}_n^k\left(\alpha \right)-E^k\left(\alpha \right)){\eta }_k^{\left[\alpha \right]}& \hfill \\ \hfill =\sqrt{n}(({\widehat{\eta }}_k^{\left[\alpha \right]},{\eta }_k^{\left[\alpha \right]})-1){\eta }_k^{\left[\alpha \right]}+\frac{1}{({\widehat{\eta }}_k^{\left[\alpha \right]},{\eta }_k^{\left[\alpha \right]})}{\psi }_k\left(\sqrt{n}({\widehat{E}}_n^k\left(\alpha \right)-E^k\left(\alpha \right))\right).& \hfill \end{array}$$ (5.51)

By (5.50), the first term in the last line converges to 0 in probability and $({\widehat{\eta }}_k^{\left[a\right]},{\eta }_k^{\left[a\right]})\to 1$ in probability. Hence, $\left(\sqrt{n}({\widehat{\eta }}_1^{\left[\alpha \right]}-{\eta }_1^{\left[\alpha \right]}),\cdots ,\sqrt{n}({\widehat{\eta }}_K^{\left[\alpha \right]}-{\eta }_K^{\left[\alpha \right]})\right)$ has the same limit distribution as ${\mathrm{\Psi }}_K\left(\sqrt{n}({\widehat{E}}_n^k\left(\alpha \right)-E^k\left(\alpha \right))\right)$ which converges to a Gaussian random element with mean zero and covariance operator ${\mathrm{\Psi }}_K{\mathrm{\Phi }}_K\mathrm{\Theta }{iQi}^{\ast }{\mathrm{\Theta }}^{\ast }{\mathrm{\Phi }}_K^{\ast }{\mathrm{\Psi }}_K^{\ast }$ by Lemmas 8 and 16.

Define linear maps ϕ_k : C_H[0, α₀ → C_H[0, α₀], 1 ≤ k ≤ K, such that for any Λ ∈ C_H[0, α₀],

$${\mathfrak{U}}_k\left(\mathrm{\Lambda }\right)=\left\{\left({\eta }_k^{\left[\alpha \right]},E^k\left(\alpha \right)\left({\psi }_k\left(\mathrm{\Lambda }\right)\right)\left(\alpha \right)\right)+\left({\eta }_k^{\left[\alpha \right]},\mathrm{\Lambda }\left(\alpha \right){\eta }_k^{\left[\alpha \right]}\right)+\left(\left({\psi }_k\left(\mathrm{\Lambda }\right)\right)\left(\alpha \right),E^k\left(\alpha \right){\eta }_k^{\left[\alpha \right]}\right),0\le \alpha \le {\alpha }_0\right\},$$

where Ψ_k is defined in (5.47) and (Ψ_k(Λ))(α) denotes the value of Ψ_k(Λ) at α. Define a liner map ${\mathfrak{U}}_k:C_H[0,{\alpha }_0]\to C_{ℝ}[0,{\alpha }_0],1\le k\le K$ such that for any (Λ, …, Λ_K),

$${\mathrm{\mho }}_K({\mathrm{\Lambda }}_1,\cdots ,{\mathrm{\Lambda }}_K)=({\mathfrak{U}}_1\left({\mathrm{\Lambda }}_1\right),\cdots ,{\mathfrak{U}}_K\left({\mathrm{\Lambda }}_K\right)).$$ (5.52)

It is easy to see that ℧_K is a bounded operator.

Lemma 18

The sequence ${\{\sqrt{n}({\widehat{\lambda }}_k^{\left[\alpha \right]}-{\lambda }_k^{\left[\alpha \right]}),1\le k\le K\}}_n$ of stochastic processes has sample paths in ${\prod }_{k=1}^K{C}_{ℝ}[0,{\alpha }_0]$ and converges in distribution to a Gaussian random, element with zero and covariance operator ${\mathrm{\mho }}_K{\mathrm{\Phi }}_K\mathrm{\Theta }{iQi}^{\ast }{\mathrm{\Theta }}^{\ast }{\mathrm{\Phi }}_K^{\ast }{\mathrm{\mho }}_K^{\ast }$.

Proof. The continuities of ${\widehat{\lambda }}_k^{\left[\alpha \right]}$ and ${\lambda }_k^{\left[\alpha \right]}$ follow from Lemma 11 and the inequalities

$$|{\widehat{\lambda }}_k^{\left[\alpha \right]}-{\widehat{\lambda }}_k^{\left[{\alpha }^{\prime }\right]}|\le \parallel S_{\alpha }\widehat{\mathrm{\Gamma }}{S}_{\alpha }-S_{{\alpha }^{\prime }}\widehat{\mathrm{\Gamma }}{S}_{{\alpha }^{\prime }}\parallel ,|{\lambda }_k^{\left[\alpha \right]}-{\lambda }_k^{\left[{\alpha }^{\prime }\right]}|\le \parallel S_{\alpha }\mathrm{\Gamma }{S}_{\alpha }-S_{{\alpha }^{\prime }}\mathrm{\Gamma }{S}_{{\alpha }^{\prime }}\parallel ,$$

for any 0 ≤ α, α′ ≤ α₀. In ${\Omega }_0^c$,

$$\begin{array}{cc}\hfill & \sqrt{n}({\widehat{\lambda }}_k^{\left[\alpha \right]}-{\lambda }_k^{\left[\alpha \right]})=\sqrt{n}\left(({\widehat{\eta }}_k^{\left[\alpha \right]},{\widehat{E}}_n^k\left(\alpha \right){\widehat{\eta }}_k^{\left[\alpha \right]})-({\eta }_k^{\left[\alpha \right]},E^k\left(\alpha \right){\eta }_k^{\left[\alpha \right]})\right)\hfill \\ \hfill =& \sqrt{n}\left(({\widehat{\eta }}_k^{\left[\alpha \right]},{\widehat{E}}_n^k\left(\alpha \right){\widehat{\eta }}_k^{\left[\alpha \right]})-({\widehat{\eta }}_k^{\left[\alpha \right]},{\widehat{E}}_n^k\left(\alpha \right){\eta }_k^{\left[\alpha \right]})\right)\hfill \\ \hfill & +\sqrt{n}\left(({\widehat{\eta }}_k^{\left[\alpha \right]},{\widehat{E}}_n^k\left(\alpha \right){\eta }_k^{\left[\alpha \right]})-({\widehat{\eta }}_k^{\left[\alpha \right]},E_n^k\left(\alpha \right){\eta }_k^{\left[\alpha \right]})\right)\hfill \\ \hfill & +\sqrt{n}\left(({\widehat{\eta }}_k^{\left[\alpha \right]},E_n^k\left(\alpha \right){\eta }_k^{\left[\alpha \right]})-({\eta }_k^{\left[\alpha \right]},E_n^k\left(\alpha \right){\eta }_k^{\left[\alpha \right]})\right)\hfill \\ \hfill =& \left({\widehat{\eta }}_k^{\left[\alpha \right]},{\widehat{E}}_n^k\left(\alpha \right)\sqrt{n}({\widehat{\eta }}_k^{\left[\alpha \right]}-{\eta }_k^{\left[\alpha \right]})\right)+\left({\widehat{\eta }}_k^{\left[\alpha \right]},\sqrt{n}({\widehat{E}}_n^k\left(\alpha \right)-E_n^k\left(\alpha \right)){\eta }_k^{\left[\alpha \right]}\right)\hfill \\ \hfill & +\left(\sqrt{n}({\widehat{\eta }}_k^{\left[\alpha \right]}-{\eta }_k^{\left[\alpha \right]}),E_n^k\left(\alpha \right){\eta }_k^{\left[\alpha \right]}\right)\hfill \end{array}$$ (5.53)

By Lemmas 16 and 17, ${\widehat{\eta }}_k^{\left[\alpha \right]}\to {\eta }_k^{\left[\alpha \right]}$ and ${\widehat{E}}_k^{\left[\alpha \right]}\to E_k^{\left[\alpha \right]}$ in probability. Hence by (5.53), $\sqrt{n}({\widehat{\lambda }}_k^{\left[\alpha \right]}-{\lambda }_k^{\left[\alpha \right]})$ has the same limit distribution as

$$\left({\eta }_k^{\left[\alpha \right]},E_n^k\left(\alpha \right)\sqrt{n}({\widehat{\eta }}_k^{\left[\alpha \right]}-{\eta }_k^{\left[\alpha \right]})\right)+\left({\eta }_k^{\left[\alpha \right]},\sqrt{n}({\widehat{E}}_n^k\left(\alpha \right)-E_n^k\left(\alpha \right)){\eta }_k^{\left[\alpha \right]}\right)+\left(\sqrt{n}({\widehat{\eta }}_k^{\left[\alpha \right]}-{\eta }_k^{\left[\alpha \right]}),E_n^k\left(\alpha \right){\eta }_k^{\left[\alpha \right]}\right)$$

which, by (5.51), has the same distribution as

$$\begin{array}{cc}\hfill & \left({\eta }_k^{\left[\alpha \right]},E_n^k\left(\alpha \right){\psi }_k\left(\sqrt{n}({\widehat{E}}_n^k\left(\alpha \right)-E_n^k\left(\alpha \right))\right)\right)+\left({\eta }_k^{\left[\alpha \right]},\sqrt{n}({\widehat{E}}_n^k\left(\alpha \right)-E_n^k\left(\alpha \right)){\eta }_k^{\left[\alpha \right]}\right)+\left({\psi }_k\left(\sqrt{n}({\widehat{E}}_n^k\left(\alpha \right)-E_n^k\left(\alpha \right))\right),E_n^k\left(\alpha \right){\eta }_k^{\left[\alpha \right]}\right)\hfill \\ \hfill =& {\mathfrak{U}}_k\left(\sqrt{n}({\widehat{E}}_n^k\left(\alpha \right)-E_n^k\left(\alpha \right))\right).\hfill \end{array}$$

Hence, ${\{\sqrt{n}({\widehat{\lambda }}_k^{\left[\alpha \right]}-{\lambda }_k^{\left[\alpha \right]}),1\le k\le K\}}_n$ has the same limit distribution as ${\mathrm{\mho }}_K\left(\sqrt{n}({\widehat{E}}_n^1\left(\alpha \right)-E_n^1\left(\alpha \right)),\cdots ,\sqrt{n}({\widehat{E}}_n^K\left(\alpha \right)-E_n^K\left(\alpha \right))\right)$ which converges to a Gaussian random element with mean zero and covariance operator ${\mathrm{\mho }}_K{\mathrm{\Phi }}_K\mathrm{\Theta }{iQi}^{\ast }{\mathrm{\Theta }}^{\ast }{\mathrm{\Phi }}_K^{\ast }{\mathrm{\mho }}_K^{\ast }$ by Lemmas 8 and 16.

Define a linear map ${\Im }_K:{\prod }_{k=1}^K{C}_{L^2\left([a,b]\right)}[0,{\alpha }_0]\to {\prod }_{k=1}^K{C}_{L^2\left([a,b]\right)}[0,{\alpha }_0]$ such that for any $({\Lambda }_1,\cdots ,{\Lambda }_K)\in {\prod }_{k=1}^K{C}_{L^2}\left([a,b]\right)[0,{\alpha }_0]$,

$${\Im }_K({\mathrm{\Lambda }}_1,\cdots ,{\mathrm{\Lambda }}_K)=\left\{\left(\frac{1}{\parallel S_{\alpha }{\eta }_1^{\left[\alpha \right]}\parallel }{S}_{\alpha }{\mathrm{\Lambda }}_1\left(\alpha \right),\cdots ,\frac{1}{\parallel S_{\alpha }{\eta }_K^{\left[\alpha \right]}\parallel }{S}_{\alpha }{\mathrm{\Lambda }}_K\left(\alpha \right)\right),0\le \alpha \le {\alpha }_0\right\}.$$ (5.54)

ℑ_K is a bounded operator.

Lemma 19

The sequence ${\{\sqrt{n}({\widehat{\gamma }}_k^{\left[\alpha \right]}-{\gamma }_k^{\left[\alpha \right]}),1\le k\le K\}}_n$ of stochastic processes has sample paths in ${\prod }_{k=1}^K{C}_{L^2\left([a,b]\right)}[0,{\alpha }_0]$ a.s. and converges in distribution to a Gaussian random element with mean zero and covariance operator ${\Im }_K{\mathrm{\Psi }}_K{\mathrm{\Phi }}_K\mathrm{\Theta }{iQi}^{\ast }{\mathrm{\Theta }}^{\ast }{\mathrm{\Phi }}_K^{\ast }{\mathrm{\Psi }}_K^{\ast }{\Im }_K^{\ast }$.

Proof. By (5.31),

$${\widehat{\gamma }}_k^{\left[\alpha \right]}=\frac{S_{\alpha }{\widehat{\eta }}_k^{\left[\alpha \right]}}{\parallel S_{\alpha }{\widehat{\eta }}_k^{\left[\alpha \right]}\parallel },{\gamma }_k^{\left[\alpha \right]}=\frac{S_{\alpha }{\eta }_k^{\left[\alpha \right]}}{\parallel S_{\alpha }{\eta }_k^{\left[\alpha \right]}\parallel }.$$

Therefore,

$$\begin{array}{cc}\hfill \sqrt{n}({\widehat{\gamma }}_k^{\left[\alpha \right]}-{\gamma }_k^{\left[\alpha \right]})=\sqrt{n}\left(\frac{S_{\alpha }{\widehat{\eta }}_k^{\left[\alpha \right]}}{\parallel S_{\alpha }{\widehat{\eta }}_k^{\left[\alpha \right]}\parallel }-\frac{S_{\alpha }{\eta }_k^{\left[\alpha \right]}}{\parallel S_{\alpha }{\eta }_k^{\left[\alpha \right]}\parallel }\right)& \hfill \\ \hfill =\sqrt{n}\left(\frac{S_{\alpha }{\widehat{\eta }}_k^{\left[\alpha \right]}}{\parallel S_{\alpha }{\widehat{\eta }}_k^{\left[\alpha \right]}\parallel }-\frac{S_{\alpha }{\widehat{\eta }}_k^{\left[\alpha \right]}}{\parallel S_{\alpha }{\eta }_k^{\left[\alpha \right]}\parallel }\right)+\sqrt{n}\left(\frac{S_{\alpha }{\widehat{\eta }}_k^{\left[\alpha \right]}}{\parallel S_{\alpha }{\eta }_k^{\left[\alpha \right]}\parallel }-\frac{S_{\alpha }{\eta }_k^{\left[\alpha \right]}}{\parallel S_{\alpha }{\eta }_k^{\left[\alpha \right]}\parallel }\right)& \hfill \\ \hfill =\sqrt{n}\left(\frac{1}{\parallel S_{\alpha }{\widehat{\eta }}_k^{\left[\alpha \right]}\parallel }-\frac{1}{\parallel S_{\alpha }{\eta }_k^{\left[\alpha \right]}\parallel }\right)S_{\alpha }{\widehat{\eta }}_k^{\left[\alpha \right]}+\frac{1}{\parallel S_{\alpha }{\eta }_k^{\left[\alpha \right]}\parallel }{S}_{\alpha }\left(\sqrt{n}({\widehat{\eta }}_k^{\left[\alpha \right]}-{\eta }_k^{\left[\alpha \right]})\right)& \hfill \end{array}$$ (5.55)

Because

$$\parallel S_{\alpha }{\widehat{\eta }}_k^{\left[\alpha \right]}-S_{\alpha }{\eta }_k^{\left[\alpha \right]}\parallel \le \parallel {\widehat{\eta }}_k^{\left[\alpha \right]}-{\eta }_k^{\left[\alpha \right]}\parallel \to 0$$

in probability, by the definition (5.54) of ℑ_K, (5.55) and Lemma 17, ${\{\sqrt{n}({\widehat{\gamma }}_k^{\left[\alpha \right]}-{\gamma }_k^{\left[\alpha \right]}),1\le k\le K\}}_n$ has the same limit distribution as ${\Im }_K\left({\{\sqrt{n}({\widehat{\gamma }}_k^{\left[\alpha \right]}-{\gamma }_k^{\left[\alpha \right]}),1\le k\le K\}}_n\right)$ which converges to a Gaussian random element with mean zero and covariance operator ${\Im }_K{\mathrm{\Psi }}_K{\mathrm{\Phi }}_K\mathrm{\Theta }{iQi}^{\ast }{\mathrm{\Theta }}^{\ast }{\mathrm{\Phi }}_K^{\ast }{\mathrm{\Psi }}_K^{\ast }{\Im }_K^{\ast }$.

Proof of Corollary 4.1

By Lemma 18 and Lemma 19, the stochastic processes ${\{\sqrt{n}({\widehat{\lambda }}_k^{\left[\alpha \right]}-{\lambda }_k^{\left[\alpha \right]}),1\le k\le K\}}_n$ and ${\{\sqrt{n}({\widehat{\gamma }}_k^{\left[\alpha \right]}-{\gamma }_k^{\left[\alpha \right]}),1\le k\le K\}}_n$ convergence in distribution, hence they are tight by Theorem 5.2 in Billinsley [1] since C_H[0, α₀] and C_L²[a,b][0, α₀] are both complete and separable. Therefore, for any ∈ > 0, one can find a positive number M depending on ∈ such that

$$\begin{array}{cc}\hfill & \underset{n}{\sup }P(\underset{1\le k\le K}{\max }\underset{0\le \alpha \le {\alpha }_0}{\sup }\left|\sqrt{n}({\widehat{\lambda }}_k^{\left[\alpha \right]}-{\lambda }_k^{\left[\alpha \right]})\right|\ge M)\le ∊,\hfill \\ \hfill & \underset{n}{\sup }P(\underset{1\le k\le K}{\max }\underset{0\le \alpha \le {\alpha }_0}{\sup }\left|\sqrt{n}({\widehat{\gamma }}_k^{\left[\alpha \right]}-{\gamma }_k^{\left[\alpha \right]})\right|\ge M)\le ∊.\hfill \end{array}$$

In other words,

$$\begin{array}{cc}\hfill {\widehat{\lambda }}_k^{\left[\alpha \right]}-{\lambda }_k^{\left[\alpha \right]}=O_p\left(\frac{1}{\sqrt{n}}\right),& \hfill \\ \hfill {\widehat{\gamma }}_k^{\left[\alpha \right]}-{\gamma }_k^{\left[\alpha \right]}=O_p\left(\frac{1}{\sqrt{n}}\right)& \hfill \end{array}$$

uniformly in α, which combines Theorem 4.1 to get our corollary.

Proof of Corollary 4.2

First, we have decompositions

$$\begin{array}{cc}\hfill \sqrt{n}({\widehat{\lambda }}_k^{\left[{\alpha }_n\right]}-{\lambda }_k)=\sqrt{n}({\widehat{\lambda }}_k^{\left[{\alpha }_n\right]}-{\lambda }_k^{\left[{\alpha }_n\right]})+\sqrt{n}({\lambda }_k^{\left[{\alpha }_n\right]}-{\lambda }_k),& \hfill \\ \hfill \sqrt{n}({\widehat{\gamma }}_k^{\left[{\alpha }_n\right]}-{\gamma }_k)=\sqrt{n}({\widehat{\gamma }}_k^{\left[{\alpha }_n\right]}-{\gamma }_k^{\left[{\alpha }_n\right]})+\sqrt{n}({\gamma }_k^{\left[{\alpha }_n\right]}-{\gamma }_k).& \hfill \end{array}$$

Under the conditions on α_n for eigenvalues and eigenfunctions respectively, by Theorem 4.1, we have $\sqrt{n}({\lambda }_k^{\left[{\alpha }_n\right]}-{\lambda }_k)\to 0$ and $\sqrt{n}({\gamma }_k^{\left[{\alpha }_n\right]}-{\gamma }_k)\to 0$ respectively. Since ${\{\sqrt{n}({\widehat{\gamma }}_k^{\left[\alpha \right]}-{\gamma }_k^{\left[\alpha \right]}),1\le k\le K,0\le \alpha \le {\alpha }_0\}}_n$ and ${\{\sqrt{n}({\widehat{\lambda }}_k^{\left[\alpha \right]}-{\lambda }_k^{\left[\alpha \right]}),1\le k\le K,0\le \alpha \le {\alpha }_0\}}_n$ converge in distribution by Theorem 4.2, they are tight. Hence, the asymptotic normalities of $\sqrt{n}({\widehat{\lambda }}_k^{\left[{\alpha }_n\right]}-{\lambda }_k^{\left[{\alpha }_n\right]})$ and $\sqrt{n}({\widehat{\gamma }}_k^{\left[{\alpha }_n\right]}-{\gamma }_k^{\left[{\alpha }_n\right]})$ follows from Theorem 4.2 and the following lemma. Then the corollary follows at once.

Lemma 20

Suppose that F is a metric space with distance d. Let C_F[0, α₀] denote the continuous function on [0, α₀] taking values in F. Suppose we have a sequence {Y_n(α), 0 ≤ α ≤ α₀, n ∈ ℕ} of stochastic processes has sample paths in C_F[0, α₀]. Assume that Y_n is tight and Y_n(0) converges in distribution to a random element Y in F, then for any sequence α_n of positive numbers converging to 0, Y_n(α_n) also converges in distribution to Y.

Proof. First, we show that for any ∈ > 0 we can find δ > 0 such that

$$\underset{n}{\sup }P(\underset{0\le {\alpha }^{\prime },{\alpha }^{″}\le \delta }{\sup }d(Y_n\left({\alpha }^{\prime }\right),Y_n\left({\alpha }^{″}\right))>∊)\le ∊.$$

Since Y_n is tight, we can find a compact subset Χ of C_F[0, α₀] such that

$$\underset{n}{\sup }P(Y_n\notin \mathrm{\Xi })\le ∊.$$

We can find a finite number of Λ₁, … Λ_m ∈ Χ such that for any Λ ∈ Χ, we can find i such that ${\sup }_{0\le \alpha \le {\alpha }_0}d({\Lambda }_i\left(\alpha \right),\Lambda \left(\alpha \right))\le \frac{∊}{3}$. Furthermore, we can find δ > 0 such that,

$$\underset{1\le i\le m}{\max }\underset{0\le {\alpha }^{\prime },{\alpha }^{″}\le \delta }{\sup }d({\mathrm{\Lambda }}_i\left({\alpha }^{\prime }\right),{\mathrm{\Lambda }}_i\left({\alpha }^{″}\right))\le \frac{∊}{3}.$$

Now it is easy to see that for any Λ ∈ Χ,

$$\underset{0\le {\alpha }^{\prime },{\alpha }^{″}\le \delta }{\sup }d(\mathrm{\Lambda }\left({\alpha }^{\prime }\right),\mathrm{\Lambda }\left({\alpha }^{″}\right))\le ∊.$$

Hence,

$$\underset{n}{\sup }P(\underset{0\le {\alpha }^{\prime },{\alpha }^{″}\le \delta }{\sup }d(Y_n\left({\alpha }^{\prime }\right),Y_n\left({\alpha }^{″}\right))>∊)\le \underset{n}{\sup }P(Y_n\notin \mathrm{\Xi })\le ∊.$$

If α_n ≤ δ, we have

$$P(d(Y_n\left(0\right),Y_n\left({\alpha }_n\right))>∊)\le ∊.$$

Since ∈ is arbitrary, d(Y_n(0), Y_n(α_n)) → 0 in probability.