Inference for modulated stationary processes

Abstract

We study statistical inferences for a class of modulated stationary processes with time-dependent variances. Due to non-stationarity and the large number of unknown parameters, existing methods for stationary or locally stationary time series are not applicable. Based on a self-normalization technique, we address several inference problems, including self-normalized central limit theorem, self-normalized cumulative sum test for the change-point problem, long-run variance estimation through blockwise self-normalization, and self-normalization-based wild boot-strap. Monte Carlo simulation studies show that the proposed self-normalization-based methods outperform stationarity-based alternatives. We demonstrate the proposed methodology using two real data sets: annual mean precipitation rates in Seoul during 1771–2000, and quarterly U.S. Gross National Product growth rates during 1947–2002.

1. Introduction

In time series analysis, stationarity requires that dependence structure be sustained over time, and thus we can borrow information from one time period to study model dynamics over another period; see Fan and Yao (2003) for nonparametric treatments and Lahiri (2003) for various resampling and block bootstrap methods. In practice, however, many climatic, economic, and financial time series are non-stationary and therefore challenging to analyze. First, since dependence structure varies over time, information is more localized. Second, non-stationary processes often require extra parameters to account for time-varying structure. One way to overcome these issues is to impose certain local stationarity; see, for example, Dahlhaus (1997) and Adak (1998) for spectral representation frameworks and Dahlhaus and Polonik (2009) for a time domain approach.

In this article we study a class of modulated stationary processes [see Adak (1998)]:

$$X_i=\mu +{\sigma }_i{e}_i,i=1,\dots ,n,$$ (1.1)

where e_i are stationary time series with zero mean, and σ_i > 0 are unknown constants adjusting for time-dependent variances. Then X_i oscillates around the constant mean μ, whereas its variance changes over time in an unknown manner. In the special case of σ_i ≡ 1, (1.1) reduces to stationary case. If σ_i = s(i/n) for a Lipschitz continuous function s(t) on [0, 1], then (1.1) is locally stationary. For the general non-stationary case (1.1), the number of unknown parameters is larger than the number of observations, and it is infeasible to estimate σ_i. Due to non-stationarity and the large number of unknown parameters, existing methods that are developed for (locally) stationary processes are not applicable, and our main purpose is to develop new statistical inference techniques.

First, we establish a uniform strong approximation result which can be used to derive a self-normalized central limit theorem (CLT) for the sample mean X̅ of (1.1). For stationary case σ_i ≡ 1, by Fan and Yao (2003), under mild mixing conditions,

$$\sqrt{n}(\overline{X}-\mu )\Rightarrow N(0,{\tau }^2),\text{ where }{\tau }^2={\gamma }_0+2{\displaystyle \sum _{k=1}^{\mathrm{\infty }}}{\gamma }_k\text{ and }{\gamma }_k=\text{Cov}(e_i,e_{i+k}).$$ (1.2)

For the modulated stationary case (1.1), it is non-trivial whether $\sqrt{n}(\overline{X}-\mu )$ has a CLT without imposing further assumptions on σ_i and the dependence structure of e_i. Moreover, even when the latter CLT exists, it is difficult to estimate the limiting variance due to the large number of unknown parameters; see De Jong and Davidson (2000) for related work assuming a near-epoch dependent mixing framework. Zhao (2011) studied confidence interval construction for μ in (1.1) by assuming a block-wise asymptotically-equal-cumulative-variance assumption. The latter assumption is rather restrictive and essentially requires that block averages be asymptotically independent and identically distributed (IID). In this article, we deal with the more general setting (1.1). Under a strong invariance principle assumption, we establish a self-normalized CLT with the self-normalizing constant adjusting for time-dependent non-stationarity. The obtained CLT is an extension of the classical CLT for IID data or stationary time series to modulated stationary processes. Furthermore, we extend the idea to linear combinations of means over different time periods, which allows us to address inference regarding mean levels over multiple time periods.

Second, we study wild bootstrap for modulated stationary processes. Since the seminal work of Efron (1979), a great deal of research has been done on bootstrap under various settings, ranging from bootstrap for IID data in Efron (1979), wild bootstrap for independent observations with possibly non-constant variances in Wu (1986) and Liu (1988), to various block bootstrap and resampling methods for stationary time series in Künsch (1989), Politis and Romano (1994), Bühlmann (2002), and the monograph Lahiri (2003). With the established self-normalized CLT, we propose a wild bootstrap procedure that is tailored to deal with modulated stationary processes: the dependence is removed through a scaling factor, and the non-constant variance structure of the original data is preserved in the wild bootstrap data-generating mechanism. Our simulation study shows that the wild bootstrap method outperforms the widely used stationarity-based block bootstrap.

Third, we address change-point analysis. The change-point problem has been an active area of research; see Pettitt (1980) for proportion changes in binary data, Horváth (1993) for mean and variance changes in Gaussian observations, Bai and Perron (1998) for coefficient changes in linear models, Aue et al. (2008a) for coefficient changes in polynomial regression with uncorrelated errors, Aue et al. (2008b) for mean change in time series with stationary errors, Shao and Zhang (2010) for change-points for stationary time series, and the monograph by Csörgő and Horváth (1997) for more discussion. Most of these works deal with stationary and/or independent data. Hansen (2000) studied tests for constancy of parameters in linear regression models with non-stationary regressors and conditionally homoscedastic martingale difference errors. Here we consider

$$H_0:X_i={\mu }_i+{\sigma }_i{e}_i,{\mu }_1=\cdots ={\mu }_n,H_a:{\mu }_1=\cdots ={\mu }_J\ne {\mu }_{J+1}=\cdots ={\mu }_n,$$ (1.3)

where J is an unknown change-point. The aforementioned works mainly focused on detecting changes in mean while the error variance is constant. On the other hand, researchers have also realized the importance of the variance/covariance structure in change-point analysis. For example, Inclán and Tiao (1994) studied change in variance for independent data, and Aue et al. (2009) and Berkes, Gombay and Horváth (2009) considered change in covariance for time series data. To our knowledge, there has been almost no attempt to advance change-point analysis under the non-constant variances framework in (1.3). Andrews (1993) studied change-point problem under near-epoch dependence structure that allows for non-stationary processes, but his Assumption 1 (c) on page 830 therein essentially implies that the process has constant variance. The popular cumulative sum (CUSUM) test is developed for stationary time series and does not take into account the time-dependent variances. Using the self-normalization idea, we propose a self-normalized CUSUM test and a wild bootstrap method to obtain its critical value. Our empirical studies show that the usual CUSUM tests tend to over-reject the null hypothesis in the presence of non-constant variances. By contrast, the self-normalized CUSUM test yields size close to the nominal level.

Fourth, we estimate the long-run variance τ² in (1.2). Long-run variance plays an essential role in statistical inferences involving time series. Most works in the literature deal with stationary processes through various block bootstrap and subsampling approaches; see Carlstein (1986), Künsch (1989), Politis and Romano (1994), Götze and Künsch (1996), and the monograph Lahiri (2003). De Jong and Davidson (2000) established the consistency of kernel estimators of covariance matrices under a near epoch dependent mixing condition. Recently, Müller (2007) studied robust long-run variance estimation for locally stationary process. For model (1.1), the error process {e_i} is contaminated with unknown standard deviations {σ_i}, and we apply blockwise self-normalization to remove non-stationarity, resulting in asymptotically stationary blocks.

Fifth, the proposed methods can be extended to deal with linear regression model

$$X_i=U_i\beta +{\sigma }_i{e}_i,$$ (1.4)

where U_i = (u_i,1, …, u_i,p) are deterministic covariates, and β = (β₁, …, β_p)′ is the unknown column vector of parameters. For p = 2, Hansen (1995) established the asymptotic normality of the least-squares estimate of the slope parameter under a fairly general framework of non-stationary errors. While Hansen (1995) assumed that the errors form a martingale difference array so that they are uncorrelated, the framework in (1.4) is more general in that it allows for correlations. On the other hand, Hansen (1995) allowed the conditional volatilities to follow an autoregressive model, hence introducing stochastic volatilities. Phillips, Sun and Jin (2007) considered (1.4) for stationary errors and their approach is not applicable here due to the unknown non-constant variances ${\sigma }_i^2$. In Section 2.6 we consider self-normalized CLT for the least-squares estimator of β in (1.4). In the polynomial regression case u_i,r = (i/n)^r−1, Aue et al. (2008a) studied a likelihood-based test for constancy of β in (1.4) for uncorrelated errors with constant variance. Due to the presence of correlation and time-varying variances, it is more challenging to study the change-point problem for (1.4) and this is beyond the scope of this article.

The rest of this article is organized as follows. We present theoretical results in Section 2. Sections 3–4 contain Monte Carlo studies and applications to two real data sets.

2. Main results

For sequences {a_n} and {b_n}, write a_n = O(b_n), a_n = o(b_n), and a_n ≍ b_n, respectively, if |a_n/b_n| < c₁, a_n/b_n → 0, and c₂ < |a_n/b_n| < c₃, for some constants 0 < c₁, c₂, c₃ < ∞. For q > 0 and a random variable e, write e ∈ ℒ^q if ‖e‖_q ≔ {E(|e|^q)}^1/q < ∞.

2.1. Uniform approximations for modulated stationary processes

In (1.1), assume without loss of generality that E(e_i) = 0 and $E(e_i^2)=1$ so that {e_i} and $\{e_i^2-1\}$ are centered stationary processes. With the convention $S_0=S_0^{*}=0$, define

$$S_i={\displaystyle \sum _{j=1}^i}{e}_j\text{ and }{S}_i^{*}={\displaystyle \sum _{j=1}^i}(e_j^2-1),i=1,2,\dots .$$ (2.1)

Assumption 2.1. There exist standard Brownian motions {B_t} and $\{B_t^{*}\}$ such that

$$\underset{1\le i\le n}{\text{max}}|S_i-\tau B_i|=o_{\mathrm{a}.\mathrm{s}.}({\mathrm{\Delta }}_n)\text{ and }\underset{1\le i\le n}{\text{max}}|S_i^{*}-\tau *B_i^{*}|=o_{\mathrm{a}.\mathrm{s}.}({\mathrm{\Delta }}_n),$$ (2.2)

where Δ_n is the approximation error, τ² and τ^*2 are the long-run variances of {e_i} and $\{e_i^2-1\}$, respectively. Further assume τ² > 0 to avoid the degenerate case τ² = 0.

The uniform approximations in (2.2) are generally called strong invariance principle. The two Brownian motions {B_t} and $\{B_t^{*}\}$ may be defined on different probability spaces and hence are not jointly distributed, which is not an issue because our argument does not depend on their joint distribution. To see how to use (2.2), under H₀ in (1.3), consider

$$F_j=j({\underset{¯}{X}}_j-\mu )\text{ and }{\underset{¯}{V}}_j^2={\displaystyle \sum _{i=1}^j}{(X_i-{\underset{¯}{X}}_j)}^2,\text{ where }{\underset{¯}{X}}_j=j^{-1}{\displaystyle \sum _{i=1}^j}{X}_i.$$ (2.3)

Theorem 2.1 below presents uniform approximations for F_j and $V_j^2$. Define

$$r_n=|{\sigma }_n|+{\displaystyle \sum _{i=2}^n}|{\sigma }_i-{\sigma }_{i-1}|\text{ \hspace{1em}and \hspace{1em}}{r}_n^{*}=|{\sigma }_n^2|+{\displaystyle \sum _{i=2}^n}|{\sigma }_i^2-{\sigma }_{i-1}^2|,$$ (2.4)

$${\displaystyle {\sum }_j^2}={\displaystyle \sum _{i=1}^j}{\sigma }_i^2\text{ \hspace{1em}and \hspace{1em}}{\displaystyle {\sum }_j^{*2}}={\left({\displaystyle \sum _{i=1}^j}{\sigma }_i^4\right)}^{1/2}.$$ (2.5)

Theorem 2.1. Let (2.2) hold. For any c ∈ (0, 1], the uniform approximations hold:

$$\underset{\mathit{\text{cn}}\le j\le n}{\text{max}}\left|F_j-\tau {\displaystyle \sum _{i=1}^j}{\sigma }_i(B_i-B_{i-1})\right|=O_{\mathrm{a}.\mathrm{s}.}(r_n{\mathrm{\Delta }}_n),$$ (2.6)

$$\underset{\mathit{\text{cn}}\le j\le n}{\text{max}}\left|{\underset{¯}{V}}_j^2-{\mathrm{\Sigma }}_j^2\right|=O_{\mathrm{p}}\{(r_n^2{\mathrm{\Delta }}_n^2+{\mathrm{\Sigma }}_n^2)/n+{\mathrm{\Sigma }}_n^{*2}+r_n^{*}{\mathrm{\Delta }}_n\}.$$ (2.7)

Theorem 2.1 provides quite general results under (2.2). We now discuss sufficient conditions for (2.2). Shao (1993) obtained sufficient mixing conditions for (2.2). In this article, we briefly introduce the framework in Wu (2007). Assume that e_i has the causal representation e_i = G (…, ε_i−1, ε_i), where ε_i are IID innovations, and G is a measurable function such that e_i is well-defined. Let ${\{{\epsilon }_i^{\prime }\}}_{i\in \mathbb{Z}}$ be an independent copy of {ε_i}_i∈ℤ. Assume

$${\displaystyle \sum _{i=1}^{\mathrm{\infty }}}i{\Vert e_i-e_i^{\prime }\Vert }_8<\mathrm{\infty },\text{ where \hspace{1em}}{e}_i^{\prime }=G(\dots ,{\epsilon }_{-1},{\epsilon }_0^{\prime },{\epsilon }_1\dots ,{\epsilon }_{i-1},{\epsilon }_i).$$ (2.8)

Proposition 2.1 below follows from Corollary 4 in Wu (2007).

Proposition 2.1. Assume that (2.8) holds. Then (2.2) holds with Δ_n = n^1/4 log(n), the optimal rate up to a logarithm factor.

For linear process $e_i={\displaystyle {\sum }_{j=0}^{\mathrm{\infty }}}{a}_j{\epsilon }_{i-j}$ with ε_i ∈ ℒ⁸ and E (ε_i) = 0, ${\Vert e_i-e_i^{\prime }\Vert }_8={\Vert {\epsilon }_0-{\epsilon }_0^{\prime }\Vert }_8|a_i|$. If ${\displaystyle {\sum }_{i=1}^{\mathrm{\infty }}}i|a_i|$ < ∞, then (2.2) holds with Δ_n = n^1/4 log(n). For many nonlinear time series, ${\Vert e_i-e_i^{\prime }\Vert }_8$ decays exponentially fast and hence (2.8) holds; see Section 3.1 of Wu (2007). From now on we assume (2.2) holds with Δ_n = n^1/4 log(n).

Remark 2.1. If e_i are IID with E(e_i) = 0 and e_i ∈ ℒ^q for some 2 < q ≤ 4, the celebrated “Hungarian embedding” asserts that ${\displaystyle {\sum }_{j=1}^i}{e}_j$ satisfies a strong invariance principle with the optimal rate o_a.s. (n^1/q). Thus, it is necessary to have the moment assumption e_i ∈ ℒ⁸ in Proposition 2.1 in order to ensure strong invariance principles for both S_i and $S_i^{*}$ in (2.1) with approximation rate n^1/4 log(n). On the other hand, one can relax the moment assumption by loosening the approximation rate. For example, by Corollary 4 in Wu (2007), assume e_i ∈ ℒ^2q for some q > 2 and ${\displaystyle {\sum }_{i=1}^{\mathrm{\infty }}}i{\Vert e_i-e_i^{*}\Vert }_{2q}<\mathrm{\infty }$, then (2.2) holds with Δ_n = n^1/min(q,4) log(n).

As shown in Examples 2.1–2.3 below, r_n and $r_n^{*}$ in (2.4) often have tractable bounds.

Example 2.1. If σ_i is non-decreasing in i, then σ_n ≤ r_n ≤ 2σ_n and ${\sigma }_n^2\le r_n^{*}\le 2{\sigma }_n^2$. If σ_i is non-increasing in i, then r_n = σ₁ and $r_n^{*}={\sigma }_1^2$. If σ_i are piecewise constants with finitely many pieces, then $r_n,r_n^{*}=O(1)$.

Example 2.2. Let σ_i = s(i/n^γ) for γ ∈ [0, 1] and a Lipschitz continuous function s(t), t ∈ [0, ∞), sup_{t∈[0, ∞)} s(t) < ∞. Then $r_n,r_n^{*}=O(n^{1-\gamma })$. If γ = 1, we obtain locally stationary case with time window i/n ∈ [0, 1]; if γ ∈ [0, 1), we have infinite time window [0, ∞) as n/n^γ → ∞, which may be more reasonable for data with a long time horizon.

Example 2.3. If σ_i = i^βL(i) for a slowly varying function L(·) such that L(cx)/L(x) → 1 as x → ∞ for all c > 0. Then we can show r_n = O{n^βL(n)} or O(1) and $r_n^{*}=O\{n^{2\beta }{L}^2(n)\}$ or O(1), depending on whether β > 0 or β < 0. For the boundary case β = 0, assume L (i+1)/L(i) = 1+O(1/i) uniformly, then $r_n=L(n)+O(1){\displaystyle {\sum }_{i=2}^n}L(i)/i=O\{\text{log}(n){\text{max}}_{1\le i\le n}L(i)\}$. Similarly, $r_n^{*}=O\{\text{log}(n){\text{max}}_{1\le i\le n}{L}^2(i)\}$.

2.2. Self-normalized central limit theorem

In this section we establish a self-normalized CLT for the sample average X̅ . To understand how non-stationarity makes this problem difficult, elementary calculation shows

$$\text{Var}\{\sqrt{n}(\overline{X}-\mu )\}=\frac{{\gamma }_0}{n}{\displaystyle \sum _{i=1}^n}{\sigma }_i^2+\frac{2}{n}{\displaystyle \sum _{1\le i<j\le n}}{\sigma }_i{\sigma }_j{\gamma }_{j-i}≔{\tau }_n^2,$$ (2.9)

where γ_k = Cov(e₀, e_k). In the stationary case σ_i ≡ 1, under condition ${\displaystyle {\sum }_{k=0}^{\mathrm{\infty }}}|{\gamma }_k|<\mathrm{\infty },{\tau }_n^2\to {\tau }^2$, the long-run variance in (1.2). For non-constant variances, it is difficult to deal with ${\tau }_n^2$ directly, due to the large number of unknown parameters and complicated structure. See De Jong and Davidson (2000) for a kernel estimator of ${\tau }_n^2$ under a near-epoch dependent mixing framework.

To attenuate the aforementioned issue, we apply the uniform approximations in Theorem 2.1. Assume that (2.10) below holds. Note that the increments B_i − B_i−1 of standard Brownian motions are IID standard normal random variables. By (2.6), n(X̅ −μ) is equivalent to $N(0,{\tau }^2{\mathrm{\Sigma }}_n^2)$ in distribution. By (2.7), V̲_n/Σ_n → 1 in probability. By Slutsky’s theorem, we have the following Proposition 2.2.

Proposition 2.2. Let (2.2) hold with Δ_n = n^1/4 log(n). For $r_n,r_n^{*},{\mathrm{\Sigma }}_n^2,{\mathrm{\Sigma }}_n^{*2}$ in (2.4)–(2.5), assume

$${\delta }_n=r_n{\mathrm{\Delta }}_n/{\mathrm{\Sigma }}_n+(r_n^{*}{\mathrm{\Delta }}_n+{\mathrm{\Sigma }}_n^{*2})/{\mathrm{\Sigma }}_n^2\to 0.$$ (2.10)

Recall ${\underset{¯}{V}}_n^2$ in (2.3). Then as n → ∞, n(X̅ − μ)/V̲_n ⇒ N(0, τ²). Consequently, a (1 − α) asymptotic confidence interval for μ is X̅ ± z_α/2τ̂V̲_n/n, where τ̂ is a consistent estimate of τ (Section 2.5 below), and z_α/2 is (1 − α/2) standard normal quantile.

Proposition 2.2 is an extension of the classical CLT for IID data or stationary processes to modulated stationary processes. If X_i are IID, then n(X̅ − μ)/V̲_n ⇒ N(0, 1). In Proposition 2.2, τ² can be viewed as the variance inflation factor due to the dependence of {e_i}. For stationary data, the sample variance ${\underset{¯}{V}}_n^2/n$ is a consistent estimate of the population variance. Here, for non-constant variances case (1.1), by (2.7) in Theorem 2.1, ${\underset{¯}{V}}_n^2/n$ can be viewed as an estimate of the time-average “population variance” ${\mathrm{\Sigma }}_n^2/n$. So, we can interpret the CLT in Proposition 2.2 as a self-normalized CLT for modulated stationary processes with the self-normalizing term V̲_n adjusting for non-stationarity due to σ₁, …, σ_n and τ² accounting for dependence of {e_i}. Clearly, parameters σ₁, …, σ_n are canceled out through self-normalization. Finally, condition (2.10) is satisfied for Example 2.2 with γ > 3/4 and Example 2.3 with β > −1/4.

In classical statistics, the width of confidence intervals usually shrinks as sample size increases. By Proposition 2.2 and Theorem 2.1, the width of the constructed confidence interval for μ is proportional to V̲_n/n or, equivalently, Σ_n/n. Thus, a necessary and sufficient condition for shrinking confidence interval is ${\displaystyle {\sum }_{i=1}^n}{\sigma }_i^2/n^2\to 0$, which is satisfied if ${\sigma }_i=o(\sqrt{i})$. An intuitive explanation is as follows. For IID data, sample mean converges at a rate of $O(\sqrt{n})$. In (1.1), if σ_i grows faster than $O(\sqrt{i})$, the contribution of a new observation is negligible relative to its noise level.

Example 2.4. If σ_i ≍ i^β with β ∈ [0, 1/2), the length of confidence interval is proportional to Σ_n/n ≍ n^β−1/2. In particular, if c₁ < σ_i < c₂ for some positive constants c₁ and c₂, then Σ_n/n achieves the optimal rate O(n^−1/2). If σ_i ≍ log(i), then ${\mathrm{\Sigma }}_n/n\asymp \text{log}(n)/\sqrt{n}$.

The same idea can be extended to linear combinations of means over multiple time periods. Suppose we have observations from k consecutive time periods 𝒯₁, …, 𝒯_k, each of the form (1.1) with different means, denoted by μ₁, …, μ_k, and each having time-dependent variances. Let ν = β₁μ₁ + ⋯ + β_kμ_k for given coefficients β₁, …, β_k. For example, if we are interested in mean change from 𝒯₁ to 𝒯₂, we can take ν = μ₂ − μ₁; if we are interested in whether the increase from 𝒯₃ to 𝒯₄ is larger than that from 𝒯₁ to 𝒯₂, we can let ν = (μ₄ − μ₃) − (μ₂ − μ₁). Proposition 2.3 below extends Proposition 2.2 to multiple means.

Proposition 2.3. Let ν = β₁μ₁ + ⋯ + β_kμ_k. For 𝒯_j, denote its sample size n_j and its sample average X̅ (j). Assume that (2.10) holds for each individual time periods 𝒯_j and for simplicity that n₁, …, n_k are of the same order. Then

$$\frac{{\displaystyle {\sum }_{j=1}^k}{\beta }_j\overline{X}(j)-\nu }{{\mathrm{\Lambda }}_n}\Rightarrow N(0,{\tau }^2),\mathit{\text{ where \hspace{1em}}}{\mathrm{\Lambda }}_n^2={\displaystyle \sum _{j=1}^k}\left\{\frac{{\beta }_j^2}{n_j^2}{\displaystyle \sum _{i\in {𝒯}_j}}{[X_i-\overline{X}(j)]}^2\right\}.$$

2.3. Wild bootstrap for self-normalized statistic

Recall σ_ie_i in (1.1). Suppose we are interested in the self-normalized statistic

$$H_n=\frac{{\displaystyle {\sum }_{i=1}^n}{\sigma }_i{e}_i}{\sqrt{{\displaystyle {\sum }_{i=1}^n}{\sigma }_i^2{e}_i^2}}.$$

For problems with small sample sizes, it is natural to use bootstrap distribution instead of the convergence H_n ⇒ N(0, τ²) in Proposition 2.2. Wu (1986) and Liu (1988) have pioneered the work on wild bootstrap for independent data with non-identical distributions. We shall extend their wild bootstrap procedure to the modulated stationary process (1.1).

Let {α_i} be IID random variables independent of {e_i} satisfying α_i ∈ ℒ³, E(α_i) = 0, $E({\alpha }_i^2)=1$. Define the self-normalized statistic based on new data:

$$H_n^{*}=\frac{{\displaystyle {\sum }_{i=1}^n}{\xi }_i}{\sqrt{{\displaystyle {\sum }_{i=1}^n}{({\xi }_i-\overline{\xi })}^2}},\text{ where \hspace{1em}}{\xi }_i={\sigma }_i{e}_i{\alpha }_i\text{ and \hspace{1em}}\overline{\xi }=\frac{{\xi }_1+\cdots +{\xi }_n}{n}.$$

Clearly, ξ_i inherits the non-stationarity structure of σ_ie_i by writing ${\xi }_i={\sigma }_i{e}_i^{*}\text{ with }{e}_i^{*}=e_i{\alpha }_i$. On the other hand, for the new error process $\{e_i^{*}\},E(e_i^{*2})=E(e_i^2)=1\text{ and Cov}(e_i^{*},e_j^{*})=0$ for i ≠ j. Thus, $\{e_i^{*}\}$ is a white noise sequence with long-run variance one. By Proposition 2.2, the scaled version H_n/τ ⇒ N(0, 1) is robust against the dependence structure of {e_i}, so we expect that $H_n^{*}$ should be close to H_n/τ in distribution.

Theorem 2.2. Let the conditions in Proposition 2.2 hold. Further assume

$${\left({\displaystyle \sum _{i=1}^n}{\sigma }_i^3\right)}^2{\left({\displaystyle \sum _{i=1}^n}{\sigma }_i^2\right)}^{-3}\to 0.$$ (2.11)

Let τ̂ be a consistent estimate of τ . Denote by ℙ* the conditional law given {e_i}. Then

$$\underset{x\in ℝ}{\text{sup}}|{\mathbb{P}}^{*}(H_n^{*}\le x)-\mathbb{P}(H_n/\hat{\tau }\le x)|\to 0,\mathit{\text{ in probability}}.$$ (2.12)

Theorem 2.2 asserts that, $H_n^{*}$ behaves like the scaled version H_n/τ̂, with the scaling factor τ̂ coming from the dependence of {e_i}. Here we use the sample mean X̅ in (1.1) to illustrate a wild bootstrap procedure to obtain the distribution of n(X̅ − μ)/(τV̲_n) in Proposition 2.2.

• (i)Apply method in Section 2.5 to X₁, …, X_n to obtain a consistent estimate τ̂ of τ.
• (ii)Subtract the sample mean X̅ from data to obtain ε_i = X_i − X̅, i = 1, …, n.
• (iii)Generate IID random variables α₁, …, α_n satisfying E(α_i) = 0, $E({\alpha }_i^2)=1$.
• (iv)
  Based on ε_i in (ii) and α_i in (iii), generate bootstrap data ${\xi }_i^b={\epsilon }_i{\alpha }_i$, and compute

  $$H_n^b=\frac{{\displaystyle {\sum }_{i=1}^n}{\xi }_i^b}{{\hat{\tau }}^b\sqrt{{\displaystyle {\sum }_{i=1}^n}{({\xi }_i^b-{\overline{\xi }}^b)}^2}},$$

  where τ̂^b is a long-run variance estimate (see Section 2.5) for bootstrap data ${\xi }_i^b$.
• (vi)Repeat (ii)–(iv) many times and use the empirical distribution of those realizations of $H_n^b$ as the distribution of n(X̅ − μ)/(τV̲_n).

The proposed wild bootstrap is an extension of that in Liu (1988) for independent data to modulated stationary case, and it has two appealing features. First, the scaling factor τ̂ makes the statistic independent of the dependence structure. Second, the bootstrap data-generating mechanism is adaptive to unknown time-dependent variances $\{{\sigma }_i^2\}$. For the distribution of α_i in step (iii), we use ℙ(α_i = −1) = ℙ(α_i = 1) = 1/2, which has some desirable properties. For example, it preserves the magnitude and range of the data. As shown by Davidson and Flachaire (2008), for certain hypothesis testing problems in linear regression models with symmetrically distributed errors, the bootstrap distribution is exactly equal to that of the test statistic; see Theorem 1 therein.

For the purpose of comparison, we briefly introduce the widely used block bootstrap for a stationary time series {X_i} with mean μ. By (1.2), $\sqrt{n}(\overline{X}-\mu )\Rightarrow N(0,{\tau }^2)$. Suppose that we want to bootstrap the distribution of $\sqrt{n}(\overline{X}-\mu )$. Let k_n, ℓ_n, ℐ₁, …, ℐ_ℓn be defined as in Section 2.5 below. The non-overlapping block bootstrap works as follows

1. Take a simple random sample of size ℓ_n with replacement from the blocks ℐ₁, …, ℐ_ℓn, and form the bootstrap data $X_1^b,\dots ,X_{n\prime }^b,n\prime =k_n{\ell }_n$, by pooling together X_i’s for which the index i is within those selected blocks.

2. Let X̅^b be the sample average of $X_1^b,\dots ,X_{n\prime }^b$. Compute ${\mathrm{\Xi }}_n=\sqrt{n\prime }\{{\overline{X}}^b-E^{*}({\overline{X}}^b)\},\text{ where }{E}^{*}({\overline{X}}^b)={\displaystyle {\sum }_{i=1}^{n\prime }}{X}_i/n\prime$ is the conditional expectation of X̅^b given {X_i}.

3. Repeat (i)–(ii) many times and use the empirical distribution of Ξ_n’s as the distribution of $\sqrt{n}(\overline{X}-\mu )$.

In step (ii), another choice is the studentized version ${\mathrm{\Xi ̃}}_n=\sqrt{n\prime }\{{\overline{X}}^b-E^{*}({\overline{X}}^b)\}/{\hat{\tau }}^b$, where τ̂^b is a consistent estimate of τ based on bootstrap data. Assuming stationarity and k_n → ∞, the blocks are asymptotically independent and share the same model dynamics as the whole data, which validates the above block bootstrap. We refer the reader to Lahiri (2003) for detailed discussions. For a non-stationary process, block bootstrap is no longer valid, because individual blocks are not representative of the whole data. By contrast, the proposed wild bootstrap is adaptive to unknown dependence and the non-constant variances structure.

2.4. Change-point analysis: self-normalized CUSUM test

To test a change-point in the mean of a process {X_i}, two popular CUSUM-type tests [see Section 3 of Robbins et al. (2011) for a review and related references] are

$$T_n^1=\underset{\mathit{\text{cn}}\le j\le (1-c)n}{\text{max}}\frac{{\hat{\tau }}^{-1}|S_X(j)|}{\sqrt{j(1-j/n)}}\text{ \hspace{1em}and \hspace{1em}}{T}_n^2=\underset{\mathit{\text{cn}}\le j\le (1-c)n}{\text{max}}{\hat{\tau }}^{-1}|S_X(j)|,$$ (2.13)

where τ̂² is a consistent estimate of the long-run variance τ² of {X_i}, and

$$S_X(j)=(1-\frac{j}{n}){\displaystyle \sum _{i=1}^j}{X}_i-\frac{j}{n}{\displaystyle \sum _{i=j+1}^n}{X}_i.$$ (2.14)

Here c > 0 (c = 0.1 in our simulation studies) is a small number to avoid the boundary issue. For IID data, j(1 − j/n) is proportional to the variance of S_X (j), so $T_n^1$ is a studentized version of $T_n^2$. For IID Gaussian data, $T_n^1$ is equivalent to likelihood ratio test; see Csörgő and Horváth (1997). Assume that, under null hypothesis,

$${\{n^{-1/2}{\displaystyle \sum _{i=1}^{\lfloor \mathit{\text{nt}}\rfloor }}[X_i-E(X_i)]\}}_{0\le t\le 1}\Rightarrow \tau {\{B_t\}}_{0\le t\le 1},\text{ in the Skorohod space},$$ (2.15)

for a standard Brownian motion {B_t}_t≥0. The above convergence requires finite-dimensional convergence and tightness; see Billingsley (1968). By the continuous mapping theorem, $T_n^1\Rightarrow {\text{max}}_{c\le t\le 1-c}|B_t-{\mathit{\text{t B}}}_1|/\sqrt{t(1-t)}\text{ and }{T}_n^2/\sqrt{n}\Rightarrow {\text{max}}_{c\le t\le 1-c}|B_t-{\mathit{\text{t B}}}_1|$.

For the modulated stationary case (1.3), (2.15) is no longer valid. Moreover, since $T_n^1\text{ and }{T}_n^2$ do not take into account the time-dependent variances ${\sigma }_i^2$, an abrupt change in variances may lead to a false rejection of H₀ when the mean remains constant. For example, our simulation study in Section 3.3 shows that the empirical false rejection probability for $T_n^1\text{ and }{T}_n^2$ is about 10% for nominal level 5%. To alleviate the issue of non-constant variances, we adopt the self-normalization approach as in previous sections. Recall F_j and V̲_j in (2.3). For each fixed cn ≤ j ≤ (1 − c)n, by Theorem 2.1 and Slutsky’s theorem, F_j/V̲_j ⇒ N(0, τ²) in distribution, assuming the negligibility of the approximation errors. Therefore, the self-normalization term V̲_j can remove the time-dependent variances. In light of this, we can simultaneously self-normalize the two terms ${\displaystyle {\sum }_{i=1}^j}{X}_i\text{ and }{\displaystyle {\sum }_{i=j+1}^n}{X}_i$ in (2.14) and propose the self-normalized test statistic

$$T_n^{\text{SN}}=\underset{\mathit{\text{cn}}\le j\le (1-c)n}{\text{max}}{\hat{\tau }}^{-1}|T_n(j)|,\text{ where \hspace{1em}}{T}_n(j)=\frac{S_X(j)}{\sqrt{{(1-j/n)}^2{\underset{¯}{V}}_j^2+{(j/n)}^2{\overline{V}}_j^2}}.$$ (2.16)

Here, ${\underset{¯}{V}}_j^2$ is defined as in (2.3), ${\overline{V}}_j^2={\displaystyle {\sum }_{i=j+1}^n}{(X_i-{\overline{X}}_j)}^2\text{ with }{\overline{X}}_j={(n-j)}^{-1}{\displaystyle {\sum }_{i=j+1}^n}{X}_i$.

Theorem 2.3. Assume (2.2) holds. Let δ_n → 0 be as in (2.10). Under H₀, we have $\underset{\mathit{\text{cn}}\le j\le (1-c)n}{\text{max}}|T_n(j)-\tau {\tilde{T}}_n(j)|=O_{\mathrm{p}}({\delta }_n)$, where ${\tilde{T}}_n(j)=\frac{(1-j/n){\displaystyle {\sum }_{i=1}^j}{\sigma }_i(B_i-B_{i-1})-j/n{\displaystyle {\sum }_{i=j+1}^n}{\sigma }_i(B_i-B_{i-1})}{\sqrt{{(1-j/n)}^2{\displaystyle {\sum }_{i=1}^j}{\sigma }_i^2+{(j/n)}^2{\displaystyle {\sum }_{i=j+1}^n}{\sigma }_i^2}}$.

By Theorem 2.3, under H₀, $T_n^{\text{SN}}$ is asymptotically equivalent to max_{cn≤j≤(1−c)n}|T̃_n(j)|. Due to the self-normalization, for each j, the time-dependent variances are removed and T̃_n(j) ~ N(0, 1) has a standard normal distribution. However, T̃_n(j) and e T̃_n(j′) are correlated for j ≠ j′. Therefore, {T̃_n(j)} is a non-stationary Gaussian process with a standard normal marginal density. Due to the large number of unknown parameters σ_i, it is infeasible to obtain the null distribution directly. On the other hand, Theorem 2.3 establishes the fact that, asymptotically, the distribution of $T_n^{\text{SN}}$ in (2.16) depends only on σ₁, …, σ_n and is robust against the dependence structure of {e_i}, which motivates us to use the wild bootstrap method in Section 2.3 to find the critical value of $T_n^{\text{SN}}$.

1. Compute T_n(j) and find Ĵ = argmax_{cn≤j≤(1−c)n}|T_n(j)|.

2. Divide the data into two blocks X₁, …, X_Ĵ and X_Ĵ+1, …, X_n. Within each block, subtract the sample mean from the observations therein to obtain centered data. Pool all centered data together and denote them by ε₁, …, ε_n.

3. Based on ε₁, …, ε_n, obtain an estimate τ̂ of τ . See Section 2.5 below.

4. Compute the test statistic $T_n^{\text{SN}}$ in (2.16).

5. Based on ε_i in (ii), use the wild bootstrap method in Section 2.3 to generate synthetic data ξ₁, …, ξ_n, and use (i)–(iv) to compute the bootstrap test statistic $T_n^b$ based on the bootstrap data ξ₁, …, ξ_n.

6. Repeat (v) many times and find (1 − α) quantile of those $T_n^b’\mathrm{s}$.

As argued in Section 2.3, the synthetic data-generating scheme (v) inherits the time-varying non-stationarity structure of the original data. Also, the statistic $T_n^{\text{SN}}$ is robust against the dependence structure, which justifies the proposed bootstrap method. If H₀ is rejected, the change-point is then estimated by Ĵ = argmax_{cn≤j≤(1−c)n} |T_n(j)|.

If there is no evidence to reject H₀, we briefly discuss how to apply the same methodology to test H̃₀ : σ₁ = ⋯ = σ_J ≠ σ_J+1 = ⋯ = σ_n, that is, whether there is a change-point in the variances ${\sigma }_i^2$. By (1.1), we have ${(X_i-\mu )}^2={\sigma }_i^2+{\sigma }_i^2{\zeta }_i,\text{ where }{\zeta }_i=e_i^2-1$ has mean zero. Therefore, testing a change-point in the variances ${\sigma }_i^2$ of X_i is equivalent to testing a change-point in the mean of the new data X̃_i = (X_i − X̅)².

2.5. Long-run variance estimation

To apply the results in Sections 2.2–2.4, we need a consistent estimate of the long-run variance τ². Most existing works deal with stationary time series through various block bootstrap and subsampling approaches; see Lahiri (2003) and references therein. Assuming a near-epoch dependent mixing condition, De Jong and Davidson (2000) established the consistency of a kernel estimator of $\text{Var}({\displaystyle {\sum }_{i=1}^n}{X}_i)$, and their result can be used to estimate ${\tau }_n^2$ in (2.9) for the CLT of $\sqrt{n}(\overline{X}-\mu )$. However, for the change-point problem in Section 2.4, we need an estimator of the long-run variance τ² of the unobservable process {e_i}, so the method in De Jong and Davidson (2000) is not directly applicable.

To attenuate the non-stationarity issue, we extend the idea in Section 2.2 to blockwise self-normalization. Let k_n be the block length. Denote by ℓ_n = ⌊n/k_n⌋ the largest integer not exceeding n/k_n. Ignore the boundary and divide 1, …, n into ℓ_n blocks

$${ℐ}_j=\{(j-1)k_n+1,\dots ,{\mathit{\text{jk}}}_n\},j=1,\dots ,{\ell }_n.$$ (2.17)

Recall the overall sample mean X̅ . For each block j, define the self-normalized statistic

$$D_j=\frac{k_n[\overline{X}(j)-\overline{X}]}{V(j)},\text{ where \hspace{1em}}\overline{X}(j)=\frac{1}{k_n}{\displaystyle \sum _{i\in {ℐ}_j}}{X}_i,V^2(j)={\displaystyle \sum _{i\in {ℐ}_j}}{[X_i-\overline{X}(j)]}^2.$$ (2.18)

By Proposition 2.2, the self-normalized statistics D₁, …, D_ℓn ~ N(0, τ²) are asymptotically IID. Thus, we propose estimating τ² by

$${\hat{\tau }}^2=\frac{1}{{\ell }_n}{\displaystyle \sum _{j=1}^{{\ell }_n}}{D}_j^2.$$ (2.19)

As in (2.4)–(2.5), we define the quantities on block j

$$r(j)=|{\sigma }_{{\mathit{\text{j k}}}_n}|+{\displaystyle \sum _{i\in {ℐ}_j}}|{\sigma }_i-{\sigma }_{i-1}|\text{ \hspace{1em}and \hspace{1em}}{r}^{*}(j)=|{\sigma }_{{\mathit{\text{j k}}}_n}^2|+{\displaystyle \sum _{i\in {ℐ}_j}}|{\sigma }_i^2-{\sigma }_{i-1}^2|,$$ (2.20)

$${\mathrm{\Sigma }}^2(j)={\displaystyle \sum _{i\in {ℐ}_j}}{\sigma }_i^2\text{ \hspace{1em}and \hspace{1em}}{\mathrm{\Sigma }}^{*2}(j)={\left({\displaystyle \sum _{i\in {ℐ}_j}}{\sigma }_i^4\right)}^{1/2}.$$ (2.21)

Theorem 2.4. Let (2.2) hold with Δ_n = n^1/4 log(n). Recall r_n, Σ_n in (2.4)–(2.5). Define

$$M_n=\frac{1}{k_n}+\underset{1\le j\le {\ell }_n}{\text{max}}\frac{{\mathrm{\Sigma }}^{*2}(j)+r^{*}(j){\mathrm{\Delta }}_n}{{\mathrm{\Sigma }}^2(j)}+\underset{1\le j\le {\ell }_n}{\text{max}}\frac{r(j){\mathrm{\Delta }}_n}{\mathrm{\Sigma }(j)}.$$ (2.22)

Assume that r_nΔ_n/Σ_n → 0 and

$${\mathrm{\chi }}_n={\ell }_n^{-1/2}+\text{log}(n)M_n+\sqrt{\text{log}(n)}\frac{{\mathrm{\Sigma }}_n}{{\ell }_n^2}{\displaystyle \sum _{j=1}^{{\ell }_n}}\frac{1}{\mathrm{\Sigma }(j)}+\frac{{\mathrm{\Sigma }}_n^2}{{\ell }_n^3}{\displaystyle \sum _{j=1}^{{\ell }_n}}\frac{1}{{\mathrm{\Sigma }}^2(j)}\to 0.$$ (2.23)

Then τ̂² − τ² = O_p(χ_n). Consequently, τ̂ is a consistent estimate of τ.

Consider Example 2.2 with γ ∈ [0, 1). Then ${\chi }_n\asymp \sqrt{\text{log}(n)/{\ell }_n}+{\text{log}}^2(n)(n^{1/4}/\sqrt{k_n}+n^{5/4-\gamma }/k_n+\sqrt{k_n}{n}^{1/4-\gamma })$. For γ ∈ (3/4, 1), it can be shown that the optimal rate is χ_n ≍ n^−1/8 log^5/4(n) when k_n ≍ n^3/4 log^3/2(n). In Example 2.3 with σ_i = i^β for some β ∈ [0, 1), elementary but tedious calculations show that the optimal rate is

$${\mathrm{\chi }}_n\asymp \{\begin{array}{ccc}{n}^{-1/8}{\text{ log}}^{5/4}(n),\hfill & k_n\asymp n^{3/4}{\text{ log}}^{3/2}(n),\hfill & \beta \in [0,3/4],\hfill \\ n^{\frac{\beta -1}{5-4\beta }}{\{\text{log}(n)\}}^{\frac{8(1-\beta )}{5-4\beta }},\hfill & k_n\asymp n^{\frac{4.5-4\beta }{5-4\beta }}{\{\text{log}(n)\}}^{\frac{4}{5-4\beta }},\hfill & \beta \in (3/4,1).\hfill \end{array}$$

2.6. Some possible extensions

The self-normalization approaches in Sections 2.2–2.5 can be extended to linear regression model (1.4) with modulated stationary time series errors. The approach in Phillips, Sun and Jin (2007) is not applicable here due to non-stationarity. For simplicity, we consider the simple case that p = 2, U_i = (1, i/n), and β = (β₀, β₁)′. Hansen (1995) studied a similar setting for martingale difference errors. Denote by β̂₀ and β̂₁ the simple linear regression estimates of β₀ and β₁ given by

$${\hat{\beta }}_1=\frac{n{\displaystyle {\sum }_{i=1}^n}{\mathit{\text{iX}}}_i-{\displaystyle {\sum }_{i=1}^n}i{\displaystyle {\sum }_{i=1}^n}{X}_i}{{\displaystyle {\sum }_{i=1}^n}{i}^2-{\left({\displaystyle {\sum }_{i=1}^n}i\right)}^2/n}\text{ and \hspace{1em}}{\hat{\beta }}_0=\overline{X}-{\hat{\beta }}_1(n+1)/(2n).$$ (2.24)

Then simple algebra shows that

$${\hat{\beta }}_0-{\beta }_0=\frac{2}{n^2-n}{\displaystyle \sum _{i=1}^n}(2n-3i+1){\sigma }_i{e}_i,{\hat{\beta }}_1-{\beta }_1=\frac{6}{n^2-1}{\displaystyle \sum _{i=1}^n}(2i-n-1){\sigma }_i{e}_i.$$

The latter expressions are linear combinations of {e_i}. Thus, by the same argument in Proposition 2.2 and Theorem 2.1, we have self-normalized CLTs for β̂₀ and β̂₁.

Theorem 2.5. Let s_i,0 = (2n − 3i + 1)σ_i and s_i,₁ = (2i − n − 1)σ_i. Assume that {s_i,0}_1≤i≤n and {s_i,1}_1≤i≤n satisfy condition (2.10). Then as n → ∞,

$$\frac{n^2({\hat{\beta }}_0-{\beta }_0)}{2V_{n,0}}\Rightarrow N(0,{\tau }^2),\mathit{\text{ where }}{V}_{n,0}^2={\displaystyle \sum _{i=1}^n}{(2n-3i+1)}^2{(X_i-{\hat{\beta }}_0-{\hat{\beta }}_{1}i/n)}^2,$$

$$\frac{n^2({\hat{\beta }}_1-{\beta }_1)}{6V_{n,1}}\Rightarrow N(0,{\tau }^2),\mathit{\text{ where }}{V}_{n,1}^2={\displaystyle \sum _{i=1}^n}{(2i-n-1)}^2{(X_i-{\hat{\beta }}_0-{\hat{\beta }}_{1}i/n)}^2.$$

The long-run variance τ² can be estimated using the idea of blockwise self-normalization in Section 2.5. Let k_n, ℓ_n and ℐ_j be defined as in Section 2.5. Then we propose

$${\hat{\tau }}^2=\frac{1}{{\ell }_n}{\displaystyle \sum _{j=1}^{{\ell }_n}}{D}_j^2,\text{ where }{D}_j=\frac{{\displaystyle {\sum }_{i\in {ℐ}_j}}(X_i-{\hat{\beta }}_0-{\hat{\beta }}_{1}i/n)}{\sqrt{{\displaystyle {\sum }_{i\in {ℐ}_j}}{(X_i-{\hat{\beta }}_0-{\hat{\beta }}_{1}i/n)}^2}}.$$ (2.25)

Here, D₁, …, D_ℓn are asymptotically IID normal random variables with mean zero and variance τ². Consistency can be established under similar conditions as in Theorem 2.4.

For the general linear regression model (1.4), the linearly weighted average structure of linear regression estimates allows us to obtain self-normalized CLTs as in Theorem 2.5 under more complicated conditions. Also, it is possible to extend the proposed method to the nonparametric regression model with time-varying variances

$$X_i=f(i/n)+{\sigma }_i{e}_i,$$ (2.26)

where f(·) is a nonparametric time trend of interest. Nonparametric estimates, for example the Nadaraya-Watson estimate, are usually based on locally weighted observations. The latter feature allows us to derive similar self-normalized CLT. However, the change-point problem for (1.4) and (2.26) will be more challenging, and Aue et al. (2008a) studied (1.4) for uncorrelated errors with constant variance. Also, it is more difficult to address the bandwidth selection issues; see Altman (1990) for related contribution when σ_i ≡ 1. It remains a direction of future research to investigate (1.4) and (2.26).

3. Simulation study

3.1. Selection of block length k_n for τ̂

Recall that D₁, …, D_ℓn in (2.25) are asymptotically IID normal random variables. To get a sensible choice of the block length parameter k_n, we propose a simulation-based method by minimizing the empirical mean squared error (MSE).

1. Simulate n IID standard normal random variables Z₁, …, Z_n.

2. Based on Z₁, …, Z_n, obtain τ̂ with block length k.

3. Repeat (i)–(ii) many times, compute empirical MSE(k) as the average of realizations of (τ̂ − 1)², and find the optimal k by minimizing MSE(k).

We find that the optimal block length k is about 12 for n = 120, about 15 for n = 240, about 20 for n = 360, 600, and about 25 for n = 1200.

3.2. Empirical coverage probabilities

Let sample size n = 120. Recall e_i and σ_i in (1.1). For σ_i, consider four choices:

• A1 : σ_i = 0.21_i≤n/2 + 0.61_i>n/2,

• A2 : σ_i = 0.2{1 + cos²(i/n^4/5)},

• A3 : σ_i = 0.2 + 0.1 log(1 + |i − n/2|),

• A4 : σ_i = 0.3 + ϕ(i/60),

where ϕ is the standard normal density and 1 is the indicator function. The sequences A1–A4 exhibit different patterns, with a piecewise constancy for A1, a cosine shape for A2, a sharp change around time n/2 for A3, and a gradual downtrend for A4. Let ε_i be IID N(0,1). For e_i, we consider both linear and nonlinear processes:

• B1 : $e_i=\{{\eta }_i-E({\eta }_i)\}/\sqrt{\text{Var}({\eta }_i)},\text{ where }{\eta }_i=\theta |{\eta }_{i-1}|+\sqrt{1-{\theta }^2}{\epsilon }_i,|\theta |<1$.

• B2 : $e_i={\displaystyle \sum _{j=0}^{\mathrm{\infty }}}{a}_j{\epsilon }_{i-j},\text{ where }{a}_j=\frac{{(j+1)}^{-\beta }}{\sqrt{{\displaystyle {\sum }_{j=0}^{\mathrm{\infty }}}{(j+1)}^{-2\beta }}},\beta >1/2$.

For B1, by Wu (2007), (2.8) holds. By Andel, Netuka and Svara (1984), $E({\eta }_i)=\theta \sqrt{2/\pi }$ and Var(η_i) = 1 − 2θ²/π. To examine how the strength of dependence affects the performance, we consider θ = 0, 0.4, 0.8, representing independence, intermediate, and strong dependence, respectively. For B2 with β > 2, (2.2) holds with Δ_n = n^1/4 log(n), and we consider three cases β = 2.1, 3, 4. To assess the effect of block length k_n, three choices k_n = 8, 10, 12 are used. Thus, we consider all 72 combinations of {A1, A2, A3, A4} × {B1, θ = 0, 0.4, 0.8; B2, β = 2.1, 3, 4} × {k_n = 8, 10, 12}.

Without loss of generality we examine coverage probabilities based on 10³ realized confidence intervals for μ = 0 in (1.1). We compare our self-normalization-based confidence intervals to some stationarity-based methods. For (1.1), if we pretend that the error process {ẽ_i = σ_ie_i} is stationary, then we can use (1.2) to construct an asymptotic confidence interval for μ. Under stationarity, the long-run variance τ² of {ẽ_i} can be similarly estimated through the block method in Section 2.5 by using the non-normalized version $D_j=\sqrt{k_n}[\overline{X}(j)-\overline{X}]$ in (2.25); see Lahiri (2003). Thus, we compare two self-normalization-based methods and three stationarity-based alternatives: self-normalization-based confidence intervals through the asymptotic theory in Proposition 2.2 (SN) and the wild bootstrap (WB) in Section 2.3; stationarity-based confidence intervals through the asymptotic theory (1.2) (ST), non-overlapping block bootstrap (BB), and studentized non-overlapping block bootstrap (SBB) in Section 2.3. From the results in Table 1, we see that the coverage probabilities of the proposed self-normalization-based methods (columns SN and WB) are close to the nominal level 95% for almost all cases considered. By contrast, the stationarity-based methods (columns ST, BB and SBB) suffer from substantial undercoverage, especially when dependence is strong (θ = 0.8 in Table 1 (a) and β = 2.1 in Table 1 (b)). For the two self-normalization-based methods, WB slightly outperforms SN.

Table 1.

Coverage probabilities (in percentage) for μ in (1.1) with e_i from B1 [Table 1 (a)] and B2 [Table 1 (b)]. Nominal level is 95%. SN and WB denote self-normalization-based confidence intervals using asymptotic theory in Proposition 2.2 and the wild bootstrap procedure, respectively; ST, BB, SBB denote stationarity-based confidence intervals using asymptotic theory in (1.2), non-overlapping block bootstrap, and studentized non-overlapping block bootstrap, respectively.

(a): Model B1
θ	kn	σi	SN	WB	ST	BB	SBB	σi	SN	WB	ST	BB	SBB
0.0	8		98.0	94.7	93.1	92.2	92.8		96.6	95.2	92.3	92.5	92.5
	10	A1	98.2	95.0	92.6	92.4	92.2	A2	94.6	94.6	90.0	89.5	89.4
	12		98.1	95.6	91.7	91.4	91.1		92.1	93.7	89.7	89.5	89.6
	8		96.4	95.0	92.5	92.3	92.0		96.6	95.6	93.1	92.6	93.0
	10	A3	94.7	94.7	90.8	90.6	90.6	A4	95.1	95.1	91.4	91.3	91.3
	12		93.7	94.8	90.8	90.4	90.5		92.9	93.7	89.8	89.7	89.5
0.4	8		98.7	95.9	92.7	92.6	92.9		96.6	95.3	92.5	92.4	92.0
	10	A1	98.5	95.7	92.8	92.7	92.3	A2	95.4	95.4	91.6	91.1	91.6
	12		98.0	95.0	90.8	90.8	90.2		92.5	94.0	89.4	89.1	89.4
	8		96.6	95.2	91.7	91.7	91.6		95.4	94.1	90.8	90.9	90.6
	10	A3	95.3	95.5	91.5	91.3	91.5	A4	95.0	94.8	91.2	90.7	90.8
	12		93.1	94.6	90.2	89.9	89.9		94.1	95.1	90.3	89.8	90.1
0.8	8		97.9	94.6	87.8	86.8	87.3		96.1	94.7	87.2	87.3	87.0
	10	A1	97.6	95.5	87.3	87.0	86.7	A2	93.3	92.9	86.4	86.8	86.1
	12		97.3	94.0	85.8	85.5	85.1		92.6	93.4	86.5	86.4	86.4
	8		94.8	93.5	85.7	85.7	86.0		95.5	94.7	86.3	86.1	86.1
	10	A3	93.5	93.8	85.7	85.5	85.2	A4	95.3	95.1	88.5	88.3	88.5
	12		92.4	93.3	87.2	86.7	86.9		92.6	94.2	86.3	85.8	85.7

(b): Model B2
β	kn	σi	SN	WB	ST	BB	SBB	σi	SN	WB	ST	BB	SBB
4.0	8		97.6	94.9	91.8	91.4	91.9		95.9	94.2	91.9	92.0	91.1
	10	A1	97.7	93.2	88.9	88.1	88.3	A2	95.7	95.7	92.1	91.8	92.1
	12		97.9	95.5	90.7	90.2	90.0		93.3	94.6	90.0	89.9	89.7
	8		94.6	93.3	89.8	89.5	89.5		95.6	94.7	91.3	91.7	91.0
	10	A3	95.1	95.2	91.6	91.4	91.5	A4	95.4	95.9	92.8	92.2	93.0
	12		93.8	95.4	90.8	90.6	90.2		93.9	94.9	88.9	88.5	88.6
3.0	8		99.1	95.7	91.1	91.0	91.2		95.8	94.6	90.4	89.8	90.1
	10	A1	98.5	96.4	91.6	90.9	91.1	A2	95.6	95.2	92.1	91.9	91.5
	12		97.9	94.6	89.6	89.3	89.0		94.1	95.0	90.5	90.2	90.4
	8		95.9	94.6	92.0	91.9	91.7		96.0	94.5	90.6	90.4	90.3
	10	A3	94.3	94.4	90.0	89.9	89.8	A4	94.3	94.4	89.2	89.3	88.9
	12		93.2	94.5	88.9	88.6	88.7		93.1	94.1	89.6	88.9	88.8
2.1	8		97.1	92.5	86.2	86.2	85.5		95.7	93.8	88.9	89.0	88.7
	10	A1	97.6	94.7	89.2	88.9	88.6	A2	93.5	93.6	88.8	88.8	88.4
	12		97.2	95.1	87.9	87.5	87.7		92.6	93.9	88.0	87.6	87.7
	8		94.0	93.7	88.5	88.4	88.3		95.0	93.1	88.8	88.7	88.6
	10	A3	93.3	93.8	88.1	87.9	87.8	A4	94.1	94.2	89.1	88.8	89.1
	12		92.9	94.7	89.1	88.4	88.4		91.5	92.6	87.7	87.5	87.5

3.3. Size and power study

In (1.3), we use the same setting for σ_i and e_i as in Section 3.2. For mean μ_i, we consider μ_i = λ1_i>40, λ ≥ 0, and compare the test statistics $T_n^1,T_n^2$ in (2.13) and $T_n^{\text{SN}}$ in (2.16). First, we compare their size under the null with λ = 0. The critical value of $T_n^{\text{SN}}$ is obtained using the wild bootstrap in Section 2.4; for $T_n^1\text{ and }{T}_n^2$, their critical values are based on the block bootstrap in Section 2.3. In each case, we use 10³ bootstrap samples, nominal level 5%, and block length k_n = 10, and summarize the empirical sizes (under the null λ = 0) in Table 2 based on 10³ realizations. While $T_n^{\text{SN}}$ has size close to 5%, $T_n^1\text{ and }{T}_n^2$ tend to over-reject the null, and the false rejection probabilities can be three times the nominal level of 5%. Next, we compare the size-adjusted power. Instead of using the bootstrap methods to obtain critical values, we use 95% quantiles of 10⁴ realizations of the test statistics when data are simulated directly from the null model so that the empirical size is exactly 5%. Figure 1 presents the power curves for combinations {A1–A4} × {B1 with θ = 0.4; B2 with β = 3.0} with 10³ realizations each. For A1, $T_n^{\text{SN}}$ outperforms $T_n^1\text{ and }{T}_n^2$; for A2–A4, there is a moderate loss of power for $T_n^{\text{SN}}$. Overall, $T_n^{\text{SN}}$ has power comparable to other two tests. In practice, however, the null model is unknown, and when one turns to the bootstrap method to obtain the critical values, the usual CUSUM tests $T_n^1\text{ and }{T}_n^2$ will likely over-reject the null as shown in Table 2. In summary, with such small sample size and complicated time-varying variances structure, $T_n^{\text{SN}}$ along with the wild bootstrap method delivers reasonably good power and the size is close to nominal level.

Table 2.

Size (in percentage) comparison of $T_n^1\text{ and }{T}_n^2$ in (2.13) and $T_n^{\text{SN}}$ in (2.16), with sample size n = 120, nominal level 5%, and block length k_n = 10.

	Model B1				Model B2
σi	θ	$T_n^{\text{SN}}$	$T_n^1$	$T_n^2$	β	$T_n^{\text{SN}}$	$T_n^1$	$T_n^2$
	0.0	4.9	9.1	8.4	2.1	7.3	12.2	13.4
A1	0.4	4.7	9.4	9.6	3.0	4.7	8.6	9.2
	0.8	6.0	15.1	14.7	4.0	5.6	9.9	7.7
	0.0	5.7	8.2	6.1	2.1	5.8	9.5	8.6
A2	0.4	6.1	8.9	6.8	3.0	5.3	9.6	6.8
	0.8	7.3	12.6	9.3	4.0	4.2	7.5	4.2
	0.0	5.0	5.7	4.8	2.1	5.5	7.7	6.7
A3	0.4	5.3	6.9	5.4	3.0	5.8	6.1	4.9
	0.8	7.0	9.8	10.0	4.0	5.0	6.5	4.2
	0.0	5.4	8.4	6.0	2.1	6.9	8.8	7.1
A4	0.4	5.7	7.9	5.2	3.0	4.8	6.6	6.3
	0.8	7.2	11.1	9.2	4.0	5.3	6.2	5.8

Figure 1.

Size-adjusted power curves for $T_n^1$ (dashed curve) and $T_n^2$ (dotdash curve) in (2.13) and $T_n^{\text{SN}}$ (solid curve) in (2.16) as functions of change size λ (horizontal axis) with sample size n = 120 and block length k_n = 10. For (A1,B1)–(A4,B1), the error process {e_i} is from B1 with θ = 0.4; for (A1,B2)–(A4,B2), the error process {e_i} is from B2 with β = 3.0.

Finally, we point out that the proposed self-normalization-based methods are not robust to models with time-varying correlation structures. For example, consider the model e_i = 0.3e_i−1 + ε_i for 1 ≤ i ≤ 60 and e_i = 0.8e_i−1 + ε_i for 61 ≤ i ≤ n, where ε_i are IID N(0,1). With k_n = 10, the sizes (nominal level 5%) for the three tests $T_n^{\text{SN}},T_n^1,T_n^2$ are 0.154, 0.196, 0.223 for A1. Future research directions include (i) developing tests for change in the variance or covariance structure for (1.1) [See Inclán and Tiao (1994), Aue et al. (2009) and Berkes, Gombay and Horváth (2009) for related contributions]; and (ii) developing methods that are robust to changes in correlations.

4. Applications to two real data sets

4.1. Annual mean precipitation in Seoul during 1771–2000

The data set consists of annual mean precipitation rates in Seoul during 1771–2000; see Figure 2 for a plot. The mean levels seem to be different for the two time periods 1771–1880 and 1881–2000. Ha and Ha (2006) assumed the observations are IID under the null hypothesis. As shown in Figure 2, the variations change over time. Also, the autocorrelation function plot (not reported here) indicates strong dependence up to lag 18. Therefore, it is more reasonable to apply our self-normalization-based test that is tailored to deal with modulated stationary processes. With sample size n = 230, by the method in Section 3.1, the optimal block length is about 15. Based on 10⁵ bootstrap samples as described in Section 2.4, we obtain the corresponding p-values 0.016, 0.005, 0.045, 0.007, with block length k_n = 12, 14, 16, 18, respectively. For all choices of k_n, there is compelling evidence that a change-point occurred at year 1880. While our result is consistent with that of Ha and Ha (2006), our modulated stationary time series framework seems to be more reasonable. Denote by μ₁ and μ₂ the mean levels over pre-change and post-change time periods 1771–1880 and 1881–2000. For the two sub-periods with sample sizes 110 and 120, the optimal block length is about 12. With k_n = 12, applying the wild bootstrap in Section 2.3 with 10⁵ bootstrap samples, we obtain 95% confidence intervals [121.7, 161.3] for μ₁, [100.9, 114.3] for μ₂. For the difference μ₁ − μ₂, with optimal block length k_n = 15, the 95% wild bootstrap confidence interval is [19.6, 48.2]. Note that the latter confidence interval for μ₁ − μ₂ does not cover zero, which provides further evidence for μ₁ ≠ μ₂ and the existence of a change-point at year 1880.

Figure 2.

Annual mean precipitation in Seoul during 1771–2000.

4.2. Quarterly U.S. GNP growth rates during 1947–2002

The data set consists of quarterly U.S. Gross National Product (GNP) growth rates from the first quarter of 1947 to the third quarter of 2002; see Section 3.8 in Shumway and Stoffer (2006) for a stationary autoregressive model approach. However, the plot in Figure 3 suggests a non-stationary pattern: the variation becomes smaller after year 1985 whereas the mean level remains constant. Moreover, the stationarity test in Kwiatkowski et al. (1992) provides fairly strong evidence for non-stationarity with a p-value of 0.088. With the block length k_n = 12, 14, 16, 18, we obtain the corresponding p-values 0.853, 0.922, 0.903, 0.782, and hence there is no evidence to reject the null hypothesis of a constant mean μ. Based on k_n = 15, the 95% wild bootstrap confidence interval for μ is [0.66%, 1.00%]. To test whether there is a change-point in the variance, by the discussion in the last paragraph of Section 2.4, we consider X̃_i = (X_i − X̲_n)². With k_n = 12, 14, 16, 18, the corresponding p-values are 0.001, 0.006, 0.001, 0.010, indicating strong evidence for a change-point in the variance at year 1984. In summary, we conclude that there is no change-point in the mean level, but there is a change-point in the variance at year 1984.

Figure 3.

Quarterly U.S. GNP growth rates during 1947–2002.

Appendix: Proofs

Proof of Theorem 2.1. Let $r_j=|{\sigma }_j|+{\displaystyle {\sum }_{i=2}^j}|{\sigma }_i-{\sigma }_{i-1}|$. By the triangle inequality, we have r_j ≤ r_n. Recall S_i in (2.2). By the summation by parts formula, (2.6) follows via

$$F_j={\displaystyle \sum _{i=1}^j}{\sigma }_i(S_i-S_{i-1})={\sigma }_j{S}_j+{\displaystyle \sum _{i=1}^{j-1}}({\sigma }_i-{\sigma }_{i+1})S_i={\sigma }_j\tau B_j+{\displaystyle \sum _{i=1}^{j-1}}({\sigma }_i-{\sigma }_{i+1})\tau B_i+O_{\mathrm{a}.\mathrm{s}.}(r_n{\mathrm{\Delta }}_n)=\tau {\displaystyle \sum _{i=1}^j}{\sigma }_i(B_i-B_{i-1})+O_{\mathrm{a}.\mathrm{s}.}(r_n{\mathrm{\Delta }}_n).$$ (5.1)

By Kolmogorov’s maximal inequality for independent random variables, for δ > 0,

$$P\left\{\underset{1\le j\le n}{\text{max}}\left|{\displaystyle \sum _{i=1}^j}{\sigma }_i(B_i-B_{i-1})\right|\ge \delta {\mathrm{\Sigma }}_n\right\}\le {(\delta {\mathrm{\Sigma }}_n)}^{-2}E\left[{\left\{{\displaystyle \sum _{i=1}^n}{\sigma }_i(B_i-B_{i-1})\right\}}^2\right]={\delta }^{-2}.$$ (5.2)

Thus, by (5.1), max_1≤j≤n |F_j| = O_p(Σ_n + r_nΔ_n). Observe that

$${\underset{¯}{V}}_j^2-{\mathrm{\Sigma }}_j^2=W_j-F_j^2/j,\text{ where \hspace{1em}}{W}_j={\displaystyle \sum _{i=1}^j}{\sigma }_i^2(e_i^2-1).$$ (5.3)

By (2.2), the same argument in (5.1) and (5.2) shows $W_j=O_{\mathrm{p}}({\mathrm{\Sigma }}_n^{*2}+r_n^{*}{\mathrm{\Delta }}_n)$, uniformly. The desired result then follows via (5.3).

Proof of Theorem 2.2. Denote by Φ(x) the standard normal distribution function. By Proposition 2.2 and Slutsky’s theorem, ℙ(H_n/τ̂ ≤ x) → Φ(x) for each fixed x ∈ ℝ. Since Φ(x) is a continuous distribution, sup_x∈ℝ |ℙ(H_n/τ̂ ≤ x) − Φ(x)| = 0. It remains to prove ${\text{sup}}_{x\in ℝ}|{\mathbb{P}}^{*}(H_n^{*}\le x)-\mathrm{\Phi }(x)|\to 0$, in probability. Notice that, conditioning on {e_i}, {ξ_i} are independent random variables with zero mean. By the Berry-Esséen bound in Bentkus, Bloznelis and Götze (1996), there exists a finite constant c such that

$$\underset{x\in ℝ}{\text{sup}}|{\mathbb{P}}^{*}(H_n^{*}\le x)-\mathrm{\Phi }(x)|\le c{\displaystyle \sum _{i=1}^n}{E}^{*}({|{\xi }_i|}^3){\left\{{\displaystyle \sum _{i=1}^n}{E}^{*}({|{\xi }_i|}^2)\right\}}^{-3/2},$$ (5.4)

where E^* denotes conditional expectations given {e_i}. Clearly, $E^{*}({|{\xi }_i|}^2)={\sigma }_i^2{e}_i^{2}E({\alpha }_1^2)\text{ and }E({|{\xi }_i|}^3)={\sigma }_i^3|e_i^3|E(|{\alpha }_1^3|)$. Thus, under the assumption e_i ∈ ℒ³, we have ${\displaystyle {\sum }_{i=1}^n}{E}^{*}({|{\xi }_i|}^3)=O_{\mathrm{p}}({\displaystyle {\sum }_{i=1}^n}{\sigma }_i^3)$. Meanwhile, by the proof of Theorem 2.1, ${\displaystyle {\sum }_{i=1}^n}{E}^{*}(|{\xi }_i{|}^2)={\displaystyle {\sum }_{i=1}^n}{\sigma }_i^2{e}_i^2=\{1+o_{\mathrm{p}}(1)\}{\displaystyle {\sum }_{i=1}^n}{\sigma }_i^2$. Therefore, the desired result follows from (5.4) in view of (2.11).

Proof of Theorem 2.3. For cn ≤ j ≤ (1 − c)n, c ≤ (1 − j/n), j/n ≤ 1 − c. For S_X (j) in (2.14), by (2.6), we have max_{cn≤j≤(1−c)n} |S_X(j) − τ S̃_X (j)| = O_a.s.(r_nΔ_n), where

$${\tilde{S}}_X(j)=\left(1-\frac{j}{n}\right){\displaystyle \sum _{i=1}^j}{\sigma }_i(B_i-B_{i-1})-\frac{j}{n}{\displaystyle \sum _{i=j+1}^n}{\sigma }_i(B_i-B_{i-1}).$$

By (2.7), ${\text{max}}_{\mathit{\text{cn}}\le j\le (1-c)n}|{(1-j/n)}^2{\underset{¯}{V}}_j^2+{(j/n)}^2{\overline{V}}_j^2-V_j^2|=O_{\mathrm{p}}({\varpi }_n)$, where

$$V_j^2={(1-j/n)}^2{\displaystyle \sum _{i=1}^j}{\sigma }_i^2+{(j/n)}^2{\displaystyle \sum _{i=j+1}^n}{\sigma }_i^2\text{ and }{\varpi }_n=(r_n^2{\mathrm{\Delta }}_n^2+{\mathrm{\Sigma }}_n^2)/n+{\mathrm{\Sigma }}_n^{*2}+r_n^{*}{\mathrm{\Delta }}_n.$$

For cn ≤ j ≤ (1 − c)n, $V_j^2\ge c^2{\mathrm{\Sigma }}_n^2$. Thus, condition (2.10) implies ${\varpi }_n=o(V_j^2)\text{ and }{\{V_j^2+O_{\mathrm{p}}({\varpi }_n)\}}^{1/2}=V_j+O_{\mathrm{p}}({\varpi }_n/V_j)$ . Therefore, uniformly over cn ≤ j ≤ (1 − c)n,

$$T_n(j)-\tau {\tilde{T}}_n(j)=\frac{\tau {\tilde{S}}_X(j)+O_{\mathrm{a}.\mathrm{s}}(r_n{\mathrm{\Delta }}_n)}{V_j+O_{\mathrm{p}}({\varpi }_n/V_j)}-\frac{\tau {\tilde{S}}_X(j)}{V_j}=O_{\mathrm{p}}\left\{\frac{r_n{\mathrm{\Delta }}_n}{V_j}+\frac{{\varpi }_n{\tilde{S}}_X(j)}{V_j^3}\right\}.$$

By (5.2), max_j |S̃_X (j)| = O_p(Σ_n). Thus, the result follows in view of V_j ≥ cΣ_n.

Proof of Theorem 2.4. Condition M_n → 0 implies max_1≤j≤ℓn r(j)Δ_n/Σ(j) → 0. By (2.7),

$${\omega }_j≔\frac{V^2(j)}{{\mathrm{\Sigma }}^2(j)}-1=O_{\mathrm{p}}\left\{\frac{{\mathrm{\Sigma }}^{*2}(j)+r^{*}(j){\mathrm{\Delta }}_n}{{\mathrm{\Sigma }}^2(j)}+\frac{1}{k_n}\right\}=O_{\mathrm{p}}(M_n)\to 0.$$ (5.5)

Define U_j = Σ⁻¹(j) ∑_i∈ℐj σ_i(B_i − B_i−1). Clearly, U₁, …, U_ℓn are independent standard normal random variables. Thus, ${\text{max}}_{1\le j\le {\ell }_n}|U_j|=O_{\mathrm{p}}\{\sqrt{\text{log}({\ell }_n)}\}=O_{\mathrm{p}}\{\sqrt{\text{log}(n)}\}$. By (2.6), X̲_n − μ = O_p{(Σ_n + r_nΔ_n)/n} = O_p(Σ_n/n). Recall the definition of D_j in (2.18). By the same argument in (2.6), using $\sqrt{1+x}=1+O(x)$ as x → 0, we have

$$D_j=\frac{k_n\{\overline{X}(j)-\mu \}}{\mathrm{\Sigma }(j)}\frac{1}{\sqrt{1+{\omega }_j}}+\frac{k_n(\mu -{\underset{¯}{X}}_n)}{\mathrm{\Sigma }(j)}\frac{1}{\sqrt{1+{\omega }_j}}=\left[\tau U_j+O_{\mathrm{a}.\mathrm{s}}\left\{\frac{r(j){\mathrm{\Delta }}_n}{\mathrm{\Sigma }(j)}\right\}\right]\{1+O({\omega }_j)\}+O_{\mathrm{p}}\left\{\frac{k_n{\mathrm{\Sigma }}_n}{n\mathrm{\Sigma }(j)}\right\}=\tau U_j+O_{\mathrm{p}}\left\{\sqrt{\text{log}(n)}{M}_n+\frac{{\mathrm{\Sigma }}_n}{{\ell }_n\mathrm{\Sigma }(j)}\right\}.$$

By the latter expression and log(n)M_n → 0, we can easily verify τ̂² − τ² = O_p(χ_n).