Estimation of Dynamic Networks for High-Dimensional Nonstationary Time Series

Abstract

This paper is concerned with the estimation of time-varying networks for high-dimensional nonstationary time series. Two types of dynamic behaviors are considered: structural breaks (i.e., abrupt change points) and smooth changes. To simultaneously handle these two types of time-varying features, a two-step approach is proposed: multiple change point locations are first identified on the basis of comparing the difference between the localized averages on sample covariance matrices, and then graph supports are recovered on the basis of a kernelized time-varying constrained $L_1$-minimization for inverse matrix estimation (CLIME) estimator on each segment. We derive the rates of convergence for estimating the change points and precision matrices under mild moment and dependence conditions. In particular, we show that this two-step approach is consistent in estimating the change points and the piecewise smooth precision matrix function, under a certain high-dimensional scaling limit. The method is applied to the analysis of network structure of the S&P 500 index between 2003 and 2008.

1. Introduction

Networks are useful tools to visualize the relational information among a large number of variables. An undirected graphical model belongs to a rich class of statistical network models that encodes conditional independence [1]. Canonically, Gaussian graphical models (or their normalized version partial correlations [2]) can be represented by the inverse covariance matrix (i.e., the precision matrix), where a zero entry is associated with a missing edge between two vertices in the graph. Specifically, two vertices are not connected if and only if they are conditionally independent, given the value of all other variables.

On one hand, there is a large volume of literature on estimating the (static) precision matrix for graphical models in the high-dimensional setting, where the sample size and the dimension are both large [3,4,5,6,7,8,9,10,11,12,13,14,15,16]. Most of the earlier work along this line assumes that the underlying network is time-invariant. This assumption is quite restrictive in practice and hardly plausible for many real-world applications, such as gene regulatory networks, social networks, and stocking market, where the underlying data generating mechanisms are often dynamic. On the other hand, dynamic random networks have been extensively studied from the perspective of large random graphs, such as community detection and edge probability estimation for dynamic stochastic block models (DSBMs) [17,18,19,20,21,22,23,24,25,26,27,28,29,30]. Such approaches do not model the sampling distributions of the error (or noise), since the “true” networks are connected with random edges sampled from certain probability models, such as the Erdős–Rényi graphs [31] and random geometric graphs [32].

In this paper, we view the (time-varying) networks of interests as non-random graphs. We adopt the graph signal processing approach for denoising the nonstationary time series and target on estimating the true unknown underlying graphs. Despite the recent attempts towards more flexible time-varying models [33,34,35,36,37,38,39,40], there are still a number of major limitations in the current high-dimensional literature. First, theoretical analysis was derived under the fundamental assumption that the observations are either temporally independent, or the temporal dependence has very specific forms, such as Gaussian processes or (linear) vector autoregression (VAR) [14,33,34,37,41,42,43]. Such dynamic structures are unduly demanding in view that many time series encountered in real applications have very complex nonlinear spatial-temporal dependency [44,45]. Second, most existing work assumes the data have time-varying distributions with sufficiently light tails, such as Gaussian graphical models and Ising models [33,34,36,41,42]. Third, in change point estimation problems for high-dimensional time series, piecewise constancy is widely used [41,42,46,47], which can be fragile in practice. For instance, financial data often appears to have time-dependent cross-volatility with structural breaks [48]. For resting-state fMRI signals, correlation analysis reveals both slowly varying and abruptly changing characteristics corresponding to modularities in brain functional networks [49,50].

Advances in analyzing high-dimensional (stationary) time series have been made recently to address the aforementioned nonlinear spatial-temporal dependency issue [14,37,43,51,52,53,54,55,56,57]. In [53,56,57], the authors considered the theoretical properties of regularized estimation of covariance and precision matrices, based on various dependence measures of high-dimensional time series. Reference [38] considered the non-paranormal graphs that evolve with a random variable. Reference [37] discussed the joint estimation of Gaussian graphical models based on a stationary VAR(1) model with special coefficient matrices, which may also depend on certain covariates. The authors applied a constrained $L_1$-minimization for inverse matrix estimation (CLIME) estimator with a kernel estimator of covariance matrix and developed consistency in the graph recovery at a given time point. Reference [14] studied the recovery of the Granger causality across time and nodes assuming a stationary Gaussian VAR model with unknown order.

In this paper, we focus on the recovery of time-varying undirected graphs on the basis of the regularized estimation of the precision matrices for a general class of nonstationary time series. We simultaneously model two types of dynamics: abrupt changes with an unknown number of change points and the smooth evolution between the change points. In particular, we study a class of high-dimensional piecewise locally stationary processes in a general nonlinear temporal dependency framework, where the observations are allowed to have a finite polynomial moment.

More specifically, there are two main goals of this paper: first, to estimate the change point locations, as well as the number of change points, and second, to estimate the smooth precision matrix functions between the change points. Accordingly, our proposed method contains two steps. In the first step, the maximum norm of the local difference matrix is computed at each time point and the jumps in the covariance matrices are detected at the location where the maximum norms are above a certain threshold. In the second step, the precision matrices before and after the jump are estimated by a regularized kernel smoothing estimator. These two steps are recursively performed until a stopping criterion is met. Moreover, a boundary correction procedure based on data reflection is considered to reduce the bias near the change point.

We provide an asymptotic theory to justify the proposed method in high dimensions: point-wise and uniform rates of convergence are derived for the change point estimation and graph recovery under mild and interpretable conditions. The convergence rates are determined via subtle interplay among the sample size, dimensionality, temporal dependence, moment condition, and the choice of bandwidth in the kernel estimator. Our results are significantly more involved than problems for sub-Gaussian tails and independent samples. We highlight that uniform consistency in terms of time-varying network structure recovery is much more challenging and difficult than pointwise consistency. For the multiple change point detection problem, we also characterize the threshold of the difference statistic that gives a consistent selection of the number of change points.

We fix some notations: Positive, finite, and non-random constants, independent of the sample size n and dimension p, are denoted by $C,C_1,C_2,\cdots$, whose values may differ from line to line. For the sequence of real numbers, $a_n$ and $b_n$, we write $a_n=O\left(b_n\right)$ or $a_n\lesssim b_n$ if $lim{sup}_{n\to \infty }(a_n/b_n)\le C$ for some constant $C<\infty$ and $a_n=o\left(b_n\right)$ if ${lim}_{n\to \infty }(a_n/b_n)=0$. We say $a_n\asymp b_n$ if $a_n=O\left(b_n\right)$ and $b_n=O\left(a_n\right)$. For a sequence of random variables $Y_n$ and a corresponding set of constants $a_n$, denote $Y_n=O_{\mathbb{P}}\left(a_n\right)$ if for any $\epsilon >0$ there is a constant $C>0$ such that $\mathbb{P}(|Y_n|/a_n>C)<\epsilon$ for all n. For a vector $\mathbf{x}\in {\mathbb{R}}^p$, we write $|\mathbf{x}|={\left({\sum }_{j=1}^p{x}_j^2\right)}^{1/2}$. For a matrix $\Sigma$, ${|\Sigma |}_1={\sum }_{j,k}|{\sigma }_{jk}|$, ${|\Sigma |}_{\infty }={max}_{j,k}|{\sigma }_{jk}|$, ${|\Sigma |}_{L_1}={max}_k{\sum }_j|{\sigma }_{jk}|$, ${|\Sigma |}_F={\left({\sum }_{j,k}{\sigma }_{jk}^2\right)}^{1/2}$ and $\rho (\Sigma )=max\{|\Sigma \mathbf{x}|:|\mathbf{x}|=1\}$. For a random vector $\mathbf{z}\in {\mathbb{R}}^p$, write $\mathbf{z}\in {\mathcal{L}}^a$, $a>0$, if ${\parallel \mathbf{z}\parallel }_a={:\left[\mathbb{E}\right(|\mathbf{z}|}^a{\left)\right]}^{1/a}<\infty$. Let $∥\mathbf{z}∥={∥\mathbf{z}∥}_2$. Denote $a\wedge b=min(a,b)$ and $a\vee b=max(a,b)$.

The rest of the paper is organized as follows: Section 2 presents the time series model, as well as the main assumptions, which can simultaneously capture the smooth and abrupt changes. In Section 3, we introduce the two-step method that first segments the time series based on the difference between the localized averages on sample covariance matrices and then recovers the graph support based on a kernelized CLIME estimator. In Section 4, we state the main theoretical results for the change point estimation and support recovery. Simulation examples are presented in Section 5 and a real data application is given in Section 6. Proof of main results can be found in Section 7.

2. Time Series Model

We first introduce a class of causal vector stochastic processes. Next, we state the assumptions to derive an asymptotic theory in Section 4 and explain their implications. Let ${\epsilon }_i\in {\mathbb{R}}^p,i\in \mathbb{Z}$ be independent and identically distributed (i.i.d.) random vectors and ${\mathcal{F}}_i=(\dots ,{\epsilon }_{i-1},{\epsilon }_i)$ be a shift process. Let ${\mathbf{X}}_i^{\circ }\left(t\right)=(X_{i1}^{\circ }\left(t\right),\cdots ,X_{ip}^{\circ }\left(t\right))$ be a p-dimensional nonstationary time series generated by

$$\begin{array}{c}\hfill {\mathbf{X}}_i^{\circ }\left(t\right)=\mathbf{H}({\mathcal{F}}_i;t),\end{array}$$ (1)

where $\mathbf{H}(·;·)=(H_1(·;·),\dots ,H_p(·;·))$ is an ${\mathbb{R}}^p$-valued jointly measurable function. Suppose we observe the data points ${\mathbf{X}}_i={\mathbf{X}}_{i,n}={\mathbf{X}}_i^{\circ }\left(t_i\right)$ at the evenly spaced time intervals $t_i=i/n,i=1,2,\cdots ,n$,

$$\begin{array}{c}\hfill {\mathbf{X}}_{i,n}=\mathbf{H}({\mathcal{F}}_i;i/n).\end{array}$$ (2)

We drop the subscription n in ${\mathbf{X}}_{i,n}$ in the rest of this section. Since our focus is to study the second-order properties, the data is assumed to have a mean of zero.

Model (1) is first introduced in [58]. The stochastic process ${\left(X_i^{\circ }\left(t\right)\right)}_{i\in \mathbb{Z},t\in [0,1)}$ can be thought of as a triangular array system, double indexed by i and t, while the observations ${\left(X_i\right)}_{i=1}^n$ are sampled from the diagonal of the array. On one hand, when fixing the time index t, the (vertical) process ${\left(X_i^{\circ }\left(t\right)\right)}_{i\in \mathbb{Z}}$ is stationary. On the other hand, since $\mathbf{H}({\mathcal{F}}_i;t_i)$ is allowed to vary with $t_i$, the diagonal process (2) is able to capture nonstationarity.

The process ${\left({\mathbf{X}}_i\right)}_{i\in \mathbb{Z}}$ is causal or non-anticipative as ${\mathbf{X}}_i$ is an output of the past innovations ${\left({\epsilon }_j\right)}_{j\le i}$ and does not depend on future innovations. In fact, it covers a broad range of linear and nonlinear, stationary and non-stationary processes, such as vector auto-regressive moving average processes, locally stationary processes, Markov chains, and nonlinear functional processes  [53,58,59,60,61].

Motivated by real applications where nonstationary time series data can involve both abrupt breaks and smooth varies between the breaks, we model the underlying processes as piecewise locally stationary with a finite number of structural breaks.

Definition 1 (Piecewise locally stationary time series model).

Define ${\mathrm{PLS}}_{\iota }([0,1],L)$ as the collection of mean-zero piecewise locally stationary processes on $[0,1]$, if for each ${\left(X\left(t\right)\right)}_{0\le t\le 1}\in$${\mathrm{PLS}}_{\iota }([0,1],L)$, there is a nonnegative integer ι such that $X\left(t\right)$ is piecewise stochastic Lipschitz continuous in t with Lipschitz constant L on the interval $[t^{\left(l\right)},t^{(l+1)}),l=0,\cdots ,\iota$, where $0=t^{\left(0\right)}<t^{\left(1\right)}\cdots <t^{\left(\iota \right)}<t^{(\iota +1)}=1$. A vector stochastic process ${\left(\mathbf{X}\left(t\right)\right)}_{0\le t\le 1}\in {\mathrm{PLS}}_{\iota }([0,1],L)$ if all coordinates belong to ${\mathrm{PLS}}_{\iota }([0,1],L)$. For the process ${\left(X_0^{\circ }\left(t\right)\right)}_{0\le t\le 1}$ defined in (1), this means that there exists a non-negative integer ι and a constant $L>0$, such that

$$\underset{1\le j\le p}{max}∥H_j({\mathcal{F}}_0;t)-H_j({\mathcal{F}}_0;t^{\prime })∥\le L|t-t^{\prime }|for all{t}^{\left(l\right)}\le t,t^{\prime }<t^{(l+1)},0\le l\le \iota .$$

Remark 1.

If we assume ${\left({\mathbf{X}}_i^{\circ }\left(t\right)\right)}_{0\le t\le 1}\in {\mathrm{PLS}}_{\iota }([0,1],L),i\in \mathbb{Z}$, then it follows that for each $i^{\prime }=i-k,\dots ,i+k$, where $k/n\to 0$, and that $t^{\left(l\right)}\le i,i^{\prime }<t^{(l+1)}$ for some $0\le l\le \iota$, we have

$$\underset{1\le j\le p}{max}\parallel H_j({\mathcal{F}}_{i^{\prime }};i/n)-H_j({\mathcal{F}}_{i^{\prime }};i^{\prime }/n)\parallel \le Lk/n=o\left(1\right).$$

In other words, within a locally stationary time period, in a local window of i, ${\left(X_{i^{\prime }j}\right)}_{i-k\le i^{\prime }\le i+k}$ can be approximated by the stationary process ${\left(X_{i^{\prime }j}^{\circ }(i/n)\right)}_{i-k\le i^{\prime }\le i+k}$ for each $j=1,\dots ,p$. This justifies the terminology of local stationarity.

The covariance matrix function of the underlying process is $\Sigma \left(t\right)={\left({\sigma }_{jk}\left(t\right)\right)}_{1\le j,k\le p}$, $t\in [0,1]$, where ${\sigma }_{jk}\left(t\right)=\mathbb{E}(H_j({\mathcal{F}}_0;t)H_k({\mathcal{F}}_0;t))$, and the precision matrix function is $\Omega \left(t\right)=\Sigma {\left(t\right)}^{-1}={\left({\omega }_{jk}\left(t\right)\right)}_{1\le j,k\le p}$. The graph at time t is denoted by $G\left(t\right)=(\mathcal{V},\mathcal{E}(t\left)\right)$, where $\mathcal{V}$ is the vertex set and $\mathcal{E}\left(t\right)=\{(j,k):{\omega }_{jk}\left(t\right)\ne 0\}$. Note that ${\left({\mathbf{X}}_i^{\circ }\left(t\right)\right)}_t\in {\mathrm{PLS}}_{\iota }([0,1],L),i\in \mathbb{Z}$ implies piecewise Lipschitz continuity in $\Sigma \left(t\right)$ except at the breaks $t^{\left(1\right)},\cdots ,t^{\left(\iota \right)}$. In particular, if ${sup}_{0\le t\le 1}{max}_{1\le j\le p}∥H_j({\mathcal{F}}_0;t)∥\le C$ for some constant $C>0$, then

$$\begin{array}{ccc}\hfill {|\Sigma \left(s\right)-\Sigma \left(t\right)|}_{\infty }& \le & 2CL|s-t|,\forall s,t\in [t^{\left(l\right)},t^{(l+1)}),l=0,\dots ,\iota .\hfill \end{array}$$ (3)

The reverse direction is not necessarily true, i.e., (3) does not indicate ${\left({\mathbf{X}}_i^{\circ }\left(t\right)\right)}_t\in {\mathrm{PLS}}_{\iota }([0,1],L)$, $i\in \mathbb{Z}$ in general. As a trivial example, let ${\epsilon }_{ij}=2^{-1/2}$ with probability $2/3$ and $\sqrt{2}$ with probability $1/3$ i.i.d for all $i,j$. At time $t_k=k/n$, let $X_{ij}^{\circ }\left(t_k\right)={(-1)}^k{\sqrt{t}}_k{\epsilon }_{ij}$. Then for any k and $k^{\prime }$ such that $k+k^{\prime }$ is odd, $|\Sigma \left(t_k\right)-\Sigma \left(t_{k^{\prime }}\right){|}_{\infty }=|t_k-t_{k^{\prime }}|$, while $\parallel X_{01}^{\circ }\left(t_k\right)-X_{01}^{\circ }\left(t_{k^{\prime }}\right){\parallel }_2=\sqrt{t_k}+\sqrt{t_{k^{\prime }}}$.

Assumption 1 (Piecewise smoothness).

(i) Assume ${\left({\mathbf{X}}_i^{\circ }\left(t\right)\right)}_{0\le t\le 1}\in {\mathrm{PLS}}_{\iota }([0,1],L)$ for each $i\in \mathbb{Z}$, where $L>0$ and $\iota \ge 0$ are constants independent of n and p. (ii) For each $l=0,\dots ,\iota$, and $1\le j,k\le p$, we have ${\sigma }_{jk}\left(t\right)\in {\mathcal{C}}^2[t^{\left(l\right)},t^{(l+1)})$.

Now we introduce the temporal dependence measure. We quantify the dependence of ${\left({\mathbf{X}}_i^{\circ }\left(t\right)\right)}_{i\in \mathbb{Z}}$ by the dependence adjusted norm (DAN) (cf. [62]). Let ${\epsilon }_i^{\prime }$ be an independent copy of ${\epsilon }_i$ and ${\mathcal{F}}_{i,\left\{m\right\}}=(\dots ,{\epsilon }_{i-m-1},{\epsilon }_{i-m}^{\prime },{\epsilon }_{i-m+1},\dots ,{\epsilon }_i)$. Denote ${\mathbf{X}}_{i,\left\{m\right\}}^{\circ }\left(t\right)=\left(X_{i1,\left\{m\right\}}^{\circ }\left(t\right),\dots ,X_{ip,\left\{m\right\}}^{\circ }\left(t\right)\right)$, where $X_{ij,\left\{m\right\}}^{\circ }\left(t\right)=H_j({\mathcal{F}}_{i,\left\{m\right\}};t)$, $1\le j\le p$. Here ${\mathbf{X}}_{i,\left\{m\right\}}^{\circ }\left(t\right)$ is a coupled version of ${\mathbf{X}}_i^{\circ }\left(t\right)$, with the same generating mechanism and input, except that ${\epsilon }_{i-m}$ is replaced by an independent copy ${\epsilon }_{i-m}^{\prime }$.

Definition 2 (Dependence adjusted norm (DAN)).

Let constants $a\ge 1,A>0$. Assume ${sup}_{0\le t\le 1}{\parallel X_{1j}^{\circ }\left(t\right)\parallel }_a<\infty ,j=1,\dots ,p$. Define the uniform functional dependence measure for the sequences ${\left(X_{ij}^{\circ }\left(t\right)\right)}_{i\in \mathbb{Z},t\in [0,1]}$ of form (1) as

$${\theta }_{m,a,j}=\underset{0\le t\le 1}{sup}{\parallel X_{ij}^{\circ }\left(t\right)-X_{ij,\left\{m\right\}}^{\circ }\left(t\right)\parallel }_a,j=1,\dots ,p,$$

and ${\Theta }_{m,a,j}={\sum }_{i=m}^{\infty }{\theta }_{i,a,j}$. The dependence adjusted norm of ${\left(X_{ij}^{\circ }\left(t\right)\right)}_{i\in \mathbb{Z},t\in [0,1]}$ is defined as

$${∥X_{·,j}∥}_{a,A}=\underset{m\ge 0}{sup}{(m+1)}^A{\Theta }_{m,a,j},$$

whenever ${∥X_{·,j}∥}_{a,A}<\infty$.

Intuitively, the physical dependence measure quantifies the adjusted stochastic difference between the random variable and its coupled version by replacing past innovations. Indeed, ${\theta }_{m,a,j}$ measures the impact on $X_{ij}^{\circ }\left(t\right)$ uniform over t by replacing ${\mathbf{\epsilon }}_{i-m}$ while freezing all the other inputs, while ${\Theta }_{m,a,j}$ quantifies the cumulative influence of replacing ${\mathbf{\epsilon }}_{-m}$ on ${\left(X_{ij}^{\circ }\left(t\right)\right)}_{i\ge 0}$ uniform over t. Then ${∥X_{·,j}∥}_{a,A}$ controls the uniform polynomial decay in the lag of the cumulative physical dependence, where a depends on the the tail of marginal distributions of $X_{1,j}^{\circ }\left(t\right)$ and A quantifies the polynomial decay power and thus the temporal dependence strength. It is clear that ${∥X_{·,j}∥}_{a,A}$ is a semi-norm, i.e., it is subaddative and absolutely homogeneous.

Assumption 2 (Dependence and moment conditions).

Let ${\mathbf{X}}_i^{\circ }\left(t\right)$ be defined in (1) and ${\mathbf{X}}_i$ in (2). There exist $q>2$ and $A>0$ such that

$${\nu }_{2q}:=\underset{t\in [0,1]}{sup}\underset{1\le j\le p}{max}\mathbb{E}{|X_j^{\circ }\left(t\right)|}^{2q}<\infty \mathrm{and}{N}_{X,2q}:=\underset{1\le j\le p}{max}{∥X_{·,j}∥}_{2q,A}<\infty .$$ (4)

We let $M_{X,q}:={\left({\sum }_{1\le j\le p}{∥X_{·,j}∥}_{2q,A}^q\right)}^{1/q}$ and write $N_X=N_{X,4}$, $M_X=M_{X,2}$. The quantities $M_{X,q}$ and $N_{X,2q}$ measure the $L^q$-norm aggregated effect and the largest effect of the element-wise DANs respectively. Both quantities play a role in the convergence rates of our estimator.

Obviously, we have $\parallel X_{ij}-X_{ij,\left\{m\right\}}{\parallel }_a\le {\theta }_{m,a,j}$ and ${max}_{1\le j\le p}\mathbb{E}{|X_{ij}|}^{2q}\le {\nu }_{2q}$ for all $1\le i\le n$. In contrast to other works in a high-dimensional covariance matrix and network estimation, where sub-Gaussian tails and independence are the keys to ensure consistent estimation. Assumption 2 only requires that the time series have a finite polynomial moment, and it allows linear and nonlinear processes with short memory in the time domain.

Example 1 (Vector linear process).

Consider the following vector linear process model

$$\mathbf{H}({\mathcal{F}}_i;t)=\sum _{m=0}^{\infty }{A}_m\left(t\right){\mathbf{\epsilon }}_{i-m},$$

where ${\mathbf{\epsilon }}_i=({\epsilon }_1,\dots ,{\epsilon }_p)$ and ${\epsilon }_{ij}$ are i.i.d. with mean 0 and variance 1, and $\parallel {\epsilon }_{ij}{\parallel }_q\le C_q$ for each $i\in \mathbb{Z}$ and $1\le j\le p$ with some constants $q>2$ and $C_q>0$. The vector linear process is commonly seen in literature and application [63]. It includes the time-varying VAR model where $A_m\left(t\right)=A{\left(t\right)}^m$ as a special example.

Suppose that the coefficient matrices $A_m\left(t\right)={\left(a_{m,jk}\left(t\right)\right)}_{1\le j,k\le p},m=0,1,\dots$ satisfy the following condition.

• (A1)  For each $1\le j,k\le p$, $a_{m,jk}\left(t\right)\in {\mathcal{C}}^2[0,1].$
• (A2)  For each $1\le j\le p$, there is a constant $C_{A,j}>0$ such that for each $t\in [0,1]$, ${\sum }_{k=1}^p{a}_{m,jk}{\left(t\right)}^2\le C_{A,j}{(m+1)}^{-2(A+1)}$ for all $m\ge 0$.
• (A3)  For any $t,t^{\prime }\in [0,1]$, ${\sum }_{m=0}^{\infty }{\sum }_{k=1}^p{[a_{m,jk}\left(t\right)-a_{m,jk}\left(t^{\prime }\right)]}^2\le L^2{|t-t^{\prime }|}^2$ for each $j=1,\dots ,p$.

Note that

$$\begin{array}{cc}\hfill {\sigma }_{jk}\left(t\right)& =\sum _{m\ge 0}{A}_{m,j·}^{\top }\left(t\right)A_{m,k·}\left(t\right),\hfill \\ \hfill {\Theta }_{m,q,j}& \le 2C_q\sqrt{q-1}\sum _{m=0}^{\infty }{\left(A_{m,j·}^{\top }{A}_{m,j·}\right)}^{1/2},\hfill \\ \hfill \parallel X_{ij}^{\circ }\left(t\right)-X_{ij}^{\circ }\left(t^{\prime }\right){\parallel }^2& =\sum _{m=0}^{\infty }{A}_{m,j·}\sum _{k=1}^p{[a_{m,jk}\left(t\right)-a_{m,jk}\left(t^{\prime }\right)]}^2,\hfill \end{array}$$

where $A_{m,j·}\left(t\right)$ is the jth row of $A_m\left(t\right)$. Under conditions (A1)–(A3), one can easily verify that for each $1\le j,k\le p$, the process satisfies: (1) ${\sigma }_{jk}\left(t\right)\in {\mathcal{C}}^2[0,1]$; (2) $\parallel X_{·,j}{\parallel }_{q,A}\le C_q\sqrt{(q-1)C_{A,j}}$ (due to Burkholder’s inequality, cf. [64]); (3) $\parallel H_j({\mathcal{F}}_0;t)-H_j({\mathcal{F}}_0;t^{\prime })\parallel \le L|t-t^{\prime }|$.

Conditions (A1)–(A3) implicitly impose smoothness in each entry of the coefficient matrices, sparseness in each column of the entry and evolution, and polynomial decay rate in the lag m of each entry and its derivative.

For $1\le l\le \iota$, let ${\delta }_{jk}\left(t^{\left(l\right)}\right):={\sigma }_{jk}\left(t^{\left(l\right)}\right)-{\sigma }_{jk}(t^{\left(l\right)}-)$ and $\Delta \left(t^{\left(l\right)}\right)={\left({\delta }_{jk}\left(t^{\left(l\right)}\right)\right)}_{1\le j,k\le p}$, where ${\sigma }_{jk}(t^{\left(l\right)}-)={lim}_{t\to t^{\left(l\right)}-}{\sigma }_{jk}\left(t\right)$ is well-defined in view of (3). We assume that the change points are separated and sizeable.

Assumption 3 (Separability and sizeability of change points).

There exist positive constants $c_1\in (0,1)$ and $c_2>0$ independent of n and p such that ${max}_{0\le l\le \iota }(t^{(l+1)}-t^{\left(l\right)})\ge c_1$ and $\delta \left(t_l\right):={|\Delta \left(t_l\right)|}_{\infty }\ge c_2$.

In the high-dimensional context, we assume that the inverse covariance matrices are sparse in the sense of their $L_1$ norms.

Assumption 4 (Sparsity of precision matrices).

The precision matrix ${|\Omega \left(t\right)|}_{L^1}\le {\kappa }_p$ for each $t\in [0,1]$, where ${\kappa }_p$ is allowed to grow with p.

If we further assume that the eigenvalues of the covariance matrices are bounded from below and above, i.e., there exists a constant $0<c<1$, such that $c\le {inf}_{t\in [0,1]}{|\Sigma \left(t\right)|}_2\le {sup}_{t\in [0,1]}{|\Sigma \left(t\right)|}_2\le c^{-1}$, then the covariance matrices and precision matrices are well-conditioned. In particular, as $|\Omega \left(t\right)-\Omega \left(t^{\prime }\right)|\le c^{-2}|\Sigma \left(t\right)-\Sigma \left(t^{\prime }\right)|$, a small perturbation in the covariance matrix would guarantee a small change of the same order in the precision matrix under the spectral norm.

3. Method: Change Point Estimation and Support Recovery

In graphical models (such as the Gaussian graphical model or partial correlation graph), network structures relevant to correlations or partial correlations are second-order characteristics of the data distributions. Specifically, the existence of edges coincides with non-zero entries of the inverse covariance matrix. We consider the dynamics of time series with both structural breaks and smooth changes. The piecewise stochastic Lipschitz continuity in Definition 1 allows the time series to have discontinuity in the covariance matrix function at time points $t^{\left(l\right)},l=1,\dots ,\iota$ (i.e., change points), while only smooth changes (i.e., twice continuous differentiability of the covariance matrix function in Assumptions 1) can occur between the change points.

In the presence of change points, we must first remove the change points before applying any smoothing procedures since ${|\Omega \left(t\right)-\Omega (t-)|}_{\infty }\ge {|\Sigma \left(t\right)|}_{L^1}^{-1}{|\Sigma (t-)|}_{L^1}^{-1}{|\Delta \left(t\right)|}_{\infty }$, i.e., a non-negligible abrupt change in the covariance matrix will result in a substantial change of the graph structure for sparse and smooth covariance matrices. Thus our proposed graph recovery method consists of two steps: change point detection and support recovery.

Let $h\equiv h_n>0$ be a bandwidth parameter such that $h=o\left(1\right)$ and $n^{-1}=o\left(h\right)$, and ${\mathcal{D}}_h\left(0\right)=\{h,h+1/n,\dots ,1-h\}$ be a search grid in $(0,1)$. Define

$$D\left(s\right)=n^{-1}\left(\sum _{i=0}^{hn-1}{\mathbf{X}}_{ns-i}{\mathbf{X}}_{ns-i}^{\top }-\sum _{i=1}^{hn}{\mathbf{X}}_{ns+i}{\mathbf{X}}_{ns+i}^{\top }\right),s\in {\mathcal{D}}_h\left(0\right).$$ (5)

To estimate the change points, compute

$${\widehat{s}}_1={\mathrm{argmax}}_{s\in {\mathcal{D}}_h\left(0\right)}{|D\left(s\right)|}_{\infty }.$$ (6)

The following steps are performed recursively. For $l=1,2,\dots$, let

$$\begin{array}{cc}\hfill & {\mathcal{D}}_h\left(l\right)={\mathcal{D}}_h(l-1)\cap {\{{\widehat{s}}_l-2h,\cdots ,{\widehat{s}}_l+2h\}}^c,\end{array}$$ (7)

$$\begin{array}{c}{\widehat{s}}_{l+1}=arg{max}_{s\in {\mathcal{D}}_h\left(l\right)}{|D\left(s\right)|}_{\infty },\end{array}$$ (8)

until the following criterion is attained:

$$\begin{array}{c}\hfill \underset{s\in {\mathcal{D}}_h\left(l\right)}{max}{|D\left(s\right)|}_{\infty }<\nu ,\end{array}$$ (9)

where $\nu$ is an early stopping threshold. The value of $\nu$ is determined in Section 4, which depends on the dimension and sample size, as well as the serial dependence level, tail condition, and local smoothness. Since our method only utilizes data in the localized neighborhood, multiple change points can be estimated and ranked in a single pass, which offers some computational advantage than the binary segmentation algorithm [41,46].

Once the change points are claimed, in the second step, we consider recovering the networks from the locally stationary time series before and after the structural breaks. In [11], where $X_i,i=1,\dots ,n$ are assumed with an identical covariance matrix, the precision matrix $\widehat{\Omega }$ is estimated as,

$$\begin{array}{c}\hfill {\widehat{\Omega }}_{\lambda }=arg\underset{\Omega \in {\mathbb{R}}^{p\times p}}{min}{|\Omega |}_1\mathrm{s}.\mathrm{t}.{|\widehat{\Sigma }\Omega -{\mathrm{Id}}_p|}_{\infty }\le \lambda ,\end{array}$$ (10)

where $\widehat{\Sigma }$ is the sample covariance matrix. Inspired by (10), we apply a kernelized time-varying (tv-) CLIME estimator for the covariance matrix functions of the multiple pieces of locally stationary processes before and after the structural breaks. Let

$$\begin{array}{c}\hfill \widehat{\Sigma }\left(t\right)=\sum _{i=1}^{n}w(t,t_i){\mathbf{X}}_i{\mathbf{X}}_i^{\top },\end{array}$$ (11)

where

$$\begin{array}{c}\hfill w(t,i)=\frac{K_b(t_i,t)}{{\sum }_{i=1}^n{K}_b(t_i,t)}\end{array}$$ (12)

and $K_b(u,v)=K(|u-v|/b)/b$. The bandwidth parameter b satisfies that $b=o\left(1\right)$ and $n^{-1}=o\left(b\right)$. Denote $B_n=nb$. The kernel function $K(·)$ is chosen to have properties as follows.

Assumption 5 (Regularity of kernel function).

The kernel function $K(·)$ is non-negative, symmetric, and Lipschitz continuous with bounded support in $[-1,1]$, and that ${\int }_{-1}^{1}K\left(u\right)du=1$.

Assumption 5 is a common requirement on the kernel functions and can be fulfilled by a range of kernel functions, such as the uniform kernel, triangular kernel, and the Epanechnikov kernel. Now the tv-CLIME estimator of the precision matrix $\Omega \left(t\right)$ is defined by $\tilde{\Omega }\left(t\right)={\left({\tilde{\omega }}_{jk}\left(t\right)\right)}_{1\le j,k\le p}$, where ${\tilde{\omega }}_{jk}\left(t\right)=min({\widehat{\omega }}_{jk}\left(t\right),{\widehat{\omega }}_{kj}\left(t\right))$, and $\widehat{\Omega }\left(t\right)\equiv {\widehat{\Omega }}_{\lambda }\left(t\right)={\left({\widehat{\omega }}_{jk}\left(t\right)\right)}_{1\le j,k\le p}$,

$$\begin{array}{c}\hfill {\widehat{\Omega }}_{\lambda }\left(t\right)=arg\underset{\Omega \in {\mathbb{R}}^{p\times p}}{min}{|\Omega |}_1\mathrm{s}.\mathrm{t}.{|\widehat{\Sigma }\left(t\right)\Omega -{\mathrm{Id}}_p|}_{\infty }\le \lambda .\end{array}$$ (13)

Similar hybridized kernel smoothing and the CLIME method for estimating the sparse and smooth transition matrices in high-dimensional VAR model has been considered in [65], where change point is not considered. Thus in the current setting we need to carefully control effect of (consistently) removing the change points before smoothing.

Then, the network is estimated by the “effective support” defined as follows.

$$\begin{array}{c}\hfill \widehat{G}(t;u)={\left({\widehat{g}}_{jk}(t;u)\right)}_{1\le j,k\le p},\mathrm{where}{\widehat{g}}_{jk}(t;u)=\mathbb{I}\left\{|{\tilde{\omega }}_{jk}\left(t\right)|\ge u\right\}.\end{array}$$ (14)

It should be noted that the (vanilla) kernel smoothing estimator (11) of the covariance matrix does not adjust for the boundary effect due to the change points in the covariance matrice function. Thus, in the neighborhood of the change points, a larger bias can be induced in estimating $\Sigma \left(t\right)$ by $\widehat{\Sigma }\left(t\right)$. As a remedy, we apply the following reflection procedure for boundary correction. Suppose $t\in {\widehat{\mathcal{T}}}_{b+h^2}\left(j\right)$ for $1\le j\le \iota$, Denote ${\widehat{\mathcal{T}}}_d\left(j\right):=[{\widehat{s}}_j-d,{\widehat{s}}_j+d)$ for $d\in (0,1)$. We replace (11) by

$$\widehat{\Sigma }\left(t\right)=\sum _{i=1}^{n}w(t,t_i){\breve{\mathbf{x}}}_i{\breve{\mathbf{x}}}_i^{\top },$$

and then apply the rest of the tv-CLIME approach. Here

$$\begin{array}{c}\hfill {\breve{\mathbf{x}}}_i=\left\{\begin{array}{cc}{\mathbf{x}}_i\hfill & \mathrm{if} (i-{\widehat{s}}_{j}n)(t-{\widehat{s}}_{j}n)\ge 0;\hfill \\ {\mathbf{x}}_{2{\widehat{s}}_{j}n-i}\hfill & \mathrm{otherwise}.\hfill \end{array}\right.\end{array}$$ (15)

4. Theoretical Results

In this section, we derive the theoretical guarantees for the change point estimation and graph support recovery. Roughly speaking, Proposition 1 and 2 below show that under appropriate conditions, if each element of the covariance matrix varies smoothly in time, one can obtain an accurate snapshot estimation of the precision matrices as well as the time-varying graphs with high probability via the proposed kernel smoothed constrained $l_1$ minimization approach.

Define $J_{q,A}(n,p)=M_{X,q}{\left(p{\varpi }_{q,A}\left(n\right)\right)}^{1/q}$, where ${\varpi }_{q,A}\left(n\right)=n,n{(logn)}^{1+2q},n^{q/2-Aq}$ if $A>1/2-1/q$, $A=1/2-1/q$, and $0<A<1/2-1/q$, respectively.

Proposition 1 (Rate of convergence for estimating precision matrices: pointwise and uniform).

Suppose Assumptions 2, 4, and 5 hold with $\iota =0$. Let $B_n=bn$ for $n^{-1}=o\left(b\right)$ and $b=o\left(1\right)$.

• (i) 
  Pointwise. Choose the parameter ${\lambda }^{\circ }\ge C{\kappa }_p(b^2+B_n^{-1}{J}_{q,A}(B_n,p)+N_X{(logp/B_n)}^{1/2})$ in the tv-CLIME estimator ${\widehat{\Omega }}_{{\lambda }^{\circ }}\left(t\right)$ in (13), where C is a sufficiently large constant independent of n and p. Then for any $t\in [b,1-b]$, we have

  $$\begin{array}{cc}\hfill |{\widehat{\Omega }}_{{\lambda }^{\circ }}{\left(t\right)-\Omega \left(t\right)|}_{\infty }& =O_{\mathbb{P}}\left({\kappa }_p{\lambda }^{\circ }\right).\hfill \end{array}$$ (16)
• (ii) 
  Uniform. Choose ${\lambda }^{\diamond }\ge C{\kappa }_p\left(b^2+B_n^{-1}{J}_{q,A}(n,p)+N_X{B}_n^{-1}{(nlog\left(p\right))}^{1/2}\right)$ in the tv-CLIME estimator ${\widehat{\Omega }}_{{\lambda }^{\circ }}\left(t\right)$ in (13), where C is a sufficiently large constant independent of n and p. Then we have

  $$\begin{array}{c}\hfill \underset{t\in [b,1-b]}{sup}{|{\widehat{\Omega }}_{{\lambda }^{\diamond }}\left(t\right)-\Omega \left(t\right)|}_{\infty }=O_{\mathbb{P}}\left({\kappa }_p{\lambda }^{\diamond }\right).\end{array}$$ (17)

The optimal order of the bandwidth parameter $b=b_{♯}$ in (17) is the solution to the following equation:

$$\begin{array}{ccc}\hfill b^2& =& B_n^{-1}max(J_{q,A}(n,p),N_X{(nlog\left(p^2\right))}^{1/2}),\hfill \end{array}$$

which implies that the closed-form expression for $b_{♯}$ is given by

$$\begin{array}{c}\hfill b_{♯}=C_1{\left(n^{-1}{J}_{q,A}(n,p)\right)}^{1/3}+C_2{N}_X^{1/3}{n}^{-1/6}log{\left(p\right)}^{1/6}\end{array}$$

for some constants $C_1$ and $C_2$ that are independent of n and p.

Given a finite sample, to distinguish the small entries in the precision matrix from the noise is challenging. Since a smaller magnitude of a certain element of the precision matrix implies a weaker connection of the edge in the graphical model, we instead consider the estimation of significant edges in the graph. Define the set of significant edges at level u as ${\mathcal{E}}^{*}(t;u)=\left\{(j,k):g_{jk}^{*}(t;u)\ne 0\right\}$, where

$$g_{jk}^{*}(t;u)=\mathbb{I}\left\{|{\omega }_{jk}\left(t\right)|>u\right\}.$$

Then, as a consequence of (17), we have the following support recovery consistency result.

Proposition 2 (Consistency of support recovery: significant edges).

Choose u as $u_{♯}=C_0{\kappa }_p^2{b}_{♯}^2$, where $C_0$ is taken as a sufficiently large constant independent of n and p. Suppose that $u_{♯}=o\left(1\right)$ as $n,p\to \infty$. Then under conditions of Proposition 1, we have that as $n,p\to \infty$,

$$\begin{array}{c}\hfill \mathbb{P}\left(\underset{t\in [b,1-b]}{sup}\sum _{(j,k)\in {\mathcal{E}}^c\left(t\right)}\mathbb{I}\left\{{\widehat{g}}_{jk}(t;u_{♯})\ne 0\right\}\ne 0\right)\to 0,\end{array}$$ (18)

$$\begin{array}{c}\hfill \mathbb{P}\left(\underset{t\in [b,1-b]}{sup}\sum _{(j,k)\in {\mathcal{E}}^{*}(t;2u_{♯})}\mathbb{I}\left\{{\widehat{g}}_{jk}(t;u_{♯})=0\right\}\ne 0\right)\to 0.\end{array}$$ (19)

Proposition 2 shows that the pattern of significant edges in the time-varying true graphs $G\left(t\right),t\in [b,1-b]$, can be correctly recovered with high probability. However, it is still an open question to what extent the edges with magnitude below u can be consistently estimated, which can be naturally studied in the multiple hypothesis testing framework. Nonetheless, hypothesis testing for graphical models on the nonstationary high-dimensional time series is rather challenging. We leave it as a future problem.

Propositions 1 and 2 together yield that the consistent estimation of the precision matrices and the graphs can be achieved before and after the change points. Now, we provide the theoretical result of the change point estimation. Theorem 1 below shows that if the change points are separated and sizable, then we can consistently identify them via the single pass segmentation approach under suitable conditions. Denote

$$h_{\diamond }=C_1{\left(n^{-1}{J}_{q,A}(n,p)\right)}^{1/3}+C_2{N}_X^{1/3}{n}^{-1/6}log{\left(p\right)}^{1/6},$$

where $C_1$ and $C_2$ are constants independent of n and p.

Theorem 1 (Consistency of change point estimation).

Assume ${\mathbf{X}}_i\in {\mathbb{R}}^p$ admits the form (2). Suppose that Assumptions 2 to 3 are satisfied. Choose the bandwidth $h=h_{\diamond }$, and $\nu =(1+L)h_{\diamond }^2$ in (5) and (9) respectively. Assume that $h_{\diamond }=o\left(1\right)$ as $n,p\to \infty$. We find that there exist constants $C_1,C_2,C_3$ independent of n and p, such that

$$\begin{array}{c}\hfill \mathbb{P}(|\widehat{\iota }-\iota |>0)\le C_1{\left(\frac{p{\varpi }_{q,A}\left(n\right)M_{X,q}^q{\nu }_{2q}^q}{n^q{c}_2^q}\right)}^{1/3}+C_2{p}^{2}exp\left\{-C_3{\left(\frac{n{log}^2\left(p\right)}{N_X^2}\right)}^{1/3}\right\}.\end{array}$$ (20)

Furthermore, in the event $\{\iota =\widehat{\iota }\}$, the ordered change-point estimator $({\widehat{s}}_{\left(1\right)}<{\widehat{s}}_{\left(2\right)}<\cdots <{\widehat{s}}_{\left(\widehat{\iota }\right)})$ defined in (7) satisfies

$$\begin{array}{cc}\hfill & \underset{1\le j\le \iota }{max}|{\widehat{s}}_{\left(j\right)}-t^{\left(j\right)}|=O_{\mathbb{P}}\left(h_{\diamond }^2\right).\end{array}$$ (21)

Proposition 2 and Theorem 1 together indicate the consistency in the snapshot estimation of the time-varying graphs before and after the change points. In a close neighborhood of the change points, we have the following result for the recovery of the time-varying network. Denote $\mathcal{S}:=\left[b_{♯},1-b_{♯}]\cap ({\cup }_{1\le j\le \widehat{\iota }}{\widehat{\mathcal{T}}}_{h_{\diamond }^2+b_{♯}}^c\left(j\right)\right)$ as the time intervals between the estimated change points, and $\mathcal{N}:=[0,b_{♯})\cup \left({\cup }_{1\le j\le \widehat{\iota }}({\widehat{\mathcal{T}}}_{h_{\diamond }^2+b_{♯}}\cap {\widehat{\mathcal{T}}}_{h_{\diamond }^2}^c)\right)\cup (1-b_{♯},1]$ as the recoverable neighborhood of the jump.

Theorem 2.

Let Assumptions 2 to 5 be satisfied. We have the following results as $n,p\to \infty$.

• (i) 
  Between change points. For $t\in \mathcal{S}$, take $b=b_{♯}$ and $u=u_{♯}$, where $b_{♯}$ and $u_{♯}$ are defined in Proposition 2. Suppose $u_{♯}=o\left(1\right)$. We have

  $$\begin{array}{cc}\hfill & \underset{t\in \mathcal{S}}{sup}\underset{j,k}{max}|{\widehat{\sigma }}_{j,k}\left(t\right)-{\sigma }_{j,k}\left(t\right)|=O_{\mathbb{P}}\left(b_{♯}^2\right).\hfill \end{array}$$ (22)
  Choose the penalty parameter as ${\lambda }_{♯}:=C_1{\kappa }_p{b}_{♯}^2$, where $C_1$ is a constant independent of n and p. Then

  $$\begin{array}{c}\hfill \underset{t\in \mathcal{S}}{sup}{|{\widehat{\Omega }}_{{\lambda }_{♯}}\left(t\right)-\Omega \left(t\right)|}_{\infty }=O_{\mathbb{P}}\left({\kappa }_p^2{b}_{♯}^2\right).\end{array}$$
  Moreover,

  $$\begin{array}{c}\hfill \mathbb{P}\left(\underset{t\in \mathcal{S}}{sup}\sum _{(j,k)\in {\mathcal{E}}^c\left(t\right)}\mathbb{I}\left\{{\widehat{g}}_{j,k}(t;u_{♯})\ne 0\right\}=0\right)\to 1,\end{array}$$ (23)

  $$\begin{array}{c}\hfill \mathbb{P}\left(\underset{t\in \mathcal{S}}{sup}\sum _{(j,k)\in {\mathcal{E}}^{*}(t;2u_{♯})}\mathbb{I}\left\{{\widehat{g}}_{jk}(t;u_{♯})=0\right\}=0\right)\to 1.\end{array}$$ (24)
• (ii) 
  Around change points. For $s\in \mathcal{N}$, take $b=b_{\star }:=C_1{\left(n^{-1}{J}_{q,A}(n,p)\right)}^{1/2}+C_2{N}_X^{1/2}{n}^{-1/4}log{\left(p\right)}^{1/4}$, and $u=u_{\star }:=C_0{\kappa }_p^2{b}_{\star }$, where $C_0$, $C_1$ and $C_2$ are constants independent of n and p. Suppose $u_{\star }=o\left(1\right)$. We have

  $$\begin{array}{cc}\hfill & \underset{t\in \mathcal{N}}{sup}\underset{j,k}{max}|{\widehat{\sigma }}_{j,k}\left(t\right)-{\sigma }_{j,k}\left(t\right)|=O_{\mathbb{P}}\left(b_{\star }\right).\hfill \end{array}$$
  Choose the penalty parameter as ${\lambda }_{\star }:=C_1{\kappa }_p{b}_{\star }$, where $C_1$ is a constant independent of n and p. Then

  $$\begin{array}{c}\hfill \underset{t\in \mathcal{N}}{sup}{|{\widehat{\Omega }}_{{\lambda }_{\star }}\left(t\right)-\Omega \left(t\right)|}_{\infty }=O_{\mathbb{P}}\left({\kappa }_p^2{b}_{\star }\right).\end{array}$$ (25)
  Moreover,

  $$\begin{array}{c}\hfill \mathbb{P}\left(\underset{t\in \mathcal{N}}{sup}\sum _{(j,k)\in {\mathcal{E}}^c\left(t\right)}\mathbb{I}\left\{{\widehat{g}}_{j,k}(t;u_{\star })\ne 0\right\}=0\right)\to 1,\end{array}$$ (26)

  $$\begin{array}{c}\hfill \mathbb{P}\left(\underset{t\in \mathcal{N}}{sup}\sum _{(j,k)\in {\mathcal{E}}^{*}(t;2u_{\star })}\mathbb{I}\left\{{\widehat{g}}_{j,k}(t;u_{\star })=0\right\}=0\right)\to 1.\end{array}$$ (27)

Note that the convergence rates for the covariance matrix entries and precision matrix entries in case (ii) around the jump locations are slower than those for points well separated from the jump locations in case (i). This is because on the boundary due to the reflection, the smooth condition may no longer hold true. Indeed, we only take advantage of the Lipschitz continuous property of the covariance matrix function. Thus, we lose one degree of regularity in the covariance matrix function, and the bias term $b^2$ in the convergence rate of the between-jump area becomes b around the jumps. We also note that around the smaller neighborhood of the jump $\mathcal{J}:={\cup }_{1\le j\le \widehat{\iota }}{\widehat{\mathcal{T}}}_{h_{\diamond }^2}$, due to the larger error in the change point estimation, consistent recovery of the graphs is not achievable.

5. A Simulation Study

We simulate data from the following multivariate time series model:

$$X_i=\sum _{m=0}^{100}{A}_m\left(i\right){ϵ}_{i-m},i=1,\dots ,n,$$

where $A_m\left(i\right)\in {\mathbb{R}}^{p\times p},1\le m\le 100,1\le i\le n$, and ${ϵ}_{i-m}={({ϵ}_{i-m,1},\dots ,{ϵ}_{i-m,p})}^{\top }$, with ${ϵ}_{m,k}$, $m\in \mathbb{Z}$, $j=1,\dots ,p$ generated as i.i.d. standardized $T\left(8\right)$ random variables. In the simulation, we fix $n=1000$ and vary $p=50$ and $p=100$. For each $m=1,\dots ,100$, the coefficient matrices $A_m\left(i\right)={(1+m)}^{-\beta }{B}_m\left(i\right)$, where $\beta =1$, and $B_m\left(1\right)$ is an $R^{p\times p}$ block diagonal matrix. The $5\times 5$ diagonal blocks in $B_m\left(i\right)$ are fixed with i.i.d. $N(0,1)$ entries and all the other entries are 0.

We consider the number of abrupt changes is $\iota =2$ and $(n{t}^{\left(1\right)},n{t}^{\left(2\right)})=(300,650)$. The matrix $A_0\left(i\right)$ is set to be a zero matrix for $i=1,2,\dots ,299$, while $A_0\left(i\right)=A_0\left(299\right)+\mathbf{\alpha }{\mathbf{\alpha }}^{\top }$, $i=300,301,\dots ,649$, and $A_0\left(i\right)=A_0\left(649\right)-\mathbf{\alpha }{\mathbf{\alpha }}^{\top }$, $i=650,651,\dots ,1000$, where the first 20 entries in $\mathbf{\alpha }$ are taken to be a constant ${\delta }_0$ and the others are 0.

We let the coefficient matrices $A_1\left(i\right)={\left\{a_{m,jk}\left(i\right)\right\}}_{1\le j,k\le p}$ evolve at each time point, such that two entries are soft-thresholded and another two elements increase. Specifically, at time i, we randomly select two elements from the support of $A_1\left(i\right)$, which are denoted as $\left\{a_{1,j_l^{\star }{k}_l^{\star }}\left(i\right)\right\},l=1,2$ and that $a_{1,j^{\star }{k}^{\star }}\left(i\right)\ne 0$, and set them to $a_{1,j_l^{\star }{k}_l^{\star }}^{\star }\left(i\right)=\mathrm{sign}\left(a_{1,j_l^{\star }{k}_l^{\star }}\left(i\right)\right)(|a_{1,j_l^{\star }{k}_l^{\star }}\left(i\right)-0.05|)$. We also randomly select two elements from $A_1^{\star }\left(i\right)$ and increase their values by $0.03$.

Figure 1 and Figure 2 show the support of the true covariance matrices at $i=100,200,\dots ,900$.

Figure 1.

Support of the true covariance matrices, $p=50$.

Figure 2.

Support of the true covariance matrices, $p=100$.

In detecting the change points, the cutoff value $\nu$ of detection is chosen as follows. After removing the neighborhood of detected change points, we obtain ${\mathcal{D}}_h^{\left(l\right)}$ by ordering ${\mathcal{D}}_h^{\left(l\right)},\dots {\mathcal{D}}_h^{\left(\mathfrak{l}\right)}$, where $\mathfrak{l}$ is obtained from (9) with $\nu =0$. For $l=1,2,\dots ,\mathfrak{l}-1$, compute

$${\mathcal{R}}_h^{\left(l\right)}=\frac{{\mathcal{D}}_h^{\left(l\right)}}{{\mathcal{D}}_h^{(l+1)}}.$$

We let $\widehat{\iota }=arg{max}_{0\le l\le \mathfrak{l}-1}{\mathcal{R}}_h^{\left(l\right)}$ and set $\nu ={\mathcal{D}}_h^{\left(\widehat{\iota }\right)}$.

We report the number of estimated jumps and the average absolute estimation error, where the average absolute estimation error is the mean of the distance between the estimated change points and the true change points. As is shown in Table 1 and Table 2, there is an apparent improvement in the estimation accuracy as the jump magnitude increases and dimension decreases. The detection is relatively robust to the choice of bandwidth.

Table 1.

Average distance.

	Bandwidth	0.14	0.16	0.18	0.2	0.22	0.24
$p=50$	${\delta }_0=1$	23.4	21.0	17.47	16.6	14.7	16.5
	${\delta }_0=2$	7.4	6.9	8.3	8.1	7.2	6.3
$p=100$	${\delta }_0=1$	37.2	30.1	26.4	25.5	21.2	21.3
	${\delta }_0=2$	7.8	8.2	9.9	6.9	8.9	7.6

Table 2.

Number of estimated change points.

	Bandwidth	0.14	0.16	0.18	0.2	0.22	0.24
$p=50$	${\delta }_0=1$	2.38	2.16	1.99	2.00	2.00	2.00
	${\delta }_0=2$	2.46	2.31	2.00	2.00	2.00	2.00
$p=100$	${\delta }_0=1$	2.25	2.09	1.99	1.99	2.00	2.00
	${\delta }_0=2$	2.38	2.19	2.00	2.00	2.00	2.00

We evaluate the support recovery performance of the time-varying CLIME at the lattice $100,200,\dots ,900$ with $\lambda =0.02,0.06,0.1$. We take the uniform kernel function and the bandwidth is fixed as $0.2$. At each time point $t_0$, two quantities are computed: sensitivity and specificity, which are defined as:

$$\begin{array}{cc}\hfill \mathrm{sensitivity}& =\frac{{\sum }_{1\le j,k\le p}\mathbb{I}\{{\widehat{g}}_{jk}(t_0;u)\ne 0,g_{jk}(t_0;u)\ne 0\}}{{\sum }_{1\le j,k\le p}\mathbb{I}\{g_{jk}(t_0;u)\ne 0\}},\hfill \\ \hfill \mathrm{specificity}& =\frac{{\sum }_{1\le j,k\le p}\mathbb{I}\{{\widehat{g}}_{jk}(t_0;u)=0,g_{jk}(t_0;u)=0\}}{{\sum }_{1\le j,k\le p}\mathbb{I}\{g_{jk}(t_0;u)=0\}}.\hfill \end{array}$$

We plot the Receiver Operating Characteristic (ROC) curve, that is, sensitivity against 1-specificity. From Figure 3 and Figure 4 we observe that, due to a screening step, the support recovery is robust to the choice of $\lambda$, except at the change points, where a non-negligible estimation error of the covariance matrix is induced and the overall estimation is less accurate. As the effective dimension of the network remains the same at $p=50$ and $p=100$ by the construction of the coefficient matrix $A_m\left(i\right)$, there is no significant difference in the ROC curves at different dimensions.

Figure 3.

ROC curve of the time-varying CLIME, $p=50$.

Figure 4.

ROC curve of the time-varying CLIME, $p=100$.

6. A Real Data Application

Understanding the interconnection among financial entities and how they vary over time provides investors and policy makers with insights into risk control and decision making. Reference [66] presents a comprehensive study of the applications of network theory in financial systems. In this section, we apply our method to a real financial dataset from Yahoo! Finance (finance.yahoo.com). The data matrix contains daily closing prices of 420 stocks that are always in the S&P 500 index between 2 January 2002 through 30 December 2011. In total, there are $n=2519$ time points. We select 100 stocks with the largest volatility and consider their log-returns; that is, for $j=1,\dots ,100$,

$$X_{ij}=log\left(p_{i+1,j}/p_{ij}\right),$$

where $p_{ij}$ is the daily closing price of the stock j at time point i. We first compute the statistic (5) and (6) for the change point detection. We look at the top three statistics for different bandwidths. For bandwidth $k=n^{-1/5}=0.21$, we rank the test statistic and find that the location for the top change point is: 7 February 2008 ($n_{{\widehat{s}}_1}=1536$), which is shown in Figure 5. The detected change point is quite robust to a variety of choices of bandwidth. Our result is partially consistent with the change point detection method in [48]. In particular, the two breaks in 2006 and 2007 were also found in [48] and it is conjectured that the 2007 break may be associated to the U.S. house market collapse. Meanwhile, it is interesting to observe the increased volatility before the 2008 financial crisis.

Figure 5.

Break size $|D_s{|}_{\infty }$. From 4 February 2004, to 30 November 2009.

Next, we estimate the time-varying networks before and after the change point at 26 May 2006 with the largest jump size. Specifically, we look at four time points at: 813, 828, 888, and 903, corresponding to 23 March 2006, 13 April 2006, 11 July 2006, and 1 August 2006. We use tv-CLIME (13) with the Epanechnikov kernel with the same bandwidth as in the change point detection to estimate the networks at the four points. Optimal tuning parameter $\lambda$ is automatically selected according to the stability approach [67]. The following matrix shows the number of different edges at those four time points. It is observed that the time of the first two time points (813 and 828) and the last two (888 and 903) has a higher similarity than across the change point at time 858. The estimated networks are shown in Figure 6. Networks in the first and second row are estimated before and after the estimated change point at time 858, respectively. It is observed that at each time point the companies in the same section tend to be clustered together such as companies in the Energy section: OXY, NOV, TSO, MRO, and DO (highlighted in cyan). In addition, the distance matrix of estimated networks is estimated as

$$\left(\begin{array}{cccc}0& 332& 350& 396\\ 332& 0& 394& 428\\ 350& 394& 0& 234\\ 396& 428& 234& 0\end{array}\right).$$

Figure 6.

Estimated networks at time points 813, 828, 888, and 903, corresponding to 23 March 2006, 13 April 2006, 11 July 2006, and 1 August 2006. Colors correspond to the nine sections in the S&P dataset.

7. Proof of Main Results

7.1. Preliminary Lemmas

Lemma 1.

Let ${\left(Y_i\right)}_{i\in Z}$ be a sequence that admits (2). Assume $Y_i\in {\mathcal{L}}^q$ for $i=1,2,\cdots$, and the dependence adjusted norm (DAN) of the corresponding underlying array $\left(Y_i^{\circ }\left(t\right)\right)$ satisfies $\parallel Y_{·}{\parallel }_{q,A}<\infty$ for $q>2$ and $A>0$. Let ${\left(\omega (t,t_i)\right)}_{i=1}^n$ be defined in (12) and suppose that the kernel function $K(·)$ satisfies Assumption 5. Denote ${\varpi }_{q,A}\left(n\right)=n,n{(logn)}^{1+2q},n^{q/2-Aq}$ if $A>1/2-1/q$, $A=1/2-1/q$, and $0<A<1/2-1/q$, respectively. Then there exist constants $C_1,C_2$ and $C_3$ independent of n, such that for all $x>0$,

$$\begin{array}{c}\hfill \underset{t\in (0,1)}{sup}\mathbb{P}\left(\left|\sum _{i=1}^{n}w(t,t_i)\left(Y_i-\mathbb{E}\left(Y_i\right)\right)\right|>x\right)\le C_1\frac{{\varpi }_{q,A}\left(B_n\right){∥Y_{·}∥}_{q,A}^q}{B_n^q{x}^q}+C_{2}exp\left(\frac{-C_3{B}_n{x}^2}{{∥Y_{·}∥}_{2,A}^2}\right).\end{array}$$ (28)

$$\begin{array}{c}\hfill \mathbb{P}\left(\underset{t\in (0,1)}{sup}\left|\sum _{i=1}^{n}w(t,t_i)\left(Y_i-\mathbb{E}\left(Y_i\right)\right)\right|>x\right)\le C_1\frac{{\varpi }_{q,A}\left(n\right){∥Y_{·}∥}_{q,A}^q}{B_n^q{x}^q}+C_{2}exp\left(\frac{-C_3{B}_n^2{x}^2}{n{∥Y_{·}∥}_{2,A}^2}\right).\end{array}$$ (29)

Proof. 

Let $S_i={\sum }_{j=1}^i\left(Y_i-\mathbb{E}\left(Y_i\right)\right)$. Note that

$$\begin{array}{cc}\hfill \underset{t\in (0,1)}{sup}\left|\sum _{i=1}^{n}w(t,t_i)Y_i\right|& =\underset{t\in (0,1)}{sup}\left|\sum _{i=1}^{n}w(t,t_i)(S_i-S_{i-1})\right|\hfill \\ \hfill & \le \underset{t}{sup}\left|\sum _{i=1}^{n-1}\left[\left(w(t,t_i)-w(t,t_{i+1})\right)S_i\right]\right|+\underset{t}{sup}\left|w(t,1)S_n\right|\hfill \\ \hfill & \lesssim B_n^{-1}\underset{1\le i\le n}{max}|S_i|,\hfill \end{array}$$

where the last inequality follows from the fact that ${sup}_t{\sum }_{i=1}^n|w(t,t_i)-w(t-t_{i+1})|\asymp B_n^{-1}$, due to Assumption 5.

To see (29), it suffices to show

$$\begin{array}{c}\hfill \mathbb{P}\left(\underset{1\le i\le n}{max}|S_i|>x\right)\le C_1\frac{{\varpi }_{q,A}\left(n\right){∥Y_{·}∥}_{q,A}^q}{x^q}+C_{2}exp\left(\frac{-C_3{x}^2}{n{∥Y_{·}∥}_{2,A}^2}\right).\end{array}$$ (30)

Now, we develop a probability deviation inequality for ${max}_{1\le i\le n}|{\sum }_{j=1}^i{\alpha }_j{Y}_j|$, where ${\alpha }_j\ge 0$, $1\le j\le n$ are constants such that ${\sum }_{1\le j\le n}{\alpha }_j=1$. Denote ${\mathcal{P}}_0\left(Y_i\right)=\mathbb{E}(Y_i|{\epsilon }_i)-\mathbb{E}\left(Y_i\right)$ and

$${\mathcal{P}}_k\left(Y_i\right)=\mathbb{E}(Y_i|{\epsilon }_{i-k},\dots ,{\epsilon }_i)-\mathbb{E}(Y_i|{\epsilon }_{i-k+1},\dots ,{\epsilon }_i).$$

Then we can write

$$\begin{array}{cc}\hfill {\displaystyle \underset{1\le i\le n}{max}}|{\displaystyle \sum _{j=1}^i}{\alpha }_j{Y}_j|& \le {\displaystyle \underset{1\le i\le n}{max}}|{\displaystyle \sum _{j=1}^i}{\alpha }_j{\mathcal{P}}_0\left(Y_j\right)|+{\displaystyle \underset{1\le i\le n}{max}}|{\displaystyle \sum _{k=1}^n}{\displaystyle \sum _{j=1}^i}{\alpha }_j{\mathcal{P}}_k\left(Y_j\right)|\hfill \\ \hfill & +{\displaystyle \underset{1\le i\le n}{max}}|{\displaystyle \sum _{k=n+1}^{\infty }}{\displaystyle \sum _{j=1}^i}{\alpha }_j{\mathcal{P}}_k\left(Y_j\right)|.\hfill \end{array}$$ (31)

Note that ${\left({\mathcal{P}}_0\left(Y_j\right)\right)}_{j\in \mathbb{Z}}$ is an independent sequence. By Nagaev’s inequality and Ottaviani’s inequality, we have that

$$\begin{array}{cc}\hfill \mathbb{P}({\displaystyle \underset{1\le i\le n}{max}}|{\displaystyle \sum _{j=1}^i}{\alpha }_j{\mathcal{P}}_0\left(Y_j\right)|\ge x)& \lesssim \frac{{\sum }_{j=1}^n{\alpha }_j^q{∥{\mathcal{P}}_0\left(Y_j\right)∥}_q^q}{x^q}+exp\left(-\frac{C_3{x}^2}{{\sum }_{j=1}^n{\alpha }_j^2{\parallel {\mathcal{P}}_0\left(Y_j\right)\parallel }_2^2}\right)\hfill \\ \hfill & \lesssim \frac{{\sum }_{j=1}^n{\alpha }_j^q}{x^q{\parallel Y_j\parallel }_q}+exp\left(-C_3\frac{x^2}{{\sum }_{j=1}^n{\alpha }_j^2}\right),\hfill \end{array}$$ (32)

where the last inequality holds because $\parallel {\mathcal{P}}_0\left(Y_j\right){\parallel }_q\le 2{\parallel Y_j\parallel }_q$ by Jensen’s inequality. Since ${\sum }_{j=i+1}^{\infty }{\alpha }_j{\mathcal{P}}_k\left(Y_j\right)$ is a martingale difference sequence with respect to $\sigma ({\epsilon }_{i+1-k},{\epsilon }_{i+2-k},\dots )$, we have that $|{\sum }_{k=1+n}^{\infty }{\sum }_{j=i+1}^n{\alpha }_j{\mathcal{P}}_k\left(Y_j\right)|$ is a non-negative sub-martingale. Then by Doob’s inequality and Burkholder’s inequality, we have

$$\begin{array}{cc}\hfill & \mathbb{P}\left({\displaystyle \underset{1\le i\le n}{max}}|{\displaystyle \sum _{k=n+1}^{\infty }}{\displaystyle \sum _{j=1}^i}{\alpha }_j{\mathcal{P}}_k\left(Y_j\right)|\ge x\right)\hfill \\ \hfill & \le \mathbb{P}\left(|{\displaystyle \sum _{k=n+1}^{\infty }}{\displaystyle \sum _{j=1}^n}{\alpha }_j{\mathcal{P}}_k\left(Y_j\right)|\ge \frac{x}{2}\right)+\mathbb{P}\left({\displaystyle \underset{1\le i\le n}{max}}|{\displaystyle \sum _{k=n+1}^{\infty }}{\displaystyle \sum _{j=1+i}^n}{\alpha }_j{\mathcal{P}}_k\left(Y_j\right)|\ge \frac{x}{2}\right)\hfill \\ \hfill & \lesssim \frac{{∥{\sum }_{k=1+n}^{\infty }{\sum }_{j=1}^n{\alpha }_j{\mathcal{P}}_k\left(Y_j\right)∥}_q^q}{x^q}\hfill \\ \hfill & \lesssim \frac{{\left({\sum }_{j=1}^n{\alpha }_j^2\right)}^{q/2}{\Theta }_{n,q}^q}{x^q}\le \frac{{\Theta }_{n,q}^q{n}^{q/2-1}{\sum }_{j=1}^n{\alpha }_j^q}{x^q}.\hfill \end{array}$$ (33)

Now, we deal with the term ${max}_{1\le i\le n}|{\sum }_{k=1}^n{\sum }_{j=1}^i{\alpha }_j{\mathcal{P}}_k\left(Y_j\right)|$. Define $a_m=min(2^m,n)$ and $M_n=\lceil logn/log2\rceil$. Then

$$\begin{array}{c}\hfill \underset{1\le i\le n}{max}|\sum _{k=1}^n\sum _{j=1}^i{\alpha }_j{\mathcal{P}}_k\left(Y_j\right)|\le \sum _{m=1}^{M_n}\underset{1\le i\le n}{max}|\sum _{l=1}^{\lceil i/a_m\rceil }\sum _{j=1+(l-1)a_m}^{min(l{a}_m,i)}\sum _{k=1+a_{m-1}}^{a_m}{\alpha }_j{\mathcal{P}}_k\left(Y_j\right)|.\end{array}$$ (34)

Let ${\mathcal{A}}_{odd}=\{1\le l\le \lceil i/a_m\rceil ,l \mathrm{is} \mathrm{odd}\}$ and ${\mathcal{A}}_{even}=\{1\le l\le \lceil i/a_m\rceil ,l \mathrm{is} \mathrm{even}\}$. We have

$$\begin{array}{c}\hfill \mathbb{P}\left(\underset{1\le i\le n}{max}|\sum _{l=1}^{\lceil i/a_m\rceil }{Z}_{l,m,i}|\ge x\right)\le \mathbb{P}\left(\underset{1\le i\le n}{max}|\sum _{{\mathcal{A}}_{odd}}{Z}_{l,m,i}|\ge x/2\right)+\mathbb{P}\left(\underset{1\le i\le n}{max}|\sum _{{\mathcal{A}}_{even}}{Z}_{l,m,i}|\ge x/2\right),\end{array}$$

where we have that $Z_{l,m,i}:={\sum }_{j=1+(l-1)a_m}^{min(l{a}_m,i)}{\alpha }_j{\mathcal{P}}_{a_{m-1}}^{a_m}\left(Y_j\right)$ is independent of $Z_{l+2,m,i}$ for $1\le l\le \lceil i/a_m\rceil ,1\le m\le M_n,1\le i\le n$, as ${\mathcal{P}}_{a_{m-1}}^{a_m}\left(Y_j\right):={\sum }_{k=1+a_{m-1}}^{a_m}{\mathcal{P}}_k\left(Y_j\right)$ is $a_m$-dependent. Therefore, we can apply Ottaviani’s inequality and Nagaev’s inequality for independent variables. As a consequence,

$$\mathbb{P}\left(\underset{1\le i\le n}{max}|\sum _{l=1}^{\lceil i/a_m\rceil }{Z}_{l,m,i}|\ge x\right)\lesssim \frac{{\sum }_{1\le l\le \lceil n/a_m\rceil }{\parallel Z_{l,m,n}\parallel }_q^q}{x^q}+exp\left(-\frac{C_3{x}^2}{{\sum }_{1\le l\le \lceil n/a_m\rceil }{\parallel Z_{l,m,n}\parallel }_2^2}\right).$$

Again, by Burkholder’s inequality, we have that for $q\ge 2$,

$$\begin{array}{cc}\hfill \parallel Z_{l,m,n}{\parallel }_q& \le \sum _{k=1+a_{m-1}}^{a_m}{\parallel \sum _{j=1+(l-1)a_m}^{min(l{a}_m,n)}{\alpha }_j{\mathcal{P}}_k\left(Y_j\right)\parallel }_q\hfill \\ \hfill & \lesssim {\left(\sum _{j=1+(l-1)a_m}^{min(l{a}_m,n)}{\alpha }_j^2\right)}^{1/2}({\Theta }_{a_{m-1}}-{\Theta }_{a_m}).\hfill \end{array}$$

Note ${\sum }_{j=1+(l-1)a_m}^{min(l{a}_m,n)}{\alpha }_j^2\le a_m^{(q-2)/q}{\left({\sum }_{j=1+(l-1)a_m}^{min(l{a}_m,n)}{\alpha }_j^q\right)}^{2/q}$. Let ${\tau }_m=m^{-2}/{\sum }_{m=1}^{M_n}{m}^{-2}$, and we have ${\tau }_m\asymp m^{-2}$ as $1\le {\sum }_{m=1}^{M_n}{m}^{-2}\le {\pi }^2/6$. In respect to (34), we have that

$$\begin{array}{cc}\mathbb{P}\left(\underset{1\le i\le n}{max}|\sum _{k=1}^n\sum _{j=1}^i{\mathcal{P}}_k\left(Y_j\right)|\ge x\right)\hfill & \le \sum _{m=1}^{M_n}\mathbb{P}\left(\underset{1\le i\le n}{max}|\sum _{l=1}^{\lceil i/a_m\rceil }{Z}_{l,m,i}|\ge {\tau }_{m}x\right)\hfill \\ \hfill & \lesssim \frac{{\sum }_{i=1}^n{\alpha }_j^q}{x^q}{\parallel Y_{·}\parallel }_{q,A}^q\sum _{m=1}^{M_n}{\tau }_m^{-q}{a}_m^{(1/2-A)q-1}+\sum _{m=1}^{M_n}exp\left(-\frac{C_3{x}^2{\tau }_m^2{a}_m^{2A}}{{\sum }_{j=1}^n{\alpha }_j^2{\parallel Y_{·}\parallel }_{2,A}^2}\right).\hfill \end{array}$$ (35)

Note ${\sum }_{m=1}^{M_n}{\tau }_m^{-q}{a}_m^{(1/2-A)q-1}\asymp n^{-1}{\varpi }_{q,A}\left(n\right)$, and

$$\sum _{m=1}^{M_n}exp\left(-\frac{C_3{x}^2{\tau }_m^2{a}_m^{2A}}{{\sum }_{j=1}^n{\alpha }_j^2{\parallel Y_{·}\parallel }_{2,A}^2}\right)\lesssim exp\left(-\frac{C_3{x}^2}{{\sum }_{j=1}^n{\alpha }_j^2{\parallel Y_{·}\parallel }_{2,A}^2}\right).$$

Combining (31), (32), (33) and (35), we obtain

$$\begin{array}{cc}\hfill & \mathbb{P}\left(\underset{1\le i\le n}{max}|\sum _{j=1}^i{\alpha }_j\left(Y_j-\mathbb{E}\left(Y_j\right)\right)|>x\right)\hfill \\ \hfill & \le C_1\frac{{\varpi }_{q,A}\left(n\right){\sum }_{j=1}^n{\alpha }_j^q{∥Y_{·}∥}_{q,A}^q}{n{x}^q}+C_{2}exp\left(\frac{-C_3{x}^2}{{\sum }_{j=1}^n{\alpha }_j^2{∥Y∥}_{2,A}^2}\right).\hfill \end{array}$$ (36)

Now, we have (30) by taking ${\alpha }_j=n^{-1}$ for $j=1,\dots ,n$. Note that since $K(·)$ has bounded support, for any given $t\in [b,1-b]$, we have

$$\begin{array}{c}\hfill \mathbb{P}\left(|\sum _{i=1}^{n}w(t,t_i)(Y_i-\mathbb{E}{Y}_i)|>x\right)\le \mathbb{P}\left(|\sum _{i=-B_n}^{B_n}w(t,t_{tn+i})(Y_{tn+i}-\mathbb{E}{Y}_{tn+i})|>x\right)\\ \hfill \le C_1\frac{{\varpi }_{q,A}\left(B_n\right){\sum }_{i=-B_n}^{B_n}w{(t,t_{tn+i})}^q{∥Y_{·}∥}_{q,A}^q}{B_n{x}^q}+C_{2}exp\left(\frac{-C_3{x}^2}{{\sum }_{i=-B_n}^{B_n}w{(t,t_{tn+i})}^2{∥Y_{·}∥}_{2,A}^2}\right).\end{array}$$

Therefore (28) follows from (36) by taking ${\alpha }_j=w(t,tn+j)$, and note that for any $t\in [b,1-b]$, ${\sum }_{i=-B_n}^{B_n}w{(t,t_{tn+i})}^{\beta }\asymp B_n^{1-\beta }$ for a constant $\beta \ge 2$. □

Lemma 2.

Suppose ${\left(X_{ij}\right)}_{i\in \mathbb{Z},1\le j\le p}$ satisfys Assumption 2. Furthermore, let Assumption 5 hold. Let ${\varpi }_{q,A}\left(n\right)$ be defined as in Lemma 1. Then there exist constants $C_1,C_2$, and $C_3$ independent of n and p, such that for all $x>0$, we have

$$\begin{array}{ccc}\hfill & \hfill & {\displaystyle \underset{t\in (0,1)}{sup}}\mathbb{P}\left(|{\displaystyle \sum _{i=1}^n}\omega (t,t_i)\left({\mathbf{X}}_i{\mathbf{X}}_i^{\top }-\mathbb{E}\left({\mathbf{X}}_i{\mathbf{X}}_i^{\top }\right)\right){|}_{\infty }\ge x\right)\hfill \\ \hfill & \hfill & \le C_1{\nu }_{2q}^q\frac{p{\varpi }_{q,A}\left(B_n\right)M_{X,q}^q}{B_n^q{x}^q}+C_2{p}^{2}exp\left(-C_3\frac{B_n{x}^2}{{\nu }_4^2{N}_X^2}\right),\hfill \end{array}$$ (37)

and

$$\begin{array}{ccc}\hfill & \hfill & \mathbb{P}\left({\displaystyle \underset{t\in (0,1)}{sup}}|{\displaystyle \sum _{i=1}^n}w(t,t_i)\left({\mathbf{X}}_i{\mathbf{X}}_i^{\top }-\mathbb{E}\left({\mathbf{X}}_i{\mathbf{X}}_i^{\top }\right)\right){|}_{\infty }\ge x\right)\hfill \\ \hfill & \hfill & \le C_1{\nu }_{2q}^q\frac{p{\varpi }_{q,A}\left(n\right)M_{X,q}^q}{B_n^q{x}^q}+C_2{p}^{2}exp\left(-C_3\frac{B_n^2{x}^2}{n{\nu }_4^2{N}_X^2}\right).\hfill \end{array}$$ (38)

Proof. 

For $1\le j,k\le p$, let $Y_{i,jk}=X_{ij}{X}_{ik}$. We now check the conditions in Lemma 1 for ${\left(Y_{i,jk}\right)}_{1\le i\le n}$. Denote $Y_{i,jk,\left\{m\right\}}=X_{ij,\left\{m\right\}}{X}_{ik,\left\{m\right\}}$. Then the uniform functional dependence measure of ${\left(Y_{i,jk}\right)}_i$ is

$$\begin{array}{ccc}\hfill {\theta }_{m,q,jk}^Y& =& \underset{i}{sup}{\parallel Y_{i,jk}-Y_{i,jk,\left\{m\right\}}\parallel }_q\hfill \\ & =& \underset{i}{sup}{\parallel X_{ij}{X}_{ik}-X_{ij,\left\{m\right\}}{X}_{ik,\left\{m\right\}}\parallel }_q\hfill \\ & \le & \underset{i}{sup}\parallel X_{ij}(X_{ik}-X_{ik,\left\{m\right\}}){\parallel }_q+\underset{i}{sup}{\parallel X_{ik,\left\{m\right\}}(X_{ij}-X_{ij,\left\{m\right\}})\parallel }_q.\hfill \end{array}$$

Thus the DAN of the process $Y_{·,jk}$ satisfies that

$$\begin{array}{c}\hfill \parallel Y_{·,jk}{\parallel }_{q,A}\le \underset{i}{sup}\parallel X_{ij}{\parallel }_{2q}\parallel X_{·,k}{\parallel }_{2q,A}+\underset{i}{sup}\parallel X_{ik}{\parallel }_{2q}\parallel X_{·,j}{\parallel }_{2q,A}\le {\nu }_q(\parallel X_{·,k}{\parallel }_{2q,A}+\parallel X_{·,j}{\parallel }_{2q,A}).\end{array}$$

The result follows immediately from Lemma 1 and the Bonferroni inequality. □

Lemma 3.

We adopt the notation in Lemma 2. Suppose Assumptions 2, 1, and 5 hold with $\iota =0$. Recall $B_n=nb$, where $b\to 0$ and $B_n/\sqrt{n}\to \infty$ as $n\to \infty$. Then there exists a constant C independent of n and p such that $\widehat{\Sigma }\left(t\right)$ in (11) satisfies that for any $t\in [c,1-c]$,

$$\begin{array}{cc}\hfill |\widehat{\Sigma }{\left(t\right)-\Sigma \left(t\right)|}_{\infty }& =O_{\mathbb{P}}\left(b^2+M_{X,q}{\nu }_{2q}{B}_n^{-1}{\left(p{\varpi }_{q,A}\left(B_n\right)\right)}^{1/q}+{\nu }_4{N}_X{(logp/B_n)}^{1/2}\right).\hfill \end{array}$$ (39)

Furthermore,

$$\begin{array}{c}\hfill \underset{t\in [c,1-c]}{sup}{|\widehat{\Sigma }\left(t\right)-\Sigma \left(t\right)|}_{\infty }=O_{\mathbb{P}}\left(b^2+M_{X,q}{\nu }_{2q}{B}_n^{-1}{\left(p{\varpi }_{q,A}\left(n\right)\right)}^{1/q}+{\nu }_4{N}_X{B}_n^{-1}{[nlogp]}^{1/2}\right).\end{array}$$ (40)

Proof. 

First, we have

$$\begin{array}{c}\hfill \mathbb{E}{\widehat{\sigma }}_{jk}\left(t\right)-{\sigma }_{jk}\left(t\right)=\sum _{i=1}^{n}w(t,t_i)[{\sigma }_{jk}\left(t_i\right)-{\sigma }_{jk}\left(t\right)].\end{array}$$

Approximating the discrete summation with integral, we obtain for all $1\le j,k\le p$,

$$\begin{array}{c}\hfill \underset{t\in [b,1-b]}{sup}\left|\mathbb{E}{\widehat{\sigma }}_{jk}\left(t\right)-{\sigma }_{jk}\left(t\right)-{\int }_{-1}^{1}K\left(u\right)[{\sigma }_{jk}(ub+t)-{\sigma }_{jk}\left(t\right)]du\right|=O\left(B_n^{-1}\right).\end{array}$$

By Assumption 1, we have

$$\begin{array}{cc}\hfill {\sigma }_{jk}(ub+t)-{\sigma }_{jk}\left(t\right)& =ub{\sigma }_{jk}^{\prime }\left(t\right)+\frac{1}{2}{u}^2{b}^2{\sigma }_{jk}^{″}\left(t\right)+o\left(b^2{u}^2\right).\hfill \end{array}$$

Thus we have ${sup}_{t\in [c,1-c]}{|\mathbb{E}\widehat{\sigma }\left(t\right)-\sigma \left(t\right)|}_{\infty }=O\left(B_n^{-1}+b^2\right)$, in view of Assumption 5. By Lemma 2, we have

$$\begin{array}{cc}\hfill \underset{t\in (0,1)}{sup}\mathbb{P}\left({\left|\widehat{\Sigma }\left(t\right)-\mathbb{E}\widehat{\Sigma }\left(t\right)\right|}_{\infty }\ge x\right)& \le C_{1}p{\nu }_q^q\frac{M_{X,q}^q{\varpi }_{q,A}\left(B_n\right)}{B_n^q{x}^q}+C_2{p}^{2}exp\left(-C_3\frac{B_n{x}^2}{N_X^2}\right).\hfill \end{array}$$

Denote $u=C_4\left(M_{X,q}{\nu }_{2q}{B}_n^{-1}{\left(p{\varpi }_{q,A}\left(B_n\right)\right)}^{1/q}+{\nu }_4{N}_X{(logp/B_n)}^{1/2}\right)$ for a large enough constant $C_4$, then for any $t\in (0,1)$,

$$\begin{array}{c}\hfill {\left|\widehat{\Sigma }\left(t\right)-\mathbb{E}\widehat{\Sigma }\left(t\right)\right|}_{\infty }=O_{\mathbb{P}}\left(u\right).\end{array}$$

Thus (39) is proved. The result (40) can be obtained similarly. □

7.2. Proof of Main Results

Proof of Proposition 1. 

Given (39) and (40), the proof of (16) is standard. (See, e.g., Theorem 6 of [11]). For ${\lambda }^{\circ }$ and ${\lambda }^{*}$ given in Proposition 1, by Lemma 3, we have that, respectively,

$$\begin{array}{cc}\hfill {\lambda }^{\circ }& \ge \underset{t}{sup}\mathbb{E}\left({\kappa }_p{|\widehat{\Sigma }\left(t\right)-\Sigma \left(t\right)|}_{\infty }\right),\hfill \end{array}$$ (41)

$$\begin{array}{cc}\hfill {\lambda }^{\diamond }& \ge \mathbb{E}\left({\kappa }_p\underset{t}{sup}{|\widehat{\Sigma }\left(t\right)-\Sigma \left(t\right)|}_{\infty }\right).\hfill \end{array}$$ (42)

Then note that for any $t\in [0,1]$, for any $\lambda >0$,

$$\begin{array}{cc}\hfill & |{\widehat{\Omega }}_{\lambda }{\left(t\right)-\Omega \left(t\right)|}_{\infty }\le {|\Omega \left(t\right)|}_{L_1}{|\Sigma \left(t\right){\widehat{\Omega }}_{\lambda }\left(t\right)-{\mathrm{Id}}_p|}_{\infty }\hfill \\ \hfill & \le {|\Omega \left(t\right)|}_{L_1}\left[|\widehat{\Sigma }\left(t\right){\widehat{\Omega }}_{\lambda }\left(t\right)-{\mathrm{Id}}_p{|}_{\infty }+|(\Sigma \left(t\right)-\widehat{\Sigma }\left(t\right)){\Omega \left(t\right)|}_{\infty }+|{\widehat{\Omega }}_{\lambda }{\left(t\right)-\Omega \left(t\right)|}_{L^1}{|\widehat{\Sigma }\left(t\right)-\Sigma \left(t\right)|}_{\infty }\right]\hfill \end{array}$$

where by construction, we have $|\widehat{\Sigma }\left(t\right){\widehat{\Omega }}_{\lambda }\left(t\right)-{\mathrm{Id}}_p{|}_{\infty }\le \lambda$ and $|{\widehat{\Omega }}_{\lambda }{\left(t\right)-\Omega \left(t\right)|}_{L^1}\le 2{\kappa }_p$. Consequently,

$$\begin{array}{c}\hfill |{\widehat{\Omega }}_{\lambda }{\left(t\right)-\Omega \left(t\right)|}_{\infty }\le {\kappa }_p\left(\lambda +3{\kappa }_p{|\widehat{\Sigma }\left(t\right)-\Sigma \left(t\right)|}_{\infty }\right).\end{array}$$ (43)

Then (16) and (17) follow from (41) to (43). □

Proof of Proposition 2. 

Theorem 2 is an immediate result of (17). □

Proof of Theorem 1. 

Denote $r_j,1\le j\le \iota$ as the time point(s) of the time of jump ordered decreasingly in the sense of the infinite norm of covariance matrices, i.e., $|\Delta \left(r_1\right){|}_{\infty }\ge |\Delta \left(r_2\right){|}_{\infty }\ge \dots \ge |\Delta \left(r_{\iota }\right){|}_{\infty }\ge {|\Delta \left(s\right)|}_{\infty }$ for $s\in (0,1)\cap {\{r_1,\dots ,r_{\iota }\}}^c$. (Temporal order is applied if there is a tie.) Let ${\mathcal{T}}_h\left(j\right)=[r_j-h,r_j+h)$. For $h=o\left(1\right)$, as a result of Assumption 3, ${\mathcal{T}}_h\left(j\right)\cap {\mathcal{T}}_h\left(i\right)=\varnothing$ if $i\ne j$ for n sufficiently large. That is to say, each time point $s\in (0,1)$ is in the neighborhood of, at most, one change point.

For any $s\in [t^{\left(j\right)},t^{(j+1)})$, $j=0,1,\dots ,\iota$, denote $\mathbb{D}\left(s\right)=\mathbb{E}\left[D\right(s\left)\right]$ and

$${\mathbb{D}}^{\diamond }\left(s\right)=\left\{\begin{array}{cc}(h-s+t^{\left(j\right)})\Delta \left(t^{\left(j\right)}\right),& t^{\left(j\right)}\le s<t^{\left(j\right)}+h\\ 0,& t^{\left(j\right)}+h\le s<t^{(j+1)}-h\\ (h+s-r)\Delta \left(t^{(j+1)}\right),& t^{(j+1)}-h\le s\le t^{(j+1)}.\end{array}\right.$$ (44)

Then, for $s\in {\cup }_{1\le j\le \iota }[t^{\left(j\right)}+h,<t^{(j+1)}-h)$, by (3), we have

$${|\Sigma (s+t)-\Sigma \left(s\right)|}_{\infty }\le Lt,\forall |t|\le h,$$

we can easily verify that

$$\underset{s\in [0,1]}{sup}{|\mathbb{D}\left(s\right)-{\mathbb{D}}^{\diamond }\left(s\right)|}_{\infty }\le L{h}^2.$$ (45)

Note that $|{\mathbb{D}}^{\diamond }{\left(s\right)|}_{\infty }$ is maximized at $s=r_1$ and $|{\mathbb{D}}^{\diamond }\left(r_1\right){|}_{\infty }=h{|\Delta \left(r_1\right)|}_{\infty }$. By the triangle inequalities, we have that for some positive constant C, for any $s\in [0,1]$,

$$\begin{array}{ccc}|\mathbb{D}\left(r_1\right){|}_{\infty }-{|\mathbb{D}\left(s\right)|}_{\infty }\hfill & \ge \hfill & h{c}_2-|\mathbb{D}\left(r_1\right)-{\mathbb{D}}^{\diamond }\left(r_1\right){|}_{\infty }-|{\mathbb{D}}^{\diamond }\left(s\right){|}_{\infty }-{|\mathbb{D}\left(s\right)-{\mathbb{D}}^{\diamond }\left(s\right)|}_{\infty }\hfill \\ \hfill & \ge \hfill & h{c}_2-{|{\mathbb{D}}^{\diamond }\left(s\right)|}_{\infty }-2L{h}^2\hfill \\ \hfill & \ge \hfill & c_2(|s-r_1|\wedge h)-2L{h}^2.\hfill \end{array}$$ (46)

On the other hand, since $|D\left(r_1\right){|}_{\infty }\le {|D\left({\widehat{s}}_1\right)|}_{\infty }$, we have

$$\begin{array}{cc}|\mathbb{D}\left(r_1\right){|}_{\infty }-{|\mathbb{D}\left({\widehat{s}}_1\right)|}_{\infty }\hfill & \le |D\left(r_1\right){|}_{\infty }-|D\left({\widehat{s}}_1\right){|}_{\infty }+|\mathbb{D}\left(r_1\right)-D\left(r_1\right){|}_{\infty }+{|\mathbb{D}\left({\widehat{s}}_1\right)-D\left({\widehat{s}}_1\right)|}_{\infty }\hfill \\ \hfill & \le |\mathbb{D}\left(r_1\right)-D\left(r_1\right){|}_{\infty }+{|\mathbb{D}\left({\widehat{s}}_1\right)-D\left({\widehat{s}}_1\right)|}_{\infty }.\hfill \end{array}$$ (47)

Denote the event $\mathcal{A}:=\{{sup}_{s\in [h,1-h]}|D\left(s\right)-\mathbb{D}\left(s\right){|}_{\infty }\le h_{\diamond }^2\}$ and let ${\mathbf{Y}}_{\mathbf{i}}={\left(Y_{i,jk}\right)}_{1\le j,k\le p}$, $Y_{i,jk}=X_{ij}{X}_{ik}-{\sigma }_{i,jk}$. Note that

$$|D_{jk}\left(s\right)-{\mathbb{D}}_{jk}\left(s\right)|=\frac{1}{n}\left|\sum _{i=1}^{hn}{Y}_{n_s+1-i,jk}-\sum _{i=1}^{hn}{Y}_{n_s+i,jk}\right|.$$ (48)

By Lemma 2, we have for any $x>0$,

$$\begin{array}{c}\hfill \mathbb{P}\left(\underset{s\in [h,1-h]}{sup}{|D\left(s\right)-\mathbb{D}\left(s\right)|}_{\infty }\ge x\right)\le C_1\frac{p{\varpi }_{q,A}\left(n\right)M_{X,q}^q{\nu }_{2q}^q}{n^q{x}^q}+C_2{p}^{2}exp\left(-C_3\frac{n{x}^2}{N_X^2}\right).\end{array}$$ (49)

It follows that

$$|\mathbb{D}\left(r_1\right){|}_{\infty }-{|\mathbb{D}\left({\widehat{s}}_1\right)|}_{\infty }=O_{\mathbb{P}}\left(h^{-1}{J}_{q,A}(n,p)+N_X{h}^{-1}{(n^{-1}log\left(p\right))}^{1/2}\right).$$

Taking $h=h_{\diamond }$, we have

$$|{\widehat{s}}_1-r_1|=O_{\mathbb{P}}\left(h_{\diamond }^2\right).$$

Furthermore, we have

$$\mathbb{P}\left(\mathcal{A}\right)\ge 1-C_1{\left(\frac{p{\varpi }_{q,A}\left(n\right)M_{X,q}^q{\nu }_{2q}^q}{n^q{c}_2^q}\right)}^{1/3}-C_2{p}^{2}exp\left(-C_3{\left(\frac{n{log}^2\left(p\right)}{N_X^2}\right)}^{1/3}\right).$$

Let ${\mathcal{A}}_k:=\{{max}_{1\le j\le k}|{\widehat{s}}_j-r_j|\le c_2^{-1}2(L+1)h_{\diamond }^2\}$ for some $1\le k\le \iota$. Assume ${\mathcal{A}}_k\subset \mathcal{A}$. Under ${\mathcal{A}}_k$ we have that $[r_j-h_{\diamond },r_j+h_{\diamond })\subset {\widehat{\mathcal{T}}}_{2h_{\diamond }}\left(j\right)=:[{\widehat{s}}_j-2h_{\diamond },{\widehat{s}}_j+2h_{\diamond })$ for $1\le j\le k$ and $r_{k+1}\notin {\cup }_{1\le j\le k}{\widehat{\mathcal{T}}}_{2h_{\diamond }}\left(j\right)$ as a consequence of Assumption 3. According to (46) and (47), we have if $\mathcal{A}$ is true, $|{\widehat{s}}_{k+1}-r_{k+1}|\le c_2^{-1}2(L+1)h_{\diamond }^2$, which implies ${\mathcal{A}}_{k+1}\subset \mathcal{A}$. The result (21) follows from deduction.

Suppose $\mathcal{A}$ holds. By the choice of $\nu$, as a consequence of (45) and (49), and that $\nu \ll h_{\diamond }$, we have that

$$\underset{s\in [0,1]}{sup}{|D\left(s\right)-{\mathbb{D}}^{\diamond }\left(s\right)|}_{\infty }\le \nu .$$

As a result,

$$\underset{1\le j\le \iota }{min}{|D\left(r_j\right)|}_{\infty }\ge c_2{h}_{\diamond }-\nu \ge \nu ,$$

i.e., $\widehat{\iota }\ge \iota$. On the other hand, since ${\cup }_{1\le j\le \iota }{\widehat{\mathcal{T}}}_{2h_{\diamond }}\left(j\right)$ is excluded from the searching region for $s_{\iota +1}$, we have

$$\underset{s\in {\left({\cup }_{1\le j\le \iota }{\widehat{\mathcal{T}}}_{2h_{\diamond }}\left(j\right)\right)}^c}{sup}{|D\left(s\right)|}_{\infty }\le \nu .$$

In other words, $\{\widehat{\iota }=\iota \}\subset \mathcal{A}$. Thus (20) is proved. □

Proof of Theorem 2. 

We adopt the notations in the proof of Theorem 1 and assume that $\mathcal{E}$ holds. Similar to Lemma 3, we have that by Lemma 2, for any $t\in (0,1)$,

$$\begin{array}{c}\hfill {\left|\widehat{\Sigma }\left(t\right)-\mathbb{E}\widehat{\Sigma }\left(t\right)\right|}_{\infty }=O_{\mathbb{P}}\left(u\right),\end{array}$$

where $u=C_4\left(M_{X,q}{\nu }_{2q}{B}_n^{-1}{\left(p{\varpi }_{q,A}\left(B_n\right)\right)}^{1/q}+{\nu }_4{N}_X{(logp/B_n)}^{1/2}\right)$ for a large enough constant $C_4$.

Since under $\mathcal{E}$, ${\mathcal{T}}_b\left(j\right)\subset {\widehat{\mathcal{T}}}_{b+h_{\diamond }^2}\left(j\right)$. For $t\in {\left({\cup }_{1\le j\le \iota }{\widehat{\mathcal{T}}}_{b+h_{\diamond }^2}\left(j\right)\right)}^c\cap [b,1-b]$, we have that for all $1\le j,k\le p$,

$$\begin{array}{cc}\hfill \left|\mathbb{E}{\widehat{\sigma }}_{jk}\left(t\right)-{\sigma }_{jk}\left(t\right)\right|& ={\int }_{-1}^{1}K\left(u\right)[{\sigma }_{jk}(ub+t)-{\sigma }_{jk}\left(t\right)]du+O\left(B_n^{-1}\right)\hfill \\ \hfill & =b{\sigma }_{jk}^{\prime }\left(t\right){\int }_{-1}^{1}uK\left(u\right)du+\left(\frac{1}{2}{b}^2{\sigma }_{jk}^{″}\left(t\right)+o\left(b^2\right)\right){\int }_{-1}^1{u}^{2}K\left(u\right)du+O\left(B_n^{-1}\right)\hfill \\ \hfill & =O(b^2+B_n^{-1}).\hfill \end{array}$$

On the other hand, for $t\in {\cup }_{1\le j\le \iota }\left({\widehat{\mathcal{T}}}_{b+h_{\diamond }^2}\left(j\right)\cap {\mathcal{T}}_{h_{\diamond }^2}^c\left(j\right)\right)\cup [0,b]\cup [1-b,1]$, due to reflection, we no longer have that differentiability. As a result of the Lipschitz continuity, we get

$$\begin{array}{c}\hfill \left|\mathbb{E}{\widehat{\sigma }}_{jk}\left(t\right)-{\sigma }_{jk}\left(t\right)\right|={\int }_{-1}^{1}K\left(u\right)[{\sigma }_{jk}(ub+t)-{\sigma }_{jk}\left(t\right)]du+O\left(B_n^{-1}\right)=O(b+B_n^{-1}).\end{array}$$

The result (22) follows by the choices of b. The rest of the proof are similar to that of Proposition 1 and Theorem 2. □

Author Contributions

Methodology, M.X., X.C., W.B.W.; writing—original draft preparation, M.X., X.C., W.B.W.; writing—review and editing, M.X., X.C., W.B.W., software, M.X. All authors have read and agreed to the published version of the manuscript.

Funding

X.C.’s research is supported in part by NSF CAREER Award DMS-1752614 and UIUC Research Board Award RB18099. W.B.W.’s research is supported in part by NSF DMS-1405410.

Conflicts of Interest

The authors declare no conflict of interest.