Bootstrap Confidence Intervals for Multiple Change Points Based on Two-Stage Procedures

Abstract

This paper investigates the construction of confidence intervals for multiple change points in linear regression models. First, we detect multiple change points by performing variable selection on blocks of the input sequence; second, we re-estimate their exact locations in a refinement step. Specifically, we exploit an orthogonal greedy algorithm to recover the number of change points consistently in the cutting stage, and employ the sup-Wald-type test statistic to determine the locations of multiple change points in the refinement stage. Based on a two-stage procedure, we propose bootstrapping the estimated centered error sequence, which can accommodate unknown magnitudes of changes and ensure the asymptotic validity of the proposed bootstrapping method. This enables us to construct confidence intervals using the empirical distribution of the resampled data. The proposed method is illustrated with simulations and real data examples.

1. Introduction

Multiple change point detection is common in many applications, such as signal processing [1], medical diagnosis [2], industrial control [3], and oceanography [4]. For example, in the case of the detection of changes in one’s heart rates, the accurate identification of the change points can decompose the data into stationary segments, allowing clinicians to identify the behavior of the heart and diagnose diseases. The detection of multiple change points has important practical implications, which has prompted extensive research in the statistical community.

Implementing appropriate hypothesis testing is an important method for detecting multiple change points. Here are some examples. Ref. [5] proposed sup-Wald-type tests to test the null hypothesis of no change against alternative hypotheses containing an arbitrary number of changes to identify multiple structural changes in linear models. At the same time, the null hypothesis of s changes against $s+1$ changes is tested to determine the number of breakpoints. Later, ref. [6] developed dynamic programming principles to estimate multiple change points in linear regression. Ref. [7] proposed a genetic algorithm for detecting multiple breakpoints. However, these methods are very time-consuming. There is another line of research on multiple change point detection that transforms the problem into a variable selection problem. For example, ref. [8] used LASSO [9] to estimate the locations of multiple change points in a one-dimensional piecewise constant signal observed in white noise. Ref. [10] introduced a two-stage procedure based on adaptive LASSO [11], SCAD [12], and MCP [13] regularization methods that can simultaneously detect multiple change points in linear models. In addition, the two-stage procedure has been extended to accelerated failure time (AFT) models [14].

Despite the aforementioned developments, research on the inference for multiple change points remains limited. Under the null hypothesis, asymptotic [15] or approximate [16] distributions of test statistics have been derived, which enables quantifying uncertainty in the number of change points. For many testing procedures in change point analysis, the computation of critical values often relies on the asymptotic behavior of the test statistic under the null hypothesis. However, the convergence of the test statistic to its limiting distribution is often slow, and in some cases, the exact form of this distribution remains unknown. It has been noted that the bootstrap method is a computer-based statistical inference technique that can provide answers to a variety of statistical questions without relying on formulas. For example, ref. [17] proposed a bootstrap method in which the estimated error sequence is resampled with replacement to obtain confidence intervals for change points. Ref. [18] introduced an asymptotically valid confidence region for a single change point through the inversion of bootstrap tests. Ref. [19] studied the application of a circular overlapping block bootstrap method in the context of an at-most-one-change time series model with an abrupt change in the mean and dependent errors. In the field of a single change point estimation, the bootstrap technique has emerged as a valuable tool for approximating unknown probability distributions and the characteristics of change point estimators. These methods make it possible for us to provide confidence intervals for multiple change points. Ref. [20] addressed this problem in mean shift models by proposing the bootstrap construction of pointwise and uniform confidence intervals for multiple change points based on a moving sum procedure.

This paper aims to construct confidence intervals for multiple change points. By [10], the multiple change point detection problem can be formulated as a model selection problem. Before using bootstrapping, it is important to ensure that the estimates of the number of change points and the estimates of the within-segment parameters are consistent. However, regularization methods such as the LASSO and SCAD may suffer from the bias problem inherited from the penalty function [12,21]. Specifically, LASSO may select more irrelevant variables, leading to an overestimation of the number of change points. To alleviate this problem, we consider adopting the $L_0$-regularization method to achieve consistency between the estimates of the number of change points and their locations. Although the subset selection, while unbiased, is often computationally expensive, several optimization strategies and algorithms have been proposed to overcome the computational difficulties. For example, ref. [22] introduced an iterative algorithm called the orthogonal greedy algorithm (OGA) for high-dimensional regression models, which sequentially selects input variables to be included in the linear regression model and proposed a procedure named OGA + HDIC + Trim. OGA is a fast stepwise forward regression method that starts with a null model and adds predictors via component-wise linear least squares estimation. HDIC is a high-dimensional information criterion used to enter predictors into the model along the OGA path. The Trim method defines a subset to exclude additional irrelevant variables from OGA + HDIC. To construct confidence intervals for multiple change points, we proceed as follows. We first cut the data sequence into segments. Due to its model selection consistency and expected convergence properties, we apply OGA + HDIC + Trim to find data segments containing change points. We then utilize sup-Wald-type test statistics to locate them. Finally, we apply the bootstrap method to construct confidence intervals for multiple change points using the bootstrap estimated centered error sequence.

The main contributions of this paper are as follows. As described above, we first propose a two-stage procedure to detect multiple change points by using OGA + HDIC + Trim procedure and sup-Wald-type test statistics. In the first stage, we cut the data sequence into segments. In addition, we give the asymptotic distributions of the change point estimators under certain conditions. Based on this framework, we further explore the application of bootstrapping techniques in constructing confidence intervals for the change points and demonstrate the validity of the bootstrap method, i.e., the proposed bootstrap $100(1-\alpha )\%$-confidence intervals asymptotically attain the coverage probability of $1-\alpha$ for a given $\alpha \in (0,1)$ [20]. Last but not least, we illustrate the effectiveness and applicability of the proposed method through extensive simulation studies and a real data example, respectively.

The rest of this paper is organized as follows. Section 2 details the detection of multiple change points using the OGA + HDIC + Trim procedure and sup-Wald-type test statistics. Furthermore, Section 3 introduces the resampling bootstrap method for the change point estimators based on a two-stage procedure and Section 4 gives the theoretical properties of the proposed bootstrap method. Section 5 presents extensive simulation studies. Section 6 applies the proposed method to the seismograms of the 1982 Urakawa–Oki earthquake. Technical proofs of the main results are relegated to the Appendix A. In this paper, vectors and matrices are denoted in bold type.

2. Multiple Change Point Detection Based on Two-Stage Procedures

Assume that $({\mathit{x}}_i,y_i),i=1,\dots ,n,$ satisfy the following linear regression model with s change points located at $a_1<\dots <a_s$ as

$$\begin{array}{cc}\hfill y_i& ={\mathit{x}}_i^{\top }\left[{\mathit{\beta }}_1+\sum _{l=1}^s{\mathit{\delta }}_{l}I(a_l<i\le n)\right]+{\epsilon }_i\hfill \\ \hfill & =\left\{\begin{array}{cc}{\mathit{x}}_i^{\top }{\mathit{\beta }}_1+{\epsilon }_i,\hfill & \mathrm{if}1\le i\le a_1,\hfill \\ {\mathit{x}}_i^{\top }({\mathit{\beta }}_1+{\mathit{\delta }}_1)+{\epsilon }_i,\hfill & \mathrm{if}{a}_1<i\le a_2,\hfill \\ \cdots \hfill & \cdots \hfill \\ {\mathit{x}}_i^{\top }({\mathit{\beta }}_1+\sum _{l=1}^s{\mathit{\delta }}_l)+{\epsilon }_i,\hfill & \mathrm{if}{a}_s<i\le n,\hfill \end{array}\right.\hfill \end{array}$$ (1)

where n is the sample size, ${\mathit{x}}_i={(x_{i,1},\dots ,x_{i,q})}^{\top }$ is a sequence of q-dimensional predictors; ${\mathit{\beta }}_1={({\beta }_{1,1},\dots ,{\beta }_{q,1})}^{\top }\ne \mathbf{0}$ is an unknown q-dimensional vector of regression coefficients; s is an unknown number of change points; $1<a_1<\dots <a_s<n$ are unknown change points; ${\mathit{\delta }}_l={({\delta }_{1,l},\dots ,{\delta }_{q,l})}^{\top },l=1,\dots ,s$, are unknown changes in regression coefficient vectors at change points; and ${\epsilon }_i$’s are unobservable random errors.

The estimation of the regression coefficients ${\mathit{\beta }}_1$ and $({\mathit{\beta }}_1+{\sum }_{l=1}^j{\mathit{\delta }}_l),j=1,\dots s$ is hampered by the unknown parameters s, $a_1,\dots ,a_s$. In light of [23], we propose to replace s in (1) with a predetermined number of segments, denoted by $p_n$, to facilitate the estimation of the regression coefficients. This replacement allows us to identify the total number of change points. Specifically, we partition the data sequence into $p_n$ segments, where $p_n\to \infty$ as $n\to \infty$. This ensures that all segments excluding the first segment have length m, while the length of the first segment is $n-(p_n-1)m$. Let ${\mathcal{Q}}_1=$$\left\{1,2,\dots ,n-\left(p_n-1\right)m\right\}$ and ${\mathcal{Q}}_l=\left\{n-\left(p_n-l+1\right)m+1,\dots ,n-\left(p_n-l\right)m\right\}, l=2,\dots ,p_n$. See (3) for more details on segmentation. We assume that $m<min(a_{j+1}-a_j)/2$ for $j=1,\cdots ,s$, so that each segment ${\mathcal{Q}}_l$, for $l=1,\cdots ,p_n$, contains at most one change point.

We denote the segment as $k_j$ if $a_j\in {\mathcal{Q}}_{k_j}$. This partitioning results in the following model:

$$\begin{array}{cc}\hfill y_i& ={\mathit{x}}_i^{\top }\left[{\mathit{\beta }}_1+\sum _{l=2}^{p_n}{\mathit{d}}_{l}I\left(n-\left(p_n-l+1\right)m<i\le n\right)-{\mathit{\omega }}_i\right]+{\epsilon }_i,\hfill \end{array}$$ (2)

where

$$\left\{\begin{array}{cc}{\mathit{d}}_{k_j}={\mathit{\delta }}_j\ne \mathbf{0},\hfill & \mathrm{for}j=1,\dots ,s,\hfill \\ {\mathit{d}}_l=\mathbf{0},\hfill & \mathrm{for}\mathrm{any}l\notin \left\{k_1,\dots ,k_s\right\},\hfill \end{array}\right.$$

and ${\mathit{\omega }}_i={\mathit{d}}_{k_j}I(i\in {\mathcal{T}}_j)$ with ${\mathcal{T}}_j=\left\{n-\left(p_n-k_j+1\right.\right.$) $\left.m+1,\dots ,a_j\right\}$. ${\mathit{\omega }}_i=\mathbf{0}$ for all $i\notin {\cup }_{j=1}^s{\mathcal{Q}}_{k_j}$.

Remark 1.

By the definition of m, any two segments also contain at most one change point. From (2), it follows that

$$\begin{array}{cc}\hfill y_i& =\left\{\begin{array}{cc}{\mathit{x}}_i^{\top }\left[{\mathit{\beta }}_1+\sum _{l=1}^{j-1}{\mathit{\delta }}_l\right]+{\epsilon }_i,\hfill & \mathrm{if}i\in {\mathcal{Q}}_{k_j-1},\hfill \\ {\mathit{x}}_i^{\top }\left[{\mathit{\beta }}_1+\sum _{l=1}^j{\mathit{\delta }}_l-{\mathit{\omega }}_i\right]+{\epsilon }_i,\hfill & \mathrm{if}i\in {\mathcal{Q}}_{k_j},\hfill \\ {\mathit{x}}_i^{\top }\left[{\mathit{\beta }}_1+\sum _{l=1}^j{\mathit{\delta }}_l\right]+{\epsilon }_i,\hfill & \mathrm{if}i\in {\mathcal{Q}}_{k_j+1}.\hfill \end{array}\right.\hfill \end{array}$$

Based on the partition, we can obtain least-squares estimates ${\widehat{\mathit{\gamma }}}_l$ for $l=k_j-1,k_j,k_j+1$ within each segment.

• 1.  When $a_j=n-(p_n-k_j)m$, the change point coincides with the pre-specified cut-point and ${\mathit{\omega }}_i={\mathit{\delta }}_j$. The regression coefficients corresponding to the three segments are denoted by ${\mathit{\gamma }}_l,l=k_j-1,k_j,k_j+1$, and are equal to ${\mathit{\beta }}_1+{\sum }_{l=1}^{j-1}{\mathit{\delta }}_l$, ${\mathit{\beta }}_1+{\sum }_{l=1}^{j-1}{\mathit{\delta }}_l$, and ${\mathit{\beta }}_1+{\sum }_{l=1}^j{\mathit{\delta }}_l$, respectively. The segment ${\mathcal{Q}}_{k_j}$ that contains the change point $a_j$ can be identified by ${\widehat{\mathit{\gamma }}}_{k_j+1}-{\widehat{\mathit{\gamma }}}_{k_j}\ne \mathbf{0}$.
• 1.  When $a_j<n-(p_n-k_j)m$, due to the existence of ${\mathit{\omega }}_i$, the linear regression model is misspecified on ${\mathcal{Q}}_{k_j}$, causing ${\widehat{\mathit{\gamma }}}_{k_j}$ to converge to the pseudo-true value ${\mathit{\gamma }}_{k_j}$ under model misspecification [24,25]. Since ${\mathit{\gamma }}_{k_j}$ is different from both ${\mathit{\beta }}_1+{\sum }_{l=1}^{j-1}{\mathit{\delta }}_l$ and ${\mathit{\beta }}_1+{\sum }_{l=1}^j{\mathit{\delta }}_l$, ${\mathcal{Q}}_{k_j}$ can be determined by the following: ${\widehat{\mathit{\gamma }}}_{k_j}-{\widehat{\mathit{\gamma }}}_{k_j-1}\ne \mathbf{0}$ and ${\widehat{\mathit{\gamma }}}_{k_j+1}-{\widehat{\mathit{\gamma }}}_{k_j}\ne \mathbf{0}$.

According to Remark 1, we reformulate the change-point detection problem as a high-dimensional variable selection task by constructing differences between coefficients. Therefore, (2) can be rewritten in matrix form as follows:

$${\mathit{y}}_n={\mathit{X}}_n\mathit{\theta }+{\mathit{e}}_n+{\mathit{\epsilon }}_n,$$ (3)

where ${\mathit{y}}_n={(y_1,y_2,\dots ,y_n)}^{\top }$,

$${\mathit{X}}_n=\left({\mathit{X}}^{\left(1\right)},\dots ,{\mathit{X}}^{\left(p_n\right)}\right)=\left(\begin{array}{cccc}{\mathbf{X}}_{\left(1\right)}& \mathbf{0}& \cdots & \mathbf{0}\\ {\mathbf{X}}_{\left(2\right)}& {\mathbf{X}}_{\left(2\right)}& \cdots & \mathbf{0}\\ ⋮& ⋮& \ddots & ⋮\\ {\mathbf{X}}_{\left(p_n\right)}& {\mathbf{X}}_{\left(p_n\right)}& \cdots & {\mathbf{X}}_{\left(p_n\right)}\end{array}\right),$$

${\mathit{X}}_{\left(1\right)}={({\mathit{x}}_1,\cdots ,{\mathit{x}}_{n-(p_n-1)m})}^{\top }$ and ${\mathit{X}}_{\left(j\right)}={({\mathit{x}}_{n-(p_n-j+1)m+1},\dots ,{\mathit{x}}_{n-(p_n-j)m})}^{\top }$. In addition, the definitions of $\mathit{\theta }={({\mathit{\theta }}_1^{\top },\dots ,{\mathit{\theta }}_{p_n}^{\top })}^{\top }={\left({\mathit{\beta }}_1^{\top },{({\mathit{\gamma }}_2-{\mathit{\beta }}_1)}^{\top },\dots ,{({\mathit{\gamma }}_{p_n}-{\mathit{\gamma }}_{p_n-1})}^{\top }\right)}^{\top }$, ${\mathit{\epsilon }}_n={({\epsilon }_1,\dots ,{\epsilon }_n)}^{\top }$, and ${\mathit{e}}_n$ represents the artificial n-dimensional error due to model misspecification, whose elements are equal to zero for all $i\notin {\cup }_{j=1}^s{\mathcal{Q}}_{k_j}$.

It can be seen that ${\mathit{\theta }}_l=\mathbf{0}$ if $l\ne 1,k_j,$ or $k_j+1$ for $j=1,\dots ,s$. Let ${\mathcal{A}}_n=\{1,k_1,k_1+1,k_2,k_2+1,\dots ,k_s,k_s+1\}$. It is important to note that ${\mathit{\theta }}_l={\mathbf{0}}_{q\times 1}$ is evident for all $l\notin {\mathcal{A}}_n$. Therefore, the estimation of ${\mathcal{A}}_n$ comes down to identifying the non-zero elements from $\mathit{\theta }$, which is the focus of the subsequent subsections.

2.1. Segment Selection

By the definition of m, each segment ${\mathcal{Q}}_{k_j}$ contains a change point. In this subsection, our goal is to select segments ${\mathcal{Q}}_{k_j}$ for $j=1,\cdots ,s$. From the above statement, we can see that selecting all non-zero elements of $\mathit{\theta }$ can yield the estimation of ${\mathcal{A}}_n$ effectively. Therefore, after the cutting stage, the problem of detecting multiple change points is reduced to a variable selection problem in high-dimensional scenarios. We use the OGA + HDIC + Trim method for the segment selection.

We rewrite the model (3) as follows:

$${\mathit{y}}_n=\sum _{j=1}^{r_n}{\mathit{z}}_j{\theta }_j+{\tilde{\mathit{\epsilon }}}_n,$$ (4)

where $r_n=p_n·q$, $\{{\mathit{z}}_1,\dots ,{\mathit{z}}_{r_n}\}$ are all the column vectors of ${\mathit{x}}_n$, ${\tilde{\mathit{\epsilon }}}_n={\mathit{e}}_n+{\mathit{\epsilon }}_n$ and $\mathit{\theta }={({\theta }_1,\cdots ,{\theta }_{r_n})}^{\top }$. Without loss of generality, replace ${\mathit{y}}_n$ by ${\mathit{y}}_n-{\mathit{y}}_n^{\top }{\mathbf{1}}_n$ and ${\mathit{z}}_j$ by ${\mathit{z}}_j-{\mathit{z}}_j^{\top }{\mathbf{1}}_n$. Define ${\widehat{\sigma }}_{J_d}^2=n^{-1}{\mathit{y}}_n^{\top }({\mathit{I}}_n-{\mathit{H}}_{J_d}){\mathit{y}}_n$, where ${\mathit{H}}_{J_d}$ is the orthogonal projection matrix onto the linear span of $\{{\mathit{z}}_j,j\in J_d\}$. For convenience, denote ${\mathit{H}}_{\varnothing }=\mathbf{0}$. The proposed OGA+HDIC+Trim algorithm is outlined in Algorithm 1.

Algorithm 1 OGA + HDIC + Trim

Require: response vector ${\mathit{y}}_n\in {\mathbb{R}}^n$, regressor matrix ${\mathit{X}}_n\in {\mathbb{R}}^{n\times r_n}$.  Initialzation: set $d=0$, ${\mathit{r}}^{\left(0\right)}={\mathit{y}}_n$ and ${\widehat{J}}_0=\varnothing$.  While $d=0,\dots ,D_n$ do     Compute ${\widehat{j}}_{d+1}=arg{max}_{1\le j\le r_n,j\notin {\widehat{J}}_{d-1}}|{\mathit{z}}_j^{\top }{\mathit{r}}^{\left(d\right)}|/(n^{1/2}\parallel {\mathit{z}}_j\parallel )$ and update ${\widehat{J}}_{d+1}={\widehat{J}}_d\bigcup {\widehat{j}}_{d+1}$;     Compute ${\mathit{z}}_{{\widehat{j}}_{d+1}}^{\perp }$ via ${\mathit{z}}_{{\widehat{j}}_{d+1}}^{\perp }=({\mathit{I}}_n-{\sum }_{\ell =1}^d{\mathit{H}}_{{\widehat{j}}_{\ell }}^{\perp }){\mathit{z}}_{{\widehat{j}}_{d+1}}$, where ${\mathit{H}}_{{\widehat{j}}_{\ell }}^{\perp }={\mathit{z}}_{{\widehat{j}}_{\ell }}{\mathit{z}}_{{\widehat{j}}_{\ell }}^{\top }/{\parallel {\mathit{z}}_{{\widehat{j}}_{\ell }}\parallel }^2$;     Compute ${\mathit{r}}^{(d+1)}$ via ${\mathit{r}}^{(d+1)}=({\mathit{I}}_n-{\mathit{H}}_{{\widehat{j}}_{d+1}}^{\perp }){\mathit{r}}^{\left(d\right)}$.  end     Compute the minimum of HDIC via $$\begin{array}{cc}\hfill \mathrm{HDIC}\left({\widehat{J}}_d\right)& =log{\widehat{\sigma }}_{{\widehat{J}}_d}^2+♯\left({\widehat{J}}_d\right)c_{n}log\left(r_n\right)/n,\hfill \\ \hfill {\widehat{d}}_n& =arg\underset{1\le d\le D_n}{min}\mathrm{HDIC}\left({\widehat{J}}_d\right).\hfill \end{array}$$ (5)  return ${\widehat{\mathcal{J}}}_n$ via $${\widehat{\mathcal{J}}}_n=\left\{\begin{array}{cc}\{{\widehat{j}}_{\ell }:\mathrm{HDIC}\left({\widehat{J}}_d-\left\{{\widehat{j}}_{\ell }\right\}\right)>\mathrm{HDIC}\left({\widehat{J}}_d\right),1\le \ell \le {\widehat{d}}_n\},\hfill & \mathrm{if}{\widehat{d}}_n>1;\hfill \\ \left\{{\widehat{j}}_1\right\},\hfill & \mathrm{if}{\widehat{d}}_n=1.\hfill \end{array}\right.$$

In this context, the value of $D_n$ denotes the maximum number of iterations, and the convergence rate theory of OGA in [22] shows that $D_n=O\left(n^{1/2}{(log{r}_n)}^{-1/2}\right)$. The parameter $c_n$ satisfies the conditions $c_n\to \infty$ and $c_{n}log{r}_n=o\left(n^{1-2\gamma }\right)$, where $\gamma \in [0,1)$. Let $\widehat{\mathit{\theta }}={({\widehat{\mathit{\theta }}}_1^{\top },\dots ,{\widehat{\mathit{\theta }}}_{p_n}^{\top })}^{\top }$ be the estimate obtained by applying the OGA+HDIC+Trim procedure. Therefore, we derive an estimate of ${\mathcal{A}}_n$ from ${\widehat{\mathcal{J}}}_n$ as ${\widehat{\mathcal{A}}}_n=\{l:{\widehat{\mathit{\theta }}}_l\ne \mathbf{0},l=1,\dots ,p_n\}.$ Denote

$${\widehat{\mathcal{C}}}_n=\left\{l:l\in {\widehat{\mathcal{A}}}_n,l+1\notin {\widehat{\mathcal{A}}}_n,l=2,\dots ,p_n+1\right\}=\left\{{\widehat{k}}_1,\dots ,{\widehat{k}}_{\widehat{s}}\right\},$$ (6)

where ${\widehat{k}}_1<\cdots <{\widehat{k}}_{\widehat{s}}$. It is clear that if $l+1\notin {\widehat{\mathcal{A}}}_n,l\in {\widehat{\mathcal{A}}}_n$ and $l-1\in {\widehat{\mathcal{A}}}_n$, then $l\in {\widehat{\mathcal{C}}}_n$ and $l-1\notin {\widehat{\mathcal{C}}}_n$. Let ${\mathcal{Q}}_{\left(j\right)}={\mathcal{Q}}_{k_j}\cup {\mathcal{Q}}_{k_j+1}$. It only includes the change point $a_j$ since $m<{min}_j(a_{j+1}-a_j)/2$, and it ensures that no change point overlaps with the cut-point of ${\mathcal{Q}}_{\left(j\right)}$. By following the steps outlined above and considering that the Wald test cannot detect the location of the change point at the partition boundary, we derive the following selected segments:

$${\widehat{\mathcal{Q}}}_{\left(j\right)}={\widehat{\mathcal{Q}}}_{{\widehat{k}}_j}\bigcup {\widehat{\mathcal{Q}}}_{{\widehat{k}}_j+1}=\{n-(p_n-{\widehat{k}}_j+1)m+1,n-(p_n-{\widehat{k}}_j-1)m\},$$ (7)

where ${\widehat{\mathcal{Q}}}_{{\widehat{k}}_j}=\{n-(p_n-{\widehat{k}}_j+1)m+1,\cdots ,n-(p_n-{\widehat{k}}_j)m\}$.

2.2. Refining

By Theorem 1 in Section 4, $\widehat{s}$ converges to s as $n\to \infty$. Hence, we assume that for a large n, there exists ${\widehat{\mathcal{Q}}}_{\left(j\right)}$ such that $a_j\in {\widehat{\mathcal{Q}}}_{\left(j\right)}$. We now show how to estimate this change point. Note that for $i\in {\widehat{\mathcal{Q}}}_{\left(j\right)}=\{n-(p_n-{\widehat{k}}_j+1)m+1,n-(p_n-{\widehat{k}}_j-1)m\}$, we have

$$y_i={\mathit{x}}_i^{\top }\left[{\mathit{\beta }}_j+{\mathit{\delta }}_{j}I(a_j<i\le n_j^{\left(r\right)})\right]+{\epsilon }_i,n_j^{\left(l\right)}\le i\le n_j^{\left(r\right)},$$ (8)

where $n_j^{\left(l\right)}=n-(p_n-{\widehat{k}}_j+1)m+1$ and $n_j^{\left(r\right)}=n-(p_n-{\widehat{k}}_j-1)m$. Here, ${\mathit{\delta }}_j={\mathit{\beta }}_{j+1}-{\mathit{\beta }}_j$ and ${\mathit{\beta }}_j$ and ${\mathit{\beta }}_{j+1}$ are unknown q-dimensional vectors of the regression coefficients on the line segment $\{n_j^{\left(l\right)},a_j\}$ and $\{a_j,n_j^{\left(r\right)},\}$, respectively. $n_j^{\left(r\right)}-n_j^{\left(l\right)}+1=2m$.

We compute the sup-Wald test statistics [26] and estimate $a_j$ by

$${\widehat{a}}_{j,h}=arg\underset{h}{max}{\widehat{\mathit{\delta }}}_{j;h}^{\top }\left({\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h}\right){\widehat{\mathit{\delta }}}_{j;h},n_j^{\left(l\right)}+q<h<n_j^{\left(r\right)}-q,$$ (9)

where ${\mathit{Z}}_{j,h}={(\mathbf{0},\cdots ,\mathbf{0},{\mathit{x}}_{h+1},{\mathit{x}}_{h+2},\cdots ,{\mathit{x}}_{n_j^{\left(r\right)}})}^{\top }$, ${\mathit{X}}_j={({\mathit{x}}_{n_j^{\left(l\right)}},{\mathit{x}}_{n_j^{\left(l\right)}+1}.\cdots ,{\mathit{x}}_{n_j^{\left(r\right)}})}^{\top }$, and ${\mathit{M}}_j={\mathit{I}}_n-{\mathit{X}}_j{\left({\mathit{X}}_j^{\top }{\mathit{X}}_j\right)}^{-1}{\mathit{x}}_j^{\top }$. We also obtain the estimates $({\widehat{\mathit{\beta }}}_{j;h},{\widehat{\mathit{\delta }}}_{j;h})$ of $({\mathit{\beta }}_j,{\mathit{\delta }}_j)$ by regressing ${\mathit{y}}_{\left(j\right)}={(y_{n_j^{\left(l\right)}},\cdots ,y_{n_j^{\left(r\right)}})}^{\top }$ on ${\mathit{X}}_j$ and ${\mathit{Z}}_{j,h}$, respectively. The limiting behavior of ${\widehat{a}}_{j,h}$ is given in Section 4.

Since the multiple change points are not dense in the data sequence, without loss of generality, we assume that $m<min(a_{j+1}-a_j)/2,$$j=1,\dots ,s-1$, so that each segment ${\mathcal{Q}}_{\left(j\right)}$ contains at most one change point. Note that if m is too small, it leads to inconsistency in estimating the regression parameters and increases the computational time. Therefore, we need to avoid choosing too small a value for m. To address this issue, we define $m=\lceil c_0\sqrt{n}\rceil$ according to Theorem 2 in Section 3, where $c_0$ serves as a tuning parameter, and $\lceil ·\rceil$ is the ceiling function. We study the range of values of $c_0$ on the interval $[0.1,1.5]$. The final value of m is determined using the Bayesian Information Criterion (BIC) as follows:

$$\widehat{m}=arg\underset{m}{min}log\left\{\sum _{i=1}^n{\left(y_i-{\mathit{x}}_i^{\top }\left[{\widehat{\mathit{\beta }}}_{1,h}+\sum _{j=1}^{\widehat{s}}{\widehat{\mathit{\delta }}}_{j,h}I({\widehat{a}}_{j,h}<i\le n)\right]\right)}^2\right\}+\widehat{s}·qlogn.$$

3. Bootstrap Confidence Intervals for Multiple Change Points

In this section, we construct bootstrap confidence intervals for multiple change points. It helps us to study the behavior of multiple change points in a linear regression model. Obviously, the two-stage procedure inherently involves the quantification of uncertainty in the number of change points and their respective locations.

We define the estimated residuals ${\widehat{\epsilon }}_i$ and the centered residuals ${\tilde{\epsilon }}_i$ as follows:

$$\begin{array}{cc}\hfill {\widehat{\epsilon }}_i& =y_i-{\mathit{x}}_i^{\top }\left[{\widehat{\mathit{\beta }}}_{1,h}+\sum _{l=1}^{\widehat{s}}{\widehat{\mathit{\delta }}}_{l,h}I({\widehat{a}}_{l,h}<i\le n)\right],\hfill \\ \hfill {\tilde{\epsilon }}_i& ={\widehat{\epsilon }}_i-{\displaystyle \frac{1}{{\widehat{a}}_{j,h}-{\widehat{a}}_{j-1,h}}}\sum _{l={\widehat{a}}_{j-1,h}+1}^{{\widehat{a}}_{j,h}}{\widehat{\epsilon }}_l,{\widehat{a}}_{j-1,h}<i\le {\widehat{a}}_{j,h},\hfill \end{array}$$ (10)

where ${\widehat{a}}_{0,h}=1$ and ${\widehat{a}}_{\widehat{s}+1,h}=n$. Let ${\epsilon }_{{\widehat{a}}_{j-1,h}+1}^{\ast },\dots ,{\epsilon }_{{\widehat{a}}_{j,h}}^{\ast }$ be independently and identically distributed (i.i.d.) random variables sampled from the empirical distribution function of $\{{\tilde{\epsilon }}_{{\widehat{a}}_{j-1,h}+1},\dots ,{\tilde{\epsilon }}_{{\widehat{a}}_{j,h}}\}$. We then consider the bootstrap observations defined as follows:

$$\begin{array}{cc}\hfill y_i^{\ast }& ={\mathit{x}}_i^{\top }\left[{\widehat{\mathit{\beta }}}_{1,h}+\sum _{l=1}^{\widehat{s}}{\widehat{\mathit{\delta }}}_{l,h}I({\widehat{a}}_{l,h}<i\le n)\right]+{\epsilon }_i^{\ast },i=1,\dots ,n,\hfill \end{array}$$ (11)

where ${\epsilon }_i^{\ast }$ represents the bootstrapped version of the residuals. To obtain an approximation to the distribution of ${\widehat{a}}_{j;h}$ in (9), a bootstrap statistic of the estimate is defined as

$${\widehat{a}}_{j^{\ast };h_b}^{\ast }=arg\underset{h_b}{max}{\widehat{\mathit{\delta }}}_{j^{\ast };h_b}^{\ast \top }\left({\mathit{z}}_{j^{\ast },h_b}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j^{\ast },h_b}\right){\widehat{\mathit{\delta }}}_{j^{\ast };h_b}^{\ast },j^{\ast }=1,\dots ,{\widehat{s}}^{\ast },$$ (12)

where the number of change points ${\widehat{s}}^{\ast }$ and the ${\widehat{\mathcal{C}}}_n^{\ast }$ are obtained by performing OGA on ${\mathit{y}}_n^{\ast }={(y_1^{\ast },\dots ,y_n^{\ast })}^{\top }$ as in (6). The data sequence segmentation remains unchanged. In this case, ${\mathit{Z}}_{j^{\ast },h_b}={(\mathbf{0},\dots ,\mathbf{0},{\mathit{x}}_{h_b+1},\dots ,{\mathit{x}}_{n_{j^{\ast }}^{\left(r\right)}})}^{\top }$, and $\left({\widehat{\mathit{\beta }}}_{j^{\ast };h_b}^{\ast },{\widehat{\mathit{\delta }}}_{j^{\ast };h_b}^{\ast }\right)$ represents the least-squares estimates obtained by regressing ${\mathit{y}}_{\left(j^{\ast }\right)}^{\ast }={(y_{n_{j^{\ast }}^{\left(l\right)}}^{\ast },\dots ,y_{n_{j^{\ast }}^{\left(r\right)}}^{\ast })}^{\top }$ on ${\mathit{X}}_{j^{\ast }}$ and ${\mathit{Z}}_{j^{\ast },h_b}$. Next, we describe the construction of bootstrap CIs for multiple change points.

• 1.We generate a bootstrap sample ${(y_1^{\ast },\dots ,y_n^{\ast })}^{\top }$ by randomly sampling residuals ${({\epsilon }_1^{\ast },\dots ,{\epsilon }_n^{\ast })}^{\top }$ from the set $\{{\tilde{\epsilon }}_1,\dots ,{\tilde{\epsilon }}_n\}$ as in (11);
• 2.We apply the two-stage procedure and compute the local maximizer obtained as in (12) for each estimated segment;
• 3.For a given bootstrap sample size B, we repeat Steps 1-2 B times and record ${\widehat{a}}_{j^{\ast };h_b}^{\ast \left(b\right)}$, $j^{\ast }=1,\cdots ,{\widehat{s}}^{\ast }$, where $b=1,\dots ,B$.

Therefore, the bootstrap-based approximation for the change point $a_j$ can be constructed. Generally, for any $\alpha \in (0,1)$, the bootstrap $100(1-\alpha )\%$ confidence interval for the change point $a_j$ is given by the following:

$${\mathrm{CIs}}_j\left(\alpha \right)=\left[{\widehat{a}}_{j,h}+q_U(\alpha /2),{\widehat{a}}_{j,h}+q_L(\alpha /2)\right],$$ (13)

where $q_U(\alpha /2)=sup\left\{x;{\displaystyle \frac{1}{B}}{\sum }_{b=1}^{B}I\left({\widehat{a}}_{j^{\ast },h_b}^{\ast \left(b\right)}-{\widehat{a}}_{j;h}\le x\right)\le \alpha /2\right\}$, and $q_L(\alpha /2)=inf\left\{x;{\displaystyle \frac{1}{B}}{\sum }_{b=1}^{B}I\left({\widehat{a}}_{j^{\ast },h_b}^{\ast \left(b\right)}-{\widehat{a}}_{j;h}\ge x\right)\ge 1-\alpha /2\right\}.$

If ${\widehat{\mathcal{C}}}_n\subseteq {\widehat{\mathcal{C}}}_n^{\ast }$, then ${\widehat{a}}_{j;h_b}^{\ast }$ is an estimate of ${\widehat{a}}_{j;h}$ for $j=1,\dots ,\widehat{s}$. If some elements of ${\widehat{\mathcal{C}}}_n$ are not in the set ${\widehat{\mathcal{C}}}_n^{\ast }$, then ${\widehat{a}}_{j;h}$ has no corresponding bootstrap estimate for some j. Hence, the bootstrap CI of ${\widehat{a}}_{j;h}$ is constructed using $\{{\widehat{a}}_{j;h_b}^{\ast \left(b\right)}:{\widehat{a}}_{j;h_b}^{\ast \left(b\right)}\notin \varnothing ,b=1,\dots ,B^{\ast }\}$ instead of $\{{\widehat{a}}_{j^{\ast };h_b}^{\ast \left(1\right)},\dots ,{\widehat{a}}_{j^{\ast };h_b}^{\ast \left(B\right)}\}$ in (13), which yields

$${\mathrm{CIs}}_j^{\ast }\left(\alpha \right)=\left[{\widehat{a}}_{j,h}+q_U^{\ast }(\alpha /2),{\widehat{a}}_{j,h}+q_L^{\ast }(\alpha /2)\right],$$ (14)

where

$$q_U^{\ast }(\alpha /2)=sup\left\{x:{\displaystyle \frac{1}{B^{\ast }}}\sum _{b=1}^{B^{\ast }}I\left({\widehat{a}}_{j,h_b}^{\ast \left(b\right)}-{\widehat{a}}_{j;h}\le x\right)\le \alpha /2\right\},$$

and

$$q_L^{\ast }(\alpha /2)=inf\left\{x:{\displaystyle \frac{1}{B^{\ast }}}\sum _{b=1}^{B^{\ast }}I\left({\widehat{a}}_{j,h_b}^{\ast \left(b\right)}-{\widehat{a}}_{j;h}\ge x\right)\ge 1-\alpha /2\right\}.$$

4. Theoretical Validity of the Bootstrap Confidence Intervals

To investigate the performance of the bootstrap CIs for multiple change points, we make the following assumptions.

Assumption 1.

If $s⩾1$, then $a_j/n\to {\tau }_j>0$ for $1\le j\le s.$ Furthermore, if $s⩾2$, then ${min}_{1\le j\le s-1}\left({\tau }_{j+1}-{\tau }_j\right)>0$.

Assumption 2.

$\left\{{\epsilon }_i,i=1,2,\dots ,n\right\}$ is a sequence of independently and identically distributed random variables with $E\left({\epsilon }_i\right)=0$ and $E\left({\epsilon }_i^2\right)={\sigma }^2$.

Assumption 3.

$a_j-n_j^{\left(l\right)}=\lceil 2{\tau }_{j}m\rceil$, where ${\tau }_j\in (0,1)$ and $\lceil ·\rceil$ is the ceilling function.

Assumption 4.

${sup}_{(t_2-t_1)\ge 1}∥{\sum }_{i=t_1}^{t_2}{\mathit{x}}_i{\mathit{x}}_i^{\top }/(t_2-t_1)∥$ is stochastically bounded. ${\epsilon }_i$ is independent of the regressor ${\mathit{x}}_j$ for all i and j.

Assumption 5.

${\mathit{\delta }}_j\to \mathbf{0}$, and ${\mathit{\delta }}_j^{-1}{\left(2m\right)}^{-1/2+\alpha }=o\left(1\right)$ for some $\alpha \in (0,1/2)$ as $n\to \infty$.

By Assumption 1, it follows that $m/(a_{j+1}-a_j)\to 0$, i.e., there is at most one change point in each segment for a large n. Assumption 2 follows that the residuals are independent and identically distributed and justifies the use of bootstrapping to estimate the central error, helping generate the sample distribution of the change points. Assumption 3 assumes that the shift point is bounded from the endpoints for asymptotic purposes. Assumption 4 requires that there is enough data around the change point and at the beginning and end of the sample so that the change point can be identified. The asymptotic distribution of ${\widehat{a}}_{j,h}$ depends on various unknown quantities, with the magnitude of the change ${\mathit{\delta }}_j$ being the most significant. Assumption 5 is the minimum signal amplitude of the regression coefficient in the high-dimensional setting. The shift amplitude cannot be too small; otherwise, the change point will not be identified. Based on this, we give the necessary Assumption 5. Next, we establish the consistency of the number of change points $\widehat{s}$ and the change point estimator ${\widehat{a}}_{j,h}$. The following theorem provides the consistency of the estimated number of change points.

Theorem 1.

Suppose that $m\to \infty$, $p_n\to \infty$, $D_n=O\left({(n/log\left(r_n\right))}^{1/2}\right)$, and $log\left(r_n\right)/n\to 0$ as $n\to \infty$. Under Assumptions 1–5, we have

$$\begin{array}{cc}\hfill & \underset{n\to \infty }{lim}P(\widehat{s}=s)=1;\hfill \\ \hfill & \underset{n\to \infty }{lim}P\left(a_j\in {\widehat{\mathcal{Q}}}_{\left(j\right)},j=1,\dots ,s\mid \widehat{s}=s\right)=1,\hfill \end{array}$$ (15)

where $\widehat{s}$ and ${\widehat{k}}_j,j=1,\dots ,\widehat{s}$ are given in (6).

Theorem 1 extends Theorem 4 in [22] to the multiple change points detection case. The consistency and asymptotic distribution of change points estimators are given below.

Theorem 2.

If ${\sum }_{i=t_1}^{t_2}{\mathit{x}}_i{\mathit{x}}_i^{\top }/\left(t_1-t_2\right){\to }_{p}V$ as $t_2-t_1\to \infty$, under Assumptions 1–4, we have that when $n\to \infty$, ${\widehat{a}}_{j,h}-a_j=O_p(\parallel {\mathit{\delta }}_j{\parallel }^{-2})$ and

$${\displaystyle \frac{{\mathit{\delta }}_j^{\top }V{\mathit{\delta }}_j}{{\sigma }^2}}({\widehat{a}}_{j,h}-a_j){\to }_{d}argmax\{W\left(c\right)-\left|c\right|/2:c\in \mathcal{R}\},$$

where V is a strictly positive definite matrix, $\left\{W\right(c):c\in \mathcal{R}\}$ is a two-sided Wiener process, and ${\mathit{\delta }}_j$ is a fixed value or satisfies ${\mathit{\delta }}_j\to 0$, as specified in Assumption 5.

Subsequently, we establish the validity of the bootstrap CIs in (14). For future reference, we define some notations here. Any symbol with a superscript ∗ denotes an object under the bootstrap probability measure, rather than the original measure used in some of the other sections. For example, $E^{\ast }(·)$ denotes the conditional expectation with respect to the bootstrap probability measure conditional on the original data. Similarly, $P^{\ast }(·)$ denotes the conditional probability under the bootstrap measure.

Theorem 3.

Under the assumptions of Theorem 2, we have

$$\underset{x\in \mathcal{R}}{sup}|P^{\ast }({\widehat{a}}_{j,h_b}^{\ast }-{\widehat{a}}_{j,h}\le x)-P({\widehat{a}}_{j,h}-a_j\le x)|{\to }_{p}0.$$

The proofs of Theorems 2 and 3 are given in Appendix A.

Combining Theorem 3 with Theorems 1 and 2 establishes the validity of the bootstrap method for multiple change points.

Corollary 1.

Under Assumptions 1–5, we have that, as $n\to \infty$,

$$\underset{x\in \mathcal{R}}{sup}\left|P^{\ast }\left({\cap }_{j=1}^s\left\{{\widehat{a}}_{j,h_b}^{\ast }-{\widehat{a}}_{j,h}\le x\right\}\right)-P\left({\cap }_{j=1}^s\left\{{\widehat{a}}_{j,h}-a_j\le x\right\}\right)\right|{\to }_{p}0.$$

Since $\mathrm{P}\left(a_j\in {\mathrm{CIs}}_j^{\ast }\left(\alpha \right)\right)\to 1-\alpha$ for each $j=1,\dots ,s$, by the Bonferroni correction, we have $\mathrm{P}\left({\cap }_{j=1}^s\left\{a_j\in {\mathrm{CIs}}_j^{\ast }(\alpha /s)\right\}\right)\to 1-\alpha$. The asymptotic validity of the proposed bootstrap CIs in (14) follows.

5. Simulation

In this section, we first present the simulation results for change point detection given in Section 2 and compare them with the two-stage multiple change point detection procedure involving LASSO (TSMCD_lasso) by [10]. We also construct confidence intervals using the bootstrap method proposed in Section 3. We denote the CIs in (14) by bootstrap_oga,wald.

5.1. Detection of Multiple Change Points

We consider the simulation setting where the change points $a_j$, $j=1,2,3$ are, respectively, 150, 300, and 450, respectively, and generate data from the model

$$\begin{array}{cc}\hfill y_t=& 2cos(t\pi /30)+2sin(t\pi /30)+0.1y_{t-1}\hfill \\ \hfill & +(-3cos(t\pi /30)+sin(t\pi /30)+0.2y_{t-1})I_{(150,600]}\left(t\right)\hfill \\ \hfill & +(2cos(t\pi /30)-0.3y_{t-1})I_{(300,600]}\left(t\right)\hfill \\ \hfill & +(2cos(t\pi /30)+2sin(t\pi /30))I_{(450,600]}\left(t\right)+{\epsilon }_t,\hfill \end{array}$$ (16)

where ${\epsilon }_1,\dots ,{\epsilon }_n$ are independent and follow the standard normal distribution. The simulated data are shown in Figure 1. In this model, $\{y_t,t=1,\cdots ,600\}$ represents a periodic autocorrelation sequence with a period of 30 and an autocorrelation order of 1.

Figure 1.

Simulated regression data with change points indicated by dotted lines.

We perform 1000 Monte Carlo simulations for multiple change point estimation in Table 1. According to [22], we take $c_n=2$ in (5), similar to AIC. We focus on counting the number of events for which $\left\{\right|{\widehat{a}}_j-a_j|\le 5\}$. The percentage of the correct identifications of all change points, denoted as $c_{\mathrm{all}}(\%)$, reflects the proportion of replicates for which $|{\widehat{a}}_j-a_j|\le 5$ for all j. In addition, $c_j(\%)$ represents the proportion of replicates for which $|{\widehat{a}}_j-a_j|\le 5$. The mean and standard error of the estimated change points are calculated for the replicates for which the difference between the estimated change points and the true value is less than or equal to 50 (i.e., $|{\widehat{a}}_j-a_j|\le 50$).

Table 1.

Performance of different methods for multiple change point detection.

Method	${\mathit{c}}_{\mathbf{all}}$		${\mathit{c}}_1$	${\mathit{c}}_2$	${\mathit{c}}_3$
TSPoga,wald	90.70		98.50	96.80	97.80
		Mean	150.34	300.41	449.77
		SE	1.66	2.31	2.22
TSMCDlasso	72.60		95.20	95.80	96.40
		Mean	150.61	300.43	450.16
		SE	2.42	2.31	2.19

From Table 1, we can see that TSP_oga,wald generally outperforms TSMCD_lasso in terms of yielding a correct identification rate. It is noteworthy that TSMCD_lasso performs significantly worse than TSP_oga,wald in identifying all change points, especially at $a_1$, as evidenced by the lower $c_1$ value. TSP_oga,wald and TSMCD_lasso show comparable estimation accuracy in terms of the mean and standard error (SE) of the estimated change points.

5.2. Bootstrap CIs

All results are based on 500 realizations in the simulation setting, and we use $B=500$ to give the corresponding bootstrap CIs. We assume the confidence level is $(1-\alpha )\in \{0.9,0.95\}$. For each j, the coverage of the bootstrap CI is calculated as the proportion of simulated realizations where ${\mathrm{CIs}}_j^{\ast }\left(\alpha \right)$ contains $a_j$. For all j, the coverage of the bootstrap CIs is calculated as the proportion of simulated realizations, where ${\cup }_{j=1}^s{\mathrm{CIs}}_j^{\ast }(\alpha /s)$ contains all $a_j$.

Table 2 confirms the effectiveness of our bootstrap method, as the empirical coverage probability for each $a_j$ is close to the nominal level. The overall coverage of the bootstrap confidence intervals for all j is slightly higher than the nominal level, which may be due to the complexity of multiple change point detection. The average computational time for the bootstrap_oga,wald procedure is 1.44 min per Monte Carlo replication, as measured on an Intel(R) Core(TM) i9-14900K processor (3.20 GHz) with 64 GB of RAM.

Table 2.

Performance of bootstrap_oga,wald.

$(1-\mathit{\alpha })$%	${\mathit{a}}_{\mathbf{all}}$	${\mathit{a}}_1$	${\mathit{a}}_2$	${\mathit{a}}_3$
90	93.80	91.80	93.80	91.00
95	96.80	95.80	95.80	95.60

6. Empirical Application

In this section, we illustrate the proposed method by an application to the east–west component of seismograms, recorded at Iwanai station during the first foreshock of the Urakawa–Oki earthquake in 1982. This dataset has been previously studied by [27]. The time series data are analyzed using autoregressive models (AR) of an order of 5. The estimates and visualization of the 95% confidence level bootstrap CIs are presented in Table 3 and Figure 2, respectively.

Table 3.

Bootstrap CIs for the change points.

Change Point	95% Bootstrap CIs	90% Bootstrap CIs
3074	[3072, 3084]	[3073, 3080]
3914	[3877, 3952]	[3882, 3948]

Figure 2.

Change points estimated by our method (vertical lines), with shaded areas representing the 95% confidence intervals around the change points. The results for the two change points are shown in red and blue, respectively.

It can be seen from Table 3 that change points are detected at 3074 and 3914. In [27], they found the estimated change points to be 3079 and 3929. It is noteworthy that [27]s’ results and ours are close. In geology, these two change points represent the arrival times of P-waves and S-waves, which are two types of seismic waves. From Figure 2, it is clear that the confidence interval (CI) of the first change point is narrower than that of the second change point.

This example demonstrates the applicability and effectiveness of our method in detecting change points in seismic data. By accurately identifying the locations of structural faults, we can gain insights into the underlying geological processes and improve our understanding of seismic events.

7. Conclusions

This paper effectively addresses the bias issues often encountered when using OGA for model selection and the estimation of segments containing change points in the cutting stage. The accuracy of multiple change point estimation is improved by applying sup-Wald-type test statistics in the refining stage. The proposed method successfully constructs confidence intervals for multiple change points by using a bootstrapping technique with a two-stage procedure to quantify the uncertainty of multiple change points. The reliable construction of confidence intervals makes this method a valuable addition to the field of change point analysis and regression modeling. The bootstrapping technique also guarantees asymptotic validity. Numerical studies demonstrate the statistical accuracy of the proposed method. Our method can also be applied together with block bootstrapping to the parameter changes of linear regression models with dependent errors.

Appendix A

Proof of Theorem 2.

We only sketch this proof because it is similar to the proof of Propositions 1 and 2 in [26]. To prove Theorem 2, we define $V\left(h\right)={\widehat{\mathit{\delta }}}_{j;h}^{\top }\left({\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h}\right){\widehat{\mathit{\delta }}}_{j;h}$. For the sake of simplicity, we denote $h_0=a_j$, which means that $h_0$ is the change point in ${\mathcal{I}}_{k_j}$. If $h=h_0$, we can obtain that ${\mathit{Z}}_{j,h}={\mathit{Z}}_{j,h_0}$, where ${\mathit{Z}}_{j,h_0}={(\mathbf{0},\cdots ,\mathbf{0},{\mathit{x}}_{h_0},\cdots ,{\mathit{x}}_{n_j^{\left(r\right)}})}^{\top }$. By Equation (9), ${\widehat{a}}_{j,h}=arg{max}_{h}V\left(h\right)$. Note that

$$\begin{array}{cc}\hfill & {\widehat{\mathit{\delta }}}_{j;h}={\left({\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h}\right)}^{-1}({\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h_0}{\mathit{\delta }}_j+{\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j\mathit{\epsilon }),\hfill \\ \hfill & {\widehat{\mathit{\delta }}}_{j,h_0}={\mathit{\delta }}_j+{\left({\mathit{Z}}_{j,h_0}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h_0}\right)}^{-1}{\mathit{Z}}_{j,h_0}^{\top }{\mathit{M}}_j\mathit{\epsilon }.\hfill \end{array}$$

It follows that

$$\begin{array}{cc}\hfill V\left(h\right)-V\left(h_0\right)& ={\mathit{\delta }}_j^{\top }\left\{\left({\mathit{Z}}_{j,h_0}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h}\right){\left({\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h}\right)}^{-1}\left({\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h_0}\right)-\left({\mathit{Z}}_{j,h_0}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h_0}\right)\right\}{\mathit{\delta }}_j\hfill \\ & +v\left(h\right),\hfill \end{array}$$

where

$$\begin{array}{cc}\hfill v\left(h\right)& =2{\mathit{\delta }}_j^{\top }\left({\mathit{Z}}_{j,h_0}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h}\right){\left({\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h}\right)}^{-1}{\mathit{Z}}_{j,h_0}^{\top }{\mathit{M}}_j\mathit{\epsilon }-2{\mathit{\delta }}_j^{\top }{\mathit{Z}}_{j,h_0}^{\top }{\mathit{M}}_j\mathit{\epsilon }\hfill \\ & +{\mathit{\epsilon }}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h}{\left({\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h}\right)}^{-1}{\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j\mathit{\epsilon }-{\mathit{\epsilon }}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h_0}{\left({\mathit{Z}}_{j,h_0}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h_0}\right)}^{-1}{\mathit{Z}}_{j,h_0}^{\top }{\mathit{M}}_j\mathit{\epsilon }.\hfill \end{array}$$

Define for $h\ne h_0$,

$$\begin{array}{cc}\hfill g\left(h\right)& ={\mathit{\delta }}_j^{\top }\left|\left({\mathit{Z}}_{j,h_0}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h}\right){\left({\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h}\right)}^{-1}\left({\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h_0}\right)\right.\hfill \\ & \left.-{\mathit{Z}}_{j,h_0}{\mathit{M}}_j{\mathit{Z}}_{j,h_0}\right|{\mathit{\delta }}_j/\left|h_0-h\right|.\hfill \end{array}$$

When $h=h_0$, define $g\left(h\right)={\mathit{\delta }}_j^{\top }{\mathit{\delta }}_j$. Then, we have

$$\begin{array}{cc}\hfill V\left(h\right)-V\left(h_0\right)& =-|h_0-h|g\left(h\right)+v\left(h\right).\hfill \end{array}$$ (A1)

Let

$${\mathit{Z}}_{\mathsf{\Delta }}=\left\{\begin{array}{cc}{\mathit{Z}}_{j,h}-{\mathit{Z}}_{j,h_0}={(\mathbf{0},\cdots ,\mathbf{0},{\mathit{x}}_{h+1},\dots ,{\mathit{x}}_{h_0},\mathbf{0},\dots ,\mathbf{0})}^{\top },\hfill & h<h_0,\hfill \\ {\mathit{Z}}_{j,h_0}-{\mathit{Z}}_{j,h}={(\mathbf{0},\cdots ,\mathbf{0},{\mathit{x}}_{h_0+1},\cdots ,{\mathit{x}}_h,\mathbf{0},\dots ,\mathbf{0})}^{\top },\hfill & h>h_0,\hfill \\ \mathbf{0},\hfill & h=h_0.\hfill \end{array}\right.$$

We have ${\mathit{Z}}_{j,h_0}={\mathit{Z}}_{j,h}-{\mathit{Z}}_{\mathsf{\Delta }}sgn(h_0-h)$. It follows that

$$\begin{array}{cc}\hfill v\left(h\right)& =v_1\left(h\right)+v_2\left(h\right)+v_3\left(h\right)+v_4\left(h\right)+v_5\left(h\right)\hfill \\ & =\left[2{\mathit{\delta }}_j^{\top }{\mathit{Z}}_{\mathsf{\Delta }}^{\top }\mathit{\epsilon }\right]sgn(h_0-h)\hfill \\ & -\left[2{\mathit{\delta }}_j^{\top }{\mathit{Z}}_{\mathsf{\Delta }}^{\top }{\mathit{X}}_j{\left({\mathit{X}}_j^{\top }{\mathit{X}}_j\right)}^{-1}{\mathit{X}}_j^{\top }\mathit{\epsilon }\right]sgn(h_0-h)\hfill \\ \hfill & -\left[2{\mathit{\delta }}_j^{\top }\left({\mathit{Z}}_{\mathsf{\Delta }}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h}\right){\left({\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h}\right)}^{-1}{\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j\mathit{\epsilon }\right]sgn(h_0-h)\hfill \\ \hfill & +{\mathit{\epsilon }}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h}{\left({\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h}\right)}^{-1}{\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j\mathit{\epsilon }\hfill \\ & -{\mathit{\epsilon }}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h_0}{\left({\mathit{Z}}_{j,h_0}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h_0}\right)}^{-1}{\mathit{Z}}_{j,h_0}^{\top }{\mathit{M}}_j\mathit{\epsilon }.\hfill \end{array}$$ (A2)

We now establish the convergence rate of the change point estimator ${\widehat{a}}_{j,h}$ in Theorem 2. By Lemma A.2 in [26], there exists $\lambda >0$ such that for every $ϵ>0$, ${inf}_{|h-h_0|>C\parallel {\delta }_j{\parallel }^{-2}}g\left(h\right)\ge \lambda {\parallel {\mathit{\delta }}_j\parallel }^2$ has a probability of at least $1-ϵ$. Therefore, we only need to prove that

$$\begin{array}{cc}\hfill & P\left(\right|{\widehat{a}}_{j,h}-a_j|>C\parallel {\mathit{\delta }}_j{\parallel }^{-2})\hfill \\ \hfill \le & P\left(\underset{\left|h-h_0\right|>C{\parallel {\delta }_j\parallel }^{-2}}{sup}V\left(h\right)\ge V\left(h_0\right)\right)\hfill \\ \hfill \le & P\left(\underset{h\in K\left(C\right)}{sup}\left|{\displaystyle \frac{v\left(h\right)}{(h_0-h){\parallel {\mathit{\delta }}_j\parallel }^2}}\right|>\lambda \right)\le ϵ,\hfill \end{array}$$ (A3)

where $K\left(C\right)=\left\{h:\left|h-h_0\right|>C{\parallel {\mathit{\delta }}_j\parallel }^{-2}\mathrm{and}{n}_j^{\left(l\right)}+\eta N_j\le h\le n_j^{\left(r\right)}-\eta N_j\right\}$ for a small number of $\eta >0$. It is easy to show that

$$\begin{array}{cc}\hfill \underset{h\in K\left(C\right)}{sup}{\displaystyle \frac{|v_2\left(h\right)|}{|h_0-h|\parallel {\mathit{\delta }}_j{\parallel }^2}}& \le 2\parallel {\mathit{\delta }}_j{\parallel }^{-1}\parallel {\mathit{Z}}_{\mathsf{\Delta }}^{\top }{\mathit{x}}_j/(h_0-h)\parallel \parallel {\left({\mathit{x}}_j^{\top }{\mathit{x}}_j\right)}^{-1}{\mathit{x}}_j^{\top }\mathit{\epsilon }\parallel \hfill \\ & =O_p(\parallel {\mathit{\delta }}_j{\parallel }^{-1}{N}_j^{-1/2})=o_p\left(1\right),\hfill \\ \hfill \underset{h\in K\left(C\right)}{sup}{\displaystyle \frac{v_3\left(h\right)}{|h_0-h|\parallel {\mathit{\delta }}_j{\parallel }^2}}& \le \parallel {\mathit{\delta }}_j{\parallel }^{-1}\parallel \left({\mathit{Z}}_{\mathsf{\Delta }}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h}\right)/(h_0-h)\parallel \parallel {\left({\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h}\right)}^{-1}{\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j\mathit{\epsilon }\parallel \hfill \\ & =O_p(\parallel {\mathit{\delta }}_j{\parallel }^{-1}{N}_j^{-1/2})=o_p\left(1\right),\hfill \\ \hfill \underset{h\in K\left(C\right)}{sup}{\displaystyle \frac{|v_4\left(h\right)|}{|h_0-h|\parallel {\mathit{\delta }}_j{\parallel }^2}}& \le \parallel {\mathit{\delta }}_j{\parallel }^{-2}\parallel {\left({\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h}\right)}^{-1/2}{\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{\epsilon }\parallel }^2/|h_0-h|\hfill \\ & =O_p(\parallel {\mathit{\delta }}_j{\parallel }^{-2}|h_0-h{|}^{-1})=O_p\left(1\right),\hfill \\ \hfill \underset{h\in K\left(C\right)}{sup}{\displaystyle \frac{|v_5\left(h\right)|}{|h_0-h|\parallel {\mathit{\delta }}_j{\parallel }^2}}& \le \parallel {\mathit{\delta }}_j{\parallel }^{-2}\parallel {\left({\mathit{Z}}_{j,h_0}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h_0}\right)}^{-1/2}{\mathit{Z}}_{j,h_0}^{\top }{\mathit{M}}_j{\mathit{\epsilon }\parallel }^2/|h_0-h|\hfill \\ & =O_p(\parallel {\mathit{\delta }}_j{\parallel }^{-2}|h_0-h{|}^{-1})=O_p\left(1\right).\hfill \end{array}$$

By Lemma A.3 in [26], there exists a large C such that

$$\begin{array}{cc}& P\left(\underset{h\in K\left(C\right)}{sup}{\displaystyle \frac{|v_1\left(h\right)|}{|h_0-h|\parallel {\mathit{\delta }}_j{\parallel }^2}}>{\displaystyle \frac{\lambda }{5}}\right)\hfill \\ \hfill \le & P\left(\underset{h\in K\left(C\right)}{sup}\parallel {\mathit{\delta }}_j{\parallel }^{-1}\parallel {\mathit{Z}}_{\mathsf{\Delta }}^{\top }\mathit{\epsilon }/(h_0-h)\parallel >{\displaystyle \frac{\lambda }{10}}\right)\le {\displaystyle \frac{100B}{{\lambda }^{2}C}}\le {\displaystyle \frac{ϵ}{5}}.\hfill \end{array}$$

This completes the proof of (A3).

To study the limiting distribution of ${\widehat{a}}_{j,h}$, we need to investigate the behavior of $V\left(h\right)$ on $D\left(C\right)=\{h:|h-h_0|\le C\parallel {\mathit{\delta }}_j{\parallel }^{-2}\}$. Since $\parallel {\mathit{X}}_j^{\top }{\mathit{Z}}_{\mathsf{\Delta }}\parallel =O_p(\parallel {\mathit{\delta }}_j{\parallel }^{-2})$ and $\parallel {\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{Z}}_{\mathsf{\Delta }}\parallel =O_p(\parallel {\mathit{\delta }}_j{\parallel }^{-2})$, we have

$$\begin{array}{cc}\hfill |h_0-h|g\left(h\right)& ={\mathit{\delta }}_j^{\top }\left\{\left({\mathit{Z}}_{j,h_0}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h}\right){\left({\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h}\right)}^{-1}\left({\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h_0}\right)-\left({\mathit{Z}}_{j,h_0}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h_0}\right)\right\}{\mathit{\delta }}_j\hfill \\ \hfill & ={\mathit{\delta }}_j^{\top }\{{\mathit{Z}}_{\mathsf{\Delta }}^{\top }{\mathit{Z}}_{\mathsf{\Delta }}-{\mathit{Z}}_{\mathsf{\Delta }}^{\top }{\mathit{X}}_j{\left({\mathit{X}}_j^{\top }{\mathit{X}}_j\right)}^{-1}{\mathit{X}}_j^{\top }{\mathit{Z}}_{\mathsf{\Delta }}\}{\mathit{\delta }}_j\hfill \\ \hfill & -{\mathit{\delta }}_j\left\{\left({\mathit{Z}}_{\mathsf{\Delta }}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h}\right){\left({\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h}\right)}^{-1}\left({\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{Z}}_{\mathsf{\Delta }}\right)\right\}{\mathit{\delta }}_j\hfill \\ \hfill & ={\mathit{\delta }}_j^{\top }{\mathit{Z}}_{\mathsf{\Delta }}^{\top }{\mathit{Z}}_{\mathsf{\Delta }}{\mathit{\delta }}_j+o_p\left(1\right).\hfill \end{array}$$

Consider $v\left(h\right)$ in (A2). It is straightforward to prove that if $|h-h_0|\le C\parallel {\mathit{\delta }}_j{\parallel }^{-2}$, then $v_2\left(h\right)=O_p(\parallel {\mathit{\delta }}_j{\parallel }^{-1}{N}_j^{-1/2})=o_p\left(1\right)$, and $v_3\left(h\right)=O_p(\parallel {\mathit{\delta }}_j{\parallel }^{-1}{N}_j^{-1/2})=o_p\left(1\right)$. It can also be shown that

$$\begin{array}{cc}\hfill v_4\left(h\right)+v_5\left(h\right)& ={\mathit{\epsilon }}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h_0}\left[{\left({\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h}\right)}^{-1}-{\left({\mathit{Z}}_{j,h_0}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h_0}\right)}^{-1}\right]{\mathit{Z}}_{j,h_0}^{\top }{\mathit{M}}_j\mathit{\epsilon }\hfill \\ & +{\mathit{\epsilon }}^{\top }{\mathit{M}}_j{\mathit{Z}}_{\mathsf{\Delta }}{\left({\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h}\right)}^{-1}{\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j\mathit{\epsilon }\hfill \\ & +{\mathit{\epsilon }}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h_0}{\left({\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h}\right)}^{-1}{\mathit{Z}}_{\mathsf{\Delta }}^{\top }{\mathit{M}}_j\mathit{\epsilon }=o_p\left(1\right)\hfill \end{array}$$

on $D\left(C\right)$. Therefore, we obtain that

$$V\left(h\right)-V\left(h_0\right)=-{\mathit{\delta }}_j^{\top }{\mathit{Z}}_{\mathsf{\Delta }}^{\top }{\mathit{Z}}_{\mathsf{\Delta }}{\mathit{\delta }}_j+\left[2{\mathit{\delta }}_j^{\top }{\mathit{Z}}_{\mathsf{\Delta }}^{\top }\mathit{\epsilon }\right]sgn(h_0-h)+o_p\left(1\right).$$ (A4)

In light of the proof of Proposition 2 in [26], we obtain the limiting distribution in Theorem 2. □

In the following, we introduce lemmas that are instrumental in proving Theorem 3.

Lemma A1.

If ${\mathit{\delta }}_j$ is fixed or ${\mathit{\delta }}_j\to 0$ but satisfies Assumption 5, then as $N_j\to \infty$,

$${\mathrm{var}}^{\ast }({\epsilon }_{n_j^{\left(l\right)}}^{\ast })=N_j^{-1}\sum _{i=n_j^{\left(l\right)}}^{n_j^{\left(r\right)}}{\left({\epsilon }_i-{\overline{\epsilon }}_{N_j}\right)}^2+O_p\left(N_j^{-1}\right).$$

where ${\epsilon }_{n_j^{\left(l\right)}}^{\ast }$ is defined in (11).

Proof f Lemma A1.

In view of (1) and (10), we have

$$\begin{array}{cc}\hfill {\tilde{\epsilon }}_i& ={\left[{\mathit{x}}_{i}I(a_j<i\le n_j^{\left(r\right)})-{\displaystyle \frac{1}{n_j^{\left(r\right)}-a_j}}\sum _{l=a_j+1}^{n_j^{\left(r\right)}}{\mathit{x}}_l\right]}^{\top }{\mathit{\delta }}_j+{({\mathit{x}}_i-{\overline{\mathit{x}}}_j)}^{\top }({\mathit{\beta }}_j-{\widehat{\mathit{\beta }}}_{j,h})\hfill \\ \hfill & -{\left[{\mathit{x}}_{i}I({\widehat{a}}_{j,h}<i\le n_j^{\left(r\right)})-{\displaystyle \frac{1}{N_j}}\sum _{l={\widehat{a}}_{j,h}+1}^{n_j^{\left(r\right)}}{\mathit{x}}_l\right]}^{\top }{\widehat{\mathit{\delta }}}_{j,h}+{\epsilon }_i-{\overline{\epsilon }}_{N_j},\hfill \end{array}$$ (A5)

where ${\overline{\mathit{x}}}_j={\displaystyle \frac{1}{N_j}}{\sum }_{l=n_j^{\left(l\right)}}^{n_j^{\left(r\right)}}{\mathit{x}}_l$ and ${\overline{\epsilon }}_{N_j}={\displaystyle \frac{1}{N_j}}{\sum }_{l=n_j^{\left(l\right)}}^{n_j^{\left(r\right)}}{\epsilon }_l$.

Assume $a_j<{\widehat{a}}_{j,h}$ (the other case can be handled in a similar way). Since $E^{\ast }({\epsilon }_{n_j^{\left(l\right)}}^{\ast })=N_j^{-1}{\sum }_{i=n_j^{\left(l\right)}}^{n_j^{\left(r\right)}}{\tilde{\epsilon }}_i=0$, we have ${\mathrm{var}}^{\ast }({\epsilon }_{n_j^{\left(l\right)}}^{\ast })=N_j^{-1}{\sum }_{i=n_j^{\left(l\right)}}^{n_j^{\left(r\right)}}{\tilde{\epsilon }}_i^2$. It follows from (A5) that

$$\begin{array}{cc}\hfill {\mathrm{var}}^{\ast }({\epsilon }_{n_j^{\left(l\right)}}^{\ast })& ={\displaystyle \frac{1}{N_j}}\sum _{i=n_j^{\left(l\right)}}^{n_j^{\left(r\right)}}{({\epsilon }_i-{\overline{\epsilon }}_{N_j})}^2+{\displaystyle \frac{2}{N_j}}\sum _{i=n_j^{\left(l\right)}}^{n_j^{\left(r\right)}}{({\mathit{x}}_i-{\overline{\mathit{x}}}_j)}^{\top }({\mathit{\beta }}_j-{\widehat{\mathit{\beta }}}_{j,h})({\epsilon }_i-{\overline{\epsilon }}_{N_j})\hfill \\ \hfill & +{\displaystyle \frac{1}{N_j}}\sum _{i=n_j^{\left(l\right)}}^{n_j^{\left(r\right)}}{\left[{({\mathit{x}}_i-{\overline{\mathit{x}}}_j)}^{\top }({\mathit{\beta }}_j-{\widehat{\mathit{\beta }}}_{j,h})\right]}^2+{\displaystyle \frac{1}{N_j^2}}{\left(\sum _{i={\widehat{a}}_{j,h}+1}^{n_j^{\left(r\right)}}{\mathit{x}}_i^{\top }({\mathit{\delta }}_j-{\widehat{\mathit{\delta }}}_{j,h})\right)}^2\hfill \\ & +{\displaystyle \frac{1}{N_j^2}}{\left(\sum _{i=a_j+1}^{{\widehat{a}}_{j,h}}{\mathit{x}}_i^{\top }{\mathit{\delta }}_j\right)}^2+{\displaystyle \frac{2}{N_j^2}}\sum _{i=a_j+1}^{{\widehat{a}}_{j,h}}{\mathit{x}}_i^{\top }{\mathit{\delta }}_j\sum _{l={\widehat{a}}_{j,h}+1}^{n_j^{\left(r\right)}}{\mathit{x}}_l^{\top }({\mathit{\delta }}_j-{\widehat{\mathit{\delta }}}_{j,h}).\hfill \end{array}$$ (A6)

According to Corollary 1 in [26], we have ${\mathit{\beta }}_j-{\widehat{\mathit{\beta }}}_{j,h}=O_p\left(N_j^{-1/2}\right)$ and ${\mathit{\delta }}_j-{\widehat{\mathit{\delta }}}_{j,h}=O_p\left(N_j^{-1/2}\right)$, which, combined with ${\widehat{a}}_{j,h}=a_j+O_p(\parallel {\mathit{\delta }}_j{\parallel }^{-2})$, yields that $N_j^{-1}{\sum }_{i=a_j+1}^{{\widehat{a}}_{j,h}}{\mathit{x}}_i^{\top }{\mathit{\delta }}_j=O_p(N_j^{-1}\parallel {\mathit{\delta }}_j{\parallel }^{-1})$ and $N_j^{-1}{\sum }_{l={\widehat{a}}_{j,h}+1}^{n_j^{\left(r\right)}}{\mathit{x}}_l^{\top }({\mathit{\delta }}_j-{\widehat{\mathit{\delta }}}_{j,h})=O_p\left(N_j^{-1/2}\right)$. This completes the proof.

□

Lemma A2.

For every $ϵ>0$,

$$\underset{N_j\to \infty }{lim\; sup}{P}^{\ast }\left(\left|N_j^{-1}\sum _{i=n_j^{\left(l\right)}}^{n_j^{\left(r\right)}}{\left({\epsilon }_i^{\ast }\right)}^2-Va{r}^{\ast }({\epsilon }_{n_j^{\left(l\right)}}^{\ast })\right|\ge ϵ\right)\le ϵ{\sigma }^2.$$

Proof of Lemma A2.

We have for every $ϵ>0$ and every $\eta >0$,

$$\begin{array}{cc}\hfill & P^{\ast }\left(\left|N_j^{-1}\sum _{i=n_j^{\left(l\right)}}^{n_j^{\left(r\right)}}{\epsilon }_i^{\ast 2}-{\mathrm{var}}^{\ast }({\epsilon }_{n_j^{\left(l\right)}}^{\ast })\right|\ge ϵ\right)\hfill \\ \hfill =& P^{\ast }\left(\left|N_j^{-1}\sum _{i=n_j^{\left(l\right)}}^{n_j^{\left(r\right)}}{\epsilon }_i^{\ast 2}-N_j^{-1}\sum _{i=n_j^{\left(l\right)}}^{n_j^{\left(r\right)}}{\tilde{\epsilon }}_i^2\right|\ge ϵ\right)\hfill \\ \hfill \le & P^{\ast }\left(\left|N_j^{-1}\sum _{i=n_j^{\left(l\right)}}^{n_j^{\left(r\right)}}{\epsilon }_i^{\ast 2}I\left(\right|{\epsilon }_i^{\ast }|\ge \eta \sqrt{N_j})-N_j^{-1}\sum _{i=n_j^{\left(l\right)}}^{n_j^{\left(r\right)}}{\tilde{\epsilon }}_i^{2}I\left(\right|{\tilde{\epsilon }}_i|\ge \eta \sqrt{N_j})\right|\ge ϵ\right)\hfill \\ \hfill & +P^{\ast }\left(\left|N_j^{-1}\sum _{i=n_j^{\left(l\right)}}^{n_j^{\left(r\right)}}{\epsilon }_i^{\ast 2}I\left(\right|{\epsilon }_i^{\ast }|<\eta \sqrt{N_j})-N_j^{-1}\sum _{i=n_j^{\left(l\right)}}^{n_j^{\left(r\right)}}{\tilde{\epsilon }}_i^{2}I\left(\right|{\tilde{\epsilon }}_i|<\eta \sqrt{N_j})\right|\ge ϵ\right)\hfill \end{array}$$ (A7)

Clearly, for every $ϵ>0$ and every $\eta >0$, as $N_j\to \infty$, we have

$$N_j^{-1}\sum _{i=n_j^{\left(l\right)}}^{n_j^{\left(r\right)}}{\tilde{\epsilon }}_i^{2}I\left(\right|{\tilde{\epsilon }}_i|\ge \eta \sqrt{N_j})\to 0a.s.$$

It follows that

$$P^{\ast }\left(N_j^{-1}\sum _{i=n_j^{\left(l\right)}}^{n_j^{\left(r\right)}}{\epsilon }_i^{\ast 2}I\left(\left|{\epsilon }_i^{\ast }\right|\ge \eta \sqrt{N_j}\right)\ge ϵ/2\right)\le {\displaystyle \frac{2}{ϵ}}{N}_j^{-1}\sum _{i=n_j^{\left(l\right)}}^{n_j^{\left(r\right)}}{\tilde{\epsilon }}_i^{2}I\left(\left|{\tilde{\epsilon }}_i\right|\ge \eta \sqrt{N_j}\right)\to 0,\mathrm{a}.\mathrm{s}.$$

Since

$$\begin{array}{cc}& \underset{N_j\to \infty }{lim\; sup}{P}^{\ast }\left(\left|N_j^{-1}\sum _{i=n_j^{\left(r\right)}}^{n_j^{\left(l\right)}}{\epsilon }_i^{\ast 2}I\left(\left|{\epsilon }_i^{\ast }\right|<\eta \sqrt{N_j}\right)-N_j^{-1}\sum _{i=n_j^{\left(r\right)}}^{n_j^{\left(l\right)}}{\tilde{\epsilon }}_i^{2}I\left(\left|{\tilde{\epsilon }}_i\right|<\eta \sqrt{N_j}\right)\right|\ge ϵ/2\right)\hfill \\ \hfill \le & {\displaystyle \frac{4}{{ϵ}^2}}\underset{N_j\to \infty }{lim\; sup}{N}_j^{-2}\sum _{i=n_j^{\left(r\right)}}^{n_j^{\left(l\right)}}{\tilde{\epsilon }}_i^{4}I\left(\left|{\tilde{\epsilon }}_i\right|<\eta \sqrt{N_j}\right)\hfill \\ \hfill \le & {\displaystyle \frac{4}{{ϵ}^2}}{\eta }^2\underset{n\to \infty }{lim\; sup}{N}_j^{-1}\sum _{i=n_j^{\left(r\right)}}^{n_j^{\left(l\right)}}{\tilde{\epsilon }}_i^2={\displaystyle \frac{4}{{ϵ}^2}}{\eta }^2{\sigma }^2\mathrm{a}.\mathrm{s}.,\hfill \end{array}$$

choosing ${\eta }^2={ϵ}^3/8$ gives the assertion in Lemma A2. □

Lemma A3.

Under the assumption that ${\sum }_{i=t_1}^{t_2}{\mathit{x}}_i{\mathit{x}}_i^{\top }/\left(t_1-t_2\right){\to }_{p}V$ as $t_2-t_1{\to }_p\infty$, for every $ϵ>0$, there exists $\lambda >0$ such that $g^{\ast }\ge \lambda {\parallel {\widehat{\mathit{\delta }}}_{j,h}\parallel }^2$ with a probability of at least $1-ϵ$, where $g^{\ast }={inf}_{\left|k_2-h\right|>N_j\eta }{g}^{\ast }\left(h_b\right)$.

Lemma A4.

Under the assumption that ${\sum }_{i=t_1}^{t_2}{\mathit{x}}_i{\mathit{x}}_i^{\top }/\left(t_1-t_2\right){\to }_{p}V$ as $t_2-t_1\to \infty$, for every $ϵ>0$ and $\lambda >0$, there exists $N_0>0$, such that $N_j>N_0$, $P^{\ast }\left(\right|h_b-h|>N_j\eta )<ϵ$.

The proofs of Lemmas A3 and A4 follow from [26] (see Lemmas A.2 and A.4). We next establish the consistency of ${\widehat{a}}_{j,h_b}^{\ast }$ in Lemma A5, which serves as a key step in proving Theorem 3.

Lemma A5.

If ${\mathit{\delta }}_j$ is fixed or ${\mathit{\delta }}_j\to 0$ but satisfies Assumption 5, under Assumptions 1–4, we have

$${\widehat{a}}_{j,h_b}^{\ast }-{\widehat{a}}_{j,h}=O_p(\parallel {\widehat{\mathit{\delta }}}_{j,h}{\parallel }^{-2}).$$

Proof of Lemma A5.

We define $V^{\ast }\left(h_b\right)={\left({\widehat{\mathit{\delta }}}_{j;h_b}^{\ast }\right)}^{\top }\left({\mathit{Z}}_{j,h_b}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h_b}\right){\widehat{\mathit{\delta }}}_{j;h_b}^{\ast }$. By Equation (12), ${\widehat{a}}_{j,h_b}^{\ast }=arg{max}_{h_b}{V}^{\ast }\left(h_b\right)$. Denote $h={\widehat{a}}_{j,h}$. If $h_b=h$, then ${\mathit{Z}}_{j,h_b}={\mathit{Z}}_{j,h}$. It can be seen from (11) that

$$\begin{array}{cc}& {\widehat{\mathit{\delta }}}_{j;h_b}^{\ast }={\left({\mathit{Z}}_{j,h_b}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h_b}\right)}^{-1}({\mathit{Z}}_{j,h_b}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h}{\widehat{\mathit{\delta }}}_{j,h}+{\mathit{Z}}_{j,h_b}^{\top }{\mathit{M}}_j{\mathit{\epsilon }}^{\ast }),\hfill \\ & {\widehat{\mathit{\delta }}}_{j;h}^{\ast }={\widehat{\mathit{\delta }}}_{j,h}+{\left({\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h}\right)}^{-1}{\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{\epsilon }}^{\ast }.\hfill \end{array}$$

It follows that

$$\begin{array}{cc}\hfill V^{\ast }\left(h_b\right)-V^{\ast }\left(h\right)& ={\widehat{\mathit{\delta }}}_{j,h}^{\top }\left\{\left({\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h_b}\right){\left({\mathit{Z}}_{j,h_b}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h_b}\right)}^{-1}\left({\mathit{Z}}_{j,h_b}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h}\right)-\left({\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h}\right)\right\}{\widehat{\mathit{\delta }}}_{j,h}\hfill \\ & +v^{\ast }\left(h_b\right),\hfill \end{array}$$

where

$$\begin{array}{cc}\hfill v^{\ast }\left(h_b\right)& =2{\widehat{\mathit{\delta }}}_{j,h}^{\top }\left({\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h_b}\right){\left({\mathit{Z}}_{j,h_b}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h_b}\right)}^{-1}{\mathit{Z}}_{j,h_b}^{\top }{\mathit{M}}_j{\mathit{\epsilon }}^{\ast }-2{\widehat{\mathit{\delta }}}_{j,h}^{\top }{\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{\epsilon }}^{\ast }\hfill \\ & +{\mathit{\epsilon }}^{\ast \top }{\mathit{M}}_j{\mathit{Z}}_{j,h_b}{\left({\mathit{Z}}_{j,h_b}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h_b}\right)}^{-1}{\mathit{Z}}_{j,h_b}^{\top }{\mathit{M}}_j{\mathit{\epsilon }}^{\ast }-{\mathit{\epsilon }}^{\ast \top }{\mathit{M}}_j{\mathit{Z}}_{j,h}{\left({\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h}\right)}^{-1}{\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{\epsilon }}^{\ast }.\hfill \end{array}$$

Define

$$g^{\ast }\left(h_b\right)=\left\{\begin{array}{cc}{\widehat{\mathit{\delta }}}_{j,h}^{\top }\left|\left({\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h_b}\right){\left({\mathit{Z}}_{j,h_b}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h_b}\right)}^{-1}\left({\mathit{Z}}_{j,h_b}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h}\right)-{\mathit{Z}}_{j,h}{\mathit{M}}_j{\mathit{Z}}_{j,h}\right|{\widehat{\mathit{\delta }}}_{j,h}/\left|h-h_b\right|,\hfill & \mathrm{if} h_b\ne h\hfill \\ {\widehat{\mathit{\delta }}}_{j,h}^{\top }{\widehat{\mathit{\delta }}}_{j,h},\hfill & \mathrm{if} h_b=h.\hfill \end{array}\right.$$

We have

$$V^{\ast }\left(h_b\right)-V^{\ast }\left(h\right)=-|h-h_b|g^{\ast }\left(h_b\right)+v^{\ast }\left(h_b\right).$$ (A8)

Again, since $V^{\ast }\left(h_b\right)\ge V^{\ast }\left(h\right)$ by definition, it suffices to prove, by Lemma A4, that

$$P^{\ast }\left(\underset{\left|h_b-h\right|>C{\parallel {\widehat{\delta }}_{j,h}\parallel }^{-2}}{sup}{V}^{\ast }\left(h_b\right)\ge V^{\ast }\left(h\right)\right)<ϵ.$$

By Lemmas A1 and A2, for any $ϵ>0$ and $\eta >0$, we have $P^{\ast }\left(\left|h_b-h\right|>N_j\eta \right)<ϵ$ for a large $N_j$. Therefore, to prove the above equation, we only need to show that

$$P^{\ast }\left(\underset{h_b\in K^{\ast }\left(C\right)}{sup}V\left(h_b\right)\ge V\left(h\right)\right)<ϵ,$$

where $K^{\ast }\left(C\right)=\left\{h_b:\left|h_b-h\right|>C{\parallel {\widehat{\mathit{\delta }}}_{j,h}\parallel }^{-2}and{n}_j^{\left(l\right)}+N_j\eta \le h_b\le n_j^{\left(r\right)}-\eta N_j\right\}$ for a small number $\eta >0$. Finding ${sup}_{h_b\in K^{\ast }\left(C\right)}V\left(h_b\right)$ is equivalent to a restricted search, this is legitimate only after the consistency has been established. Thus, by Lemma A3, it suffices to show that

$$P^{\ast }\left(\underset{h_b\in K^{\ast }\left(C\right)}{sup}{\displaystyle \frac{|v^{\ast }\left(h_b\right)|}{|h-h_b|\parallel {\widehat{\mathit{\delta }}}_{j,h}{\parallel }^2}}>\lambda \right)<ϵ.$$ (A9)

Next, consider $v^{\ast }\left(h_b\right)$. Denote

$${\mathit{Z}}_{{\mathsf{\Delta }}_b}=\left\{\begin{array}{cc}{\mathit{Z}}_{j,h_b}-{\mathit{Z}}_{j,h}={(\mathbf{0},\cdots ,\mathbf{0},{\mathit{x}}_{h_b+1},\dots ,{\mathit{x}}_h,\mathbf{0},\dots ,\mathbf{0})}^{\top },\hfill & h_b<h,\hfill \\ {\mathit{Z}}_{j,h}-{\mathit{Z}}_{j,h_b}={(\mathbf{0},\cdots ,\mathbf{0},{\mathit{x}}_{h+1},\cdots ,{\mathit{x}}_{h_b},\mathbf{0},\dots ,\mathbf{0})}^{\top },\hfill & h_b>h,\hfill \\ \mathbf{0},\hfill & h_b=h.\hfill \end{array}\right.$$

It follows that ${\mathit{Z}}_{j,h_b}={\mathit{Z}}_{j,h}+{\mathit{Z}}_{{\mathsf{\Delta }}_b}sgn\left(h-h_b\right)$. Thus, we have

$$\begin{array}{cc}\hfill v^{\ast }\left(h_b\right)& =v_1^{\ast }\left(h_b\right)+v_2^{\ast }\left(h_b\right)+v_3^{\ast }\left(h_b\right)+v_4^{\ast }\left(h_b\right)+v_5^{\ast }\left(h_b\right)\hfill \\ & =\left[2{\widehat{\mathit{\delta }}}_{j,h}^{\top }{\mathit{Z}}_{{\mathsf{\Delta }}_b}^{\top }{\mathit{\epsilon }}^{\ast }\right]sgn(h-h_b)\hfill \\ & -\left[2{\widehat{\mathit{\delta }}}_{j,h}^{\top }{\mathit{Z}}_{{\mathsf{\Delta }}_b}^{\top }{\mathit{x}}_j{\left({\mathit{x}}_j^{\top }{\mathit{x}}_j\right)}^{-1}{\mathit{x}}_j^{\top }{\mathit{\epsilon }}^{\ast }\right]sgn(h-h_b)\hfill \\ \hfill & -\left[2{\widehat{\mathit{\delta }}}_{j,h}^{\top }\left({\mathit{Z}}_{{\mathsf{\Delta }}_b}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h_b}\right){\left({\mathit{Z}}_{j,h_b}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h_b}\right)}^{-1}{\mathit{Z}}_{j,h_b}^{\top }{\mathit{M}}_j{\mathit{\epsilon }}^{\ast }\right]sgn(h-h_b)\hfill \\ \hfill & +{\mathit{\epsilon }}^{\ast \top }{\mathit{M}}_j{\mathit{Z}}_{j,h_b}{\left({\mathit{Z}}_{j,h_b}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h_b}\right)}^{-1}{\mathit{Z}}_{j,h_b}^{\top }{\mathit{M}}_j{\mathit{\epsilon }}^{\ast }\hfill \\ & -{\mathit{\epsilon }}^{\ast \top }{\mathit{M}}_j{\mathit{Z}}_{j,h}{\left({\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h}\right)}^{-1}{\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{\epsilon }}^{\ast }.\hfill \end{array}$$ (A10)

By Lemmas A1 and A2, we deduce that ${\left({\mathit{X}}_j^{\top }{\mathit{X}}_j\right)}^{-1}{\mathit{X}}_j^{\top }{\mathit{\epsilon }}^{\ast }=N_j^{-1/2}{O}_{p^{\ast }}\left(1\right).$ Since $N_j^{-1/2+\alpha }{\mathit{\delta }}_j^{-1}\to 0$ and

$$N_j^{-1/2+\alpha }{\mathit{\delta }}_j^{-1}-N_j^{-1/2+\alpha }{\widehat{\mathit{\delta }}}_{j,h}^{-1}=N_j^{-1/2+\alpha }{\mathit{\delta }}_j^{-1}({\widehat{\mathit{\delta }}}_{j,h}-{\mathit{\delta }}_j){\widehat{\mathit{\delta }}}_{j,h}^{-1}=O_{p^{\ast }}\left(N_j^{-1+\alpha }{\mathit{\delta }}_j^{-2}\right),$$

by Corollary 1 in [26], we have $N_j^{-1/2+\alpha }{\widehat{\mathit{\delta }}}_{j,h}^{-1}$ tends to zero. It follows from ${\mathit{Z}}_{{\mathsf{\Delta }}_b}^{\top }{\mathit{X}}_j=|h-h_b|O_p\left(1\right)$ that

$$\begin{array}{cc}\hfill \underset{h_b\in K^{\ast }\left(C\right)}{sup}{\displaystyle \frac{|v_2^{\ast }\left(h_b\right)|}{|h-h_b|\parallel {\widehat{\mathit{\delta }}}_{j,h}{\parallel }^2}}& \le 2\parallel {\widehat{\mathit{\delta }}}_{j,h}{\parallel }^{-1}\parallel {\mathit{Z}}_{{\mathsf{\Delta }}_b}^{\top }{\mathit{X}}_j/(h-h_b)\parallel \parallel {\left({\mathit{X}}_j^{\top }{\mathit{X}}_j\right)}^{-1}{\mathit{X}}_j^{\top }{\mathit{\epsilon }}^{\ast }\parallel \hfill \\ & =O_{p^{\ast }}(\parallel {\widehat{\mathit{\delta }}}_{j,h}{\parallel }^{-1}{N}_j^{-1/2})=o_{p^{\ast }}\left(1\right).\hfill \end{array}$$

Similarly, we have $\parallel \left({\mathit{Z}}_{{\mathsf{\Delta }}_b}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h_b}\right)\parallel =|h-h_b|O_{p^{\ast }}\left(1\right)$, and ${\left({\mathit{Z}}_{j,h_b}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h_b}\right)}^{-1}{\mathit{Z}}_{j,h_b}^{\top }{\mathit{M}}_j{\mathit{\epsilon }}^{\ast }=N_j^{-1/2}{O}_{p^{\ast }}\left(1\right)$ uniformly on $K^{\ast }\left(C\right)$, which implies

$$\begin{array}{cc}\hfill \underset{h_b\in K^{\ast }\left(C\right)}{sup}{\displaystyle \frac{v_3\left(h_b\right)}{|h-h_b|\parallel {\widehat{\mathit{\delta }}}_{j,h}{\parallel }^2}}& \le \parallel {\widehat{\mathit{\delta }}}_{j,h}{\parallel }^{-1}\parallel \left({\mathit{Z}}_{{\mathsf{\Delta }}_b}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h_b}\right)/(h-h_b)\parallel \parallel {\left({\mathit{Z}}_{j,h_b}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h_b}\right)}^{-1}{\mathit{Z}}_{j,h_b}^{\top }{\mathit{M}}_j{\mathit{\epsilon }}^{\ast }\parallel \hfill \\ & =O_{p^{\ast }}(\parallel {\widehat{\mathit{\delta }}}_{j,h}{\parallel }^{-1}{N}_j^{-1/2})=o_{p^{\ast }}\left(1\right).\hfill \end{array}$$

Furthermore, it is easy to show that the expressions ${sup}_{h_b\in K^{\ast }\left(C\right)}{\displaystyle \frac{|v_4^{\ast }\left(h_b\right)|}{|h-h_b|\parallel {\widehat{\delta }}_{j,h}{\parallel }^2}}$ and

${sup}_{h_b\in K^{\ast }\left(C\right)}{\displaystyle \frac{|v_5^{\ast }\left(h\right)|}{|h-h_b|\parallel {\widehat{\delta }}_{j,h}{\parallel }^2}}$ are uniformly $O_{p^{\ast }}\left(1\right)$ on $K^{\ast }\left(C\right)$. By Lemma A.3 in [26],

$$\begin{array}{cc}& P^{\ast }\left(\underset{h_b\in K^{\ast }\left(C\right)}{sup}{\displaystyle \frac{|v_1^{\ast }\left(h_b\right)|}{|h-h_b|\parallel {\widehat{\mathit{\delta }}}_{j,h}{\parallel }^2}}>{\displaystyle \frac{\lambda }{5}}\right)\hfill \\ \hfill \le & P^{\ast }\left(\underset{h_b\in K^{\ast }\left(C\right)}{sup}\parallel {\widehat{\mathit{\delta }}}_{j,h}{\parallel }^{-1}\parallel {\mathit{Z}}_{{\mathsf{\Delta }}_b}^{\top }{\mathit{\epsilon }}^{\ast }/(h-h_b)\parallel >{\displaystyle \frac{\lambda }{10}}\right)\le {\displaystyle \frac{100B}{{\lambda }^{2}C}}\le {\displaystyle \frac{ϵ}{5}}\hfill \end{array}$$

for large C. In conclution, the consistency of the bootstrap estimator has been established. □

Proof of Theorem 3.

Lemma A5 implies that when C is large, ${\widehat{a}}_{j,h_b}^{\ast }$ has a high probability of being outside the set $K^{\ast }\left(C\right)$. Let $D^{\ast }\left(C\right)$ denote the complement set of $K^{\ast }\left(C\right)$ such that $D^{\ast }\left(C\right)=\{h_b:|h_b-h|\le C\parallel {\widehat{\mathit{\delta }}}_{j,h}{\parallel }^{-2}\}$. To study the limiting distribution, we consider the expression of $V^{\ast }\left(h_b\right)-V^{\ast }\left(h\right)$ on $D^{\ast }\left(C\right)$.

We use ${\mathit{Z}}_{j,h}={\mathit{Z}}_{j,h_b}-{\mathit{Z}}_{{\mathsf{\Delta }}_b}sgn\left(h-h_b\right)$ to obtain that

$$\begin{array}{cc}\hfill \left|h-h_b\right|g^{\ast }\left(h_b\right)& ={\widehat{\mathit{\delta }}}_{j,h}^{\top }\{{\mathit{Z}}_{j,h}{\mathit{M}}_j{\mathit{Z}}_{j,h}-{\mathit{Z}}_{j,h}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h_b}{\left({\mathit{Z}}_{j,h_b}{\mathit{M}}_j{\mathit{Z}}_{j,h_b}\right)}^{-1}{\mathit{Z}}_{j,h_b}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h}\}{\widehat{\mathit{\delta }}}_{j,h}\hfill \\ \hfill & ={\widehat{\mathit{\delta }}}_{j,h}^{\top }\{{\mathit{Z}}_{{\mathsf{\Delta }}_b}^{\top }{\mathit{Z}}_{{\mathsf{\Delta }}_b}-{\mathit{Z}}_{{\mathsf{\Delta }}_b}^{\top }{\mathit{X}}_j{\left({\mathit{X}}_j^{\top }{\mathit{X}}_j\right)}^{-1}{\mathit{X}}_j^{\top }{\mathit{Z}}_{{\mathsf{\Delta }}_b}\}{\widehat{\mathit{\delta }}}_{j,h}\hfill \\ & -{\widehat{\mathit{\delta }}}_{j,h}^{\top }{\mathit{Z}}_{{\mathsf{\Delta }}_b}^{\top }{\mathit{M}}_j{\mathit{Z}}_{j,h_b}{\left({\mathit{Z}}_{j,h_b}{\mathit{M}}_j{\mathit{Z}}_{j,h_b}\right)}^{-1}{\mathit{Z}}_{j,h_b}^{\top }{\mathit{M}}_j{\mathit{Z}}_{{\mathsf{\Delta }}_b}{\widehat{\mathit{\delta }}}_{j,h}.\hfill \end{array}$$ (A11)

Since $\parallel {\mathit{X}}_j^{\top }{\mathit{Z}}_{{\mathsf{\Delta }}_b}\parallel =O_{p^{\ast }}(\parallel {\widehat{\mathit{\delta }}}_{j,h}{\parallel }^{-2})$ and $\parallel {\mathit{Z}}_{j,h_b}^{\top }{\mathit{M}}_j{\mathit{Z}}_{{\mathsf{\Delta }}_b}\parallel =O_{p^{\ast }}(\parallel {\widehat{\mathit{\delta }}}_{j,h}{\parallel }^{-2})$, we have

$$|h-h_b|g^{\ast }\left(h_b\right)={\widehat{\mathit{\delta }}}_{j,h}^{\top }{\mathit{Z}}_{{\mathsf{\Delta }}_b}^{\top }{\mathit{Z}}_{{\mathsf{\Delta }}_b}{\widehat{\mathit{\delta }}}_{j,h}+o_{p^{\ast }}\left(1\right).$$

Since $\left|h-h_b\right|\le C{\parallel {\widehat{\mathit{\delta }}}_{j,h}\parallel }^{-2}$, we deduce that $v_2^{\ast }\left(h_b\right)$ and $v_3^{\ast }\left(h_b\right)$ are both bounded by $O_{p^{\ast }}(N_j^{-1/2}\parallel {\widehat{\mathit{\delta }}}_{j,h}{\parallel }^{-1})=o_{p^{\ast }}\left(1\right)$, and $v_4^{\ast }\left(h_b\right)$ and $v_5^{\ast }\left(h_b\right)$ are both bounded by $o_{p^{\ast }}\left(1\right)$ uniformly on $D^{\ast }\left(C\right)$.

By the above results, we obtain that

$$\begin{array}{cc}\hfill V^{\ast }\left(h_b\right)-V^{\ast }\left(h\right)& =-{\widehat{\mathit{\delta }}}_{j,h}^{\top }{\mathit{Z}}_{{\mathsf{\Delta }}_b}^{\top }{\mathit{Z}}_{{\mathsf{\Delta }}_b}{\widehat{\mathit{\delta }}}_{j,h}+2{\widehat{\mathit{\delta }}}_{j,h}^{\top }{\mathit{Z}}_{{\mathsf{\Delta }}_b}^{\top }{\mathit{\epsilon }}^{\ast }sgn\left(h-h_b\right)+o_{p^{\ast }}\left(1\right),\hfill \end{array}$$

which combined with (A4) completes the proof. □

Author Contributions

Conceptualization, L.H., B.J. and Y.W.; methodology, L.H. and B.J.; data analysis, L.H. and F.W.; writing, L.H., Y.W. and F.W. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

Not applicable.

Data Availability Statement

The data presented in this study are available on request from the corresponding authors.

Conflicts of Interest

The authors declare no conflicts of interest.

Funding Statement

This work is supported by the National Natural Science Foundation of China (12231017, 72293573); and the Natural Science and Engineering Research Council of Canada (RGPIN-2023-05655).