Efficient multiple change point detection for high-dimensional generalized linear models

Abstract

Change point detection for high-dimensional data is an important yet challenging problem for many applications. In this paper, we consider multiple change point detection in the context of high-dimensional generalized linear models, allowing the covariate dimension p to grow exponentially with the sample size n. The model considered is general and flexible in the sense that it covers various specific models as special cases. It can automatically account for the underlying data generation mechanism without specifying any prior knowledge about the number of change points. Based on dynamic programming and binary segmentation techniques, two algorithms are proposed to detect multiple change points, allowing the number of change points to grow with n. To further improve the computational efficiency, a more efficient algorithm designed for the case of a single change point is proposed. We present theoretical properties of our proposed algorithms, including estimation consistency for the number and locations of change points as well as consistency and asymptotic distributions for the underlying regression coefficients. Finally, extensive simulation studies and application to the Alzheimer’s Disease Neuroimaging Initiative data further demonstrate the competitive performance of our proposed methods.

1. INTRODUCTION

With the advance of technology, complex large-scale data are prevalent in various scientific fields. Data heterogeneity creates great challenges for the analysis of complex data that may not be well approximated by a common distribution. Change point detection is a powerful tool to deal with data heterogeneity. Since the seminal work by Page (1955), there is a growing literature on change point detection with a wide range of applications, including genomics (Braun et al., 2000), finance (Pesaran & Pick, 2007; Fan et al., 2011), and social networks (Raginsky et al., 2012).

In this paper, we consider multiple change point detection for a general framework of high-dimensional generalized linear models (GLMs). Suppose we have n independent observations $\{Y_i,{\mathit{X}}_i{\}}_{i=1}^n$ with

$$g({\mu }_i)={\mathit{X}}_i^T{\beta }^{(i)}\text{for}i=1,\dots ,n,$$ (1)

where $Y_i\in \mathcal{Y}\subset \mathbb{R}$ is the real-valued response for the i-th observation, X_i = (X_i1,…,X_ip)^⊤ is the corresponding covariate vector in $\mathcal{X}\subset {\mathbb{R}}^p$, ${\mu }_i=\mathbb{E}(Y_i\mid {\mathit{X}}_i)$, g(·) is the link function, and ${\beta }^{(i)}=({\beta }_1^{(i)},\dots ,{\beta }_p^{(i)}{)}^{\mathsf{T}}\in {\mathbb{R}}^p$ is the unknown regression coefficient vector for the i-th observation. Then we consider estimating multiple change points with piecewise constant coefficients for Model (1). More specifically, let $\tilde{k}\ge 0$ be the true number of unknown change points along with the true location vector $\tilde{\tau }=({\tilde{\tau }}_0,{\tilde{\tau }}_1\dots ,{\tilde{\tau }}_{\tilde{k}},{\tilde{\tau }}_{\tilde{k}+1}{)}^{\mathsf{T}}$ with $0={\tilde{\tau }}_0<{\tilde{\tau }}_1<{\tilde{\tau }}_2<\dots <{\tilde{\tau }}_{\tilde{k}}<{\tilde{\tau }}_{\tilde{k}+1}=1$. Then, the unknown $\tilde{k}$ change points divide the n time-ordered observations into $\tilde{k}+1$ intervals and the underlying regression coefficients β⁽ⁱ⁾ have the following form:

$${\beta }^{(i)}=\{\begin{array}{cc}{\beta }^0,\hfill & \text{if}\tilde{k}=0,\hfill \\ {\beta }^0(j),\hfill & \text{if}{\tilde{\tau }}_{j-1}<i∕n\le {\tilde{\tau }}_j,j=1,\dots ,\tilde{k}+1,\hfill \end{array}\phantom{\}}$$ (2)

where ${\beta }^0(j)=({\beta }_1^0(j),\dots ,{\beta }_p^0(j){)}^{\mathsf{T}}\in {\mathbb{R}}^p$ denotes the underlying true regression coefficients in the j-th interval. We focus on change point detection, which consists of estimating: (a) the number of change points $(\tilde{k})$; (b) the locations of change points $(\tilde{\tau })$; (c) the regression coefficients β⁰(j) in each segmentation, where j = 1,…, $\tilde{k}+1$.

There is a growing literature on change point detection. Most existing papers focus on change point problems in the mean, variance, or covariance matrix either for a fixed p (Kirch et al., 2015; Zhang & Lavitas, 2018) or for a growing p (Frick et al., 2014; Jirak 2015; Barigozzi et al., 2018; Wang & Samworth, 2018; Wang et al., 2021b). Progress has been made in the literature for detection of multiple change points as well (Lavielle & Teyssiére, 2006; Aue et al., 2009; Harchaoui & Lévy-Leduc, 2010; Cho & Fryzlewicz, 2015). Despite progress on change point detection, much fewer papers appear in the literature on regression change point problems, especially for high-dimensional models. The main difficulty comes from the complexity of both calculation and theoretical analysis arising from the growing dimension.

For regression problems, penalized techniques such as Lasso (Tibshirani, 1996) are popular in dealing with high-dimensional data. Some theoretical properties of the Lasso and various extensions can be found in Fan & Peng (2004), Candes & Tao (2007), and van de Geer et al. (2014). For a general overview and recent developments, we refer to Fan & Lv (2010) and Tibshirani (2011). In terms of change point detection based on Lasso, some methods exist for solving regression change point problems both in low and high dimensions. For example, designed for a fixed p, Ciuperca (2014) considered multiple change point estimation based on the Lasso. Qian & Su (2016) and Li et al. (2016) proposed a systematic change point estimation framework based on the adaptive fused Lasso. When the data dimension p grows to infinity, Lee et al. (2016) considered high-dimensional linear models with a possible change point and proposed a method for estimating regression coefficients as well as the unknown threshold parameter. As an extension, Leonardi & Bühlmann (2016) proposed computationally efficient algorithms for the number and locations of multiple change points in the context of high-dimensional linear models. Recently, Liu et al. (2019) investigated simultaneous change point detection and identification based on a de-biased Lasso process. Wang et al. (2021a) developed variance projected wild binary segmentation (VPWBS) for multiple change point detection.

Note that the above-mentioned papers focused on change point detection based on linear models with a continuous response, and thus are not directly applicable to the analysis of categorical or count response variables in practice. GLM can be very useful in this situation since it covers the exponential family distributions for the response variable. Because of its generality, GLM is widely used in various applications such as genetics, economics, and epidemiology. Several papers studied low-dimensional, single change point problems in the context of GLM (Lee & Seo, 2008; Lee et al., 2011; Fong et al., 2015). To the best of our knowledge, change point detection for high-dimensional GLMs has not been studied in the literature. Hence, it is desirable to consider a flexible and general framework for analyzing high-dimensional data with heterogeneity. Motivated by this, in this paper, we consider computationally efficient multiple change point detection in the context of high-dimensional GLMs. Our main contributions are summarized as follows:

• We consider change point problems in a more flexible and general framework of high-dimensional GLMs, allowing the data dimension p to grow exponentially with the sample size n. It covers various model settings including linear models, logistic, and probit models as special cases. As far as we know, change point detection for high-dimensional logistic and probit models has not been considered in the literature.

• Under the above framework, we propose a three-step procedure to estimate the number and locations of change points based on the Lasso estimator of the regression coefficients. The basic idea is to choose a useful contrast function J(τ(k)), which satisfies $J(\widehat{\tau }(\widehat{k}))<J(\tau (k))$ for any τ(k). To solve this optimization problem, we propose two algorithms based on dynamic programming and binary segmentation techniques, which have computational costs of O(n²GLMLasso(n)) and O(n log(n)GLMLasso(n)), respectively, where GLMLasso(n) is the cost to compute the Lasso estimator for the GLM with the sample size n. We also propose a much more efficient approach for the single change point case, with a computational cost of O(log(n)GLMLasso(n)). To the best of our knowledge, this is the most computationally efficient algorithm available for detecting a single change point in GLMs.

• We examine some theoretical properties of our proposed change point estimators computed by the three algorithms. To be specific, under some mild conditions, both the dynamic programming and binary segmentation techniques can obtain a consistent estimator for the number and locations of the true change points with a rate of $O_p(\sqrt{\log (p)∕n})$, which covers the case with an asymptotically growing number of change points. Moreover, the estimation error of the Lasso estimator of underlying regression coefficients can be bounded to o_p(1). To achieve further statistical prediction and inference, we introduce the de-biased Lasso estimator of the underlying regression coefficients in each segmentation, which is shown to be asymptotically normal. As for the third efficient approach designed for single change point cases, we establish that it can identify the location of the change point with high estimation accuracy. Finally, the competitive performance of our proposed methods is demonstrated by extensive numerical results as well as application to the Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset.

For a better understanding of our work, we would like to point out its relationship with several related papers. Compared to Lee et al. (2011), which considered single change point detection for the binary response variable with low dimensional covariates, we overcome the challenges of the computational and theoretical complexity arising from the growing dimension and number of change points. Meanwhile, to address the issue of the unknown multiple change points, we construct accurate and effective algorithms based on two techniques, dynamic programming and binary segmentation. These techniques are popular for multiple change point detection and were previously studied by Lavielle & Teyssiére (2006), Boysen et al. (2009), Harchaoui & Lévy-Leduc (2010), Cho & Fryzlewicz (2012, 2015), and Leonardi & Bühlmann (2016). Our extension to GLMs involves several technical challenges to overcome. One substantial difficulty comes from the complex form of the contrast functions compared to the least squares for linear models considered in Leonardi & Bühlmann (2016).

The rest of this paper is organized as follows. In Section 2, we introduce our methodology and demonstrate how our proposed three algorithms detect change points. In Section 3, the corresponding theoretical results of the change points computed by different algorithms are established. We investigate the performance of our proposed methods by extensive numerical results as well as a real data application in Section 4. We summarize the paper in Section 6. Detailed proofs of the main theorems and some useful lemmas are given in the Appendix.

2. METHODOLOGY

In this section, we introduce our new methodology for Model (1) with multiple unknown change points. In particular, in Section 2.1, some notation is introduced. In Section 2.2, we present a three-step change point estimator including the number and the locations of change points. Meanwhile, the regression coefficients in each segment are estimated based on the Lasso. In Sections 2.2.1 and 2.2.2, based on dynamic programming and binary segmentation techniques, two algorithms are proposed to detect multiple change points. To further improve the computational efficiency, in Section 2.2.3, we present a much more efficient algorithm designed for the case of a single change point.

2.1. Notation

We first introduce some notation. For a vector $\mathbf{a}=(a_1,\dots ,a_p{)}^{\mathsf{T}}\in {\mathbb{R}}^p$, we denote $\Vert \mathit{a}{\Vert }_1={\sum }_{i=1}^p\mid a_i\mid ,\Vert \mathit{a}{\Vert }_2=({\sum }_{i=1}^p{a}_i^2{)}^{1∕2}$, and ∥a∥_∞ = max_1≤i≤p∣a_i∣. For two real-valued sequences a_n and b_n, we set a_n = O(b_n) if there exists a constant C such that ∣a_n∣ ≤ C_n∣b_n∣, for a sufficiently large n. We set a_n = o(b_n) if a_n/b_n → 0 as n → ∞. For a sequence of random variables {ξ₁, ξ₂,…}, we set ${\xi }_n\stackrel{\mathbb{P}}{\to }\xi$ if ξ_n converges to ξ in probability as n → ∞. We also denote ξ_n = o_p(1) if ${\xi }_n\stackrel{\mathbb{P}}{\to }0$. Given an interval (u,v) ⊂ [0,1] such that $u,v\in L^1=\{i∕n,i=1,\dots ,n,n\in \mathbb{N}\}$, we denote the vector (Y_un+1,…,Y_vn)^⊤ by Y_(u,v) and the vector (ϵ_un+1,…,ϵ_vn)^⊤ by ϵ_(u,v). Analogously, we use X_(u,v) to denote the (v − u)n × p dimensional matrix $({\mathit{X}}_{(u,v)}^{(1)},\dots ,{\mathit{X}}_{(u,v)}^{(p)})$, where ${\mathit{X}}_{(u,v)}^{(j)}=(X_{un+1}^{(j)},\dots ,X_{vn}^{(j)}{)}^{\mathsf{T}}$ with j = 1,…,p, and we use ${\widehat{\beta }}_{(u,v)}$ to denote the Lasso estimator based on the observations Y_(u,v) and X_(u,v). For a set A, we use #A to denote its cardinality. For any x ≥ 0, we use [x] to denote the largest integer less than or equal to x. We use C₁,C₂,… to denote generic positive constants that may vary in different places.

2.2. New Estimation and Algorithms

Let Y = (Y₁,…, Y_n)^⊤ denote the n × 1 response vector, and X the n × p design matrix with X_i = (X_i1, …, X_ip)^⊤ being its i-th row for 1 ≤ i ≤ n. In this paper, we assume $\{{\mathit{X}}_i{\}}_{i=1}^n$ are independently and identically distributed (i.i.d.) p-dimensional random vectors with mean zero and covariance matrix Σ = Cov(X₁). Furthermore, for $j=1,\dots ,\tilde{k}+1$, we denote by S^(j) the set of non-zero elements of the regression coefficients ${\beta }^0(j),\text{i.e.},S^{(j)}=\#\{\ell :{\beta }_{\ell }^0(j)\ne 0\text{for}\ell =1,\dots ,p\}$. For any given partition τ = (τ₀, τ₁, …, τ_k, τ_k+1)^⊤, we denote the j-th interval by I_j(τ) = (τ_j−1, τ_j), the length of the j-th interval by r_j (τ) = τ_j − τ_j−1, the shortest interval length by r(τ) = min_i≤j≤k+1 r_j (τ), and the change point number by l(τ). Moreover, we denote the minimum interval length by δ.

We are now ready to introduce our change point estimator in detail. We consider the Lasso-type ℓ₁-penalized estimator for high-dimensional GLMs. Such estimators have some desirable properties. In particular, van de Geer (2008) derived some theoretical properties including consistency and the oracle inequality, based on which our algorithms are mainly constructed. More specifically, let ${\rho }_{\beta }(\mathit{x},y):\mathcal{X}\times \mathcal{Y}\to \mathbb{R}$ be some loss function relative to g(·). For instance, if g(·) is the logit function, ρβ(x, y) will then be the negative-likelihood function in the form log(1 + e^x⊤β) − yx^⊤β. For $\beta \in {\mathbb{R}}^p$, we define ${\stackrel{.}{\rho }}_{\beta }≔\frac{\partial {\rho }_{\beta }}{\partial \beta }$ and ${\ddot{\rho }}_{\beta }≔\frac{\partial {\rho }_{\beta }}{\partial \beta \partial {\beta }^T}$. Note that such complex loss functions lead to substantial difficulty for the estimation of change points as well as regression coefficients. Given data observations $\{Y_i,{\mathit{X}}_i{\}}_{i=1}^n$, the Lasso-based GLM method solves the following ℓ₁ penalized problem:

$$\widehat{\beta }=\text{arg}\underset{\beta }{\text{min}}\left\{\frac{1}{n}\sum _{i=1}^n{\rho }_{\beta }({\mathit{X}}_i,Y_i)+\lambda \Vert \beta {\Vert }_1\right\}.$$ (3)

Since we consider heterogeneous data with possible multiple change points, we cannot use Equation (3) directly to obtain parameter estimation. The main challenge is that both the number $(\tilde{k})$ and the locations $(\tilde{\tau })$ are unknown. To solve this issue, we consider three steps.

Before introducing the change point estimator, we first demonstrate how to estimate the regression coefficients for each segment. To be specific, for any given candidate partition τ = (τ₀, τ₁,…, τ_k, τ_k+1)^⊤, with τ_j ∈ L¹, j = 1,…, k + 1, we obtain the estimator with the ℓ₁ penalty in each segment by, for j = 1,…, k + 1,

$$\widehat{\beta }(\tau ,j)=\underset{\beta }{\text{arg min}}\left\{\frac{1}{n}\sum _{i=n{\tau }_{j-1}+1}^{n{\tau }_j}{\rho }_{\beta }({\mathit{X}}_i,Y_i)+{\lambda }_j\sqrt{({\tau }_j-{\tau }_{j-1})}\Vert \beta {\Vert }_1\right\},$$ (4)

where λ_j is the non-negative regularization parameter.

Based on Equation (4), our new algorithms for estimating both $\tilde{k}$ and $\tilde{\tau }$ are summarized into the following three steps.

Step 1 (Search the “best” partition): Given the candidate number of change points k, we find the “best” partition $\widehat{\tau }(k)=({\widehat{\tau }}_1,\dots ,{\widehat{\tau }}_k{)}^{\mathsf{T}}$ that minimizes the total loss function (contrast function):

$$\widehat{\tau }(k)=\underset{\tau =({\tau }_0,\dots ,{\tau }_{k+1}{)}^{\mathsf{T}}}{\text{arg min}}J(\tau ,\widehat{\beta }(\tau ),\mathbf{X},\mathit{Y})+\gamma (k+1),$$ (5)

where $\widehat{\beta }(\tau )≔(\widehat{\beta }(\tau ,1{)}^{\mathsf{T}},\dots ,\widehat{\beta }(\tau ,k+1{)}^{\mathsf{T}}{)}^{\mathsf{T}}$, $J(\tau ,\widehat{\beta }(\tau ),\mathbf{X},\mathit{Y})≔{\sum }_{j=1}^{k+1}{P}_n\rho \left(I_j(\tau ),\widehat{\beta }(\tau ,j))\right)$, and $P_n\rho (I_j(\tau ),\beta )≔\frac{1}{n}{\sum }_{i=n{\tau }_{j-1}+1}^{n{\tau }_j}{\rho }_{\beta }({\mathit{X}}_i,Y_i)$.

Step 2 (Estimate number of change points): We put $\widehat{\tau }(k)$ into $J(\tau ,\widehat{\beta }(\tau ),\mathbf{X},\mathit{Y})$ and obtain the minimum loss function associated with k as

$$G(k)≔J(\widehat{\tau }(k),\widehat{\beta }(\widehat{\tau }(k)),\mathbf{X},\mathit{Y})+\gamma (k+1).$$ (6)

Then, we find the “best” estimation which minimizes G(k) with a penalty:

$$\widehat{k}=\underset{k}{\text{arg min}}G(k).$$ (7)

Step 3 (Estimate locations of change points): We put $\widehat{k}$ into Step 1 and obtain the final change point estimator $\widehat{\tau }≔\widehat{\tau }(\widehat{k})=({\widehat{\tau }}_1,\dots ,{\widehat{\tau }}_{\widehat{k}}{)}^{\mathsf{T}}$ by

$$\widehat{\tau }=\underset{\tau =({\tau }_0,\dots ,{\tau }_{\widehat{k}+1}{)}^{\mathsf{T}}}{\text{arg min}}J(\tau ,\widehat{\beta }(\tau ),\mathbf{X},\mathit{Y}).$$ (8)

Combining Steps 1-3, our final change point estimators $\widehat{k}$ and $\widehat{\tau }$ can be obtained equivalently in the following form:

$$\widehat{\tau }=\underset{k}{\text{arg min}}\underset{\tau :l(\tau )=k}{\text{min}}\left\{J(\tau ,\widehat{\beta }(\tau ),\mathbf{X},\mathit{Y})+\gamma (k+1)\right\}.$$ (9)

After the above three steps, we obtain the change-point estimators $\widehat{\tau }$. As for β⁰, we recommend two different Lasso estimators of the underlying regression coefficients β⁰, serving different purposes for practitioners. In particular, naturally, we can use the Lasso estimator of β⁰ to select variables and make a prediction, which is defined for $j=1,\dots ,\widehat{k}+1$ as

$$\widehat{\beta }(\widehat{\tau },j)=\underset{\beta }{\text{arg min}}\left\{\frac{1}{n}\sum _{i=n{\widehat{\tau }}_{j-1}+1}^{n{\widehat{\tau }}_j}{\rho }_{\beta }({\mathit{X}}_i,Y_i)+{\lambda }_j\sqrt{({\widehat{\tau }}_j-{\widehat{\tau }}_{j-1})}\Vert \beta {\Vert }_1\right\}.$$

For further statistical inference including confidence intervals and hypothesis testing, van de Geer et al. (2014) proposed the de-biased Lasso estimator and analyzed its asymptotic properties for the homogeneous model under high-dimensional setups. Similarly, for the heterogeneous observations, we construct a de-biased Lasso estimator for the underlying regression coefficients for each segmentation for $j=1,\dots ,\widehat{k}+1$ as

$$\tilde{\beta }(\widehat{\tau },j)=\widehat{\beta }(\widehat{\tau },j)-\widehat{\Theta }{P}_n{\stackrel{.}{\rho }}_{\widehat{\beta }(\widehat{\tau },j)},$$

where the precision matrix estimator $\widehat{\Theta }={\widehat{\Theta }}_{\text{Lasso}}$ can be constructed using nodewise Lasso with $\widehat{\Sigma }≔P_n{\ddot{\rho }}_{\widehat{\beta }(\widehat{\tau },j)}$ as input (van de Geer et al., 2014).

In what follows, we introduce three specific algorithms for solving Equation (9). Note that J(τ, β, X, Y) and P_nρ(I_j(τ), β) are the loss function for all intervals and the j-th interval, respectively. Meanwhile, λ_j in Equation (4) and γ in Equation (5) are positive tuning parameters that encourage coefficient and segment sparsity, respectively. We adopt a cross-validation approach to make a proper choice of these two tuning parameters λ and γ. To compute $\widehat{\beta }(\tau ,j)$ in Equation (4), we can use, for example, the R package glmnet (https://glmnet.stanford.edu). It is worth mentioning the following two remarks for our proposed estimator: (1) If the number of change points $\tilde{k}$ is known, we just use Step 1 by plugging in $k=\tilde{k}$ to directly obtain the locations of change points as follows:

$$\widehat{\tau }(\tilde{k})=\underset{\tau =({\tau }_0,\dots ,{\tau }_{\tilde{k}+1}{)}^{\mathsf{T}}}{\text{arg min}}J(\tau ,\widehat{\beta }(\tau ),\mathbf{X},\mathit{Y}).$$ (10)

In this case, our method covers the setting considered by Lee et al. (2016), where at most one change point is assumed. (2) When no change point occurs ($\tilde{k}=0$), our proposed method can still work. Hence, our proposed method can automatically account for the underlying data generation mechanism ($\tilde{k}=0$ or $\tilde{k}>0$) without specifying any prior knowledge about the number of change points $\tilde{k}$. Furthermore, as shown by our extensive numerical studies, our new algorithms can estimate $\tilde{k}$ with high accuracy.

Our main goal is to design efficient algorithms that solve the optimization problem in Equation (9) of the form $J(\tau ,\widehat{\beta })+\text{Pen}(\tau )$. To address this issue, three algorithms are proposed next.

2.2.1. Dynamic programming approach

We introduce a general approach based on the Dynamic Programming Algorithm (DPA), which works well for our change point problem (Eq. 9). It is well-known that DPA has excellent accuracy since it considers the global solution of Equation (9). It is widely used in multiple change point detection including the efficient, parallelized approaches introduced recently by Tickle et al. (2020). More details can be found in Boysen et al. (2009) and Leonardi & Bühlmann (2016). Next, we present how to use this technique to solve Equation (9) in detail.

For any given v ∈ {i/n : i = 1, …,n}, consider the sample (Y_(0,v], X_(0,v]). Given a candidate change point number k, denote F_k(v) as the minimum value as follows:

$$F_k(v)≔\underset{\tau :l(\tau )=k}{\text{min}}\sum _{j=1}^{k+1}\left(P_n\rho \left(I_j(v\tau ),\widehat{\beta }(\tau ,j)\right)+\gamma \right).$$ (11)

One can see that the optimal k + 1 segments $\{({\tau }_{j-1},{\tau }_j){\}}_{j=1}^{k+1}$ corresponding to the change point vector τ = (τ₀, τ₁, …, τ_k+1)^⊤ obtained from Equation (11), consist of the optimal first k segments $\{({\tau }_{j-1},{\tau }_j){\}}_{j=1}^k$ and a single segment (τ_k, τ_k+1)^⊤. Recall that τ₀ := 0 and τ_k+1 := 1. Then τ_k is the rightmost change point estimator. Furthermore, by definition of F_k(v), $\{({\tau }_{j-1},{\tau }_j){\}}_{j=1}^k$ obtained from Equation (11) is also a minimizer of F_k−1(τ_k). Hence, the last change point τ_k is the minimizer of $F_{k-1}(u)+P_n\rho ((u,v),{\widehat{\beta }}_{(u,v)})+\gamma$ with u < v.

The above observation motivates us to use the dynamic programming recursion to calculate F_k(v) with v ∈ {i/n : i = 1, …, n}. In particular, for any v ∈ {i/n : i = 1, …, n}, define

$$F_0(v)=P_n\rho \left((0,v),{\widehat{\beta }}_{(0,v)}\right)+\gamma .$$ (12)

Then, the dynamic programming recursion proceeds as follows:

$$F_k(v)=\underset{\underset{u<v}{u\in \{i∕n:i=1,\dots ,n\}}}{\text{min}}\left\{F_{k-1}(u)+P_n\rho \left((u,v),{\widehat{\beta }}_{(u,v)}\right)+\gamma \right\},v\in \{i∕n:i=1,\dots ,n\}.$$ (13)

Define V_n = {i/n : i = 1, …, n}. Based on Equations (12) and (13), we can obtain {F₁(v), v ∈ V_n}, {F₂(v), v ∈ V_n}, …, and {F_kmax (v), v ∈ V_n}, where k_max (in our case k_max + 1 = 1/δ) is an “upper bound” of the number of change points. See Section 3.2 for more details. By considering G(k) in (6), we have F_k(1) = G(k) with k = 1, …, k_max. Hence, we are ready to estimate the change point number by

$$\widehat{k}=\underset{k=1,\dots ,k_{\max }}{\text{arg min}}{F}_k(1).$$ (14)

The corresponding locations of change points $\widehat{\tau }=(0,{\widehat{\tau }}_1,\dots ,{\widehat{\tau }}_{\widehat{k}},1{)}^{\mathsf{T}}$ can be obtained by

$${\widehat{\tau }}_j=\underset{u\in V_n,u<{\widehat{\tau }}_{j+1}}{\text{arg min}}\{F_{j-1}(u)+P_n\rho \left((u,{\widehat{\tau }}_{j+1}),{\widehat{\beta }}_{(u,{\widehat{\tau }}_{j+1})}\right)+\gamma ,\text{for}j=\widehat{k},\dots ,1.$$ (15)

The following Algorithm 1 describes our procedure for obtaining $\widehat{k}$ and $\widehat{\tau }$ based on the DPA. Note that DPA solves Equation (9) with globally optimal solutions, which have excellent estimation accuracy. Furthermore, as shown in Leonardi & Buhlmann (2016), it has the computational cost of O(n²GlmLasso(n)) operations. This can be computationally expensive especially when n is very large. Hence, it is desirable to consider a more efficient approach. Next, we introduce an efficient approach based on binary segmentation, which can ensure almost the same estimation accuracy as that of DPA.

Algorithm 1 : Dynamic programming procedure for change point detection in high-dimensional GLMs.

$$\begin{array}{cc}\hfill & \mathrm{Input}:\text{Given the dataset}\{\mathbf{X},\mathit{Y}\},\text{set the value of}{k}_{\max }.\hfill \\ \hfill & \mathbf{Step}1:\text{Based on Equations}(12)\text{-}(13),\text{compute}{F}_k(1)\text{for}k=1,\dots ,k_{\max }.\hfill \\ \hfill & \mathbf{Step}2:\text{Obtain estimate of the number}\text{of change points}\widehat{k}\text{by Equation}(14).\hfill \\ \hfill & \mathbf{Step}3:\text{Obtain estimate of the}\text{change point locations}\widehat{\tau }\text{by Equation}(15).\hfill \\ \hfill & \mathrm{Output}:\text{Algorithm}1\text{provides}\text{the}\text{change}\text{point}\text{estimator}\widehat{\tau }(\widehat{k})=\hfill \\ \hfill & (0,{\widehat{\tau }}_1,\dots ,{\widehat{\tau }}_{\widehat{k}},1{)}^{\mathsf{T}},\text{including both the number and locations}.\hfill \end{array}$$

2.2.2. Binary segmentation approach

Next we introduce an approach based on the binary segmentation algorithm (BSA) examined in Cho & Fryzlewicz (2012; 2015), and Leonardi & Bühlmann (2016), which is shown to be much more efficient compared to DPA. The main idea of BSA for solving the change point problem for GLMs (Eq. 9) is that for each candidate search interval (u, v), we use the penalized loss function to determine whether a new change point s can be added. If s is identified, then the interval (u, v) is split into two subintervals (u, s) and (s, v) and we conduct the above procedure on (u, s) and (s, v) separately. This algorithm is continued until no new subintervals can be added. In particular, for any given u, v ∈ V_n := {i/n : i = 1, …, n}, we define

$$Z(u,v)=\{\begin{array}{cc}{P}_n\rho \left((u,v],{\widehat{\beta }}_{(u,v]}\right)+\gamma ,\hfill & \text{if}(v-u)n\ge 1\hfill \\ 0,\hfill & \text{otherwise},\hfill \end{array}\phantom{\}}$$ (16)

and

$$h(u,v)=\underset{s\in \{u\}\cup [u+\delta ,v-\delta ]}{\text{arg min}}\{Z(u,s)+Z(s,v)\}.$$ (17)

Then we present our BSA-based algorithm as follows.

Algorithm 2 : Binary segmentation procedure for change point detection in high-dimensional GLMs.

$$\begin{array}{cc}\hfill & \mathrm{Input}:\text{Given the dataset}\{\mathbf{X},\mathit{Y}\},\text{initialize the set}\text{of change}\text{point pairs}T=\hfill \\ \hfill & \{0,1\}.\hfill \\ \hfill & \mathbf{Step}1:\text{For each}\text{pair}\{u,v\}\text{in}T,\text{compute}s=h(u,v)\text{as defined in Equation}\hfill \\ \hfill & (17).\text{If}s>u,\text{add new}\text{pairs of nodes}\{u,s\}\text{and}\{s,v\}\text{to}T\text{and update}T\text{as}\hfill \\ \hfill & T=T\cup \{u,s\}\cup \{s,v\}.\hfill \\ \hfill & \mathbf{Step}2:\text{Repeat Step}1\text{until no more new pair of nodes can be added}.\text{Denote the}\hfill \\ \hfill & \text{terminal set of change point pairs by}{T}_{\text{final}}={\cup }_{i=1}^q\{u_i,v_i\}.\hfill \\ \hfill & \mathrm{Output}:\text{Algorithm}2\text{provides the change point estimator}{\widehat{\tau }}^b=({\widehat{\tau }}_0^b,\dots ,{\widehat{\tau }}_{{\widehat{k}}^b+1}^b{)}^{\mathsf{T}},\hfill \\ \hfill & \text{where}{\widehat{k}}^b=\#T_{\text{final}}\text{and}0={\widehat{\tau }}_0^b<{\widehat{\tau }}_1^b<\dots <{\widehat{\tau }}_{{\widehat{k}}^b}^b<{\widehat{\tau }}_{{\widehat{k}}^b+1}^b=1\in T_{\text{final}},\text{including}\hfill \\ \hfill & \text{both the number and locations}.\hfill \end{array}$$

Note that this approach searches much fewer candidates for finding a new change point as compared to DPA, which makes it more computationally efficient. More specifically, as shown by Leonardi & Bühlmann (2016), BSA has a computational cost of O(n log(n)GlmLasso(n)) operations. Furthermore, in Section 3.2, we prove that the change point estimator computed by Algorithm 2 enjoys almost the same estimation accuracy as that of Algorithm 1.

2.2.3. A fast screening approach for single change point models

So far, we have proposed two efficient algorithms in Sections 2.2.1 and 2.2.2 for solving Equation (9). In this section, we show that under the single change point models, the computational cost can be further reduced. As far as we know, our fast screening approach (FSA) is novel for detecting a single change point in regression models. The main idea is that for detecting a single change point, if we have some prior information about its location, it is not necessary to search all candidate subintervals that have been adopted in the BSA-based algorithm. To see this, we recall Z(u, v) as defined in Equation (16). For τ_f ∈ (0, 1), we define the statistics as W_τ^f((u, v)) = Z(u, u + τ^f (v − u)) + Z(u + τ^f (v − u),v). Based on W_τ^f ((u, v)), we have the following key observation: Consider any subinterval (u, v) containing a single change point $\tilde{\tau }\in (u,v)$. If $\tilde{\tau }$ lies in the first half interval of (u, v), i.e., $\tilde{\tau }\in (u,u+\frac{1}{2}(v-u))$, with a high probability, we can prove that W_1/4((u, v)) < W_3/4((u, v)). If $\tilde{\tau }$ lies in the second half interval of (u, v), i.e., $\tilde{\tau }\in (u+\frac{1}{2}(v-u),v)$, with a high probability, we have W_3/4((u, v)) < W_1/4((u, v)). The above observation motivates us to design Algorithm 3 for fast change point identification.

Note that Algorithm 3 does not need to search through all data points, and it can quickly identify the half-interval where the change point is located via comparing the quarter, half, and three-quarter values of W(T_i) in each iteration. As a result, it only takes O(log(n)GLMLasso(n)) computational operations to detect the change point. Hence, as compared to Algorithms 1 and 2, the computational cost can be dramatically reduced. Its computational benefits will be validated by our numerical experiments in Section 4.

Algorithm 3 : A fast screening approach for single change point detection in high-dimensional GLMs.

$$\begin{array}{cc}\hfill & \mathrm{Input}:\text{Input the dataset}\{\mathbf{X},\mathit{Y}\}.\hfill \\ \hfill & \mathbf{Step}0:\text{Set}{u}_0=0,v_0=1.\hfill \\ \hfill & \mathbf{Step}1:\text{For each iteration}i=0,1,2,\dots ,\text{let}{T}_i=[u_i,v_i].\text{Calculate the values}\hfill \\ \hfill & \text{of}{W}_{\frac{1}{4}}(T_i),W_{\frac{1}{2}}(T_i),\text{and}{W}_{\frac{3}{4}}(T_i).\text{Set}M(T_i)=\text{min}(W_{\frac{1}{4}}(T_i),W_{\frac{1}{2}}(T_i),W_{\frac{3}{4}}(T_i)).\hfill \\ \hfill & \text{For each iteration}i,\text{consider the following three cases:}\hfill \\ \hfill & \text{If}M(T_i)=W_{\frac{1}{4}}(T_i),\text{set}{T}_{i+1}=[u_i,u_i+1∕2(v_i-u_i)];\hfill \\ \hfill & \text{if}M(T_i)=W_{\frac{1}{2}}(T_i),\text{set}{T}_{i+1}=[u_i+1∕4(v_i-u_i),u_i+3∕4(v_i-u_i)];\hfill \\ \hfill & \text{if}M(T_i)=W_{\frac{3}{4}}(T_i),\text{set}{T}_{i+1}=[u_i+1∕2(v_i-u_i),v_i].\hfill \\ \hfill & \mathbf{Step}2:\text{Repeat Step}2\text{until}[n\ast (v_{i^{\ast }}-u_{i^{\ast }})]\le 4\text{holds for some}{i}^{\ast }.\text{Denote}{T}_{i^{\ast }}\hfill \\ \hfill & \text{by}\widehat{T}.\hfill \\ \hfill & \mathbf{Step}3:\text{Calculate}{\widehat{\tau }}^f={\text{arg min}}_{\tau \in \widehat{T}}{W}_{{\tau }^f}(\widehat{T}).\hfill \\ \hfill & \mathrm{Output}:\text{This algorithm provides a single change point estimator}{\widehat{\tau }}^f.\hfill \end{array}$$

3. THEORETICAL PROPERTIES

We examine the theoretical properties of our proposed three approaches. In particular, we first show that our estimation of change points and regression coefficients is consistent and has the same rates of convergence as those of linear models. Secondly, for GLMs with change points, we reconstruct our assumptions and lemmas for analyzing high-dimensional data with heterogeneity based on the work of van de Geer (2008). Note that van de Geer (2008) considered the ℓ₁ penalized estimation of the regression coefficients under the setting of all observations from the same GLM. In particular, in Section 3.1, we introduce some assumptions. In Section 3.2, we present theoretical results of the change point estimator computed by the new algorithms.

Before presenting the theoretical results, we introduce some additional notation. For any u, v ∈ V_n := {i/n : i = 1, …, n} with u < v, we denote, for a function $\rho (\mathit{x},y):\mathcal{X}\times \mathcal{Y}\to \mathbb{R}$, the subinterval-based theoretical mean and empirical mean by $P\rho ((u,v))≔\frac{1}{n}{\sum }_{i=un+1}^{vn}\mathbb{E}\rho ({\mathit{X}}_i,Y_i)$, and $P_n\rho ((u,v))≔\frac{1}{n}{\sum }_{i=un+1}^{vn}\rho ({\mathit{X}}_i,Y_i)$, respectively. For convenience, we denote Pρ = Pρ((0, 1)) and P_nρ = P_nρ((0, 1)). Consider a linear subspace $\mathcal{F}≔\left\{f_{\beta }(\mathit{x})={\mathit{x}}^{\mathsf{T}}\beta :\beta \in {\mathbf{R}}^p\right\}$. For a $f_{\beta }\in \mathcal{F}$, define ρ_fβ(x, y) = ρ(f_β (x), y). Then the empirical risk and theoretical risk at f are defined as P_nρf and P_ρf, respectively. Furthermore, we define the target as the minimizer of the theoretical risk $f^0≔{\text{arg min}}_{f\in \mathcal{F}}P{\rho }_f$ and ${\beta }^0≔{\text{arg min}}_{\beta \in {\mathbb{R}}^p}P{\rho }_{f_{\beta }}$, where β⁰ can be regarded as the “truth”. By definition, we have f⁰(x) = x^⊤β⁰. For $f_{\beta }\in \mathcal{F}$, the excess risk is defined as $\mathcal{E}(f_{\beta })≔P\left({\rho }_{f_{\beta }}-{\rho }_{f^0}\right)$ . Lastly, for any subinterval (u, v), we define the oracle ${\beta }_{(u,v)}^{\ast }$ as ${\beta }_{(u,v)}^{\ast }≔{\text{arg min}}_{\beta \in {\mathbb{R}}^p}\left\{\mathcal{E}(f_{\beta })\right\}$. The corresponding estimation error is then denoted as ϵ* := (P_n − P) ρf_β*.

3.1. Basic assumptions

We introduce some assumptions as follows.

Assumption A (loss function). The loss function ρf(x, y) := ρ(f(x), y) is convex for all $y\in \mathbb{R}$. Moreover, it satisfies the Lipschitz property:

$$\mid \rho (f_{\beta }(x),y)-\rho (f_{\tilde{\beta }}(x),y)\mid \le \mid f_{\beta }(x)-f_{\tilde{\beta }}(x)\mid ,\forall (x,y)\in \mathcal{X}\times \mathcal{Y},\forall \beta ,\tilde{\beta }\in {\mathbb{R}}^p.$$

Assumption B (design matrix). There exists K_X < ∞ such that ∥X_i∥_∞ ≤ K_X and $\mathbb{E}({\mathit{X}}_i)=0$ hold for all i = 1, …, n.

Assumption C (margin condition). There exists an η > 0 and a strictly convex increasing function G(x), such that for all $\beta \in {\mathbb{R}}^p$ with ∥f_β − f⁰∥_∞ ≤ η, one has

$$\mathcal{E}(f_{\beta })\ge G(\Vert f_{\beta }-f^0\Vert ),$$

where there exists a constant C such that G(x) ≥ Cx² for any positive x.

Assumption D (compatibility condition). The compatibility condition is met for the set $S_{\ast }={\cup }_{j=1}^{\tilde{k}+1}{S}^{(j)}$ (S^(j) defined in Section 2.2) with constant ϕ_* > 0, if for all β ∈ R^pp satisfying ${\Vert {\beta }_{S_{\ast }^c}\Vert }_1\le 3\Vert {\beta }_{S_{\ast }}{\Vert }_1$, it holds that

$$\Vert {\beta }_{S_{\ast }}{\Vert }_1^2\le ({\beta }^{\mathsf{T}}{\mathbf{X}}^{\mathsf{T}}\mathbf{X}\beta )s_{\ast }∕{\varphi }_{\ast }^2,$$

where s_* := #S_* is the cardinality of S_*.

Assumption E (parameter space). For k₀ > 1, there exist constants m_* > 0 and M_* > 0 such that

$$\underset{1\le j\le j\le k\le \tilde{k}+1}{\text{min}}\frac{{\Vert {\sum }_{r=i}^j\gamma (i,r,j){\beta }^0(r)-{\sum }_{r=j+1}^k\gamma (j+1,r,k){\beta }^0(r)\Vert }_1}{s_{\ast }}\ge m_{\ast },$$

where $\gamma (i,j,k)=\frac{{\tilde{\tau }}_j-{\tilde{\tau }}_{j-1}}{{\tilde{\tau }}_k-{\tilde{\tau }}_{i-1}}$,

$$s_{\ast }=o(\sqrt{n}∕\log (p)),\underset{1\le j\le \tilde{k}+1}{\max }\Vert {\beta }^0(j){\Vert }_{\infty }\le M_{\ast },\text{and}\underset{1<j\le \tilde{k}+1}{\max }\Vert {\beta }^0(j)-{\beta }^0(j-1){\Vert }_{\infty }\le M_{\ast }.$$

Note that in the case $\tilde{k}=1$, the former condition reduces to ∥β⁰(1) − β⁰(2)∥₁≥ m_*s_*.

We assume in Assumption A that the loss function ρ is Lipschitz in f, which allows us to bound the loss function by the difference between estimated regression parameters and the corresponding true parameters. Many functions can meet this condition: for example, the negative-likelihood function of the logistic regression model. Assumption B imposes relatively weak conditions on the covariates, which covers a wide range of distributional patterns. Assumption C (margin condition) is assumed for a “neighbourhood” of the target linear function f⁰ = X^Tβ⁰ and is a common condition for analyzing the GLM. See Section 6.4 in Bühlmann & van de Geer (2011) for more details. Assumption D (compatibility condition) for the design matrix X allows us to establish oracle results for Lasso estimation. Note that one can verify that Assumption D is a sufficient condition of Assumption C in van de Geer (2008) by choosing the function $D(\mathcal{K})=\#\mathcal{K}{\beta }^2$, where $\#\mathcal{K}$ is the cardinality of the set $\mathcal{K}\subset \{1,\dots ,p\}$ defined in Assumption C of van de Geer (2008). Assumption E presents the minimum and maximum differences between the true regression parameters, which allow us to detect the change points. Furthermore, the sparsity of regression coefficients is required to guarantee the consistency of our proposed estimators. Assumption F introduced in the Appendix imposes some technical conditions on the tuning parameter λ for the Lasso estimation as well as the tuning parameter γ for the change point estimation. Assumption G includes the required condition for the limiting property of the de-biased Lasso estimator.

3.2. Main results

We are ready to present some theoretical results of our proposed three new algorithms. Before that, we denote $c^{\ast }=m_{\ast }^2{\varphi }_{\ast }^2∕M^{\ast }$ and let d_* be a constant. See more details in Lemma 5. We first present the property of the estimators computed by DPA in Algorithm 1.

Theorem 1 Suppose Assumptions A-G hold with log(p) = o(n). Then, for a given C₁ > 0, with probability at least 1 − 7 exp $\left(-C_1\frac{n^2}{\log (p)}\right)$, we have that

1. $l(\widehat{\tau })=\tilde{k}$

2. $\Vert \widehat{\tau }-\tilde{\tau }{\Vert }_1\le c^{\ast }\sqrt{\delta }\lambda$

3. $\sum _{j=1}^{\tilde{k}+1}\mid P_n\rho \left(I_j(\widehat{\tau }),\widehat{\beta }(\widehat{\tau },j)\right)-P_n\rho \left(I_j(\widehat{\tau }),{\beta }^0(j)\right)\mid +\lambda r_j(\widehat{\tau })\Vert \widehat{\beta }(\widehat{\tau },j)-{\beta }^0(j){\Vert }_1\le (\tilde{k}+1)d_{\ast }{s}_{\ast }{\lambda }^2$;

4. for each $j\in \{1,\dots ,\widehat{k}\}$,

   $$\sqrt{n}\left({\tilde{\beta }}_s(\widehat{\tau },j)-{\beta }_s^0(j)\right)∕{\widehat{\sigma }}_{j,s}=V_{j,s}+o_{\mathbf{P}}(1)fors\in \{1,\dots ,p\},$$

   where ${\tilde{\beta }}_s(\widehat{\tau },j)$ is the s-th component of $\tilde{\beta }(\widehat{\tau },j),V_{j,s}\sim \mathcal{N}(0,1)$ and ${\widehat{\sigma }}_{j,s}^2≔({\widehat{\Theta }}_j{P}_n{\stackrel{.}{\rho }}_{\widehat{\beta }}{\stackrel{.}{\rho }}_{\widehat{\beta }}^T{\widehat{\Theta }}_j^T{)}_{s,s}$.

Theorem 1 demonstrates that Algorithm 1 can identify both the number and locations of multiple change points with high estimation accuracy. In particular, the first result shows that we can obtain a consistent estimator $l(\widehat{\tau })$ for the true number of change points. As for the locations, the second result indicates that our multiple change point estimator $\widehat{\tau }$ converges to the true change point vector $\tilde{\tau }$ with a rate of $O_p(\sqrt{\log (p)∕n})$. Furthermore, the third result implies that we can bound the prediction error or the estimation error of underlying regression parameters within a rate of $O_p(\tilde{k}{s}_{\ast }{\lambda }^2)$ or $O_p(\tilde{k}{s}_{\ast }\lambda )$. Result (4) implies the asymptotic normality of the de-biased Lasso estimator $\tilde{\beta }(\widehat{\tau })$, which allows the wider statistical inference including confidence intervals and hypothesis testing.

Based on Theorem 1, some other interesting conclusions can be made. To simplify the discussion, we require that all the $\tilde{k}+1$ change point intervals are within the same order of magnitude. Recall δ as the minimum length of change point intervals as defined in Section 2.2. Then we have $\tilde{k}(k_{\max })=O(1∕\delta )$. Furthermore, according to δ, the following two cases are considered: (1) δ = O(1) and (2) δ = o(1).

For the first case, we have $\tilde{k}=O(1)$, which means that the number of change points is fixed and does not increase with the sample size n. Furthermore, considering Assumption (F1), we have $\lambda =O(\sqrt{\log (p)∕n})$. Hence, the three results in Theorem 1 reduce to:

$$\begin{gathered} \Vert \widehat{\tau }-\tilde{\tau }{\Vert }_1=O_p(\sqrt{\log (p)∕n}), \\ \sum _{j=1}^{\tilde{k}+1}\mid P_n\rho \left(I_j(\widehat{\tau }),\widehat{\beta }(\widehat{\tau },j)\right)-P_n\rho (I_j(\widehat{\tau }),{\beta }^0(j))\mid =O_p\left(s_{\ast }\frac{\log (p)}{n}\right),\text{and} \\ \sum _{j=1}^{\tilde{k}+1}\Vert \widehat{\beta }(\widehat{\tau },j)-{\beta }^0(j){\Vert }_1=O_p\left(s_{\ast }\sqrt{\frac{\log (p)}{n}}\right). \end{gathered}$$ (18)

Considering Equation (18), our results are consistent with the Lasso estimation results derived in van de Geer (2008) and estimation consistency is guaranteed as long as $s_{\ast }\sqrt{\log (p)∕n}=o(1)$ holds.

We next consider the second case with δ = o(1). In this case, we allow the number of change points grow with n. Noting that $\lambda \sqrt{\delta }=O(\sqrt{\log (p)∕n})$, the three results in Theorem 1 reduce to:

$$\begin{gathered} \Vert \widehat{\tau }-\tilde{\tau }{\Vert }_1=O_p(\sqrt{\log (p)∕n}), \\ \sum _{j=1}^{\tilde{k}+1}\mid P_n\rho \left(I_j(\widehat{\tau }),\widehat{\beta }(\widehat{\tau },j)\right)-P_n\rho (I_j(\widehat{\tau }),{\beta }^0(j))\mid =O_p\left(s_{\ast }\frac{\log (p)}{n{\delta }^2}\right),\text{and} \\ \sum _{j=1}^{\tilde{k}+1}\Vert \widehat{\beta }(\widehat{\tau },j)-{\beta }^0(j){\Vert }_1=O_p\left(s_{\ast }{\delta }^{-3∕2}\sqrt{\frac{\log (p)}{n}}\right). \end{gathered}$$ (19)

Hence, by Equation (19), the estimation consistency can still be obtained as long as $s_{\ast }{\delta }^{-3∕2}\sqrt{\frac{\log (p)}{n}}=o(1)$ holds. In other words, the number of change points $\tilde{k}$ can not grow faster than the order of ${\left(\frac{n}{\log (p)s_{\ast }^2}\right)}^{1∕3}$.

Next, we present theoretical results of change point estimators computed by BSA.

Theorem 2 Suppose Assumptions A-G hold with log(p) = o(n). For a given C₂ > 0, with probability at least 1 − 7 exp $\left(-C_2\frac{n^2}{\log (p)}\right)$, we have that

1. $l({\widehat{\tau }}^b)=\tilde{k}$;

2. $\Vert {\widehat{\tau }}^b-\tilde{\tau }{\Vert }_1\le c^{\ast }\sqrt{\delta }\lambda$;

3. $\sum _{j=1}^{\tilde{k}+1}\mid P_n\rho \left(I_j({\widehat{\tau }}^b),\widehat{\beta }({\widehat{\tau }}^b,j)\right)-P_n\rho \left(I_j({\widehat{\tau }}^b),{\beta }^0(j)\right)\mid +\lambda r_j({\widehat{\tau }}^b)\Vert \widehat{\beta }({\widehat{\tau }}^b,j)-{\beta }^0(j){\Vert }_1\le (\tilde{k}+1)d_{\ast }{s}_{\ast }{\lambda }^2$;

4. for each $j\in \{1,\dots ,\widehat{k}\}$,

   $$\sqrt{n}\left({\tilde{\beta }}_s(\widehat{\tau },j)-{\beta }_s^0(j)\right)∕{\widehat{\sigma }}_{j,s}=V_{j,s}+o_{\mathbf{P}}(1),fors\in \{1,\dots ,p\},$$

   where ${\tilde{\beta }}_s(\widehat{\tau },j)$ is the s-th component of $\tilde{\beta }(\widehat{\tau },j)$, $V_{j,s}\sim \mathcal{N}(0,1)$ and ${\widehat{\sigma }}_{j,s}^2≔({\widehat{\Theta }}_j{P}_n{\stackrel{.}{\rho }}_{\widehat{\beta }}{\stackrel{.}{\rho }}_{\widehat{\beta }}^T{\widehat{\Theta }}_j^T{)}_{s,s}$.

Theorem 2 shows similar results as those of Theorem 1 in terms of consistency of both the number and locations of change points. Furthermore, Theorem 2 allows us to use a much more efficient algorithm to detect multiple change points for GLMs, which enjoys almost the same estimation accuracy as that of the global solutions. The efficiency will be further investigated in our numerical experiments.

Finally, we establish theoretical properties of FSA proposed in Algorithm 3 for single change point models.

Theorem 3 Suppose Assumptions A-F hold with log(p) = o(n). Assume that the true single change point $\tilde{\tau }\in (0,1∕2)$. For a given C₃ > 0, with probability at least 1 − 7 exp $\left(-C_3\frac{n^2}{\log (p)}\right)$, we have that

$$W_{\frac{1}{4}}((0,1))<W_{\frac{3}{4}}((0,1)).$$ (20)

Theorem 3 justifies the validity of Algorithm 3 and demonstrates that the cost of identifying a single change point in GLMs can be reduced to only O(log(n)GLMLasso(n)) computational operations.

4. SIMULATION STUDIES

In this section, we investigate the numerical performance of our three proposed change point detection procedures in various model settings. For the design matrix X, we generate X_i i.i.d. from $\mathcal{N}(0,\Sigma )$. We first consider two types of covariance matrix structures including independent and weakly dependent settings as follows:

Case 1: Σ = I_p×p;

Case 2: Σ = Σ* with ${\Sigma }^{\ast }=({\sigma }^{\ast }{)}_{i,j=1}^p$, where ${\sigma }_{i,j}^{\ast }={0.8}^{\mid i-j\mid }$ for 1 ≤ i, j ≤ p.

We consider logistic regression models. For i = 1, …, n, we generate Y_i ∈ {0,1} with $g(\mathbb{P}(Y_i=1))-\log \frac{\mathbb{P}(Y_i=1)}{1-\mathbb{P}(Y_i=1)}={\mathit{X}}_i^T{\beta }^{(i)}$. Then the responses $\{Y_i{\}}_{i=1}^n$ are generated from the following Binomial distribution: $Y_i\mid {\mathit{X}}_i\sim \text{Bin}(1,\frac{\text{exp}({\mathit{X}}_i^T{\beta }^{(i)})}{1+\text{exp}({\mathit{X}}_i^T{\beta }^{(i)})})$.

For this model setup, we investigate the performance of our approaches in terms of accuracy and efficiency. For efficiency, we compare our proposed algorithms in terms of the computational cost. Note that BSA and DPA are designed for multiple change point detection. In order to compare efficiency reasonably for the cases with no change point and a single change point, we set these two algorithms to stop after one screening by making k_max = 1. To show the accuracy, we record the mean, mean squared error (MSE), and error rate (proportion of false positives) of the change point estimators including the number and locations. We compare the corresponding results with the following existing methods:

• Lee et al. (2011) (denoted by Lee2011), which is based on the maximum score estimation.

• Qian and Su (2016) (denoted by SGL), which proposed a systematic estimation framework based on the adaptive fused Lasso in linear regression models. To be specific, they estimate $\{{\beta }_t{\}}_{t=1}^n$ by minimizing the ℓ₂-loss with the fused Lasso penalty. In this paper, we modify SGL by replacing the ℓ₂-loss with the loss ρ defined in Section 2 for high-dimensional GLMs.

• Wang et al. (2021a) (denoted by VPWBS), i.e., the variance projected wild binary segmentation based on the sparse group Lasso estimator for linear regression models. In particular, they projected the high-dimensional time series $\{{\mathit{X}}_i,Y_i{\}}_{i=1}^n$ onto the univariate time series $\{z_i(u){\}}_{i=1}^n$. The optimal projection direction u is obtained by local group Lasso screening (LGS). Then they conducted mean change point detection by wild binary segmentation (WBS) on the univariate time series $\{z_i(u){\}}_{i=1}^n$. Note that, for linear models, LGS performs a variant of the group Lasso on any subsample $\{{\mathit{X}}_i,Y_i{\}}_{i=s+1}^e$, and computes

  $$\begin{gathered} ({\widehat{\alpha }}_1,{\widehat{\alpha }}_2,\widehat{\nu })\leftarrow \underset{\begin{array}{c}\hfill \nu \in [s^{\prime }+1,e^{\prime }-1]\hfill \\ \hfill {\alpha }_1,{\alpha }_2\in {\mathbb{R}}^p\hfill \end{array}}{\text{arg min}}\{\sum _{i=s+1}^{\nu }(Y_i-{\mathit{X}}_i^{\mathsf{T}}{\alpha }_1{)}^2+\sum _{t=\nu +1}^e(Y_i-{\mathit{X}}_i^{\mathsf{T}}{\alpha }_2{)}^2\phantom{\}} \\ +\phantom{\{}{\lambda }_G\sum _{j=1}^p\sqrt{(\nu -s)({\alpha }_{1,j}{)}^2+(e-\nu )({\alpha }_{2,j}{)}^2}\}, \end{gathered}$$ (21)

  where s′ and e′ serve as boundary trimming parameters with s + 1 ≤ s′ + 1 < e′ ≤ e, and λ_G is the tuning parameter for the group penalty. For a better comparison in the context of high-dimensional GLMs, we modify this ℓ₂-loss based method in Wang et al. (2021a) by replacing the ℓ₂-loss in Equation (21) with the loss _ρβ(X_i, Y_i) defined in Section 2.

For our proposed approaches, the regression coefficients are computed by the R package glmnet (https://glmnet.stanford.edu). All numerical results are based on 100 replications, except for the test by Lee2011 which is based on 500 replications.

4.1. Tuning parameter selection

It is essential to properly choose the values of tuning parameters for accurate estimation results. We develop a cross-validation approach for GLMs to choose the parameters λ and γ, which encourage regression coefficient and segmentation sparsity, respectively. To be specific, let samples with odd indices be the training set (X₁, X₃, … X_n−3, X_n−1) and the others be the validation set (X₂, X₄, …, X_n−2, X_n). For each of two tuning parameters λ, γ, we conduct our procedure on the training set and obtain the estimated change point $\widehat{\tau }(\widehat{k})$ and underlying regression coefficients $\widehat{\beta }(\widehat{\tau },j),j=1,\dots ,\widehat{k}$. Let ${\widehat{f}}_i={\mathit{X}}_i^{\mathsf{T}}\widehat{\beta }(\widehat{\tau },j)$, for i/n ∈ I_j(τ) and i = 1, …,n. We can calculate the validation loss as:

$$CV(\lambda ,\gamma )=\frac{2}{n}\sum _{i:i\text{mod}2\equiv 1}\rho ({\widehat{f}}_i,Y_i),$$

where ρ is the loss function of the GLMs and depends on the link function. For the specific regression models such as the linear model and logistic regression model, the corresponding validation losses are defined as:

$$\begin{gathered} C{V}_{LM}(\lambda ,\gamma )=\frac{2}{n}\sum _{i:i\text{mod}2\equiv 1}({\widehat{f}}_i-Y_i{)}^2,\text{and} \\ C{V}_{Logic}(\lambda ,\gamma )=\frac{2}{n}\sum _{i:i\text{mod}2\equiv 1}\log (1+e^{{\widehat{f}}_i})-y{\widehat{f}}_i. \end{gathered}$$

Then we choose (λ, γ) corresponding to the lowest validation loss. Note that it is time-consuming to use the cross-validation procedure to choose the tuning parameters for our various model settings. Based on our extensive numerical simulations, we find that our methods are stable over a certain range of tuning parameters. Hence, we use an empirical choice of the parameters λ and γ to save computational cost. In particular, we set $\lambda =c(\sqrt{\log (2p)∕n}+\log (2p)∕n)$, with c ∈ (0.15, 0.25). As for γ, we set γ = δλ. Recall δ is the minimum interval length and δ_n is the minimum interval size, which controls the maximum number of change points. Note that δ is of key importance for our theoretical guarantee discussed in Section 3.2, and needs to be carefully chosen in simulation studies.

In order to ensure the effective fitting of the regression model, we need to guarantee a sufficient sample size for each interval. According to our numerical studies, setting δ ∈ (0.1, 0.25) works well. To investigate how sensitive our proposed methods are to the choice of these tuning parameters, we consider various values of λ and γ by setting the sample size n ∈ {200, 300, 1000} and data dimension p ∈ {200, 300, 400}. Note that our proposed methods can automatically account for the underlying data generation mechanism and does not need to know the number of change points. To justify this, in what follows, we present our numerical results under three different cases: (1) $\tilde{k}=0$, (2) $\tilde{k}=1$, and (3) $\tilde{k}=3$, which correspond to data with no change point, one change point, and multiple change points, respectively.

4.2. No change point models

We consider the alternative scenario where no change point occurs. In this case, the underlying regression coefficients satisfy ${\beta }^{(i)}={\beta }^0≔({\beta }_1^0,\dots ,{\beta }_p^0{)}^{\mathsf{T}}$ for i = 1, …, n. We set the sample size n = 200 and the data dimension p ∈ {200, 300,400}. For $s\in {\mathcal{S}}^0$, we generate ${\beta }_s^0\stackrel{\text{iid}}{\sim }\mathrm{U}(0,2)$, where ${\mathcal{S}}^0$ denotes the set of non-zero elements of β⁰ with $\#{\mathcal{S}}^0=[\log (p)]$.

We implement the corresponding algorithms independently on a 2.50GHz CPU (Linux) with 6 cores and 4GB of RAM. As shown in Figure 1 (left), the computational cost of BSA grows moderately (12s to 737s) as the data dimension increases from 400 to 2000, while the computational cost of DPA grows exponentially (80s to 32000s). As for the accuracy, the error rates of VPWBS, DPA, and BSA are zero in almost all cases, which suggests these three approaches have almost the same accuracy when no change point occurs. SGL tends to overestimate the number of change points for the homogeneous observations. Note that Lee2011 has relatively large errors, which suggests that it may be unreliable in high-dimensional settings. Thus, we do not include it in our comparisons for the single change point models.

Figure 1:

Efficiency of change point estimation with p = 2n. The left panel shows computational costs of BSA and DPA per replication under the model with no change point. The right panel shows computational costs of BSA and DPA per replication under the model of three change points.

4.3. Single change point models

Next we consider the alternative scenario where (β⁽ⁱ⁾)_1≤i≤n have a common change point location ${\tilde{\tau }}_1$, where ${\tilde{\tau }}_1\in \{0.5,07\}$. We set the sample size n = 300 and the data dimension p ∈ {200, 300, 400}. Furthermore, we assume the regression coefficients have support set $\{{\mathcal{S}}^1,{\mathcal{S}}^2\}$. For $s\in {\mathcal{S}}^1$, we set ${\beta }_s(1)\stackrel{\text{iid}}{\sim }\mathrm{U}(0,2)$. Then, for $s\in {\mathcal{S}}^2$, we set β_s(2) = β_s(1) + δ_s with ${\delta }_s(1)\stackrel{\text{i.i.d.}}{\sim }\mathrm{U}(0,10\sqrt{\log (p)∕n})$. For each replication, the support sets ${\mathcal{S}}^1$, ${\mathcal{S}}^2$ of regression coefficients are randomly selected from the set {1, 2,…, 0.3p} with $\#{\mathcal{S}}^1=\#{\mathcal{S}}^2=[\log (p)]$.

Figure 2 (left) indicates that the computational cost of the BSA grows gradually with the data dimension increasing from 400 to 2000 as compared to the exponential growth of the DPA. Meanwhile, the computational cost of FSA in Algorithm 3 increases slowly as compared to the “exponential” growth of the BSA, as shown in Figure 2 (right). This suggests that FSA is more preferable for single change point models. Furthermore, to investigate the computational efficiency for high-dimensional cases, we present the computational cost of our three proposed approaches in Figure 3. It implies that our proposed approaches have stable and good performance as data dimension p grows.

Figure 2:

Efficiency of change point estimation under the single change point model with n ∈ {200, 400, 600, 800, 1000} and p = 2n. The left panel shows the computational costs of BSA and DPA per replication. The right panel shows computational costs per replication of BSA and FSA. The change point is fixed at τ₁ = 0.5.

Figure 3:

Efficiency of change point estimation under the single change point model with p ∈ {600, 1200, 1800, 2400, 3000} and n = 300. The left panel shows the computational costs of BSA and DPA per replication. The right panel shows computational costs per replication of BSA and FSA. The change point is fixed at τ₁ = 0.5.

As for the accuracy, we record the percentage of replications (Rate (%)) in which DPA and BSA correctly identified a single change point. Note that in Tables 2 and 3, MSEs for the number and location are expressed as factors of 10⁻² and 10⁻⁴, respectively. We can see that DPA, BSA, and VPWBS can identify a single change point with high rates of success. Furthermore, DPA generally has the best performance for estimating the single change point location. VPWBS performs better than BSA especially when the change occurs near the edge. Both DPA and BSA perform slightly better than FSA. Note that all the proposed algorithms perform better the closer the change point location is to the middle of the data observations, e.g., ${\tilde{\tau }}_1=0.5$.

Table 2:

Single change point detection for Case 1 with Σ = I_p×p under various dimensions and change point locations, based on 100 replications.

			dimension p
	${\tilde{\tau }}_1$	method	200	300	400
Number	0.5	SGL	2.49 ∣ 19	2.36 ∣ 17	2.48 ∣ 23
Mean∣Rate(%)		VPWBS	1.02 ∣ 96	1.01 ∣ 98	1.02 ∣ 97
		DPA	1.00 ∣ 100	1.01 ∣ 99	1.00 ∣ 100
		BSA	1.00 ∣ 100	1.00 ∣ 100	1.00 ∣ 100
	0.7	SGL	2.48 ∣23	2.48 ∣23	2.64 ∣14
		VPWBS	1.01 ∣ 97	1.04 ∣ 96	0.99 ∣ 95
		DPA	1.04 ∣ 96	1.02 ∣ 98	1.03 ∣ 97
		BSA	1.00 ∣ 100	1.00 ∣ 100	1.00 ∣ 100
Location	0.5	SGL	-	-	-
Mean∣MSE(10−4)		VPWBS	0.497 ∣ 2.244	0.500 ∣ 3.590	0.496 ∣ 6.322
		DPA	0.499 ∣ 1.836	0.503 ∣ 2.256	0.499 ∣ 3.254
		BSA	0.498 ∣ 2.446	0.501 ∣ 3.779	0.499 ∣ 6.445
		FSA	0.495 ∣ 19.21	0.496 ∣ 9.326	0.493 ∣ 11.18
	0.7	SGL	-	-	-
		VPWBS	0.699 ∣ 2.442	0.701 ∣ 6.963	0.693 ∣ 7.710
		DPA	0.694 ∣ 3.933	0.693 ∣ 4.630	0.691 ∣ 5.086
		BSA	0.691 ∣ 6.884	0.688 ∣ 14.29	0.686 ∣ 11.28
		FSA	0.684 ∣ 25.09	0.678 ∣ 43.94	0.666 ∣ 37.07

Table 3:

Single change point detection for Case 2 with Σ = Σ* under various dimensions and change point locations. The numerical results are based on 100 replications.

			dimension p
	${\tilde{\tau }}_1$	method	200	300	400
Number	0.5	SGL	1.14 ∣ 32	1.30 ∣ 41	1.52 ∣ 40
Mean∣Rate(%)		VPWBS	1.02 ∣ 98	1.04 ∣ 96	1.03 ∣ 98
		DPA	1.01 ∣ 99	1.00 ∣ 100	1.00 ∣ 100
		BSA	1.00 ∣ 100	1.00 ∣ 100	1.00 ∣ 100
	0.7	SGL	0.72 ∣ 20	1.16 ∣ 26	1.14 ∣36
		VPWBS	0.97 ∣ 97	1.01 ∣ 99	1.05 ∣ 96
		DPA	1.00 ∣ 100	1.02 ∣98	1.01 ∣ 99
		BSA	0.99 ∣ 99	1.00 ∣98	1.01 ∣ 99
Location	0.5	SGL	-	-	-
Mean∣MSE		VPWBS	0.504 ∣ 5.056	0.495 ∣ 3.146	0.498 ∣ 3.572
		DPA	0.498 ∣ 2.145	0.502 ∣ 1.423	0.499 ∣ 3.670
		BSA	0.493 ∣ 5.897	0.498 ∣ 3.347	0.498 ∣ 4.326
		FSA	0.495 ∣ 12.21	0.489 ∣ 20.10	0.492 ∣ 9.374
	0.7	SGL	-	-	-
		VPWBS	0.694 ∣ 5.959	0.694 ∣ 7.297	0.701 ∣ 6.950
		DPA	0.690 ∣ 5.748	0.690 ∣ 6.411	0.691 ∣ 12.75
		BSA	0.689 ∣ 9.419	0.694 ∣ 3.973	0.688 ∣ 11.44
		FSA	0.689 ∣ 22.41	0.683 ∣ 24.20	0.676 ∣ 27.66

4.4. Multiple change point models

Finally we consider the alternative scenario where (β⁽ⁱ⁾)_1≤i≤n has multiple change point locations ${\tilde{\tau }}_2$ with ${\tilde{\tau }}_2=(0,0.25,0.5,0.75,1{)}^{\mathsf{T}}$. We set the sample size n = 1000 and the data dimension p ∈ {200, 300,400}. For $s_1\in {\mathcal{S}}^1$, we set ${\beta }_{s_1}(1)\stackrel{\text{iid}}{\sim }\mathrm{U}(0,2)$. Then, for $s_j\in {\mathcal{S}}^j(j=2,3,4)$, we set β_sj(j) = β_sj(j − 1) + (j − 1)δ_sj with ${\delta }_{s_j}\stackrel{\text{i.i.d.}}{\sim }\mathrm{U}(0,10\sqrt{\log (p)∕n})$. For each replication, the support set of regression coefficients $\mathcal{S}$ is randomly selected from the set {1, 2, …, 0.3p} with $\#{\mathcal{S}}^j=[\log (p)]$, j = 1, 2, 3, 4.

We first analyze the efficiency. The results are similar to the other cases. Figure 4 shows that the computational cost of BSA grows gradually with the number of change points. In contrast, the cost of the DPA grows substantially. This suggests that the efficiency of BSA is not sensitive to the number of change points. To compare accuracy, similarly to the previous analysis, we record the percentage of replications in which DPA and BSA can correctly identify the three change points. As shown in Tables 4 and 5, DPA generally has the best performance. VPWBS performs slightly better than BSA. However, there is not much difference in performance among DPA, BSA, and VPWBS in terms of identifying the number and locations of multiple change points.

Figure 4:

Efficiency of change point estimation under the model of multiple change points with n ∈ {200, 400, 600, 800, 1000} and p = 2n. The left panel shows the computational costs of BSA per replication in the settings with different numbers of change points. The right panel shows computational cost of DPA per replication in the settings with different numbers of change points.

Table 4:

Multiple change point detection for Case 1 with Σ = I_p×p under various dimensions. The numerical results are based on 100 replications.

τ2 =	(0.25, 0.5, 0.75)⊤	p	Accuracy for the following methods
			SGL	VPWBS	DPA	BSA
		200	3.19 ∣ 55	3.02 ∣ 98	3.00 ∣ 100	2.95 ∣ 95
number	Mean ∣ Rate(%)	300	3.47 ∣ 55	3.03 ∣ 95	3.00 ∣ 100	2.98 ∣ 98
		400	3.50 ∣ 44	3.04 ∣ 93	3.00 ∣ 100	2.94 ∣ 94
		200	-	0.248 ∣ 1.108	0.249 ∣ 0.665	0.249 ∣ 1.821
location 1	Mean ∣ MSE (10−5)	300	-	0.250 ∣ 2.100	0.250 ∣ 0.865	0.247 ∣ 4.168
		400	-	0.249 ∣ 3.913	0.251 ∣ 1.847	0.256 ∣ 5.736
		200	-	0.499 ∣ 0.894	0.500 ∣ 0.353	0.499 ∣ 2.099
location 2	Mean ∣ MSE (10−5)	300	-	0.499 ∣ 1.204	0.501 ∣ 0.508	0.499 ∣ 6.148
		400	-	0.498 ∣ 2.490	0.500 ∣ 0.981	0.500 ∣ 6.736
		200	-	0.751 ∣ 0.701	0.750 ∣ 1.002	0.749 ∣ 1.689
location 3	Mean ∣ MSE (10−5)	300	-	0.750 ∣ 1.370	0.751 ∣ 2.481	0.745 ∣ 7.968
		400	-	0.748 ∣ 2.268	0.749 ∣ 0.688	0.748 ∣ 2.258

Table 5:

Multiple change point detection for Case 2 with Σ = Σ* under various dimensions. The numerical results are based on 100 replications.

τ 2	= (0.25, 0.5, 0.75)⊤	p	Accuracy for the following methods
			SGL	VPWBS	DPA	BSA
		200	2.65 ∣ 30	3.00 ∣ 100	3.00 ∣ 100	2.99 ∣ 99
number	Mean ∣ Rate(%)	300	2.08 ∣ 17	2.97 ∣ 99	3.00 ∣ 100	2.98 ∣ 98
		400	2.47 ∣ 25	3.00 ∣ 100	3.00 ∣ 100	2.98 ∣ 98
		200	-	0.247 ∣ 2.435	0.250 ∣ 1.961	0.251 ∣ 3.457
location 1	Mean ∣ MSE (10−5)	300	-	0.249 ∣ 3.164	0.251 ∣ 2.844	0.250 ∣ 6.585
		400	-	0.251 ∣ 1.687	0.251 ∣ 2.497	0.249 ∣ 4.994
		200	-	0.487 ∣ 2.040	0.500 ∣ 1.069	0.499 ∣ 2.667
location 2	Mean ∣ MSE (10−5)	300	-	0.502 ∣ 1.403	0.501 ∣ 0.894	0.499 ∣ 4.363
		400	-	0.499 ∣ 2.212	0.501 ∣ 3.613	0.500 ∣ 4.192
		200	-	0.748 ∣ 1.242	0.750 ∣ 0.711	0.748 ∣ 1.674
location 3	Mean ∣ MSE (10−5)	300	-	0.752 ∣ 1.672	0.750 ∣ 0.486	0.749 ∣ 1.398
		400	-	0.747 ∣ 4.511	0.750 ∣ 0.580	0.747 ∣ 4.881

It is worth mentioning that our proposed methods have good performance in all three models with various data dimensions, sample sizes, and numbers of change points, which suggests that the methods are robust to the suggested choice of tuning parameters λ and γ.

5. REAL DATA ANALYSIS

To illustrate the usefulness of our proposed methods, we consider the Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset (www.loni.ucla.edu/ADNI). The ADNI dataset contains disease state information on different subjects including normal controls (NC), mild cognitive impairment (MCI), and Alzheimer’s disease (AD) as well as some biological markers including features derived from magnetic resonance imaging (MRI) and positron emission tomography (PET). It is very useful for clinical diagnosis and prevention to study how to measure the progression of AD using the images. For example, in the AD-related literature (see, for example, Reiss & Ogden, 2010), it is popular to use structural MRI or PET to predict the current disease status of the patient (binary response variable), which can be regarded as a classification problem. Usually, they treat data as homogeneous and ignore the effect of other covariate variables, such as age, gender and so on. Hence, an interesting question is whether the generalized linear structure between the disease status and biomarkers (MRI or PET) changes due to other covariates. If it changes, how can one estimate (a) the number of change points, (b) the locations of change points, and (c) the regression coefficients (selected variables) in each segmentation? In our study, we address these issues by detecting change points in the generalized linear structure between the disease status and the MRI features together with some covariates. In this application, we choose age as the covariate, which is of particular interest in AD studies.

We use the MRI data of 405 subjects including 220 normal controls and 185 AD patients from the ADNI data. For each subject, we obtain the corresponding status (AD/NC), age, and 93 MRI features after using the data processing method proposed in Zhang & Shen (2012). In our model, the predictive variables X = (X₁, …, X₉₃) are the 93 MRI features, which are scaled to have mean 0 and variance 1, and the response variable is the binary status obtained by setting Y_i = 1 {subject i is an AD patient}, and 0 otherwise. For this dataset $\{{\mathit{X}}_i,Y_i{\}}_{i=1}^n$, consider the following logistic regression model: $\log \frac{(\mathbb{P}(Y_i=1))}{(\mathbb{P}(Y_i=0))}={\mathit{X}}_i^{\mathsf{T}}{\beta }^{(i)}$. Our goal is to detect potential change points of regression coefficients in $\{{\beta }^{(i)}{\}}_{i=1}^n$. Taking the effect of different samples into consideration, in our analysis, we divide the data into two parts: training and testing datasets. More specifically, we randomly select 40 subjects from the whole set of 405 subjects according to the empirical distribution of age in Figure 5 (left) as the testing sample and use the remaining 365 subjects as the training data. Then we sort those 365 observations in the training data by age and use BSA to estimate the number and location of change points. We choose the tuning parameters λ and γ as suggested in Section 4. The above process is repeated for 100 times. Since BSA is more computationally efficient than DPA, we use only BSA to analyze the data.

Figure 5:

The left panel shows distribution of age among 405 subjects. The right panel shows estimated numbers of change points based on 100 replications.

Figure 5 (right) demonstrates the estimated numbers of change points for 100 replications. To be specific, among the 100 replications, 80% of the estimated numbers of change points are 1 (k = 3), which suggests that there is a change point in the context of logistic regression between the disease status and the MRI features due to the change of age. Moreover, Table 6 summarizes the change point estimation results computed by BSA. For the change point estimation, 90% of change points were estimated by BSA to be located at 80 years old and the mean is 79.95 years old. This implies that there are significant differences between the regression models of the disease status and the MRI features under 80 and over 80 years old.

Table 6:

Change point estimation and prediction for ADNI data. The change point estimator is obtained by using BSA based on 100 replications.

training/testing sample	methods	location of change point	Averaged MSE
365/40	BSA	79.75	0.380
	Lasso-based	-	0.408

In terms of prediction, Table 6 provides the prediction results computed by BSA and the Lasso-based method, where for each replication, we use the training sample to select models and use the testing sample to predict. Note that the Lasso-based method treats the data as homogeneous while we consider a heterogeneous model. For the prediction result, we calculate the predictive MSE on the testing set for these two methods. Our proposed method obtains better prediction performance, which is demonstrated by a 7% lower averaged predictive MSE than that of the Lasso-based method. This suggests that treating the data as heterogeneous and using our method to select models can predict better.

For variable selection, Figure 6 shows selected features before and after the estimated change point of 80 years old. We see that models both under 80 and over 80 both select the 69th and 83rd features, which correspond to the hippocampal and amygdala regions, respectively. These regions are known to be related to AD by many previous studies (Zhang & Shen, 2012). Moreover, there are a few different features selected separately by these two models under 80 and over 80. We believe these features deserve more scientific attention and more researches are needed to study their associations with AD together with age.

Figure 6:

Frequency of features selected before the change point (left) and after the change point (right) for the ADNI dataset based on 100 replications. Features selected by models both before and after the change point are shown in red.

6. SUMMARY

In this paper, we provide a three-step procedure for change point detection in the context of high-dimensional GLMs, where the dimension p can be much larger than the sample size n. It is worth mentioning that our proposed method can automatically account for the underlying data generation mechanism ($\tilde{k}=0$ or $\tilde{k}>0$) without specifying any prior knowledge about the number of change points $\tilde{k}$. Moreover, based on dynamic programming and binary segmentation techniques, two algorithms, DPA and BSA, are proposed to detect multiple change points. To further improve the computational efficiency, we present a much more efficient algorithm designed for the case of a single change point. Furthermore, we investigate the theoretical properties of our proposed change point estimators computed by the three algorithms. Estimation consistency for the number and locations of change points is established. Finally, we demonstrate the efficiency and accuracy of our proposed methods by extensive numerical results under various model settings. A real data application to the ADNI dataset also demonstrates the usefulness of our proposed methods.

Table 1:

Change point detection for Cases 1 and 2 under the model with no change point. The numerical results are based on averages of 100 replications.

Case	Measurement	p	Accuracy for the following methods
			SGL	VPWBS	Lee2011	DPA	BSA
		200	0.65	0.01	0.73	0.00	0.00
Σ = I	error rate	300	0.85	0.01	0.71	0.00	0.00
		400	0.86	0.01	0.74	0.00	0.00
		200	0.38	0.00	0.71	0.00	0.00
Σ = Σ*	error rate	300	0.44	0.01	0.71	0.00	0.00
		400	0.47	0.00	0.72	0.00	0.00

APPENDIX

Notation

We introduce some notation. The random part, empirical process, is defined as $\left\{v_n(\beta )≔(P_n-P){\rho }_{f_{\beta }}:\beta \in {\mathbb{R}}^p\right\}$. We recall the Lasso estimator is, for j = 1, …, k, $\widehat{\beta }(j)={\text{arg min}}_{\beta }\left\{P_n{\rho }_{f_{\beta }}+\lambda r_j(\tau )\Vert \beta {\Vert }_1\right\}$. We write $\widehat{f}=f_{\widehat{\beta }}$ and $\mathcal{E}(\beta )≔P({\rho }_{f_{\beta }}-{\rho }_{f_{{\beta }^0}})$ for convenience. Recall that for any (u, v), the oracle β* is defined as ${\beta }_{(u,v)}^{\ast }≔{\text{arg min}}_{\beta }\left\{\mathcal{E}(f_{\beta })\right\}$, which is the best approximation of β⁰ under the compatibility condition, β* = β⁰, if there is no change point between u and v. The estimation error is denoted as ϵ* := v_n(β*) = (P_n − P)_ρfβ*. We define, for some positive constant L > 0, Z_L := sup_{∥β−β*∥1≤L} ∣v_n(β) − v_n (β*)∣ . We set L* := ϵ*/λ₀, and require a relatively small L*, so this indicates that ϵ* ≤ λ⁰. Based on this, for any u, v ∈ V_n := {i/n : i =1, …, n} with u < v, we define two important sets as follows:

$${\mathcal{T}}_0≔\{{\mathbf{Z}}_{L^{\ast }}\le {ϵ}^{\ast }\le {\lambda }_0\},$$ (1)

$${\mathcal{T}}_1≔\left\{\underset{(u,v)}{\max }{\Vert {\widehat{\Sigma }}_{(u,v)}-(v-u)\Sigma \Vert }_{\infty }\le {\lambda }_1\right\}.$$ (2)

We introduce the following assumptions.

Assumption F. We require some technical conditions as follows:

(F1) $\lambda \sqrt{\delta }\ge 8{\lambda }_0$, where ${\lambda }_0=O(\sqrt{\log (p)∕n})$.

(F2) $\gamma >3d_{\ast }{s}_{\ast }{\lambda }^2$, where d_* = O(1) with detailed definition introduced in the Appendix .

(F3) $\frac{\delta m_{\ast }^2{\varphi }_{\ast }^2{s}_{\ast }}{8}>\gamma +22{ϵ}^{\ast }$, where ${ϵ}^{\ast }=O(s_{\ast }\log (p)∕n)$.

(F4) $\frac{\lambda \delta m_{\ast }^2{\varphi }_{\ast }^2{s}_{\ast }}{8}>22{ϵ}^{\ast }-\lambda \delta M_{\ast }$.

Assumption G. We require some conditions for achieving limit properties for the de-biased Lasso estimator:

(G1) The derivatives $\stackrel{.}{\rho }(Y,a)≔\frac{d}{da}\rho (Y,a),\ddot{\rho }(Y,a)≔\frac{d^2}{d{a}^2}\rho (Y,a)$ exist for all y, a, and for some δ-neighborhood (δ > 0), $\ddot{\rho }(Y,a)$ is Lipschitz:

$$\underset{a_0\in \{{\mathit{X}}_i^{\mathsf{T}}{\beta }^0\}\mid a-a_0\mid \vee \mid \widehat{a}-a_0\mid \le \delta }{\max }\underset{Y\in \mathcal{Y}}{\text{sup}}\frac{\mid \ddot{\rho }(Y,a)-\ddot{\rho }(Y,\widehat{a})\mid }{\mid a-\widehat{a}\mid }\le 1$$

Moreover,

$$\underset{a_0\in \{{\mathit{X}}_i^{\mathsf{T}}{\beta }^0\}}{\max }\underset{Y\in \mathcal{Y}}{\text{sup}}\mid \stackrel{.}{\rho }(y,a_0)\mid =O(1),\text{and}\underset{a_0\in \{{\mathit{X}}_i^{\mathsf{T}}{\beta }^0\}\mid a-a_0\mid \le \delta }{\max }\text{sup sup}\mid \ddot{\rho }(Y,a)\mid =O(1).$$

(G2) It holds that ${\Vert P_n{\ddot{\rho }}_{\widehat{\beta }}{\widehat{\Theta }}_j^T-e_j\Vert }_{\infty }=O_P({\lambda }_{\ast })$.

(G3) It holds that ${\Vert \mathbf{X}{\widehat{\Theta }}_j^T\Vert }_{\infty }=O_P(K)$ and ${\Vert {\widehat{\Theta }}_j\Vert }_1=O_P(\sqrt{s_{\ast }})$

(G4) It holds that ${\Vert (P_n-P){\stackrel{.}{\rho }}_{{\beta }^0}{\stackrel{.}{\rho }}_{{\beta }^0}^T\Vert }_{\infty }=O_P(K^2\lambda )$ and moreover ${\max }_{j}1∕{\left(\widehat{\Theta }P{\stackrel{.}{\rho }}_{{\beta }^0}{\stackrel{.}{\rho }}_{{\beta }^0}^T{\widehat{\Theta }}^T\right)}_{j,j}=O(1)$.

(G5) For every j, the random variable $\frac{\sqrt{n}{\left(\widehat{\Theta }{P}_n{\stackrel{.}{\rho }}_{{\beta }^0}\right)}_j}{\sqrt{{\left(\widehat{\Theta }P{\stackrel{.}{\rho }}_{{\beta }^0}{\stackrel{.}{\rho }}_{{\beta }^0}^T{\widehat{\Theta }}^T\right)}_{j,j}}}$ converges weakly to a N(0,1)-distribution.

Useful Lemmas

We introduce some useful lemmas that are essential for our main results. More specifically, Lemma 1 presents the upper bound of the difference of the subinterval penalized empirical average of the loss function based on the Lasso and oracle estimators. Corollary 1 shows the equation in Lemma 1 holds with high probability. Lemma 2 provides the results for the subinterval based on the compatibility condition. The margin condition based on the oracle estimator is updated in Lemma 3. Lemma 4 presents the lower bound of the difference of the loss function based on the oracle estimator of adjacent subintervals. At last, Lemma 5 provides the upper bound of the difference of the subinterval penalized empirical average of the loss function based on the oracle estimator and the truth. Next, we will introduce these useful lemmas in detail.

Lemma 1 (Oracle inequality for the Lasso) Suppose Assumptions A-F hold for all ${\Vert \widehat{\beta }-{\beta }^{\ast }\Vert }_1\le L^{\ast }$, as well as $\Vert f_{\beta }-f_{\beta }^{\ast }{\Vert }_{\infty }\le L^{\ast }{K}_X$. Suppose that λ satisfies the inequality $\lambda \sqrt{\delta }\ge 8{\lambda }_0$. Then on the set ${\mathcal{T}}_0\cup {\mathcal{T}}_1$ given in (1)-(2), we have

$$\mid P_n\rho \left((u,v),{\widehat{\beta }}_{(u,v)}\right)-P_n\rho \left((u,v),{\beta }_{(u,v)}^{\ast }\right)\mid +\lambda \sqrt{(u-v)}\Vert \widehat{\beta }-{\beta }^{\ast }{\Vert }_1\le 6{ϵ}^{\ast },$$ (3)

where there exists a constant C₃ > 0 such that ϵ* ≤ C₃s* log(p)/n.

Proof. Since the Assumptions hold, on ${\mathcal{T}}_0\cup {\mathcal{T}}_1$, $8{\lambda }_0<\lambda \sqrt{\delta }<\lambda \sqrt{(v-u)}$, we have

$$\mid P\rho \left((u,v),{\widehat{\beta }}_{(u,v)}\right)-P\rho \left((u,v),{\beta }_{(u,v)}^{\ast }\right)\mid +\lambda \sqrt{(u-v)}\Vert \widehat{\beta }-{\beta }^{\ast }{\Vert }_1\le 4{ϵ}^{\ast },$$

by Theorem 6.4 in Bühlmann & van de Geer (2011). By the definition of P_nρ, Pρ, and v(β) introduced in Section 3.1, we can obtain

$$\begin{gathered} \mid P_n\rho \left((u,v),{\widehat{\beta }}_{(u,v)}\right)-P_n\rho \left((u,v),{\beta }_{(u,v)}^{\ast }\right)\mid \\ \le \mid P\rho \left((u,v),{\widehat{\beta }}_{(u,v)}\right)-P\rho \left((u,v),{\beta }_{(u,v)}^{\ast }\right)\mid +\mid v({\beta }^{\ast })-v(\widehat{\beta })\mid . \end{gathered}$$

If the condition $\Vert \widehat{\beta }-{\beta }^{\ast }{\Vert }_1\le L^{\ast }$ holds, on the set ${\mathcal{T}}_0\cup {\mathcal{T}}_1$, we can have

$$\mid P_n\rho \left((u,v),{\widehat{\beta }}_{(u,v)}\right)-P_n\rho \left((u,v),{\beta }_{(u,v)}^{\ast }\right)\mid +\lambda \sqrt{(u-v)}\Vert \widehat{\beta }-{\beta }^{\ast }{\Vert }_1\le 4{ϵ}^{\ast }+2{ϵ}^{\ast }=6{ϵ}^{\ast },$$

which completes the proof.

Corollary 1 Suppose Assumptions A-F hold. Let $a_n≔4\left(\sqrt{\frac{2\log (2p)}{n}}+\frac{\log (2p)}{n}{K}_X\right)$ and

$${\lambda }_0≔{\lambda }_0(t)≔a_n\left(1+t\sqrt{2(1+2a_n{K}_X)}+\frac{2t^2{a}_n{K}_X}{3}\right).$$

Then we have, with probability at least $1-7exp[-n{a}_n^{-2}{t}^2]=1-\text{exp}(-C_1\frac{n^2}{\log (p)})$, that Equation (3) holds. We refer to Theorem 2.1 in van de Geer (2008).

Lemma 2 Suppose Assumption D and $s_{\ast }{\lambda }_1\le \frac{{\varphi }_{\ast }^2}{32}$ hold. Then on the set ${\mathcal{T}}_0\cup {\mathcal{T}}_1$, we have, for all (u, v) ∈ {i/n, i = 1, …, n} and all $\beta \in {\mathbb{R}}^p$ with ∥β_S*∥₁ ≤ 3∥β_S*∥₁,

$$\Vert {\beta }_{S_{\ast }}{\Vert }_1^2\le \frac{({\beta }^{\mathsf{T}}{\widehat{\Sigma }}_{(u,v)}\beta )s_{\ast }}{(v-u){\varphi }_{\ast }^2}.$$

Proof. By Assumption D (the compatibility condition), for any u, v ∈ {i/n, n = 1, …, n}, we have

$$\Vert {\beta }_{S_{\ast }}{\Vert }_1^2\le \frac{\Vert f_{\beta }{\Vert }^2(v-u)s_{\ast }}{(v-u){\varphi }^2}=\frac{\left({\beta }^T(v-u)\Sigma \beta \right)s_{\ast }}{(v-u){\varphi }_{\ast }^2},$$

for all $\beta \in {\mathbb{R}}^p$ that satisfy ∥β_S*∥₁ ≤ 3 ∥β_S*∥₁. Then the matrix (v − u)Σ satisfies the compatibility condition for the set S_* with constant $\sqrt{(v-u){\psi }_{\ast }}$. By Corollary 6.8 in n Bühlmann and van de Geer (2011), if $s_{\ast }{\lambda }_1\le \frac{{\varphi }_{\ast }^2}{32}$, the compatibility condition also holds for the set S_* and the matrix ${\widehat{\Sigma }}_{(u,v)}$, with ${\varphi }_{{\widehat{\Sigma }}_{(u,v)}}^2\ge (v-u){\varphi }_{\ast }^2∕2$. Then we can obtain, for all $\beta \in {\mathbb{R}}^n$ that satisfy ∥β_Sε∥₁ ≤ 3 ∥β_S.∥₁,

$$\Vert {\beta }_{S_{\cdot }}{\Vert }_1^2\le \frac{\left({\beta }^T{\widehat{\Sigma }}_{(u,x)}\beta \right)s_{\ast }}{{\varphi }_{{\widehat{\Sigma }}_{(0,v)}}^2}\le \frac{2\left({\beta }^T{\widehat{\Sigma }}_{(u,t)}\beta \right)s_{\ast }}{(v-u){\varphi }_{\ast }^2}.$$

Lemma 3 By Assumption B, there exists an η ≥ 0 and strictly convex increasing G, such that for all ${\beta }_1,{\beta }_2\in {\mathbb{R}}^p$ with ∥f_β1 − f⁰∥_∞ ≤ η/2,∥f_β − f⁰∥_∞ ≤ η ∥β* − β⁰∥ ≤ η, we have

$$\mid P_n\rho (\beta )-P_n\rho ({\beta }^{\ast })\mid \ge C\Vert \mathit{X}((\beta -{\beta }^{\ast })){\Vert }_2^2-2{ϵ}^{\ast }.$$

Proof. According to Assumption C (margin condition), we can directly have

$$\mathcal{E}(f_{\beta })\ge G(\Vert f_{\beta }-f^0\Vert )\ge C\Vert f_{\beta }-f^0{\Vert }^2.$$

By the definition of $\mathcal{E}(f_{\beta })$, v(β) introduced in Notation and the triangle inequality, we have

$$P_n\rho (\beta )-P_n\rho ({\beta }^{\ast })\ge C\Vert f_{\beta }-f^0{\Vert }^2-2{ϵ}^{\ast }.$$

By the definition of β*, under the compatibility condition, β* is the best approximation of β⁰: β* = β⁰. Then we have

$$P_n\rho (\beta )-P_n\rho ({\beta }^{\ast })\ge C\Vert \mathit{X}((\beta -{\beta }^{\ast })){\Vert }_2^2-2{ϵ}^{\ast }.$$

Lemma 4 Suppose k₀ > 1 and that Assumptions A-F and $s_{\ast }{\lambda }_1\le \frac{{\varphi }_{\ast }^2}{32}$ hold. Then on ${\mathcal{T}}_0\cap {\mathcal{T}}_1$, if $(u,v)\subset ({\tilde{\tau }}_{j-1}-c^{\ast }\sqrt{\delta }\lambda ,{\tilde{\tau }}_{j+1}+c^{\ast }\sqrt{\delta }\lambda )$ and $u<{\tilde{\tau }}_j<v$ for some j = 2, …, $\tilde{k}-1$, we have

$$\begin{gathered} \mid P_n\rho ((u,{\tilde{\tau }}_j),{\beta }_{(u,v)}^{\ast })-P_n\rho ((u,{\tilde{\tau }}_j),{\beta }_{(u,{\tilde{\tau }}_j)}^{\ast })\mid +\mid P_n\rho (({\tilde{\tau }}_j,v),{\beta }_{(u,v)}^{\ast })-P_n\rho (({\tilde{\tau }}_j,v),{\beta }_{({\tilde{\tau }}_j,v)}^{\ast })\mid \\ \ge \frac{\text{min}({\tilde{\tau }}_j-u,v-{\tilde{\tau }}_j)m_{\ast }^2{\varphi }_{\ast }^2{s}_{\ast }}{8}-4{ϵ}^{\ast }. \end{gathered}$$

Proof. By Lemmas 2 and 3, if $(u,v)\subset ({\tilde{\tau }}_{j-1}-c^{\ast }\sqrt{\delta }\lambda ,{\tilde{\tau }}_{j+1}+c^{\ast }\sqrt{\delta }\lambda )$ and $u<{\tilde{\tau }}_j<v$ for some j = 2, …, $\tilde{k}-1$, we have

$$\begin{gathered} \mid P_n\rho ((u,{\tilde{\tau }}_j),{\beta }_{(u,v)}^{\ast })-P_n\rho ((u,{\tilde{\tau }}_j),{\beta }_{(u,{\tilde{\tau }}_j)}^{\ast })\mid +\mid P_n\rho (({\tilde{\tau }}_j,v),{\beta }_{(u,v)}^{\ast })-P_n\rho (({\tilde{\tau }}_j,v),{\beta }_{({\tilde{\tau }}_j,v)}^{\ast })\mid \\ \ge C\Vert \mathit{X}({\beta }_{(u,v)}^{\ast }-{\beta }_{(u,{\tilde{\tau }}_j)}^{\ast }){\Vert }_2^2+C\Vert \mathit{X}({\beta }_{(u,v)}^{\ast }-{\beta }_{({\tilde{\tau }}_j,v)}^{\ast }){\Vert }_2^2-4{ϵ}^{\ast } \\ \ge \frac{({\tilde{\tau }}_j-u){\Vert {\beta }_{(u,v]}^{\ast }-{\beta }_{(u,{\tilde{\tau }}_j)}^{\ast }\Vert }_1^2{\varphi }_{\ast }^2}{2s_{\ast }}+\frac{(v-{\tilde{\tau }}_j){\Vert {\beta }_{(u,v)}^{\ast }-{\beta }_{({\tilde{\tau }}_j,v]}^{\ast }\Vert }_1^2{\varphi }_{\ast }^2}{2s_{\ast }}-4{ϵ}^{\ast }. \end{gathered}$$

Now observe that

$$(v-u){\beta }_{(u,v]}^{\ast }=({\tilde{\tau }}_j-u){\beta }_{(u,{\tilde{\tau }}_j)}^2+(v-{\tilde{\tau }}_j){\beta }_{({\tilde{\tau }}_j,i)}^{\ast }.$$

Then

$${\beta }_{(u,v)}^{\ast }-{\beta }_{(u,{\tilde{\tau }}_j)}^{\ast }=\left(\frac{v-{\tilde{\tau }}_j}{v-u}\right)\left({\beta }_{({\tilde{\tau }}_j,v]}^{\ast }-{\beta }_{(u,{\tilde{\tau }}_j)}^{\ast }\right),$$

by Assumption E. If $(u,v)\subset ({\tilde{\tau }}_{j-1}-c^{\ast }\sqrt{\delta }\lambda ,{\tilde{\tau }}_{j+1}+c^{\ast }\sqrt{\delta }\lambda )$ and $u<{\tilde{\tau }}_j<v$ for some j = 2, …, $\tilde{k}-1$, we have

$${\Vert {\beta }_{(u,v]}^{\ast }-{\beta }_{(u,{\tilde{\tau }}_j]}^{\ast }\Vert }_1\ge \frac{(v-{\tilde{\tau }}_j)m_{\ast }{s}_{\ast }}{(v-u)},{\Vert {\beta }_{(u,x]}^{\ast }-{\beta }_{({\tilde{\tau }}_j,x)}^{\ast }\Vert }_1\ge \frac{({\tilde{\tau }}_j-u)m_{\ast }{s}_{\ast }}{(v-u)}.$$ (4)

Then, by the above Equation (4) and straightforward calculations, we can obtain

$$\begin{gathered} \frac{({\tilde{\tau }}_j-u){\Vert {\beta }_{(u,v]}^{\ast }-{\beta }_{(u,{\tilde{\tau }}_j)}^{\ast }\Vert }_1^2{\varphi }_{\ast }^2}{2s_{\ast }}+\frac{(v-u){\Vert {\beta }_{(u,v]}^{\ast }-{\beta }_{({\tilde{\tau }}_j,v]}^{\ast }\Vert }_1^2{\varphi }_{\ast }^2}{2s_{\ast }} \\ \ge \frac{({\tilde{\tau }}_j-u{)}^2+(v-{\tilde{\tau }}_j{)}^2}{(v-u{)}^2}\frac{m_{\ast }^2{\varphi }_{\ast }^2{s}_{\ast }}{2}-4{ϵ}^{\ast }. \end{gathered}$$

As $(({\tilde{\tau }}_j-u{)}^2+(v-{\tilde{\tau }}_j{)}^2)∕(v-u{)}^2\ge 1∕2$, we can complete the proof. ■

Lemma 5 Suppose Assumptions A-F hold and let $(u,v)\subset \left({\tilde{\tau }}_{j-1}-c^{\ast }\sqrt{\delta }\lambda ,{\tilde{\tau }}_j+c^{\ast }\sqrt{\delta }\lambda \right)\cap (0,1]$ for some j = 1, …, $\tilde{k}+1$, and $s_{\ast }{\lambda }_1\le \frac{{\varphi }_{\ast }^2}{32}$, $c^{\ast }\sqrt{\delta }\lambda <r(\tilde{\tau })$. Then on the set ${\mathcal{T}}_0\cap {\mathcal{T}}_1$ we have

$$\mid P_n\rho \left((u,v),{\widehat{\beta }}_{(u,v)}\right)-P_n\rho \left((u,v),{\beta }^0(j)\right)\mid +\lambda \sqrt{(u-v)}\Vert {\widehat{\beta }}_{(u,v)}-{\beta }^0(j){\Vert }_1\le d_{\ast }{s}_{\ast }{\lambda }^2,$$

where $b=1\{u<{\tilde{\tau }}_{j-1}\}+1\{{\tilde{\tau }}_j<v\}$, $d_{\ast }=((b^2{K}_X^2{c}^{\ast }{M}_{\ast }+b)c^{\ast }{M}_{\ast }+6C_4)$.

Proof. Firstly, by straightforward calculations, we can obtain

$$\begin{gathered} \mid P_n\rho \left((u,v),{\widehat{\beta }}_{(u,v)}\right)-P_n\rho \left((u,v),{\beta }^0(j)\right)\mid +\lambda (u-v)\Vert \widehat{\beta }-{\beta }^0(j){\Vert }_1 \\ \le \mid P_n\rho \left((u,v),{\widehat{\beta }}_{(u,v)}\right)-P_n\rho \left((u,v),{\beta }_{(u,v)}^{\ast }\right)\mid +\mid P_n\rho \left((u,v),{\beta }_{(u,v)}^{\ast }\right)-P_n\rho \left((u,v),{\beta }^0(j)\right)\mid \\ +\lambda \sqrt{(u-v)}\Vert \widehat{\beta }-{\beta }_{(u,v)}^{\ast }{\Vert }_1+\lambda \sqrt{(u-v)}\Vert {\beta }_{(u,v)}^{\ast }-{\beta }^0(j){\Vert }_1. \end{gathered}$$

According to Lemma 1 and on the set ${\mathcal{T}}_0$, we can obtain

$$\mid P_n\rho \left((u,v),{\widehat{\beta }}_{(u,v)}\right)-P_n\rho \left((u,v),{\beta }_{(u,v)}^{\ast }\right)\mid +\lambda \sqrt{(u-v)}\Vert \widehat{\beta }-{\beta }_{(u,v)}^{\ast }{\Vert }_1\le 6{ϵ}^{\ast }\le 6C_4{s}^{\ast }{\lambda }^2.$$ (5)

Now, we will present the bias between ${\beta }_{u,v}^{\ast }$ and ${\beta }_j^0$, with $(u,v]\subset ({\tilde{\tau }}_{j-1}-c^{\ast }\sqrt{\delta }\lambda ,{\tilde{\tau }}_j+c^{\ast }\sqrt{\delta }\lambda )$. Since $\lambda c^{\ast }\sqrt{\delta }<r(\tau )$, by Assumption E, we can have that

$$\begin{gathered} \Vert {\beta }_{(u,v)}^{\ast }-{\beta }^0(j){\Vert }_{\infty }\le \frac{\max ({\tilde{\tau }}_{j-1}-u,0)}{(v-u)}\Vert {\beta }^0(j)-{\beta }^0(j-1){\Vert }_{\infty } \\ +\frac{\max (v-{\tilde{\tau }}_j,0)}{(v-u)}\Vert {\beta }^0(j+1)-{\beta }^0(j){\Vert }_{\infty } \\ \le \frac{b{M}^{\ast }{c}^{\ast }\lambda }{\sqrt{v-u}}. \end{gathered}$$ (6)

Furthermore, combining Assumption A, Assumption C, Equation (6), and the Cauchy-Schwarz inequality, we have

$$\begin{gathered} \mid P_n\rho \left((u,v),{\beta }_{(u,v)}^{\ast }\right)-P_n\rho \left((u,v),{\beta }^0(j)\right)\mid +\lambda \sqrt{(u-v)}\Vert {\beta }_{(u,v)}^{\ast }-{\beta }^0(j){\Vert }_1 \\ \le (v-u)\Vert \mathit{X}({\beta }_{(u,v)}^{\ast }-{\beta }^0(j)){\Vert }_2^2+\lambda \sqrt{(u-v)}\Vert {\beta }_{(u,v)}^{\ast }-{\beta }^0(j){\Vert }_1 \\ \le (b^2{K}_X^2{M}^{\ast }{s}_{\ast }+b)c^{\ast }{M}^{\ast }{s}_{\ast }{\lambda }^2. \end{gathered}$$ (7)

Combining Equations (5) and (7) can complete the proofs. ■

Proof of Theorem 1. To simplify the notation, we denote the value of the penalized total loss function corresponding to the change point vector by

$$H(\tau )=\sum _{j=1}^{l(\tau )+1}{P}_n\rho \left(I_j(\tau ),\widehat{\beta }(\tau ,j)\right)+\gamma l(\tau ).$$ (8)

First, we will show that if the assumptions hold, we must have $l(\widehat{\tau })=\tilde{k}$ and $\Vert \widehat{\tau }-\tilde{\tau }{\Vert }_1\le c^{\ast }\sqrt{\delta }\lambda$. On the contrary, we assume $\widehat{\tau }$ does not satisfy the above two results. We can distinguish three possible cases:

Case 1: Change point number is overestimated, $l(\widehat{\tau })>\tilde{k}$. There exist some i, 1 ≤ i ≤ $\widehat{k}-1$, such that $\{{\widehat{\tau }}_{i-1},{\widehat{\tau }}_i,{\widehat{\tau }}_{i+1}\}\subset ({\tilde{\tau }}_{j-1}-c^{\ast }\sqrt{\delta }\lambda ,{\tilde{\tau }}_j+c^{\ast }\sqrt{\delta }\lambda )$ for some j, 1 ≤ j ≤ $\tilde{k}$.

Case 2: Change point number is underestimated, $l(\widehat{\tau })<\tilde{k}$. For some $j=1,\dots ,\tilde{k}-1$, we have $\widehat{\tau }\cap ({\tilde{\tau }}_j-c^{\ast }\sqrt{\delta }\lambda ,{\tilde{\tau }}_j-c^{\ast }\sqrt{\delta }\lambda )=\varnothing$ and $\widehat{\tau }\cap ({\tilde{\tau }}_j-c^{\ast }\sqrt{\delta }\lambda ,{\tilde{\tau }}_j-c^{\ast }\sqrt{\delta }\lambda )=\varnothing$.

Case 3: Change point number is correctly estimated, $l(\widehat{\tau })=\tilde{k}$. However, for some $j=1,\dots ,\tilde{k}-1$, we have $\widehat{\tau }\cap ({\tilde{\tau }}_j-c^{\ast }\sqrt{\delta }\lambda ,{\tilde{\tau }}_j-c^{\ast }\sqrt{\delta }\lambda )=\varnothing$ and $\widehat{\tau }\cap ({\tilde{\tau }}_j-c^{\ast }\sqrt{\delta }\lambda ,{\tilde{\tau }}_j-c^{\ast }\sqrt{\delta }\lambda )\ne \varnothing$.

We first consider Case 1, where we have $l(\widehat{\tau })>\tilde{k}$ and there exists some i, such that $\{{\widehat{\tau }}_{i-1},{\widehat{\tau }}_i,{\widehat{\tau }}_{i+1}\}\subset ({\tilde{\tau }}_{j-1},{\tilde{\tau }}_j)$ for some $j,1\le j\le \tilde{k}$. We define

$$\tau =({\widehat{\tau }}_1,\dots ,{\widehat{\tau }}_{i-1},{\widehat{\tau }}_{i+1},\dots ,{\widehat{\tau }}_{l(\widehat{\tau })}).$$

Then we get a new change point vector τ with $l(\tau )=l(\widehat{\tau })-1$. Denote the intervals by $S_1=({\widehat{\tau }}_{i-1},{\widehat{\tau }}_i],S_2=({\widehat{\tau }}_i,{\widehat{\tau }}_{i+1}]$, and $S=({\widehat{\tau }}_{i-1},{\widehat{\tau }}_{i+1}]$, and then we obtain

$$H(\tau )-H(\widehat{\tau })=P_n\rho \left(S,{\widehat{\beta }}_S\right)-P_n\rho \left(S_1,{\widehat{\beta }}_{S_1}\right)-P_n\rho \left(S_2,{\widehat{\beta }}_{S_2}\right)-\gamma .$$ (9)

By the definition of the Lasso estimator $\widehat{\beta }$ and the triangle inequality, we can directly have

$$P_n\rho \left(S,{\widehat{\beta }}_S\right)\le P_n\rho \left(S,{\beta }^0(j)\right)+\lambda \sqrt{\Vert S\Vert }{\Vert {\beta }^0(j)-{\widehat{\beta }}_J\Vert }_1.$$ (10)

Then, combining Equations (9)-(10), we can directly have

$$\begin{gathered} H(\tau )-H(\widehat{\tau })\le P_n\rho \left(S,{\beta }^0(j)\right)-P_n\rho \left(S_1,{\widehat{\beta }}_{S_1}\right)-P_n\rho \left(S_2,{\widehat{\beta }}_{S_2}\right) \\ +\lambda \sqrt{\Vert S\Vert }{\Vert {\beta }^0(j)-{\widehat{\beta }}_J\Vert }_1-\gamma . \end{gathered}$$ (11)

Using some straightforward calculations and the triangle inequality, we have

$$\begin{gathered} P_n\rho (S,{\beta }^0(j))-P_n\rho (S_1,{\widehat{\beta }}_{S_1})-P_n\rho (S_2,{\widehat{\beta }}_{S_2}) \\ =P_n\rho (S_1,{\beta }^0(j))+P_n\rho (S_2,{\beta }_j^0)-P_n\rho (S_1,{\widehat{\beta }}_{S_1})-P_n\rho (S_2,{\widehat{\beta }}_{S_2}) \\ \le \mid P_n\rho (S_1,{\beta }^0(j))-P_n\rho (S_1,{\widehat{\beta }}_{S_1})\mid +\mid P_n\rho (S_2,{\beta }^0(j))-P_n\rho (S_2,{\widehat{\beta }}_{S_2})\mid . \end{gathered}$$ (12)

By Lemma 5 and the above Equation (12), with (u, v) = S₁, S₂, then we have

$$P_n\rho (S,{\beta }^0(j))-P_n\rho (S_1,{\widehat{\beta }}_{S_1})-P_n\rho (S_2,{\widehat{\beta }}_{S_2})\le 2d_{\ast }{s}^{\ast }{\lambda }^2.$$ (13)

Also by Lemma 5, for the second term in Equation (10), we can directly have

$$\lambda \sqrt{\Vert S\Vert }{\Vert {\beta }^0(j)-{\widehat{\beta }}_J\Vert }_1\le d_{\ast }{s}^{\ast }{\lambda }^2,$$ (14)

and therefore, by combining Equation (11) and Equations (13)-(14), we can easily obtain

$$H(\tau )-H(\widehat{\tau })\le 3d_{\ast }{s}^{\ast }{\lambda }^2-\gamma .$$ (15)

According to the Assumption F2, we obtain $M(\tau )<M(\widehat{\tau })$, which is a contradiction.

For Case 2, where we have $l(\widehat{\tau })<\tilde{k}$, we define a new change points vector $\tau =\widehat{\tau }\cup \{{\tilde{\tau }}_j\}$, that is,

$$\tau =({\widehat{\tau }}_1,\dots ,{\widehat{\tau }}_{r_i-1},{\widehat{\tau }}_{r_i},{\widehat{\tau }}_{r_i+1},\dots ,{\widehat{\tau }}_{l(\widehat{\tau })+1}),$$ (16)

where ${\widehat{\tau }}_{r_i}={\tilde{\tau }}_j$. We obtain a new change point vection τ with $l(\tau )=l(\widehat{\tau })+1$. Also we denote the intervals by $S_1=({\widehat{\tau }}_{r_i-1},{\widehat{\tau }}_{r_i}]$, $S_2=({\widehat{\tau }}_{r_i},{\widehat{\tau }}_{r_i+1}]$ and $S=({\widehat{\tau }}_{r_i-1},{\widehat{\tau }}_{r_i+1}]$, then we have

$$H(\widehat{\tau })-H(\tau )=P_n\rho \left(S,{\widehat{\beta }}_S\right)-P_n\rho \left(S_1,{\widehat{\beta }}_{S_1}\right)-P_n\rho \left(S_2,{\widehat{\beta }}_{S_2}\right)-\gamma .$$ (17)

By Lemma 1, for u, v ∈ V_n, we can obtain $P_n\rho \left((u,v),{\widehat{\beta }}_{(u,v)}\right)-P_n\rho \left((u,v),{\beta }_{(u,v)}^{\ast }\right)\le 6{ϵ}^{\ast }$. Thus, by this inequality (with (u, v) = S₁, S₂ and S), the triangle inequality and Lemma 4, we can have

$$\begin{gathered} P_n\rho \left(S,{\widehat{\beta }}_S\right)-P_n\rho \left(S_1,{\widehat{\beta }}_{S_1}\right)-P_n\rho \left(S_2,{\widehat{\beta }}_{S_2}\right) \\ \ge P_n\rho \left(S,{\beta }_S^{\ast }\right)-P_n\rho \left(S_1,{\beta }_{S_1}^{\ast }\right)-P_n\rho \left(S_2,{\beta }_{S_2}^{\ast }\right)-3\ast (6{ϵ}^{\ast }) \\ \ge \frac{\delta m_{\ast }^2{\varphi }_{\ast }^2{s}_{\ast }}{8}-4{ϵ}^{\ast }-3\ast (6{ϵ}^{\ast }). \end{gathered}$$ (18)

Then, by combining the above Equations (17)-(18), we can directly have

$$H(\widehat{\tau })-H(\tau )\ge \frac{\delta m_{\ast }^2{\varphi }_{\ast }^2{s}_{\ast }}{8}-4{ϵ}^{\ast }-3\ast (6{ϵ}^{\ast })-\gamma ,$$ (19)

by Assumption F3, we have $H(\widehat{\tau })>H(\tau )$, which is a contradiction.

For Case 3 with $l(\widehat{\tau }=\tilde{k})$, we must add some points and remove others to obtain a good change point estimator. Then we define ${\tau }^{(1)}=\widehat{\tau }\bigcup \{{\tilde{\tau }}_j\}$ with ${\tau }_{r_i}^{(1)}={\tilde{\tau }}_j$. We denote the intervals by $S_1=({\widehat{\tau }}_{r_i-1},{\widehat{\tau }}_{r_i}]$, $S_2=({\widehat{\tau }}_{r_i},{\widehat{\tau }}_{r_i+1}]$, and $S=({\widehat{\tau }}_{r_i-1},{\widehat{\tau }}_{r_i+1}]$, and then we can obtain

$$H(\widehat{\tau })-H({\tau }^{(1)})=P_n\rho \left(S,{\widehat{\beta }}_S\right)-P_n\rho \left(S_1,{\widehat{\beta }}_{S_1}\right)-P_n\rho \left(S_2,{\widehat{\beta }}_{S_2}\right)-\gamma .$$ (20)

Without loss of generality, we assume ∣ S₁ ∣< δ and define a new partition ${\tau }^{(1)}\setminus \{{\tau }_{r_i-1}^{(1)}\}$. By denoting $K_1=({\tau }_{r_i-1}^{(2)},{\tau }_{r_i-1}^{(1)})$ and I = K₁ ∪ S₁, then we have that

$$H({\tau }^{(1)})-H(\tau )=P_n\rho \left(K_1,{\widehat{\beta }}_{K_1}\right)+P_n\rho \left(S_1,{\widehat{\beta }}_{S_1}\right)-P_n\rho \left(I,{\widehat{\beta }}_I\right)+\gamma .$$ (21)

Thus, by combining Equations (20) and (21), with straightforward calculations, we have

$$\begin{gathered} H(\widehat{\tau })-H(\tau )=H(\widehat{\tau })-H\left({\tau }^{(1)}\right)+H\left({\tau }^{(1)}\right)-H(\tau ) \\ =P_n\rho \left(S,{\widehat{\beta }}_S\right)-P_n\rho \left(S_2,{\widehat{\beta }}_{S_2}\right)+P_n\rho \left(K_1,{\widehat{\beta }}_{K_1}\right)-P_n\rho \left(I,{\widehat{\beta }}_I\right), \end{gathered}$$

by the definition of the Lasso estimator and the triangle inequality, we can obtain

$$\begin{gathered} \sqrt{\mid I\mid }{\Vert {\beta }_{K_1}^{\ast }-{\widehat{\beta }}_I\Vert }_1\le \sqrt{\mid I\Vert }{\beta }_I^{\ast }-{\widehat{\beta }}_I\Vert {}_1+\sqrt{\mid I\mid }\Vert {\beta }_{K_1}^{\ast }-{\beta }_I^{\ast }{\Vert }_1 \\ \le \sqrt{\mid I\mid }{\Vert {\beta }_I^{\ast }-{\widehat{\beta }}_I\Vert }_1+\sqrt{\mid I\mid }\frac{\delta }{\sqrt{\mid I\mid }}M,s_{\ast } \\ \le \sqrt{\mid I\mid }{\Vert {\beta }_I^{\ast }-{\widehat{\beta }}_I\Vert }_1+\delta M_{\ast }{s}_{\ast }. \end{gathered}$$ (22)

By Equation (22), Lemma 1, Lemma 4, and straightforward calculations, we have

$$\begin{gathered} P_n\rho \left(S,{\widehat{\beta }}_S\right)-P_n\rho \left(S_2,{\widehat{\beta }}_{S_2}\right)+P_n\rho \left(K_1,{\widehat{\beta }}_{K_1}\right)-P_n\rho \left(I,{\widehat{\beta }}_I\right) \\ \ge P_n\rho \left(S,{\beta }_S^{\ast }\right)-P_n\rho \left(S_2,{\beta }_{S_2}^{\ast }\right)+P_n\rho \left(K_1,{\beta }_{K_1}^{\ast }\right)-18{ϵ}^{\ast }-P_n\rho \left(I,{\beta }_{K_1}^{\ast }\right)-\lambda \sqrt{\mid I\mid }{\Vert {\beta }_{K_1}^{\ast }-{\widehat{\beta }}_I\Vert }_1 \\ \ge P_n\rho \left(S,{\beta }_S^{\ast }\right)-P_n\rho \left(S_2,{\beta }_{S_2}^{\ast }\right)-P_n\rho \left(S_1,{\beta }_{S_1}^{\ast }\right)-18{ϵ}^{\ast }-\lambda \mid I\mid {\Vert {\beta }_I^{\ast }-{\widehat{\beta }}_I\Vert }_1-\lambda \delta M_{\ast }{s}_{\ast } \\ \ge \frac{\lambda \delta m_{\ast }^2{\varphi }_s^2{s}_{\ast }}{8}-22{ϵ}^{\ast }-\lambda \delta M_{\ast }. \end{gathered}$$

According to the Assumption F4, we can obtain $H(\widehat{\tau })-H(\tau )>0$, which is a contradiction. Above all, the first two results (1) and (2) in Theorem 1 have been proved. Result (3) in Theorem 1 can be directly obtained by combining (2) in Theorem 1 with Lemma 5.

Now we prove the fourth result in Theorem 1. Using that by condition $\mid x_i{\widehat{\Theta }}_j^T\mid ={\mathcal{O}}_{\mathbb{P}}(K)$, we have

$$\widehat{\Theta }{P}_n{\stackrel{.}{\rho }}_{\widehat{\beta }}=\widehat{\Theta }{P}_n{\stackrel{.}{\rho }}_{{\beta }^0}+\widehat{\Theta }{P}_n{\ddot{\rho }}_{\widehat{\beta }}\left(\widehat{\beta }-{\beta }^0\right)+{\text{Rem}}_1,$$

where

$$\begin{gathered} {\text{Rem}}_1={\mathcal{O}}_{\mathbb{P}}(K)\sum _{i=1}^n{\mid x_i\left(\widehat{\beta }-{\beta }^0\right)\mid }^2∕n=\mathcal{O}(K){\Vert \mathbf{X}\left(\widehat{\beta }-{\beta }^0\right)\Vert }_n^2 \\ ={\mathcal{O}}_{\mathbb{P}}(K{s}_0{\lambda }^2)=o_{\mathbb{P}}(1), \end{gathered}$$

it follows that

$$\begin{gathered} \tilde{\beta }(\tau ,j)-{\beta }^0(j)=\widehat{\beta }(\tau ,j)-{\beta }^0(j)-\widehat{\Theta }{P}_n{\stackrel{.}{\rho }}_{\widehat{\beta }} \\ =\widehat{\beta }(\tau ,j)-{\beta }^0(j)-{\widehat{\Theta }}_j{P}_n{\stackrel{.}{\rho }}_{{\beta }^0}-{\widehat{\Theta }}_j{P}_n{\ddot{\rho }}_{\widehat{\beta }}\left(\widehat{\beta }(\tau ,j)-{\beta }^0(j)\right)-{\text{Rem}}_1 \\ =-\widehat{\Theta }{P}_n{\stackrel{.}{\rho }}_{{\beta }^0}-\left(\widehat{\Theta }{P}_n{\ddot{\rho }}_{\widehat{\beta }}-I\right)\left(\widehat{\beta }(\tau ,j)-{\beta }^0(j)\right)-{\text{Rem}}_1 \\ =-\widehat{\Theta }{P}_n{\stackrel{.}{\rho }}_{{\beta }^0}-{\text{Rem}}_2. \end{gathered}$$

By the proof of Theorem 3.1 in van de Geer et al. (2014), we have $\widehat{\Theta }{P}_n{\ddot{\rho }}_{\widehat{\beta }}-I=O(\lambda )$. According to the third result in Theorem 1, we have $\widehat{\beta }(\tau ,j)-{\beta }^0(j)=O_P(s_{\ast }\lambda )$. Then, it follows that

$$\mid {\text{Rem}}_2\mid \le \mid {\text{Rem}}_1\mid +O(\lambda ){\Vert \widehat{\beta }(\tau ,j)-{\beta }^0(j)\Vert }_1=o_P\left(n^{-1∕2}\right)+O_P\left(s_{\ast }{\lambda }^2\right)=o_P\left(n^{-1∕2}\right),$$

since the condition $s_{\ast }{\lambda }^2=o\left(s_{\ast }\frac{\log p}{n}\right)=o\left(n^{-1∕2}\right)$ holds. By straight calculations, we have

$$\begin{gathered} \sqrt{n}\left(\tilde{\beta }(\widehat{\tau },j)-{\beta }^0(j)\right)=-\sqrt{n}\widehat{\Theta }{P}_n{\stackrel{.}{\rho }}_{{\beta }^0}-\sqrt{n}{\text{Rem}}_2 \\ =-\sqrt{n}\widehat{\Theta }{P}_n{\stackrel{.}{\rho }}_{{\beta }^0}-o_P(1). \end{gathered}$$

By the proof of Theorem 3.1 in van de Geer et al. (2014), we can easily conclude that

$$\sqrt{n}\left({\tilde{\beta }}_s(\widehat{\tau },j)-{\beta }_s^0(j)\right)∕{\widehat{\sigma }}_{j,s}=V_{j,s}+o_{\mathbf{P}}(1),s\in \{1,\dots ,p\},$$

where $V_{j,s}\sim \mathcal{N}(0,1)$ and ${\widehat{\sigma }}_s^2≔({\widehat{\Theta }}_j{P}_n{\stackrel{.}{\rho }}_{\widehat{\beta }}{\stackrel{.}{\rho }}_{\widehat{\beta }}^T{\widehat{\Theta }}_j^T{)}_{s,s}$. ■

Proof of Theorem 2. First we will show that under the conditions of Theorem 1, on ${\mathcal{T}}_0\cap {\mathcal{T}}_1$, we have three cases:

Case A: Change point number is overestimated, $l(\widehat{\tau })>\tilde{k}$. We have $\tilde{k}=1$ and $l({\widehat{\tau }}^b)=1$.

Case B: Change point number is underestimated, $l(\widehat{\tau })<\tilde{k}$. For $\tilde{k}>1$, we have h(0, 1) and $l({\widehat{\tau }}^b)=0$.

Case C: Change point number is correctly estimated, $l(\widehat{\tau })=\tilde{k}$. For $\tilde{k}>1$, we have h(0, 1) ∈ [δ, 1 − δ].

This fact can be derived straightforwardly from the proof of Theorem 1, as the objective functions coincide for 1 or 2 segments; that is, for all u ∈ [0, 1],

$$H((0,u,1))=Z(0,u)+Z(u,1).$$

For Case A, suppose $\tilde{k}=1$ and τ = (0, 1). Like in the proof of Case 1 in Theorem 1, we can obtain

$$H({\tau }^0)<\underset{u\in (\delta ,1-\delta )}{\text{min}}H((0,u,1)),$$

and h(0, 1) = 0. We suppose $\tilde{k}>1$ and $h(0,1)\notin {\cup }_{j=1}^{\tilde{k}-1}({\tilde{\tau }}_j-c^{\ast }\delta \lambda ,{\tilde{\tau }}_j-c^{\ast }\delta \lambda )$. We define τ⁽⁰⁾ = (0, h(0, 1), 1), ${\tau }^{(1)}={\tau }^{(0)}\cup \{{\tilde{\tau }}_j\}$, τ⁽²⁾ = τ⁽¹⁾\{h(0, 1)}.

For Case B, h(0, 1) = 0, like in the proof of Case 2 in Theorem 1, we can obtain H(τ⁽⁰⁾) > H(τ⁽¹⁾). For Case C, h(0, 1) ∈ [δ, 1 − δ], like in the proof of Case 3 in Theorem 1, we can obtain

$$H\left({\tau }^{(0)}\right)-H\left({\tau }^{(2)}\right)=H\left({\tau }^{(0)}\right)-H\left({\tau }^{(1)}\right)+H\left({\tau }^{(1)}\right)-H\left({\tau }^{(2)}\right)>0.$$

Since h(0, 1) minimizes Equation (17), all three cases result in a contradiction. Then, we can replace (0,1) by each sub-interval and obtain the same results, which completes the proof. ■

Proof of Theorem 3. Firstly we recall our proposed statistics as follows:

$$\begin{gathered} W_{1∕4}-W_{3∕4} \\ =P_n\rho ((0,\frac{1}{4}),{\widehat{\beta }}_{(\frac{1}{4},1)})+P_n\rho ((\frac{1}{4},1),{\widehat{\beta }}_{(\frac{1}{4},1)})-P_n\rho ((0,\frac{3}{4}),{\widehat{\beta }}_{(0,\frac{3}{4})})-P_n\rho ((\frac{3}{4},1),{\widehat{\beta }}_{(\frac{3}{4},1)}). \end{gathered}$$

Then, for convenience, we consider two cases for the change point location denoted by τ:

Case E: 0 < τ ≤ 1/4,

Case F: 1/4 < τ < 1/2.

Next, we will discuss each case in detail. Case E: In this case, 0 < τ ≤ 1/4, so we have

$$\begin{gathered} W_{1∕4}-W_{3∕4}=\left(P_n\rho ((0,\frac{1}{4}),{\widehat{\beta }}_{(0,\frac{1}{4})})-P_n\rho ((0,\frac{1}{4}),{\widehat{\beta }}_{(0,\frac{3}{4})})\right) \\ +\left(P_n\rho ((\frac{1}{4},\frac{3}{4}),{\widehat{\beta }}_{(\frac{1}{4},1)})-P_n\rho ((\frac{1}{4},\frac{3}{4}),{\widehat{\beta }}_{(0,\frac{3}{4})})\right) \\ +\left(P_n\rho ((\frac{1}{4},1),{\widehat{\beta }}_{(\frac{1}{4},1)})-P_n\rho ((\frac{3}{4},1),{\widehat{\beta }}_{(\frac{3}{4},1)})\right). \end{gathered}$$

We observe there is no change point in (1/4, 1). So the Lasso estimations of any subinterval (u, v) ⊂ (1/4, 1) make no difference and we can replace ${\widehat{\beta }}_{(\frac{1}{4},1)}$ by ${\widehat{\beta }}_{(\frac{1}{4},\frac{3}{4})}$ or ${\widehat{\beta }}_{(\frac{3}{4},1)}$, and it follows that

$$\begin{gathered} W_{1∕4}-W_{3∕4}=\left(P_n\rho ((0,\frac{1}{4}),{\widehat{\beta }}_{(0,\frac{1}{4})})-P_n\rho ((0,\frac{1}{4}),{\widehat{\beta }}_{(0,\frac{3}{4})})\right) \\ +\left(P_n\rho ((\frac{1}{4},\frac{3}{4}),{\widehat{\beta }}_{(\frac{1}{4},\frac{3}{4})})-P_n\rho ((\frac{1}{4},\frac{3}{4}),{\widehat{\beta }}_{(0,\frac{3}{4})})\right). \end{gathered}$$ (23)

We denote the intervals by $J_1=(0,\frac{1}{4})$, $J_2=(\frac{1}{4},\frac{3}{4})$, and $J=(0,\frac{3}{4})$. (23) can be organized as follows:

$$\begin{gathered} W_{1∕4}-W_{3∕4} \\ =\left(P_n\rho (J_1,{\widehat{\beta }}_{J_1})-P_n\rho (J_1,{\widehat{\beta }}_J)\right)+\left(P_n\rho (J_2,{\widehat{\beta }}_{J_2})-P_n\rho (J_2,{\widehat{\beta }}_J)\right) \\ =P_n\rho (J_1,{\widehat{\beta }}_{J_1})+P_n\rho (J_2,{\widehat{\beta }}_{J_2})-P_n\rho (J,{\widehat{\beta }}_J). \end{gathered}$$

Then using the same argument as Case 1 in Theorem 1, we can obtain W_1/4 < W_3/4.

Case F: In this case, 1/4 < τ < 1/2, we observe that there is no change point within these two intervals (0, 1/4) and (3/4, 1). Then, by straightforward calculations, we can obtain

$$\begin{gathered} W_{1∕4}-W_{3∕4}=P_n\rho ((\frac{1}{4},1),{\widehat{\beta }}_{(\frac{1}{4},1)})-P_n\rho ((0,\frac{3}{4}),{\widehat{\beta }}_{(0,\frac{3}{4})} \\ =P_n\rho ((\frac{1}{4},\frac{3}{4}),{\widehat{\beta }}_{(\frac{1}{4},1)})+P_n\rho ((\frac{3}{4},1),{\widehat{\beta }}_{(\frac{1}{4},1)})-P_n\rho ((0,\frac{1}{4}),{\widehat{\beta }}_{(0,\frac{3}{4})}-P_n\rho ((\frac{1}{4},\frac{3}{4}),{\widehat{\beta }}_{(0,\frac{3}{4})}). \end{gathered}$$

We denote the intervals by (0, 1/4) = K₁, (1/4, 3/4) = J₁, (3/4, 1) = J₂, and it follows that

$$\begin{gathered} W_{1∕4}-W_{3∕4} \\ =P_n\rho (J_1,{\widehat{\beta }}_J)+P_n\rho (J_2,{\widehat{\beta }}_J)-P_n\rho (K_1,{\widehat{\beta }}_I)-P_n\rho (J_1,{\widehat{\beta }}_I). \end{gathered}$$

Then using the same argument as Case 3 in Theorem 1, we can obtain W_1/4 < W_3/4. ■