Consistent Model Selection in Segmented Line Regression

Abstract

The Schwarz criterion or Bayes Information Criterion (BIC) is often used to select a model dimension, and some variations of the BIC have been proposed in the context of change-point problems. In this paper, we consider a segmented line regression model with an unknown number of change-points and study asymptotic properties of Schwarz type criteria in selecting the number of change-points. Noticing the overestimating tendency of the traditional BIC observed in some empirical studies and being motivated by asymptotic behavior of the modified BIC proposed by Zhang and Siegmund (2007), we consider a variation of the Schwarz type criterion that applies a harsher penalty equivalent to the model with one additional unknown parameter per segment. For the segmented line regression model without the continuity constraint, we prove the consistency of the number of change-points selected by the criterion with such type of a modification and summarize the simulation results that support the consistency. Further simulations are conducted for the model with the continuity constraint, and we empirically observe that the asymptotic behavior of this modified version of BIC is comparable to that of the criterion proposed by Liu, Wu, and Zidek (1997).

1 Introduction

A main concern in regression model selection is how to select the “best” set of independent variables, and two major approaches of the model selection are hypothesis testing and information criteria approaches. In the context of change-point problems, both approaches have been applied to select the number of change-points, and their analytic and empirical properties have been investigated by many researchers. One of widely used information criteria is the Bayes Information Criterion (BIC) proposed by Schwarz (1978). This Schwarz criterion selects the model dimension by finding the Bayes solution that maximizes a posterior probability of the model, and Schwarz (1978) derived the following criterion by evaluating the leading terms of its asymptotic expansion:

$$\mathrm{SC}\left(p\right)=\underset{{\theta }_p}{\sup }\log \left(\mathrm{lik}\left({\theta }_p\right)\right)-\frac{p}{2}\log n=\log \left(\mathrm{lik}\left({\widehat{\theta }}_p\right)\right)-\frac{p}{2}\log n,$$

where lik(θ_p) is the likelihood function of θ_p for the model with dimension p and ${\widehat{\theta }}_p$ is the maximum likelihood estimator of θ_p. As in general information criteria, the Schwarz criterion has two parts, the log of the maximized likelihood function and the penalty function that penalizes for the model dimension, and the method selects the model that maximizes SC(p). Note that its validity is established in Schwarz (1978) for “the case of independent, identically distributed observations, and linear models.”

Yao (1988) studied the problem to select the number of change-points in means of normally distributed random variables, where the total number of unknown parameters for the model with k change-points is p = 2(k + 1). For the number of change-points estimated by minimizing –SC(p) with p = 2k, Yao (1988) proved its consistency. Lee (1997) considered a similar type of a criterion to select the number of change-points in a sequence of random variables from an exponential family distribution. Under some mild conditions on spacings of successive change-points, Lee (1997) proved the consistency of the number of change-points estimated by the Schwarz type criterion whose penalty term is greater than 2k(1 + ε₀) log n for some ε₀ > 0. Zhang and Siegmund (2007) noted that the usage of the Schwarz criterion “is not theoretically justified” in their situation due to irregularities in the likelihood function and proposed a modified BIC derived as an asymptotic approximation of the Bayes factor to determine the number of change-points in means of normally distributed random variables. For other types of modifications and applications for detecting mean changes, see Ninomiya (2005), Pan and Chen (2006), and Hannart and Naveau (2012).

In the context of segmented line regression, similar approaches have been proposed to select the number of change-points. Kim et al. (2000) proposed the permutation test to select the number of change-points in the segmented line regression model where segments are assumed to be continuous at change-points, called the joinpoint regression model in their paper. Kim et al. (2009) considered the traditional BIC,

$$\mathrm{BIC}\left(k\right)=-\frac{2}{n}\mathrm{SC}\left(2k\right)=\log ({\mathrm{RSS}}_k∕n)+2k\frac{\log n}{n},$$

where RSS_k is the residual sum of squares for the model with k change-points, and compared its performance with those of the permutation test procedure of Kim et al. (2000) and the method based on generalized cross validation used in MARS of Friedman (1991). Note that the penalty term of $2k\frac{\log n}{n}$ is chosen based on 2k + 3 unknown parameters for the joinpoint regression model with k change-points. Liu et al. (1997) considered a general segmented line regression model allowing a discontinuity at the change-point and non-Gaussian errors, proposed a penalty term with a bigger order than that of BIC(k), and proved the consistency of the dimension selected by minimizing their criterion:

$$\mathrm{MIC}\left(k\right)=\log ({\mathrm{RSS}}_k∕(n-p^{\ast })+p^{\ast }\frac{c_0{\left(\log n\right)}^{2+{\delta }_0}}{n},$$

where p* = p*(k) = (k + 1)p + k for the model with k change-points and p covariates and c₀ and δ₀ are positive constants. Two Bayesian model selection methods based on the Bayes factor and a Bayesian version of BIC were developed in Tiwari et al. (2005) who investigated their empirical properties via simulations and compared their performances with that of the permutation procedure of Kim et al. (2000). Martinez-Beneito et al. (2011) also proposed a Bayesian model selection method that provides posterior probabilities and is flexible to work with Poisson count data.

This paper is motivated from empirical results where the traditional BIC indicated a tendency to over-estimate the number of change-points (See Table 1 of Kim et al. (2009), Table 1 of Zhang and Siegmund (2007)). When the argument of Zhang and Siegmund (2007) is applied to segmented line regression, the penalty of the modified BIC is harsher than that of the traditional BIC, asymptotically corresponding to one additional unknown parameter per segment under some conditions, and this motivated us to consider a BIC type criterion whose penalty is $4k\frac{\log n}{n}$ for the segmented line regression model without the continuity constraint and $3k\frac{\log n}{n}$ for the model with the continuity constraint. Note that for segmented line regression with k change-points, the number of unknown parameters is 3k + 3 for the model without the continuity constraint and 2k + 3 for the model with the continuity constraint. Let

$${\mathrm{BIC}}_d\left(k\right)=\log ({\mathrm{RSS}}_k∕n)+P{E}_d\left(k\right)=\log ({\mathrm{RSS}}_k∕n)+dk\frac{\log n}{n},$$ (1)

for penalty coefficients, d. Then the traditional BICs are BIC₃ for the unconstrained model and BIC₂ for the constrained model.

Table 1.

Unconstrained model with k₀ = 1 and x_i = i/n for i = 1,..., n. E(Y|x) = (1 + x)I(x ≤ 0.5) + (1.35 + 0.5x)I(x > 0.5)

n	Prob	σ = 0.1			σ = 0.05
		BIC3	BIC4	MIC	BIC3	BIC4	MIC
30	$\widehat{P}(\widehat{\kappa }<k_0)$ $\widehat{P}(\widehat{\kappa }=k_0)$ $\widehat{P}(\widehat{\kappa }>k_0)$	$0.47\stackrel{‒}{6}$ 0.360 $0.16\stackrel{‒}{3}$	$0.76\stackrel{‒}{3}$ 0.210 $0.02\stackrel{‒}{6}$	$0.44\stackrel{‒}{6}$ $0.38\stackrel{‒}{3}$ 0.170	0.030 $0.72\stackrel{‒}{3}$ $0.24\stackrel{‒}{6}$	$0.12\stackrel{‒}{3}$ 0.830 $0.04\stackrel{‒}{6}$	$0.02\stackrel{‒}{6}$ 0.730 $0.24\stackrel{‒}{3}$
50	$\widehat{P}(\widehat{\kappa }<k_0)$ $\widehat{P}(\widehat{\kappa }=k_0)$ $\widehat{P}(\widehat{\kappa }>k_0)$	$0.38\stackrel{‒}{6}$ $0.52\stackrel{‒}{6}$ $0.08\stackrel{‒}{6}$	0.670 0.320 0.010	0.450 0.510 0.040	0 $0.85\stackrel{‒}{3}$ $0.14\stackrel{‒}{6}$	0.010 $0.97\stackrel{‒}{3}$ $0.01\stackrel{‒}{6}$	$0.00\stackrel{‒}{3}$ 0.910 $0.08\stackrel{‒}{6}$
75	$\widehat{P}(\widehat{\kappa }<k_0)$ $\widehat{P}(\widehat{\kappa }=k_0)$ $\widehat{P}(\widehat{\kappa }>k_0)$	$0.19\stackrel{‒}{3}$ $0.74\stackrel{‒}{6}$ 0.060	$0.49\stackrel{‒}{3}$ $0.50\stackrel{‒}{6}$ 0	$0.34\stackrel{‒}{6}$ $0.62\stackrel{‒}{3}$ 0.030	0 $0.92\stackrel{‒}{3}$ $0.07\stackrel{‒}{6}$	$0.00\stackrel{‒}{3}$ 0.980 $0.01\stackrel{‒}{6}$	$0.00\stackrel{‒}{3}$ $0.97\stackrel{‒}{6}$ 0.020
150	$\widehat{P}(\widehat{\kappa }<k_0)$ $\widehat{P}(\widehat{\kappa }=k_0)$ $\widehat{P}(\widehat{\kappa }>k_0)$	$0.04\stackrel{‒}{3}$ 0.900 $0.05\stackrel{‒}{6}$	$0.14\stackrel{‒}{6}$ $0.84\stackrel{‒}{6}$ $0.00\stackrel{‒}{6}$	0.130 $0.86\stackrel{‒}{3}$ $0.00\stackrel{‒}{6}$	0 $0.95\stackrel{‒}{3}$ $0.04\stackrel{‒}{6}$	0 0.990 0.010	0 0.990 0.010
200	$\widehat{P}(\widehat{\kappa }<k_0)$ $\widehat{P}(\widehat{\kappa }=k_0)$ $\widehat{P}(\widehat{\kappa }>k_0)$	$0.00\stackrel{‒}{6}$ $0.96\stackrel{‒}{3}$ 0.030	0.040 $0.95\stackrel{‒}{6}$ $0.00\stackrel{‒}{3}$	$0.04\stackrel{‒}{3}$ $0.95\stackrel{‒}{3}$ $0.00\stackrel{‒}{3}$	0 $0.96\stackrel{‒}{3}$ $0.03\stackrel{‒}{6}$	0 $0.99\stackrel{‒}{6}$ $0.00\stackrel{‒}{3}$	0 $0.99\stackrel{‒}{6}$ $0.00\stackrel{‒}{3}$

Our interest in this paper is on asymptotic behavior of BIC_d, a simple model selection criterion whose penalty term has the same order as that of the traditional BIC, and we focus on asymptotic properties of BIC₄ for the model without the continuity constraint and BIC₃ for the model with the continuity constraint. In Section 2, we formally introduce the unconstrained model and prove the consistency of the model dimension selected by BIC₄ for the unconstrained model with Gaussian errors. This result provides a consistent model selection criterion that imposes an asymptotically milder penalty than MIC does. In Section 3, we present the results of a simulation study where we compare the performance of BIC₄ with those of BIC₃ and MIC. Section 4 includes empirical results and discussion on the constrained case where the segments are constrained to be continuous at the change-points. Further discussion is presented in Section 5.

2 Selection Methods and Consistency: Unconstrained Model

Suppose that we observe (x₁, y₁), . . ., (x_n, y_n) and consider a segmented line regression model such that

$$y_i={\beta }_{j,0}+{\beta }_{j,1}{x}_i+{∊}_i,\text{if}{\tau }_{j-1}<x_i\le {\tau }_j(j=1,\dots ,\kappa +1),$$ (2)

where κ is the unknown number of change-points, the τ's are unknown change-points with ${\tau }_0={\min }_i{x}_i-\frac{1}{n}$ and τ_κ+1 = max_i x_i, and the ε_i are independent N(0, σ²).

Let us consider the Schwarz type criterion (1) defined above and estimate κ as

$$\widehat{\kappa }={\mathrm{argmin}}_{0\le k\le K}{\mathrm{BIC}}_d\left(k\right),$$

where K is a pre-determined maximum number of change-points. For the segmented line regression model without the continuity constraint, (2), we consider BIC₃ as the traditional BIC based on 3k + 3 unknown parameters, and study the large sample properties of BIC₄, which is a modified version motivated by Zhang and Siegmund (2007).

Liu et al. (1997) considered a segmented line regression model that allows a discontinuity in the mean function and non-Gaussian errors, and their main idea to prove the consistency of $\widehat{\kappa }$ was that the difference between the variances estimated under the true model and a model with a larger number of change-points is in the order of $\frac{{\left(\log n\right)}^2}{n}$, which seemed to have motivated their choice of the penalty term. Thus, arguments used in Liu et al. (1997) can not be applied in our situation where the penalty term is a constant multiple of $\frac{\log n}{n}$. Yao (1988) proved the consistency of the number of change-points estimated by the traditional BIC, BIC₂, in the context of simple mean changes. He used properties of a normal distribution to show that the difference between the variances estimated under the true model and the model with a larger number of change-points than the true model, can be bounded above by the difference between the penalty terms of the two models. However, his argument to bound the difference does not work in our situation, and the upper bound in our case is achieved with d ≥ 4 using properties of a chi-square distribution.

Rewrite the model (2) as

$$y_i={\beta }_{j,0}+{\beta }_{j,1}{x}_i+{∊}_i={\mathit{x}}_i^{\prime }{\beta }_j+{∊}_i\text{if}{x}_i\in ({\tau }_{j-1},{\tau }_j],(j=1,\dots ,\kappa +1),$$ (3)

where x_i = (1, x_i)′, β_j = (β_j,0, β_j,1)′ and the ε_i are independent and normally distributed with mean zero and variance σ². Without loss of generality, we assume that 0 < x_i < 1 for i = 1, . . ., n. Note that this model can be expressed as

$$\mathit{y}=\sum _{j=1}^{\kappa +1}X({\tau }_{j-1},{\tau }_j){\beta }_j+∊,$$

where y = (y₁, y₂, . . ., y_n)′, X = (1, x) with x = (x₁, x₂, . . ., x_n)′ and 1 = (1, 1, . . ., 1)′, and X(α, η) = I(α, η)X with I(α, η) = diag(1_{{x1 ∈ (α,η]}}, . . ., 1_{{xn ∈ (α,η]}}) for any α < η. Note that 1_A = 1 if A is true and 0 otherwise.

Let ${\mathcal{D}}_k=\{{(t_1,\dots ,t_k)}^{\prime }:t_0<t_1<\cdots <t_k<t_{k+1}\}$ with t₀ = 0 and t_k+1 = 1. For a change-point vector, ${\mathit{t}}_k={(t_1.\dots ,t_k)}^{\prime }\in {\mathcal{D}}_k$, denote the design matrix and the corresponding hat matrix as

$$\mathit{X}\left({\mathit{t}}_k\right)=(X(t_0,t_1),X(t_1,t_2),\dots ,X(t_k,t_{k+1}))$$

and

$$H\left({\mathit{t}}_k\right)=\mathit{X}\left({\mathit{t}}_k\right){\left(\mathit{X}{\left({\mathit{t}}_k\right)}^{\prime }\mathit{X}\left({\mathit{t}}_k\right)\right)}^{-1}\mathit{X}{\left({\mathit{t}}_k\right)}^{\prime }=\sum _{j=1}^{k+1}H(t_{j-1},t_j),$$

where H(α, η) is the projection matrix based on X(α, η). Then the sum of squared residuals at given t_k is

$$S_n\left({\mathit{t}}_k\right)={\mathit{y}}^{\prime }(I-H\left({\mathit{t}}_k\right))\mathit{y}=\sum _{j=1}^{k+1}{S}_n(t_{j-1},t_j),$$

where S_n(α, η) = y′(I(α, η) – H(α, η))y, and thus the sum of squared residuals for the model with k change-points is obtained as

$$RS{S}_k=\underset{{\mathit{t}}_k}{\inf }{S}_n\left({\mathit{t}}_k\right)=S_n\left({\widehat{\tau }}_k\right).$$

We also denote the true values of κ, σ² and τ as κ₀, ${\sigma }_0^2$, and ${\tau }^0={({\tau }_1^0,{\tau }_2^0,\dots ,{\tau }_{{\kappa }_0}^0)}^{\prime }$, respectively.

Assumption 1

The regressor variable x has a positive and continuous density function in any small neighborhoods of τ⁰. Also x is independent of the error, ε.

For the case of nonrandom design points, the following assumption can replace Assumption 1. Note that the data spacing should be of order O(1/n) to satisfy this assumption.

Assumption 1’

Let {δ_n} be a sequence of constants for which O(1/n) ≤ δ_n = o(1). The number of data points in any small neighborhoods of ${\tau }_r^0$, (r = 1, . . ., κ₀) with volume δ_n is of order at least δ_n.

Theorem 1

Suppose that the segmented line regression model without the continuity constraint (3) satisfies one of Assumption 1 or 1’. Let k < κ₀. Then, for any constant d > 0,

$$P({\mathrm{BIC}}_d\left(k\right)>{\mathrm{BIC}}_d\left({\kappa }_0\right))\to 1$$

as n → ∞.

Proof

Let d be an arbitrary positive constant. By Lemma 5.3 and Lemma 5.4(i) of Liu et. al.(1997), it is true that for some C > 0,

$${\widehat{\sigma }}_k^2=\frac{RS{S}_k}{n}=\frac{S_n\left({\widehat{\tau }}_k\right)}{n}>{\sigma }_0^2+C$$

with probability going to 1 as n → ∞. Since ${\widehat{\sigma }}_{{\kappa }_0}^2=\frac{RS{S}_{{\kappa }_0}}{n}$ converges to ${\sigma }_0^2$ in probability (that is, ${\widehat{\sigma }}_{{\kappa }_0}^2\stackrel{p}{\to }{\sigma }_0^2$), it is true that

$${\mathrm{BIC}}_d\left(k\right)-{\mathrm{BIC}}_d\left({\kappa }_0\right)=\log \left(\frac{{\widehat{\sigma }}_k^2}{{\widehat{\sigma }}_{{\kappa }_0}^2}\right)+\frac{d(k-{\kappa }_0)}{n}\log n>0$$

with probability going to 1 as n → ∞.

Theorem 1 guarantees that for any d > 0, $\widehat{\kappa }$ based on BIC_d is consistent from the below. For the consistency of $\widehat{\kappa }$ from the above, we now get the result for d ≥ 4 under the assumption that the mean function of the regression model is discontinuous.

In order to prove that for d ≥ 4 and for k > κ₀, P(BIC_d(k) > BIC_d(κ₀)) → 1 as n → ∞, we first state the following lemmas whose proofs are given in Appendix. In the lemmas and Theorem 2, we let ${\delta }_n=\frac{{\left(\log n\right)}^2}{n}$, and for 1 ≤ r ≤ κ₀, let

$$B_r\left(n\right)=\{{\mathit{t}}_k\in {\mathcal{D}}_k:\mid t_j-{\tau }_r^0\mid >{\delta }_n\text{for}1\le j\le k\}.$$

Lemma 1

Suppose that the segmented line regression model without the continuity constraint (3) satisfies one of Assumption 1 or 1’. Then, for k > κ₀,

$$P({\widehat{\tau }}_k\in {\cup }_{r=1}^{{\kappa }_0}{B}_r\left(n\right))\to 0$$

as n → ∞.

Let ξ_k+3κ0 be the ordered set of {t_k, τ⁰, τ⁰ – δ_n1, τ⁰ + δ_n1}. Let $T_r=({\tau }_j^0-{\delta }_n,{\tau }_j^0+{\delta }_n]$ for r = 1, . . ., κ₀. Let $D_0=(0,{\tau }_1^0-{\delta }_n]$, $D_{{\kappa }_0}=({\tau }_{{\kappa }_0}^0+{\delta }_n,1]$, and for r = 1, . . ., κ₀ – 1, let $D_r=({\tau }_r^0+{\delta }_n,{\tau }_{r+1}^0-{\delta }_n]$. Let

$$\begin{array}{cc}\hfill G_{n,1}=& \sum _{({\xi }_{l=1},{\xi }_l]\subset {\cup }_{r=1}^{{\kappa }_0}{T}_r}{∊}^{\prime }H({\xi }_{l-1},{\xi }_l)∊,\hfill \\ \hfill G_{n,2}=& \sum _{({\xi }_{l=1},{\xi }_l]\subset {\cup }_{r=0}^{{\kappa }_0}{D}_r}{∊}^{\prime }H({\xi }_{l-1},{\xi }_l)∊.\hfill \end{array}$$

Lemma 2

Suppose that the segmented line regression model without the continuity constraint (3) satisfies one of Assumption 1 or 1’. Then, for large n,

$$G_{n,1}=O_p\left(\log \log n\right).$$

Lemma 3

Suppose that the segmented line regression model without the continuity constraint (3) satisfies one of Assumption 1 or 1’. Then, for k > κ₀ and any ${\mathit{t}}_k\in {\cap }_{r=1}^{{\kappa }_0}{B}_r{\left(n\right)}^c$, it is true that for any δ > 0,

$$P(G_{n,2}\le (k-{\kappa }_0-1)4(1+\delta ){\sigma }_0^2\log n)\to 1$$

as n → ∞.

Theorem 2

Suppose that the segmented line regression model without the continuity constraint (3) satisfies one of Assumption 1 or 1’. Let k > κ₀. Then, for any constant d ≥ 4,

$$P({\mathrm{BIC}}_d\left(k\right)>{\mathrm{BIC}}_d\left({\kappa }_0\right))\to 1$$

as n → ∞.

Proof

Without loss of generality, we suppose k = κ₀ + 1. Since ${\widehat{\sigma }}_{{\kappa }_0}^2\stackrel{p}{\to }{\sigma }_0^2$ and ${\widehat{\sigma }}_k^2\stackrel{p}{\to }{\sigma }_0^2$, it is true that for any small δ > 0,

$$\begin{array}{cc}\hfill & {\mathrm{BIC}}_d\left(k\right)-{\mathrm{BIC}}_d\left({\kappa }_0\right)\hfill \\ \hfill =& \log \left(\frac{S_n\left({\widehat{\tau }}_k\right)}{n}\right)-\log \left(\frac{S_n\left({\widehat{\tau }}_{{\kappa }_0}\right)}{n}\right)+\frac{d(k-{\kappa }_0)}{n}\log n\hfill \\ \hfill =& \log \left(1-\frac{{\widehat{\sigma }}_{{\kappa }_0}^2-{\widehat{\sigma }}_k^2}{{\widehat{\sigma }}_{{\kappa }_0}^2}\right)+\frac{d(k-{\kappa }_0)}{n}\log n\hfill \\ \hfill \ge & (1+\delta )\left(\frac{{\widehat{\sigma }}_k^2-{\widehat{\sigma }}_{{\kappa }_0}^2}{{\widehat{\sigma }}_{{\kappa }_0}^2}\right)+\frac{d(k-{\kappa }_0)}{n}\log n\hfill \end{array}$$

with probability going to 1 as n → ∞.

Since $P({\widehat{\tau }}_k\in {\cup }_{r=1}^{{\kappa }_0}{B}_r\left(n\right))\to 0$ by Lemma 1, we now may focus on t_k's in ${\cap }_{r=1}^{{\kappa }_0}{B}_r{\left(n\right)}^c$. Let ξ_k+3κ0 be the ordered set of {t_k, τ⁰, τ⁰ – δ_n1, τ⁰ + δ_n1}. Then

$$\begin{array}{cc}\hfill & S_n\left({\mathit{t}}_k\right)-S_n\left({\widehat{\tau }}_{{\kappa }_0}\right)\hfill \\ \hfill \ge & S_n\left({\xi }_{k+3{\kappa }_0}\right)-S_n\left({\tau }^0\right)\hfill \\ \hfill \ge & -\sum _{l=1}^{k+3{\kappa }_0+1}{∊}^{\prime }H({\xi }_{l-1},{\xi }_l)∊\hfill \\ \hfill =& -\{\sum _{{\xi }_{l-1},{\xi }_l]\subset {\cup }_{r=1}^{{\kappa }_0}{T}_r}{∊}^{\prime }H({\xi }_{l-1},{\xi }_l)∊+\sum _{({\xi }_{l-1},{\xi }_l]\subset {\cup }_{r=0}^{{\kappa }_0}{D}_r}{∊}^{\prime }H({\xi }_{l-1},{\xi }_l)∊\}\hfill \\ \hfill =& -(G_{n,1}+G_{n,2}),\hfill \end{array}$$

where ξ₀ = 0 and ξ_k+3κ0+1 = 1.

By Lemma 2 and Lemma 3, the following holds for any ${\mathit{t}}_k\in {\cap }_{r=1}^{{\kappa }_0}{B}_r{\left(n\right)}^c$: it is true that for any δ > 0,

$$S_n\left({\mathit{t}}_k\right)-S_n\left({\widehat{\tau }}_{{\kappa }_0}\right)>-(\delta +(k-{\kappa }_0-1)4(1+\delta )){\sigma }_0^2\log n$$

and ${\widehat{\alpha }}_{{\kappa }_0}^2>{\sigma }_0^2(1-\delta )$, with probability going to 1 as n → ∞. Therefore, for any δ > 0, ${\mathrm{BIC}}_d\left(k\right)-{\mathrm{BIC}}_d\left({\kappa }_0\right)>-(1+\delta )(\delta +(k-{\kappa }_0-1)4(1+\delta )){\sigma }_0^2\log n∕n{\sigma }_0^2(1-\delta )+d(k-{\kappa }_0)\log n∕n$ with probability going to 1 as n → ∞. For d ≥ 4, we can choose δ small enough such that

$$(1+\delta )(\delta +(k-{\kappa }_0-1)4(1+\delta ))∕(1-\delta )<d(k-{\kappa }_0),$$

and thus

$${\mathrm{BIC}}_d\left(k\right)-{\mathrm{BIC}}_d\left({\kappa }_0\right)>0$$

with probability going to 1 as n → ∞.

Remark

Theorem 1 and Theorem 2 imply that the consistency of $\widehat{\kappa }$ is achieved by BIC_d for d ≥ 4. Because the penalty term of MIC for the one independent variable case with p = 1, $(2k+1)\frac{c_0{\left(\log n\right)}^{2+{\delta }_0}}{n}$, is larger than $4k\frac{\log n}{n}$ for large n, the result in this section indicates that the consistent model selection can be performed with a criterion whose penalty is asymptotically milder than that of MIC under the assumption of Gaussian errors.

3 Simulations: Unconstrained Model

This section summarizes simulations conducted to study asymptotic behavior of BIC₄ whose consistency is proved in Section 2 and to compare its performance with those of BIC₃, the traditional BIC, and MIC whose consistency is established in Liu et al. (1997). For each case of the model parameters chosen, 300 replications of data sets are simulated, and $P(\widehat{\kappa }<{\kappa }_0)$, $P(\widehat{\kappa }-{\kappa }_0)$, and $P(\widehat{\kappa }>{\kappa }_0)$ are estimated where κ₀ is the true number of change-points and $\widehat{\kappa }(0\le \widehat{\kappa }\le K)$ is the estimate obtained by each selection method. The value of K was set as 4, and $\stackrel{‒}{3}$ or $\stackrel{‒}{6}$ represents a repeating decimal.

Tables 1, 2, and 3 summarize the results for unconstrained model cases. Tables 1 and 2 consider the cases with one change-point for two different x configurations, fixed x in Table 1 and random x in Table 2. Table 3 is for the case with two change-points where x is fixed. Note that MIC of Liu et al. (1997) uses the penalty term, PE(k) = p*c₀(ln n)^2+δ₀/n = (2k + 1)(0.299)(ln n)^2.1/n for the model with one independent variable, which is based on p* = (k+1)p+k = 2k+1 with p = 1 in their equation (2.3). According to Liu et al. (1997), δ₀ = 0.1 is arbitrarily chosen and then c₀ = 0.299 is chosen by forcing the penalty term of MIC to be equal to that of the Schwarz criterion. For the model with one independent variable, the penalty term of MIC is set to be equal to that of BIC₂ at n = 20, but MIC is not exactly same as BIC₂ when n = 20 because MIC used an unbiased estimator of σ² while BIC₂ uses the maximum likelihood estimator of σ² under the Gaussian error assumption. When the full version of MIC is compared to BIC_d, the penalty term of MIC is similar to that of BIC₃ when n = 30 or 50 and to that of BIC₄ when n = 150 or 200, respectively, which is empirically indicated in Tables 1, 2, and 3. For n very large, MIC imposes a penalty harsher than that of BIC₄.

Table 2.

Unconstrained model with k₀ = 1 and x_i ~ N(0.5, 1) for i = 1,..., n. E(Y|x) = (1 + x)I(x ≤ 0.5) + (1.35 + 0.5x)I(x > 0.5)

n	Prob	σ = 0.2			σ = 0.1
		BIC3	BIC4	MIC	BIC3	BIC4	MIC
30	$\widehat{P}(\widehat{\kappa }<k_0)$ $\widehat{P}(\widehat{\kappa }=k_0)$ $\widehat{P}(\widehat{\kappa }>k_0)$	$0.08\stackrel{‒}{3}$ $0.70\stackrel{‒}{3}$ $0.21\stackrel{‒}{3}$	0.300 $0.66\stackrel{‒}{6}$ $0.03\stackrel{‒}{3}$	$0.07\stackrel{‒}{6}$ $0.70\stackrel{‒}{6}$ $021\stackrel{‒}{6}$	0 0.760 0.240	$0.00\stackrel{‒}{6}$ $0.94\stackrel{‒}{3}$ 0.050	0 0.760 0.240
50	$\widehat{P}(\widehat{\kappa }<k_0)$ $\widehat{P}(\widehat{\kappa }=k_0)$ $\widehat{P}(\widehat{\kappa }>k_0)$	$0.01\stackrel{‒}{6}$ 0.830 $0.15\stackrel{‒}{3}$	$0.07\stackrel{‒}{6}$ $0.89\stackrel{‒}{6}$ $0.02\stackrel{‒}{6}$	$0.02\stackrel{‒}{3}$ $0.89\stackrel{‒}{3}$ $0.08\stackrel{‒}{3}$	0 $0.84\stackrel{‒}{3}$ $0.15\stackrel{‒}{6}$	0 0.980 0.020	0 $0.91\stackrel{‒}{3}$ $0.08\stackrel{‒}{6}$
150	$\widehat{P}(\widehat{\kappa }<k_0)$ $\widehat{P}(\widehat{\kappa }=k_0)$ $\widehat{P}(\widehat{\kappa }>k_0)$	0 $0.95\stackrel{‒}{3}$ $0.04\stackrel{‒}{6}$	0 0.990 0.010	0 0.990 0.010	0 $0.95\stackrel{‒}{6}$ $0.04\stackrel{‒}{3}$	0 $0.99\stackrel{‒}{6}$ $0.00\stackrel{‒}{3}$	0 $0.99\stackrel{‒}{6}$ $0.00\stackrel{‒}{3}$

Table 3.

Unconstrained model with k₀ = 2 and x_i = i/n for i = 1,..., n. E(Y|x) = (1 + x)I(x ≤ 0.5) + (1.35 + 0.5x)I(0.5 < x ≤ 0.7) + (0.75 + 1.5x)I(x > 0.7)

n	Prob	σ = 0.05			σ = 0.02
		BIC3	BIC4	MIC	BIC3	BIC4	MIC
30	$\widehat{P}(\widehat{\kappa }<k_0)$ $\widehat{P}(\widehat{\kappa }=k_0)$ $\widehat{P}(\widehat{\kappa }>k_0)$	0.590 $0.24\stackrel{‒}{3}$ $0.16\stackrel{‒}{6}$	$0.87\stackrel{‒}{3}$ $0.10\stackrel{‒}{3}$ $0.02\stackrel{‒}{3}$	0.590 $0.24\stackrel{‒}{3}$ $0.16\stackrel{‒}{6}$	$0.02\stackrel{‒}{6}$ $0.67\stackrel{‒}{3}$ 0.300	$0.11\stackrel{‒}{3}$ $0.79\stackrel{‒}{3}$ $0.09\stackrel{‒}{3}$	$0.02\stackrel{‒}{6}$ $0.68\stackrel{‒}{6}$ $0.28\stackrel{‒}{6}$
50	$\widehat{P}(\widehat{\kappa }<k_0)$ $\widehat{P}(\widehat{\kappa }=k_0)$ $\widehat{P}(\widehat{\kappa }>k_0)$	$0.63\stackrel{‒}{6}$ $0.30\stackrel{‒}{6}$ $0.05\stackrel{‒}{6}$	0.890 $0.10\stackrel{‒}{6}$ $0.00\stackrel{‒}{3}$	$0.72\stackrel{‒}{3}$ 0.250 $0.02\stackrel{‒}{6}$	0 $0.82\stackrel{‒}{3}$ $0.17\stackrel{‒}{6}$	0 $0.96\stackrel{‒}{6}$ $0.03\stackrel{‒}{3}$	0 $0.89\stackrel{‒}{3}$ $0.10\stackrel{‒}{6}$
150	$\widehat{P}(\widehat{\kappa }<k_0)$ $\widehat{P}(\widehat{\kappa }=k_0)$ $\widehat{P}(\widehat{\kappa }>k_0)$	0.190 0.760 0.050	$0.38\stackrel{‒}{6}$ $0.60\stackrel{‒}{6}$ $0.00\stackrel{‒}{6}$	$0.37\stackrel{‒}{6}$ $0.61\stackrel{‒}{6}$ $0.00\stackrel{‒}{6}$	0 $0.94\stackrel{‒}{6}$ $0.05\stackrel{‒}{3}$	0 0.990 0.010	0 0.990 0.010

Results in Tables 1, 2, and 3 indicate that in the unconstrained case, the consistency of $\widehat{\kappa }$ estimated by BIC₄ or MIC, which is theoretically proved, is empirically supported with $P(\widehat{\kappa }={\kappa }_0)$ approaching one as n increases or the effect size increases. Based on Theorem 1, a choice of d related to under-fitting is not much of an issue with n large, and our interest is on the over-fitting probabilities of these selection methods when n is large. The over-fitting probabilities of BIC₄ and MIC converge to zero as n increases, but the over-fitting probability of BIC₃, the traditional BIC, is shown to be considerably larger than those of BIC₄ and MIC even for large n, although they seem to decrease as n increases.

In small sample situations, however, performances of these selection criteria depend more heavily on the effect size, the change size relative to the variability. For example, when the sample size is small and the effect size is small (for example, see the cases with σ = 0.1 and n = 30 or 50 in Table 1), the traditional BIC, BIC₃, shows the highest probability of correct selection (PCP), while BIC₄ shows a very conservative performance. However, when the effect size is large enough to have the PCP of BIC₄ greater than 0.80, BIC₄ achieves the highest PCP even in small sample cases (for example, see the cases with σ = 0.05 and n = 30 or 50 in Table 1). Regardless of the sample size, the over-fitting probabilities of the traditional BIC, BIC₃, is significantly larger than that of BIC₄. The performance of MIC is similar to that of BIC₃ when n = 30, 50 and to that of BIC₄ when n = 150, 200.

4 Selection Methods in Constrained Model

Recall the segmented line regression model introduced in (2):

$$y_i={\beta }_{j,0}+{\beta }_{j,1}{x}_i+{∊}_i,\text{if}{\tau }_{j-1}<x_i\le {\tau }_j(j=1,\dots ,\kappa +1).$$

When the segments are assumed to be continuous at the change-points,

$${\beta }_{j,0}+{\beta }_{j,1}{\tau }_j={\beta }_{j+1,0}+{\beta }_{j+1,1}{\tau }_j,$$

for j = 1, . . ., κ, and this model can also be represented as

$$y_i={\beta }_0+{\beta }_1{x}_i+{\delta }_1{(x_i-{\tau }_1)}^{+}+\cdots +{\delta }_{\kappa }{(x_i-{\tau }_{\kappa })}^{+}+{∊}_i,$$ (4)

where a⁺ = max(0, a).

For this segmented line regression model with the continuity constraint, called a joinpoint regression model in Kim et al. (2000), the total number of unknown parameters is 2k + 3 for the model with k change-points, and so the traditional BIC uses d = 2 as in the simple mean change case. We call this traditional BIC as BIC₂. Similarly as in the unconstrained case, we consider BIC₃ as a modified version in the constrained case.

Because Lemma 5.3 of Liu et al. (1997) also holds for the segmented line regression model with the continuity constraint, Theorem 1 holds for the constrained case as well. However, the arguments used to prove Theorem 2, more specifically the arguments used in the proof of Lemma 1, can not be used to prove that the over-fitting probability of BIC₃ converges to zero in the constrained case, and in this section, we investigate asymptotic behavior of BIC₃ via simulations. Although detailed proofs are provided only for the unconstrained model case, Liu et al. (1997) indicated the consistency of MIC in the constrained model case as well and the comparison of BIC₃ and MIC is expected to provide some insights on the consistency of BIC₃.

Table 4, 5, and 6 include simulation results for constrained model cases, and we used the same simulation setting as in Tables 1, 2, and 3, except that in Table 4, K = 3 is used due to an excessive computation time. Tables 4, 5, and 6 include results for BIC₂ that is the traditional BIC for the constrained model, BIC₃ that we anticipate to be consistent, and MIC whose properties are studied in Liu et al. (1997). Note that MIC of Liu et al. (1997) uses the same penalty term for both constrained and unconstrained models, which was (2k + 1)c₀(ln n)^2+δ₀/n = (2k + 1)(0.299)(ln n)^2.1/n for the model with one independent variable.

Table 4.

Constrained model with k₀ = 1 and x_i = i/n for i = 1,..., n. E(Y|x) = 1 + x – 0.5(x – 0.5)⁺

n	Prob	σ = 0.1			σ = 0.05
		BIC2	BIC3	MIC	BIC2	BIC3	MIC
30	$\widehat{P}(\widehat{\kappa }<k_0)$ $\widehat{P}(\widehat{\kappa }=k_0)$ $\widehat{P}(\widehat{\kappa }>k_0)$	$0.61\stackrel{‒}{3}$ 0.340 $0.04\stackrel{‒}{6}$	0.810 $0.18\stackrel{‒}{6}$ $0.00\stackrel{‒}{3}$	$0.80\stackrel{‒}{3}$ $0.19\stackrel{‒}{3}$ $0.00\stackrel{‒}{3}$	$0.04\stackrel{‒}{6}$ 0.880 $0.07\stackrel{‒}{3}$	$0.17\stackrel{‒}{6}$ $0.81\stackrel{‒}{3}$ 0.010	$0.15\stackrel{‒}{6}$ 0.830 $0.01\stackrel{‒}{3}$
50	$\widehat{P}(\widehat{\kappa }<k_0)$ $\widehat{P}(\widehat{\kappa }=k_0)$ $\widehat{P}(\widehat{\kappa }>k_0)$	$0.45\stackrel{‒}{6}$ $0.51\stackrel{‒}{3}$ 0.030	0.730 0.270 0	0.770 0.230 0	$0.00\stackrel{‒}{6}$ 0.920 $0.07\stackrel{‒}{3}$	0.040 $0.95\stackrel{‒}{3}$ $0.00\stackrel{‒}{6}$	0.060 $0.93\stackrel{‒}{6}$ $0.00\stackrel{‒}{3}$
75	$\widehat{P}(\widehat{\kappa }<k_0)$ $\widehat{P}(\widehat{\kappa }=k_0)$ $\widehat{P}(\widehat{\kappa }>k_0)$	$0.30\stackrel{‒}{6}$ 0.650 $0.04\stackrel{‒}{3}$	$0.64\stackrel{‒}{6}$ 0.350 $0.00\stackrel{‒}{3}$	$0.75\stackrel{‒}{6}$ $0.24\stackrel{‒}{3}$ 0	0 $0.93\stackrel{‒}{6}$ $0.06\stackrel{‒}{3}$	$0.00\stackrel{‒}{3}$ 0.990 $0.00\stackrel{‒}{6}$	$0.01\stackrel{‒}{3}$ $0.98\stackrel{‒}{3}$ $0.00\stackrel{‒}{3}$
150	$\widehat{P}(\widehat{\kappa }<k_0)$ $\widehat{P}(\widehat{\kappa }=k_0)$ $\widehat{P}(\widehat{\kappa }>k_0)$	0.100 $0.87\stackrel{‒}{3}$ $0.02\stackrel{‒}{6}$	0.280 $0.71\stackrel{‒}{6}$ $0.00\stackrel{‒}{3}$	$0.49\stackrel{‒}{3}$ $0.50\stackrel{‒}{6}$ 0	0 $0.96\stackrel{‒}{6}$ $0.03\stackrel{‒}{3}$	0 $0.99\stackrel{‒}{6}$ $0.00\stackrel{‒}{3}$	0 1 0
200	$\widehat{P}(\widehat{\kappa }<k_0)$ $\widehat{P}(\widehat{\kappa }=k_0)$ $\widehat{P}(\widehat{\kappa }>k_0)$	$0.02\stackrel{‒}{3}$ $0.96\stackrel{‒}{3}$ $0.01\stackrel{‒}{3}$	$0.11\stackrel{‒}{6}$ $0.88\stackrel{‒}{3}$ 0	$0.35\stackrel{‒}{3}$ $0.64\stackrel{‒}{6}$ 0	0 $0.98\stackrel{‒}{3}$ $0.01\stackrel{‒}{6}$	0 1 0	0 1 0

Table 5.

Constrained model with k₀ = 1 and x_i ~ N(0.5, 1) for i = 1,..., n. E(Y|x) = 1 + x – 0.5(x – 0.5)⁺

n	Prob	σ = 0.2			σ = 0.1
		BIC2	BIC3	MIC	BIC2	BIC3	MIC
30	$\widehat{P}(\widehat{\kappa }<k_0)$ $\widehat{P}(\widehat{\kappa }=k_0)$ $\widehat{P}(\widehat{\kappa }>k_0)$	0.100 $0.82\stackrel{‒}{3}$ $0.07\stackrel{‒}{6}$	0.290 $0.70\stackrel{‒}{6}$ $0.00\stackrel{‒}{3}$	$0.27\stackrel{‒}{6}$ 0.720 $0.00\stackrel{‒}{3}$	0 0.910 0.090	$0.00\stackrel{‒}{3}$ $0.98\stackrel{‒}{3}$ $0.01\stackrel{‒}{3}$	$0.00\stackrel{‒}{3}$ $0.98\stackrel{‒}{3}$ $0.01\stackrel{‒}{3}$
50	$\widehat{P}(\widehat{\kappa }<k_0)$ $\widehat{P}(\widehat{\kappa }=k_0)$ $\widehat{P}(\widehat{\kappa }>k_0)$	$0.01\stackrel{‒}{6}$ $0.92\stackrel{‒}{6}$ $0.05\stackrel{‒}{6}$	0.080 $0.91\stackrel{‒}{6}$ $0.00\stackrel{‒}{3}$	$0.10\stackrel{‒}{3}$ $0.89\stackrel{‒}{3}$ $0.06\stackrel{‒}{3}$	0 $0.93\stackrel{‒}{6}$ $0.06\stackrel{‒}{3}$	0 $0.98\stackrel{‒}{6}$ $0.01\stackrel{‒}{3}$	0 $0.99\stackrel{‒}{3}$ $0.00\stackrel{‒}{6}$
150	$\widehat{P}(\widehat{\kappa }<k_0)$ $\widehat{P}(\widehat{\kappa }=k_0)$ $\widehat{P}(\widehat{\kappa }>k_0)$	0 $0.96\stackrel{‒}{6}$ $0.03\stackrel{‒}{3}$	0 1 0	0 1 0	0 $0.96\stackrel{‒}{3}$ $0.03\stackrel{‒}{6}$	0 1 0	0 1 0

Table 6.

Constrained model with k₀ = 2 and x_i = i/n for i = 1,..., n. E(Y|x) = 1 + x – 0.5(x – 0.5)⁺ + (x – 0.7)⁺

n	Prob	σ = 0.5			σ = 0.02
		BIC2	BIC3	MIC	BIC2	BIC3	MIC
30	$\widehat{P}(\widehat{\kappa }<k_0)$ $\widehat{P}(\widehat{\kappa }=k_0)$ $\widehat{P}(\widehat{\kappa }>k_0)$	$0.58\stackrel{‒}{6}$ $0.36\stackrel{‒}{3}$ 0.050	0.860 0.140 0	$0.85\stackrel{‒}{3}$ $0.14\stackrel{‒}{6}$ 0	0 0.920 0.080	$0.02\stackrel{‒}{6}$ $0.95\stackrel{‒}{6}$ $0.01\stackrel{‒}{6}$	$0.02\stackrel{‒}{3}$ 0.960 $0.01\stackrel{‒}{6}$
50	$\widehat{P}(\widehat{\kappa }<k_0)$ $\widehat{P}(\widehat{\kappa }=k_0)$ $\widehat{P}(\widehat{\kappa }>k_0)$	$0.50\stackrel{‒}{3}$ $0.43\stackrel{‒}{3}$ $0.06\stackrel{‒}{3}$	$0.81\stackrel{‒}{6}$ $0.18\stackrel{‒}{3}$ 0	0.890 0.110 0	0 $0.89\stackrel{‒}{3}$ $0.10\stackrel{‒}{6}$	$0.01\stackrel{‒}{3}$ $0.96\stackrel{‒}{3}$ $0.02\stackrel{‒}{3}$	0.020 $0.96\stackrel{‒}{6}$ $0.01\stackrel{‒}{3}$
150	$\widehat{P}(\widehat{\kappa }<k_0)$ $\widehat{P}(\widehat{\kappa }=k_0)$ $\widehat{P}(\widehat{\kappa }>k_0)$	$0.09\stackrel{‒}{3}$ $0.86\stackrel{‒}{3}$ $0.04\stackrel{‒}{3}$	0.280 $0.71\stackrel{‒}{3}$ $0.00\stackrel{‒}{6}$	0.530 0.470 0	0 $0.93\stackrel{‒}{3}$ $0.06\stackrel{‒}{6}$	0 $0.99\stackrel{‒}{6}$ $0.00\stackrel{‒}{3}$	0 1 0

The results summarized in Tables 4, 5, and 6 empirically support that the under-fitting probabilities of BIC₂, BIC₃, and MIC approaching zero as n increases. The over-fitting probabilities of BIC₃ and MIC seem to converge to zero as the sample size increases and/or the effect size gets larger. Similarly as in the unconstrained case, the traditional BIC for the constrained case, BIC₂, tends to overestimate the true number of change-points (κ), especially when the effect size is large, and the probability of overestimating κ is significantly different from zero even for n = 150 or 200 in most of cases, although it seems to decrease as n increases. Via numerical study not reported here, we also observed that the residual sum of squares of the constrained model fit change usually at a slower speed than those of the unconstrained model fit do as the number of change-points increases, and thus we expect the consistency of BIC_d to be achieved in the constrained model case with a less severe penalty than that of BIC₄ whose consistency was proved in the unconstrained model case.

5 Discussion

In this paper, we considered information based selection criteria to estimate the number of change-points in segmented line regression, and proved the consistency of the number of change-points estimated by BIC₄ for the unconstrained model with normal errors. The simulation results summarized in Tables 1, 2, and 3 for the unconstrained model support the theoretical results that (i) the under-fitting probability of BIC_d converges to zero for any d > 0 (Theorem 1) and (ii) the over-fitting probability of BIC₄ converges to zero (Theorem 2), while the traditional BIC, BIC₃, indicates a non-negligible over-fitting tendency compared to those of BIC₄ and MIC.

For the constrained case where the line segments are assumed to be continuous at the change-points, we examined empirical properties of BIC₃ that is expected to improve large sample performance of the traditional BIC, BIC₂. The fact that the under-fitting probability of BIC_d converges to zero for any d > 0, is empirically justified in Tables 4, 5, and 6. Significantly higher over-fitting probabilities of the traditional BIC, BIC₂, is also observed even for large n, compared to those of BIC₃ and MIC.

In general, a relatively large penalty, that is, large d, will reduce the probability of over-estimating the number of change-points, but it will raise a risk of under-estimation. In that regards, our interest in this paper was on the smallest d for which BIC_d produces a consistent estimator $\widehat{\kappa }$. In the normally distributed sequence of random variables, the traditional BIC, BIC₂, produces a consistent estimator of κ according to Yao (1988), although its over-estimating tendency is observed even for n = 497 in Table 1 of Zhang and Siegmund (2007). Although the consistency of $\widehat{\kappa }$ in the unconstrained case was proved only for d ≥ 4 in our paper, BIC_d with d < 4 may provide a consistent model selection, and we hope future research to improve our results in this paper.

Another related future research problem is to identify a selection procedure that selects the best model for various sample sizes and effect sizes. In our simulation study, we observed that the traditional BIC performs better when the effect size is small, and in our future research, we plan to investigate small sample properties of various selection criteria. Any one type of criterion may not provide a uniform optimality, and a revised selection method such as a weighted BIC, ${\sum }_d{w}_d{\mathrm{BIC}}_d$, where the weight depends on effect sizes, sample size, etc., may achieve a relatively high probability of correct selection.

BIC₄/BIC₃ studied in this paper for the unconstrained/constrained model cases, respectively, were motivated from the modified BIC of Zhang and Siegmund (2007) that are derived as an asymptotic approximation of the Bayes factor. The original form of the Bayes factor could be used as a selection measure, instead of its asymptotic approximation. However, we have observed in our preliminary simulation study that even the performance of MBIC derived for the segmented line regression models depend on the x-configurations, and thus it may not be straightforward to study analytic properties of the criterion using the original Bayes factor.

Highlights.

• Segmented line regression models, constrained and unconstrained, are studied.

• Consistency of the estimated number of change-points is investigated.

• Simulation studies are conducted to assess the performance of the proposed criteria.

Appendix: Proofs of Lemmas

Lemma A.1

For any δ > 0,

$$P\left(\underset{\alpha <\eta }{\sup }{∊}^{\prime }H(\alpha ,\eta )∊>4{\sigma }_0^2(1+\delta )\log n\right)\to 0$$

as n → ∞.

Proof

Let X be a χ² distributed random variable with k degrees of freedom. For some constant c_k > 0,

$$P(X>x)=c_k{x}^{\frac{k}{2}-1}{e}^{-\frac{x}{2}}+o\left(x^{\frac{x}{2}-1}{e}^{-\frac{x}{2}}\right)$$

as x → ∞. For given α < η, $\frac{{∊}^{\prime }H(\alpha ,\eta )∊}{{\sigma }_0^2}$ has a χ² distribution with 2 degrees of freedom. So, it is true that for some constant c₀ > 0 and for δ > 0,

$$\begin{array}{cc}\hfill & P\left(\underset{\alpha <\eta }{\sup }{∊}^{\prime }H(\alpha ,\eta )∊>4{\sigma }_0^2(1+\delta )\log n\right)\hfill \\ \hfill =& P\left(\underset{x_i<x_j}{\max }{∊}^{\prime }H(x_i,x_j)∊>4{\sigma }_0^2(1+\delta )\log n\right)\hfill \\ \hfill \le & c_0\frac{n(n-1)}{2}{n}^{-2-2\delta }\hfill \\ \hfill \to & 0\hfill \end{array}$$

as n gets large.

Lemma A.2

For 1 ≤ j ≤ n – 2,

$$\underset{2\le k\le n-j}{\max }{∊}^{\prime }H(x_j,x_{j+k})∊=O_p\left(\log \log n\right),$$

and for 3 ≤ j ≤ n,

$$\underset{2\le k\le j-1}{\max }{∊}^{\prime }H(x_{j-k},x_j)∊=O_p\left(\log \log n\right).$$

Proof

For 2 ≤ k ≤ n – j, the rank of H(x_j, x_j+k) is 2, and so there exists 2 × n matrix Q(k) with full row rank 2, such that

$${∊}^{\prime }H(x_j,x_{j+k})∊={∊}^{\prime }Q{\left(k\right)}^{\prime }Q\left(k\right)∊=\sum _{l=1}^{2}u{\left(k\right)}_l^2,$$

where Q(k)′ = (q(k)₁, q(k)₂) and $u{\left(k\right)}_l=\mathit{q}{\left(k\right)}_l^{\prime }∊$, l = 1, 2. Then ${\sum }_{l=1}^2\mathit{q}{\left(k\right)}_l^{\prime }\mathit{q}{\left(k\right)}_l=tr\left(H(x_j,x_{j+k})\right)=2$. For each k and integer i, let $a_{lki}=\sqrt{k}q{\left(k\right)}_{l,i}$, so that

$${∊}^{\prime }H(x_j,x_{j+k})∊=\frac{S_{1k}^2+S_{2k}^2}{k},$$

where $S_{lk}={\sum }_{i=j+1}^{j+k}{a}_{lki}{∊}_i$. Then by Theorem 1 of Lai and Wei(1982),

$$\underset{2\le k\le n-j}{\max }{∊}^{\prime }H(x_j,x_{j+k})∊=O_p\left(\log \log n\right).$$

Similarly, we can show that

$$\underset{2\le k\le j-1}{\max }{∊}^{\prime }H(x_{j-k},x_j)∊=O_p\left(\log \log n\right).$$

Proof of Lemma 1

It is good enough to consider the case of k = κ₀ + 1. For any t_k ∈ B_r(n), let ξ_k+κ0+1 be an ordered set of $\{{\mathit{t}}_k,{\tau }_1^0,\dots ,{\tau }_{r-1}^0,{\tau }_r^0-{\delta }_n,{\tau }_r^0+{\delta }_n,{\tau }_{r+1}^0,\dots ,{\tau }_{{\kappa }_0}^0\}$. Then

$$S_n\left({\mathit{t}}_k\right)-S_n({\tau }^0,1)\ge S_n\left({\xi }_{k+{\kappa }_0+1}\right)-S_n\left({\tau }^0\right)\ge S_n\left({\xi }_{k+{\kappa }_0+1}\right)-{∊}^{\prime }∊.$$

For $({\xi }_{l-1},{\xi }_l]\not\subset ({\tau }_r^0-{\delta }_n,{\tau }_r^0+{\delta }_n]$,

$$S_n({\xi }_{l-1},{\xi }_l)={∊}^{\prime }(I({\xi }_{l-1},{\xi }_l)-H({\xi }_{l-1},{\xi }_l))∊=\sum _{i\in \{i:x_i\in ({\xi }_{l-1},{\xi }_l]\}}{∊}_i^2-O_p\left(\log n\right)$$

by Lemma A.1.

For 1 ≤ r ≤ κ₀. let $A_r\left(n\right)=\{i:{\tau }_r^0-{\delta }_n<x_i\le {\tau }_r^0+{\delta }_n\}$. Then

$$\begin{array}{cc}\hfill S_n({\tau }_r^0-{\delta }_n,{\tau }_r^0+{\delta }_n)=& \sum _{i\in A_r\left(n\right)}{(E\left(y_i\right)+{∊}_i-{\widehat{y}}_i)}^2\hfill \\ \hfill =& \sum _{i\in A_r\left(n\right)}\{{∊}_i^2+{(E\left(y_i\right)-{\widehat{y}}_i)}^2+2(E\left(y_i\right)-{\widehat{y}}_i){∊}_i\}\hfill \end{array}$$

Since E(y) is discontinuous at ${\tau }_r^0$ while ŷ is continuous on (${\tau }_r^0-{\delta }_n$, ${\tau }_r^0+{\delta }_n$), for some constant c₀,

$${(E\left(y_i\right)-{\widehat{y}}_i)}^2>c_0$$

for $i\in I_r\subset A_r\left(n\right)$ for which #(I_r) is of order at least nδ_n. So by the weak law of large numbers, there exists c₁ > 0 such that

$$S_n({\tau }_r^0-{\delta }_n,{\tau }_r^0+{\delta }_n)-\sum _{i\in A_r\left(n\right)}{∊}_i^2>c_1{\delta }_{n}n=c_1{\left(\log n\right)}^2.$$

Then

$$S_n\left({\xi }_{k+{\kappa }_0+1}\right)-{∊}^{\prime }∊>-O_p\left(\log n\right)+c_1{\left(\log n\right)}^2>0$$

for large n.

Proof of Lemma 2

The number of observations in subintervals (${\tau }_r^0-{\delta }_n,{\tau }_r^0$) and (${\tau }_r^0,{\tau }_r^0+{\delta }_n$) is of order (log n)² by Assumption 1 or 1’. Then, by Lemma A.1,

$$G_{n,1}=O_p\left(\log \left({\left(\log n\right)}^2\right)\right)=O_p\left(\log \log n\right).$$

Proof of Lemma 3

For ${\mathit{t}}_k=(t_1,\dots ,t_k)\in {\cap }_{r=1}^{{\kappa }_0}{B}_r{\left(n\right)}^c$,

$$\#\{t_j:t_j\in {\cup }_{r=0}^{{\kappa }_0}{D}_r\}\le k-{\kappa }_0.$$

Suppose that k – κ₀ such points, ${\xi }_1^{\prime },\dots ,{\xi }_{k-{\kappa }_0}^{\prime }$, are in D_r for some r. Then

$$G_{n,2}={∊}^{\prime }H(0,{\tau }_1^0-{\delta }_n)∊+{∊}^{\prime }H({\tau }_{{\kappa }_0}^0+{\delta }_n,1)∊+\sum _{j\ne r}{∊}^{\prime }H({\tau }_j^0+{\delta }_n,{\tau }_{j+1}^0-{\delta }_n)∊+\sum _{({\xi }_{l-1},{\xi }_l]\subset D_r}{∊}^{\prime }H({\xi }_{l-1},{\xi }_l)∊.$$

Since ε′H(α, η)ε for given (α, η) is O_p(1), the first three terms of the right-hand side are O_p(1). So,

$$G_{n,2}=O_p\left(1\right)+{∊}^{\prime }H({\tau }_r^0+{\delta }_n,{\xi }_1^{\prime })∊+{∊}^{\prime }H({\xi }_{k-{\kappa }_0}^{\prime },{\tau }_{r+1}^0-{\delta }_n)∊+\sum _{j=1}^{k-{\kappa }_0-1}{∊}^{\prime }H({\xi }_j^{\prime },{\xi }_{j+1}^{\prime })∊.$$

By Lemma A.2, the second and the third terms of the right-hand side of the above equation are of order log log n. Then, by Lemma A.1, it is true that for any δ > 0,

$$\sum _{j=1}^{k-{\kappa }_0-1}{∊}^{\prime }H({\xi }_j^{\prime },{\xi }_{j+1}^{\prime })∊\le (k-{\kappa }_0-1)4(1+\delta ){\sigma }_0^2\log n,$$

with probability going to 1 as n → ∞. If the points ${\xi }_1^{\prime },\dots ,{\xi }_{k-{\kappa }_0}^{\prime }$ are located in distinct D_r's, we can get an even smaller upper bound.