Moving Block Bootstrap for Analyzing Longitudinal Data

Abstract

In a longitudinal study subjects are followed over time. I focus on a case where the number of replications over time is large relative to the number of subjects in the study. I investigate the use of moving block bootstrap methods for analyzing such data. Asymptotic properties of the bootstrap methods in this setting are derived. The effectiveness of these resampling methods is also demonstrated through a simulation study.

1. Introduction

Many longitudinal designs are the case that the number of subjects n is large and the number of replications m_i is bounded. Liang and Zeger (1986) proved that the generalized estimating equation (GEE) estimator is consistent and asymptotically normal with misspecification of covariance parameters. That asymptotic property is considered when the number of subjects n goes to infinity and m_i is bounded. Xie and Yang (2003) proved the almost sure existence and strong consistency of GEE estimators.

Alternatively in this article, I will consider the case of the longitudinal design in which the number of subjects n is bounded and the number of replications m, the same number for all subjects, is large. The model on which we focus is given by

$$y_i=x_i\beta +e_i,$$ (1)

where i = 1, n, …, x_i is m × p design matrix, β is p × 1 vector of unknown parameters, y_i = (y_i1, …, y_im)′, and e_i = (e_i1, …, e_im)′.

I focus on the following estimating equation. The estimator β̂ is called a regression estimator if it solves

$$\sum _{i=1}^{n_0}{x}_i^{\prime }{V}_i^{-1}(y_i-x_i\beta )=0.$$ (2)

The repeated observations are correlated for each subject. Since bootstrap methods that resample small subjects or resample observations independently may not work well, we will investigate the moving block bootstrap method developed for time-series-correlated data.

The remaining part of this paper is organized as follows. In Sec. 2, we present some literature review. In Sec. 3, we explore the use of moving block bootstrap for analyzing longitudinal data. In Sec. 4, we demonstrate the moving block bootstrap justification in longitudinal data theoretically and empirically. In Sec. 5, we conclude the paper with a brief discussion.

2. Literature Review

Efron (1979) introduced the bootstrap procedure for estimating sampling distributions of statistics based on independent and identically distributed (i.i.d. observations. It is well known, in the i.i.d. setup, that the bootstrap often offers more accurate approximations than classical large sample approximations, e.g., Singh (1981) and Babu (1986). However, when the observations are not necessarily independent, the classical bootstrap no longer succeeds, as shown by Singh (1981).

In a time series case, Lahiri (1996) applied the moving block bootstrap method to multiple linear regression models

$$y_j=x_j^{\prime }\beta +{∊}_j,j=1,\dots ,m,$$ (3)

Where x_j's are known p × 1 vectors, β is a p × 1 vector of parameters, and ∊₁, ∊₂, …, ∊_m are stationary, strongly mixing random variables. If β̂_m is an M-estimator of β corresponding to some score function ϕ, under some conditions, a two-term Edgeworth expansion for studentized multivariate M-estimator was observed. Also, Lahiri (1996) showed that the block bootstrap has a second-order correctness for some suitable bootstrap analogs of studentized β̂_m.

Hall et al. (1995) showed that the optimal asymptotic rate of the block size for the moving blocks method depends significantly on context, being equal to m^1/3, m^1/4, and m^1/5 in the cases of variance or bias estimation, estimation of a one-sided distribution function, and estimation of a two-sided distribution function, respectively. The latter two quantities are needed for the construction of equal-tailed and symmetric confidence intervals, respectively. Hall et al. (1995) present a practical rule for selecting the block size empirically. It is based on the fact that the asymptotic formula is b ∼ Cm^1/k, where k = 3, 4, or 5 is known, and C is a constant that depends on the underlying process. The rule suggested provides a way for estimating the optimal block for a time series of smaller length than the original.

Paparoditis and Politis (2002) presented a new block bootstrap variation, the tapered block bootstrap, which is applicable in the general time series case of approximately linear statistics. The asymptotic validity and the favorable bias properties of the tapered block bootstrap are shown in two important cases: smooth function of means and M-estimators.

3. Moving Block Bootstrap for Longitudinal Data

The major drawback with model-based resampling is that in practice not only the parameters of a model, but also its structure, must be identified from the data. If the chosen structure is incorrect, the resampled series will be generated for the wrong model, and they will not have the same statistical properties as the original data. The model-based approach is inconsistent if the model used for resampling is misspecified.

The moving bootstrap involves resampling possibly overlapping blocks. The mixed block bootstrep (MBB) does not force one to select a model and the only parameter required is the block length. If the block is long enough the original dependence will be reasonably preserved in the resampled series. This approximation is better if the dependence is weak and the blocks are as long as possible, thus preserving the dependence more faithfully. But the distinct values of the statistics must be as numerous as possible to provide a good estimate of the distribution of the statistics and this points toward short blocks.

Unless the length of the series is considerable to accommodate longer and more number of blocks, the preservation of the dependence structure may be difficult, especially for complex, long-range dependent structures. In such cases, the block resampling scheme tends to generate resampled series that are less dependent than the original ones. Furthermore, the resampled series often exhibits artifacts that are caused by joining randomly selected blocks. Then, the asymptotic variance–covariance matrices of the estimators based on the original series and those based on the bootstrap series are different and a modification of the original scheme is needed. This suggests a strategy intermediate between model-based and block resampling. The idea comes from a procedure of pre-whitening the original dependent series by fitting a model and is intended to remove much of the dependence between the original observations. A resampling series is generated by block resampling of residuals from the simple fitted model, and the innovation series is then post-blackened by applying the simple estimated model to the resampled innovations. The post-blackened version works more consistently in practice (Davison and Hinkley, 1997).

Bühlmann (1997) suggested the sieve bootstrap that is model based. The autoregressive (AR)(p) model is just used to filter the residual series. If the model used in the sieve bootstrap is not appropriate, the resulting residuals cannot be treated as i.i.d. A hybrid approach between the model based method and moving block bootstrap, named post-blacken bootstrap, was suggested by Davison and Hinkley (1997). The procedure is similar to the sieve bootstrap, but the residuals from AR(p) model are not resampled in an iid manner but by using the MBB bootstrap. If some residual dependent structure is present in the AR residuals, this is kept from the blockwise bootstrap. The linear model is used to pre-whiten the series by fitting the model that is intended to remove much of the dependence present the observations. A series of innovations is then generated by block resampling of residuals obtained from the fitted model, and the innovation series is then post-blackened by applying the estimated model to the resampled innovations.

A block bootstrap algorithm in a longitudinal model is given as follows. We continue to assume (1) as our longitudinal model under consideration.

1. Let ê_ij, i = 1, …, n₀, j = 1, …, m be the residuals from the model fit.

   $${\widehat{e}}_{ij}=y_{ij}-x_{ij}\widehat{\beta },$$ (4)

   where β̂ is the ordinary least square estimate.

2. Now assuming that m = bk with b and k integers: Let $B_1^{\ast },\dots ,B_k^{\ast }$ denotes k uniform draws with replacement from the integers {0, 1, …, m−b}. These represent the starting point for each block of length b. A block bootstrap resample of residuals, $({\widehat{e}}_{i1}^{\ast },\dots ,{\widehat{e}}_{im}^{\ast })$, is defined by:

   $${\widehat{e}}_{i,(j-1)b+s}^{\ast }={\widehat{e}}_{i,B_j^{\ast }+s},(1\le j\le k,1\le s\le b)\text{for each}i.$$ (5)

3. The bootstrapped response, $y_{ij}^{\ast }$, are then generated from the estimated model with residuals ${\widehat{e}}_{ij}^{\ast }$ and the original covariates:

   $$y_{ij}^{\ast }=x_{ij}\widehat{\beta }+{\widehat{e}}_{ij}^{\ast }.$$ (6)

4. From the resampled responses, $y_{ij}^{\ast }$, and original covariates, we fit the model and obtain new parameter estimates.

5. Repeating steps (2) through (4) a large number, R, of times one obtains R bootstrap replicates from which features of the distribution of the parameter estimates can be estimated. In particular, the bootstrap variance estimates are simply variance of the B computed values for each parameter.

I consider the six different kinds of block bootstrap methods in a balanced longitudinal design in which the number of subjects is small and the number of replications is large:

• Case 1, MBB1 (Within block bootstrap): For each i subject, we construct overlapping blocks with m − b + 1 blocks and block size b, i.e B₁, …, B_m−b+1. Let us define m/b = k which is assumed to be an integer for simplicity, in general k = [m/b]. We can add the k blocks with replacement among B₁, …, B_m−b+1. We get the $B_1^{\ast },\dots ,B_k^{\ast }$ with kb = m, and create ${\widehat{e}}_{i1}^{\ast },\dots ,{\widehat{e}}_{im}^{\ast }$ from ê_i1, …, ê_im, where ê_ij = y_ij − β̂₀ − β̂₁x_ij. We can add up to n₀ individuals and plug this into the model and the results is a pseudo sample series $y_{11}^{\ast },\dots ,y_{n_{0}m}^{\ast }$. From the model $y_{ij}^{\ast }={\widehat{\beta }}_0+{\widehat{\beta }}_1{x}_{ij}+{\widehat{e}}_{ij}^{\ast }$, we fit model and produce new parameters ${\widehat{\beta }}_0^{\ast }$ and ${\widehat{\beta }}_1^{\ast }$.

• Case 2, MBB2 (Mixed block bootstrap): We have m − b + 1 blocks and block size b, i.e B₁, …, B_m−b+1 and add up to n₀ subjects. We sample n₀k blocks with replacements among B₁, …, B_n0(m−b+1). We construct $B_1^{\ast },\dots ,B_{n_{0}k}^{\ast }$ with kb = m, and plug this into the model and obtain a pseudo series $y_{11}^{\ast },\dots ,y_{n_{0}m}^{\ast }$. Similarly, from the model $y_{ij}^{\ast }={\widehat{\beta }}_0+{\widehat{\beta }}_1{x}_{ij}+{\widehat{e}}_{ij}^{\ast }$, we fit the model and produce new parameters ${\widehat{\beta }}_0^{\ast }$ and ${\widehat{\beta }}_1^{\ast }$.

• Case 3, One-line moving block bootstrap: One can make up to one long series and perform the moving block bootstrap using a time series without splitting the different individual consecutive data.

  Case 4, Standard bootstrap: This is a special case of b = 1 in MBB2.

• Case 5, Resampling subject bootstrap: This is a special case of b = m in MBB2.

• Case 6, Stratified standard bootstrap: This is a special case of b = 1 in MBB1.

4. Justification of Moving Block Bootstrap in Longitudinal Data

I consider the justification of moving block bootstrap in longitudinal data. We focus on the within block bootstrap method (MBB1) in the six different kinds of scenario in previous section. I follow the Lahiri's (1996) assumptions. Let's consider the relationship between the GEE and M-estimators. The robust approach can be extended to the regression setup to analyze a predictor−outcome relationship. Suppose we have model (1) with n = n₀. The estimator β̂ is called a robust regression estimator or an M-estimator if it solves

$$\sum _{i=1}^{n_0}\sum _{j=1}^m{x}_{ij}^{\prime }\psi (y_{ij}-x_{ij}\beta )=0,$$ (7)

for some choice of function ψ(·).

4.1 Expansion for M-estimator

It is known that

$${\left(\sum _{i=1}^{n_0}\sum _{j=1}^m{x}_{ij}^{\prime }{x}_{ij}\right)}^{1/2}({\widehat{\beta }}_{n_{0}m}-\beta )\sim N_p(0,\frac{\mathrm{E}{\psi }^2(e_{11})}{{(\mathrm{E}\psi \prime (e_{11}))}^2}{I}_p),$$ (8)

where I_p denotes the identity matrix of order p.

Let ê_ij = y_ij − x_ijβ̂_n0m denote residuals. Define

$${\sigma }_i(k)=\mathrm{E}\psi (e_{i1})\psi (e_{i(1+k)}),k\ge 0;\tau ={\tau }_i=\mathrm{E}\psi \prime (e_{i1})\text{for each}i{\widehat{\sigma }}_{im}(k)={(m-k)}^{-1}\sum _{j=1}^{m-k}\psi ({\widehat{e}}_{ij})\psi ({\widehat{e}}_{i(j+k)}),0\le k\le m-1,{\widehat{\tau }}_{im}=m^{-1}\sum _{j=1}^m\psi \prime ({\widehat{e}}_{ij}).$$

Also, let σ_i (k) = σ(k) and σ̂_im(k) = σ̂_m(k)

Assumption 1.

• (A.1)

  1. ψ is twice differentiable, and ψ″ satisfies a Lipschitz condition of order δ₁ > 0,
  2. ψ, ψ′, ψ″ are bounded.
• (A.2)

  1. for each i Eψ(e_i1) = 0, τ ≡ Eψ′ (e_i1) ≠ 0,
  2. ${\sigma }_{\infty }\equiv \sigma (0)-2{\sum }_{k=1}^{\infty }|\sigma (k)|>0$.
• (A.3) There exists ρ > 0 such that

  1. $sup\{|P(A\cap B)-P(A)P(B)|:A\in {\mathcal{F}}_{-\infty }^r$, $B\in {\mathcal{F}}_{r+k}^{\infty },r\ge 1\}\le {\rho }^{-1}exp(-\rho k)$ for all k ≥ 1,
  2. for all r ≥ 1, and all k ≥ ρ⁻¹, there exists a ${\mathcal{F}}_{r-k}^{r+k}$-measurable random variable ẽ_ir,k such that E|e_ir − ẽ_ir,k| ≤ ρ⁻¹ exp(−ρk),
  3. for all r, k, q ≥ ρ⁻¹ and $A\in {\mathcal{F}}_{r-q}^{r+q},\mathrm{E}|P(A|{\mathcal{F}}_j:j\ne r)-P(A|{\mathcal{F}}_j:0<|j-r|\le q+k)|\le {\rho }^{-1}exp(-\rho k)$, and
  4. for all r ≥ ρ⁻¹, k ≤ r and all t_r−k, …, t_r+k ∈ ℝ with |t_r| > ρ, $\mathrm{E}|\mathrm{E}(exp(\sqrt{-1}{\sum }_{i=1}^{n_0}{\sum }_{j=r-k}^{r+k}{t}_j\psi (e_{ij}))|{\mathcal{F}}_j:j\ne r|<exp(-\rho )$.

• (A.4) max{‖x_ij‖ : 1 ≤ j ≤ m} = O(1) and lim inf_m→∞m⁻¹λ_m ≡ λ > 0, where λ_m denotes the smallest eigenvalue of $({\sum }_{j=1}^m{x}_{ij}^{\prime }{x}_{ij})$.

Let $D_{n_{0}m}={({\sum }_{i=1}^{n_0}{\sum }_{j=1}^m{x}_{ij}^{\prime }{x}_{ij})}^{-1/2}$ and $d_{ij}=D_{n_{0}m}{x}_{ij}^{\prime }$, 1 ≤ j ≤ m. When e_ij are weakly dependent for each i, the asymptotic covariance of $D_{n_{0}m}^{-1}({\widehat{\beta }}_{n_{0}m}-\beta )$ matrix is given by

$${\mathit{Cov}}_{n_{0}m}\equiv {(\mathrm{E}\psi \prime (e_{11}))}^{-2}\times \sum _{i=1}^{n_0}\sum _{k=0}^m{L}_{ikm}\mathrm{E}\psi (e_{i1})\psi (e_{i(1+k)}),$$ (9)

where L_i0m = I_p and $L_{ikm}={\sum }_{j=1}^{m-k}(d_{ij}{d}_{i(j+k)}^{\prime }+d_{i(j+k)}{d}_{ij}^{\prime })$, 1 ≤ k ≤ m − 1.

To define the studentized version of β̂_n0m, note that the asymptotic matrix $D_{n_{0}m}^{-1}({\widehat{\beta }}_{n_{0}m}-\beta )$ is given by

$${\sum }_{n_{0}m}\equiv \mathit{Cov}(\sum _{i=1}^{n_0}\sum _{j=1}^m{d}_{ij}\psi (e_{ij}))=\sum _{i=1}^{n_0}\sum _{k=0}^{m-1}{L}_{ikm}\sigma (k).$$ (10)

Therefore, a natural estimator of Σ_n0m is

$${\widehat{\sum }}_{n_{o}m}=\sum _{i=1}^{n_0}\sum _{k=0}^l{L}_{ikm}{\widehat{\sigma }}_m(k),$$ (11)

where 1 ≤ l ≡ l_m ≤ m − 1 is an integer. If l → ∞ slowly with m, then ‖Σ̂_n0m − Σ_n0m‖ = o_p(1). Σ̂_n0m is non singular with high probability for m large, and can be inverted to define the studentized statistic,

$$T_{n_{0}m}={\widehat{\sum }}_{n_{0}m}^{-1/2}{D}_{n_{0}m}^{-1}({\widehat{\beta }}_{n_{0}m}-\beta ).$$ (12)

Next, I extend Lahiri's (1996) results for longitudinal case: assume that (A.1),(A.2),(A.3)(i),(ii), and (A.4) hold. Then, there exists a sequence of statistics {β̂_m} such that

$$P({\widehat{\beta }}_{n_{0}m}\text{satisfies}(7)\text{and}{\Vert D_{n_{0}m}^{-1}({\widehat{\beta }}_{n_{0}m}-\beta )\Vert }^2\le Clogm)=1-o(m^{-1/2}).$$ (13)

If we have a unique solution β̂_n0m, then $\Vert D_{n_{0}m}^{-1}({\widehat{\beta }}_{n_{0}m}-\beta )\Vert =O_P({(logm)}^{1/2})$. When (7) has a unique solution, one can obtain the strong consistency of β̂_n0m as in Lahiri (1992). The next result gives a first-order Edgeworth expansion for the studentized M-estimator.

Theorem 1. Assume that Assumptions (A.1)–(A.4) hold and that {β̂_n0m} is a sequence of measurable solutions of (7). Then, there exist's a polynomial p_m(·) on ℝ^p such that

$$\underset{B\in \mathcal{B}}{sup}|P(T_{n_{0}m}\in B)-{\int }_B(1+p_{n_{0}m}(x))d\mathrm{\Phi }(x)|=o(m^{-1/2})$$ (14)

for every class ℬ of Borel subsets of ℝ^p satisfying

$$\underset{B\varepsilon \mathcal{B}}{sup}\mathrm{\Phi }({(\partial B)}^{∊})=O(∊)\mathit{\text{as}}\eta \downarrow 0.$$ (15)

Here ‖p_n0mϕ‖_∞ = O(m^−1/2) with sup norm ‖‖_∞, Φ denotes the standard normal distribution onℝ^p(p ≥ 1), and the coefficient of p_n0m(·) are continuous functions of cross-product moments of ψ(e_ij), ψ′ (e_ij), and ψ″ (e_ij). Here ∂B denote's the boundary of a set B ⊆ ℝ^p and (∂B)^∊ = {x : ‖x − y‖ < ∊ for some y ∈ ∂B}.

4.2. Expansion for Bootstrap M-estimator

Define the bootstrap M-estimator ${\widehat{\beta }}_{n_{o}m}^{\ast }$ as a solution of the equation in t ∈ ℝ^p

$$\sum _{i=1}^{n_0}(\sum _{j=1}^m{x}_{ij}^{\prime }(\psi (y_{ij}^{\ast }-x_{ij}t))-{\widehat{\mu }}_m)=0,$$ (16)

where ${\widehat{\mu }}_m=\frac{1}{b}{\mathrm{E}}_{n_{0}m}^{\ast }\{\psi ({\widehat{e}}_{11}^{\ast })+\dots +\psi ({\widehat{e}}_{1b}^{\ast })\}$ and $y_{ij}^{\ast }$ is given in (6). The ${\sum }_{n_{0}m}^{\ast }$ is the conditional covariance matrix of ${\sum }_{i=1}^{n_0}{\sum }_{j=1}^m{d}_{ij}\psi ({\widehat{e}}_{ij}^{\ast })$ which is given by

$$\sum _{i=1}^{n_0}\sum _{u=1}^k{\mathit{Cov}}_m(\sum _{j=1}^b{d}_{i,(u-1)b+j}\psi ({\widehat{e}}_{ij}^{\ast })).$$ (17)

The natural estimator of ${\sum }_{n_{0}m}^{\ast }$ is

$${\widehat{\sum }}_{n_{0}m}^{\ast }=\sum _{i=1}^{n_0}\sum _{j=0}^{b-1}\sum _{u=1}^k\sum _{l=1}^{b-j}{D}_{i,luj}^{\ast }{\widehat{\sigma }}_{n_{0}m}^{\ast }(j),$$ (18)

where $D_{luj}^{\ast }=(1-2^{-1}I(j=0))({\stackrel{\sim }{D}}_{luj}^{\ast }+{\stackrel{\sim }{D}}_{luj}^{{\ast }^{\prime }})$, ${\stackrel{\sim }{D}}_{luj}^{\ast }=d_{(u-1)b+l}{d}_{(u-1)b+l+j}^{\prime }$. The bootstrap version $T_{n_{0}m}^{\ast }$ of T_n0m is given by

$$T_{n_{0}m}^{\ast }={({\widehat{\sum }}_{n_{0}m}^{\ast })}^{-1/2}{D}_{n_{0}m}^{-1}({\widehat{\beta }}_{n_{0}m}^{\ast }-{\widehat{\beta }}_{n_{0}m}).$$ (19)

By assumptions, there exists a sequence of statistics $\{{\widehat{\beta }}_{n_{0}m}^{\ast }\}$ such that

$$P({\widehat{\beta }}_{n_{0}m}^{\ast }\text{satisfies}(16)\text{and}{\Vert D_{n_{0}m}^{-1}({\widehat{\beta }}_{n_{0}m}^{\ast }-{\widehat{\beta }}_{n_{0}m})\Vert }^2\le Clogm)=1-o_p(m^{-1/2}).$$ (20)

Theorem 2. Assume that the conditions in Theorem 1. hold. Suppose that $T_{n_{0}m}^{\ast }$ is defined for some measurable sequence $\{{\widehat{\beta }}_{n_{0}m}^{\ast }\}$ satisfying (20) and also suppose that m^δb⁻¹ = O(1) and b = O(m^(1−κ)/4) for some δ > 0, and κ > max{p + 3, 5}δ₀. Then

$$\underset{B\in \mathcal{B}}{sup}|P^{\ast }(T_{n_{0}m}^{\ast }\in B)-P(T_{n_{0}m}\in B)|=o_p(m^{-1/2})$$ (21)

for any class ℬ of Borel subset of ℝ^p.

Theorem 2 shows that the MBB indeed provides more accurate approximation for studentized multivariate M-estimator of the regression parameter vector β than normal approximation. Consequently, Theorem 2 is useful for constructing second-order correct multivariate inference procedures for β under multiple regression model. The studentized moving block bootstrap statistics obtain the second-order accuracy for the bounded n = n₀ case.

4.3. A Simulation Study

The block bootstrap captures the dependence in the series of residuals without the need to know the correlation structure. It can be simple and account for correlations in a regression model with correlated error. To obtain the bootstrap version of β̂_n0m, first form the observed blocks of residual length b as ξ_ih = (ê_ij, …, ê_i(h+b−1)), 1 ≤ h ≤ q, where q = m − b + 1 and ê_ij = y_ij − x_ijβ̂_n0m, 1 ≤ j ≤ m, 1 ≤ i ≤ n. Next draw ${\xi }_{i1}^{\ast },\dots ,{\xi }_{ik}^{\ast }$ randomly, with replacements from ξ_i1, …, ξ_iq, where m/b = k is assumed to be an integer for simplicity. Note that each ${\xi }_{ik}^{\ast }$ has b components. Denote the lth component of ${\xi }_{ik}^{\ast }$, 1 ≤ l ≤ b by ${\xi }_{ikl}^{\ast }$. Also, set ${\widehat{e}}_{i((b-1)k+l)}^{\ast }={\xi }_{ikl}^{\ast }$, 1 ≤ l ≤ b, and we have the bootstrap pseudo-observations

$$y_{ij}^{\ast }=x_{ij}{\widehat{\beta }}_{nm}+{\widehat{e}}_{ij}^{\ast },1\le i\le n_0,1\le j\le m.$$ (22)

Adapting Shorack's approach, we obtain the bootstrapped estimator ${\widehat{\beta }}_{n_{0}m}^{\ast }$ as a solution of the equation t ∈ R^p,

$$g_{n_{0}m}^{\prime }=\sum _{i=1}^{n_0}\sum _{j=1}^m{x}_{ij}^{\prime }((y_{ij}^{\ast }-x_{ij}\beta )-{\widehat{\mu }}_{nm})=0,$$ (23)

where ${\widehat{\mu }}_{n_{0}m}=b^{-1}{E}_{n_{0}m}\{e_{11}^{\ast }+\dots +e_{n_{0}b}^{\ast }\}$, and E_n0m denotes the conditional expectations under the MBB resampling scheme, given ê₁₁, …, ê_nm. Centering the above equation by μ̂_n0m makes the estimating equation conditionally unbiased at β = β̂_n0m and ensures the bootstrap analog. The bootstrap estimator is as follows:

$${\widehat{\beta }}_{n_{0}m}^{\ast }={\left(\sum _{i=1}^{n_0}\sum _{j=1}^m{x}_{ij}^{\prime }{x}_{ij}\right)}^{-1}\sum _{i=1}^{n_0}\sum _{j=1}^m{x}_{ij}^{\prime }(y_{ij}^{\ast }-{\widehat{\mu }}_{n_{0}m}).$$ (24)

Consider the following specific model in simulation work:

$$y_{ij}={\beta }_0+{\beta }_1{x}_{ij}+{\gamma }_i+e_{ij},i=1,\dots ,n_0,j=1,\dots ,m,$$ (25)

where e_ij = ϕe_i(j−1) + u_ij,

$${\gamma }_i~N(0,{\sigma }_{\gamma }^2)\text{and}{u}_{ij}~N(0,{\sigma }_u^2).$$

In particular, let n₀ = 5, m = 20, and x_ij = (1,…, 20)′.

$${\beta }_0=10,{\beta }_1=1,b=4,k=5,$$

$${\sigma }_{\gamma }^2=1,{\sigma }_u^2=1\text{and}\varphi =0.75.$$

4.4. Bootstrapping the Distribution of Statistics

Let R be the number of bootstrap simulations (r = 1, …, R), and β̂* be the bootstrap estimate of β for the r samples. The important result is that the distribution of β̂*, estimated by the empirical distribution function of the β̂^*,r(r = 1, …, R), approximates the distribution of β̂. Now the studentized statistic has the following form:

$${\widehat{T}}^{\ast }=\frac{{\widehat{\beta }}^{\ast }-\widehat{\beta }}{{\widehat{S}}_{{\widehat{\beta }}^{\ast }}^{\ast }}.$$ (26)

The difference between the distribution functions of T̂ and T̂* tends to 0, when the number of observations is large; thus we can use the quartiles of T̂* instead of T̂ to construct intervals or tests. Let (T̂^*,r, r = 1, …, R) be the rth sample of T̂*, where T̂* is calculated in the same way as T̂, replacing y_ij with $y_{ij}^{\ast }$. Let q̂_α be the percentile of the T̂^*,r. It can be shown that P(T̂ ≤ q̂_α) tends to α, when m tends to infinity. This gives a bootstrap confidence interval for β

$${\widehat{I}}_R=[{\widehat{\beta }}^{\ast }-{\widehat{q}}_{1-\frac{\alpha }{2}}{\widehat{S}}_{{\widehat{\beta }}^{\ast }}^{\ast },{\widehat{\beta }}^{\ast }-{\widehat{q}}_{\frac{\alpha }{2}}{\widehat{S}}_{{\widehat{\beta }}^{\ast }}^{\ast }].$$ (27)

For large m and R, the coverage probability of Î_R is close to 1 − α. The bootstrap estimation of the variance is calculated using the empirical variance of the R sample (β̂^*,r, r = 1, …, R):

$${\widehat{S}}^{\ast 2}=\frac{1}{R-1}\sum _{r=1}^R{({\widehat{\beta }}^{\ast ,r}-\overline{{\widehat{\beta }}^{\ast }})}^2,$$ (28)

where $\overline{{\widehat{\beta }}^{\ast }}$ is the sample mean $\overline{{\widehat{\beta }}^{\ast }}={\sum }_{r=1}^R{\widehat{\beta }}^{\ast ,r}/R$.

Coverage accuracy, where coverage is the probability that a confidence interval includes β, is the important property for a confidence interval procedure. Bootstrap confidence interval methods differ in their asymptotic properties. Our simulation results are given in Table 1. MBB1 and MBB2 are similar to each other. Those two block bootstrap methods obtained correct coverage probability at the nominal level of 95%. The standard bootstrap or stratified ordinary bootstrap did not perform well in highly correlated longitudinal data with low coverage probabilities.

Table 1.

Coverage probability and length of CI $({\widehat{\beta }}_1^{\ast })$: 500 replications; ϕ = 0.75, SOB is stratified ordinary bootstrap, SB is a standard bootstrap estimation, and β̂₁ is a robust estimation with unknown covariance structure

Methods	CI $({\widehat{\beta }}_1^{\ast })$	Probability	Length
MBB1	(0.849, 1.168)	0.949	0.318
MBB2	(0.844, 1.163)	0.952	0.320
SOB	(0.805, 1.091)	0.768	0.286
SB	(0.804, 1.086)	0.747	0.282
β̂1	(0.840, 1.169)	0.950	0.329

5. Concluding Remarks and Discussion

The moving block bootstrap methods are used for analyzing longitudinal data in which a small number of subjects have a large number of replications over time by investigating the efficacy and utility of the methodology, theoretically and empirically, through a small simulation study. Those have second-order optimality in the case of dependent stationary data, under regular conditions.

Quasi-likelihood approaches such as the GEE of Liang and Zeger (1986) and quadratic inference function (QIF) of Qu et al. (2000) are useful for modeling longitudinal data in the form of large sample short time series. However, their advantages of simplicity and robustness against the misspecification of the correlation structure are offset by their loss of estimation efficiency and lack of procedure for model assessment and selection. In other words, these estimators have consistency properties, but are not fully efficient. To offset this inefficiency, subjects rather than observations can be resampled to obtain an estimation efficiency that takes correlation structure into account.

In a longitudinal study with a small sample and a long time series in which modeling the repeated time pattern is necessary, we assume a stationary process. The stationary process assumption represents a fundamentally different stochastic mechanism from other methods used to govern the structure and behavior of transitions over time. Using a moving block bootstrap estimation procedure is preferable in case of cluster high-correlated data with an equal large number of clusters or in spatial strong-correlated data collected from a large number of locations.

When both the number of subjects and the number of replications are large, the correlation structure is determined by a trade off between the number of nuisance parameters and the closeness of the mathematical model to the true underlying structure. The question is, which provides better estimation efficiency: a simpler correlation with small nuisance parameters or a model closer to the true structure with many nuisance parameters. Further simulations using the bootstrap procedure will be investigated in the future study.

Appendix

Proof of Theorem 1

Proof. We follow Lahiri (1996) notation and definitions. For a smooth function h : ℝ^p → ℝ, let us D_jh denote the partial derivative of h(x) with respect to the jth coordinate of x, 1 ≤ j ≤ p. For p × 1 vectors $v={(v_1,\dots ,v_p)}^{\prime }\in {\mathbb{Z}}_{+}^p$ and w = (w₁,…, w_p)′ ∈ ℝ^p, let |v| = v₁ + … + v_p, $v!=v_1!\mathrm{\dots }{v}_p!,w^v={\mathrm{\Pi }}_{i=1}^p{(w_i)}^{v_i}$, and $\Vert w\Vert ={(w_1^2+\dots +w_p^2)}^{1/2}$. Let D^v denote the differential operator $D_1^{v_1}\dots D_p^{v_p}$, namely, $D^v={\mathrm{\Pi }}_{j=1}^p{(\frac{\partial }{\partial t^{(j)}})}^{v(j)}$. For $v\in {\mathbb{Z}}_{+}^p$ with 1 ≤ |v| ≤ s, let χ_v denote the vth cumulant and μ_v is the vth moment of w. Note that ${\sqrt{-1}}^{|v|}{\mu }_v=D^v\widehat{\mathrm{\Phi }}(0)$, $D^{\alpha }\widehat{\mu }(t)={(\sqrt{-1})}^{|\alpha |}\int w^{\alpha }{e}^{\sqrt{-1}{t}^{\prime }w}\mu (dw)$, t ∈ ℝ^p, and ${\sqrt{-1}}^{|v|}{\chi }_v=(D^{v}log\widehat{\mathrm{\Phi }})(0)$.

We consider w is a ℝ^p -valued random vector with Ew =0 and E‖w‖^s < ∞ for some integer s ≥ 3.

Let m₃ = [log m log log(3 + m)], v_1m = m^−1/2(log m)^1/2, v_m = m^−1/2, v_2m = v_m(log m)⁻¹, and v_3m = v_m(log m)^−3/2. Furthermore, define

$$\begin{array}{ccc}{G}_{n_{0}m}=D_{n_{0}m}^{-1}({\widehat{\beta }}_{n_{0}m}-\beta ),& G_{1n_{0}m}=\sum _{i=1}^{n_0}\sum _{j=1}^m{d}_{ij}\psi (e_{ij})& d_{ij}={\left(\sum _{i=1}^{n_0}\sum _{j=1}^m{x}_{ij}^{\prime }{x}_{ij}\right)}^{-1/2}{x}_{ij},\\ D_{n_{0}m}={\left(\sum _{i=1}^{n_0}\sum _{j=1}^m{x}_{ij}^{\prime }{x}_{ij}\right)}^{-1/2},& \stackrel{\sim }{\psi }=\psi (\cdot )-{\widehat{\mu }}_{n_{0}m},& A_{n_{0}m}=\sum _{i=1}^{n_0}\sum _{j=1}^m{d}_{ij}{d}_{ij}^{\prime }{\psi }^{\prime }(e_{ij}),\end{array}$$

A = τI_p = Eψ′ (e_ij)I_p, τ = τ_i = Eψ′ (e_i1) for each i, w_i1j = ψ (e_i1), W_i2j = ψ′(e_ij) − τ, W_i3j = ψ″(e_ij) − Eψ″(e_i1), and W_i4j(k) = ψ(e_ij)ψ(e_i(j+k)) − σ_i(k). Also, write $\chi (U)={(-1)}^{p/2}{D}_1\dots D_p\mathrm{E}exp(\sqrt{-1}{t}^{\prime }U){|}_{t=0}$ for a random vector U in ℝ^p.

Let $\mathrm{\Delta }=D_{n_{0}m}^{-1}(t-\beta )$, t ∈ ℝ^p. Then, by Taylor's expansion, one can get

$$[\sum _{i=1}^{n_0}\sum _{j=1}^m{d}_{ij}{d}_{ij}^{\prime }{\psi }^{\prime }(e_{ij})]\mathrm{\Delta }=\sum _{i=1}^{n_0}\sum _{j=1}^m{d}_{ij}\psi (e_{ij})+\frac{1}{2}\sum _{i=1}^{n_0}\sum _{j=1}^m{d}_{ij}{(d_{ij}^{\prime }\mathrm{\Delta })}^2{\psi }^{″}(e_{ij})+R_{n_{0}m}(t),$$ (29)

Where $\Vert R_{n_{0}m}(t)\Vert \le C{\sum }_{i=1}^{n_0}{\sum }_{j=1}^m{\Vert d_{ij}\Vert }^{3+{\delta }_1}{\Vert \mathrm{\Delta }\Vert }^{2+\delta }$, t ∈ ℝ^p.

Following Lahiri (1992), we obtain that

$$G_{n_{0}m}=(A^{-1}+{\tau }^{-2}(A-A_{n_{0}m}))G_{1n_{0}m}+{(2{\tau }^3)}^{-1}\sum _{i=1}^{n_0}\sum _{j=1}^m{d}_{ij}{(d_{ij}^{\prime }{\theta }_{1n_{0}m})}^2\mathrm{E}{\psi }^{″}(e_{ij})+R_{n_{0}m}^{\prime },$$ (30)

where $P(\Vert R_{n_{0}m}^{\prime }\Vert >C({\sigma }_{\infty })v_{2m})=o(v_m)$,

$$T_{n_{0}m}={\sum }_{n_{0}m}^{-1/2}[G_{1n_{0}m}+{\tau }^{-1}{G}_{1n_{0}m}(m^{-1}\sum _{j=1}^{n_0}\sum _{j=1}^m{W}_{i2j}-{(m\tau )}^{-1}\sum _{i=1}^{n_0}\sum _{j=1}^m(d_{ij}^{\prime }{G}_{1n_{0}m})\mathrm{E}{\psi }^{″}(e_{i1}))+{\tau }^{-1}(A-A_{n_{0}m})G_{1n_{0}m}+{(2{\tau }^2)}^{-1}\sum _{i=1}^{n_0}\sum _{j=1}^m{d}_{ij}{(d_{ij}^{\prime }{G}_{1n_{0}m})}^2\mathrm{E}{\psi }^{″}(e_{i1})]+\sum _{|\beta |=1}{({\widehat{\sum }}_{n_{0}m})}^{\beta }{D}^{\beta }({\sum }_{n_{0}m}^{-1/2})G_{1n_{0}m}+R_{n_{0}m}^{†}.$$ (31)

If we have T_n0m = T_1n0m + R_n0ms, where R_n0ms is the remainder term that under the moment condition E‖y₁₁‖^s < ∞ satisfies P(‖R_n0ms‖ > δ_{m, s}) = δ_{m, s} for some sequence δ_{m, s} = o(m^{−(s−2)/2)}, then the random variable T_1n0m is called a (s − 2)th -order stochastic approximation to T_n0m. Note that the (s − 2)th order Edgeworth expansions for T_n0m and T_1n0m coincide. The reason for T_1n0m is that the first term is the same as T_n0m, but the remaining terms consist of all independent variables for deriving a simpler expansion. The stochastic approximation T_1n0m can be expressed in the form

$$T_{1n_{0}m}={\sum }_{n_{0}m}^{-1/2}{G}_{1n_{0}m}+\sum _{r=1}^p{G}_{1n_{0}m}^{\prime }{\mathrm{\Lambda }}_{n_{0}rm}{G}_{1n_{0}m}{q}_r+\sum _{r=1}^p{\stackrel{\sim }{W}}_{2n_{0}m}^{\prime }{\mathrm{\Lambda }}_{1n_{0}rm}{G}_{1n_{0}m}{q}_r+\sum _{|v|=1}{({\widehat{\sum }}_{1n_{0}m})}^v{\mathrm{\Lambda }}_{vm}{G}_{1n_{0}m},$$ (32)

where q₁ = (1, 0,…, 0),…, q_p = (0, 0,…, 1) are the standard basis of ℝ^p, ${\stackrel{\sim }{W}}_{2n_{0}m}={({(A-A_{n_{0}m})}^{\prime }:{\sum }_{i=1}^{n_0}{m}^{-1}{\sum }_{j=1}^m{W}_{i2j})}^{\prime }$ and Λ_n0rm, Λ_1n0rm, Λ_vm are non random matrices satisfying max{m^1/2‖Λ_n0rm‖ + ‖Λ_1n0rm‖ + ‖Λ_vm‖ : 1 ≤ r ≤ p, |v| = 1} = o(1). In the following C, C(·) denotes pure constants which depend on each arguments, and the dependance of C(·) on p, α, and the finite moments of ψ (e_ij), ψ′(e_ij), and ψ″(e_ij) will be suppressed for notational simplicity. Using Lahiri's (1992, 1996) arguments, we can show that

$$P(\Vert \sum _{i=1}^{n_0}\sum _{j=1}^m{d}_{ij}\psi (e_{ij})\Vert >C{(logm)}^{1/2})=o(m^{-1/2}),$$ (33)

$$P(\Vert A_{n_{0}m}-A\Vert >C{m}^{-1/4}{(logm)}^{-2})=o(m^{-1/2}),$$ (34)

$$P(|\sum _{i=1}^{n_0}\sum _{j=1}^m{d}_{iju}{d}_{ijl}{d}_{ijz}|>C{m}^{-5/8})\le C{m}^{-3/4}$$ (35)

for all 1 ≤ u, l, z ≤ p, where d_iju denote the uth component of d_ij, we have T_n0m = T_1n0m + R_n0m, where P(‖R_n0m‖ > Cv_2m) = o(v_m). We know that the first order Edgeworth expansions for T_1n0m and T_n0m coincide.

Let $G_{2n_{0}m}={\sum }_{i=1}^{n_0}{\sum }_{j=a}^m{d}_{ij}{W}_{i1j}$ with a = [m^{(1−2δ₀)/2}], $A_{2n_{0}m}={(({\sum }_{i=1}^{n_0}{\sum }_{j=a}^m{d}_{ijk}{d}_{ijl}{W}_{i2j}))}_{p\times p}$, ${\widehat{W}}_{2n_{0}m}={((A_{2n_{0}m}^{\prime }-\mathrm{E}{A}_{2n_{0}m}^{\prime }):{\sum }_{i=1}^{n_0}{m}^{-1}{\sum }_{j=1}^m{W}_{i2j})}^{\prime }$, ${\widehat{\sum }}_{2n_{0}m}={\sum }_{i=1}^{n_0}{\sum }_{k=0}^l{(m-k)}^{-1}{L}_{ikm}{\sum }_{j=a}^{m-k}[W_{i4j}-G_{2n_{0}m}^{\prime }{\gamma }_{ijk}]$, and

$$T_{2n_{0}m}={\sum }_{n_{0}m}^{-1/2}{G}_{1n_{0}m}+\sum _{r=1}^p{G}_{2n_{0}m}^{\prime }{\mathrm{\Lambda }}_{n_{0}rm}{G}_{2n_{0}m}{q}_r+\sum _{r=1}^p{\widehat{W}}_{2n_{0}m}^{\prime }{\mathrm{\Lambda }}_{1n_{0}rm}{G}_{2n_{0}m}{q}_r+\sum _{|v|=1}{({\widehat{\sum }}_{2n_{0}m})}^v{\mathrm{\Lambda }}_{vm}{G}_{2n_{0}m}.$$ (36)

The reason for T_2n0m is that the first term is the same as T_1n0m, but the remaining terms consist of truncated independent variables for obtaining the simplified forms of expansion. Using an Edgeworth expansion under dependence for T_1n0m (Lahiri 1994, 1996), we have

$$P(\Vert T_{1n_{0}m}-T_{2n_{0}m}\Vert >C{v}_m)=o(v_m).$$ (37)

We have the same first-order Edgeworth expansions for T_2n0m, T_1n0m, and T_n0m; namely, three statistics are close to each other.

We obtain that

$$\underset{B\in \mathcal{B}}{sup}|P(T_{n_{0}m}\in B)-{\int }_B(1+p_{n_{0}m}(x))d\mathrm{\Phi }(x)|=o(m^{-1/2}),$$ (38)

with ‖p_n0mϕ‖_∞) = O(m^−1/2). The proof is then complete.

Proof of Theorem 2

Proof. Let $G_{n_{0}m}^{\ast }=D_{n_{0}m}^{-1}({\widehat{\beta }}_{n0m}^{\ast }-{\widehat{\beta }}_{n_{0}m})$, $G_{1n_{0}m}^{\ast }={\sum }_{i=1}^{n_0}{\sum }_{u=1}^k{W}_{i1u}^{\ast }$, $A_{n_{0}m}^{\ast }={\sum }_{u=1}^k{W}_{i2u}^{\ast }$, ${\widehat{A}}_{n_{0}m}=E_{n_{0}m}{A}_{n_{0}m}^{\ast }$, ${\tau }_{1n_{0}m}^{\ast }=m^{-1}{\sum }_{i=1}^{n_0}{\sum }_{j=1}^m{\psi }^{\prime }({\widehat{e}}_{ij}^{\ast })$, ${\widehat{\tau }}_{1n_{0}m}=E_{n_{0}m}^{\ast }({\tau }_{1n_{0}m}^{\ast })$, ${\xi }_{uj}^{\ast }=j\text{th}$, component of $B_u^{\ast }$, for u = 1, k,…,

$$W_{i1u}^{\ast }=\sum _{j=1}^b{d}_{i((u-1)b+j)}\widehat{\psi }({\xi }_{uj}^{\ast }),W_{i2u}^{\ast }=\sum _{j=1}^b{d}_{i((u-1)b+j)}{d}_{i((u-1b+j)}^{\prime }{\psi }^{\prime }({\xi }_{uj}^{\ast }),1\le k\le m.$$

As in the proof of Theorem 4.5.1 and in Lahiri's (1996) result, we have

$$T_{n_{0}m}^{\ast }=T_{1n_{0}m}^{\ast }+R_{n_{0}m}^{†\ast }$$ (39)

where $P_{n_{0}m}(\Vert R_{n_{0}m}^{†\ast }\Vert >C{v}_{2m})=O_p(v_{3m})$. We also use $T_{1n_{0}m}^{\ast }$ and $T_{2n_{0}m}^{\ast }$, which is the same definition of T_1n0m and T_2n0m in Theorem 4.5.1.

Now using the results of Bhattachararya and Ranga Rao (1986) and Lahiri (1996), We have the same first-order Edgeworth expansion forms for $T_{2n_{0}m}^{\ast }$, $T_{1n_{0}m}^{\ast }$, and $T_{n_{0}m}^{\ast }$, since those are close to each other. We obtain

$$\underset{B\in \mathcal{B}}{sup}|P^{\ast }(T_{n_{0}m}^{\ast }\in B)-P(T_{n_{0}m}\in B)|=o_p(v_m).$$ (40)