Unified inference for sparse and dense longitudinal models

Summary

In longitudinal data analysis, statistical inference for sparse data and dense data could be substantially different. For kernel smoothing estimate of the mean function, the convergence rates and limiting variance functions are different under the two scenarios. The latter phenomenon poses challenges for statistical inference as a subjective choice between the sparse and dense cases may lead to wrong conclusions. We develop self-normalization based methods that can adapt to the sparse and dense cases in a unified framework. Simulations show that the proposed methods outperform some existing methods.

1. Introduction

Longitudinal models have extensive applications in biomedical, psychometric and environmental sciences (Fitzmaurice et al., 2004; Wu & Zhang, 2006). In longitudinal studies, repeated measurements are recorded over time from subjects, and therefore measurements from the same subject are correlated. One popular framework is to assume that the observations from each subject are noisy discrete realizations of an underlying process {ξ(·)}:

$$Y_{ij}={\xi }_i(X_{ij})+\sigma (X_{ij}){\epsilon }_{ij}(i=1,\dots ,n;j=1,\dots ,n_i).$$ (1)

Here Y_ij is the measurement at time X_ij from subject i, {ξ_i(·)} are independent realizations of an underlying process {ξ(·)}, ε_ij are errors with E(ε_ij) = 0 and $E({\epsilon }_{ij}^2)=1$, n_i is the number of measurements collected on subject i, and n is the total number of subjects.

There are two typical approaches to taking between-subject variation into account: functional principal component analysis (Yao et al., 2005a,b; Yao, 2007; Ma et al., 2012) and the mixed-effects approach (Wu & Zhang, 2002; Zhang & Chen, 2007). The basic idea of the latter is to decompose {ξ_i(·)} into a fixed population mean μ(·) = E{ξ_i(·)} and a subject-specific random trajectory υ_i(·) with E{υ_i(x)} = 0 and covariance function γ(x, x′) = cov{υ_i(x), υ_i(x′)}. Then (1) becomes

$$Y_{ij}=\mu (X_{ij})+{\upsilon }_i(X_{ij})+\sigma (X_{ij}){\epsilon }_{ij}(i=1,\dots ,n;j=1,\dots ,n_i).$$ (2)

The goal is to estimate the population mean μ(·) and construct a confidence interval for it.

Depending on the number of measurements within subjects, model (2) has two scenarios: dense and sparse longitudinal data. Dense longitudinal data allow n_i → ∞ and a conventional estimation approach is to smooth each individual curve and then construct an estimator based on the smoothed curves (Ramsay & Silverman, 2005; Hall et al., 2006; Zhang & Chen, 2007). In sparse longitudinal data, the n_i are either bounded or independent and identically distributed with E(n_i) < ∞, and due to the sparse observations from individual subjects, it is essential to pool data together (Yao et al., 2005a; Hall et al., 2006; Yao, 2007; Ma et al., 2012).

In practice, the boundary between dense and sparse cases may not always be clear, and such ambiguity could pose challenges for statistical inference, since different researchers may likely classify the same data set differently. To address this issue, Li & Hsing (2010) proposed a unified weighted local linear estimator of μ(x). However, as shown in Section 2, the latter estimator has different convergence rates and limiting variances under the two scenarios. Therefore, to construct a confidence interval for μ(x), one should make a subjective decision whether to treat the data as sparse or dense. In Section 2, we show that the constructed confidence intervals based on a sparse or dense assumption could differ substantially, depending on many unknown factors. Another challenging issue is that the limiting variance function contains the unknown functions γ(x, x) and σ²(x). As shown by Wu & Zhang (2002), Yao et al. (2005a,b), Müller (2005) and Li & Hsing (2010), covariance estimation requires extra smoothing procedures.

We develop two unified nonparametric approaches that can successfully solve the aforementioned issues. First, we establish a unified convergence theory so that inference can be conducted without deciding whether the data are dense or sparse. Second, the unknown limiting variance is canceled out through a self-normalization technique, and thus the proposed methods do not require estimation of the functions γ(x, x) and σ²(x). The first approach introduces a unified self-normalized central limit theorem that can adapt to both cases. The second approach constructs a self-normalizer based on recursive estimates of the mean function. The related methods have been explored mainly under parametric settings for time series data (Lobato, 2001; Kiefer & Vogelsang, 2005; Shao, 2010). In the longitudinal setting, our development of the self-normalization method is more attractive due to the sparse and dense scenario and the more complicated structure such as the within-subject covariance and overall noise variance function. Simulations show that the proposed methods outperform some existing methods.

2. Motivation

For model (2), we consider two scenarios: (i) sparse longitudinal data: n₁, … , n_n are independent and identically distributed positive-integer-valued random variables with E(n_i) < ∞; and (ii) dense longitudinal data: n_i ≥ M_n for some M_n → ∞ as n → ∞.

Throughout we let f(·) denote the density function of X_ij and let x be an interior point of the support of f(·). Li & Hsing (2010) proposed a sample-size weighted local linear estimator of μ(x). For technical convenience, we consider the weighted local constant estimator

$${\widehat{\mu }}_n(x)=\underset{\theta }{\text{argmin}}\sum _{i=1}^n\frac{1}{n_i}\sum _{j=1}^{n_i}{(Y_{ij}-\theta )}^{2}K\left(\frac{X_{ij}-x}{b}\right)=\frac{G_n}{H_n},$$ (3)

where K is a kernel function satisfying ∫_ℝ K(u)du = 1 and b > 0 is a bandwidth, with

$$H_n=\sum _{i=1}^n\frac{1}{n_i}\sum _{j=1}^{n_i}K\left(\frac{X_{ij}-x}{b}\right),G_n=\sum _{i=1}^n\frac{1}{n_i}\sum _{j=1}^{n_i}{Y}_{ij}K\left(\frac{X_{ij}-x}{b}\right).$$ (4)

The convergence rates and limiting variances are different for sparse and dense longitudinal data. To gain intuition about this, write

$${\widehat{\mu }}_n(x)-\mu (x)-\frac{1}{H_n}\sum _{i=1}^n\frac{1}{n_i}\sum _{j=1}^{n_i}\{\mu (X_{ij})-\mu (x)\}K\left(\frac{X_{ij}-x}{b}\right)=\frac{1}{H_n}\sum _{i=1}^n{\xi }_i,$$ (5)

where the right hand side determines the asymptotic distribution of μ̂_n(x), with

$${\xi }_i=\frac{1}{n_i}\sum _{j=1}^{n_i}{\xi }_{ij},{\xi }_{ij}=\{{\upsilon }_i(X_{ij})+\sigma (X_{ij}){\epsilon }_{ij}\}K\left(\frac{X_{ij}-x}{b}\right).$$ (6)

Recall γ(x, x′) = cov{υ_i(x), υ_i(x′)}. For j ≠ j′, by E(ξ_ijξ_ij′) = E{E(ξ_ijξ_ij′ | X_ij, X_ij′)},

$$E({\xi }_{ij}{\xi }_{ij\prime })=E\{\gamma (X_{ij},X_{ij\prime })K(\frac{X_{ij}-x}{b}\left)K\right(\frac{X_{ij\prime }-x}{b}\left)\right\}\approx b^2{f}^2(x)\gamma (x,x).$$ (7)

Throughout, c_n ≈ d_n means that c_n/d_n → 1. Similarly,

$$E({\xi }_{ij}^2)=E\{E({\xi }_{ij}^2|X_{ij})\}\approx bf(x){\psi }_K\{\gamma (x,x)+{\sigma }^2(x)\},{\psi }_K={\int }_{ℝ}{K}^2(u)du.$$ (8)

Applying (7)–(8) to $\text{var}({\xi }_i|n_i)=n_i^{-2}\{{\sum }_{1\le j\ne j\prime \le n_i}E({\xi }_{ij}{\xi }_{ij\prime })+{\sum }_{j=1}^{n_i}E({\xi }_{ij}^2)\}$, we obtain

$$\text{var}({\xi }_i|n_i)\approx (1-1/n_i)b^2{f}^2(x)\gamma (x,x)+f(x){\psi }_K\{\gamma (x,x)+{\sigma }^2(x)\}b/n_i.$$ (9)

For sparse case with b → 0, var(ξ_i | n_i) ≈ bf(x)ψ_K{γ(x, x) + σ²(x)}/n_i; for dense case with n_i ≥ M_n and M_nb → ∞, var(ξ_i | n_i) ≈ b²f²(x)γ(x, x).

Theorem 1. Assume Assumption 1 in the Appendix. Let f(x) be the density of X_ij. Write

$${\psi }_K={\int }_{ℝ}{K}^2(u)du,\rho (x)=\{\frac{\mu ″(x)}{2}+\frac{\mu \prime (x)f\prime (x)}{f(x)}\}{\int }_{ℝ}{u}^{2}K(u)du.$$

1. Sparse data: Assume nb → ∞ and sup_n nb⁵ < ∞. Then

   $$\surd (nb)\{{\widehat{\mu }}_n(x)-\mu (x)-b^2\rho (x)\}\to N\{0,s_{\text{sparse}}^2(x)\},$$ (10)

   where $s_{\text{sparse}}^2(x)=\tau {\psi }_K\{\gamma (x,x)+{\sigma }^2(x)\}/f(x)$ and τ = E(1/n₁).
2. (ii) Dense data: Assume n_i ≥ M_n, M_nb → ∞, nb → ∞ and sup_n nb⁴ < ∞. Then

   $$\surd n\{{\widehat{\mu }}_n(x)-\mu (x)-b^2\rho (x)\}\to N\{0,s_{\text{dense}}^2(x)\},s_{\text{dense}}^2(x)=\gamma (x,x).$$ (11)

It is worth mentioning some related results. Li & Hsing (2010) established the uniform consistency of μ̂_n(x) with different rates under the sparse and dense cases, but they did not obtain the asymptotic distribution. Wu & Zhang (2002) also showed that the local polynomial mixed-effects estimator has different convergence rates and limiting variances under the two scenarios. Under a Karhunen–Loève representation of longitudinal models, Yao (2007) studied the sparse case by allowing n_i to be dependent on n; see also Ma et al. (2012).

By Theorem 1, the confidence interval for μ(x) is different under the two cases. Let z_1−α/2 be the 1 − α/2 standard normal quantile. Then an asymptotic 1 − α confidence interval for μ(x) is

$${\widehat{\mu }}_n(x)-b^2\widehat{p}(x)\pm z_{1-\alpha /2}{(nb)}^{-1/2}{[\widehat{\tau }{\psi }_K\{\widehat{\gamma }(x,x)+{\widehat{\sigma }}^2(x)\}/\widehat{f}(x)]}^{1/2}$$ (12)

for sparse data, or

$${\widehat{\mu }}_n(x)-b^2\widehat{\rho }(x)\pm z_{1-\alpha /2}{n}^{-1/2}{\{\widehat{\gamma }(x,x)\}}^{1/2}$$ (13)

for dense data. Here, $\widehat{\tau }=n^{-1}{\sum }_{i=1}^n{n}_i^{-1}$, γ̂(x, x), σ̂²(x), f̂(x) and ρ̂(x) are consistent estimates of τ, γ(x, x), σ²(x), f(x) and ρ(x). The ratio of the lengths of the two confidence intervals is R = [ψ_K τ̂{1 + σ̂²(x)/γ̂(x, x)}/{b f̂(x)}]^1/2, which depends on the denseness parameter τ, the signal-to-noise ratio γ(x, x)/σ²(x), the bandwidth b and the design density f(·). The further away R is from one, the larger the discrepancy between the two constructed confidence intervals.

Remark 1. In the dense case, suppose n_i is proportional to M_n → ∞. Theorem 1 (ii) studies the case M_nb → ∞. If M_nb → 0, then the leading term in (9) is f(x)ψ_K{γ(x, x) + σ²(x)}b/n_i. If M_nb is bounded away from 0 and ∞, then both terms in (9) are of the same order. If b is proportional to (nM_n)^−1/5, then a sufficient condition for M_nb → ∞ is $M_n^4/n\to \infty$. In many practical problems, n is about 30–200, M_n is about 10–30, and $M_n^4/n$ is sufficiently large.

3. Unified approaches for sparse and dense data

3·1. A unified self-normalized central limit theorem

The discussion in Section 2 suggests a need for a unified approach. For independent and identically distributed random variables Z₁, … , Z_n, de la Peña et al. (2009) gave an extensive account of the asymptotic properties of the self-normalized statistic ${\sum }_{i=1}^n{Z}_i/\surd ({\sum }_{i=1}^n{Z}_i^2)$. In this section, we present a unified self-normalized central limit theorem for μ̂_n(x). For H_n in (4), define

$$U_n^2(x)=\frac{1}{H_n^2}\sum _{i=1}^n{[\frac{1}{n_i}\sum _{j=1}^{n_i}\{Y_{ij}-\widehat{\mu }(X_{ij})\}K(\frac{x-X_{ij}}{b}\left)\right]}^2.$$

Theorem 2. Assume Assumption 1 in the Appendix. Suppose nb/ log n → ∞, sup_n nb⁵ < ∞ for sparse data or n_i ≥ M_n, M_nb → ∞, nb²/ log n → ∞, sup_n nb⁴ < ∞ for dense data. Then {μ̂_n(x) − μ(x) − b² ρ(x)}/U_n(x) → N(0, 1) in both the sparse and dense settings.

Many papers treat sparse and dense data separately. For example, Yao et al. (2005a,b), Yao (2007) and Ma et al. (2012) studied sparse longitudinal data. For the local polynomial mixed-effects estimator, Wu & Zhang (2002) obtained different central limit theorems under the two scenarios. By contrast, Theorem 2 establishes a unified central limit theorem, which can be used to construct a unified asymptotic pointwise 1 − α confidence interval for μ(x):

$${\widehat{\mu }}_n(x)-b^2\widehat{\rho }(x)\pm z_{1-\alpha /2}{U}_n(x).$$ (14)

While the confidence intervals (12)–(13) require estimation of the within-subject covariance function γ(x, x) and the overall noise variance function σ²(x), (14) avoids such extra smoothing steps and can adapt to the sparse or dense setting through the self-normalizer U_n(x).

To select the bandwidth b, we adopt subject-based cross-validation (Rice & Silverman, 1991). The idea is to leave one subject out in model fitting, validate the fitted model using the left-out subject, and choose the optimal bandwidth by minimizing the prediction error:

$$b^{*}=\underset{b}{\text{argmin}}\text{SJCV}(b),\text{SJCV}(b)=\sum _{i=1}^n\frac{1}{n_i}\sum _{j=1}^{n_i}{\{Y_{ij}-{\widehat{\mu }}^{(-i)}(X_{ij})\}}^2,$$ (15)

where μ̂⁽⁻ⁱ⁾(x) represents the estimator of μ(x) based on data from all but the ith subject.

In practice, it is difficult to estimate the bias b²ρ(x) due to the unknown derivatives f′, μ′, μ″. In our simulations, we use K(u) = 2G(u) − G(u/ √ 2)/ √2 with G(u) the standard normal density. Then ∫_ℝ u²K(u)du = 0 and ρ(x) = 0. However, this does not solve the bias issue. For example, if f and μ are four times differentiable, then we have the higher order bias term O(b⁴). The bias issue is inherently difficult and there is no good solution so far.

3·2. Self-normalization based on recursive estimates

In this section we introduce another self-normalization method based on recursive estimates. For m = 1, … , n, denote by μ̂_m(x) the estimator in (3) based on observations from the first m subjects. Then μ̂₁(x), … , μ̂_n(x) are estimates of μ(x) with increasing accuracy. Moreover, μ̂_m(x) has similar asymptotic normality as in (10)–(11). For example, for each 0 < t ≤ 1, the counterpart of (10) for sparse data is $\surd (ntb)\{{\widehat{\mu }}_{\lfloor nt\rfloor }(x)-\mu (x)-b^2\rho (x)\}\to N\{0,s_{\text{sparse}}^2(x)\}$. Throughout, ⌊z⌋ is the integer part of z. Therefore, μ̂_n(x) and μ̂_⌊nt⌋(x) have proportional convergence rates and the same limiting variance, which motivates us to consider certain ratios between μ̂_n(x) and μ̂_⌊nt⌋ (x) to cancel out the convergence rates and limiting variance.

Since the above analysis holds for all 0 < t ≤ 1, we consider an aggregated version

$$T_n(x)=\frac{{\widehat{\mu }}_n(x)-\mu (x)-b^2\rho (x)}{V_n(x)},V_n(x)=n^{-3/2}{\left\{\sum _{m=\lfloor cn\rfloor }^n{m}^2{|{\widehat{\mu }}_m(x)-{\widehat{\mu }}_n(x)|}^2\right\}}^{1/2}.$$

Throughout c > 0 is a small constant included to avoid unstable estimation at the boundary. By our simulations, c =0·1 works reasonably well. Intuitively, we may interpret μ̂_m(x), m = 1, … , n, as observations from a population with mean μ(x) and treat μ̂_n(x) as sample average. Thus, V_n(x) can be viewed as a weighted sample standard deviation with the weight m² reflecting the accuracy of μ̂_m(x), and V_n(x) mimics the usual normalizer in the Student-t distribution.

Theorem 3. Assume the conditions in Theorem 1. Let {B_t} be a standard Brownian motion. Then $T_n(x)\to B_1/{\{{\int }_c^1{(B_t-t{B}_1)}^{2}dt\}}^{1/2}$ under either the sparse or the dense settings.

By Theorem 3, an asymptotic pointwise 1 − α confidence interval for μ(x) is μ̂_n(x) − b² ρ̂(x) ± q_1−α/2V_n(x), where q_1−α/2 is the 1 − α/2 quantile of the limiting distribution. The latter confidence interval is the same for both scenarios, with the convergence rate and limiting variance being built into the self-normalizer V_n(x) implicitly. Our method can be viewed as an extension of the parametric self-normalization methods in Lobato (2001), Kiefer & Vogelsang (2005) and Shao (2010) for time series data to the nonparametric longitudinal model (2).

In practice, however, subjects have no natural ordering, and we can use the average of multiple copies of $V_n^2(x)$ through permuting the subjects. For a large n, since it is computationally infeasible to enumerate all permutations, we consider only a fixed number, say T, of random permutations. Denote the corresponding V_n(x) by $V_n^1(x),\dots ,V_n^T(x)$. Consider

$${\tilde{T}}_n(x)=\frac{{\widehat{\mu }}_n(x)-\mu (x)-b^2\rho (x)}{{\tilde{V}}_n(x)},{\tilde{V}}_n^2(x)=\frac{1}{T}\sum _{r=1}^T{\{V_n^T(x)\}}^2.$$

By the above analysis, the asymptotic distribution of T̃_n(x) is the same under both the sparse and dense settings. However, it is not clear whether T̃_n(x) is asymptotically normally distributed. Nevertheless, in light of the asymptotic normality of μ̂_n(x), the proof of Theorem 3 and $E\{{\int }_c^1{(B_t-t{B}_1)}^{2}dt\}=(1-3c^2+2c^3)/6$, we propose the pointwise confidence interval

$${\widehat{\mu }}_n(x)-b^2\widehat{\rho }(x)\pm z_{1-\alpha /2}{\tilde{V}}_n(x)\surd c_1,c_1=6/(1-3c^2+2c^3).$$ (16)

Here z_1−α/2 is defined in (12). We call it the rule-of-thumb self-normalization based confidence interval. Our quantile-quantile studies show that the empirical quantile of T̃_n(x) with 200 permutations matches well with that of N(0, c₁) under the settings in Section 4.

4. Numerical results

Following Li & Hsing (2010), we consider the model

$$Y_{ij}=\mu (X_{ij})+\sum _{k=1}^3{\alpha }_{ik}{\mathrm{\Phi }}_k(X_{ij})+\sigma {\epsilon }_{ij}(i=1,\dots ,n;j=1,\dots ,n_i),$$

where α_ik ~ N (0, ω_k) and ε_ij ~ N(0, 1). Let μ(x) = 5 (x − 0.6)², Φ₁(x) = 1, Φ₂ (x) = √2 sin(2πx), Φ₃(x) = √2 cos(2πx), (ω₁, ω₂, ω₃) =(0·6,0·3,0·1), and n = 200. Then the variance function γ(x, x) =0·6+0·6sin²(2πx)+0·2cos²(2πx). Two noise levels σ = 1, 2 are considered. The design points X_ij are uniformly distributed on [0, 1]. For the vector N = (n₁, … , n_n) of the number of measurements on individual subjects, we consider four cases

$$N_1:n_i~U[\{2,3,\dots ,8\}];N_2:n_i~U[\{15,16,\dots ,35\}];$$ (17)

$$N_3:n_i~U[\{30,31,\dots ,70\}];N_4:n_i~U[\{150,151,\dots ,250\}].$$ (18)

Here U[𝒟] stands for the discrete uniform distribution on a finite set 𝒟.

We compare six confidence intervals: the two self-normalization based confidence intervals in (14) and (16) with 200 permutations, the asymptotic normality based confidence intervals (12)–(13) assuming sparse and dense data, respectively, the bootstrap confidence interval with 200 bootstrap replications from sampling subjects with replacement, and the confidence interval

$${\widehat{\mu }}_n(x)-b^2\widehat{\rho }(x)\pm z_{1-\alpha /2}{n}^{-1/2}{\{(1-\widehat{\tau })\gamma (x,x)+\widehat{\tau }{\psi }_K\frac{\gamma (x,x)+{\sigma }^2(x)}{bf(x)}\}}^{1/2}.$$ (19)

The confidence interval (19) is practically infeasible as we need to estimate the unknown functions. Nevertheless, by using the true theoretical limiting variance function in (9), (19) serves as a standard against which we can measure the performance of other confidence intervals. When using the local linear method in Li & Hsing (2010) to estimate γ(x, x), we found that negative estimates of γ(x, x) occur frequently, especially when the noise level σ is high. For the purpose of comparison, we use the true functions γ(x, x), σ²(x) and f(x) to implement (12)–(13).

We consider two criteria: empirical coverage probabilities and lengths of confidence intervals. Let x₁ < ⋯ < x₂₀ be 20 grid points evenly spaced on [0·1, 0·9]. For each x_j and a given nominal level, we construct confidence intervals for μ(x_j), and compute the empirical coverage probabilities based on 1000 replications. For each of the six confidence intervals, we average their empirical coverage probabilities and lengths at 20 grid points. To facilitate computations in bandwidth selection, instead of using (15) for each replication, we set b to be the average of 20 optimal bandwidths in (15) based on 20 replications from each set of parameter choices.

The results are presented in Table 1. The performance of the confidence intervals (12)–(13) depends on whether the data are sparse or dense. As we increase the number of measurements on each subject from the sparse setting N₁ to the dense setting N₄, (12) under sparse assumption performs increasingly worse whereas (13) under dense assumption performs increasingly better. The simulation study further confirms the theoretical results in Theorem 1 that the confidence intervals (12)–(13) perform well only under their corresponding sparse or dense assumption. By contrast, the self-normalization based confidence intervals (14) and (16) deliver robust and superior performance: (i) they have similar widths but slightly better coverage probabilities than the bootstrap confidence interval; and (ii) they perform similar to the infeasible confidence interval (19) with true functions. Finally, (14) and (16) have comparable performance.

Table 1.

Average empirical coverage percentages and lengths, in brackets, of six confidence intervals.

1 − α	σ	N	SN1	SN2	NS	ND	NSD	BS
90%	1	N1	88·1 (0·381)	88·6 (0·386)	82·8 (0·331)	66·5 (0·236)	89·2 (0·389)	87·4 (0·377)
		N2	88·9 (0·288)	89·2 (0·290)	68·0 (0·178)	81·0 (0·236)	89·4 (0·292)	88·1 (0·284)
		N3	89·8 (0·262)	89·8 (0·263)	57·8 (0·126)	86·3 (0·236)	90·3 (0·265)	89·1 (0·258)
		N4	88·4 (0·246)	88·5 (0·247)	37·3 (0·076)	86·9 (0·236)	88·6 (0·247)	87·5 (0·242)
	2	N1	88·8 (0·528)	89·3 (0·534)	86·5 (0·497)	51·7 (0·236)	89·4 (0·537)	87·8 (0·523)
		N2	88·6 (0·330)	88·7 (0·332)	75·8 (0·243)	74·3 (0·236)	89·3 (0·335)	87·9 (0·326)
		N3	89·5 (0·293)	89·4 (0·294)	69·5 (0·183)	81·1 (0·236)	90·1 (0·297)	88·8 (0·289)
		N4	88·4 (0·257)	88·6 (0·257)	48·6 (0·106)	85·1 (0·236)	88·7 (0·258)	87·6 (0·252)
95%	1	N1	93·6 (0·454)	94·0 (0·460)	89·7 (0·394)	75·2 (0·281)	94·6 (0·464)	92·9 (0·446)
		N2	94·1 (0·343)	94·3 (0·345)	76·4 (0·212)	88·1 (0·281)	94·7 (0·347)	93·4 (0·335)
		N3	95·0 (0·312)	95·1 (0·314)	66·0 (0·150)	92·1 (0·281)	95·3 (0·316)	94·0 (0·305)
		N4	94·2 (0·293)	94·3 (0·294)	43·7 (0·090)	92·9 (0·281)	94·3 (0·294)	93·1 (0·286)
	2	N1	94·2 (0·629)	94·4 (0·636)	92·6 (0·592)	59·7 (0·281)	94·8 (0·640)	93·2 (0·618)
		N2	93·9 (0·393)	94·0 (0·395)	83·6 (0·289)	82·3 (0·281)	94·2 (0·399)	93·0 (0·385)
		N3	94·7 (0·349)	94·8 (0·351)	77·9 (0·219)	88·0 (0·281)	95·1 (0·354)	93·8 (0·341)
		N4	94·1 (0·306)	94·1 (0·307)	56·4 (0·127)	91·6 (0·281)	94·2 (0·308)	93·0 (0·298)

SN1 and SN2: the self-normalized confidence intervals in (14) and (16) with 200 permutations, respectively; NS and ND: the asymptotic normality based confidence intervals (12)–(13) assuming sparse and dense data, respectively; NSD: the infeasible confidence interval in (19); BS: bootstrap confidence interval; N₁–N₄: the numbers of measurements on individual subjects in (17)–(18).

Appendix: Regularity conditions and proofs

Assumption 1. (i) K(·) is bounded, symmetric, and has bounded support and bounded derivative. (ii) {υ_i(·)}_i, {X_ij}_ij, {ε_ij}_ij are independent and identically distributed and mutually independent. Furthermore, the density function f(·) of X_ij is twice continuously differentiable in a neighborhood of x and f(x) > 0. (iii) In a neighborhood of x, μ(·) is twice continuously differentiable, σ²(·) is continuously differentiable; in a neighborhood of (x, x), γ(x, x′) = cov{υ_i(x), υ_i(x′)} is continuously differentiable. Moreover, γ(x, x) > 0, σ²(x) > 0. (iv) E{|υ_i(·) + σ(·)ε_ij |⁴} is continuous in a neighborhood of x and E{|υ_i(x) + σ(x)ε_ij |⁴} < ∞.

Proof of Theorem 1. Let ξ_i be defined in (6). Recall the decomposition (5). Write

$$H_n=\sum _{i=1}^n{\nu }_i,{\nu }_i=\frac{1}{n_i}\sum _{j=1}^{n_i}{\nu }_{ij},{\nu }_{ij}=K\left(\frac{X_{ij}-x}{b}\right),$$ (20)

$$I_n=\sum _{i=1}^n{\zeta }_i,{\zeta }_i=\frac{1}{n_i}\sum _{j=1}^{n_i}{\zeta }_{ij},{\zeta }_{ij}=\{\mu (X_{ij})-\mu (x)\}K\left(\frac{X_{ij}-x}{b}\right).$$ (21)

By the symmetry of K and Taylor’s expansion, E(ν_ij) = {1 + O(b²)}bf(x), var(ν_ij) = O(b), E(ζ_ij) = b³f(x)ρ(x) + o(b³), var(ζ_ij) = O(b³). In either the sparse or the dense case, E(ν_i | n_i) = E(ν_ij) is non-random. Thus, var(ν_i) = E{var(ν_i | n_i)} = var(ν_ij)E(1/n_i) and $\text{var}(H_n)={\sum }_{i=1}^n\text{var}({\nu }_i)=O(b){\sum }_{i=1}^{n}E(1/n_i)$. Write ${\tau }_n=n^{-1}{\sum }_{i=1}^{n}E(1/n_i)$. Then

$$H_n=E(H_n)+O_p\{\surd \text{var}(H_n)\}=[1+O_p\{b^2+{(nb/{\tau }_n)}^{-1/2}\}]nbf(x).$$ (22)

Similarly, I_n = nb³ f (x) ρ(x) + o(nb³) + O_p{√(nb³τ_n)}. Thus,

$$I_n/H_n=b^2\rho (x)+{\delta }_n,{\delta }_n=o_p(b^2)+O_p\{\surd (b{\tau }_n/n)\}.$$ (23)

Dense case: Under given conditions, δ_n = o_p(n^−1/2) and ${\{n{b}^2{f}^2(x)\}}^{-1}\text{var}({\sum }_{i=1}^n{\xi }_i)\to \gamma (x,x)$. For distinct j, r, s, k, by the argument in (7), E(ξ_ijξ_irξ_isξ_ik) = O(b⁴), $E({\xi }_{ij}^2{\xi }_{ir}{\xi }_{is})=O(b^3),E({\xi }_{ij}^2{\xi }_{ir}^2)=O(b^2),E({\xi }_{ij}^3{\xi }_{ir})=O(b^2),E({\xi }_{ij}^4)=O(b)$. Thus, ${\sum }_{i=1}^{n}E({\xi }_i^4)=O(n{b}^4)=o\{{(b\surd n)}^4\}$. By the Lyapunov central limit theorem, ${\sum }_{i=1}^n{\xi }_i/\{b\surd nf(x)\}\to N\{0,\gamma (x,x)\}$.

Sparse case: In (5), ξ₁, … , ξ_n are independent and identically distributed. The result follows from δ_n = o_p{(nb)^−1/2} and var(ξ_i) = E{var(ξ_i | n_i)} ≈ bτψ_K f(x){γ(x, x) + σ²(x)}.

Proof of Theorem 2. By Theorem 1, it suffices to show $n{U}_n^2(x)\to s_{\text{dense}}^2(x)$ for dense data or $nb{U}_n^2(x)\to s_{\text{sparse}}^2(x)$ for sparse data. For convenience, write K_ij = K{(X_ij − x)/b}. Let

$$S_n=\sum _{i=1}^n{[\frac{1}{n_i}\sum _{j=1}^{n_i}\{Y_{ij}-{\widehat{\mu }}_n(X_{ij})\}{K}_{ij}]}^2=\sum _{i=1}^n({\xi }_i^2+{\eta }_i^2+2{\xi }_i{\eta }_i),$$ (24)

where ξ_i is defined in (6) and ${\eta }_i=n_i^{-1}{\sum }_{j=1}^{n_i}\{\mu (X_{ij})-\widehat{\mu }(X_{ij})\}{K}_{ij}$. By Theorem 3.1 in Li & Hsing (2010), |μ̂(z) − μ(z)| = O_p(ℓ_n) uniformly for z in the neighborhood of x, where ℓ_n = b² + (n/ log n)^−1/2 for dense data or ℓ_n = b² + (nb/ log n)^−1/2 for sparse data. Then ${\eta }_i=O_p({\ell }_n)n_i^{-1}{\sum }_{j=1}^{n_i}|K_{ij}|$. Using ${\xi }_i=n_i^{-1}{\sum }_{j=1}^{n_i}{\xi }_{ij}$, where ξ_ij is defined in (6), we obtain

$$\sum _{i=1}^n|{\eta }_i^2+2{\xi }_i{\eta }_i|=O_p({\ell }_n^2)\sum _{i=1}^n{(\frac{1}{n_i}\sum _{j=1}^{n_i}|K_{ij}|)}^2+O_p({\ell }_n)\sum _{i=1}^n\frac{1}{n_i^2}\sum _{j=1}^{n_i}|{\xi }_{ij}|\sum _{j=1}^{n_i}|K_{ij}|\le O_p({\ell }_n)J_n,J_n=\sum _{i=1}^n\frac{1}{n_i}\sum _{j=1}^{n_i}{K}_{ij}^2+\sum _{i=1}^n\frac{1}{n_i^2}\sum _{j=1}^{n_i}\sum _{j\prime =1}^{n_i}(K_{ij}^2+{\xi }_{ij\prime }^2).$$

Here we have used ${\ell }_n^2=o({\ell }_n),{({\sum }_{j=1}^{n_i}|K_{ij}|)}^2\le n_i{\sum }_{j=1}^{n_i}{K}_{ij}^2,2|K_{ij}{\xi }_{ij\prime }|\le K_{ij}^2+{\xi }_{ij\prime }^2$. By $E(K_{ij}^2)=O(b)\text{and}E({\xi }_{ij}^2)=O(b)$, E(J_n) = O(nb). Thus, ${\sum }_{i=1}^n|{\eta }_i^2+2{\xi }_i{\eta }_i|=O_p(nb{\ell }_n)$. By (24) and the independence of ξ₁, … , ξ_n,

$$S_n=\sum _{i=1}^{n}E({\xi }_i^2)+O_p({\chi }_n),{\chi }_n={\{\sum _{i=1}^n\text{var}({\xi }_i^2)\}}^{1/2}+nb{\ell }_n.$$

From the proof of Theorem 1, ${\{n{b}^2{f}^2(x)\}}^{-1}{\sum }_{i=1}^{n}E({\xi }_i^2)\to s_{\text{dense}}^2(x)$ for dense data or ${\{nb{f}^2(x)\}}^{-1}{\sum }_{i=1}^{n}E({\xi }_i^2)\to s_{\text{sparse}}^2(x)$ for sparse data. By (22), H_n = {1 + o_p(1)}nbf(x). Thus, it remains to show χ_n = o(nb²) for dense data or χ_n = o(nb) for sparse data. In the dense case, by the proof of Theorem 1, ${\sum }_{i=1}^n\text{var}({\xi }_i^2)\le {\sum }_{i=1}^{n}E({\xi }_i^4)=O(n{b}^4)$, and consequently χ_n = O{√ nb² + nb³ + b √ (n log n)} = o(nb²). In the sparse case, by the proof of the dense case in Theorem 1, $E({\xi }_i^4|n_i)=O(1)n_i^{-4}(n_i^4{b}^4+n_i^3{b}^3+n_i^2{b}^2+n_{i}b),E({\xi }_i^4)=E\{E({\xi }_i^4|n_i)\}=O(b)$, and thus χ_n = O{√ (nb) + nb³ + √ (nb log n)} = o(nb).

Proof of Theorem 3. Recall s_sparse(x) and s_dense(x) in Theorem 1. Let Γ_n = nbf(x)/Λ_n, Λ_n = √ (nb)f(x)s_sparse(x) for sparse data or Λ_n = b √ nf(x)s_dense(x) for dense data. Suppose we can show the weak convergence

$${\{{\mathrm{\Gamma }}_{n}t\{{\widehat{\mu }}_{\lfloor nt\rfloor }(x)-\mu (x)-b^2\rho (x)\}\}}_{c\le t\le 1}\to {\{B_t\}}_{c\le t\le 1}.$$ (25)

For convenience, we write ${ℒ}_2(g)={\{{\int }_c^1|g(t){|}^{2}dt\}}^{1/2}$ and suppress the argument x. By (25) and the continuous mapping theorem, (μ̂_n − μ − b²ρ)ℒ₂{t(μ̂_⌊nt⌋ − μ̂_n)} → B₁/ℒ₂(B_t − tB₁). By |n⁻¹⌊nt⌋ − t| ≤ n⁻¹ for t ∈ [c, 1], ℒ₂{t(μ̂_⌊nt⌋ − μ̂_n)} is asymptotically equivalent to ℒ₂{n⁻¹⌊nt⌋(μ̂_⌊nt⌋ −μ̂_n)} = V_n(x), where V_n(x) is defined in T_n(x). This completes the proof.

It remains to show (25). Recall ν_i and ζ_i in (20)–(21). As in (3) and (5),

$${\widehat{\mu }}_{\lfloor nt\rfloor }(x)-\mu (x)-\frac{1}{H_{\lfloor nt\rfloor }}\sum _{i=1}^{\lfloor nt\rfloor }{\zeta }_i=\frac{W_n(t)}{H_{\lfloor nt\rfloor }},H_{\lfloor nt\rfloor }=\sum _{i=1}^{\lfloor nt\rfloor }{\nu }_i,W_n(t)=\sum _{i=1}^{\lfloor nt\rfloor }{\xi }_i.$$

By Kolmogorov’s maximal inequality for independent random variables,

$$\underset{c\le t\le 1}{\text{sup}}|H_{\lfloor nt\rfloor }-E(H_{\lfloor nt\rfloor })|=\underset{\lfloor cn\rfloor \le m\le n}{\text{max}}|H_m-E(H_m)|=O_p\left[{\{\sum _{i=1}^n\text{var}({\nu }_i)\}}^{1/2}\right].$$

Thus, similar to (22), H_⌊nt⌋ = [1 + O_p{b² + (nb/τ_n)^−1/2}]⌊nt⌋bf(x) uniformly in c ≤ t ≤ 1. Applying the same argument to (23) gives ${\sum }_{i=1}^{\lfloor nt\rfloor }{\zeta }_i/H_{\lfloor nt\rfloor }=b^2\rho (x)+{\delta }_n$ uniformly, where δ_n is defined in (23). Thus it suffices to show {W_n(t)/Λ_n}_c≤t≤1 → {B_t}_c≤t≤1. The finite-dimensional convergence follows from the same argument in Theorem 1 and the Cramér–Wold device. It remains to prove the tightness. Let c ≤ t < t′ ≤ 1. By independence,

$${\mathrm{\Delta }}_n(t,t\prime )=E{\{\frac{W_n(t)}{{\mathrm{\Lambda }}_n}-\frac{W_n(t\prime )}{{\mathrm{\Lambda }}_n}\}}^4=\frac{1}{{\mathrm{\Lambda }}_n^4}\{\sum _{i=\lfloor nt\rfloor +1}^{\lfloor nt\prime \rfloor }E({\xi }_i^1)+6\sum _{\lfloor nt\rfloor +1\le i<k}^{\lfloor nt\prime \rfloor }E({\xi }_i^2)E({\xi }_k^2)\}.$$

By the argument in the proof of Theorem 1, in the dense case, $E({\xi }_i^2)=O(b^2),E({\xi }_i^4)=O(b^4)$, and thus Δ_n(t, t′) = O{|t − t′|/n + |t − t′|²}; in the sparse case, $E({\xi }_i^4)=O(b),E({\xi }_i^2)=O(b)$, and thus Δ_n(t, t′) = O{|t − t′|/(nb) + |t − t′|²}. This proves the tightness.