Variance Estimation in a Model with Gaussian Sub-Models

Abstract

This paper considers the problem of estimating the dispersion parameter in a Gaussian model which is intermediate between a model where the mean parameter is fully known (fixed) and a model where the mean parameter is completely unknown. One of the goals is to understand the implications of the two-step process of first selecting a model among a finite number of sub-models, and then estimating a parameter of interest after the model selection, but using the same sample data. The estimators are classified into global, two-step, and weighted-type estimators. While the global-type estimators ignore the model space structure, the two-step estimators explore the structure adaptively and can be related to pre-test estimators, and the weighted estimators are motivated by the Bayesian approach. Their performances are compared theoretically and through simulations using their risk functions based on a scale invariant quadratic loss function. It is shown that in the variance estimation problem efficiency gains arise by exploiting the sub-model structure through the use of two-step and weighted estimators, especially when the number of competing sub-models is few; but that this advantage may deteriorate or be lost altogether for some two-step estimators as the number of sub-models increases or as the distance between them decreases. Furthermore, it is demonstrated that weighted estimators, arising from properly chosen priors, outperform two-step estimators when there are many competing sub-models or when the sub-models are close to each other, whereas two-step estimators are preferred when the sub-models are highly distinguishable. The results have implications regarding model averaging and model selection issues.

1 Model Selection and Inference

In a variety of settings in statistical practice, it is common to encounter the following situation: we observe data χ from a distribution F which is only known to belong to one of p (possibly nested) sub-models ℳ₁, ℳ₂,…,ℳ_p; and given χ, we want to estimate a common parameter, or a functional, of F, denoted by τ(F). For example, we might observe χ ∼ F where F belongs to either the gamma or Weibull family of distributions, and wish to estimate the mean of F. Or, in a multiple regression setting with p possible predictors, we might want to choose one of the 2^p competing sub-models (Breiman (1992); Zhang (1992b,a)), and then estimate a common parameter such as dispersion or the conditional distribution function of the response variable.

The most frequent strategies for estimating τ(F) are: (i) utilizing an estimator developed under a larger model ℳ, which contains all sub-models; (ii) using data χ to first choose a sub-model, and then applying the estimator developed for the chosen sub-model to the same data χ; and (iii) assigning to each sub-model a plausibility measure, possibly using χ, and then forming a weighted combination of the estimators developed under each of the sub-models. In this paper we are interested in determining whether there is a preferred strategy, and whether that preferred strategy depends on the interplay among the competing sub-models, and possibly the parameter we are estimating.

Issues pertaining to the two-step process of inference after model selection and the consequences of “data double-dipping” in strategy (ii) have been discussed in the econometric literature (Judge, Bock, and Yancey (1974), Leamer (1978), Yancey, Judge, and Mandy (1983), and Wallace (1977)). Further investigations of these issues in other settings are in Potscher (1991), Buhlmann (1999), and Burnham and Anderson (1998). The third strategy has been discussed mostly in the context of model averaging, a notion that naturally arises in the Bayesian paradigm (Madigan and Raftery (1994); Raftery, Madigan, and Hoeting (1997); Hoeting, Madigan, Raftery, and Volinsky (1999); Burnham and Anderson (1998), among many others). The first strategy on the other hand may be viewed as having a nonparametric flavor. Though it is clearly intuitive that the first strategy will entail some loss in efficiency, it is not apparent whether (and when) the second strategy is preferred over the third strategy. Clearly, an examination of this problem in the general framework is important to provide guidance to practitioners regarding which strategy is better in general situations. However, a general treatment of the problem may not yield exact results, and one may need to rely on asymptotics, or local asymptotics such as in the work by Claeskens and Hjort (2003) and Hjort and Claeskens (2003).

In this paper, we focus our attention on a prototype Gaussian model which admits exact results and thereby enables concrete comparison of the three strategies. Though the specific model examined in this paper may be perceived as restrictive, it highlights the difficulties inherent in this problem. In addition, the specific estimation problem examined – the estimation of dispersion parameter – is still the subject of active research (Arnold and Villasenor (1997), Brewster and Zidek (1974), Gelfand and Dey (1977), Maatta and Casella (1990), Ohtani (2001), Pal, Ling, and Lin (1998), Rukhin (1987), Vidaković and DasGupta (1995), and Wallace (1977)).

The paper is outlined as follows. Section 2 will describe the formal setting of the specific problem considered, introduce notation, and present the global-type estimators. Section 3 will present the classical two-step estimators, whereas the Bayes and weighted estimators will be developed in Section 4. Distributional properties and risk comparison will be obtained in Section 5. Concluding remarks are given in Section 6, while Appendix A gathers the technical proofs.

2 Global-Type Estimators

We first describe the specific model examined in this paper. Let χ = (χ₁, χ₂,…,χ_n)… be a vector of IID random variables from an unknown distribution function F(x) = Pr{χ₁ ≤ x} which belongs to the two-parameter normal family of distributions ℳ = {N(μ, σ²) : (μ, σ²) ∈ Θ = ℜ × ℜ₊}. If interest is on estimating the variance σ², then the uniformly minimum variance unbiased estimator (UMVUE) of σ² is

$${\widehat{\sigma }}_{UMVU}^2=S^2=\frac{1}{n-1}{\displaystyle \sum _{i=1}^n{({\chi }_i-\overline{\chi })}^2}$$ (1)

where $\overline{\chi }=\frac{1}{n}{\displaystyle {\sum }_{i=1}^n{\chi }_i}$ (Lehmann and Casella (1998)). We adopt a decision-theoretic approach for evaluating estimators of σ² via the risk function based on the scale invariant quadratic loss function L : ℜ × Θ → ℜ

$$L(a,(\mu ,{\sigma }^2))={\left(\frac{a-{\sigma }^2}{{\sigma }^2}\right)}^2.$$ (2)

It should be pointed that the appropriateness of this loss function has been questioned, partly because of Stein (1964)’s demonstration that under this loss the UMVUE of σ² is inadmissible and dominated by the minimum risk equivariant estimator (MRE)

$${\widehat{\sigma }}_{MRE}^2=\frac{1}{n+1}{\displaystyle \sum _{i=1}^n{({\chi }_i-\overline{\chi })}^2}$$ (3)

(which also turns out to be inadmissible). However, quadratic loss functions are still popular when dealing with the estimation of variance (Arnold and Villasenor (1997), Maatta and Casella (1990), Ohtani (2001), Pal et al. (1998), Rukhin (1987), Vidaković and DasGupta (1995), and Wallace (1977)).

If the model is restricted so that μ = μ₀ where μ₀ ∈ ℜ is known, so ℳ₀ = {N(μ, σ²) : (μ, σ²) ∈ Θ₀ = {μ₀} × ℜ₊}, the UMVUE and MRE of σ² are given, respectively, by

$${\widehat{\sigma }}_{UMVU}^2({\mu }_0)=\frac{1}{n}{\displaystyle \sum _{i=1}^n{({\chi }_i-{\mu }_0)}^2}\text{and}{\widehat{\sigma }}_{MRE}^2({\mu }_0)=\frac{1}{n+2}{\displaystyle \sum _{i=1}^n{({\chi }_i-{\mu }_0)}^2.}$$ (4)

Incidentally, ${\widehat{\sigma }}_{UMVU}^2({\mu }_0)$ is also the minimax estimator of σ² since it is a limit of proper Bayes estimators and it has constant risk. Clearly, we are able to improve on the estimators derived under ℳ by exploiting the knowledge that μ = μ₀ under ℳ₀: when model ℳ₀ holds, the relative efficiency of the estimator ${\widehat{\sigma }}_{UMVU}^2({\mu }_0)$ in (4) with respect to ${\widehat{\sigma }}_{UMVU}^2$ in (1) is n/(n - 1). But suppose now that we have a model between ℳ and ℳ₀. Specifically, let p be a known positive integer, and μ = {μ₁, μ₂,…,μ_p} be a set of known real numbers, and consider the estimation of σ² under the model

$${\mathit{ℳ}}_p={\mathit{ℳ}}_p(\mu )=\{N(\mu ,{\sigma }^2):(\mu ,{\sigma }^2)\in {\mathrm{\Theta }}_p\equiv \{{\mu }_1,{\mu }_2,\dots ,{\mu }_p\}\times {\Re }_{+}\}.$$

In ℳ_p, in contrast to ℳ₀, there is some information about the possible value of μ, but we are not certain about this value. Model ℳ_p can be viewed as having p sub-models, with the ith sub-model, ℳ_p,i, being the normal class with unknown variance σ² and known mean μ_i, that is, ℳ_p,i = {N(μ_i, σ²) : σ² > 0}. This particular model arises in a variety of settings. For example, it includes decision problems with a two-element action space such as in the Neyman-Pearson hypothesis testing setting. If we further allow the possibility that μ ∈ ℜ \ {μ₁, μ₂,…,μ_p}, we obtain a generalization of the setting utilized by Stein (1964) to derive an estimator dominating ${\widehat{\sigma }}_{MRE}^2$ (see Brewster and Zidek (1974), Wallace (1977), and Maatta and Casella (1990)). The Stein estimator is given by

$${\widehat{\sigma }}_{ST}^2=\text{min}\left\{{\widehat{\sigma }}_{MRE}^2,\frac{1}{n+2}{\displaystyle \sum _{i=1}^n{\chi }_i^2}\right\}.$$ (5)

This estimator can be viewed as a preliminary test estimator (Sen and Saleh (1987), Lehmann and Casella (1998), and Sclove, Morris, and Radhakrishnan (1972)). The pre-test estimation approach proceeds by testing a null hypothesis that the parameter equals a certain value, and if it accepts the hypothesis then the estimator based on this parameter value is used; otherwise, an estimator under the general model is used. Since a test is to be performed, a level of significance needs to be specified, and so generally pre-test estimators will depend on such a specified level. Interestingly, the Stein estimator in (5), which could be derived as a pre-test estimator with hypothesis specifying that μ = 0, eliminates this dependence by utilizing an ‘optimal’ significance level. The approach implemented in our paper differs from that of pre-test estimation, since we altogether avoid the need for testing a hypothesis to derive our estimators.

Note that in problems dealing with the estimation of the normal variance, it is typically assumed that either model ℳ or model ℳ₀ holds. However, in many settings, the mean can take only a finite number of possible values, such as for example in the Neyman-Pearson lemma, where the variance estimator is typically the sample variance, which does not exploit the fact that there are only two possible means.

3 Classical Two-Step Estimators

Under ℳ_p the likelihood function for the sample realization χ = x = (x₁, x₂,…, x_n)′… is

$$L(\mu ,{\sigma }^2)=L(\mu ,{\sigma }^2|\mathit{x})={\displaystyle \prod _{i=1}^p{L}_i{({\mu }_i,{\sigma }^2)}^{M_i}}$$ (6)

where, for i = 1, 2,…, p, with I{·} denoting the indicator function and M_i = I{μ = μ_i},

$$L_i({\mu }_i,{\sigma }^2)={\left(\frac{1}{\sqrt{2\pi }}\right)}^n{\left(\frac{1}{{\sigma }^2}\right)}^{\frac{n}{2}}\text{exp}\left\{-\frac{n{\widehat{\sigma }}_i^2}{2{\sigma }^2}\right\}\text{and}{\widehat{\sigma }}_i^2=\frac{1}{n}{\displaystyle \sum _{j=1}^n{(x_j-{\mu }_i)}^2.}$$ (7)

L_i(μ_i, σ²) is maximized with respect to σ² at ${\widehat{\sigma }}_i^2$, so $L_i({\mu }_i,{\widehat{\sigma }}_i^2)={\text{sup}}_{{\sigma }^2\in {\Re }_{+}}{L}_i({\mu }_i,{\sigma }^2)$. Define the likelihood-based model selector $\widehat{M}=\widehat{M}(\mathit{\chi })$ via

$$\widehat{M}=\text{arg}\underset{1\le i\le p}{\text{max}}{L}_i({\mu }_i,{\widehat{\sigma }}_i^2)=\text{arg}\underset{1\le i\le p}{\text{min}}{\widehat{\sigma }}_i^2=\text{arg}\underset{1\le i\le p}{\text{min}}|\overline{\chi }-{\mu }_i|.$$

One could employ model selectors different from $\widehat{M}$, such as the highest posterior probability (à la Schwarz’ Bayesian criterion (SBC) (Schwarz (1978)) or the Akaike information criterion (AIC) (Akaike (1973), Burnham and Anderson (1998))). In this paper we restrict our attention to the intuitive selector $\widehat{M}$, which could actually be viewed also as a highest posterior probability model selector associated with a flat prior distribution. The maximum likelihood estimator (MLE) of σ² under ℳ_p is

$${\widehat{\sigma }}_{p,MLE}^2={\widehat{\sigma }}_{\widehat{M}}^2={\displaystyle \sum _{i=1}^{p}I\{\widehat{M}=i\}{\widehat{\sigma }}_i^2},$$ (8)

a two-step estimator, with the first stage selecting the sub-model and the second-stage using the MLE of σ² in the chosen sub-model. An alternative to the estimator (8) is to use the sub-model’s MRE instead of MLE of σ²:

$${\widehat{\sigma }}_{p,MRE}^2={\widehat{\sigma }}_{\mathit{\text{M RE}},\widehat{M}}^2={\displaystyle \sum _{i=1}^{p}I\{\widehat{M}}=i\}{\widehat{\sigma }}_{MRE,i}^2={\displaystyle \sum _{i=1}^{p}I\{\widehat{M}}=i\}\frac{n{\widehat{\sigma }}_i^2}{(n+2)}.$$ (9)

Note that the label ‘_p,MRE’ (and similar labels in the sequel) is a misnomer since this estimator need not be minimum risk equivariant under model ℳ_p. However, we keep the name for clarity.

4 Bayes and Weighted Estimators

We focus on the class of prior densities of (μ, σ²) which consists of the product of a multinomial probability function and an inverse gamma density:

$$\pi (\mu ,{\sigma }^2|\tilde{\mathit{\theta }},\kappa ,\beta )=({\displaystyle \prod _{i=1}^p{\tilde{\theta }}_i^{m_i}})\frac{{\beta }^{\kappa -1}}{\mathrm{\Gamma }(\kappa -1)}{\left(\frac{1}{{\sigma }^2}\right)}^{\kappa }\text{exp}(-\frac{\beta }{{\sigma }^2}),$$ (10)

where σ² > 0, m_i = I{μ = μ_i} so that ${\displaystyle {\sum }_{i=1}^p{m}_i=1},\text{and}0\le {\tilde{\theta }}_i\le 1\text{with}{\displaystyle {\sum }_{i=1}^p{\tilde{\theta }}_i=1,\beta >0}$, and κ > 1. From (10) and (6), we obtain the posterior density of (μ, σ²) given χ = x:

$$\pi (\mu ,{\sigma }^2|\mathit{x})=C{{\displaystyle \prod _{i=1}^p\left\{{\tilde{\theta }}_i{\left(\frac{1}{{\sigma }^2}\right)}^{\frac{n}{2}+\kappa }\text{exp}\left(-\frac{1}{{\sigma }^2}\left[\frac{n{\widehat{\sigma }}_i^2}{2}+\beta \right]\right)\right\}}}^{m_i}.$$ (11)

Note that in the “vectorized form,” and with a slight abuse of notation, π(μ, σ²|x) = π(m, σ²|x) because {μ = μ_i} = {m = 1_i}, where 1_i is an n × 1 vector with ith component equal to 1 and all others equal 0. It follows that

$$C=\frac{1}{\mathrm{\Gamma }(n/2+\kappa -1)}{\left\{{\displaystyle \sum _{i=1}^p\frac{{\tilde{\theta }}_i}{{(n{\widehat{\sigma }}_i^2/2+\beta )}^{n/2+\kappa -1}}}\right\}}^{-1}.$$ (12)

4.1 Posterior Probabilities

From the posterior distribution in (11), the marginal posterior density (with respect to counting measure) of μ, or equivalently of m, is

$$\pi (m|\mathbf{x})=C{\displaystyle \prod _{i=1}^p{\left\{{\tilde{\theta }}_i\frac{\mathrm{\Gamma }(n/2+\kappa -1)}{{(n{\widehat{\sigma }}_i^2/2+\beta )}^{n/2+\kappa -1}}\right\}}^{m_i}}={\displaystyle \prod _{i=1}^p{\{{\theta }_i(\kappa ,\beta ,n,\mathbf{x})\}}^{m_i},}$$

where, for i = 1, 2,…, p, the posterior probability that the sub-model ℳ_p,i is true, is

$${\theta }_i(\kappa ,\beta ,n,\mathit{x})=\frac{{\tilde{\theta }}_i{(n{\widehat{\sigma }}_i^2/2+\beta )}^{-(n/2+\kappa -1)}}{{\displaystyle {\sum }_{j=1}^p{\tilde{\theta }}_j}{(n{\widehat{\sigma }}_j^2/2+\beta )}^{-(n/2+\kappa -1)}}.$$ (13)

Note, as expected, that if θ̃_i > 0 and ℳ_p,i is the true sub-model, θ_i(κ, β, n, χ), when viewed as a function of χ with (μ, σ²) fixed, converges to 1 with probability one (wp1) as n → ∞. This is because if ℳ_p,i is the correct model, ${\widehat{\sigma }}_i^2$ converges wp1 to σ² by the strong law of large numbers (SLLN); whereas, for $\mathit{i\prime }\ne i,{\widehat{\sigma }}_{\mathit{i\prime }}^2$ converges wp1 to σ² + (μ_i - μ_i′)².

4.2 Estimators

The marginal posterior density function of σ² is directly obtained from (11) to be

$$\pi ({\sigma }^2|\mathit{x})=C{\displaystyle \sum _{i=1}^p{\tilde{\theta }}_i}{\left(\frac{1}{{\sigma }^2}\right)}^{(\kappa +n/2)}\text{exp}\left[-\frac{1}{{\sigma }^2}(\frac{n{\widehat{\sigma }}_i^2}{2}+\beta )\right]I\{{\sigma }^2>0\}.$$ (14)

The posterior mean, which is the Bayes estimator of σ² under the loss function L in (2), is then

$${\widehat{\sigma }}_{p,\mathit{\text{Bayes}}}^2(\kappa ,\beta ,\theta )={\displaystyle \sum _{i=1}^p{\theta }_i}(\kappa ,\beta ,n,\mathit{x})\left\{\left(\frac{n}{n+2(\kappa -2}\right){\widehat{\sigma }}_i^2+\left(\frac{2(\kappa -2)}{n+2(\kappa -2)}\right)\left(\frac{\beta }{\kappa -2}\right)\right\}.$$ (15)

Note that β/(κ − 2) is the prior mean of σ², provided κ > 2 (the condition also needed for the prior variance of σ² to exist), whereas ${\widehat{\sigma }}_i^2$ is the MLE of σ² under the ℳ_p,i model. This estimator mixes in a data-dependent manner, using the posterior probabilities of the p sub-models, the Bayes estimators of σ² from each sub-model. Furthermore, the Bayes estimator of σ² for the ℳ_p,i sub-model is a convex combination of the ℳ_p,i-model MLE and the prior mean of σ², a well-known result.

To obtain limiting Bayes estimators for σ², we consider improper priors arising by setting θ̃_i = 1/p, i = 1, 2, …, p, and β → 0. We examine four κ values: κ → 1, κ → 3/2, κ = 2, and κ = 3. The rationale for these choices is as follows: κ → 1 amounts to placing Jeffreys’ non-informative prior on σ² in each of the p sub-models, since Jeffreys’ prior for σ² (with mean known) is proportional to 1/σ² (Robert (2001)); κ → 3/2 corresponds to the Jeffreys’ prior for σ² when the mean is unknown in the normal model, since in this case Jeffreys’ prior is proportional to (1/σ²)^(3/2); κ = 2 and κ = 3 produce (limiting) Bayes estimators that are convex combinations of the sub-models’ MLEs and MREs, respectively. Table 1 lists the sub-models’ posterior probabilities and the resulting limiting Bayes estimators of σ². Each of the sub-models’ posterior probabilities associated with κ ∈ {1, 3/2, 2, 3} given in Table 1 could also be utilized to form estimators which are convex combinations of the sub-models’ MREs. These new estimators need not however be limiting Bayes with respect to our class of priors. These ‘weighted’ estimators are defined as:

$${\widehat{\sigma }}_{p,PLB1}^2=\left(\frac{n-2}{n+2}\right){\widehat{\sigma }}_{p,LB1}^2;{\widehat{\sigma }}_{p,PLB2}^2=\left(\frac{n-2}{n+2}\right){\widehat{\sigma }}_{p,LB2}^2;{\widehat{\sigma }}_{p,PLB3}^2=\left(\frac{n}{n+2}\right){\widehat{\sigma }}_{p,LB3}^2.$$ (16)

Table 1.

Sub-models’ posterior probabilities and limiting Bayes estimators of σ² for different values of κ when θ̃_i = 1/p and β → 0.

κ	Sub-model Posterior Probabilities, θi(κ, 0, n, x), i = 1, 2, …, p	Limiting Bayes Estimator $${\widehat{\sigma }}_{p,LBk}^2,k=1,2,3,4$$
1	$${\theta }_{i1}={({\widehat{\sigma }}_i^2)}^{-n/2}/{\displaystyle {\sum }_{j=1}^p{({\widehat{\sigma }}_j^2)}^{-n/2}}$$	$${\widehat{\sigma }}_{p,LB1}^2=(\frac{n}{n-2}){\displaystyle {\sum }_{i=1}^p{\theta }_{i1}{\widehat{\sigma }}_i^2}$$
3/2	$${\theta }_{i2}={({\widehat{\sigma }}_i^2)}^{-(n+1)/2}/{\displaystyle {\sum }_{j=1}^p{({\widehat{\sigma }}_j^2)}^{-(n+1)/2}}$$	$${\widehat{\sigma }}_{p,LB2}^2=(\frac{n}{n-1}){\displaystyle {\sum }_{i=1}^p{\theta }_{i2}{\widehat{\sigma }}_i^2}$$
2	$${\theta }_{i3}={({\widehat{\sigma }}_i^2)}^{-(n+2)/2}/{\displaystyle {\sum }_{j=1}^p{({\widehat{\sigma }}_j^2)}^{-(n+2)/2}}$$	$${\widehat{\sigma }}_{p,LB3}^2={\displaystyle {\sum }_{i=1}^p{\theta }_{i3}{\widehat{\sigma }}_i^2}$$
3	$${\theta }_{i4}={({\widehat{\sigma }}_i^2)}^{-(n+4)/2}/{\displaystyle {\sum }_{j=1}^p{({\widehat{\sigma }}_j^2)}^{-(n+4)/2}}$$	$${\widehat{\sigma }}_{p,LB4}^2=(\frac{n}{n+2}){\displaystyle {\sum }_{i=1}^p{\theta }_{i4}{\widehat{\sigma }}_i^2}$$

Note also from (15) that the estimators ${\tilde{\sigma }}_{LB,i}^2=(n/(n-2)){\widehat{\sigma }}_i^2$, the ones whose convex combination is being formed in ${\widehat{\sigma }}_{p,LB1}^2$, are the limiting Bayes estimators of σ² for each of the p sub-models under Jeffreys’ non-informative prior when the sub-model’s mean is known (arising from κ → 1). The estimators in Table 1 and in (16) have different flavors than the MLE of σ² given in (8): in the latter, we choose one among the p estimators of σ², while the Bayes and weighted estimators are mixing sub-model estimators according to the sub-models’ posterior probabilities.

Finally, we define the two-step estimator based on the sub-models’ limiting Bayes estimators:

$${\widehat{\sigma }}_{p,ALB}^2={\tilde{\sigma }}_{LB,\widehat{M}}^2=\left(\frac{n}{n-2}\right){\displaystyle \sum _{i=1}^{p}I\{\widehat{M}=i\}{\widehat{\sigma }}_i^2.}$$ (17)

This belongs to the same class of estimators as ${\widehat{\sigma }}_{p,MLE}^2$ and ${\widehat{\sigma }}_{p,MRE}^2$, differing just in the multipliers which are functions of n only. Note that for the purposes of obtaining risk functions, it suffices to derive formulas for the mean and variance functions of ${\widehat{\sigma }}_{p,MLE}^2$.

5 Comparison of Estimators

The goal of this section is to compare the performances of the estimators given in Table 1 and (16), with the estimators developed under ℳ $({\widehat{\sigma }}_{UMVU}^2\text{and}{\widehat{\sigma }}_{MRE}^2)$, with the two-step estimators $({\widehat{\sigma }}_{p,MLE}^2,{\widehat{\sigma }}_{p,MRE}^2,{\widehat{\sigma }}_{p,MLE}^2,\text{and}{\widehat{\sigma }}_{p,ALB}^2)$ and, for completeness, with the Stein estimator in (5). Performance will be measured by their risk functions arising from the loss function L in (2). In particular, we address the following questions: (i) How much efficiency is lost by using the estimators developed under the wider model ℳ when model ℳ_p holds? (ii) How do the limiting Bayes and weighted estimators ${\widehat{\sigma }}_{p,LBk}^2$ and ${\widehat{\sigma }}_{p,PLBk}^2$ compare with the ℳ_p MLE-based and MRE-based estimators? (iii) Do the advantages of the ℳ_p-based estimators over ℳ-based estimators decrease as the dimension p increases and/or the spacings among the μ₁,…,μ_p decrease?

5.1 Distributional Representations

It is well-known that, provided n > 1, $(n-1){\widehat{\sigma }}_{UMVU}^2/{\sigma }^2\sim {\chi }_{n-1}^2,\text{so}\mathbf{E}\{{\widehat{\sigma }}_{UMVU}^2/{\sigma }^2\}=1\text{and}\mathbf{\text{Var}}\{{\widehat{\sigma }}_{UMVU}^2/{\sigma }^2\}=2/(n-1)$. Therefore, the risk function of ${\widehat{\sigma }}_{UMVU}^2$ with respect to the loss function L in (2) is $R({\widehat{\sigma }}_{UMVU}^2,(\mu ,{\sigma }^2))=2/(n-1)$. By exploiting the relationship between ${\widehat{\sigma }}_{UMVU}^2$ and ${\widehat{\sigma }}_{MRE}^2$, the risk function of the latter is easily found to be $R({\widehat{\sigma }}_{MRE}^2,(\mu ,{\sigma }^2))=2/(n+1)$. This demonstrates the known fact that ${\widehat{\sigma }}_{UMVU}^2$ is inadmissible. To compare estimator performances, we will use ${\widehat{\sigma }}_{UMVU}^2$ as the baseline, so the efficiency of an estimator σ̂² will be given by

$$\text{Eff}({\widehat{\sigma }}^2:{\widehat{\sigma }}_{UMVU}^2)=\frac{R({\widehat{\sigma }}_{UMVU}^2,(\mu ,{\sigma }^2))}{R({\widehat{\sigma }}^2,(\mu ,{\sigma }^2))}.$$ (18)

Thus, in particular, $\text{Eff}({\widehat{\sigma }}_{MRE}^2:{\widehat{\sigma }}_{UMVU}^2)=(n+1)/(n-1)=1+2/(n-1)$.

We present some distributional properties of the estimators which will be used to derive the exact expressions of the risk functions of ${\widehat{\sigma }}_{p,MLE}^2$, and second-order approximations to the risk functions of ${\widehat{\sigma }}_{p,LBk}^2$ and ${\widehat{\sigma }}_{p,PLBk}^2$. Let Z ∼ N(0,1) and Z = (Z₁, Z₂,…,Z_n)′ ∼ N_n(0, I). For the vector of means μ = (μ₁,μ₂,…,μ_p)′ with μ_i0 being the true mean (i₀ ∈ {1,2,…,p}), we let

$$\mathbf{\Delta }\equiv \mathbf{\Delta }(\mu ,\sigma )=\frac{\mu -{\mu }_{i_0}1}{\sigma }$$ (19)

where 1 = (1,1,…,1)′. Note that this will always have a zero component under ℳ_p. In the sequel, the ‘equal-in-distribution’ relation is denoted by $‘\underset{¯}{\underset{¯}{d}}’$. To achieve a more fluid presentation, formal proofs of lemmas, propositions, theorems, and corollaries are relegated to Appendix A.

Proposition 5.1

Under ℳ_p with μ_i0 the true mean, $n{\widehat{\sigma }}_i^2/{\sigma }^2\underset{¯}{\underset{¯}{d}}W+V_i^2,i=1,2,\dots ,p,$, where $W\sim {\chi }_{n-1}^2,\mathit{V}=(V_1,V_2,\dots ,V_p{)}^{\prime }\sim N_p(-\sqrt{n}\mathbf{\Delta },\mathit{J}\equiv 11\prime )$, and W and V are independent.

From Proposition 5.1, by exploiting the independence between W and V and using the iterated expectation and covariance rules, and by noting that

$$\mathbf{E}\left\{W^{k/2}\right\}={(1/2)}^{-k/2}[\mathrm{\Gamma }((n+k-1)/2)/\mathrm{\Gamma }((n-1)/2)]$$

holds for any k < n − 1, the following corollary immediately follows.

Corollary 5.1

Under the conditions of Proposition 5.1, $n{\widehat{\sigma }}_i^2/{\sigma }^2\stackrel{d}{=}W(1+T_i^2),i=1,2,\dots ,p,$ with $\mathit{T}=(T_1,\dots ,T_p)\prime =\mathit{V}/\sqrt{W}$. The distribution of T depends on (μ, σ²) only through Δ and, provided that n > 3, the mean vector and covariance matrix of T are given, respectively, by

$$\mathbf{E}(\mathit{T})=\mathit{\nu }\equiv -\mathbf{\Delta }{C}_n\mathit{\text{and}}\mathbf{\text{Cov}}(\mathit{T},\mathit{T})=\frac{1}{n-3}\mathit{J}+(\frac{n}{(n-3)C_n^2}-1){\mathit{\nu }}^{\otimes 2}$$

with $C_n=\sqrt{n/2}[\mathrm{\Gamma }((n-2)/2)/\mathrm{\Gamma }((n-1)/2)]$.

5.2 Representation and Risk Function of ${\widehat{\sigma }}_{p,MLE}^2$

We now give a representation of ${\widehat{\sigma }}_{p,MLE}^2$ and obtain the exact expressions for its mean, variance, and risk. For a given Δ, let Δ₍₁₎ < Δ₍₂₎ < … < Δ_(p) denote the associated ordered values.

Theorem 5.1

Let μ_i0 be the true mean. Then under ℳ_p,

$$n{\widehat{\sigma }}_{p,MLE}^2/{\sigma }^2\stackrel{d}{=}W+{\displaystyle \sum _{i=1}^{p}I\{L({\mathrm{\Delta }}_{(i)}},\mathbf{\Delta })<Z<U({\mathrm{\Delta }}_{(i)},\mathbf{\Delta })\}(Z-\sqrt{n}{\mathrm{\Delta }}_{(i)}{)}^2$$

where, under the convention that Δ₍₀₎ = −∞ and Δ_(p+1) = +∞,

$$L({\mathrm{\Delta }}_{(i)},\mathbf{\Delta })=(\sqrt{n}/2)[{\mathrm{\Delta }}_{(i)}+{\mathrm{\Delta }}_{(i-1)}]\mathit{\text{and}}U({\mathrm{\Delta }}_{(i)},\mathbf{\Delta })=(\sqrt{n}/2)[{\mathrm{\Delta }}_{(i)}+{\mathrm{\Delta }}_{(i+1)}],$$

$W\sim {\chi }_{n-1}^2,Z\sim N(0,1)$ and W and Z are independent.

Define the events Ω_(i) = {L(Δ_(i), Δ) < Z < U(Δ_(i),Δ)}, i = 1, 2, …, p. The collection of sets $\left\{\right\{\widehat{M}=i\},i=1,2,\dots ,p\}$ is in one-to-one correspondence with the collection {Ω₍₁₎, Ω₍₂₎, …, Ω_(p)}, as can be seen from Theorem 5.1, and thus Ω_(i), i = 1, 2, …, p, are disjoint. Using Theorem 5.1, we can now obtain expressions of the mean and variance of ${\widehat{\sigma }}_{p,MLE}^2$. For i = 1, 2, …, p, we let

$$P_{(i)}(\mathbf{\Delta })\equiv \text{Pr}\{{\mathrm{\Omega }}_{(i)}\}=\mathrm{\Phi }(\frac{\sqrt{n}}{2}({\mathrm{\Delta }}_{(i)}+{\mathrm{\Delta }}_{(i+1)}))-\mathrm{\Phi }(\frac{\sqrt{n}}{2}({\mathrm{\Delta }}_{(i)}+{\mathrm{\Delta }}_{(i-1)})),$$ (20)

where Φ(·) is the standard normal distribution function. In the sequel, we let ϕ(·) denote the density function of a standard normal random variable.

Theorem 5.2

Under the conditions of Theorem 5.1,

$$\begin{array}{ccc}EpMLE(\mathbf{\Delta })\hfill & \equiv \hfill & \mathbf{E}\left\{\frac{{\widehat{\sigma }}_{p,MLE}^2}{{\sigma }^2}\right\}=1-\frac{2}{\sqrt{n}}{\displaystyle \sum _{i=1}^p{\mathrm{\Delta }}_{(i)}[\varphi (L({\mathrm{\Delta }}_{(i)},\mathbf{\Delta }))-\varphi (U({\mathrm{\Delta }}_{(i)},\mathbf{\Delta }))]+}\hfill \\ \hfill & \hfill & {\displaystyle \sum _{i=1}^p{\mathrm{\Delta }}_{(i)}^2}[\mathrm{\Phi }(U({\mathrm{\Delta }}_{(i)},\mathbf{\Delta }))-\mathrm{\Phi }(L({\mathrm{\Delta }}_{(i)},\mathbf{\Delta }))].\hfill \end{array}$$

Next, we present an expression for the variance function of the estimator ${\widehat{\sigma }}_{p,MLE}^2$. Towards this end, we introduce some notation to simplify the presentation. For k ∈ Ƶ₊ = {0, 1, 2, …}, define

$$\xi (k;{\mathrm{\Omega }}_{(i)})\equiv \mathbf{E}\left\{Z^{k}I({\mathrm{\Omega }}_{(i)})\right\}={\displaystyle {\int }_{L({\mathrm{\Delta }}_{(i)},\mathbf{\Delta })}^{U({\mathrm{\Delta }}_{(i)},\mathbf{\Delta })}{z}^k}\varphi (z)dz.$$

Using this, observe that by the binomial expansion, for m ∈ Ƶ₊,

$${\zeta }_{(i)}(m)\equiv \mathbf{E}\left\{I({\mathrm{\Omega }}_{(i)})(Z-\sqrt{n}{\mathrm{\Delta }}_{(i)}{)}^m\right\}={\displaystyle \sum _{k=0}^m{(-1)}^{(m-k)}}(\begin{array}{c}m\hfill \\ k\hfill \end{array})(\sqrt{n}{\mathrm{\Delta }}_{(i)}{)}^{(m-k)}\xi (k;{\mathrm{\Omega }}_{(i)}).$$ (21)

To compute the quantity ξ(k;Ω_(i)), observe that for k ∈ Ƶ₊ and t ∈ ℜ,

$${\displaystyle {\int }_{-\mathrm{\infty }}^t{z}^k\varphi (z)dz={(-1)}^k}\frac{2^{(k-1)/2}}{\sqrt{2\pi }}\mathrm{\Gamma }((k+1)/2)\times \{\begin{array}{cc}\text{Pr}\{{\mathrm{\chi }}_{k+1}^2>t^2\}\hfill & \text{if}t<0\hfill \\ \left[1+{(-1)}^k\text{Pr}\{{\mathrm{\chi }}_{k+1}^2<t^2\}\right]\hfill & \text{if}t\ge 0\hfill \end{array}.$$

Using the above formulas, we obtain ξ(k;Ω_(i)) according to

$$\xi (k;{\mathrm{\Omega }}_{(i)})={\displaystyle {\int }_{-\mathrm{\infty }}^{U({\mathrm{\Delta }}_{(i)},\mathbf{\Delta })}{z}^k\varphi (z)dz-{\displaystyle {\int }_{-\mathrm{\infty }}^{L({\mathrm{\Delta }}_{(i)},\mathbf{\Delta })}{z}^k\varphi (z)dz.}}$$ (22)

Theorem 5.3

Under the conditions of Theorem 5.1,

$$VpMLE(\mathrm{\Delta })\equiv \text{Var}\left\{\frac{{\widehat{\sigma }}_{p,MLE}^2}{{\sigma }^2}\right\}=\frac{1}{n}\left\{2(1-\frac{1}{n})+\frac{1}{n}\left[{\displaystyle \sum _{i=1}^p{\zeta }_{(i)}(4)-{({\displaystyle \sum _{i=1}^p{\zeta }_{(i)}(2)})}^2}\right]\right\}.$$

In the situation where there are only two sub-models so p = 2, the expressions for the mean and variance functions of ${\widehat{\sigma }}_{p,MLE}^2/{\sigma }^2$ simplify. These simplified forms are provided in the following corollary. The proofs of these results are straightforward, hence to conserve space, we omit them but instead refer the reader to the more detailed technical report by Dukić and Peña (2003).

Corollary 5.2

If p = 2 so Δ = (0; Δ), then under the conditions of Theorem 5.1,

$$\begin{array}{ccc}EpMLE(\mathrm{\Delta })\hfill & =\hfill & 1-\left(\frac{2}{\sqrt{n}}|\mathrm{\Delta }|\right)\left\{\varphi \left(\frac{\sqrt{n}}{2}\left|\mathrm{\Delta }\right|\right)-\left(\frac{\sqrt{n}}{2}\left|\mathrm{\Delta }\right|\right)\left[1-\mathrm{\Phi }(\frac{\sqrt{n}}{2}\left|\mathrm{\Delta }\right|)\right]\right\};\hfill \\ VpMLE(\mathrm{\Delta })\hfill & =\hfill & \frac{2}{n}+|\mathrm{\Delta }{|}^4\mathrm{\Phi }(\frac{\sqrt{n}}{2}\left|\mathrm{\Delta }\right|)\left[1-\mathrm{\Phi }(\frac{\sqrt{n}}{2}\left|\mathrm{\Delta }\right|)\right]-\frac{4}{\sqrt{n}}|\mathrm{\Delta }{|}^3\mathrm{\Phi }(\frac{\sqrt{n}}{2}\left|\mathrm{\Delta }\right|)\times \hfill \\ \hfill & \hfill & {\displaystyle {\int }_{\frac{\sqrt{n}}{2}\left|\mathrm{\Delta }\right|}^{\mathrm{\infty }}z\varphi (z)dz-\frac{4}{n^{3/2}}\left|\mathrm{\Delta }\right|\left\{{\displaystyle {\int }_{\frac{\sqrt{n}}{2}\left|\mathrm{\Delta }\right|}^{\mathrm{\infty }}{z}^3}\varphi (z)dz-{\displaystyle {\int }_{\frac{\sqrt{n}}{2}\left|\mathrm{\Delta }\right|}^{\mathrm{\infty }}z}\varphi (z)dz\right\}}+\hfill \\ \hfill & \hfill & \frac{1}{n}|\mathrm{\Delta }{|}^2\left\{6{\displaystyle {\int }_{\frac{\sqrt{n}}{2}\left|\mathrm{\Delta }\right|}^{\mathrm{\infty }}{z}^2}\varphi (z)dz-4{({\displaystyle {\int }_{\frac{\sqrt{n}}{2}\left|\mathrm{\Delta }\right|}^{\mathrm{\infty }}z}\varphi (z)dz)}^2-2\left[1-\mathrm{\Phi }(\frac{\sqrt{n}}{2}\left|\mathrm{\Delta }\right|)\right]\right\}.\hfill \end{array}$$

We note from the expression in Corollary 5.2 that, for a fixed n, lim_|Δ|→0 EpMLE(Δ) → 1 and ${\text{lim}}_{\left|\mathrm{\Delta }\right|\to 0}\mathrm{V}\mathrm{p}\mathrm{M}\mathrm{L}\mathrm{E}(\mathrm{\Delta })\to 2/n=\mathbf{\text{Var}}\{{\widehat{\sigma }}_{MLE}^2/{\sigma }^2\},\text{where}{\widehat{\sigma }}_{MLE}^2=\frac{1}{n}{\displaystyle {\sum }_{i=1}^n{({\chi }_i-{\mu }_{i_0})}^2}$, the ML (also UMVU, minimax) estimator of σ² under the true model. Also, for a fixed Δ, we see that lim_n→∞ EpMLE(Δ) → 1 and lim_n→∞ {n(VpMLE(Δ))} → 2.

The next result in Corollary 5.3 shows that even though the sub-models’ MLEs are each unbiased for σ², the two-step estimator ${\widehat{\sigma }}_{p,MLE}^2$, which employs the MLE of the sub-model selected by the model selector $\widehat{M}$, is a negatively biased estimator of σ². The result is an immediate consequence of Corollary 5.2 by noting that the continuous function g(u) = ϕ(u) − u[1 − Φ(u)] is positive by virtue of the facts that lim_u↓0h(u) > 0, lim_u→∞ h(u) = 0, and g′(u) = ϕ′(u) + uϕ(u) − [1 − Φ(u)] = −[1 − Φ(u)] < 0 since ϕ′(u) = −uϕ(u).

Corollary 5.3

Under the conditions of Corollary 5.2 with Δ ≠ 0, $\mathbf{E}\left\{{\widehat{\sigma }}_{p,MLE}^2\right\}<{\sigma }^2$ that is, ${\widehat{\sigma }}_{p,MLE}^2$ is negatively biased for σ².

Now that we have the exact expressions for the mean and variance of ${\widehat{\sigma }}_{p,MLE}^2/{\sigma }^2$, we could obtain the risk function of ${\widehat{\sigma }}_{p,MLE}^2$ under ℳ_p and loss L in (2) as

$$R({\widehat{\sigma }}_{p,MLE}^2,({\mu }_{i_0},{\sigma }^2))=\mathrm{V}\mathrm{p}\mathrm{M}\mathrm{L}\mathrm{E}(\mathrm{\Delta })+{[\mathrm{E}\mathrm{p}\mathrm{M}\mathrm{L}\mathrm{E}(\mathrm{\Delta })-1]}^2.$$ (23)

Finally, for ${\widehat{\sigma }}_{p,MLE}^2$, we address the question of what happens when p increases and the spacings in Δ decrease. This will indicate whether we will lose the advantage of ℳ_p-based estimators over ℳ-based estimators. The proof of Theorem 5.4 is rather lengthy and hence omitted; instead we refer the reader to the technical report by Dukić and Peña (2003).

Theorem 5.4

Given n fixed, if p → ∞, max_2≤i≤p |Δ_(i) − Δ_(i−1)| → 0, with Δ₍₁₎ → Δ_min ∈ (−∞, 0], and Δ_(p) → Δ_max ∈ [0, ∞), then

$$\begin{array}{c}EpMLE(\mathrm{\Delta })\to 1-\frac{1}{n}{\displaystyle {\int }_{\sqrt{n}{\mathrm{\Delta }}_{\mathit{\text{min}}}}^{\sqrt{n}{\mathrm{\Delta }}_{\mathit{\text{max}}}}{w}^2}\varphi (w)dw+\frac{2}{\sqrt{n}}\left\{{\mathrm{\Delta }}_{\mathit{\text{min}}}\varphi (\sqrt{n}{\mathrm{\Delta }}_{\mathit{\text{min}}})-{\mathrm{\Delta }}_{\mathit{\text{max}}}\varphi (\sqrt{n}{\mathrm{\Delta }}_{\mathit{\text{max}}})\right\}+\hfill \\ \left\{{({\mathrm{\Delta }}_{\mathit{\text{min}}})}^2\mathrm{\Phi }(\sqrt{n}{\mathrm{\Delta }}_{\mathit{\text{min}}})+{({\mathrm{\Delta }}_{\mathit{\text{max}}})}^2[1-\mathrm{\Phi }(\sqrt{n}{\mathrm{\Delta }}_{\mathit{\text{max}}})]\right\};\hfill \end{array}$$

$$\begin{array}{c}VpMLE(\mathbf{\Delta })\to \frac{2}{n}\left(1-\frac{1}{n}\right)+\hfill \\ \frac{1}{n^2}[\left\{\mathbf{E}[{(Z-\sqrt{n}{\mathrm{\Delta }}_{\mathit{\text{min}}})}^{4}I(Z<\sqrt{n}{\mathrm{\Delta }}_{\mathit{\text{min}}})]+\mathbf{E}[{(Z-\sqrt{n}{\mathrm{\Delta }}_{\mathit{\text{max}}})}^{4}I(Z>\sqrt{n}{\mathrm{\Delta }}_{\mathit{\text{max}}})]\right\}-\hfill \\ {\left\{\mathbf{E}[{(Z-\sqrt{n}{\mathrm{\Delta }}_{\mathit{\text{min}}})}^{2}I(Z<\sqrt{n}{\mathrm{\text{Δ}}}_{\mathit{\text{min}}})]+\mathbf{E}[{(Z-\sqrt{n}{\mathrm{\Delta }}_{\mathit{\text{max}}})}^{2}I(Z>\sqrt{n}{\mathrm{\Delta }}_{\mathit{\text{max}}})]\right\}}^2].\hfill \end{array}$$

Letting Δ_min → − ∞ and Δ_max → ∞, EpMLE(Δ) → 1 − 1/n and VpMLE(Δ) → (2/n) (1 − 1/n).

Using Theorem 5.4 we could now address the issue of whether we lose the advantage by utilizing the two-step estimator which was developed under model ℳ_p over the estimator developed under the more general model ℳ when p increases. For this purpose we have the following corollary.

Corollary 5.4

With n > 1 fixed, if as p → ∞, max_2≤i≤p |Δ_(i) − Δ_(i−1)| → 0, Δ₍₁₎ → −∞, and Δ_(p) → ∞, then (i) $\mathit{\text{Eff}}({\widehat{\sigma }}_{p,MLE}^2:{\widehat{\sigma }}_{UMVU}^2)\to 2n^2/[(n-1)(2n-1)]>1$; (ii) $\mathit{\text{Eff}}({\widehat{\sigma }}_{p,MRE}^2:{\widehat{\sigma }}_{UMVU}^2)\to 2{(n+2)}^2/[(n-1)(2n+7)]>1$; (iii) $\mathit{\text{Eff}}({\widehat{\sigma }}_{p,MRE}^2:{\widehat{\sigma }}_{p,MLE}^2)\to (2n-1){(n+2)}^2/[(2n+7)n^2]>1$; and (iv) $\mathit{\text{Eff}}({\widehat{\sigma }}_{p,MRE}^2:{\widehat{\sigma }}_{MRE}^2)\to 2{(n+2)}^2/[(n+1)(2n+7)]<1$. In addition, ${\widehat{\sigma }}_{p,ALB}^2$ is dominated by ${\widehat{\sigma }}_{UMVU}^2$.

The fourth result in Corollary 5.4 indicates that when the number of sub-models increases indefinitely the estimator ${\widehat{\sigma }}_{MRE}^2$ (which is the minimum risk estimator under the general model ℳ) dominates the two-step estimator ${\widehat{\sigma }}_{p,MRE}^2$ (which was developed by exploiting the sub-model structure of ℳ_p). Using the limiting results for p = 2 and as |Δ| → 0, stated after Corollary 5.2, we find that the limiting risk function of ${\widehat{\sigma }}_{p,MRE}^2$ is 2/(n+2), which is smaller than 2/(n+1), the risk function of ${\widehat{\sigma }}_{MRE}^2$. This shows that when the number of sub-models is small, we can gain efficiency by using the two-step estimator developed under model ℳ_p These results agree with our intuition: when the number of sub-models increases it is better to utilize the best estimator developed under the more general model. However, as it will be seen in the simulation studies reported later in the paper, the weighted and Bayes-type estimators’ performance seems not degraded by an increase in the number of sub-models.

5.3 Representation of Limiting Bayes and Weighted Estimators

We now provide distributional representations useful for the limiting Bayes estimators ${\widehat{\sigma }}_{p,LBk}^{2}s$ and the weighted estimators ${\widehat{\sigma }}_{p,PLBk}^{2}s$ under ℳ_p, in order to find an approximation to the risk functions of these estimators. For α > 0, define the “umbrella” estimator as

$${\widehat{\sigma }}_{p,LB}^2\equiv {\widehat{\sigma }}_{p,LB}^2(\alpha )={\displaystyle \sum _{i=1}^p\left\{\frac{{({\widehat{\sigma }}_i^2)}^{-\alpha }}{{\displaystyle {\sum }_{j=1}^p{({\widehat{\sigma }}_j^2)}^{-\alpha }}}\right\}}{\widehat{\sigma }}_i^2.$$ (24)

Individual estimators are easily derived from this umbrella estimator by choosing an appropriate α. For example:

$${\widehat{\sigma }}_{p,LB1}^2=(\frac{n}{n-2}){\widehat{\sigma }}_{p,LB}^2(n/2)\text{and}{\widehat{\sigma }}_{p,PLB1}^2=(\frac{n}{n+2}){\widehat{\sigma }}_{p,LB}^2(n/2).$$

Theorem 5.5

Under ℳ_p where μ_i0 is the true mean, for a fixed α > 0, $n{\widehat{\sigma }}_{p,LB}^2/{\sigma }^2\stackrel{d}{=}W(1+H(\mathit{T}))$, where

$$H(\mathit{T})\equiv H(\mathit{T};\alpha )={\displaystyle \sum _{i=1}^p{\theta }_i}(\mathit{T})T_i^2\mathit{\text{with}}{\theta }_i(\mathit{T})\equiv {\theta }_i(\mathit{T};\alpha )=\frac{{(1+T_i^2)}^{-\alpha }}{{\displaystyle {\sum }_{j=1}^p{(1+T_j^2)}^{-\alpha }}},i=1,2,\dots ,p.$$

Consequently, the distribution of ${\widehat{\sigma }}_{p,LB}^2/{\sigma }^2$ depends on (μ, σ²) only through Δ.

From the distributional representation in Theorem 5.5, a closed-form expression for the risk function of ${\widehat{\sigma }}_{p,LB}^2$ will be difficult to obtain because of the adaptive, i.e., data-dependent, nature of the mixing probabilities θ_i(T) and the fact that these are rational functions of T. To obtain an approximation to the risk function of ${\widehat{\sigma }}_{p,LB}^2$ we used a second-order Taylor expansion of the function H(T) about T = ν, the mean vector of T. For notation, let

$$H\equiv H(\mathit{\nu });{\mathit{H}}^{(1)}\equiv {\nabla }_{\mathit{T}}H(\mathit{T}){|}_{\mathit{T}=\mathit{\nu }};\text{and}{\mathit{H}}^{(2)}\equiv \frac{{\partial }^2}{\partial \mathit{T}\partial \mathit{T\prime }}H(\mathit{T}){|}_{\mathit{T}=\mathit{\nu }}.$$

A second-order Taylor approximation for ${\widehat{\sigma }}_{p,LB}^2/{\sigma }^2$ is provided by

$$\frac{{\widehat{\sigma }}_{p,LB}^2}{{\sigma }^2}\stackrel{d}{\approx }\frac{W}{n}\left\{1+H+{\mathit{H}}^{(1{)}^{\prime }}(\mathit{T}-\mathit{\nu })+\frac{1}{2}(\mathit{T}-\mathit{\nu }{)}^{\prime }{\mathit{H}}^{(2)}(\mathit{T}-\mathit{\nu })\right\}.$$ (25)

From this approximate representation, we are able to obtain approximate expressions for the mean and variance of the Bayes estimator. These mean and variance expressions, which involve the constant C_n defined in Corollary 5.1, are given in the next two theorems. The proofs require several intermediate results (contained in lemmas), and these are presented in Appendix A.

Theorem 5.6

Under ℳ_p, a second-order approximation to the mean of ${\widehat{\sigma }}_{p,LB}^2/{\sigma }^2$ is

$$E_2(\mathbf{\Delta })\equiv \left(1-\frac{1}{n}\right)(1+H)+\frac{1}{2}\left(\frac{1}{C_n^2}-1+\frac{3}{n}\right)(\mathit{\nu \prime }{\mathit{H}}^{(2)}\mathit{\nu })-\frac{1}{n}\left\{({\mathit{H}}^{(1{)}^{\prime }}\mathit{\nu })-\frac{1}{2}(\mathrm{1\prime }{\mathit{H}}^{(2)}1)\right\}.$$

Theorem 5.7

Under ℳ_p, a second-order approximation to the variance of ${\widehat{\sigma }}_{p,LB}^2/{\sigma }^2$ is $V_2(\mathbf{\Delta })\equiv \frac{1}{n}\{\mathrm{V}\mathrm{E}(\mathbf{\Delta })+\mathrm{E}\mathrm{V}(\mathrm{\Delta })\}$, where

$$\begin{array}{c}\mathrm{V}\mathrm{E}(\mathbf{\Delta })\equiv 2\left(1-\frac{1}{n}\right){\left(1+H-{\mathit{H}}^{(1{)}^{\prime }}\mathit{\nu }+\frac{1}{2}{\mathit{\nu }}^{\prime }{\mathit{H}}^{(2)}\mathit{\nu }\right)}^2+\left(\frac{n-1}{C_n^2}-\frac{{(n-2)}^2}{n}\right)\times \hfill \\ {\left({\mathit{H}}^{(1{)}^{\prime }}\mathit{\nu }-{\mathit{\nu }}^{\prime }{\mathit{H}}^{(2)}\mathit{\nu }\right)}^2+2\left(1-\frac{2}{n}\right)\left(1+H-{\mathit{H}}^{(1{)}^{\prime }}\mathit{\nu }+\frac{1}{2}{\mathit{\nu }}^{\prime }{\mathit{H}}^{(2)}\mathit{\nu }\right)\left({\mathit{H}}^{(1{)}^{\prime }}\mathit{\nu }-{\mathit{\nu }}^{\prime }{\mathit{H}}^{(2)}\mathit{\nu }\right);\hfill \end{array}$$

$$\begin{array}{c}\mathrm{E}\mathrm{V}(\mathbf{\Delta })\equiv \frac{1}{2n}{\left(\mathrm{1\prime }{\mathit{H}}^{(2)}1\right)}^2+\left(1-\frac{1}{n}\right){\left({\mathit{H}}^{(1{)}^{\prime }}1-\mathrm{1\prime }{\mathit{H}}^{(2)}\mathit{\nu }\right)}^2+\hfill \\ 2\left(1-\frac{2}{n}\right)\left({\mathit{H}}^{(1{)}^{\prime }}1-\mathrm{1\prime }{\mathit{H}}^{(2)}\mathit{\nu }\right)\left(\mathrm{1\prime }{\mathit{H}}^{(2)}\mathit{\nu }\right)+\frac{1}{C_n^2}{\left(\mathrm{1\prime }{\mathit{H}}^{(2)}\mathit{\nu }\right)}^2.\hfill \end{array}$$

From these expressions, we can compute second-order approximations to the risk functions of ${\widehat{\sigma }}_{p,LB1}^2$ according to the formula

$$R\left({\widehat{\sigma }}_{p,LB1}^2,({\mu }_{i_0},{\sigma }^2)\right)\approx {\left(\frac{n}{n-2}\right)}^2{V}_2(\mathbf{\Delta };\alpha =n/2)+{\left[\left(\frac{n}{n-2}\right)E_2(\mathbf{\Delta };\alpha =n/2)-1\right]}^2,$$ (26)

where E₂(Δ;α) ≡ E₂(Δ) and V₂(Δ;α) ≡ V₂(Δ) are given in Theorem 5.6 and Theorem 5.7, respectively. For other limiting Bayes and weighted estimators in Table 1 and (16), analogous approximate risk expressions can be obtained similarly as for ${\widehat{\sigma }}_{p,LB1}^2$.

Lastly, still for a given α > 0, we present a few expressions for the components $H_k^{(1)}(\mathit{T})$, k ∈ {1, 2, …, p} of the p × 1 vector H⁽¹⁾ (T) and the components $H_{kl}^{(2)}(\mathit{T})$, k, l ∈ {1, 2, …, p} of the p × p matrix H⁽²⁾ T, which when evaluated at T = ν yield H⁽¹⁾ and H⁽²⁾, respectively. From the expressions for H(T) and θ_i(T) in Corollary 5.1, we find that for j,k ∈ {1, 2, …, p},

$$H_k^{(1)}(\mathit{T})\equiv \frac{\partial }{\partial T_k}H(\mathit{T})=2{\theta }_k(\mathit{T})T_k+{\displaystyle {\sum }_{i=1}^p{\theta }_{ik}^{(1)}}(\mathit{T})T_i^2;$$

$$H_{kl}^{(2)}(\mathit{T})\equiv \frac{{\partial }^2}{\partial T_k\partial T_l}H(\mathit{T})=2{\theta }_k(\mathit{T})I\{k=l\}+2[{\theta }_{kl}^{(1)}(\mathit{T})T_k+{\theta }_{lk}^{(1)}(\mathit{T})T_l]+{\displaystyle {\sum }_{i=1}^p{\theta }_{ikl}^{(2)}(\mathit{T})T_i^2;}$$

where, for i, k ∈ {1, 2, …, p}, ${\theta }_{ik}^{(1)}(\mathit{T})=(2\alpha )(T_k/(1+T_k^2)){\theta }_k(\mathit{T})[{\theta }_i(\mathit{T})-I\{k=i\}]$; and, for i,k,l ∈ {1, 2, …, p},

$$\begin{array}{c}{\theta }_{ikl}^{(2)}(\mathit{T})=(2\alpha )\{I\{k=l\}\left(\frac{1-T_k^2}{{(1+T_k^2)}^2}\right){\theta }_k(\mathit{T})[{\theta }_i(\mathit{T})-I\{k=i\}]+\hfill \\ \left(\frac{T_k}{{(1+T_k^2)}^2}\right)\left[{\theta }_{kl}^{(1)}(\mathit{T})[{\theta }_i(\mathit{T})-I\{k=i\}]+{\theta }_k(\mathit{T}){\theta }_{il}^{(1)}(\mathit{T})\right]\}.\hfill \end{array}$$

5.4 Assessing the Second-Order Approximations via Simulation

To assess the goodness of the second-order approximations, we compared the values of the means, variances, and risks of ${\widehat{\sigma }}_{p,LB}^2(\alpha =n/2)/{\sigma }^2$ based on 10,000 simulated datasets to their second-order approximations. The results revealed the same pattern across all choices of Δ. For n = 3 the approximation performs rather poorly, gradually improving with increasing n, to finally become almost identical to the simulation-based values when n is 30. In one of the worst-case scenarios, when n = 3 and Δ is symmetric with a medium-size spread (such as Δ = (−0:25; 0; 0:25)), the approximate mean values lie generally within 20% of the simulated ones. Similar behavior is shown by variances and risks also. Furthermore, as the model dimension p increases or as the separations among the sub-models’ means become smaller, the differences between simulated values and approximations also seem to diminish. With increasing n the accuracy of the approximations improves. Therefore, the second-order approximation appears to work well overall, but when n is small (less than 15) it seems better to use simulations. In the remainder of this paper, all analyses involving risks of the limiting Bayes $({\widehat{\sigma }}_{p,LBk}^{2}s)$ and weighted $({\widehat{\sigma }}_{p,PLBk}^{2}s)$ estimators are based on simulations.

5.5 Comparison of Relative Efficiencies

We now carry out the comparison of relative efficiencies of the variance estimators with respect to ${\widehat{\sigma }}_{UMVU}^2$, using simulated datasets with a variety of Δ values for n ∈ {3; 10; 30}. The results are summarized in Table 2 and Table 3 and Figure 1 and Figure 2.

Table 2.

Efficiencies (relative to of the UMVU estimator ${\widehat{\sigma }}_{UMVU}^2$) of the different variance estimators for different combinations of p, Δ, and n. For the limiting Bayes and weighted estimators, the values are based on simulation studies with 10000 replications for each combination.

Combinations of p and Δ		Efficiency %
	n	MRE	pMLE	pMRE	ALB	LB1	LB2	LB3	LB4	PLB1	PLB2	PLB3
	3	200	174	222	14	10	60	152	230	240	236	233
Δ=(−0.25,0,0.25)	10	122	117	124	71	62	89	113	128	129	129	128
p=3	30	107	106	107	91	89	99	107	105	112	112	112
	3	200	183	209	17	11	66	160	217	235	226	221
Δ=(−0.5,0,0.5)	10	122	119	124	73	59	86	110	127	129	128	127
p=3	30	107	106	110	89	84	95	104	109	111	111	111
	3	200	164	226	13	9	51	135	227	238	234	232
Δ=(0,0.25, 0.50)	10	122	114	127	66	53	78	103	131	129	128	128
p=3	30	107	105	109	88	81	92	100	107	108	108	108
	3	200	166	222	13	8	47	128	222	233	228	224
Δ=(0,0.5,1)	10	122	115	128	65	51	76	102	126	126	126	126
p=3	30	107	105	110	87	83	94	103	109	110	110	110
	3	200	174	222	14	10	58	149	234	241	238	235
Δ=(−0.25:2−4:0.25)	10	122	117	123	71	61	88	112	127	130	130	129
p=9	30	107	105	106	91	88	98	106	109	111	111	111
	3	200	174	222	14	10	57	145	234	239	237	234
Δ=(−0.25:2−5:0.25)	10	122	117	123	71	60	86	109	130	127	127	126
p=17	30	107	105	106	91	87	97	105	108	110	110	110
	3	200	164	225	13	9	52	137	230	243	240	238
Δ=(0:2−4:0.5)	10	122	114	126	66	53	77	102	129	128	127	127
p=9	30	107	104	108	88	76	87	97	109	107	107	107
	3	200	164	225	13	10	55	145	235	249	246	243
Δ=(0:2−5:0.5)	10	122	114	126	66	54	79	106	130	134	133	133
p=17	30	107	104	108	88	76	87	97	111	107	107	108

Table 3.

Efficiencies (relative to the UMVU estimator ${\widehat{\sigma }}_{UMVU}^2$) of the different variance estimators for different combinations of p, Δ, and n. For the limiting Bayes and weighted estimators, 10000 simulation replications were performed for each combination.

Combinations of p and Δ		Efficiency %
	n	MRE	pMLE	pMRE	ALB	LB1	LB2	LB3	LB4	PLB1	PLB2	PLB3
	3	200	170	232	13	8	52	143	229	243	237	234
Δ/=(0, 1)	10	122	115	134	63	55	81	107	130	129	130	130
p=2	30	107	104	111	85	85	96	105	109	112	112	112
	3	200	195	216	18	10	63	162	221	255	237	228
Δ=(−1, 0, 1)	10	122	120	134	67	51	77	103	129	126	127	128
p=3	30	107	104	111	85	84	95	103	110	111	111	111
	3	200	185	199	19	11	66	162	208	246	230	221
Δ=(−1:2−1:1)	10	122	119	124	73	52	78	103	125	126	126	125
p=5	30	107	106	110	89	81	92	101	107	108	108	108
	3	200	182	195	20	11	64	153	210	232	219	211
Δ=(−1:2−2 :1)	10	122	118	120	74	56	82	106	126	126	125	125
p=9	30	107	106	107	91	82	92	100	105	106	106	106
	3	200	181	194	20	11	65	155	207	234	220	213
Δ=(−1:2−3:1)	10	122	117	119	75	55	81	104	126	125	124	123
p=17	30	107	105	106	92	80	90	99	107	107	107	107
	3	200	181	194	20	12	72	168	206	240	226	217
Δ=(−1:2−4:1)	10	122	117	119	75	57	82	106	125	126	125	124
p=33	30	107	105	105	92	82	93	101	105	108	108	108
	3	200	181	194	20	11	65	156	208	234	221	214
Δ=(−1:2−5:1)	10	122	117	119	75	58	84	109	123	128	127	127
p=65	30	107	105	105	92	82	93	102	107	109	109	109
	3	200	181	194	20	11	69	163	207	237	224	216
Δ=(−1:2−6:1)	10	122	117	119	75	56	82	106	126	126	125	125
p=129	30	107	105	105	92	81	91	99	108	106	106	106
	3	200	181	194	20	11	67	159	205	236	223	215
Δ=(−1:2−7 :1)	10	122	117	119	75	57	83	107	124	128	127	126
p=257	30	107	105	105	92	82	92	101	108	108	108	108
	3	200	181	194	20	11	66	156	206	233	221	213
Δ=(−1:2−8:1)	10	122	117	119	75	58	84	108	127	128	127	126
p=513	30	107	105	105	92	80	91	99	106	107	107	107

Figure 1.

Relative efficiencies of pMRE with respect to MRE in a symmetric and asymmetric Δ cases, as a function of Δ_max and number of sub-models p for sample size of n = 10. The symmetric case is of form Δ = [−Δ_max : Δ_max/(p − 1) : Δ_max], while the asymmetric case is of form Δ = [0 : Δ_max/(2(p − 1)) : Δ_max].

Figure 2.

Efficiencies of the leading estimators and Stein’s estimator of σ² relative to the UMVUE ${\widehat{\sigma }}_{UMVUE}^2$ for p = 2 and Δ = (0, Δ), with Δ varying, for n = 3, 10. The connected scatterplots represent the simulated relative efficiencies based on 20000 replications for each Δ for the limiting Bayes (pLB4, lighter of the top two), weighted (pPLB1, darker of the top two), and Stein (bottom) estimators. The smooth curves correspond to theoretical efficiencies of ${\widehat{\sigma }}_{p,MRE}^2$ (top), ${\widehat{\sigma }}_{MRE}^2$ (middle), and ${\widehat{\sigma }}_{p,MLE}^2$ (bottom).

Table 2 focuses on the differences in relative effciencies between symmetric and asymmetric Δ cases. As can be seen, there does not seem to be a strong effect of the asymmetry of Δ on the estimators. In all Δ cases that we have chosen, ${\widehat{\sigma }}_{p,PLB1}^2,{\widehat{\sigma }}_{p,PLB2}^2,{\widehat{\sigma }}_{p,PLB3}^2,{\widehat{\sigma }}_{p,LB4}^2$ perform best, with the two-step estimator ${\widehat{\sigma }}_{p,MRE}^2$ following. Clearly, the best among the ℳ_p-based estimators dominate the ℳM-based estimators ${\widehat{\sigma }}_{UMVU}^2\text{and}{\widehat{\sigma }}_{MRE}^2$, with the gain in efficiency being quite impressive for small sample sizes.

Table 3 is designed to examine the impact of increasing number of sub-models. We see that when p is large the two-step estimator ${\widehat{\sigma }}_{p,MRE}^2$ becomes less efficient than the global-type estimator ${\widehat{\sigma }}_{MRE}^2$. This result is consistent with the theoretical result of Corollary 5.4. Note also that even for p as low as 33, the ratios of the relative efficiency values from Table 3 start to agree (to the third decimal) with the limiting relative efficiencies predicted by Corollary 5.4. The weighted estimators do not seem to be affected much by the increasing p, faring much better than the two-step estimators.

Figure 1 presents two contour plots of the relative efficiencies of ${\widehat{\sigma }}_{p,MRE}^2$ with respect to ${\widehat{\sigma }}_{MRE}^2$ as a function of p (where p > 3) and the range of the values in the Δ vector. In the top and bottom contour plots symmetric and asymmetric Δ cases are considered separately. The top contour plot reveals a structure that is consistent with Corollary 5.4, especially in the regions of the contour plot where p → ∞ and Δ_max → ∞ (hence Δ_min → − ∞) which fall in the top and bottom right corners, where ${\widehat{\sigma }}_{MRE}^2$ starts to dominate ${\widehat{\sigma }}_{p,MRE}^2$. Note that the 98% relative efficiency in this region is very close to the limiting 97% from Corollary 5.4. The bottom contour plot is constructed using Δ that are quite asymmetric (all Δ_min = 0), and therefore could not be compared to the predictions of Corollary 5.4. From this plot we can see however that ${\widehat{\sigma }}_{p,MRE}^2$ seems to dominate ${\widehat{\sigma }}_{MRE}^2$ everywhere.

Figure 2 explores the case when p = 2 only, so Δ = (0; Δ), and for sample sizes n = 3 and n = 10. The plots depict the efficiency of the leading estimators, together with that of the Stein estimator in (5), as a function of the magnitude of the Δ parameter. As can be seen, ${\widehat{\sigma }}_{p,MRE}^2$ performs best when |Δ| is large, with ${\widehat{\sigma }}_{p,LB4}^2$ giving a very comparable performance. The estimator ${\widehat{\sigma }}_{p,PLB1}^2$ performs better than ${\widehat{\sigma }}_{p,MRE}^2$ when |Δ| is closer to zero, but degrades in performance when |Δ| becomes large. Thus, we see that the estimators’ performances and regions where they perform well will depend to a large extent on the magnitude of Δ. In particular, it appears that the best among the weighted estimators perform very well when the |Δ| is neither too large nor too small, while the two-step estimator performs very well when |Δ| is large. This points to the following intuitive explanation: when |Δ| is large, the two models are well-separated and the model selection is easier; however, when |Δ| is neither too small nor too large, it is not so clear which model to choose, and it seems better to average over the sub-models’ estimators. When |Δ| is quite close to zero, i.e. when there is not much di erence among the sub-models, either approach to estimation works well. With regards to the Stein estimator, observe that though it dominates ${\widehat{\sigma }}_{MRE}^2$, as is expected from theory, it is at the same time dominated by the estimators ${\widehat{\sigma }}_{p,MRE}^2,{\widehat{\sigma }}_{p,PLB1}^2,\text{and}{\widehat{\sigma }}_{p,LB4}^2$.

Overall, based on the results of the risk comparison, the estimators performing best are the Bayes-type or weighted estimators ${\widehat{\sigma }}_{p,PLB1}^2\text{and}{\widehat{\sigma }}_{p,LB4}^2$, and the two-step estimator ${\widehat{\sigma }}_{p,MRE}^2$. We give a slight preference to the weighted estimators because their performance does not degrade much even when the number of sub-models increases, in contrast to the two-step estimator which becomes dominated by the ℳ-estimator ${\widehat{\sigma }}_{MRE}^2$ when p, the number of sub-models, increases.

Finally, a cautionary note arising from these efficiency studies is that one ought to be very careful in the choice of prior parameters. At least in the situation when one is concerned with variance estimation, the limiting Bayes estimators ${\widehat{\sigma }}_{p,LB1}^2\text{and}{\widehat{\sigma }}_{p,LB2}^2$, corresponding to the limiting cases of κ → 1 and κ → 3/2 respectively, perform quite poorly, especially for small sample sizes. These two estimators are dominated by the estimator ${\widehat{\sigma }}_{UMVU}^2$ in terms of risk function. However, these improper priors associated with the limiting values of κ are most likely the worst-case scenarios, and other, more carefully chosen and meaningful priors should result in improved performance.

6 Concluding Remarks

We have examined some of the issues arising when considering a model with a finite number of sub-models, where the goal is to make inference about a common parameter among these sub-models, based on a single realization of a sample. It is of interest to determine which of the three possible strategies is preferable: (i) to utilize a wider model that encompasses all competing sub-models; (ii) to adopt a two-step approach: select the sub-model, and then do inference within this chosen sub-model, but with both steps utilizing the same sample data; (iii) to do a sub-model averaging scheme where the inference procedure is formed by weighting the sub-models’ procedures, with the weights being also data-dependent. The second strategy may be labeled the classical approach, while the third strategy coincides or is motivated by the Bayesian approach.

Through a simple model prototype with a finite number of Gaussian sub-models with common variance but different means, we have studied each of the strategies, with regards to the estimation of variance. Based on the theoretical and simulated comparison of the different types of estimators, and with the estimator performance evaluated through risk functions based on quadratic loss, we have reached the following conclusions: (i) There could be considerable improvement in using estimators developed by exploiting the structure of the sub-models, over the strategy of simply using estimators from a wider model. (ii) However, the properties of these resulting estimators may be quite difficult to obtain. Furthermore, some desirable properties of the sub-model estimators, such as unbiasedness and minimum variance, may not carry-over when they are combined to form the estimator for the full model of interest. (iii) Based on the theoretical and simulated results for the variance parameter σ² considered in this paper, the weighted estimators, which were motivated and/or derived via the Bayesian approach, seem preferable over the two-step estimators even though these estimators were derived using improper priors. (iv) When the number of sub-models increases and two-step estimators are employed, it appears that their performance could degrade relative to estimators developed under a wider model, but that the weighted estimators’ performances are not necessarily affected. (v) And, finally, when developing weighted estimators through the Bayesian framework, caution must be observed in assigning prior parameters as a particular prior specification may also lead to poor estimators.

Approaches similar to the one presented here could be a useful first step in many contexts where model selection is often done and where there exist a natural notion of “distance” among models: regression, survival analysis, or goodness-of-fit testing. For example, techniques and results presented here could be extended to settings of regression with p possible predictor variables, where the goal is estimation of dispersion parameters associated with error components for the 2^p competing sub-models. There is clearly a need for studies of more complicated situations in varied settings, where multiple parameters are being estimated simultaneously and/or where sub-models are of different dimensions (Claeskens and Hjort (2003)). As was pointed out earlier, for these more general settings, exact risk expressions may not be possible and asymptotic analysis may be needed, in contrast to the situation considered in this paper where the Gaussian distributional assumption enabled us to obtain concrete results. Also, many other interesting alternative options need to be examined: for example, in the two-step approach, would it have been better to subdivide the sample data into two parts and use the first part for model selection and the second part for making inference in the chosen sub-model, an issue alluded for instance in Hastie, Tibshirani, and Friedman (2001) and investigated by Yang (2003)? Finally, we have observed in Dukić and Peña (2003) that in the case of estimation of the distribution function in this same specific model, a different conclusion holds with respect to which of the three strategies is preferable. This is consistent with recent work by Claeskens and Hjort (2003)) where they advocate the use of a focussed information criterion in model selection problems that is tailored to the specific parameter of interest.

A Appendix: Proofs of Selected Results

In this appendix we gather the technical proofs of the results presented in earlier sections.

Proof of Proposition 5.1

With $\overline{Z}=(Z\mathrm{1\prime })/n$, we have $n{\widehat{\sigma }}_i^2/{\sigma }^2=\Vert \mathit{\chi }-{\mu }_i\mathbf{1}/\sigma {\Vert }^2=\Vert [(\mathit{\chi }-{\mu }_{i_0}\mathbf{1})-({\mu }_i-{\mu }_{i_0})\mathbf{1}]/\sigma {\Vert }^2\stackrel{d}{=}\Vert \mathit{Z}-{\mathrm{\Delta }}_i\mathbf{1}{\Vert }^2=\Vert \mathit{Z}-\overline{Z}\mathbf{1}{\Vert }^2+n{(\overline{Z}-{\mathrm{\Delta }}_i)}^2$. Letting $W=\Vert \mathit{Z}-\overline{Z}\mathbf{1}{\Vert }^2$ and $V_i=\sqrt{n}(\overline{Z}-{\mathrm{\Delta }}_i)$, i = 1, 2, … p, it follows that $W\sim {\chi }_{n-1}^2$ with W and V = (V₁, V₂, …, V_p)′ independent. Furthermore, since $\overline{Z}\sim N(0,n^{-1})$, V has representation

$$\mathit{V}=Z1-\sqrt{n}\mathbf{\Delta },$$ (27)

so V ∼ N_p(0,J) since $\mathbf{\text{Cov}}(\mathit{V},\mathit{V})=\mathbf{\text{Cov}}\{Z\mathbf{1}-\sqrt{n}\mathbf{\Delta },Z\mathbf{1}-\sqrt{n}\mathbf{\Delta }\}=\mathbf{1}\mathbf{\text{Var}}(Z)\mathbf{1}\prime =\mathit{J}$.

Proof of Theorem 5.1

By Prop. 5.1, ${\widehat{\sigma }}_{p,MLE}^2/{\sigma }^2\stackrel{d}{=}\left[W+{\displaystyle {\sum }_{i=1}^{p}I\{\widehat{M}=i\}{V}_i^2}\right]/n$. Now,

$$\begin{array}{c}\{\widehat{M}=i\}=\left\{\frac{{\widehat{\sigma }}_i^2}{{\sigma }^2}<\frac{{\widehat{\sigma }}_j^2}{{\sigma }^2},j=1,2,\dots ,p;j\ne i\right\}\stackrel{d}{=}\{V_i^2<V_j^2,j=1,2,\dots ,p;j\ne i\}\hfill \\ =\left\{(Z-\sqrt{n}{\mathrm{\Delta }}_i{)}^2<(Z-\sqrt{n}{\mathrm{\Delta }}_j{)}^2,j=1,2,\dots ,p;j\ne i\right\}\text{by}(27)\hfill \\ =\left\{({\mathrm{\Delta }}_j-{\mathrm{\Delta }}_i)Z<\frac{\sqrt{n}}{2}({\mathrm{\Delta }}_j+{\mathrm{\Delta }}_i)({\mathrm{\Delta }}_j-{\mathrm{\Delta }}_i),j=1,2,\dots ,p;j\ne i\right\}\hfill \\ =\left\{Z<\frac{\sqrt{n}}{2}({\mathrm{\Delta }}_j+{\mathrm{\Delta }}_i),\forall (j\ne i,{\mathrm{\Delta }}_j>{\mathrm{\Delta }}_i)\right\}\cap \left\{Z>\frac{\sqrt{n}}{2}({\mathrm{\Delta }}_j+{\mathrm{\Delta }}_i),\forall (j\ne i,{\mathrm{\Delta }}_j<{\mathrm{\Delta }}_i)\right\}\hfill \\ =\left\{\frac{\sqrt{n}}{2}\left({\mathrm{\Delta }}_i+\underset{\{j:{\mathrm{\Delta }}_j<{\mathrm{\Delta }}_i\}}{\text{max}}{\mathrm{\Delta }}_j\right)<Z<\frac{\sqrt{n}}{2}\left({\mathrm{\Delta }}_i+\underset{\{j:{\mathrm{\Delta }}_j>{\mathrm{\Delta }}_i\}}{\text{min}}{\mathrm{\Delta }}_j\right)\right\}.\hfill \end{array}$$

But,

$$\begin{array}{c}{\displaystyle \sum _{i=1}^{p}I\left\{\frac{\sqrt{n}}{2}\left({\mathrm{\Delta }}_i+\underset{\{j:{\mathrm{\Delta }}_j<{\mathrm{\Delta }}_i\}}{\text{max}}{\mathrm{\Delta }}_j\right)<Z<\frac{\sqrt{n}}{2}\left({\mathrm{\Delta }}_i+\underset{\{j:{\mathrm{\Delta }}_j>{\mathrm{\Delta }}_i\}}{\text{min}}{\mathrm{\Delta }}_j\right)\right\}{V}_i^2}\hfill \\ ={\displaystyle \sum _{i=1}^{p}I\left\{\frac{\sqrt{n}}{2}\left({\mathrm{\Delta }}_{(i)}+\underset{\{j:{\mathrm{\Delta }}_j<{\mathrm{\Delta }}_{(i)}\}}{\text{max}}{\mathrm{\Delta }}_j\right)<Z<\frac{\sqrt{n}}{2}\left({\mathrm{\Delta }}_{(i)}+\underset{\{j:{\mathrm{\Delta }}_j>{\mathrm{\Delta }}_{(i)}\}}{\text{min}}{\mathrm{\Delta }}_j\right)\right\}}(Z-\sqrt{n}{\mathrm{\Delta }}_{(i)}{)}^2\hfill \\ ={\displaystyle \sum _{i=1}^{p}I}\left\{\frac{\sqrt{n}}{2}\left({\mathrm{\Delta }}_{(i)}+{\mathrm{\Delta }}_{(i-1)}\right)<Z<\frac{\sqrt{n}}{2}\left({\mathrm{\Delta }}_{(i)}+{\mathrm{\Delta }}_{(i+1)}\right)\right\}(Z-\sqrt{n}{\mathrm{\Delta }}_{(i)}{)}^2\hfill \\ ={\displaystyle \sum _{i=1}^{p}I\left\{L({\mathrm{\Delta }}_{(i)},\mathbf{\Delta })<Z<U({\mathrm{\Delta }}_{(i)},\mathbf{\Delta })\right\}(Z-\sqrt{n}{\mathrm{\Delta }}_{(i)}{)}^2.}\hfill \end{array}$$

Proof of Theorem 5.2

From Theorem 5.1, using ${\displaystyle {\int }_a^{b}z\varphi (z)dz=\varphi (a)-\varphi (b)}$ for a < b,

$$\begin{array}{c}\mathrm{E}\mathrm{p}\mathrm{M}\mathrm{L}\mathrm{E}(\mathbf{\Delta })=\frac{1}{n}\left\{\mathbf{E}(W)+\mathbf{E}\left\{{\displaystyle \sum _{i=1}^{p}I({\mathrm{\Omega }}_{(i)})(Z-\sqrt{n}{\mathrm{\Delta }}_{(i)}{)}^2}\right\}\right\}\hfill \\ =\frac{1}{n}\left\{(n-1)+\mathbf{E}(Z^2)-2\sqrt{n}{\displaystyle \sum _{i=1}^p{\mathrm{\Delta }}_{(i)}\mathbf{E}\{ZI({\mathrm{\Omega }}_{(i)})\}+n{\displaystyle \sum _{i=1}^p{\mathrm{\Delta }}_{(i)}^2{P}_{(i)}(\mathbf{\Delta })}}\right\}\hfill \\ =\frac{1}{n}\left\{n-2\sqrt{n}{\displaystyle \sum _{i=1}^p{\mathrm{\Delta }}_{(i)}}{\displaystyle {\int }_{L({\mathrm{\Delta }}_{(i)},\mathbf{\Delta })}^{U({\mathrm{\Delta }}_{(i)},\mathbf{\Delta })}z\varphi (z)dz+n{\displaystyle \sum _{i=1}^p{\mathrm{\Delta }}_{(i)}^2{P}_{(i)}(\mathbf{\Delta })}}\right\}\hfill \\ =1-\frac{2}{\sqrt{n}}{\displaystyle \sum _{i=1}^p{\mathrm{\Delta }}_{(i)}\left[\varphi (L({\mathrm{\Delta }}_{(i)},\mathbf{\Delta }))-\varphi (U({\mathrm{\Delta }}_{(i)},\mathbf{\Delta }))\right]}+{\displaystyle \sum _{i=1}^p{\mathrm{\Delta }}_{(i)}^2\left[\mathrm{\Phi }(U({\mathrm{\Delta }}_{(i)},\mathbf{\Delta }))-\mathrm{\Phi }(L({\mathrm{\Delta }}_{(i)},\mathbf{\Delta }))\right]}.\hfill \end{array}$$

Proof of Theorem 5.3

From Theorem 5.1, by independence of W and Z, and because Ω_(i)s depend only on Z, we have $\mathrm{V}\mathrm{p}\mathrm{M}\mathrm{L}\mathrm{E}(\mathbf{\Delta })=n^{-2}\left\{\mathbf{\text{Var}}(W)+\mathbf{\text{Var}}\left[{\displaystyle {\sum }_{i=1}^{p}I({\mathrm{\Omega }}_{(i)})(Z-\sqrt{n}{\mathrm{\Delta }}_{(i)}{)}^2}\right]\right\}$. We already know that Var(W) = 2(n − 1). On the other hand, $\mathbf{\text{Var}}\left\{{\displaystyle {\sum }_{i=1}^{p}I({\mathrm{\Omega }}_{(i)})(Z-\sqrt{n}{\mathrm{\Delta }}_{(i)}{)}^2}\right\}$ $={\displaystyle {\sum }_{i=1}^p\mathbf{\text{Var}}\left\{I({\mathrm{\Omega }}_{(i)})(Z-\sqrt{n}{\mathrm{\Delta }}_{(i)}{)}^2\right\}}+{\displaystyle {\sum }_{i\ne j}\mathbf{\text{Cov}}\left\{I({\mathrm{\Omega }}_{(i)})(Z-\sqrt{n}{\mathrm{\Delta }}_{(i)}{)}^2,I({\mathrm{\Omega }}_{(j)})(Z-\sqrt{n}{\mathrm{\Delta }}_{(j)}{)}^2\right\}}$. By the variance formula and definition of ζ_(i) (k), $\mathbf{\text{Var}}\left\{I({\mathrm{\Omega }}_{(i)})(Z-\sqrt{n}{\mathrm{\Delta }}_{(i)}{)}^2\right\}={\zeta }_{(i)}(4)-{[{\zeta }_{(i)}(2)]}^2$; and $\mathbf{\text{Cov}}\left\{I({\mathrm{\Omega }}_{(i)})(Z-\sqrt{n}{\mathrm{\Delta }}_{(i)}{)}^2,I({\mathrm{\Omega }}_{(j)})(Z-\sqrt{n}{\mathrm{\Delta }}_{(j)}{)}^2\right\}=-{\zeta }_{(i)}(2){\zeta }_{(j)}(2)$, as I(Ω_(i))I(Ω_(j)) = 0 whenever i ≠ j. Consequently, $\mathbf{\text{Var}}\left\{{\displaystyle {\sum }_{i=1}^{p}I({\mathrm{\Omega }}_{(i)})(Z-\sqrt{n}{\mathrm{\Delta }}_{(i)}{)}^2}\right\}$ $={\displaystyle {\sum }_{i=1}^p\left[{\zeta }_{(i)}(4)-{[{\zeta }_{(i)}(2)]}^2\right]}+{\displaystyle {\sum }_{i\ne j}\left[-{\zeta }_{(i)}(2){\zeta }_{(j)}(2)\right]}={\displaystyle {\sum }_{i=1}^p{\zeta }_{(i)}(4)}-{({\displaystyle {\sum }_{i=1}^p{\zeta }_{(i)}(2)})}^2$.

Proof of Corollary 5.4

From Theorem 5.4, $R\left({\widehat{\sigma }}_{p,MLE}^2,({\mu }_{i_0},{\sigma }^2)\right)=\mathrm{V}\mathrm{p}\mathrm{M}\mathrm{L}\mathrm{E}(\mathbf{\Delta })+{[\mathrm{E}\mathrm{p}\mathrm{M}\mathrm{L}\mathrm{E}(\mathbf{\Delta })-1]}^2\to (2/n)(1-1/n)+{[(1-1/n)-1]}^2=(2n-1)/n^2$. Also, since ${\widehat{\sigma }}_{p,MRE}^2=(n/(n+2)){\widehat{\sigma }}_{p,MLE}^2$, we have $R({\widehat{\sigma }}_{p,MRE}^2,({\mu }_{i_0},{\sigma }^2))$ $={(n/(n+2))}^2\mathrm{V}\mathrm{p}\mathrm{M}\mathrm{L}\mathrm{E}(\mathbf{\Delta })+{[(n/(n+2))\mathrm{E}\mathrm{p}\mathrm{M}\mathrm{L}\mathrm{E}(\mathbf{\Delta })-1]}^2\to$ ${(n/(n+2))}^2(2/n)(1-1/n)+{[(n/(n+2))(1-1/n)-1]}^2=(2n+7)[{(n+2)}^2]$. The efficiency expressions relative to ${\widehat{\sigma }}_{UMVU}^2\text{of}{\widehat{\sigma }}_{p,MLE}^2\text{and}{\widehat{\sigma }}_{p,MRE}^2$ are obtained by dividing $R({\widehat{\sigma }}_{UMVU}^2,({\mu }_{i_0},{\sigma }^2))=2/(n-1)$ by the preceding risk expressions. Both of the resulting efficiency expressions are easily shown to exceed 1. Taking the ratio of $R({\widehat{\sigma }}_{p,MLE}^2,({\mu }_{i_0},{\sigma }^2))$ and $R({\widehat{\sigma }}_{p,MRE}^2,({\mu }_{i_0},{\sigma }^2))$ yields the third expression, an expression easily shown to exceed 1. In comparing ${\widehat{\sigma }}_{MRE}^2$ and ${\widehat{\sigma }}_{p,MRE}^2$, we observe that $\text{Eff}({\widehat{\sigma }}_{p,MRE}^2:{\widehat{\sigma }}_{MRE}^2)=2{(n+2)}^2/[(n+1)(2n+7)]$. Since 2(n + 2)²/[(n + 1)(2n + 7)] − 1 = −(n−1)/[(n + 1)(2n + 7)] < 0 for n > 1, then we have established that, as p → ∞, ${\widehat{\sigma }}_{MRE}^2$ is more efficient than ${\widehat{\sigma }}_{p,MRE}^2$. To show that ${\widehat{\sigma }}_{p,ALB}^2$ is dominated by ${\widehat{\sigma }}_{UMVU}^2$, note that $R({\widehat{\sigma }}_{p,ALB}^2,({\mu }_{i_0},{\sigma }^2))=(2n-1)/[{(n-2)}^2]$, which is easily shown to exceed 2/(n − 1) whenever n > 2.

Proof of Theorem 5.5

Starting from (24) and using Corollary 5.1, we obtain

$$\frac{{\widehat{\sigma }}_{p,LB}^2}{{\sigma }^2}\stackrel{d}{=}{\displaystyle \sum _{i=1}^p\left\{\frac{{\left[\frac{W}{n}(1+T_i^2)\right]}^{-\alpha }}{{\displaystyle {\sum }_{j=1}^p{\left[\frac{W}{n}(1+T_j^2)\right]}^{-\alpha }}}\right\}}\left[\frac{W}{n}(1+T_i^2)\right]=\frac{W}{n}\left\{1+{\displaystyle \sum _{i=1}^p{\theta }_i(\mathit{T})T_i^2}\right\}.$$

The following lemma will be needed for proving Theorems 5.6 and 5.7

Lemma A.1

Under the conditions of Proposition 5.1, (i) $\mathbf{E}\{\mathit{T}-\mathit{\nu }|W\}=(\sqrt{n}/(\sqrt{W}{C}_n)-1)\mathit{\nu }$; (ii)$\mathbf{E}\{(\mathit{T}-\mathit{\nu })(\mathit{T}-\mathit{\nu }{)}^{\prime }|W\}=\mathit{J}/W+{(\sqrt{n}/(\sqrt{W}{C}_n)-1)}^2{\mathit{\nu }}^{\otimes 2}$; (iii) E{W(T − ν)} = −ν; and (iv) $\mathbf{E}\{W(\mathit{T}-\mathit{\nu })(\mathit{T}-\mathit{\nu }{)}^{\prime }\}=\mathit{J}+n(1/C_n^2-1+3/n){\mathit{\nu }}^{\otimes 2}$, with constant C_n given in Corollary 5.1.

Proof of Lemma A.1

Since $\mathbf{E}(\mathit{T}|W)=\mathbf{E}(\mathit{V}/\sqrt{W}|W)=\mathbf{E}(\mathit{V})/\sqrt{W}=\sqrt{n}\mathit{\nu }/(\sqrt{W}{C}_n)$, then the first result immediately follows. Using that E(W) = n - 1 and $\mathbf{E}(\sqrt{W})=(n-2)C_n/\sqrt{n}$, the third result follows trivially from the first identity. To prove the second result, observe that

$$\mathbf{E}(\mathit{T}\mathit{T\prime }|W)=\frac{1}{W}\mathbf{E}(\mathit{V}\mathit{V\prime })=\frac{1}{W}\mathbf{E}\{(Z\mathbf{1}-\sqrt{n}\mathbf{\Delta }{)}^{\otimes 2}\}=\frac{1}{W}\left(\mathit{J}+\frac{n}{C_n^2}{\mathit{\nu }}^{\otimes 2}\right);$$

and $\mathbf{E}\{\mathit{T}\mathit{\nu }\prime |W\}=-\sqrt{n}\mathbf{\Delta }\mathit{\nu }\prime /\sqrt{W}=\sqrt{n}{\mathit{\nu }}^{\otimes 2}/(\sqrt{W}{C}_n)$. Consequently,

$$\mathbf{E}\{(\mathit{T}-\mathit{\nu })(\mathit{T}-\mathit{\nu }{)}^{\prime }|W\}=\mathbf{E}(\mathit{T}\mathit{T\prime }|W)-2\mathbf{E}(\mathit{T}\mathit{\nu }\prime |W)+{\mathit{\nu }}^{\otimes 2}=\frac{\mathit{J}}{W}+{\left(\frac{\sqrt{n}}{\sqrt{W}{C}_n}-1\right)}^2{\mathit{\nu }}^{\otimes 2}.$$

Finally, by the iterated expectation rule, and using the expressions for E(W) and $\mathbf{E}(\sqrt{W})$,

$$\begin{array}{c}\mathbf{E}\{W(\mathit{T}-\mathit{\nu })(\mathit{T}-\mathit{\nu }{)}^{\prime }\}=\mathbf{E}\{W\mathbf{E}[(\mathit{T}-\mathit{\nu })(\mathit{T}-\mathit{\nu }{)}^{\prime }|W]\}\hfill \\ =\mathbf{E}\{\mathit{J}+\left(\frac{n}{C_n^2}-\frac{2\sqrt{n}}{C_n}\sqrt{W}+W\right){\mathit{\nu }}^{\otimes 2}\}=\mathit{J}+\left(\frac{n}{C_n^2}-2(n-2)+(n-1)\right){\mathit{\nu }}^{\otimes 2}.\hfill \end{array}$$

Simplifying leads to the expression given in the statement of the lemma.

Proof of Theorem 5.6

First, note that by using the fourth result in Lemma A.1, we have

$$\begin{array}{c}\mathbf{E}\{W(\mathit{T}-\mathit{\nu }{)}^{\prime }{\mathit{H}}^{(2)}(\mathit{T}-\mathit{\nu })\}=\mathbf{E}\{\text{tr}[{\mathit{H}}^{(2)}W(\mathit{T}-\mathit{\nu })(\mathit{T}-\mathit{\nu }{)}^{\prime }]\}\hfill \\ =\text{tr}({\mathit{H}}^{(2)}\left[\mathit{J}+n(\frac{1}{C_n^2}-1+\frac{3}{n}){\mathit{\nu }}^{\otimes 2}\right])=(\mathbf{1}\prime {\mathit{H}}^{(2)}\mathbf{1})+n(\frac{1}{C_n^2}-1+\frac{3}{n})(\mathit{\nu \prime }{\mathit{H}}^{(2)}\mathit{\nu }).\hfill \end{array}$$

From (25), and using Lemma A.1 and the preceding result, the approximation to the mean of ${\widehat{\sigma }}_{p,LB}^2/{\sigma }^2$ is obtained to be

$$E_2(\mathbf{\Delta })=\frac{1}{n}\left\{(1+H)(n-1)-{\mathit{H}}^{(1{)}^{\prime }}\mathit{\nu }+\frac{1}{2}(\mathbf{1\prime }{\mathit{H}}^{(2)}\mathbf{1})+\frac{n}{2}(\frac{1}{C_n^2}-1+\frac{3}{n})(\mathit{\nu \prime }{\mathit{H}}^{(2)}\mathit{\nu })\right\},$$

which simplifies to the expression in the statement of the theorem.

To obtain second-order approximation to the variance of ${\widehat{\sigma }}_{p,LB}^2/{\sigma }^2$, we first establish two inter-mediate lemmas concerning the conditional mean and variance, given W, of the variable

$$Q\equiv 1+H+{\mathit{H}}^{(1{)}^{\prime }}(\mathit{T}-\mathit{\nu })+\frac{1}{2}(\mathit{T}-\mathit{\nu }{)}^{\prime }{\mathit{H}}^{(2)}(\mathit{T}-\mathit{\nu }),$$ (28)

which is the second-order Taylor approximation of the function 1 + H(T) (see equation (25)).

Lemma A.2

Under conditions of Proposition 5.1,

$$\begin{array}{c}\mathbf{E}(Q|W)=\left\{1+H-({\mathit{H}}^{(1{)}^{\prime }}\mathit{\nu })+\frac{1}{2}(\mathit{\nu \prime }{\mathit{H}}^{(2)}\mathit{\nu })\right\}+\hfill \\ \frac{\sqrt{n}}{C_n}\{(H^{(1{)}^{\prime }}\mathit{\nu })-(\mathit{\nu \prime }{\mathit{H}}^{(2)}\mathit{\nu })\}\frac{1}{\sqrt{W}}+\frac{1}{2}\{(\mathbf{1}\prime {\mathit{H}}^{(2)}\mathbf{1})+\frac{n}{C_n^2}(\mathit{\nu \prime }{\mathit{H}}^{(2)}\mathit{\nu })\}\frac{1}{W}.\hfill \end{array}$$

Proof of Lemma A.2

From the proof of and results in Lemma A.1, it follows that

$$\mathbf{E}\{(\mathit{T}-\mathit{\nu }{)}^{\prime }{\mathit{H}}^{(2)}(\mathit{T}-\mathit{\nu })|W\}=\frac{1}{W}(\mathbf{1}\prime {\mathit{H}}^{(2)}\mathbf{1})+(\frac{n}{W{C}_n^2}-\frac{2\sqrt{n}}{\sqrt{W}{C}_n}+1)(\mathit{\nu \prime }{\mathit{H}}^{(2)}\mathit{\nu }).$$

From first result in Lemma A.1 and the preceding result, $\mathbf{E}(Q|W)=1+H+(\frac{\sqrt{n}}{\sqrt{W}{C}_n}-1){\mathit{H}}^{(1)\prime }\mathit{\nu }+\frac{1}{2}\{\frac{1}{w}(\mathbf{1\prime }{\mathit{H}}^{(2)}\mathbf{1})+(\frac{n}{W{C}_n^2}-\frac{2\sqrt{n}}{\sqrt{W}{C}_n}+1)(\mathit{\nu }\prime {\mathit{H}}^{(2)}\mathit{\nu })\}$, and the result follows.

Lemma A.3

Under conditions of Proposition 5.1,

$$\begin{array}{c}\mathbf{\text{Var}}(Q|W)={\left\{({\mathit{H}}^{(1{)}^{\prime }}\mathbf{1})-(\mathbf{1\prime }{\mathit{H}}^{(2)}\mathit{\nu })\right\}}^2\frac{1}{W}+\hfill \\ \frac{2\sqrt{n}}{C_n}\left\{({\mathit{H}}^{(1{)}^{\prime }}\mathbf{1})-(\mathbf{1}\prime {\mathit{H}}^{(2)}\mathit{\nu })\right\}(\mathbf{1}\prime {\mathit{H}}^{(2)}\mathit{\nu })\frac{1}{W^{3/2}}+\left\{\frac{1}{2}{(\mathbf{1\prime }{\mathit{H}}^{(2)}\mathbf{1})}^2+\frac{n}{C_n^2}{(\mathbf{1}\prime {\mathit{H}}^{(2)}\mathit{\nu })}^2\right\}\frac{1}{W^2}.\hfill \end{array}$$

Proof of Lemma A.3

First, observe that we have

$$\begin{array}{ccc}\mathbf{\text{Var}}\{{\mathit{H}}^{(1{)}^{\prime }}(\mathit{T}-\mathit{\nu })|W\}\hfill & =\hfill & \frac{1}{W}{\mathit{H}}^{(1{)}^{\prime }}\mathbf{\text{Cov}}(\mathit{V}){\mathit{H}}^{(1)}=\frac{1}{W}{\mathit{H}}^{(1{)}^{\prime }}\mathit{J}{\mathit{H}}^{(1)}=\frac{1}{W}{({\mathit{H}}^{(1{)}^{\prime }}1)}^2;\hfill \\ \mathbf{\text{Var}}\{(\mathit{T}-\mathit{\nu }{)}^{\prime }{\mathit{H}}^{(2)}(\mathit{T}-\mathit{\nu })|W\}\hfill & =\hfill & \mathbf{\text{Var}}\left\{\mathit{T\prime }{\mathit{H}}^{(2)}\mathit{T}|W\right\}+4\mathbf{\text{Var}}\left\{\mathit{\nu \prime }{\mathit{H}}^{(2)}\mathit{T}\right|W\}\hfill \\ \hfill & \hfill & -4\mathbf{\text{Cov}}\{\mathit{T\prime }{\mathit{H}}^{(2)}\mathit{T},\mathit{\nu \prime }{\mathit{H}}^{(2)}\mathit{T}|W\};\hfill \\ \mathbf{\text{Var}}\left\{\mathit{T\prime }{\mathit{H}}^{(2)}\mathit{T}\right|W\}\hfill & =\hfill & \mathbf{\text{Var}}\left\{\frac{\mathit{V\prime }}{\sqrt{W}}{\mathit{H}}^{(2)}\frac{\mathit{V}}{\sqrt{W}}|W\right\}=\frac{1}{W^2}\mathbf{\text{Var}}\{\mathit{V\prime }{\mathit{H}}^{(2)}\mathit{V}\}.\hfill \end{array}$$

From the representation of V in (27), we obtain

$$\mathbf{\text{Var}}\{\mathit{V\prime }{\mathit{H}}^{(2)}\mathit{V}\}=\mathbf{\text{Var}}\{(Z1-\sqrt{n}\mathbf{\Delta }{)}^{\prime }{\mathit{H}}^{(2)}(Z1-\sqrt{n}\mathbf{\Delta })\}2{({\mathbf{1}}^{\prime }{\mathit{H}}^{(2)}\mathbf{1})}^2+\frac{4n}{C_n^2}{(\mathbf{1\prime }{\mathit{H}}^{(2)}\mathit{\nu })}^2$$

since Var(Z²) = 2, Var(Z) = 1, and Cov(Z²,Z) = 0. Thus,

$$\mathbf{\text{Var}}\{\mathit{T\prime }{\mathit{H}}^{(2)}\mathit{T}|W\}=\frac{4}{W^2}\{\frac{1}{2}{(\mathrm{1\prime }{\mathit{H}}^{(2)}1)}^2+\frac{n}{C_n^2}{(\mathrm{1\prime }{\mathit{H}}^{(2)}\mathit{\nu })}^2\}.$$

We also have

$$\begin{array}{c}\mathbf{\text{Var}}\left\{\mathit{\nu \prime }{\mathit{H}}^{(2)}\mathit{T}|W\right\}=\mathit{\nu \prime }{\mathit{H}}^{(2)}\mathbf{\text{Cov}}(\mathit{T}|W)\frac{1}{W}{\mathit{\nu }}^{\prime }{\mathit{H}}^{(2)}\mathit{J}{\mathit{H}}^{(2)}\mathit{\nu }=\frac{1}{W}{(\mathrm{1\prime }{\mathit{H}}^{(2)}\mathit{\nu })}^2;\hfill \\ \mathbf{\text{Cov}}\{\mathit{T\prime }{\mathit{H}}^{(2)}\mathit{T},\mathit{\nu \prime }{\mathit{H}}^{(2)}\mathit{T}|W\}=\frac{1}{W^{3/2}}\mathbf{\text{Cov}}\{{\mathit{V}}^{\prime }{\mathit{H}}^{(2)}\mathit{V},\mathit{\nu \prime }{\mathit{H}}^{(2)}V\}.\hfill \end{array}$$

Again, by utilizing the representation for V in (27), we obtain

$$\mathbf{\text{Cov}}\{\mathit{V\prime }{\mathit{H}}^{(2)}\mathit{V},\mathit{\nu \prime }{\mathit{H}}^{(2)}\mathit{V}\}=\frac{2\sqrt{n}}{C_n}{(\mathbf{1}\prime {\mathit{H}}^{(2)}\mathit{\nu })}^2,$$

so $\mathbf{\text{Cov}}\{\mathit{T\prime }{\mathit{H}}^{(2)}\mathit{T},\mathit{\nu \prime }{\mathit{H}}^{(2)}\mathit{T}|W\}=\frac{2\sqrt{n}}{W^{3/2}{C}_n}{(\mathbf{1}\prime {\mathit{H}}^{(2)}\mathit{\nu })}^2$. Combining these results, we obtain

$$\begin{array}{c}\mathbf{\text{Var}}\{(\mathit{T}-\mathit{\nu }{)}^{\prime }{\mathit{H}}^{(2)}(\mathit{T}-\mathit{\nu })|W\}=\hfill \\ \frac{4}{W^2}\left\{\frac{1}{2}{(\mathbf{1}\prime {\mathit{H}}^{(2)}\mathbf{1})}^2+\frac{n}{C_n^2}{(\mathbf{1}\prime {\mathit{H}}^{(2)}\mathit{\nu })}^2\right\}+4\{\frac{1}{W}{({\mathbf{1}}^{\prime }{\mathit{H}}^{(2)}\mathit{\nu })}^2\}-4\left\{\frac{2\sqrt{n}}{W^{3/2}{C}_n}{({\mathbf{1}}^{\prime }{\mathit{H}}^{(2)}\mathit{\nu })}^2\right\}.\hfill \end{array}$$

Also,

$$\mathbf{\text{Cov}}\{{\mathit{H}}^{(1)\prime }(\mathit{T}-\mathit{\nu }),(\mathit{T}-\mathit{\nu })\prime {\mathit{H}}^{(2)}(\mathit{T}-\mathit{\nu })|W\}={\mathit{H}}^{(1)\prime }\left[\mathbf{\text{Cov}}(\mathit{T},\mathit{T}\prime {\mathit{H}}^{(2)}\mathit{T}|\mathit{W})-2\mathbf{\text{Cov}}(\mathit{T}|W){\mathit{H}}^{(2)}\mathit{\nu }\right].$$

But, once again,

$$\mathbf{\text{Cov}}(\mathit{T},\mathit{T\prime }{\mathit{H}}^{(2)}\mathit{T}|W)=\frac{1}{W^{3/2}}\mathbf{\text{Cov}}(\mathit{V},\mathit{V}{\mathit{H}}^{(2)}\mathit{V})=-\frac{2\sqrt{n}}{W^{3/2}}\mathbf{1}(\mathbf{1}\prime {\mathit{H}}^{(2)}\mathbf{\Delta })=\frac{2\sqrt{n}}{W^{3/2}{C}_n}(\mathbf{1}\prime {\mathit{H}}^{(2)}\mathit{\nu })\mathbf{1},$$

while ${\mathit{H}}^{(1{)}^{\prime }}\mathbf{\text{Cov}}(\mathit{T}|W){\mathit{H}}^{(2)}\mathit{\nu }=\frac{1}{W}({\mathit{H}}^{(1)\prime }1)(\mathrm{1\prime }{\mathit{H}}^{(2)}\mathit{\nu })$. Thus, we have

$$\mathbf{\text{Cov}}\{{\mathit{H}}^{(1{)}^{\prime }}(\mathit{T}-\mathit{\nu }),(\mathit{T}-\mathit{\nu }{)}^{\prime }{\mathit{H}}^{(2)}(\mathit{T}-\mathit{\nu })|W\}=\frac{2\sqrt{n}}{W^{3/2}{C}_n}({\mathbf{1}}^{\prime }{\mathit{H}}^{(2)}\mathit{\nu })1-\frac{2}{W}({\mathit{H}}^{(\mathbf{1}{)}^{\prime }}\mathit{1})({\mathbf{1}}^{\prime }{H}^{(2)}\mathit{\nu }).$$

Now, by using these intermediate results, we find that

$$\begin{array}{ccc}\mathbf{\text{Var}}(Q|W)\hfill & =\hfill & \mathbf{\text{Var}}\{{\mathit{H}}^{(1{)}^{\prime }}(\mathit{T}-\mathit{\nu })\}+\frac{1}{4}\mathbf{\text{Var}}\{(\mathit{T}-\mathit{\nu }{)}^{\prime }{\mathit{H}}^{(2)}(\mathit{T}-\mathit{\nu })|W\}+\hfill \\ \hfill & \hfill & (2)\left(\frac{1}{2}\right)\mathbf{\text{Cov}}\{{\mathit{H}}^{(1{)}^{\prime }}(\mathit{T}-\mathit{\nu }),(\mathit{T}-\mathit{\nu }{)}^{\prime }{\mathit{H}}^{(2)}(\mathit{T}-\mathit{\nu })|W\}\hfill \\ \hfill & =\hfill & \frac{1}{W}{({\mathit{H}}^{(1{)}^{\prime }}\mathbf{1})}^2+\frac{1}{4}\{\frac{4}{W^2}\left[\frac{1}{2}{(\mathbf{1\prime }{\mathit{H}}^{(2)}\mathbf{1})}^2+\frac{n}{C_n^2}{(\mathbf{1\prime }{\mathit{H}}^{(2)}\mathit{\nu })}^2\right]+\hfill \\ \hfill & \hfill & \frac{4}{W}{(\mathbf{1\prime }{\mathit{H}}^{(2)}\mathit{\nu })}^2-\frac{(4)(2)}{W^{3/2}}\frac{\sqrt{n}}{C_n}{(\mathbf{1\prime }{\mathit{H}}^{(2)}\mathit{\nu })}^2\}+\hfill \\ \hfill & \hfill & \frac{2}{W^{3/2}}\frac{\sqrt{n}}{C_n}({\mathit{H}}^{(1{)}^{\prime }}\mathbf{1})(\mathbf{1\prime }{\mathit{H}}^{(2)}\mathit{\nu })-\frac{2}{W}({\mathit{H}}^{(1{)}^{\prime }}\mathbf{1})(\mathbf{1\prime }{\mathit{H}}^{(2)}\mathit{\nu }).\hfill \end{array}$$

and the result follows after simplification.

Proof of Theorem 5.7

From (25) and (28), and by the iterated variance rule, an approximate variance of ${\widehat{\sigma }}_{p,LB}^2/{\sigma }^2$ is ${\mathit{V}}_2(\mathbf{\Delta })\equiv \mathbf{\text{Var}}\left\{\frac{1}{n}WQ\right\}=\frac{1}{n}\left[\mathbf{\text{Var}}\left\{\frac{w}{\sqrt{n}}\mathbf{E}(Q|W)\right\}+\mathbf{E}\left\{\frac{W^2}{n}\mathbf{\text{Var}}(Q|W)\right\}\right]\equiv \frac{1}{n}\{\mathit{V}E(\mathbf{\Delta })+EV(\mathbf{\Delta })\}$. The final expression of VE(Δ) follows from the Lemma A.2, the identities Var(W) = 2(n-1), $\mathbf{\text{Var}}(\sqrt{W})=(n-1)-{(n-2)}^2{C}_n^2/n$, and $\mathbf{\text{Cov}}(W,\sqrt{W})=(n-2)C_n/\sqrt{n}$, and

$$\begin{array}{c}\text{VE}(\mathbf{\Delta })=\frac{1}{n}\left[\mathbf{\text{Var}}\left\{W\left[1+\mathit{H}-{\mathit{H}}^{(1{)}^{\prime }}\mathit{\nu }+\frac{1}{2}{\mathit{\nu }}^{\prime }{\mathit{H}}^{(2)}\mathit{\nu }\right]+\sqrt{W}\frac{\sqrt{n}}{C_n}({\mathit{H}}^{(1{)}^{\prime }}\mathit{\nu }-{\mathit{\nu }}^{\prime }{\mathit{H}}^{(2)}\mathit{\nu })\right\}\right]\hfill \\ =\frac{1}{n}\{{\left[1+\mathit{H}-{\mathit{H}}^{(1{)}^{\prime }}\mathit{\nu }+\frac{1}{2}{\mathit{\nu }}^{\prime }{\mathit{H}}^{(2)}\mathit{\nu }\right]}^2\mathbf{\text{Var}}(W)+\frac{n}{C_n^2}{({\mathit{H}}^{(1{)}^{\prime }}\mathit{\nu }-\mathit{\nu }\prime {\mathit{H}}^{(2)}\mathit{\nu })}^2\mathbf{\text{Var}}(\sqrt{W})\hfill \\ +\frac{2\sqrt{n}}{C_n}\left[1+\mathit{H}-{\mathit{H}}^{(1)\prime }\mathit{\nu }+\frac{1}{2}\mathit{\nu }\prime {\mathit{H}}^{(2)}\mathit{\nu }\right]({\mathit{H}}^{(1)\prime }\mathit{\nu }-\mathit{\nu }\prime {\mathit{H}}^{(2)}\mathit{\nu })\mathbf{\text{Cov}}(W,\sqrt{W})\}.\hfill \end{array}$$

Also, from Lemma A.3,

$$\begin{array}{c}\text{EV}(\mathbf{\Delta })=\frac{1}{n}\{\mathbf{E}(W){\left({\mathit{H}}^{(1{)}^{\prime }}\mathbf{1}-\mathbf{1\prime }{\mathit{H}}^{(2)}\mathit{\nu }\right)}^2+\hfill \\ \frac{2\sqrt{n}}{C_n}\left({\mathit{H}}^{(1{)}^{\prime }}\mathbf{1}-\mathbf{1\prime }{\mathit{H}}^{(2)}\mathit{\nu }\right)(\mathbf{1\prime }{\mathit{H}}^{(2)}\mathit{\nu })\mathbf{E}(\sqrt{W})+\frac{1}{2}{(\mathbf{1\prime }{\mathit{H}}^{(2)}\mathbf{1})}^2+\frac{n}{C_n^2}{(\mathbf{1\prime }{\mathit{H}}^{(2)}\mathit{\nu })}^2\},\hfill \end{array}$$

which simplifies to the expression of EV(Δ) in the statement of the theorem upon substituting the expressions E(W) = n − 1 and $\mathbf{E}(\sqrt{W})=(n-2)C_n/\sqrt{n}$. This completes the proof.