Estimation of Channelized Hotelling Observer Performance with Known Class Means or Known Difference of Class Means

Abstract

This paper concerns task-based image quality assessment for the task of discriminating between two classes of images. We address the problem of estimating two widely-used detection performance measures, SNR and AUC, from a finite number of images, assuming that the class discrimination is performed with a channelized Hotelling observer. In particular, we investigate the advantage that can be gained when either (i) the means of the signal-absent and signal-present classes are both known, or (ii) when the difference of class means is known. For these two scenarios, we propose uniformly minimum variance unbiased estimators of SNR², derive the corresponding sampling distributions and provide variance expressions. In addition, we demonstrate how the bias and variance for the related AUC estimators may be calculated numerically by using the sampling distributions for the SNR² estimators. We find that for both SNR² and AUC, the new estimators have significantly lower bias and mean-square error than the traditional estimator, which assumes that the class means, and their difference, are unknown.

I. Introduction

Task-based assessment of image quality provides a pragmatic paradigm for objective evaluation and comparison of image reconstruction algorithms and imaging systems [1]. One task that is frequently of interest is the detection of a specified object in an image. For example, an important task in radiology is the discrimination of two classes of images: images with a lesion present and with no lesion present.

Lesion detectability may be appraised with both human observers and mathematical model observers [1]. However, human observers come with prominent difficulties. They are highly variable, slow, costly, and subject to fatigue, to the extent that they become impractical, especially for imaging system optimization, as noted by Park et al. [2]. By contrast, mathematical model observers [1] can be implemented efficaciously using computers. Thus, they offer an attractive alternative to human observers for task-based assessment of image quality. One type of model observer which has been given significant attention in medical imaging is the channelized Hotelling observer (CHO) [1]. Specifically, CHO methodology for image quality assessment has been applied in nuclear medicine, e.g., [3]–[8] and in x-ray computed tomography (CT) [9]–[12].

To assess the capability of a given model observer to detect lesions, summary measures of observer performance need to be estimated. Two popular measures of observer performance are the observer signal-to-noise ratio (SNR) and the area under the receiver operating characteristic curve, denoted as AUC [1] (see section II-C). In most situations, it is not possible to calculate SNR and AUC exactly, and they must be estimated from a finite collection of images. Therefore, it is important to understand the small-sample statistical properties of the estimators which are used. We note that two main approaches may be considered for the estimation of SNR and AUC: (i) an SNR-based approach, in which the SNR is first estimated, and then the AUC is obtained from the SNR estimate [1]; (ii) an AUC-based approach, in which the AUC is first estimated, and then the SNR is obtained from the AUC estimate [1]. Under conditions of normality, both approaches are valid because SNR and AUC are functions of each other [1]. Here, we focus on the SNR-based approach. Some researchers have investigated direct estimation of the AUC, e.g., [13]–[15]; these works rely on the Mann-Whitney U statistic.

Note that two main estimation strategies may be selected for the two approaches described above. In the first strategy, all available images are used together to estimate the desired figure of merit [1, p. 972]. In the second strategy, the images are divided into two groups, with one group being used to define (aka train) the observer whereas the second group is used to estimate the performance of the trained observer (aka testing the observer) [1, p. 973]. The second strategy has the advantage of yielding a negative (conservative) bias in the estimated performance [16], [17]. In this paper, we adopt the first strategy, and we will see later that negative biases may also be achieved in this case, provided that the class means, or their difference, are known.

In general, deriving the small-sample properties of estimators is difficult. However, analytical results are known for some cases. For example, if SNR² for a CHO is estimated using a scalar multiple of the two-sample Hotelling T² statistic, then the sampling distribution is known for normally distributed measurements [18, p. 216]; see section IV-A. Also, results pertaining to direct AUC estimation via the nonparametric Mann-Whitney U statistic may be found in [15]. When exact analytical results are not readily available, Taylor series approximations may be beneficial [19], [20], or one may resort to analysis based on Monte Carlo simulation and resampling techniques [16], [17], [21]–[27].

As mentioned by Barrett and Myers [1, p. 972], there is potential to improve the estimation accuracy of SNR² for a CHO if prior knowledge of the mean images with and without a lesion present is incorporated into the estimation procedure. However, to our knowledge, and according to Barrett and Myers [1, p. 973], the statistical properties of such estimators have not been investigated. We have examined this potential and report on our findings here.

Specifically, we propose uniformly minimum variance unbiased (UMVU) estimators of SNR² for a CHO in the situation when the mean images for the lesion present and lesion absent cases are known, and also when only the difference of the mean images is known. We present analytical results for the sampling distributions of these estimators and provide expressions for their variance. Using the sampling distributions of the SNR² estimators, we illustrate how the bias and variance for the related AUC estimators may be numerically evaluated, and we compare the results with a traditional estimator which assumes that the class means, and their difference, are unknown. For expositional clarity, all mathematical proofs are deferred to the appendices.

II. Background

In this section, we set our notation and review several necessary concepts. First, some probability distributions that will be used later in the paper are introduced. Afterwards, the classification task and the channelized Hotelling observers which we consider in this paper are covered. Next, two common summary measures for model observer performance, SNR and AUC, are reviewed. The remainder of the section discusses calculation of moments for AUC estimators. Throughout the paper, we write vectors using boldface and denote the transpose of vectors and matrices with the superscript T, e.g., x^T is the transpose of the vector x.

A. Some Probability Distributions

We assume that the reader is familiar with both the multivariate normal distribution and the univariate noncentral χ² distribution, e.g., see [18], [28]. A p×1 random vector X ∈ ℝ^p following a multivariate normal distribution with mean μ and covariance matrix Σ is denoted X ~ N_p(μ, Σ). A random variable X ∈ ℝ following a noncentral χ² distribution with ν degrees of freedom and noncentrality parameter, λ, is denoted $X\sim {\chi }_{\nu }^2(\lambda )$. Recall that when λ = 0, the noncentral χ² distribution becomes the central χ² distribution.

1) Inverted Gamma

The inverted gamma distribution arises as the distribution of the reciprocal of a gamma variate, and has two positive parameters, α and β. A random variable X is said to have an inverted gamma distribution if its probability density function (pdf) is of the form [29]

$$f_X(x)=\frac{{\beta }^{\alpha }{e}^{-\beta /x}}{\mathrm{\Gamma }(\alpha )x^{\alpha +1}}$$ (1)

when x > 0, and f_X(x) = 0 otherwise. Just as the central χ² distribution is a special case of the gamma distribution, the inverted central χ² distribution is a special case of the inverted gamma distribution. In particular, the reciprocal of a central χ² random variable with ν degrees of freedom is an inverted gamma random variable with α = ν/2 and β = 1/2. An inverted gamma random variable X will be denoted as X ~ IG(α, β). The mean and variance of an inverted gamma deviate are [29]

$$\text{E}[X]=\frac{\beta }{\alpha -1},\text{for}\alpha >1$$ (2)

and

$$\text{Var}[X]=\frac{{\beta }^2}{{(\alpha -1)}^2(\alpha -2)},\text{for}\alpha >2.$$ (3)

2) Noncentral F

The noncentral F distribution [18], [28]–[30] appears as the distribution of the ratio of a noncentral χ² random variable to an independent, central χ² random variable. Specifically, if $X\sim {\chi }_{{\nu }_1}^2(\lambda ),Y\sim {\chi }_{{\nu }_2}^2(0)$ and X and Y are independent, then the random variable F = (X/ν₁)/(Y/ν₂) has a noncentral F distribution with ν₁ and ν₂ degrees of freedom and noncentrality parameter, λ. In this case, we write $F\sim F_{{\nu }_1,{\nu }_2}^{\prime }(\lambda )$. An expression for the pdf of the noncentral F may be found in [28].

3) Wishart

The Wishart distribution [18], [28], [31] is a matrix variate generalization of the χ² distribution and it emerges naturally as the distribution of the sample covariance matrix for multivariate normal measurements [18, p.82], [31, p. 92–93]. Let Z₁, Z₂, … Z_n each be independently distributed according to N_p(0, Σ) and consider the p × p random matrix $W={\sum }_{i=1}^n{\mathbf{Z}}_i{\mathbf{Z}}_i^T$. The matrix W is said to have a Wishart distribution, written as W ~ W_p(n, Σ), with n degrees of freedom and p × p positive definite scale matrix Σ. When n ≥ p, W is invertible and expressions for the pdf of W may be found in [18], [28], [31]. When n < p, W is singular, and the pdf does not exist in the conventional sense, but the distribution is nonetheless defined [18, p. 85].

4) Inverted Wishart

The inverted Wishart distribution originates as the distribution of the inverse of a Wishart distributed random matrix, and it is the matrix variate generalization of the inverted gamma distribution [31, p. 111]. A p × p random matrix V which is distributed as an inverted Wishart with m degrees of freedom and a p × p positive definite parameter matrix, Ψ, will be written as V ~ IW_p(m, Ψ). The inverted Wishart distribution is defined if m > 2p and is undefined otherwise. An expression for the pdf is given in [18], [31]. Lemma 4 in appendix A clarifies the relationship between Wishart and Inverted Wishart matrices. According to this lemma, when n ≥ p, S ~ W_p(n, Σ) if and only if S⁻¹ ~ IW_p(n + p + 1, Σ⁻¹).

B. Channelized Hotelling Observers

We consider a binary classification task in which the goal is to discriminate between two classes of noisy images. The two classes will be denoted as class 1 and class 2, respectively. When the classification task corresponds to lesion detection, we assume that class 1 consists of images with no lesion present, and that class 2 consists of images with a lesion present.

For each noisy image realization, a model observer generates a statistic, t, which is compared to a threshold, t_c, in order to classify the image as belonging to either class 1 or class 2. Specifically, if t > t_c, the observer indicates that the image belongs to class 2, otherwise, the observer indicates that the image belongs to class 1 [1].

Channel filters are often added to model observers in order to reduce the dimensionality of the image data and to model human performance. Suppose that the noisy image realization is represented as a m × 1 column vector, g. We denote the number of channels by p, where usually, p is much smaller than m. The weights that make up each channel are organized into a column of a m × p channel matrix, designated as U. A channelized model observer applies the channel matrix to the image to get a p×1 output vector, v, according to v:= U^Tg. Note that a model observer without channels corresponds to the case when the channel matrix is the identity, i.e., U = I and p = m.

We denote the means of the channel outputs over class 1 and class 2 by μ_i, for i = 1, 2 respectively, and write the difference of these means as Δμ:= μ₂ − μ₁. Also, we write the covariance matrices for v over the two classes as Σ_i, for i = 1, 2 respectively, and designate their average as Σ̄:= 0.5(Σ₁ + Σ₂).

One type of model observer is the Channelized Hotelling observer (CHO) [1], which is the focus of this paper. This observer computes the observer statistic as the inner product of v with a p × 1 template, defined as w_CHO:= Σ̄⁻¹ Δμ, i.e., $t:={\mathbf{w}}_{\text{CHO}}^T\mathbf{v}$.

C. Observer Performance Measures

One measure of an observer’s performance is its signal-to-noise ratio (SNR), defined as the difference of the class means for t divided by the pooled standard deviation [1, p. 819]. The SNR is a good measure of class separability when t is normally distributed for each class, higher values of the observer SNR indicating greater class separation [1, p. 819]. The square of the SNR for a CHO may be computed as [1, p. 967]

$${\text{SNR}}^2=\mathrm{\Delta }{\mathit{\mu }}^T{\overline{\mathrm{\sum }}}^{-1}\mathrm{\Delta }\mathit{\mu }.$$ (4)

The performance of an observer may also be characterized by its receiver operating characteristic (ROC) curve, which plots the true positive fraction (TPF) versus the false positive fraction (FPF) for different values of the discrimination threshold [1, p. 814–815]. A useful summary measure for the overall performance of an observer is the area under the ROC curve, denoted as the AUC. The AUC ranges from 0.5 to 1, where higher values of the AUC indicate better class discrimination. The AUC may be interpreted as the average TPF over all FPF values. Another useful interpretation of the AUC is as the probability that the observer makes a correct classification when a randomly selected pair of images from class 1 and class 2 are compared [32, p. 77–78], [1, p. 823]. If the observer statistic, t, is normally distributed for each class, then the AUC may be computed from the SNR as [1, p. 819]

$$\text{AUC}=(1/2)[1+\text{erf}(\text{SNR}/2)],$$ (5)

where erf(z) is the conventional error function. When t is computed as a linear combination of channel outputs, the observer statistic is typically well-approximated by normal distributions for each class [1, p. 824].

D. Calculation of Moments for AUC estimators

Given a generic estimator of SNR², denoted as $\widehat{{\text{SNR}}^2}$, we may define an estimator for the AUC by substituting the square root of this generic estimator into equation (5). We denote the resulting AUC estimator as $\widehat{\text{AUC}}$.

When the sampling distribution for $\widehat{{\text{SNR}}^2}$ is known, we may use it to calculate the kth raw moment of $\widehat{\text{AUC}}$ by numerically evaluating the integral

$$\text{E}[{\widehat{\text{AUC}}}^k]={\int }_0^{\infty }{f}_X(x){\left(\frac{1}{2}[1+\text{erf}(\sqrt{x}/2)]\right)}^{k}dx,$$ (6)

where f_X(x) is the pdf for $X=\widehat{{\text{SNR}}^2}$. Therefore, we can compute the mean and variance of $\widehat{\text{AUC}}$ by evaluating equation (6) for k = 1, 2.

III. Estimation of SNR² with Known Class Means or Known Difference of Class Means

In this section, estimation of SNR² is considered for two scenarios. In the first scenario, the means for classes 1 and 2 are assumed to be known. In the second scenario, only the difference of the class means is assumed to be known. We introduce one estimator for scenario 1 and two different estimators for scenario 2. We assume that we have n independent realizations of channel outputs from class 1, ${\mathbf{v}}_1^{(1)},{\mathbf{v}}_2^{(1)},\dots ,{\mathbf{v}}_n^{(1)}$, and n independent realizations of channel outputs from class 2, ${\mathbf{v}}_1^{(2)},{\mathbf{v}}_2^{(2)},\dots ,{\mathbf{v}}_n^{(2)}$. Also, we assume that the channel outputs are normally distributed with equal class covariance matrices, i.e., ${\mathbf{v}}_i^{(1)}\sim N_p({\mathit{\mu }}_1,\mathrm{\sum })$ and ${\mathbf{v}}_i^{(2)}\sim N_p({\mathit{\mu }}_2,\mathrm{\sum })$ for i = 1, 2, … n. In this case, Σ̄ = Σ. The normality is a reasonable assumption in many CHO applications thanks to the multivariate central limit theorem, and the equal covariance assumption is easily met for low-contrast signals. After defining each estimator, we provide an analytical expression for the estimator’s sampling distribution and discuss its properties.

A. Scenario 1: Known Class Means

First, we consider the situation when the class means μ₁ and μ₂ are known, and the class covariance matrix, Σ, is unknown.

1) Estimator Definition

In this scenario, unbiased estimates for the covariance matrices of class 1 and class 2 are [18, p. 17]

$${\widehat{S}}_k:=\frac{1}{n}\sum _{i=1}^n({\mathbf{v}}_i^{(k)}-{\mathit{\mu }}_k){({\mathbf{v}}_i^{(k)}-{\mathit{\mu }}_k)}^T$$ (7)

for k = 1, 2, respectively. Let Ŝ:= 0.5(Ŝ₁ + Ŝ₂). From Lemmas 2 and 8(i) in appendix A, it follows that ((2n − p − 1)/(2n))Ŝ⁻¹ is an unbiased estimator for Σ⁻¹ if n > (p + 1)/2. Substituting this estimator for Σ⁻¹ into equation (4) yields an estimator for SNR², defined as

$${\widehat{\theta }}_1:=\left(\frac{2n-p-1}{2n}\right)\mathrm{\Delta }{\mathit{\mu }}^T{\widehat{S}}^{-1}\mathrm{\Delta }\mathit{\mu },$$ (8)

for n > (p + 1)/2.

2) Sampling Distribution

For normally distributed samples, the sampling distribution for θ̂₁ may be expressed as an inverted gamma distribution. We state this result in the following theorem, which is proved in appendix A.

Theorem 1

Suppose that θ̂₁ is computed from independent samples ${\mathbf{v}}_i^{(1)}\sim N_p({\mathit{\mu }}_1,\mathrm{\sum })$ and ${\mathbf{v}}_i^{(2)}\sim N_p({\mathit{\mu }}_2,\mathrm{\sum })$ from classes 1 and 2, respectively, where i = 1, 2, … n. Then

$${\widehat{\theta }}_1\sim IG\left(\frac{2n-p+1}{2},\left(\frac{2n-p-1}{2}\right){\text{SNR}}^2\right)$$

for n > (p + 1)/2.

3) Estimator Properties

Since the sampling distribution for θ̂₁ is an inverted gamma distribution, one may easily find expressions for the mean and variance of θ̂₁ by using Theorem 1 together with equations (2) and (3). We write these expressions in the following corollary.

Corollary 1

If θ̂₁ is computed from independent, normally distributed samples as stated in Theorem 1, then θ̂₁ is an unbiased estimator of SNR², i.e., E[θ̂₁] = SNR² for n > (p+1)/2, and its variance is

$$\text{Var}[{\widehat{\theta }}_1]=\left(\frac{2}{2n-p-3}\right){\text{SNR}}^4,\text{for}n>\frac{p+3}{2}.$$

Also, we have the pleasing result that θ̂₁ is the uniformly minimum variance unbiased (UMVU) estimator for SNR² when the class means are known. For reference, we state this result as a theorem; see appendix B for a proof.

Theorem 2

Suppose that μ₁ and μ₂ are known, and that Σ is unknown. If θ̂₁ is computed from independent, normally distributed samples as stated in Theorem 1 with n > (p+1)/2, then θ̂₁ is the unique UMVU estimator of SNR².

B. Scenario 2: Known Difference of Class Means

Next, we examine the case for which the difference of the class means, Δμ, is known but μ₁, μ₂, and Σ are unknown. We consider two estimators of SNR² for this scenario.

1) Estimator Definitions

The first estimator which we consider for scenario 2 incorporates knowledge of Δμ into the estimation of Σ. The conventional unbiased estimators for the means of classes 1 and 2 are [28, p. 77]

$${\overline{\mathbf{v}}}_k:=\frac{1}{n}\sum _{i=1}^n{\mathbf{v}}_i^{(k)}$$ (9)

for k = 1, 2, respectively. Using equations (9) together with our knowledge of Δμ, we define unbiased estimators for the means of classes 1 and 2 as

$${\stackrel{\sim }{\mathit{\mu }}}_1:=(1/2)({\overline{\mathbf{v}}}_1+{\overline{\mathbf{v}}}_2-\mathrm{\Delta }\mathit{\mu })$$ (10)

and

$${\stackrel{\sim }{\mathit{\mu }}}_2:=(1/2)({\overline{\mathbf{v}}}_1+{\overline{\mathbf{v}}}_2+\mathrm{\Delta }\mathit{\mu }).$$ (11)

Using these estimators for the class means, we define estimators for the covariance matrices of classes 1 and 2 as

$${\stackrel{\sim }{S}}_k:=\frac{1}{n-1/2}\sum _{i=1}^n({\mathbf{v}}_i^{(k)}-{\stackrel{\sim }{\mathit{\mu }}}_k){({\mathbf{v}}_i^{(k)}-{\stackrel{\sim }{\mathit{\mu }}}_k)}^T$$ (12)

for k = 1, 2, respectively. Next, we define a pooled estimator of Σ as S̃:= 0.5(S̃₁ + S̃₂). From Lemmas 2 and 8(ii) in appendix A, it follows that ((2n − p − 2)/(2n − 1))S̃⁻¹ is an unbiased estimator for Σ⁻¹ if n > (p + 2)/2. This suggests the following estimator for SNR²:

$${\widehat{\theta }}_2:=\left(\frac{2n-p-2}{2n-1}\right)\mathrm{\Delta }{\mathit{\mu }}^T{\stackrel{\sim }{S}}^{-1}\mathrm{\Delta }\mathit{\mu },$$ (13)

which is defined for n > (p + 2)/2.

The second estimator which we consider in this scenario does not use knowledge of Δμ for estimation of Σ. The conventional unbiased estimators for the covariance matrices of classes 1 and 2 have the form [28, p. 77]

$$S_k:=\frac{1}{n-1}\sum _{i=1}^n({\mathbf{v}}_i^{(k)}-{\overline{\mathbf{v}}}_k){({\mathbf{v}}_i^{(k)}-{\overline{\mathbf{v}}}_k)}^T$$ (14)

for k = 1, 2, respectively. Define a pooled estimator of Σ as S̄:= 0.5(S₁ + S₂). From Lemmas 2 and 8(iii) in appendix A, it follows that ((2n − p − 3)/(2n − 2)))S̄⁻¹ is an unbiased estimator for Σ⁻¹ if n > (p + 3)/2. This result motivates an estimator for SNR², defined as

$${\widehat{\theta }}_3:=\left(\frac{2n-p-3}{2n-2}\right)\mathrm{\Delta }{\mathit{\mu }}^T{\overline{S}}^{-1}\mathrm{\Delta }\mathit{\mu },$$ (15)

for n > (p + 3)/2.

2) Sampling Distributions

For normally distributed samples, the sampling distributions for θ̂₂ and θ̂₃ may be expressed as inverted gamma distributions. We state these results below. Proofs are provided in appendix A.

Theorem 3

Suppose that θ̂₂ is computed from independent samples ${\mathbf{v}}_i^{(1)}\sim N_p({\mathit{\mu }}_1,\mathrm{\sum })$ and ${\mathbf{v}}_i^{(2)}\sim N_p({\mathit{\mu }}_2,\mathrm{\sum })$ from classes 1 and 2, respectively, where i = 1, 2, … n. Then

$${\widehat{\theta }}_2\sim IG\left(\frac{2n-p}{2},\left(\frac{2n-p-2}{2}\right){\text{SNR}}^2\right),$$

for n > (p + 2)/2.

Theorem 4

Suppose that θ̂₃ is computed from independent samples ${\mathbf{v}}_i^{(1)}\sim N_p({\mathit{\mu }}_1,\mathrm{\sum })$ and ${\mathbf{v}}_i^{(2)}\sim N_p({\mathit{\mu }}_2,\mathrm{\sum })$ from classes 1 and 2, respectively, where i = 1, 2, … n. Then

$${\widehat{\theta }}_3\sim IG\left(\frac{2n-p-1}{2},\left(\frac{2n-p-3}{2}\right){\text{SNR}}^2\right),$$

for n > (p + 3)/2.

3) Estimator Properties

Similar to section III-A3, expressions for the mean and variance of θ̂₂ and θ̂₃ may be easily found by using equations (2) and (3) together with Theorems 3 and 4, respectively. These results are collected below.

Corollary 2

If θ̂₂ is computed from independent, normally distributed samples as stated in Theorem 3, then θ̂₂ is an unbiased estimator of SNR², i.e., E[θ̂₂] = SNR² for n > (p+2)/2, and its variance is

$$\text{Var}[{\widehat{\theta }}_2]=\left(\frac{2}{2n-p-4}\right){\text{SNR}}^4,\text{for}n>\frac{p+4}{2}.$$

Corollary 3

If θ̂₃ is computed from independent, normally distributed samples as stated in Theorem 4, then θ̂₃ is an unbiased estimator of SNR², i.e., E[θ̂₃] = SNR² for n > (p+3)/2, and its variance is

$$\text{Var}[{\widehat{\theta }}_3]=\left(\frac{2}{2n-p-5}\right){\text{SNR}}^4,\text{for}n>\frac{p+5}{2}.$$

As in section III-A3, we have a satisfying optimality property. Namely, if only the difference of class means is known, then θ̂₂ is the UMVU estimator for SNR². This property is stated precisely as the following theorem. A proof may be found in appendix B.

Theorem 5

Suppose that Δμ is known, and that μ₁, μ₂ and Σ are unknown. If θ̂₂ is computed from independent, normally distributed samples as stated in Theorem 3 with n > (p + 2)/2, then θ̂₂ is the unique UMVU estimator of SNR².

From the above results, it is clear that θ̂₃ is not the UMVU estimator for SNR² in scenario 2. However, θ̂₃ is the UMVU estimator within the family of unbiased estimators of SNR² that disregard the knowledge that Δμ is equal to μ₂ − μ₁. This statement is clarified by the following theorem. We omit the proof, which is similar to that of Theorem 5 in appendix B.

Theorem 6

Suppose that ${\mathbf{v}}_i^{(1)}\sim N_p({\mathit{\mu }}_1,\mathrm{\sum })$ and ${\mathbf{v}}_i^{(2)}\sim N_p({\mathit{\mu }}_2,\mathrm{\sum })$ are independent samples from classes 1 and 2, respectively, where i = 1, 2, … n with n > (p + 3)/2, and where μ₁, μ₂, and Σ are unknown. Let δ be an arbitrary, known p × 1 column vector, and define the estimator

$$\widehat{\phi }:=\left(\frac{2n-p-3}{2n-2}\right){\mathit{\delta }}^T{\overline{S}}^{-1}\mathit{\delta }.$$

Then φ̂ is the unique UMVU estimator of δ^TΣ⁻¹δ.

IV. Computational Evaluations

Now, we further explore the theoretical properties of the estimators introduced in the last section. We will compare these estimators to a conventional estimator for SNR², which is reviewed first. As before, we assume that there are ${\mathbf{v}}_1^{(1)},{\mathbf{v}}_2^{(1)},\dots ,{\mathbf{v}}_n^{(1)}$ independent realizations of channel outputs from class 1 and ${\mathbf{v}}_1^{(2)},{\mathbf{v}}_2^{(2)},\dots ,{\mathbf{v}}_n^{(2)}$ independent realizations of channel outputs from class 2. Also, we will be assuming that p = 40 channels are used. There is nothing special about p = 40, as opposed to another value, such as p = 4, which is typical for isotropic channels. Our choice of p = 40 is motivated by our current interest in evaluating the impact of anisotropic noise on lesion detectability [11].

A. An Estimator for SNR² When the Class Means and Their Difference Are Unknown

Suppose that the class means, μ₁ and μ₂, and their difference, Δμ, are unknown. In this situation, the conventional method to construct an estimator for SNR² is to substitute the sample estimates Δv̄:= v̄₂ − v̄₁ and S̄ for Δμ and Σ in equation (4). This approach is used in [1, p. 972], and yields an estimator defined as

$${\widehat{\theta }}_4:=\mathrm{\Delta }{\overline{\mathbf{v}}}^T{\overline{S}}^{-1}\mathrm{\Delta }\overline{\mathbf{v}},\text{for}n>\frac{p+1}{2}.$$ (16)

The sampling distribution for θ̂₄ is known in the case when the samples are normally distributed with equal covariance matrices, i.e., ${\mathbf{v}}_i^{(1)}\sim N_p({\mathit{\mu }}_1,\mathrm{\sum })$ and ${\mathbf{v}}_i^{(2)}\sim N_p({\mathit{\mu }}_2,\mathrm{\sum })$ for i = 1, 2, … n. Specifically, [18, p. 216]

$$\left(\frac{n(2n-p-1)}{4p(n-1)}\right){\widehat{\theta }}_4\sim F_{{\nu }_1,{\nu }_2}^{\prime }(\lambda ),\text{for}n>\frac{p+1}{2},$$ (17)

where $F_{{\nu }_1,{\nu }_2}^{\prime }(\lambda )$ is the non-central F distribution with ν₁ = p and ν₂ = 2n − p − 1 degrees of freedom and noncentrality parameter, λ = (n/2)SNR². Expressions for the mean and variance for the noncentral F distribution [18], [30] may be used to derive formulae for the bias and variance of θ̂₄. These formulae are [33]

$$\begin{array}{l}\text{bias}[{\widehat{\theta }}_4]=\text{E}[{\widehat{\theta }}_4]-{\text{SNR}}^2\\ =\frac{n(p+1){\text{SNR}}^2+4(n-1)p}{2n^2-np-3n}\end{array}$$ (18)

for n > (p + 3)/2, and

$$\begin{array}{l}\text{Var}[{\widehat{\theta }}_4]\\ =\frac{32{(n-1)}^2[\frac{n^2}{4}{\text{SNR}}^4+(2n-3)(n{\text{SNR}}^2+p)]}{n^2{(2n-p-3)}^2(2n-p-5)}\end{array}$$ (19)

for n > (p + 5)/2.

B. Comparison of the SNR² Estimators

In the first set of comparisons, the relative error for each of the four SNR² estimators is plotted as a function of the total number of images, 2n; see Figure 1. The plots assume SNR² values of 0.12837, 0.90987, and 3.2847, corresponding to AUC values of 0.6, 0.75, and 0.9, respectively. Here the relative error (in %) is calculated as the square root of the mean-square error (MSE) divided by the SNR² value times 100. Using the fact that the MSE is equal to the bias squared plus the variance [34], the MSE for θ̂₁, θ̂₂, θ̂₃ and θ̂₄ was calculated using the expressions from Corollaries 1, 2, and 3 and equations (18) and (19).

Fig. 1.

Relative error of SNR² estimators as a function of the total number of images, 2n. The plots assume p = 40 channels with SNR² = 0.12837 (left), 0.90987 (middle), and 3.2847 (right), corresponding to AUC values of 0.6, 0.75, and 0.9, respectively. The estimators are denoted as follows: θ̂₁ (circles), θ̂₂ (diamonds), θ̂₃ (triangles), and θ̂₄ (squares). The relative error (in %) is calculated as $(\sqrt{\text{MSE}}/({\text{SNR}}^2))\times 100$, where MSE denotes the mean-square error.

Comparing θ̂₁, θ̂₂, and θ̂₃ in the plots, we observe that their relative errors are very similar. This is not surprising, since they are all unbiased and their variance expressions are comparable. In addition, we note that for fixed n and p, the relative errors for θ̂₁, θ̂₂ and θ̂₃ are invariant over the different SNR² values. Although this invariance is only shown in the plots for particular values of n and p, it actually holds for any choice of n and p. This follows from the fact that the standard deviation of each estimator divided by SNR² only depends on n and p; this quantity is independent of SNR² and also of the correlation matrix and the class means.

On the other hand, the relative error for θ̂₄ decreases as the true SNR² value increases. Furthermore, we see that the relative error for θ̂₄ is significantly larger than that for θ̂₁, θ̂₂, and θ̂₃. For example, in the case of SNR² = 0.90987 with 2n = 250 total images, the relative error for θ̂₁ is 9.8%, compared to a relative error of 112.3% for θ̂₄.

C. Comparison of the AUC Estimators

For the next set of comparisons, we consider the AUC estimators which are found by substituting the square root of θ̂_i for SNR in equation (5). For each i ∈ {1, 2, 3, 4}, the resulting AUC estimator will be denoted as ${\widehat{\text{AUC}}}_i$.

In Figure 2, the bias magnitude and relative error for each of the AUC estimators are plotted as functions of the total number of images, 2n. The plots compare the estimator behavior for AUC values of 0.6, 0.75, and 0.9. The bias magnitude (in %) is calculated by dividing the absolute value of the bias by the AUC value and multiplying by 100. The relative error (in %) is calculated as the square root of the MSE divided by the AUC value times 100. The bias and MSE for each AUC estimator were computed from the estimator’s mean and variance, which were calculated numerically using the pdf of the corresponding θ̂_i and the method described in section II-D. In all cases, the numerical error for the mean and variance results was constrained to be less than 10⁻⁶.

Fig. 2.

Bias magnitude and relative error of AUC estimators as a function of the total number of images, 2n. The plots assume p = 40 channels with AUC = 0.6 (left), 0.75 (middle), and 0.9 (right). Bias magnitude is on the top and relative error is on the bottom. The estimators are ${\widehat{\text{AUC}}}_1$ (circles), ${\widehat{\text{AUC}}}_2$ (diamonds), ${\widehat{\text{AUC}}}_3$ (triangles), ${\widehat{\text{AUC}}}_4$ (squares). For each i ∈ {1, 2, 3, 4}, ${\widehat{\text{AUC}}}_i$ is calculated by substituting the square root of θ̂_i for SNR in equation (5). The bias magnitude (in %) for an AUC estimator, ${\widehat{\text{AUC}}}_i$, is calculated as $(\mid \text{E}[{\widehat{\text{AUC}}}_i]-\text{AUC}\mid /\text{AUC})\times 100$. The relative error (in %) is calculated as $(\sqrt{\text{MSE}}/\text{AUC})\times 100$, where MSE denotes the mean-square error.

Examining the bias magnitude plots in Figure 2 (top row), we see that the bias magnitudes for ${\widehat{\text{AUC}}}_1,{\widehat{\text{AUC}}}_2$, and ${\widehat{\text{AUC}}}_3$ are very similar, and increase at fixed n with increasing AUC values, but are generally very small. Specifically, for 2n ≥ 100, the bias magnitudes for these three estimators are all less than 0.3%. In contrast, the bias magnitude for ${\widehat{\text{AUC}}}_4$ decreases with increasing AUC values, but it is always much larger than for the other three estimators. For instance, in the case of 2n = 100, the bias magnitude for ${\widehat{\text{AUC}}}_4$ has values of 47.1%, 23.1%, and 8.4%, corresponding to AUC values of 0.6, 0.75, and 0.9, respectively. Although not reflected in the plots, the biases for ${\widehat{\text{AUC}}}_1,{\widehat{\text{AUC}}}_2$ and ${\widehat{\text{AUC}}}_3$ were negative in all cases and the bias for ${\widehat{\text{AUC}}}_4$ was positive in all cases.

Directing our attention to the relative error plots in Figure 2 (bottom row), the trends of ${\widehat{\text{AUC}}}_1,{\widehat{\text{AUC}}}_2$, and ${\widehat{\text{AUC}}}_3$ are again seen to be very similar. For 2n ≥ 100, the relative errors for these three estimators are less than 2.7%. On the other hand, the relative error for ${\widehat{\text{AUC}}}_4$ is significantly larger. For example, when 2n = 100, the relative error for ${\widehat{\text{AUC}}}_4$ has values of 47.4%, 23.4%, and 8.55% corresponding to AUC values of 0.6, 0.75, and 0.9, respectively. Comparing these relative errors to the bias magnitude values mentioned above, we see that most of the relative error for ${\widehat{\text{AUC}}}_4$ is due to bias.

V. Discussion and Conclusions

In this work, we were interested in the estimation of classical performance metrics for discrimination between two classes of images. The discrimination was assumed to be performed by a channelized Hotelling observer. Also, we assumed that the estimation is to be carried out using a small number of image realizations, as is typically the case in medical imaging.

Unfortunately, access to a limited number of image realizations implies that the estimated performance values will have some statistical errors, quantified, for example, in terms of estimator bias and variance. Barrett and Myers [1, p. 972] suggested that these statistical errors could potentially be reduced if the class means are assumed to be known. We have investigated this issue thoroughly. Assuming that the channel outputs corresponding to the two classes of images have multivariate normal distributions with equal covariance matrices, we have completely characterized the sampling distributions for three estimators of SNR² that assume that the class means, or their difference, are known. Using these distributions, we have demonstrated that a significant decrease in estimation error can be achieved when the class means, or their difference, are known.

A useful property of our three SNR² estimators is the fact that the ratio of each estimator’s standard deviation to its mean is a function of only n and p. This property allows for easy pre-selection of the number of images needed to achieve a desired level of accuracy.

Another interesting property of our three SNR² estimators is that they yield AUC estimators that have negative bias, just as estimators based on a training/testing paradigm [16], [17]. Thus, we demonstrated that the training/testing paradigm is not the only estimation strategy that can result in negative biases. Moreover, recall from Figure 2 that the biases achieved by our new estimators are fairly small.

Of course, it is natural to ask in which situations the image class means, or their difference, may be found. One situation where this is possible is for image reconstruction from simulated data with the data mean known. For example, in the case of linear reconstruction algorithms (e.g, filtered backprojection (FBP) type), which are still the methods of choice in CT, the image class means may be found by simply reconstructing the means of the data for each class. Similarly, for many types of (nonlinear) iterative reconstruction methods, the image class means may be found to first order by reconstructing the means of the data for each class [35]. This approximation has been found to be fairly accurate for the cases of expectation maximization (EM) [36], [37] and penalized-likelihood reconstruction [35].

When significant uncertainty exists for the class mean estimates, our estimators may not be satisfactory. This could be the case for iterative reconstruction methods if first-order estimates of the means are not accurate enough. This could also be the case for image quality assessment from real data, because the class means of real data are rarely known exactly. For these situations, it would be useful to understand how our estimators are affected by errors in the mean estimates. Moreover, as refined estimates of the class means may possibly be obtained at low cost, we note that knowledge of the distributions for our estimators may pave the way for new attractive estimators that incorporate prior information pertaining to the class means. These topics define interesting areas of further research.

In order to safely apply our results to a particular imaging application, we emphasize that it is necessary to verify that the channel outputs corresponding to the two classes of images are well-approximated by multivariate normal distributions that have the same covariance matrix. We are currently assessing the validity of this assumption for x-ray CT. This assessment will include a comparison of our approach to nonparametric AUC estimation using the Mann-Whitney U statistic.

Last, note that our estimators allow for a significant improvement in efficiency for the computation of detectability maps. These maps plot SNR or AUC as a function of lesion location [1, p. 858] [38]. This is one way to investigate the dependence of lesion detectability on background variation [1, p. 858–859]. In the future, we will evaluate our estimators for the purpose of building detectability maps for commercially-available FBP reconstruction methods in x-ray CT.

Appendix A

In this appendix, we prove Theorems 1, 3, and 4 which characterize the sampling distributions of θ̂₁, θ̂₂, and θ̂₃. For this task, we first review several useful properties of the Wishart, inverted Wishart, and inverted gamma distributions.

Lemma 1

If A₁ ~ W_p(m, Σ) and A₂ ~ W_p(n, Σ) are independent, p×p random matrices, then A₁ + A₂ ~ W_p(m+ n, Σ).

Proof

See [18, Thm 3.2.4, p.91] or [31, Thm 3.3.8, p.94].

Lemma 2

Let n > p + 1. If S ~ W_p(n, Σ), then

$$\text{E}[S^{-1}]=\frac{1}{n-p-1}{\mathrm{\sum }}^{-1}.$$

Proof

See [18, p. 97] or [28, p. 273].

Lemma 3

Let n ≥ p. If S ~ W_p(n, Σ) and A is a q × p matrix of rank q ≤ p, then

$${(A{S}^{-1}{A}^T)}^{-1}\sim W_q(n-p+q,{(A{\mathrm{\sum }}^{-1}{A}^T)}^{-1}).$$

Proof

See [18, Thm 3.2.11, p. 95] or [31, Thm 3.3.13, p. 97].

Lemma 4

Let n ≥ p. Then S ~ W_p(n, Σ) if and only if S⁻¹ ~ IW_p(n + p + 1, Σ⁻¹).

Proof

See [31, Thm 3.4.1, p. 111].

Lemma 5

Let n ≥ p. If S ~ W_p(n, Σ) and A is a q × p matrix of rank q ≤ p, then

$$A{S}^{-1}{A}^T\sim I{W}_q(n-p+2q+1,A{\mathrm{\sum }}^{-1}{A}^T).$$

Proof

This follows immediately from Lemmas 3 and 4.

Lemma 6

If X ~ IW₁(m, Ψ) and m > 2, then $X\sim IG(\frac{m-2}{2},\frac{\mathrm{\Psi }}{2})$.

Proof

Suppose that X ~ IW₁(m, Ψ) and m > 2. First, note that since X is an inverted Wishart matrix with p = 1, X and Ψ are scalars. For the p = 1 case, the pdf of the inverted Wishart distribution [31, p. 111] is

$$f_X(x)=\frac{{(\mathrm{\Psi }/2)}^{\frac{1}{2}(m-2)}{e}^{-\frac{\mathrm{\Psi }}{2x}}}{\mathrm{\Gamma }(\frac{1}{2}(m-2))x^{\frac{1}{2}m}}.$$ (20)

Comparing with equation (1), we recognize the above expression as the pdf for an inverted gamma random variable with α = (m − 2)/2 and β = Ψ/2.

Lemma 7

Let c > 0 be an arbitrary constant. If X ~ IG(α, β), then Y = cX ~ IG(α, cβ).

Proof

Suppose that X ~ IG(α, β) and let Y = cX. The pdf for Y may be expressed in terms of the pdf for X by applying the rule for a monotonic transformation of a random variable [39, p. 51], i.e., f_Y (y) = (1/c)f_X(y/c). Using this rule together with equation (1), we may express the pdf for Y as

$$f_Y(y)=\frac{1}{c}\left(\frac{{\beta }^{\alpha }{e}^{-c\beta /y}}{\mathrm{\Gamma }(\alpha )}\right){\left(\frac{c}{y}\right)}^{\alpha +1}=\frac{{(c\beta )}^{\alpha }{e}^{-c\beta /y}}{\mathrm{\Gamma }(\alpha )y^{\alpha +1}}.$$ (21)

Hence, Y ~ IG(α, cβ).

Next, we collect some results concerning the distributions of Ŝ, S̃, and S̄.

Lemma 8

Suppose that ${\mathbf{v}}_i^{(1)}\sim N_p({\mathit{\mu }}_1,\mathrm{\sum })$ and ${\mathbf{v}}_i^{(2)}\sim N_p({\mathit{\mu }}_2,\mathrm{\sum })$ are independent samples from classes 1 and 2, respectively, where i = 1, 2, … n. If Ŝ, S̃, and S̄ are computed from these samples according to equations (7), (12), and (14), respectively, then (i) 2nŜ ~ W_p(2n, Σ), (ii) (2n − 1) S̃ ~ W_p (2n − 1, Σ), and (iii) (2n − 2) S̄ ~ W_p(2n − 2, Σ).

Proof

1. First, observe that Ŝ₁ may be expressed as $n\widehat{S_1}={\sum }_{i=1}^n{\mathbf{Z}}_i{\mathbf{Z}}_i^T$, where ${\mathbf{Z}}_i={\mathbf{v}}_i^{(1)}-{\mathit{\mu }}_1$. Since Z₁, Z₂, … Z_n are independently distributed according to N_p(0, Σ), it follows from the definition of the Wishart distribution [18, p. 82] that nŜ₁ ~ W_p(n, Σ). By a similar argument, nŜ₂ ~ W_p(n, Σ). Because nŜ₁ and nŜ₂ are independent, Lemma 1 implies that 2nŜ = nŜ₁ + nŜ₂ ~ W_p(2n, Σ).

2. From the definition of S̃, we have

   $$\begin{array}{l}(2n-1)\stackrel{\sim }{S}=\sum _{i=1}^n({\mathbf{v}}_i^{(1)}-{\stackrel{\sim }{\mathit{\mu }}}_1){({\mathbf{v}}_i^{(1)}-{\stackrel{\sim }{\mathit{\mu }}}_1)}^T\\ +\sum _{i=1}^n({\mathbf{v}}_i^{(2)}-{\stackrel{\sim }{\mathit{\mu }}}_2){({\mathbf{v}}_i^{(2)}-{\stackrel{\sim }{\mathit{\mu }}}_2)}^T.\end{array}$$ (22)
   Substituting $({\mathbf{v}}_i^{(1)}-{\overline{\mathbf{v}}}_1+{\overline{\mathbf{v}}}_1-{\stackrel{\sim }{\mathit{\mu }}}_1)$ for $({\mathbf{v}}_i^{(1)}-{\stackrel{\sim }{\mathit{\mu }}}_1)$ and $({\mathbf{v}}_i^{(2)}-{\overline{\mathbf{v}}}_2+{\overline{\mathbf{v}}}_2-{\stackrel{\sim }{\mathit{\mu }}}_2)$ for $({\mathbf{v}}_i^{(2)}-{\stackrel{\sim }{\mathit{\mu }}}_2)$ into the definition, and rearranging, we find

   $$\begin{array}{l}(2n-1)\stackrel{\sim }{S}=\sum _{i=1}^n({\mathbf{v}}_i^{(1)}-{\overline{\mathbf{v}}}_1){({\mathbf{v}}_i^{(1)}-{\overline{\mathbf{v}}}_1)}^T\\ +\sum _{i=1}^n({\mathbf{v}}_i^{(2)}-{\overline{\mathbf{v}}}_2){({\mathbf{v}}_i^{(2)}-{\overline{\mathbf{v}}}_2)}^T\\ +n({\overline{\mathbf{v}}}_1-{\stackrel{\sim }{\mathit{\mu }}}_1){({\overline{\mathbf{v}}}_1-{\stackrel{\sim }{\mathit{\mu }}}_1)}^T\\ +n({\overline{\mathbf{v}}}_2-{\stackrel{\sim }{\mathit{\mu }}}_2){({\overline{\mathbf{v}}}_2-{\stackrel{\sim }{\mathit{\mu }}}_2)}^T.\end{array}$$ (23)
   Inserting the definitions of μ̃ ₁and μ̃₂ given by equations (10) and (11) into the last equation, and simplifying, yields

   $$\begin{array}{l}(2n-1)\stackrel{\sim }{S}=\sum _{i=1}^n({\mathbf{v}}_i^{(1)}-{\overline{\mathbf{v}}}_1){({\mathbf{v}}_i^{(1)}-{\overline{\mathbf{v}}}_1)}^T\\ +\sum _{i=1}^n({\mathbf{v}}_i^{(2)}-{\overline{\mathbf{v}}}_2){({\mathbf{v}}_i^{(2)}-{\overline{\mathbf{v}}}_2)}^T\\ +\left(\frac{n}{2}\right)(\mathrm{\Delta }\overline{\mathbf{v}}-\mathrm{\Delta }\mathit{\mu }){(\mathrm{\Delta }\overline{\mathbf{v}}-\mathrm{\Delta }\mathit{\mu })}^T.\end{array}$$ (24)

   By a theorem concerning the distribution of the sample covariance matrix [31, Thm 3.3.2, p. 92], the first and second terms in equation (24) are distributed as W_p(n − 1, Σ). Also, using standard results for the sample mean and multivariate normal distribution [18], it follows that $\sqrt{n/2}(\mathrm{\Delta }\overline{\mathbf{v}}-\mathrm{\Delta }\mu )\sim N_p(\mathbf{0},\mathrm{\sum })$. Therefore, by the definition of the Wishart distribution [18, p. 82], we see that the third term in equation (24) has a W_p(1, Σ) distribution.

   The first and second terms in equation (24) are clearly independent because they are computed from independent samples. Moreover, for the same reason, we note that the sample means, v̄₁ and v̄₂, are independent of the second and first terms, respectively. Also, by a standard theorem [31, Thm. 3.3.6, p. 92], v̄₁ and v̄₂ are independent of the first and second terms, respectively. Hence, the third term is independent of the first and second terms. Because the three terms in equation (24) are independent, we may apply Lemma 1 to conclude that (2n − 1)S̃ ~ W_p(2n − 1, Σ).

3. By a theorem for the distribution of the sample covariance matrix [31, Thm 3.3.2, p. 92], we have (n−1)S₁ ~ W_p(n−1, Σ) and (n−1)S₂ ~ W_p(n−1, Σ). In addition, since S₁ and S₂ are computed from independent samples, they are independent. Thus, Lemma 1 implies that (2n − 2)S̄ = (n − 1)S₁ + (n − 1)S₂ ~ W_p(2n − 2, Σ).

Now, we are ready to prove Theorems 1, 3, and 4. Recall that each of these estimators is computed from independent samples ${\mathbf{v}}_i^{(1)}\sim N_p({\mathit{\mu }}_1,\mathrm{\sum })$ and ${\mathbf{v}}_i^{(2)}\sim N_p({\mathit{\mu }}_2,\mathrm{\sum })$ from classes 1 and 2, respectively, where i = 1, 2, … n.

Proof of Theorem 1

Assume n > (p+ 1)/2. The definition of θ̂₂ in equation (8) may be rewritten as (1/(2n − p − 1)) θ̂₁ = Δμ^T (2nŜ)⁻¹Δμ. From Lemma 8(i), 2nŜ ~ W_p(2n, Σ). Hence, we may apply Lemma 5 and equation (4) to find that (1/(2n − p − 1)) θ̂₁ ~ IW₁(2n − p + 3, SNR²). Lemma 6 then yields (1/(2n − p −1)) θ̂₁ ~ IG((2n − p + 1)/2, (1/2)SNR²). Finally, application of Lemma 7 gives the desired result.

Proof of Theorem 3

Assume n > (p + 2)/2. The definition of θ̂₂ in equation (13) may be rewritten as (1/(2n − p − 2)) θ̂₂ = Δμ^T [(2n − 1)S̃]⁻¹Δμ. From Lemma 8(ii), (2n − 1)S̃ ~ W_p(2n − 1, Σ). Hence, we may apply Lemma 5 and equation (4) to find that (1/(2n−p−2)) θ̂₂ ~ IW₁(2n − p + 2, SNR²). Lemma 6 then yields (1/(2n − p − 2)) θ̂₂ ~ IG((2n − p)/2, (1/2)SNR²). Finally, application of Lemma 7 gives the desired result.

Proof of Theorem 4

Assume n > (p + 3)/2. The definition of θ̂₃ in equation (15) may be rewritten as (1/(2n − p − 3)) θ̂₃ = Δμ^T [(2n − 2) S̄]⁻¹Δμ. From Lemma 8(iii), (2n − 2)S̄ ~ W_p(2n − 2, Σ). Hence, we may apply Lemma 5 and equation (4) to find that (1/(2n−p−3)) θ̂₃ ~ IW₁(2n − p + 1, SNR²). Lemma 6 then yields (1/(2n −p −3)) θ̂₃ ~ IG((2n − p − 1)/2, (1/2)SNR²). Finally, application of Lemma 7 gives the desired result.

Appendix B

In this appendix, we prove Theorems 2 and 5, which state that θ̂₁ and θ̂₂ are UMVU estimators. Again, we assume that each of these estimators is computed from independent samples ${\mathbf{v}}_i^{(1)}\sim N_p({\mathit{\mu }}_1,\mathrm{\sum })$ and ${\mathbf{v}}_i^{(2)}\sim N_p({\mathit{\mu }}_2,\mathrm{\sum })$ from classes 1 and 2, respectively, where i = 1, 2, … n.

Proof of Theorem 2

Assume n > (p + 1)/2. Also, suppose that μ₁ and μ₂ are known, and that Σ is unknown. First, we show that Ŝ is a sufficient statistic for the joint pdf of the sample. The joint pdf of ${\mathbf{v}}_1^{(1)},{\mathbf{v}}_2^{(1)},\dots ,{\mathbf{v}}_n^{(1)}$ and ${\mathbf{v}}_1^{(2)},{\mathbf{v}}_2^{(2)},\dots ,{\mathbf{v}}_n^{(2)}$ is

$$f({\mathbf{v}}_1^{(1)},\dots ,{\mathbf{v}}_n^{(1)},{\mathbf{v}}_1^{(2)},\dots ,{\mathbf{v}}_n^{(2)})={(2\pi )}^{-\frac{np}{2}}\mid \mathrm{\sum }{\mid }^{-\frac{n}{2}}{e}^{\eta },$$ (25)

where

$$\begin{array}{l}\eta =-\frac{1}{2}\sum _i^n[{({\mathbf{v}}_i^{(1)}-{\mathit{\mu }}_1)}^T{\mathrm{\sum }}^{-1}({\mathbf{v}}_i^{(1)}-{\mathit{\mu }}_1)\\ +{({\mathbf{v}}_i^{(2)}-{\mathit{\mu }}_2)}^T{\mathrm{\sum }}^{-1}({\mathbf{v}}_i^{(2)}-{\mathit{\mu }}_2)].\end{array}$$ (26)

Using the additive and cyclic properties of the trace, denoted tr, we may rewrite η as

$$\begin{array}{l}\eta =-\frac{1}{2}\text{tr}({\mathrm{\sum }}^{-1}(\sum _i^n[({\mathbf{v}}_i^{(1)}-{\mathit{\mu }}_1){({\mathbf{v}}_i^{(1)}-{\mathit{\mu }}_1)}^T\\ +({\mathbf{v}}_i^{(2)}-{\mathit{\mu }}_2){({\mathbf{v}}_i^{(2)}-{\mathit{\mu }}_2)}^T])),\end{array}$$ (27)

i.e.,

$$\eta =(-n)\text{tr}\left({\mathrm{\sum }}^{-1}\widehat{S}\right).$$ (28)

Hence, the joint pdf has the form

$$\begin{array}{l}f({\mathbf{v}}_1^{(1)},\dots ,{\mathbf{v}}_n^{(1)},{\mathbf{v}}_1^{(2)},\dots ,{\mathbf{v}}_n^{(2)})\\ ={(2\pi )}^{-\frac{np}{2}}\mid \mathrm{\sum }{\mid }^{-\frac{n}{2}}exp\left[(-n)\text{tr}\left({\mathrm{\sum }}^{-1}\widehat{S}\right)\right].\end{array}$$ (29)

By the Fisher-Neyman factorization theorem [40, Thm. 6.5, p. 35] [41, Prop. IV.C.1, p. 159], Ŝ is a sufficient statistic. Moreover, because the expression in equation (29) has the form of a full rank exponential family [40, p. 23–24], Ŝ is a complete statistic [40, Thm. 6.22, p. 42]. Since (i) Ŝ is a complete sufficient statistic, (ii) θ̂₁ is an unbiased estimator of SNR², and (iii) θ̂₁ = E[θ̂₁|Ŝ], i.e., θ̂₁ is a function of Ŝ only, the Lehmann-Scheffé Theorem [40, Thm. 1.11, p. 88] [41, p. 164] implies that θ̂₁ is the unique UMVU estimator of SNR² in scenario 1.

Proof of Theorem 5

Assume n > (p + 2)/2. Also, suppose that Δμ is known, and that μ₁, μ₂, and Σ are unknown. The joint pdf of ${\mathbf{v}}_1^{(1)},{\mathbf{v}}_2^{(1)},\dots ,{\mathbf{v}}_n^{(1)}$ and ${\mathbf{v}}_1^{(2)},{\mathbf{v}}_2^{(2)},\dots ,{\mathbf{v}}_n^{(2)}$ is given by equations (25) and (26). After lengthy algebra, one may rewrite the joint pdf in the form

$$\begin{array}{l}f({\mathbf{v}}_1^{(1)},\dots ,{\mathbf{v}}_n^{(1)},{\mathbf{v}}_1^{(2)},\dots ,{\mathbf{v}}_n^{(2)})\\ ={(2\pi )}^{-\frac{np}{2}}\mid \mathrm{\sum }{\mid }^{-\frac{n}{2}}exp[(-n)\text{tr(}{\mathrm{\sum }}^{-1}{\mathit{\mu }}_{\text{1}}{\mathit{\mu }}_{\text{2}}^T\text{)}]\\ \times exp[\text{tr}({\mathrm{\sum }}^{-1}((2n-1)\stackrel{\sim }{S}\\ -n({\overline{\mathbf{v}}}_1+{\overline{\mathbf{v}}}_2-\mathrm{\Delta }\mathit{\mu }){({\overline{\mathbf{v}}}_1+{\overline{\mathbf{v}}}_2-\mathrm{\Delta }\mathit{\mu })}^T))]\\ \times exp[(-n)\text{tr}({\mathrm{\sum }}^{-1}{\mathit{\mu }}_1{({\overline{\mathbf{v}}}_1+{\overline{\mathbf{v}}}_2)}^T)].\end{array}$$ (30)

By the Fisher-Neyman factorization theorem [40, Thm. 6.5, p. 35] [41, Prop. IV.C.1, p. 159], the statistic

$$T:=\left[\begin{array}{c}{\overline{\mathbf{v}}}_1+{\overline{\mathbf{v}}}_2\\ \tau \end{array}\right],$$ (31)

where τ = (2n−1)S̃ −n(v̄₁ + v̄₂ − Δμ)(v̄₁ + v̄₂ − Δμ)^T, is sufficient. Moreover, because the expression in equation (30) has the form of a full rank exponential family [40, p. 23–24], T is a complete statistic [40, Thm. 6.22, p. 42]. Since (i) T is a complete sufficient statistic, (ii) θ̂₂ is an unbiased estimator of SNR², and (iii) θ̂₂ = E[θ̂₂|T], i.e., θ̂₂ is a function of T only, the Lehmann-Scheffé Theorem [40, Thm. 1.11, p. 88] [41, p. 164] implies that θ̂₂ is the unique UMVU estimator of SNR² in scenario 2.