Analytic, Computational, and Approximate Forms for Ratios of Noncentral and Central Gaussian Quadratic Forms

Abstract

Many useful statistics equal the ratio of a possibly noncentral chi-square to a quadratic form in Gaussian variables with all positive weights. Expressing the density and distribution function as positively weighted sums of corresponding F functions has many advantages. The mixture forms have analytic value when embedded within a more complex problem. The mixture forms also have computational value. The expansions work well with quadratic forms having few components and small degrees of freedom. A more general algorithm from earlier literature can take longer or fail to converge in the same setting. Many approximations have been suggested for the problem. a positively weighted noncentral quadratic form can always have two moments matched to a noncentral chi-square. For a single quadratic form, the noncentral form performs neither uniformly more or less accurately than older approximations. The approach also gives a noncentral F approximation for any ratio of a positively weighted noncentral form to a positively weighted central quadratic form. The method provides better accuracy for noncentral ratios than approximations based on a single chi-square. The accuracy suffices for many practical applications, such as power analysis, even with few degrees of freedom. Naturally the approximation proves much faster and simpler to compute than any exact method. Embedding the approximation in analytic expressions provides simple forms which correctly guarantee only positive values have nonzero probabilities, and also automatically reduce to partially or fully exact results when either quadratic form has only one term.

1. Introduction

Many statistics reduce to the ratio of a possibly noncentral chi-square to a positively weighted quadratic form in Gaussian variables. We focus primarily on computing exactly or accurately approximating the distribution function of such a form. The results lead us to consider more general but closely related ratios with the single chi-square replaced by a positively weighted noncentral quadratic form.

Often both the density and the cumulative distribution function (CDF) of such ratios cannot be found, except in special cases. However, the distribution function may be expressed in terms of a probability for a quadratic form with both positive and negative weights. Davies's (1980) method applies to such forms, and hence allows computing probabilities for the original ratio of interest. The approach centers on numerical integration of equation (3.2) in Imhof (1961).

Davies's approach has important drawbacks. First, it is strictly a numerical procedure, based on numerical integration of the characteristic function. Therefore it does not allow convenient analytic manipulation when nested inside other statistical problems. Second, careful implementation requires a large amount of code. Third, as Davies noted, the method may converge slowly, or fail to converge, particularly when a single term with a few degrees of freedom dominates. Forms of interest in the present article seem likely to exhibit the problems.

Interest in ratios of quadratic forms has a long history, especially from the perspective of mixture representations. Robbins and Pitman (1949) discussed mixture distributions in general. They also described series expansions for (1) a positively weighted sum of central quadratic forms; (2) a ratio of such forms; and (3) the ratio of a single, noncentral chi-square to the weighted sum of two central chi-square variables. Ruben (1962) described an expansion for the distribution function of positively weighted sums of noncentral chi-square as a mixture of central chi-square variables, as well as a second expansion as a mixture of noncentral chi-square variables. A number of Robbins and Pitman's results are special cases of Ruben's results.

Many others have contributed. Johnson and Kotz (1970, chap. 29, pp. 169–173) and Johnson, Kotz, and Balakrishnan (1994) summarized both theory and computational practice. Davies (1980) and Shively, Ansley, and Kohn (1990) devised algorithms for computing the density and CDF of a ratio of quadratic forms. Mathai and Provost (1992) reviewed approximations to the CDF of quadratic forms. One approach focuses on determining the moments, or approximations to the moments, to define an approximating CDF (Provost and Rudiuk 1992; Smith 1996). In a related approach, Lieberman (1994) described saddlepoint approximations for the density of a ratio of quadratic forms in normal variables for the central and noncentral case, respectively.

A wide variety of CDF approximations for such ratios use the CDF of another random variable to simplify computation. Matching moments of the quadratic form in the ratio to a scaled chi-square, which corresponds to Satterthwaite's (1946) approach, gives an F approximation. Expressing the desired CDF in terms of the CDF of a quadratic form with both positive and negative weights leads to a different set of approximations. Imhof (1961) described an especially accurate approximation, based on matching three moments of the quadratic form to a central chi-square. Muller and Barton (1989) suggested approximating the special case of a positively weighted sum of noncentral chi-square variables with a single scaled noncentral chi-square which matches the first noncentral moment and two corresponding central moments. The approximation works well, but not as well as Imhof's overall for many cases.

Applying Imhof's approximation to a ratio involves a quadratic form with both positive and negative coefficients. Such forms can naturally lead to a symmetric distribution, which has skewness of zero, and disallows Imhof's approximation from existing. In practice, skewness values near zero cause the approximation to fail due to chi-square degrees of freedom too large for accurate computations with finite precision arithmetic. Muller and Barton (1989) used the obvious alternative of approximating the numerator and denominator separately by a scaled chi-square, which leads to a central or noncentral F approximation. The approach guarantees a numerically stable approximation always exists for a far wider range of conditions. The approach has a simple interpretation, and automatically reduces to partially exact or exact results when one or both components of the ratio have only one term.

A scaled noncentral chi-square provides the obvious approximation for a sum of non-central chi-square variables, with a natural choice matching three moments to choose the three parameters (scale, degrees of freedom, and noncentrality). However, no such approximation applies universally (Johnson and Kotz 1970, sec. 29.5.8). As we show in Section 4, a two-moment match does exist.

Two applications will illustrate the results. The first application centers on the independent groups t test with Gaussian but heterogeneous data, the Behrens-Fisher problem. The second application involves the distribution of Cronbach's alpha (Cronbach 1951), an estimate of reliability of a measurement process, such as a psychometric test, which contains multiple measures of performance (such as items in multiple choice tests). Kistner and Muller (2004) recently demonstrated that the exact distribution can be expressed in terms of the ratio of a chi-square to a central quadratic form in Gaussian variables.

2. Notation

Lower case bold always indicates a single column, a vector, such as 1_n = [1…1]′, or λ = [λ ₁…λ_K]′, with the prime denoting the transpose. Similarly, upper case bold indicates a matrix, such as Σ, a p × p covariance matrix.

The CDF of random variable X is written F_X (x), with corresponding density denoted f_X (x). The corresponding mean and variance are written E (X) and ν(X). Writing X ∼ (μ, σ²) indicates that X follows a scalar Gaussian distribution with mean μ and variance σ², while x ∼ _p (μ, Σ) indicates the p × 1 vector x follows a multivariate Gaussian distribution. In turn, X ∼ χ² (ν, ω) indicates X follows a noncentral chi-square distribution with ν degrees of freedom and noncentrality ω. Similarly, X ∼ F(ν₁, ν₂, ω) indicates X has a noncentral F distribution with ν₁ numerator degrees of freedom, ν₂ denominator degrees of freedom, and noncentrality ω. The corresponding CDF's are denoted by F_χ2 (ν, ω) and F_F (ν₁, ν₂, ω). A weighted sum of K independent and possibly noncentral chi-square variables, with positive weights {λ_k}, degrees of freedom {ν_k}, and noncentrality parameters {ω_k} will be indicated by Q (λ, ν, ω). More precisely,

$$Q(\mathit{\lambda },\mathit{\nu },\mathit{\omega })=\sum _{k=1}^K{\lambda }_k{X}_k.$$ (2.1)

The following lemma gives a standard representation that will be used in Section 3.

Lemma 1

If X₁ ∼ χ² (ν₁,ω) independently of X₂ ∼ χ² (ν₂), and F = (X₁/ν₁)/(X₂/ν₂),

$$\text{Pr}\{F\le r\}=\text{Pr}\{(X_1/v_1)\le r({\text{X}}_2/v_2)\}={\int }_0^{\infty }{F}_{{\chi }^2}(\mathit{\text{tr}}{v}_1/v_2;v_1,\omega )f_{{\chi }^2}(t;v_2)\mathit{\text{dt}}.$$ (2.2)

3. A noncentral χ² over a positively weighted quadratic form

3.1 An Expansion in terms of F Random Variables

We use Ruben's formulas for quadratic forms with all positive weights to specify an expansion for the ratio of a possibly noncentral chi-square to a positively weighted quadratic form. The approach provides a single series expansion, which may be chosen to be a mixture, for the density and CDF in terms of F densities and CDF's. A variety of bounds are also provided. Some special cases of the results are equivalent to expansions derived by Robbins and Pitman.

We assume ${\mathbf{\lambda }}_{\text{R}}^{\prime }=[{1^{\prime }}_{\nu 1}{\lambda }_1{1^{\prime }}_{\nu 2}{\lambda }_2\dots {1^{\prime }}_{\nu K}{\lambda }_K]=\left\{{\lambda }_{\text{qR}}\right\}$ with λ_qR > 0, and ${{\omega }^{\prime }}_{\text{R}}=\left\{{\omega }_{q\text{R}}\right\}=[{1^{\prime }}_{\nu 1}{\omega }_1/{\nu }_1{1^{\prime }}_{\nu 2}{\omega }_2/{\nu }_2\dots {1^{\prime }}_{\nu K}{\omega }_K/{\nu }_K]$ with ω_qR ≥ 0. Both are ν₊ × 1, with

$${\nu }_{+}=\sum _{k=1}^K{\nu }_k.$$ (3.1)

With X₀ ∼ χ²(ν₀,ω₀), X_k ∼ χ²(ν_k), all mutually independent, and U_q ∼ χ²(1, 0), we focus on

$$\text{Pr}\{\frac{Q_1}{Q_2}\le r\}=\text{Pr}\{\frac{{\lambda }_0{X}_0}{{\sum }_{k=1}^K{\lambda }_k{X}_k}\le r\}=\text{Pr}\{\frac{{\lambda }_0{X}_0}{{\sum }_{q=1}^{\nu +}{\lambda }_{\text{R}q}{U}_q}\le r\}=\text{Pr}\{{\lambda }_0{X}_0-r\sum _{q=1}^{\nu +}{\lambda }_{\text{R}q}{U}_q\le 0\}.$$ (3.2)

Theorem 1

If X₀ ∼ χ²(ν₀, ω₀) and X_k ∼ χ²(ν_k) are all mutually independent with λ₀ > 0, λ_k > 0, then

$$\text{Pr}\{\frac{{\lambda }_0{X}_0}{{\sum }_{k=1}^K{\lambda }_k{\text{X}}_k}\le r\}=\sum _{j=0}^{\infty }{c}_j{\text{F}}_F[\frac{r\beta ({\nu }_{+}+2j)}{{\lambda }_0{\nu }_0};{\nu }_0,{\nu }_{+}+2j,{\omega }_0].$$ (3.3)

The coefficients may be computed as

$$d_m=\sum _{q=1}^{{\nu }_{+}}{(1-\beta /{\lambda }_{\text{R}q})}^m,$$ (3.4)

and

$$c_j=\{\begin{array}{c}\prod _{q=1}^{\nu +}{(\beta /{\lambda }_{\text{R}q})}^{1/2}j=0\\ \frac{1}{2j}\sum _{m=0}^{j-1}{d}_{j-m}{c}_{m}j\ge 1.\end{array}.$$ (3.5)

Choosing β ≤ λ₁ gives 0 ≤ c_j < 1 and ${\sum }_{j=0}^{\infty }{c}_j=1$, which gives a mixture.

Proof: Ruben's (1962) expansion for the CDF of a central quadratic form gives

$$\text{Pr}\{Q_1/Q_2\le r\}=\text{Pr}\{Q_1\le r{Q}_2\}={\int }_0^{\infty }{F}_{{\chi }^2}(\mathit{\text{rq}}/{\lambda }_0;{\nu }_0,{\omega }_0){\beta }^{-1}\sum _{j=0}^{\infty }{c}_j{f}_{{\chi }^2}(q{\beta }^{-1};{\nu }_{+}+2j)\mathit{\text{dq}}=\sum _{j=0}^{\infty }{c}_j{\int }_0^{\infty }{F}_{{\chi }^2}(\mathit{\text{rq}}/{\lambda }_0;{\nu }_0,{\omega }_0)f_{{\chi }^2}(q{\beta }^{-1};{\nu }_{+}+2j){\beta }^{-1}\mathit{\text{dq}}=\sum _{j=0}^{\infty }{c}_j{\int }_0^{\infty }{F}_{{\chi }^2}(\mathit{\text{tr}}\beta /{\lambda }_0;{\nu }_0,{\omega }_0)f_{{\chi }^2}(t;{\nu }_{+}+2j)\mathit{\text{dt}}.$$ (3.6)

The interchange of integration and summation is allowed by Fubini's theorem. An individual term in the series may be reduced to the desired form by using Lemma 1 to write

$${\int }_0^{\infty }{F}_{{\chi }^2}(\mathit{\text{tr}}\beta /{\lambda }_0;{\nu }_0,{\omega }_0)f_{{\chi }^2}(t;{\nu }_{+}+2j)\mathit{\text{dt}}={\int }_0^{\infty }{F}_{{\chi }^2}\left\{t\right[\frac{r\beta ({\nu }_{+}+2j)}{{\lambda }_0{\nu }_0}]\frac{{\nu }_0}{({\nu }_{+}+2j)};{\nu }_0,{\omega }_0\}{f}_{{\chi }^2}(t;{\nu }_{+}+2j)\mathit{\text{dt}}=F_F[r\beta ({\nu }_{+}+2j)/({\lambda }_0{\nu }_0);{\nu }_0,{\nu }_{+}+2j,{\omega }_0].$$ (3.7)

Specifying the coefficients completes proof. Ruben (1962) and Johnson and Kotz (1970, equations 88–89) provided explicit and convenient forms.

Johnson and Kotz summarized Ruben's extensive discussion of choices for β in terms of a bound on truncation error for the CDF (equation 43 in Johnson and Kotz 1970, p. 158). For β ≤ λ₁, which guarantees the expansion is a mixture, Ruben's bound on truncation error is least when β = λ₁. For a (possibly nonmixture) expansion with β ≥ λ_K, the bound on truncation error is least when β = λ_K. However, Ruben recommended β = 2λ₁λ_K/ (λ₁ + λ_K) to improve convergence because it yields a much smaller bound on truncation error than does β = λ_K or β = λ₁. Unfortunately, in the setting discussed here, the (generalized) bounds rarely provide any useful guidance. Hence we do not provide any details.

Corollary 1

For $y_{\mathit{\text{ik}}}\sim \mathcal{\text{N}}({\mu }_k,{\sigma }_k^2)$, i ∈ {1, …,N_k} k ∈ {1, 2}, all independent, ${\overline{y}}_k={\sum }_{i=1}^{N_k}{y}_{\mathit{\text{ik}}}/N_k={\widehat{\mu }}_k$, $S_k={\sum }_{i=1}^{N_k}{(y_{\mathit{\text{ik}}}-{\overline{y}}_k)}^2$ and s² = (S₁ + S₂) (1/N₁ + 1/N₂) / (N₁ + N₂ − 2),the Behrens-Fisher statistic is t = (ȳ₁ − ȳ₂)/s. With ${\omega }_t={({\mu }_1-{\mu }_2)}^2/({\sigma }_1^2/N_1+{\sigma }_2^2/N_2)$, β as in the theorem, c_j in terms of $\left\{{\lambda }_{\text{R}q}\right\}\in \{{\sigma }_k^2(N_1+N_2)/[N_1{N}_2(N_1+N_2-2)]\}$, the CDF of t² is

$$\text{Pr}\{t^2\le r\}=\sum _{j=0}^{\infty }{c}_j{F}_F[\frac{r\beta (N_1+N_2-2+2j)}{{\sigma }_1^2/N_1+{\sigma }_2^2/N_2};1,N_1+N_2-2+2j,{\omega }_t].$$ (3.8)

Proof: The null hypothesis is H₀ : μ₁ = μ₂, or H₀ : μ₁ − μ₂ = δ = 0. Also, $X_k\sim {\chi }_1^2(N_k-1)$. The test statistic is

$$t^2=\frac{{({\widehat{\mu }}_1-{\widehat{\mu }}_2)}^2}{(S_1+S_2)(1/N_1+1/N_2)/(N_1+N_2-2)}=\frac{{\widehat{\delta }}^2}{\frac{(N_1+N_2){\sigma }_1^2}{N_1{N}_2(N_1+N_2-2)}{X}_1+\frac{(N_1+N_2){\sigma }_2^2}{N_1{N}_2(N_1+N_2-2)}{X}_2}.$$ (3.9)

It is straightforward to prove that {S₁, S₂, μ̂₁, μ ^₂} are mutually independent. Also,

$$X_0={\widehat{\delta }}^2/({\sigma }_1^2/N_1+{\sigma }_2^2/N_2)\sim {\chi }^2(1,{\omega }_t),$$ (3.10)

The denominator is a weighted sum of independent central chi-square variables, with weights ${\lambda }_{\mathit{\text{kt}}}={\sigma }_k^2(N_1+N_2)/[N_1{N}_2(N_1+N_2-2)]$. If ${\lambda }_{0t}=({\sigma }_1^2/N_1+{\sigma }_2^2/N_2)$, then

$$t^2=\frac{{\lambda }_{0t}{X}_0/1}{{\lambda }_{1t}{X}_1+{\lambda }_{2t}{X}_2}.$$ (3.11)

In the notation of Theorem 1, ν₀ = 1, ν₁ = N₁ − 1, ν₂ = N₂ − 1, and ω₀ = ω_t. In turn, ν₊ = N₁ + N₂ − 2, and the result follows immediately.

Corollary 2

For N independent samples of y ∼ _p(μ, Σ), the matrix ${\mathbf{\text{A}}}_2={\mathbf{\text{F}}}_{\prime }^{\sum }{\mathbf{\text{A}}}_1{\mathit{\text{F}}}_{\sum },$ with ${\mathbf{\text{A}}}_1=[p/(p-1)-r_{\alpha }]{\mathbf{1}}_p{\mathbf{1}}_p^{\prime }-{\mathbf{\text{I}}}_p[p/(p-1)]$, has one positive and p − 1 negative eigenvalues, {λ_A2,k}. With β as in the theorem, c_j in terms of {λ_Rq} ∈ {λ_A2,2, …, λ_A2,p}, the CDF of the sample estimate of Cronbach's α is

$$\text{Pr}\{{\widehat{\rho }}_{\alpha }\le r_{\alpha }\}=\sum _{j=0}^{\infty }{c}_j{F}_F[\frac{\beta [(p-1)(N-1)+2j]}{{\lambda }_{A_2,1}(N-1)};N-1,(p-1)(N-1)+2j].$$ (3.12)

With intraclass correlation, ρ_I, such that ${\rho }_{\alpha }=p/[1/{\rho }_{\text{I}}+(p-1)]$,

$$\text{Pr}\{{\widehat{\rho }}_1\le r_{\text{I}}\}=\text{Pr}\{{\widehat{\rho }}_{\alpha }\le p/[1/r_{\text{I}}+(p-1)\left]\right\}.$$ (3.13)

Proof: A sample covariance matrix is C = {c_ij}, while ρ̂_I indicates an estimate, and r_I a particular value of intraclass correlation. Here ρ ^_I is

$${\widehat{\rho }}_I=\{\sum _{i=1}^{p-1}\sum _{j=i+1}^p{c}_{\mathit{\text{ij}}}/[p(p-1)]\}/(\sum _{i=1}^p{c}_{\mathit{\text{ii}}}/p),$$ (3.14)

and ρ̂_α = p/ [(1/ρ ^_I) + (p − 1)] is the estimate of Cronbach's α. With (N − 1) > p and $\sum ={\mathbf{\text{F}}}_{\sum }{\mathbf{\text{F}}}_{\sum }^{\prime },$ the distribution function of ρ̂_α exactly equals the distribution function of a weighted sum of independent central chi-square variables (Kistner and Muller 2004). The weights are {λA₂,k}, the single positive andp − 1 negative eigenvalues of ${\mathbf{\text{A}}}_2={\mathbf{\text{F}}}_{\sum }^{\prime }{\mathbf{\text{A}}}_1{\mathbf{\text{F}}}_{\sum }$, with ${\mathbf{\text{A}}}_1=[p/(p-1)-r_{\alpha }]{\mathbf{1}}_p{\mathbf{1}}_p^{\prime }-{\mathbf{\text{I}}}_p[p/(p-1)]$. In turn, with independent X_k ∼ χ² (N − 1),

$$\text{Pr}\{{\widehat{\rho }}_{\alpha }\le r_{\alpha }\}=\text{Pr}\{({\lambda }_{A_2,1}{X}_0)/(\sum _{k=2}^p|{\lambda }_{A_2,k}|X_k)\le 1\}.$$ (3.15)

Applying the theorem to the last equation directly gives Equation (3.12), the desired result for ρ_α. In turn, ρ_I is a one-to-one function of ρ_α. Hence Pr {ρ̂_I\≤ r_I} = Pr {ρ̂_α ≤ p/[1/r_I+ (p − 1)]}.

3.2 Error Bounds for the Expansion

Corollary 3

For a mixture representation, a uniform upper bound on truncation error is

$$\sum _{j=J+1}^{\infty }{c}_j{F}_F[r\beta ({\nu }_{+}+2j){({\lambda }_0{\nu }_0)}^{-1};{\nu }_0,{\nu }_{+}+2j,{\omega }_0]\le 1-\sum _{j=0}^J{c}_j.$$ (3.16)

Proof: Robbins and Pitman (1949) provided the basis of the proof in Equation (9) and surrounding text in their article. A mixture has all positive coefficients and terms, as is F_F [·]. Hence

$$|p-\sum _{j=0}^J{c}_j{F}_F[\frac{r\beta ({\nu }_{+}+2j)}{{\lambda }_0{\nu }_0};{\nu }_0,{\nu }_{+}+2j,{\omega }_0\left]\right|=\left|\sum _{j=J+1}^{\infty }{c}_j{F}_F\right[\frac{r\beta ({\nu }_{+}+2j)}{{\lambda }_0{\nu }_0};{\nu }_0,{\nu }_{+}+2j,{\omega }_0\left]\right|=\sum _{j=J+1}^{\infty }{c}_j{F}_F[\frac{r\beta ({\nu }_{+}+2j)}{{\lambda }_0{\nu }_0};{\nu }_0,{\nu }_{+}+2j,{\omega }_0]\le \sum _{j=J+1}^{\infty }{c}_j\cdot 1=1-\sum _{j=0}^J{c}_j.$$ (3.17)

Although Robbins and Pitman described series expansions for a ratio of quadratic forms, their approach to computing coefficients is very complicated. Ruben presented a convenient form for the computations. However, he considered a single quadratic form, not a ratio. We applied Ruben's method to compute the coefficients for the ratio of interest, and considered a bound on the error term of the mixture representation presented by Robbins and Pitman.

Corollary 4

For a mixture representation, a uniform lower bound on truncation error is

$$F_F[J]\cdot (1-\sum _{j=0}^J{c}_j)\le \sum _{j=J+1}^{\infty }{c}_j{F}_F[\frac{r\beta ({\nu }_{+}+2j)}{{\lambda }_0{\nu }_0};{\nu }_0,{\nu }_{+}+2j,{\omega }_0],$$ (3.18)

with F_F [j] = F_F [rβ (ν₊ + 2j + 2) / (λ₀ ν₀); ν₀,ν₊ + 2j + 2, ω₀].

Proof: If ∈ > 0, then F_F (f; υ₁, υ₂, ω₀) ≤ F_F (f; υ₁, υ₂ + ∈, ω₀) and F_F (f; υ₁, υ₂, ω₀) ≤ F_F (f + ∈; υ₁, υ₂, ω₀). Hence

$$\sum _{j=J+1}^{\infty }{c}_j{F}_F[\frac{r\beta ({\nu }_{+}+2j)}{{\lambda }_0{\nu }_0};{\nu }_0,{\nu }_{+}+2j,{\omega }_0]\ge \sum _{j=J+1}^{\infty }{c}_j{F}_F[\frac{r\beta ({\nu }_{+}+2J+2)}{{\lambda }_0{\nu }_0};{\nu }_0,{\nu }_{+}+2J+2,{\omega }_0].$$ (3.19)

In turn,

$$\sum _{j=J+1}^{\infty }{c}_j{F}_F[r\beta ({\nu }_{+}+2J+2)/({\lambda }_0{\nu }_0);{\nu }_0,{\nu }_{+}+2J+2,{\omega }_0]=\sum _{j=J+1}^{\infty }{c}_j{F}_F[J]=F_F[J]\sum _{j=J+1}^{\infty }{c}_j.$$ (3.20)

As J increases, the lower and upper bounds grow closer together. Except for very small r, even moderate values of J give F_F [J] near 1 and a narrow bounding interval. Any stopping rule should insure that the lower bound is less than the tolerable error.

Corollary 5

The $\mathit{\text{CDF}}{F}_R(r)=\text{Pr}\{{\lambda }_0{X}_0/{\sum }_{k=1}^K{\lambda }_k{X}_k\le r\}$ is an increasing function of λ_k for k ∈ {1,…, K}, for each {r, ν, ω₀}. Assuming λ_k, for k ∈ {1,…, K}, are sorted from smallest to largest, stochastic bounds on the probability are

$$F_F(\frac{{\lambda }_1{\nu }_{+}}{{\lambda }_0{\nu }_0}r;{\nu }_0,{\nu }_{+},{\omega }_0)\le F_R(r)\le F_F(\frac{{\lambda }_K{\nu }_{+}}{{\lambda }_0{\nu }_0}r;{\nu }_0,{\nu }_{+},{\omega }_0).$$ (3.21)

Proof: Here

$$\begin{array}{c}\text{Pr}\{\frac{{\lambda }_0{X}_0}{{\sum }_{k=1}^K({\lambda }_k/{\lambda }_1)X_k}\le {\lambda }_{1}r\}=F_R(r)=\text{Pr}\{\frac{{\lambda }_0{X}_0}{{\sum }_{k=1}^K({\lambda }_k/{\lambda }_1)X_k}\le {\lambda }_{1}r\}\\ \text{Pr}\{\frac{{\lambda }_0{X}_0}{({\lambda }_1/{\lambda }_1){\sum }_{k=1}^K{X}_k}\le {\lambda }_{1}r\}\le F_R(r)\le \text{Pr}\{\frac{{\lambda }_0{X}_0}{({\lambda }_K/{\lambda }_1){\sum }_{k=1}^K{X}_k}\le {\lambda }_{1}r\}\\ \text{Pr}\{\frac{X_0/{\nu }_0}{{\sum }_{k=1}^K{X}_k/{\nu }_{+}}\le \frac{{\lambda }_1{\nu }_{+}}{{\lambda }_0{\nu }_0}r\}\le F_R(r)\le \text{Pr}\{\frac{X_0/{\nu }_0}{{\sum }_{k=1}^K{X}_k/{\nu }_{+}}\le \frac{{\lambda }_K{\nu }_{+}}{{\lambda }_0{\nu }_0}r\}\\ F_F\{r\frac{{\lambda }_1{\nu }_{+}}{{\lambda }_0{\nu }_0};{\nu }_0,{\nu }_{+},{\omega }_0\}\le F_R(r)\le F_F\{r\frac{{\lambda }_K{\nu }_{+}}{{\lambda }_0{\nu }_0};{\nu }_0,{\nu }_{+},{\omega }_0\}.\end{array}$$ (3.22)

Corollary 6

If ω₀ = 0, a global bound can be specified using equation (3.18) in Ramirez and Jensen (1991). For J + 1 > (K − 2) /|log ε|, K_* the smallest even integer with K* ≥K, $A={\prod }_{i=1}^K{(\beta /{\lambda }_i)}^{1/2}$, ε = max_1≤i≤K (1 − β/λ_i), and V₀ ∼ χ² (K_*), the error in approximating Pr {R ≤ r} by truncating at J terms is bounded by

$$e_J\le \frac{\beta }{r}\cdot \frac{\Gamma (K_{\ast }/2)}{\Gamma (K/2)}\cdot \frac{A\varepsilon }{{(\varepsilon |log\varepsilon |)}^{K_{\ast }/2}}\text{Pr}\{V_0>(2J+K_{\ast }-2)|log\varepsilon |\}.$$ (3.23)

Proof: For the Cronbach α and intraclass correlation application, we are interested in a random variable of the form

$$R^{-1}=Y=\sum _{k=1}^K{\lambda }_k{X}_k/({\lambda }_0{X}_0),$$ (3.24)

with corresponding realizations r and y. We seek to compute Pr {R ≤ r}, and bound the truncation error. Integrating equation (3.5) from Ramirez and Jensen (1991) gives

$$F_Y(y)={\int }_0^y\sum _{j=0}^{\infty }\frac{c_j}{\beta }\cdot f_F(t/\beta ;K+2j,{\nu }_0-K+1)\mathit{\text{dt}}=\sum _{j=0}^{\infty }{c}_j{\int }_0^{y/\beta }{f}_F(u;K+2j,{\nu }_0-K+1)\mathit{\text{du}}=\sum _{j=0}^{\infty }{c}_j{F}_F(y/\beta ;K+2j,{\nu }_0-K+1)=\sum _{j=0}^{\infty }{c}_j[1-F_F(\beta /y;{\nu }_0-K+1,K+2j)]=1-\sum _{j=0}^{\infty }{c}_j{F}_F(\beta /y;{\nu }_0-K+1,K+2j).$$ (3.25)

In turn,

$$\text{Pr}\{R\le r\}=\text{Pr}\{1/r\le 1/R\}=1-\text{Pr}\{1/R\le 1/r\}$$ (3.26)

and

$$F_R(r)=\text{Pr}\{1/R\le r\}=1-\text{Pr}\{R\le 1/r\}=1-F_R(1/r)=1-\sum _{j=0}^{\infty }{c}_j{F}_F(\beta r^{-1};K+2j,{\nu }_0-K+1).$$ (3.27)

Differentiating with respect to r gives

$$f_R(r)=\beta r^{-2}\cdot \sum _{j=0}^{\infty }{c}_j{f}_F(\beta r^{-1};K+2j,{\nu }_0-K+1).$$ (3.28)

Therefore

$$e_J\le \frac{\beta }{r}\cdot \frac{\Gamma (K_{\ast }/2)}{\Gamma (K/2)}\cdot \frac{A\varepsilon }{{(\varepsilon |log\varepsilon |)}^{K_{\ast }/2}}\text{Pr}\{V_0>(2J+K_{\ast }-2)|log\varepsilon |\}.$$ (3.29)

Corollary 7

If ω₀ = 0, a local bound can be specified using equation (3.21) in Ramirez and Jensen (1991). For J + 1 > (ν₀ − 1) /|log εt|, (ν₀* + 1) equals the smallest even integer with (ν₀* +1) ≥ (ν₀ +1), $A={\prod }_{i=1}^K{(\beta /{\lambda }_i)}^{1/2}$, ε = max₁≤i≤ (1 − β/λ_i), c = [A · (rβ)^−(K−2)/2]/[β · Γ (K/2) · Γ[(ν₀ − K + 1)/2] · (1 + 1/rβ)^{(ν0 + 1)/2}], t = (1 + 1/rβ) / (rβ) and V₀ ∼ χ² (ν₀* + 1). The error in approximating Pr {R ≤ r} by truncating at J terms is bounded by

$$e_J(r)\le \frac{\beta }{r}\cdot \frac{c\Gamma [({\nu }_{0\ast }+1)/2]\varepsilon t}{{(\varepsilon t|log\varepsilon t|)}^{({\nu }_{0\ast }+1)/2}}\text{Pr}\{\text{V}>[2J+({\nu }_{0\ast }+1)-2]|log\varepsilon t|\}.$$ (3.30)

Proof: The proof parallels the proof of Corollary 6.

4. Approximations

4.1 A Single Noncentral Quadratic Form with Positive Weights

Existing approximations of positively weighted noncentral quadratic forms have a variety of shortcomings. They may lack accuracy in some cases (Muller and Barton 1989), or may not reduce to the exact noncentral form with equal weights (Imhof 1961). Also, Imhof's may be improper in the sense of assigning nonzero probability to negative values, which is not ideal when used in analytic manipulations. The approximation in Lemma 2 simultaneously achieves excellent accuracy, automatic reduction to special cases, and proper support in a single form. As noted earlier, no single approximation can always match three moments of a noncentral form.

Lemma 2

If $Q={\sum }_{k=1}^K{\lambda }_k{X}_k$, with λ_k > 0 and all mutually independent {X_k} for X_k ∼ χ² (ν_k, ω_k), then strictly positive {λ_*, ν_*} and ω_* ≥ 0 always exist such that Q_* = λ_*X_* with X_* ∼ χ²(ν_*, ω_*) insures E (Q) = E (Q_*) and E [Q − E (Q)]² = ν (Q) = ν(Q_*). Hence

$$\text{Pr}\{Q\le q\}\approx \text{Pr}\{Q_{\ast }\le q\}.$$ (4.1)

The required parameters are

$${\lambda }_{\ast }=(S_3+2S_4)/(S_1+2S_2)$$ (4.2)

$${\nu }_{\ast }=S_1(S_1+2S_2)/(S_3+2S_4)$$ (4.3)

$${\omega }_{\ast }=S_2(S_1+2S_2)/(S_3+2S_4),$$ (4.4)

with

$$\sum _{k=1}^K{\lambda }_k{\nu }_k=S_1={\lambda }_{\ast }{\nu }_{\ast }$$ (4.5)

$$\sum _{k=1}^K{\lambda }_k{\nu }_k+\sum _{k=1}^K{\lambda }_k{\omega }_k=S_1+S_2={\lambda }_{\ast }({\nu }_{\ast }+{\omega }_{\ast })$$ (4.6)

$$2\sum _{k=1}^K{\lambda }_k^2{\nu }_k+4\sum _{k=1}^K{\lambda }_k^2{\omega }_k=2S_3+4S_4=2{\lambda }_{\ast }^2({\nu }_{\ast }+2{\omega }_{\ast }).$$ (4.7)

Proof: If Q_* = λ_*y_* and y_* ∼ χ² (ν_*,ω_*), then E [Q|max_k(ω_k) = 0] = S1, E [Q| max_k(ω_k) > 0] =S₁+ S₂, and ν [Q| max_k(ω_k) > 0] = 2S₃ + 4S₄. Solving for the parameters of Q_* in terms of {S_m} completes the proof.

4.2 The Ratio of a Noncentral to a Central Quadratic form with All Positive Weights

Theorem 2

If $Q_1={\sum }_{k=1}^K{\lambda }_k{X}_k,$, $Q_2={\sum }_{k=K_1+1}^{K_1+K_2}{\lambda }_k{X}_k$, with all mutually independent {X_k}, λ_k > 0, X_k ∼ χ² (ν_k, ω_k), ω_k = 0 for k > K₁, then strictly positive {λ_*1, λ_*2, ν_*1, ν_*2} and ω_*1 ≥ 0 always exist such that Q_*1/λ_*1 = X_*1 ∼ χ² (ν_*1, ω_*1), Q_*2/λ_*2 = X_*2 ∼ χ²(ν_*2,0), E (Q1) = E (Q_*1), ν (Q1) = ν (Q_*1), E (Q2) = E (Q_*2) and ν (Q2) = V (Q_*2). In turn, with r_* = (rλ_*2/λ_*1) (ν_*2/ν_*1),

$$\text{Pr}\{\frac{Q_1}{Q_2}\le r\}\approx \text{Pr}\{\frac{{\lambda }_{\ast 1}{X}_{\ast 1}}{{\lambda }_{\ast 2}{X}_{\ast 2}}\le r\}=\text{Pr}\{\frac{X_{\ast 1}/{\nu }_{\ast 1}}{X_{\ast 2}/{\nu }_{\ast 2}}\le r\cdot \frac{{\lambda }_{\ast 2}/{\nu }_{\ast 1}}{{\lambda }_{\ast 1}/{\nu }_{\ast 2}}\}=F_F(r_{\ast },{\nu }_{\ast 1},{\nu }_{\ast 2},{\omega }_{\ast 1}).$$ (4.8)

Proof: Applying Lemma 2 separately to Q1 andQ2 gives the required {λ_*1, λ_*2, ν_*1, ν_*2,ω_*1}. The last equation completes the proof.

The theorem guarantees that any ratio of a positively weighted noncentral quadratic form to a positively weighted central quadratic form may be approximated by a noncentral F. If all distributions are central the approximation reduces to a Satterthwaite type. If either the numerator or denominator have all weights equal, then the approximation automatically reduces to the correct result. The approximation applies to the Behrens-Fisher statistic, as well as to Cronbach's α and intraclass correlation for general covariance. For compound symmetric covariance, the approximation automatically reduces to the appropriate exact F.

5. Numerical Results

5.1 Computing Strategy

Two problems motivated the research. We illustrate the value of the new expansions and approximations for both the Behrens-Fisher problem, as well as for intraclass correlation and Cronbach's α with general covariance.

We used Davies's algorithm and the series expansion to compute the distribution function. The series was terminated according to the error bounds, with β = λ₁ (the smallest weight). A maximum error of 0.0001 was allowed for both algorithms. We also computed Imhof and F approximations, as well as stochastic bounds.

5.2 Behrens-Fisher Statistic

Moser, Stevens, and Watts (1989) expressed the CDF of the t statistic with ${\sigma }_2^2/{\sigma }_1^2\ne 1$ as a numerical integration. They tabled the CDF for a range of values, including ω ∈ {0, 1, 5,10}, ${\sigma }_2^2/{\sigma }_1^2\in \{2,5,10\}$, and N_k ∈ {6, 51}. Table 1 contains the exact test size of the uncorrected t, as computed by the new series expansion and Davies' algorithm. All values were also checked against results of Moser et al. (1989) Table 2 contains results for non-null cases with ${\sigma }_2^2/{\sigma }_1^2=10.$ In all cases, the new series expansion provides the desired accuracy.

Table 1.

Exact and Approximate Uncorrected t (Behrens-Fisher) Test Size (ω_t = 0). Sample sizes are N₁ and N₂ for the Two Groups.

N1	N2	${\sigma }_2^2/{\sigma }_1^2$	Davies	Series	Imhof	∼̇ F	Lower	Upper
6	6	5	0.0593	0.0595	0.1018	0.0616	0.0165	0.2273
6	6	10	0.0653	0.0654	0.1133	0.0675	0.0132	0.3645
6	51	5	0.0007	0.0008	<0.0001	0.0007	0.0004	0.0984
6	51	10	0.0001	0.0002	<0.0001	0.0001	<0.0001	0.1566
51	6	5	0.2819	0.2820	0.2849	0.2822	0.0409	0.3531
51	6	10	0.3801	0.3802	0.3774	0.3809	0.0398	0.5082
51	51	5	0.0512	0.0513	0.0564	0.0512	0.0119	0.2548
51	51	10	0.0518	0.0519	0.0578	0.0518	0.0087	0.3996

Table 2.

Exact and Approximate Uncorrected t (Behrens-Fisher) Power With ${\sigma }_2^2/{\sigma }_1^2=10$. Sample Sizes are N₁ And N₂ for the Two Groups.

N1	N2	ωt	Davies	Series	Imhof	∼̇ F	Lower	Upper
6	6	5	0.5367	0.5368	0.5368	0.5365	0.2829	0.9009
6	6	10	0.8082	0.8083	0.7981	0.8077	0.5792	0.9857
6	51	5	0.0270	0.0271	0.0263	0.0270	0.0178	0.7880
6	51	10	0.1416	0.1417	0.1443	0.1416	0.1061	0.9570
51	6	5	0.9101	0.9102	0.9098	0.9102	0.5544	0.9437
51	6	10	0.9879	0.9879	0.9830	0.9879	0.8519	0.9938
51	51	5	0.6012	0.6012	0.6081	0.6012	0.3355	0.9187
51	51	10	0.8785	0.8785	0.8806	0.8785	0.6862	0.9897

Tables 1 and 2 also include the F and Imhof approximations, as well as stochastic lower and upper bounds. For the examples, the new F approximation performs nearly identically to the exact series expansion in most cases.

5.3 Cronbach's α and Intraclass Correlation

Kistner and Muller (2004) computed the exact CDF of Cronbach's α with N = 10, p = 4, and known covariance. They tabled some CDF values, including cases with constant correlation (CS, compound symmetry) for ρ = 0.5, and AR(1), with ρ ∈ {0.2, 0.5, 0.8}. The variance pattern, ${\left[{\sigma }_1^2{\sigma }_2^2{\sigma }_3^2{\sigma }_4^2\right]}^{\prime }$, was either [1 1 1 1]′, [1 2 3 4]′, or [4 3 2 1]′. An AR(1) correlation structure can be expressed as

$$\left[\begin{array}{cccc}1& \rho & {\rho }^2& {\rho }^3\\ \rho & 1& \rho & \rho \\ {\rho }^2& \rho & 1& \rho \\ {\rho }^3& {\rho }^2& \rho & 1\end{array}\right].$$ (5.1)

Table 3 displays values of the exact CDF computed with the new series and Davies' algorithm, as well as approximations and stochastic bounds. Figure 1 illustrates the corresponding exact and approximate densities, with the exact and F approximation essentially indistinguishable. The density based on the Imhof approximation has a gap caused by missing values resulting from skewness values near zero for the quadratic form. The density in the figure obviously has positive skewness. However, the weights needed are eigenvalues of ${\mathbf{\text{A}}}_2={\mathbf{\text{F}}}_{\sum }^{\prime }{\mathbf{\text{A}}}_1{\mathit{\text{F}}}_{\sum }$, and vary as a function of r_α. Hence the moments of the approximating quadratic form also vary as a function of r_α. Most awkwardly, the skewness changes from positive to zero to negative for r_α ∈ (0.78, 0.80) in the example. Corresponding approximate degrees of freedom of ν_* ≈ 10⁶ or greater pose a difficulty in computing the approximating chi-square probability.

Table 3.

Exact and Approximate CDF's of Estimated Cronbach's α for Pr{ρ̂_α ≤ 0.70 } with Σ Known, N= 10.

Correlation	ρ	Variances	Davies	Series	Imhof	∼̇ F	Lower	Upper
CS	0.5	${\sigma }_k^2\in \{1,1,1,1\}$	0.2689	0.2689	0.2755	0.2689	0.2689	0.2689
AR(1)	0.5	${\sigma }_k^2\in \{1,1,1,1\}$	0.5628	0.5627	0.5605	0.5631	0.2226	0.8427
AR(1)	0.2	${\sigma }_k^2\in \{1,1,1,1\}$	0.9442	0.9442	0.9389	0.9440	0.8778	0.9777
AR(1)	0.8	${\sigma }_k^2\in \{1,1,1,1\}$	0.0430	0.0429	0.0428	0.0429	0.0020	0.2171
CS	0.5	${\sigma }_k^2\in \{1,2,3,4\}$	0.4697	0.4696	0.4722	0.4705	0.0097	0.8746

Figure 1.

Density for ρ̂_α. Solid Line for Exact, Dashed Line for F Approximation, and Dotted Line for Imhof's Approximation for p = 4, CS, ρ = 0.5, and ${\sigma }_k^2\in \{1,2,3,4\}$.

Table 4 shows the results of the exact CDF for a range of quantiles with N = 10, p = 3, AR(1),ρ = .5, $\left[{\sigma }_1^2{\sigma }_2^2{\sigma }_3^2\right]=\left[\begin{array}{ccc}1& 2& 3\end{array}\right]$, as well as the F and Imhof approximations. Figure 2 displays corresponding exact and approximate densities, with the exact and F approximation essentially indistinguishable. The density based on the Imhof approximation is subject to the same skewness difficulty described above for r_α ∈ (0.66, 0.67).

Figure 2.

Density for ρ̂_α. Solid Line for Exact, Dashed Line for F Approximation, and Dotted Line for Imhof's Approximation for p = 3, AR(1) with ρ = 0.5, and ${\sigma }_k^2\in \{1,2,3\}$.

For all cases in Table 3 and 4, the new series expansion achieves the desired accuracy. In addition, the corresponding density calculations remain straightforward. The F approximation performs better than the Imhof approximation in all cases. The approximate density remains well-defined and simple to compute, with degrees of freedom parameters roughly 9 and 18.4 for r_α ∈ (0.78,0.80) in Figure 1, and 9 and 13.2 for r_α ∈ (0.66,0.67) in Figure 2.

Table 4.

Exact and Approximate CDF's of Estimated Cronbach's α, Pr{ρ̂_α ≤ r_α }, for a Range of Quantiles with Σ Known, AR(1), ρ = 0.5, ${\sigma }_k^2\in \{1,2,3\}$, N= 10.

rα	Davies	Series	Imhof	∼̇ F	Lower	Upper
0.10	0.0614	0.0613	0.0629	0.0614	0.0024	0.1932
0.20	0.0899	0.0898	0.0940	0.0900	0.0041	0.2605
0.30	0.1349	0.1349	0.1428	0.1353	0.0075	0.3529
0.40	0.2072	0.2072	0.2186	0.2079	0.0146	0.4766
0.50	0.3231	0.3231	0.3339	0.3242	0.0315	0.6317
0.60	0.5010	0.5010	0.5012	0.5020	0.0758	0.8008
0.70	0.7367	0.7368	0.7322	0.7361	0.2035	0.9372
0.80	0.9418	0.9418	0.9111	0.9391	0.5497	0.9942
0.90	0.9992	0.9992	0.9691	0.9989	0.9708	1

5.4 Additional Evaluations

The new scaled, noncentral chi-square and the Imhof approximations, as well as the Muller and Barton approximation, were compared to exact values for an additional wide range of quadratic forms. Cases included the sums of 2, 4, and 8 positively weighted chi-square variables for exact quantiles with probabilities in the range 0.00001–0.99999, and increments of 0.00001. Given the agreement with the displayed results, we provide only rough summaries, for the sake of brevity. For positively weighted quadratic forms, the Imhof approximation performed particularly better in the lower tail than the new approximation for all cases and only slightly better in the upper tail than the new approximation in cases with only two weighted chi-square variables. Elsewhere, the new approximation performed almost identically to Imhof's. Both greatly surpassed the Muller and Barton approximation in performance throughout the distribution for all cases.

5.5 Timing Evaluations

In order to provide stable timing estimates, we repeated calculations for each table 20,000 times. Averaging results for Tables 1 and 2, the Imhof approximation was fastest, while the F approximation took approximately 1.5 times longer. In turn, the Davies's algorithm took approximately 158.6 times longer, while the series took approximately 418.1 times longer. Averaging results for Tables 3 and 4, the Imhof approximation was fastest, while the F approximation took approximately 1.3 times longer. In turn, the Davies's algorithm took approximately 52.9 times longer, while the series took approximately 70.2 times longer.

6. Conclusions

The simple form of the series lends itself to analytic manipulation in more complex forms. Such a problem occurs in integral expressions for test size and power of internal pilot designs (Coffey and Muller 1999). The expansion has the additional advantage of being expressible as a mixture.

The series is simple to program, and appears to be numerically stable and accurate. In contrast, Davies's (more general) algorithm requires substantially more code to implement. However, overall, Davies's algorithm is much faster, especially for noncentral cases. Naturally, when approximations provide sufficient accuracy, they are much faster than the series approach or Davies's algorithm.

As Davies noted in his original article, his algorithm has difficulties when dominated by one term. Although we did not present such cases, the series appears to work better in such situations (with either only one positive term or one negative term). The role of alternative β values may also be worth considering in the study of accuracy and speed.

Approximations for the noncentral case were examined in some detail. The new approximation seems by far the best for ratios with a noncentral component. The new approximation uses four parameters (two each for the numerator and denominator), with proper support for numerator and denominator separately, and hence for the ratio. Imhof's method uses three parameters, with an improper support. Hence it is not surprising that the F approximation performs better. In addition, Imhof's approximation can be computationally ill-behaved.

In the “univariate” approach to repeated measures ANOVA, the test statistic can be expressed as the ratio of a noncentral quadratic form to a central form, all with positive weights. Such a ratio includes the ratios studied here as special cases. We expect the approaches that worked well in the present setting can be extended to our ongoing research on power approximations for the “univariate” approach to repeated measures.

We emphasize the conclusion that no single method dominates in computing probabilities involving quadratic forms. Free SAS/IML® software implementing the series method, Davies's algorithm and the F approximation is available at http://www.bios.unc.edu/∼muller.