Global Validation of Linear Model Assumptions

Abstract

An easy-to-implement global procedure for testing the four assumptions of the linear model is proposed. The test can be viewed as a Neyman smooth test and it only relies on the standardized residual vector. If the global procedure indicates a violation of at least one of the assumptions, the components of the global test statistic can be utilized to gain insights into which assumptions have been violated. The procedure can also be used in conjunction with associated deletion statistics to detect unusual observations. Simulation results are presented indicating the sensitivity of the procedure in detecting model violations under a variety of situations, and its performance is compared with three potential competitors, including a procedure based on the Box-Cox power transformation. The procedure is demonstrated by applying it to a new car mileage data set and a water salinity data set that has been used previously to illustrate model diagnostics.

1 Linear Model and its Assumptions

One of the most important models in Statistics is the linear model, in which the relationship between an observable n × 1 response vector Y and an observable n × p design matrix X of predictor variables is given by

$$\mathbf{Y}=\mathbf{X}\beta +\sigma \epsilon ,$$ (1)

where β is a p × 1 vector of unknown coefficients, σ is an unknown scale parameter, and ε is an n × 1 vector of unobservable error variables. Conditionally on X, ε has a multivariate normal distribution with mean vector 0 and covariance matrix I, the n × n identity matrix. This distributional assumption, together with the linear link specification in (1) are enumerated as four distinct assumptions: (A1) (Linearity) E{Y_i|X} = x_iβ,where x_i is the ith row of X; (A2) (Homoscedasticity) Var{Y_i|X} = σ², i = 1,2,…,n; (A3) (Uncorrelatedness) Cov{Y_i,Y_j|X} = 0,(i ≠ j); and (A4) (Normality) (Y₁,Y₂,…,Y_n)|X have a multivariate normal distribution. Assumptions (A3) and (A4) imply that, given X, Y_i, i = 1, 2, …, n,are independent normal random variables. Without loss of generality, we assume that X is of full rank with n > p, so rank(X) = p. Under (A1)–(A4),the maximum likelihood (ML) estimators of β and σ² are given, respectively, by

$$\mathbf{b}=\widehat{\beta }={({\mathbf{X}}^{\text{t}}\mathbf{X})}^{-1}{\mathbf{X}}^{\text{t}}\mathbf{Y\hspace{0.17em} }\mathrm{\hspace{0.17em} }\text{and\hspace{0.28em}}\mathrm{\hspace{0.17em} }{s}^2={\widehat{\sigma }}^2=\frac{1}{n}{\mathbf{Y}}^{\text{t}}(\mathbf{I}-\mathbf{P}[\mathbf{X}])Y;$$ (2)

where P[X] = H = X(X^tX)⁻¹X^t is the projection operator on the linear subspace generated by the columns of X. The estimator b in (2) is also the least-squares (LS) estimator of β. The usual procedures for constructing confidence ellipsoids/intervals and for testing hypotheses for β and σ² rely on the validity of (A1)–(A4). The consequences of the breakdown of any of these four assumptions are well-known, and possible remedial measures such as variable transformations, weighted regression, incorporating additional predictor variables and, if need be, the adoption of nonparametric methods, have also been discussed (see, for example, Neter, Kutner, Nachtsheim, and Wasserman 1996).

Assessment of whether assumptions (A1)–(A4) are satisfied, based on the data (Y, X), has received considerable attention. Assessment procedures typically involve the standardized residuals R, herein defined according to

$$\mathbf{R}=\frac{1}{s}(\mathbf{Y}-\mathbf{Xb})=\frac{1}{s}(\mathbf{I}-\mathbf{P}[\mathbf{X}])\mathbf{Y}.$$ (3)

There are other types of residuals that have been used in model validation and diagnostics. The studentized residuals are R′ = (R′₁, …, R′_n) with $R{\text{'}}_i=\frac{n-p}{n}\sqrt{1-h_{ii}}{R}_i$, where h_ii is the ith diagonal element of H. Other residuals are Theil (1965)’s best linear unbiased scalar covariance residuals (BLUS residuals) as well as recursive or sequential residuals; see review paper. We focus our attention on R, as this is the residual vector that naturally arises from our theoretical development.

Important work in assessing the model assumptions includes Tukey (1949) for assessing (A1); Durbin and Watson (1950, 1951) for assessing (A3); Anscombe (1961) and Anscombe and Tukey (1963) for assumptions (A4) and (A2). Many of these methods are summarized and discussed in Cook and Weisberg (1982) and Atkinson (1985). It should be noted that the residuals are not independent and may have different variances even if (A1)–(A4) hold, in contrast to the i.i.d. structure of ε = (Y − Xβ)/σ, which is the counterpart of R when the model parameters β and σ are known. The impact, especially the non-negligible change on the distributional properties of the residuals even in large samples, by the substitution of estimators for unknown parameters to obtain the residuals has been duly noted, cf., Durbin and Watson (1950); Anscombe and Tukey (1963); Theil (1965); Atkinson (1985).

In the assessment of (A1)–(A4) through graphical methods, the impact of the aforementioned substitution is mostly ignored, potentially giving rise to inaccurate assessment. Moreover, even apart from this issue, the interpretation of graphical methods is highly subjective, for though a picture is worth a thousand words, beauty is in the eye of the beholder. Furthermore, a particular plot is used to assess a specific assumption, and sometimes the synergistic impact of combinations of violations of (A1)–(A4) is not clear. It is therefore beneficial to augment these plots with a numerical measure of the degree to which (A1)–(A4) are violated.

Formal significance tests for (A1)–(A4) involve testing the null hypothesis (H₀) versus the alternative hypothesis (H₁), where

$$\begin{array}{l}{H}_0\mathrm{\hspace{0.17em} }:\mathrm{\hspace{0.17em} }\text{Assumptions\hspace{0.28em}(A1)}-\text{(A4) all hold};\\ H_1\mathrm{\hspace{0.17em} }:\mathrm{\hspace{0.17em} }\text{At leaset one of (A1)}-\text{(A4) does not hold}\text{.}\end{array}$$ (4)

The typical structure of such a test is to define a statistic S(R) whose sampling distribution is known under H₀ and such that departures from H₀ will manifest in terms of larger values of S(R). Given an observed residual vector R = r, one calculates the p-value via p = P {S(R) > S(R) >S(r) | H₀}, and the decision to reject H₀ is based on the magnitude of p. However, existing formal significance tests are typically tests for a specific assumption, hence are not simultaneous or global tests for the four assumptions (A1)–(A4). For instance, there are tests for the normality assumption (cf., Anscombe and Tukey 1963); there are tests for link mis-specifications (cf., Tukey 1949); there are tests for heterogeneity of variances (cf., Cook and Weisberg 1983; Bickel 1978; Anscombe 1961); and there are tests for the uncorrelatedness or independence of the error components (cf., Durbin and Watson 1950cf., Durbin and Watson 1951; Theil and Nagar 1961). See also Kianifard and Swallow (1996) for procedures that use the recursive residuals for significance testing of the different assumptions. The difficulty with these tests is that each is designed to detect departures from one assumption, and the impact of violations of other assumptions on this test, as well as its sensitivity against these violations are not apparent. Hence, when a specific test indicates a violation, it might be due to the violation of another assumption which affects this test. For example, a test for normality could be affected by a mis-specified link function or dependent error components. One may decide to perform tests for each of the different assumptions, but this will lead to an increase in the Type I error probability when the results of these tests are combined, though some corrective measure such as a Bonferroni adjustment could be implemented to alleviate this inflation. There is therefore a need to have a global test for all the assumptions (A1)–(A4) that controls the Type I error rate and which could be used especially if the analyst does not have an idea of which set of assumptions are violated. If such a test indicates that at least one of the assumptions is not satisfied, then directional tests may be used to determine the assumptions that have been violated. Knowing the set of assumptions that has been violated is important for instituting appropriate remedial measures, such as variable transformations, adjustments in the link function, utilizing lagged values, etc.

In this paper we propose such a global test. An important consideration in our proposal is that the procedure should be simple and easy-to-implement, but at the same time should be theoretically justifiable. Our procedure is based on the residual vector R, and the theoretical development of the procedure relies on the idea of smooth test (cf., Thomas and Pierce 1979; Rayner and Best 1986, 1989). The components of the global test can also be used as directional tests for determining the assumptions that have been violated. Because functions of R generally possess complicated distributional properties, asymptotic distributional properties for the global test are ascertained. For small sample sizes, computer-intensive methods may be employed to determine p-values. We also discuss deletion statistics based on the global statistic that can be used to identify outlying or influential observations. Moreover, the mathematical framework for the test procedure is quite general and allows for a broad class of tests to be generated by changing the embedding functions (see Section 3). However, with the goal of obtaining an easy-to-implement procedure, and to recover some currently-used directional tests, we have confined ourselves to a particular set of embedding functions. As a reviewer pointed out, a better procedure may arise via a different choice of embedding functions, but possibly at the cost of greater complexity. Even with this potential limitation, the performance of the proposed procedure is still commendable as seen in the simulation studies in Sections 5 and the applications in Section 6.

There is a deeper foundational issue regarding model-building in relation to the validation of the model assumptions and the additional inferences that are made such as testing hypotheses, construction of confidence intervals about the regression parameters, or the prediction of future observations. For example, suppose that the linear model assumptions are validated through formal and/or graphical methods using the observed data, so this validation process is subject to error, and then regression parameters are estimated using the same data and via procedures derived under the linear model and its assumptions. How should one assess the properties of these estimators in light of this two-step process? There is a growing literature and ongoing active research on the more general, but related, area of inference after model selection; see for instance Hjort and Claeskens (2003), Claeskens and Hjort (2003), and Dukić and Peña (2005), and references in these papers. This is an important issue that needs to be addressed, but this paper focuses on formally validating the model assumptions.

The paper is organized as follows. Section 2 describes and discusses the global and the component statistics. The theoretical justification of the global procedure is presented in Section 3 where it is derived as a Neyman smooth test. The asymptotic normality and the asymptotic independence of the components are established in this section. Deletion statistics, obtained by excluding an observation from the analysis, are described in Section 4. Section 5 presents simulation studies that examined the properties of the procedures. Section 6 illustrates the applications of these procedures to two real data sets. Concluding thoughts are provided in Section 7.

2 Validation Procedures

We first present the tests in this section, and then provide theoretical justification in the next section. Henceforth, we assume that X has as its first column the n × 1 vector 1 = (1, 1, …, 1)^t, so that we are incorporating an intercept term in model (1). Recalling that the ith component of the residual vector R is $R_i=(Y_i-{\widehat{Y}}_i)/ s,i=1,2,\dots ,n$, where ${\widehat{Y}}_i={\mathbf{x}}_i\mathbf{b}$ is the ith fitted value, the first three component statistics are as follows:

$${\widehat{S}}_1^2={\left\{\frac{1}{\sqrt{6n}}\underset{i=1}{\overset{n}{\Sigma }}{R}_i^3\right\}}^2;$$ (5)

$${\widehat{S}}_2^2={\left\{\frac{1}{\sqrt{24n}}\underset{i=1}{\overset{n}{\Sigma }}(R_i^4-3)\right\}}^2;$$ (6)

$${\widehat{S}}_3^2=\frac{{\left\{\frac{1}{\sqrt{n}}{\Sigma }_{i=1}^n{({\widehat{Y}}_i-\overline{Y})}^2{R}_i\right\}}^2}{(\widehat{\mathrm{\Omega }}-{\left({\mathbf{b}}^{\text{t}}{\widehat{\mathrm{\Sigma }}}_X\mathbf{b}\right)}^2-\widehat{\mathrm{\Gamma }}{\widehat{\mathrm{\Sigma }}}_X^{-1}{\widehat{\mathrm{\Gamma }}}^{\text{t}})},$$ (7)

where, with $\overline{z}=\frac{1}{n}{1}^{\text{t}}\mathbf{Z}$ for an n × q matrix Z, we define

$$\begin{array}{}\\ \widehat{\mathrm{\Omega }}=\frac{1}{n}{\mathrm{\Sigma }}_{i=1}^n{({\widehat{Y}}_i-\overline{Y})}^4;\\ {\widehat{\mathrm{\Sigma }}}_X=\frac{1}{n}{\mathrm{\Sigma }}_{i=1}^n{({\text{x}}_i-\overline{\text{x}})}^{\text{t}}({\text{x}}_i-\overline{\text{x}});\\ \widehat{\mathrm{\Gamma }}=\frac{1}{n}{\mathrm{\Sigma }}_{i=1}^n{({\widehat{Y}}_i-\overline{Y})}^2({\text{x}}_i-\overline{\text{x}}).\\ \end{array}\}$$ (8)

The fourth component statistic requires a user-supplied n × 1 vector V, which by default is set to be the standardized time sequence V = (1, 2, …, n)^t/n. It is defined via

$${\widehat{S}}_4^2={\left\{\frac{1}{\sqrt{2{\widehat{\sigma }}_V^{2}n}}\underset{i=1}{\overset{n}{\Sigma }}(V_i-\overline{V})(R_i^2-1)\right\}}^2,$$ (9)

with ${\widehat{\sigma }}_V^2=\frac{1}{n}{\Sigma }_{i=1}^n{(V_i-\overline{V})}^2$. The global test statistic is defned as

$${\widehat{G}}_4^2={\widehat{S}}_1^2+{\widehat{S}}_2^2+{\widehat{S}}_3^2+{\widehat{S}}_4^2.$$ (10)

An appealing feature of this global statistic is that variants of the statistics ${\widehat{S}}_i^2,(i=1,2,3,4)$, have been considered for significance testing purposes in earlier papers. For instance, statistics related to ${\widehat{S}}_1^2$ and ${\widehat{S}}_2^2$ have appeared in Anscombe and Tukey (1963), and a statistic related to ${\widehat{S}}_4^2$ has been considered by Cook and Weisberg (1983); Bickel (1978); Anscombe (1961) in the context of testing for heteroscedasticity. Statistic ${\widehat{S}}_3^2$ is related to test for additivity. One of the main contributions of this paper is combining these different directional statistics in a global statistic and determining its properties. We will see in ensuing sections that this combined global statistic serves as an omnibus statistic for globally testing all the assumptions of the linear model.

For large n, which for application purposes will be understood to mean that n − p ≥ 30, theglobal test for the hypotheses H₀ versus H₁ in (4) at an asymptotic significance level of α is:

$$\mathbf{\text{Global\hspace{0.28em}Test}}:\mathrm{\hspace{0.17em} }\text{Reject\hspace{0.28em}}{H}_0\mathrm{\hspace{0.17em} }\text{if\hspace{0.28em}}{\widehat{G}}_4^2>{\chi }_{4;\alpha }^2,$$ (11)

where ${\chi }_{k;\alpha }^2$ is the 100(1 − α)th percentile of a central chi-squared distribution with degrees-of-freedom (df) k. If the test in (11) leads to the rejection of H₀,the component statistics ${\widehat{S}}_1^2,\mathrm{\hspace{0.17em} }{\widehat{S}}_2^2,\mathrm{\hspace{0.17em} }{\widehat{S}}_3^2,\mathrm{\hspace{0.17em} }\text{and\hspace{0.28em}}{\widehat{S}}_4^2$ could be examined by comparing their values to ${\chi }_{1;\alpha }^2$, or perhaps more approxiately to ${\chi }_{1;\alpha /4}^2$ (see the test in (19)) or ${\chi }_{1;1-{(1-\alpha )}^{1/4}}^2$ (see the test in (20)), to get an indication of which particular assumption or assumptions have been violated. The following are rough guidelines in interpreting the values of these component statistics, with these guidelines suggested by the theoretical considerations to be presented in Section 3 and the simulation results in Section 5: (i) Skewed error distributions will usually be indicated by large values of the statistic ${\widehat{S}}_1^2$; (ii) Deviations from the normal distribution kurtosis of the true error distribution will be generally revealed by large values of statistic ${\widehat{S}}_2^2$; (iii) The use of a misspecified link function, possibly due to the absence of other predictor variables in the model, will mostly be detected by large values of the statistic ${\widehat{S}}_3^2$; (iv) The presence of heteroscedastic errors and/or dependent errors will typically manifest in large values of the statistic ${\widehat{S}}_4^2$; and (v) Simultaneous violations of at least two of the assumptions (A1)–(A4) will be manifested by large values of several of these component statistics.

3 Theoretical Development of Procedure

From (1), if the true parameter values β and σ are known, we may call, perhaps inappropriately, the error vector ε to be the vector of ‘true’ residuals R⁰. Thus, R⁰ ≡ R⁰(σ², β) = (Y −Xβ)/σ, which therefore is equal-in-distribution to the error vector ε. If H₀ holds, then the density function of R⁰ is

$$f_{{\mathbf{R}}^0}({\mathbf{r}}^0)=\prod _{i=1}^n\phi (r_i^0),$$

where $\phi (z)=\text{exp}\{-z^2 / 2\}/ \sqrt{2\pi }$ is the standard normal density function. Following idea of constructing a ‘smooth’ test (cf., Thomas and Pierce 1979; Rayner and Best 1989), we embed fR_⁰(r⁰) into a class of density functions, indexed by θ = (θ₁, θ₂, …, θ₆)^t, whose members are of the form

$$f_{{\mathbf{R}}^0}({\mathbf{r}}^0 |\theta ,\mathrm{\hspace{0.17em} }{\sigma }^2, \hspace{0.17em}\beta )=C(\theta ;\hspace{0.17em}{\sigma }^2,\hspace{0.17em}\beta )\mathrm{\hspace{0.17em} }{f}_{{\mathbf{R}}^0}({\mathbf{r}}^0)\mathrm{\hspace{0.17em} }\text{exp}\{{\theta }^{\text{t}}\mathbf{Q}({\mathbf{r}}^0;\hspace{0.17em}{\sigma }^2,\hspace{0.17em}\beta )\},$$ (12)

where $\mathbf{Q}({\mathbf{r}}^0;\hspace{0.17em}{\sigma }^2,\hspace{0.17em}\beta )\equiv {\Sigma }_{i=1}^n{\mathbf{Q}}_i(r_i^0;\hspace{0.17em}{\sigma }^2,\hspace{0.17em}\beta )$ with

$${\mathbf{Q}}_i(z;\hspace{0.17em}{\sigma }^2,\hspace{0.17em}\beta )={(z,\hspace{0.17em}{z}^2-1,z^3,z^4-3,\hspace{0.17em}{\{({\mathbf{x}}_i-\overline{\mathbf{x}})\beta \}}^{2}z,\hspace{0.17em}(V_i-\overline{V})[z^2-1])}^{\text{t}}.$$

The particular choice of the Q_i(z; σ², β) functions is motivated by our desire to recover commonly-used directional statistics. Other forms for the Q_i(z; σ², β) functions, such as trigonometric or wavelet functions, are certainly possible, and may lead to procedures with better properties. The function C(θ; σ², β) in (12) is a proportionality constant that makes $f_{{\mathbf{R}}^0}({\mathbf{r}}^0 |\theta ,\hspace{0.17em}{\sigma }^2,\hspace{0.17em}\beta )$ a density function. A straightforward calculation shows that this constant satisfies $C{(\theta ;{\sigma }^2,\beta )}^{-1}={\prod }_{i=1}^n\mathbf{E\hspace{0.17em}}[\text{exp}\{{\theta }^{\text{t}}{\mathbf{Q}}_i(Z;\hspace{0.17em}{\sigma }^2,\hspace{0.17em}\beta )\}]$, where Z is a standard normal random variable. Notice that in the in the embedding class, the null hypothesis density function obtains when θ = 0. When β = (β1,β*^t)^t is fixed, this larger family, which does not depend on β₁, is an exponential family of densities, hence possesses many of the nice properties intrinsic to exponential families. Furthermore, observe that if we allow for the case where β* = 0, then the larger model is not identifiable since (θ₅ = 0, β* ≠ 0) and (θ₅ ≠ 0, β* = 0) both lead to the same distribution. But since this model validation issue only becomes practically important in the presence of a ‘trend,’ which is the case where β* ≠ 0, then in the theoretical development we assume that β* resides in ℜ_p−1 \ {0}, which does not lead to any technical difficulties as this is still an open set in ℜ_p−1.

Let us first consider the case where β and σ² are known, so R⁰ = R⁰(σ², β) is observable. Within the embedding class of density functions specified by (12), the score test for $H_0^{\ast }:\theta =0$ versus $H_1^{\ast }:\theta \ne 0$ is easily developed. The use of score tests in this situation is appealing because it is known that score tests are endowed with a “robustness of optimality” property, see Chen (1983, 1985) regarding this property, and Cox and Hinkley (1974) for a general discussion of score tests. In our setting, it is straightforward to see that the score test statistic at θ = 0 equals

$$\mathbf{U}(\theta =0,\hspace{0.17em}{\sigma }^2,\beta )=\mathbf{Q}({\mathbf{R}}^0;{\sigma }^2,\beta ).$$

Since under $H_0,\mathrm{\hspace{0.17em} }{R}_i^0,\hspace{0.17em}(i=\mathrm{\hspace{0.17em} }1,2,\dots ,n)$, are i.i.d. standard normal variables, then for any positive integer k, $\mathbf{E}\{{[R_i^0]}^{2k+1}\}=0$ and $\mathbf{E}\{{[R_i^0]}^{2k}\}={\prod }_{j=1}^k(2j-1)$, and so the covarience matrix of $\frac{1}{\sqrt{n}}\mathbf{Q}({\mathbf{R}}^0;{\sigma }^2,\beta )$ is

$${\mathrm{\Sigma }}_{11}^{(n)}({\sigma }^2,\beta )=\left[\begin{array}{cccccc}1& 0& 3& 0& \frac{1}{n}{T}_2(\beta )& 0\\ 0& 2& 0& 12& 0& 0\\ 3& 0& 15& 0& \frac{3}{n}{T}_2(\beta )& 0\\ 0& 12& 0& 96& 0& 0\\ \frac{1}{n}{T}_2(\beta )& 0& \frac{3}{n}{T}_2(\beta )& 0& \frac{1}{n}{T}_4(\beta )& 0\\ 0& 0& 0& 0& 0& \frac{2}{n}{\Sigma }_{i=1}^n{(V_i-\overline{V})}^2\end{array}\right],$$

where $T_k(\beta )={\Sigma }_{i=1}^n{[({\mathbf{x}}_i-\overline{\mathbf{x}})\beta ]}^k$, k = 2, 4. If, as n → ∞, the following conditions are satisfied:

1. There exists a nonsingular p × p matrix Σ_x such that $\frac{1}{n}{T}_2(\beta )\stackrel{\text{pr}}{\to }{\beta }^{\text{t}}{\mathrm{\Sigma }}_X\beta ;$

2. There exists a function Ω(β) such that $\frac{1}{n}{T}_4(\beta )\stackrel{\text{pr}}{\to }\mathrm{\Omega }(\beta );$

3. There exists a ${\sigma }_V^2\in (0,\infty )$ such that $\frac{1}{n}{\Sigma }_{i=1}^n{(V_i-\overline{V})}^2\stackrel{\text{pr}}{\to }{\sigma }_V^2;$

4. $\left\{{\text{max}}_{1\le i\le n}{[({\mathbf{x}}_i-\overline{\mathbf{x}})\beta ]}^4\right\}/T_4(\beta )=o_p(1);$ and

5. $\left\{{\text{max}}_{1\le i\le n}{(V_i-\overline{V})}^2\right\}/\left\{{\Sigma }_{i=1}^n{(V_i-\overline{V})}^2\right\}=o_p(1);$

then it follows from the Lindeberg-Feller Central Limit Theorem (CLT) that, under H₀,

$$\frac{1}{\sqrt{n}}\mathbf{Q}({\mathbf{R}}^0;{\sigma }^2,\beta )\stackrel{\text{d}}{\to }N(0,{\mathrm{\Sigma }}_{11}({\sigma }^2,\beta )),$$

where

$${\mathrm{\Sigma }}_{11}({\sigma }^2,\beta )=\left[\begin{array}{cccccc}1& 0& 3& 0& {\beta }^{\text{t}}{\mathrm{\Sigma }}_X\beta & 0\\ 0& 2& 0& 12& 0& 0\\ 3& 0& 15& 0& 3{\beta }^{\text{t}}{\mathrm{\Sigma }}_X\beta & 0\\ 0& 12& 0& 96& 0& 0\\ {\beta }^{\text{t}}{\mathrm{\Sigma }}_X\beta & 0& 3{\beta }^{\text{t}}{\mathrm{\Sigma }}_X\beta & 0& \mathrm{\Omega }(\beta )& 0\\ 0& 0& 0& 0& 0& 2{\sigma }_V^2\end{array}\right].$$ (13)

In this situation where β and σ² are assumed known, notice the asymptotic dependence of the components Q₁, Q₃ and Q₅; as well Q₂ and Q₄. An asymptotic α-level score test for $H_0^{\ast }:\theta =0$ versus $H_1^{\ast }:\theta \ne 0$ rejects $H_0^{\ast }$ whenever

$$\frac{1}{n}\mathbf{Q}{({\mathbf{R}}^0;{\sigma }^2,\beta )}^{\text{t}}{[{\mathrm{\Sigma }}_{11}^{(n)}({\sigma }^2,\beta )]}^{-1}\mathrm{\hspace{0.17em} }\mathbf{Q}({\mathbf{R}}^0;{\sigma }^2,\beta )\ge {\chi }_{6;\alpha }^2.$$

However, since σ² and β are unknown, neither R⁰ nor ${\mathrm{\Sigma }}_{11}^{(n)}$ are observable. There is therefore a need to use estimators for σ² and β in R⁰(σ², β), and by substituting the ML estimators s² and b given in (2), respectively, we obtain the (estimated) residual vector R = R⁰(s², b) given in (3). To develop a test based on R, we need the asymptotic distribution of Q(R; s², b) under H₀. Towards this goal, observe that the ML estimating equations for σ² and β that give rise to s² and b are

$$A({\mathbf{R}}^0({\sigma }^2,\beta );{\sigma }^2,\beta )\equiv {\mathbf{R}}^0{({\sigma }^2,\beta )}^{\text{t}}{\mathbf{R}}^0({\sigma }^2,\beta )-n=0;$$ (14)

$$\mathbf{B}({\mathbf{R}}^0({\sigma }^2,\beta );{\sigma }^2,\beta )\equiv \sigma {\mathbf{X}}^{\text{t}}{\mathbf{R}}^0({\sigma }^2,\beta )=0.$$ (15)

Augmenting the vector Q with A and B, then invoking the Lindeberg-Feller CLT, we find that, under H₀, plus the conditions guaranteeing asymptotic normality of Q(R⁰(σ², β); σ², β) enumerated earlier,

$$\frac{1}{\sqrt{n}}\left[\begin{array}{l}\mathbf{Q}({\mathbf{R}}^0({\sigma }^2,\beta );{\sigma }^2,\beta )\\ A({\mathbf{R}}^0({\sigma }^2,\beta );{\sigma }^2,\beta )\\ \mathbf{B}({\mathbf{R}}^0({\sigma }^2,\beta );{\sigma }^2,\beta )\end{array}\right]\mathrm{\hspace{0.17em} }\stackrel{\text{d}}{\to }\mathrm{\hspace{0.17em} }N\hspace{0.17em}(0,\mathrm{\Xi }({\sigma }^2,\beta )),$$ (16)

where

$$\mathrm{\Xi }({\sigma }^2,\beta )=\left[\begin{array}{cc}{\mathrm{\Sigma }}_{11}({\sigma }^2,\beta )& {\mathrm{\Sigma }}_{12}({\sigma }^2,\beta )\\ {\mathrm{\Sigma }}_{12}{({\sigma }^2,\beta )}^{\text{t}}& {\mathrm{\Sigma }}_{22}({\sigma }^2,\beta )\end{array}\right],$$

with

$${\mathrm{\Sigma }}_{12}({\sigma }^2,\beta )=\left[\begin{array}{cc}0& \sigma {\mu }_X\\ 2& 0\\ 0& 3\sigma {\mu }_X\\ 12& 0\\ 0& \sigma [\mathrm{\Gamma }(\beta )+({\beta }^{\text{t}}{\mathrm{\Sigma }}_X\beta ){\mu }_X]\\ 0& 0\end{array}\right];\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }{\mathrm{\Sigma }}_{22}({\sigma }^2,\beta )=\left[\begin{array}{cc}2& 0\\ 0& {\sigma }^2({\mathrm{\Sigma }}_X+{\mu }_X^{\text{t}}{\mu }_X)\end{array}\right];$$

and with μ_X and Γ(β) defined according to

$$\overline{\mathbf{x}}\stackrel{\text{pr}}{\to }{\mu }_X\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }\text{and\hspace{0.28em}}\mathrm{\hspace{0.17em} }\frac{1}{n}\underset{i=1}{\overset{n}{\Sigma }}{[({\mathbf{x}}_i-\overline{\mathbf{x}})\beta ]}^2({\mathbf{x}}_i-\overline{\mathbf{x}})\stackrel{\text{pr}}{\to }\mathrm{\Gamma }(\beta ).$$

By virtue of (14) and (15), when s² and b are substituted for σ² and β, respectively, the last two components in the augmented vector are both equal to zero. Consequently, it follows by multivariate normal theory, or it could be established more formally by relying on result, that

$$\frac{1}{\sqrt{n}}\mathbf{Q}(R^0(s^2,\mathbf{b});s^2,\mathbf{b})=\frac{1}{\sqrt{n}}\mathbf{Q}(R;s^2,\mathbf{b})\mathrm{\hspace{0.17em} }\stackrel{\text{d}}{\to }\mathrm{\hspace{0.17em} }N\hspace{0.17em}(0,{\mathrm{\Xi }}_{11.2}({\sigma }^2,\beta ))$$

where ${\mathrm{\Xi }}_{11.2}({\sigma }^2,\beta )={\mathrm{\Sigma }}_{11}({\sigma }^2,\beta )-{\mathrm{\Sigma }}_{12}({\sigma }^2,\beta ){\mathrm{\Sigma }}_{22}{({\sigma }^2,\beta )}^{-1}{\mathrm{\Sigma }}_{12}{({\sigma }^2,\beta )}^{\text{t}}.$ To provide a simplified form for this limiting covariance matrix, we establish the following intermediate result.

Lemma 1 If the first column of X is 1, then ${\mu }_X{({\mathrm{\Sigma }}_X+{\mu }_X^{\text{t}}{\mu }_X)}^{-1}{\mu }_X^{\text{t}}=1.$

Proof: Write X = [1 W] so ${\mathrm{\Sigma }}_X+{\mu }_X^{\text{t}}{\mu }_X=\left[\begin{array}{cc}1& {\mu }_W\\ {\mu }_W^{\text{t}}& {\mathrm{\Sigma }}_W+{\mu }_W^{\text{t}}{\mu }_W\end{array}\right].$ Applying the partitioned matrix inverse theorem (cf., Anderson 1984, Th. A.3.3), we obtain

$${[{\mathrm{\Sigma }}_X+{\mu }_X^{\text{t}}{\mu }_X]}^{-1}=\left[\begin{array}{cc}1+{\mu }_W{\mathrm{\Sigma }}_W^{-1}{\mu }_W^{\text{t}}& -{\mu }_W{\mathrm{\Sigma }}_W^{-1}\\ -{\mathrm{\Sigma }}_W^{-1}{\mu }_W^{\text{t}}& {\mathrm{\Sigma }}_W^{-1}\end{array}\right].$$

Since μ_X = (1 μ_W), then the assertion immediately follows by matrix multiplication.||

By straightforward multiplication, and applying Lemma 1, we obtain

$$\mathrm{\Delta }({\sigma }^2,\beta )\equiv {\mathrm{\Sigma }}_{12}{\mathrm{\Sigma }}_{22}^{-1}{\mathrm{\Sigma }}_{12}^{\text{t}}=\mathrm{\hspace{0.17em} }\left[\begin{array}{cccccc}1& 0& 3& 0& {\beta }^{\text{t}}{\mathrm{\Sigma }}_X\beta & 0\\ 0& 2& 0& 12& 0& 0\\ 3& 0& 9& 0& 3{\beta }^{\text{t}}{\mathrm{\Sigma }}_X\beta & 0\\ 0& 12& 0& 72& 0& 0\\ {\beta }^{\text{t}}{\mathrm{\Sigma }}_X\beta & 0& 3{\beta }^{\text{t}}{\mathrm{\Sigma }}_X\beta & 0& \eta (\beta )& 0\\ 0& 0& 0& 0& 0& 0\end{array}\right],$$ (17)

with $\eta (\beta )={({\beta }^{\text{t}}{\mathrm{\Sigma }}_X\beta )}^2+\mathrm{\Gamma }(\beta ){\mathrm{\Sigma }}_X^{-1}\mathrm{\Gamma }{(\beta )}^{\text{t}}.$ The matrix Δ(σ², β) is the correction factor in the limiting covariance matrix arising from plugging-in s² and b for σ² and β, respectively. This factor is clearly non-negligible. Finally, from (13) and (17), a simplified form of Ξ_11.2 is

$${\mathrm{\Xi }}_{11.2}({\sigma }^2,\beta )=\left[\begin{array}{cccccc}0& 0& 0& 0& 0& 0\\ 0& 0& 0& 0& 0& 0\\ 0& 0& 6& 0& 0& 0\\ 0& 0& 0& 24& 0& 0\\ 0& 0& 0& 0& \xi ({\sigma }^2,\beta )& 0\\ 0& 0& 0& 0& 0& 2{\sigma }_V^2\end{array}\right]$$ (18)

where $\xi ({\sigma }^2,\beta )=\mathrm{\Omega }(\beta )-{({\beta }^{\text{t}}{\mathrm{\Sigma }}_X\beta )}^2-\mathrm{\Gamma }(\beta ){\mathrm{\Sigma }}_X^{-1}\mathrm{\Gamma }{(\beta )}^{\text{t}}.$ We formally state this asymptotic result as a theorem.

Theorem 1 If assumptions (A1)–(A4) hold for the linear model in (1) with X having as its first column the vector 1, and if conditions (a)–(e) enumerated earlier hold, then n^−½Q(R; s², b) converges in distribution to a zero-mean normal distribution with covariance matrix Ξ_11.2 given in (18).

Note the invariance of this asymptotic result to re-scaling, that is, the result is independent of σ. This is a consequence of the facts that the model is scale-invariant and the residual vector is scale-equivariant. The theorem also indicates that Q₁(R; s², b) and Q₂(R; s², b) are degenerate at zero, hardly a surprise since these quantities are the estimating functions for σ² and β. What is surprising, instead, is the asymptotic independence of Q₃(R; s², b) and Q₅(R; s², b), since as noted earlier, Q₃(R⁰; σ², β) and Q₅(R⁰; σ², β) are not asymptotically independent. Thus, interestingly and unexpectedly, replacing the unknown parameters by their ML estimators in the quantities Q(R⁰(σ², β); σ², β) made all the components asymptotically independent!

The quantities Ω(β), Σ_X, and Γ(β) can be consistently estimated by their empirical counterparts and with β replaced by b. Their respective estimators are those given in (8), and so we are able to obtain a consistent estimator ${\widehat{\mathrm{\Xi }}}_{11.2}$ of Ξ_11.2. The score statistic for testing $H_0^{\ast }:\theta =0$ versus $H_1^{\ast }:\theta \ne 0$, with σ² and β considered as nuisance parameters, is the quadratic form of $\frac{1}{\sqrt{n}}\mathbf{Q}(\mathbf{R};s^2,\mathbf{b})$ with quadratic matrix ${\widehat{\mathrm{\Xi }}}_{11.2}^{-}$, where for a matrix M, M⁻ denotes inverse. It is immediate to see that this statistic is

$$\frac{1}{n}\mathbf{Q}{(\mathbf{R};s^2,\mathbf{b})}^{\text{t}}{\widehat{\mathrm{\Xi }}}_{11.2}^{-}\mathbf{Q}(\mathbf{R};s^2,\mathbf{b})={\widehat{S}}_1^2+{\widehat{S}}_2^2+{\widehat{S}}_3^2+{\widehat{S}}_4^2={\widehat{G}}_4^2,$$

where ${\widehat{S}}_k^2,(k=1,2,3,4)$, and ${\widehat{G}}_4^2$ are as defined in (5), (6), (7), (9), and (10), respectively. Theorem 1 therefore justifies the use of the chi-squared distribution with four df for assessing the magnitude of ${\widehat{G}}_4^2$, as well as the one df chi-squared distributions for each of the component statistics.

Before proceeding, we mention three possible competing test procedures to the ${\widehat{G}}_4^2$-based test. These competing tests are also included in the simulation studies. The first is to perform simultaneous testing using the test statistics ${\widehat{S}}_i^2,i\mathrm{\hspace{0.17em} }=\mathrm{\hspace{0.17em} }1,\hspace{0.17em}2,\hspace{0.17em}3,\hspace{0.17em}4,$ but incorporating a Bonferroni adjustment. By virtue of the asymptotic results above, this test is as follows:

$$\mathbf{\text{BonfTest}}:\mathrm{\hspace{0.17em} }\text{Reject\hspace{0.28em}}{H}_0\mathrm{\hspace{0.17em} }\text{if\hspace{0.28em}Gmax}\equiv \underset{1\le i\le 4}{\text{max}}{\widehat{S}}_i^2>{\chi }_{1;\alpha /4}^2.$$ (19)

This amounts to rejecting H₀ if at least one of the unidirectional tests rejects H₀ at level of significance of α/4.

The second competing test arises by recognizing that under H₀, by invoking the asymptotic independence of the component statistics, the asymptotic distribution of the test statistic Gmax in (19) is $\mathbf{P}\{\text{Gmax}\le w|H_0\}\mathrm{\hspace{0.17em} }\stackrel{n\to \infty }{\to }\mathrm{\hspace{0.17em} }{[P\{{\chi }_1^2\le w\}]}^4.$ As a consequence, an asymptotic α-level test of H₀ is provided by:

$$\mathbf{\text{MaxTest}}:\mathrm{\hspace{0.17em} }\text{Reject\hspace{0.28em}}{H}_0\mathrm{\hspace{0.17em} }\text{if\hspace{0.28em}Gmax}>{\chi }_{1;1-{(1-\alpha )}^{1/4}}^2.$$ (20)

When α = .05, the critical values of the tests in (19) and (20) equal 6.239 and 6.205, respectively. This explains the almost identical behaviors of these two tests observed in the simulation studies (see Section 5).

The third competitor, referred in Section 5 as the BoxCox, is the use of the Box and Cox 1964 power transformation. The idea is to ft the linear model on the transformed responses $y_i^{\ast }(\widehat{\gamma }),i=1,2,\dots n,$ with the transformation being

$$y\mapsto y\ast (\gamma )=\{\begin{array}{ccc}(y^{\gamma }-1)/\gamma & \text{if}& \gamma \ne 0\\ \text{log}(\gamma )& \text{if}& \gamma =0\end{array},$$ (21)

where γ is the transformation parameter, and $\widehat{\gamma }$ is its ML estimate. The null hypothesis H₀ is then rejected if the null hypothesis $H_0^{\ast }:\gamma =1$ is rejected. The test for $H_0^{\ast }$ utilized a likelihood-ratio test, with the numerical implementation relying on the R language (Ihaka and Gentleman 1996) object boxcox found in the MASS package of Venables and Ripley.

4 Deletion Statistics

It is important to accompany assessment of model assumptions with investigation for unusual observations (either outlying or influential), since such observations could impact inferences regarding model validity. Unusual observations may arise either as a consequence of model violations or as rare outcomes when the data in fact adhere to the model. In the first case, exclusion of the unusual observations from analysis may have little impact because the global test remains sensitive to violations in the remaining data, or may lead to a nonsignificant global test because the excluded observations aided detection of violations substantially. In the second case, when the data meet model assumptions apart from rare exceptions, unusual observations may cause the global test to indicate violations, so that their deletion would then permit the procedure to reflect the adherence of the remaining data to the model. In any case, unusual observations should be handled with caution, and solid justification is required for their exclusion, as in the examples in Section 6.

A natural ${\widehat{G}}_4^2$-based procedure for detecting unusual observations arises from the well-known idea of deletion statistics, which reflect the change in values of statistics after the deletion of an observation. For a statistic T, denote by T[i] the value of the statistic after the ith observation is deleted. We will be interested in the quantities

$$\mathrm{\Delta }{\widehat{G}}_4^2[i]=\left[\frac{{\widehat{G}}_4^2[i]-{\widehat{G}}_4^2}{{\widehat{G}}_4^2}\right]\mathrm{\hspace{0.17em} }\times \mathrm{\hspace{0.17em} }100,\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }i=1,2,\dots ,n,$$ (22)

which represent the percent relative change in the value of the global statistic ${\widehat{G}}_4^2$ after the deletion of the ith observation. The idea is that an observation with a large absolute value of $\mathrm{\Delta }{\widehat{G}}_4^2[i]$ is either an outlier or has large influence. The sign of this global deletion statistic is also informative, since a positive (negative) value indicates that the deleted observation makes the assumptions more (less) plausible.

Related to the statistic in (22) is the p-value after the deletion of the ith observation, that is,

$$p[i]=\mathbf{P}\{{\widehat{G}}_4^2[i]>{\widehat{g}}_4^2[i]|H_0\},\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }i=1,2,\dots ,n,$$

where ${\widehat{g}}_4^2[i]$ is the observed value of the global statistic after deletion of the ith observation. The evaluation of this probability could be performed using the (approximate) chi-squared distribution with 4 df. The idea is if p[i] is quite different from the other p[j]’s, this will be indicative that the ith observation is either an outlier or an influential observation.

A potentially useful and interesting plot is the scatterplot of p[·] = (p[1],…,p[n])^t versus $\mathrm{\Delta }{\widehat{G}}_4^2[\cdot ]={(\mathrm{\Delta }{\widehat{G}}_4^2[1],\dots ,\mathrm{\Delta }{\widehat{G}}_4^2[n])}^{\text{t}}.$ Following Tukey’s (1977) idea, we indicate in our plots the observation labels of those points beyond the outer fences of either $\mathrm{\Delta }{\widehat{G}}_4^2[\cdot ]$ or p[·].Such observations are unusual in that they either have a large influence on the value or the p-value of the global statistic. This plotting idea will be demonstrated in the illustrative examples in Section 6.

5 Properties of Procedures

Simulation studies were performed to assess the achieved levels and powers of the proposed tests for small to moderate sample sizes. The simulation runs to assess the levels each had 20,000 replications (except for the BoxCox test which was added later at the suggestion of a reviewer), while the runs to determine the powers of the tests had 5,000 replications. The simulation code was in the R language (Ihaka and Gentleman 1996), using the function lm and built-in random number generators. For each set of simulation runs associated with a particular combination of simulation parameters, a common covariate sequence x₁, x₂, …, x_n, generated from the standard uniform distribution was used.

The first set of runs was to determine if the procedures achieve a pre-specified 5% level of significance for the sample sizes considered. The sample size n took values ranging from 5 to 1200; see Table 1. The response values were generated according to the model

Table 1.

Simulated levels (%) of the asymptotic 5%-level tests based on 20,000 replications, except for the Box=-Cox test which are based on 1,000 replications. The data was generated according to Y_i = x_i + ε_i where the x_is are a fixed sequence generated from a standard uniform, and ε_is are i.i.d.N(0,1) variates. The model Y_i = β₀ + β₁x_i was fitted.

RunNum	n	${\widehat{S}}_1^2$	${\widehat{S}}_2^2$	${\widehat{S}}_3^2$	${\widehat{S}}_4^2$	${\widehat{G}}_4^2$	MaxTest	BonfTest	BoxCox
1	5	0:000	0:000	12:285	0:000	0:000	0:000	0:000	4:5
2	15	2:445	0:995	6:425	2:545	2:685	2:440	2:370	4:9
3	30	3:660	2:000	5:620	3:910	4:145	3:500	3:460	4:2
4	50	4:060	2:495	5:260	4:355	4:520	3:990	3:945	4:8
5	100	4:680	3:135	4:935	4:760	5:095	4:520	4:445	5:9
6	150	4:645	3:555	5:100	4:995	5:030	4:700	4:620	4:4
7	200	4:800	3:640	5:180	5:100	5:075	4:920	4:825	5:4
8	300	4:845	3:820	4:875	4:920	4:990	4:780	4:675	5:2
9	400	4:805	4:285	5:015	5:205	5:135	5:135	5:040	5:4
10	600	5:055	4:365	4:735	4:795	5:065	5:045	4:980	3:6
11	800	4:940	4:420	5:230	4:885	5:200	5:055	4:975	5:4
12	1200	5:060	4:700	5:210	5:045	5:185	5:300	5:195	5:0

$$Y_i=x_i+\mathrm{\hspace{0.17em} }{\epsilon }_i,\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }i=1,2,\dots ,n,$$ (23)

where ε_i’s were generated from a standard normal distribution. The model

$$Y_i={\beta }_0+{\beta }_1{x}_i+\sigma {\epsilon }_i,\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }i=1,2,\dots ,n,$$ (24)

was fitted and the resulting residuals,R_i, (i = 1, 2, …, n), were used in the testing procedures. For ${\widehat{S}}_4^2,$ V was the default standardized time-sequence. Table 1 summarizes the observed empirical rejection rates. Note that for small sample sizes (n ≤ 30), the asymptotic approximation is not satisfactory. Except for the test based on ${\widehat{S}}_3^2,$ the procedures tend to be conservative. For moderate to large sample sizes, the procedures achieve significance levels close to the nominal 5%, though the ${\widehat{S}}_2^2$-based statistic has a mild degree of conservatism even when the sample size is large. The rate of convergence to the ${\chi }_1^2$ distribution for this statistic is rather slow, as has been noted in earlier papers; see for instance Doornik and Hansen (1994).

For the power simulations, n takes values in the set {15, 30, 50, 100, 200}. We examined the achieved powers of the tests for n = 15 and n = 30 because, apart from the ${\widehat{S}}_3^2$-based test, the results in Table 1 show that the tests are conservative, which may be acceptable except for a potential decrease in power. Power simulations were performed for specific types of departures from the model assumptions, and for multiple violations of the assumptions. The generic data generation model is given by

$$Y_i=x_i+{\beta }_2{x}_i^{\gamma }+{\sigma }_i^{\ast }{x}_i^{\alpha } {\epsilon }_i,\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }i=1,2,\dots ,n,$$ (25)

where ${\sigma }_i^{\ast }=1,i\le n/2$ and for $i>n/2,\mathrm{\hspace{0.17em} }{\sigma }_i^{\ast }={\sigma }_2.$ In this model, all the assumptions are satisfied whenever β₂ = 0 or γ = 0, σ² = 1, α = 0, and ε_i i.i.d. N(0, 1). To induce a dependent error structure, two models were considered. The first model, which endows the error sequence a martingale structure, has

$${\epsilon }_i=\frac{1}{\sqrt{i}}\underset{j=1}{\overset{i}{\Sigma }}{\epsilon }_j^{\ast }$$ (26)

where ${\epsilon }_j^{\ast } \text{s}$ are i.i.d. from N(0, 1). The second model induces an autoregressive [AR(1)] structure via

$${\epsilon }_1=\mathrm{\hspace{0.17em} }{\epsilon }_1^{\ast }\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }\text{and\hspace{0.28em}}\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }{\epsilon }_i=\frac{\rho {\epsilon }_{i-1}+\mathrm{\hspace{0.17em} }{\epsilon }_i^{\ast }}{\sqrt{1+{\rho }^2}},\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }i=2,\dots ,n,$$ (27)

with ρ being a dependence parameter. Extensive summaries of the simulated powers for all the sample sizes considered in the simulation and for many varieties of departures from model assumptions are summarized in a series of tables in a technical report of the same title as this manuscript, which is available upon request from the authors. To conserve space, we summarize in Table 2 representative cases for each specific departure from model assumptions. Table 3 contains the results when multiple violations occur. The conclusions obtained from this representative summary in Table 2 coincide with those obtained from the extensive tables in the technical report. In the discussion of the simulation results that follows, we also refer to and make use of the more extensive tables in the technical report.

Table 2.

Simulated powers for two sample sizes and specific type of departure from the linear model assumptions. The true model that generated the data is given in (25) with β₂ = 0, γ = 0, σ₂ = 1, α = 0, and ε i.i.d. N(0. 1), except with the specific change in the column headed ‘Relevant Model Parameters’ containing the violation.

Type of Model Violation	Relevant Model Parameters	n	${\widehat{S}}_1^2$	${\widehat{S}}_2^2$	${\widehat{S}}_3^2$	${\widehat{S}}_4^2$	${\widehat{G}}_4^2$	MaxTest	BonfTest	BoxCox
Non-Normal	t5	30	20.8	21.06	5.8	10.7	24	21	20	17.2
		100	39.1	60.80	5.3	16.7	60	57	57	28.9
Error Distribution	χ52	30	47.9	19.70	5.4	10.4	34	32	31	77.9
		100	99.0	57.54	5.4	14.7	93	96	96	100.0
Heteroscedastic	α = 2	30	26.7	59.48	13.4	26.4	59	54	54	64.9
		100	40.8	99.20	12.1	44.3	98	98	98	75.0
Error	σ2 = 2	30	12.1	10.66	6.6	40.8	27	25	25	9.7
		100	19.0	39.00	4.5	97.3	91	92	92	12.0
Misspecified Link	β2 = 3	30	3.2	1.58	29.3	3.8	11	14	14	8.8
	γ = 2	100	4.2	2.86	54.7	5.1	31	35	35	9.3
Function	β2 = 5	30	3.5	1.48	32.6	3.8	13	16	16	12.6
	γ = 2	100	4.4	2.92	95.5	5.4	82	88	88	35.7
Dependent Error	Martingale	30	15.6	7.30	2.6	39.3	27	27	26	25.4
	Type	100	56.2	37.84	1.2	73.8	75	72	72	54.1
Structure	Markov	30	7.4	0.86	6.4	22.6	14	13	12	7.3
	Type (ρ = 5)	100	28.1	26.58	3.5	51.1	55	50	50	29.2

Table 3.

Simulated powers (%) of the tests for models where all the four assumptions are violated. The number of replications is 5,000. The true model that generated the data is given in (25), and the model in (24) is fitted to the data.

	β2	γ	EDa	α	GRb	σ2	TTc	ρ	n	${\widehat{S}}_2^2$	${\widehat{S}}_2^2$	${\widehat{S}}_3^2$	${\widehat{S}}_4^2$	${\widehat{G}}_4^2$	MaxTest	BonfTest	BoxCox
1	2	2	t10	1	T	1.5	MTd	NA	15	10	8.5	11.7	3.4	13	9.7	9.6	37
2	2	2	t10	1	T	1.5	MT	NA	30	21	28.0	8.5	21.1	33	28.4	28.2	42
3	2	2	t10	1	T	1.5	MT	NA	50	38	75.6	23.7	44.0	80	75.4	75.1	68
4	2	2	t10	1	T	1.5	MT	NA	100	45	92.0	25.2	47.4	92	89.8	89.7	68
5	2	2	t10	1	T	1.5	MT	NA	200	52	99.9	42.1	85.4	100	99.9	99.9	79
6	2	0.5	χ 210	2	T	1.5	MT	NA	15	16	14.1	6.8	27.5	25	19.1	19.0	26
7	2	0.5	χ 210	2	T	1.5	MT	NA	30	40	56.9	8.2	17.8	57	52.0	51.9	45
8	2	0.5	χ 210	2	T	1.5	MT	NA	50	55	88.1	17.6	54.3	90	86.0	85.9	49
9	2	0.5	χ 210	2	T	1.5	MT	NA	100	77	99.9	11.9	49.8	100	99.5	99.5	66
10	2	0.5	χ 210	2	T	1.5	MT	NA	200	92	100.0	9.3	69.5	100	100.0	100.0	84
11	1	2	LGe	2	T	2	ARf	3	15	11	29.6	42.8	3.3	49	36.9	36.5	57
12	1	2	LG	2	T	2	AR	3	30	50	87.4	22.8	34.0	84	81.6	81.5	51
13	1	2	LG	2	T	2	AR	3	50	57	97.8	8.9	75.9	98	97.2	97.1	45
14	1	2	LG	2	T	2	AR	3	100	64	100.0	14.2	90.2	100	100.0	100.0	51
15	1	2	LG	2	T	2	AR	3	200	70	100.0	9.6	98.9	100	100.0	100.0	52
16	− 1	3	t4	1	T	2	AR	− 5	15	25	29.0	5.8	19.5	34	25.0	24.8	31
17	− 1	3	t4	1	T	2	AR	− 5	30	43	59.6	7.2	29.0	61	55.6	55.4	35
18	− 1	3	t4	1	T	2	AR	− 5	50	59	93.0	12.6	51.0	92	90.1	90.0	47
19	− 1	3	t4	1	T	2	AR	− 5	100	68	99.7	8.9	89.6	100	99.6	99.6	53
20	− 1	3	t4	1	T	2	AR	− 5	200	74	100.0	9.6	95.6	100	100.0	100.0	59

^aError Distribution

^bGrouping

^cTime Trend

^dmartingale

^eLogistic

^fAR(1)

The first type of violation examined was a non-normal error distribution. We considered several types of error distributions, broadly classified into symmetric and skewed distributions. The first four cases in Table 2 present the simulated powers of the tests when the error distribution is symmetric (t-distributed with 5 df), and is right-skewed (a centered χ² with 5 df). The technical report includes the results for other error distributions such as the logistic, double exponential, t, and χ² with df other than 5. The global test is quite good relative to the best directional test based on the four component statistics, with its power not significantly degraded by combining the four statistics, and sometimes exceeding those based on the best directional test. The best directional test statistic is ${\widehat{S}}_2^2,$ which is a kurtosis-type statistic. Notice that the ${\widehat{S}}_3^2$ test does not have any power for detecting this error distribution mis-specification. As expected, when the df of the t-distribution increases, the power of the tests decreases. The powers of the MaxTest in (20) and BonfTest in (19) are almost identical, and for these symmetric distributions are lower than the ${\widehat{G}}_4^2$-based test. Additional runs where the error distribution is a normal contaminated with a t₁- or t₃-distribution were also performed. For contaminating proportions of .1 and .3, the results indicate that the ${\widehat{G}}_4^2$-based test possesses good detection abilities for this violation, and has slightly higher power than the MaxTest and BonfTest. For the t-distributed error distribution, the performance of the BoxCox test was poor relative to the ${\widehat{G}}_4^2$-test when the sample size is large.

When the errors have shifted chi-squared distributions, the global test performs acceptably well relative to the best test among the four directional tests, with the powers slightly degraded due to combining the four directional tests, some of which do not have good power against this assumptional departure. The best directional test statistic is ${\widehat{S}}_1^2,$ which is the skewness-type statistic. ${\widehat{S}}_3^2$ does not have any detection power for this alternative. When the df increases, the power diminishes, because the chi-square distribution approaches the normal distribution. The MaxTestand the BonfTest perform just slightly better than the ${\widehat{G}}_4^2$-based test for some values of n. On the other hand, the BoxCox test performed very well for this right-skewed error distribution. Its power is significantly higher than the other tests for sample size n = 30. This superior performance of the BoxCox test could be intuitively explained by the fact that the transformation is especially appropriate for non-normal and non-symmetric error distributions. Interestingly, this non-symmetric error distribution is the only instance in Table 2 in which the BoxCox test totally dominated the other tests.

The next set of simulation runs concerns the situation where (A2) is violated, so that the conditional variances of the Y_i’s are not equal. Two models were considered for this purpose. The first model has variances that depend on the covariate values. Specifically, the true model is

$$Y_i=x_i+x_i^{\alpha }{\epsilon }_i,\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }i=1,2,\dots ,n,$$ (28)

where ε_i’s are i.i.d. from N(0; 1). The simulated powers for α = 2 are summarized in the fifth and sixth rows of Table 2. The best directional test for this departure is the ${\widehat{S}}_2^2$-test, with the global test performing best among all the tests. Again, the test based on ${\widehat{S}}_3^2$ has very low power for this heteroscedastic model, though it is not totally devoid of detection power when n = 200. The second model for heteroscedastic variances is of form

$$Y_i=\{\begin{array}{ccc}{x}_i+{\sigma }_1{\epsilon }_i& \text{for}& i\le n/2\\ x_i+{\sigma }_2{\epsilon }_i& \text{for}& i>n/2\end{array},$$ (29)

with ε_i’s also i.i.d. from N(0; 1). The seventh and eighth rows of Table 2 present the simulated powers for data arising from the model with σ₁ = 1, σ₂. = 2. The best directional test in this situation is based on ${\widehat{S}}_4^2,$ followed by the test based on ${\widehat{S}}_2^2.$ The global test also possesses acceptable power, but has lower power compared to that of ${\widehat{S}}_4^2.$ The powers of the MaxTest and BonfTest are just slightly lower than the ${\widehat{G}}_4^2$-test.

The next set of runs were for mis-specified link functions, that is, when (A1) is violated. The data analyzed were generated according to the model

$$Y_i=x_i+{\beta }_2{x}_i^{\gamma }+{\epsilon }_i,\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }i=1,2,\dots ,n,$$ (30)

with ε_i’s i.i.d. from N(0; 1). The ninth to twelfth rows of Table 2 provide the simulated powers of the tests for two different sets of (β₂, γ). Interestingly, the directional tests based on ${\widehat{S}}_1^2,{\widehat{S}}_2^2,\mathrm{\hspace{0.17em} }\text{and\hspace{0.28em}}{\widehat{S}}_4^2$ are not at all sensitive to this violation. The best directional test is based on ${\widehat{S}}_3^2.$ The global test also has detection power towards this mis-specification, although its power is quite degraded relative to that of ${\widehat{S}}_3^2,$ possibly because the other three tests have no power against this alternative. Furthermore, the MaxTest and BonfTest have better powers than the ${\widehat{G}}_4^2$-test. When β₂ = 1 and γ ∈ {.5,2}, the powers of the tests are very low. However, this should not be perceived as a defect of the tests because this is a consequence of the fact that for these parameter sets, the signal-to-noise ratios (SNR) are very low. This SNR is measured via

$$\text{SNR}=\frac{\mathbf{E}\{\text{MSE}(\text{Fitted}) | \text{True}\}-\mathbf{E}\{\text{MSE}(\text{True}) | \text{True}\}}{\mathbf{E}\{\text{MSE}(\text{True}) | \text{True}\}},$$ (31)

with E{MSE(Model A)|Model B} being the expectation of the mean-squared error when Model A is fitted with the expectation evaluated with respect to Model B. Thus, E{MSE(True)|True} = σ². It is straighforward to show that for the simulation model in (30), the SNR satisfies, for large n:

$$\text{SNR}({\beta }_2,\gamma ,\sigma )\approx {\left(\frac{{\beta }_2}{\sigma }\right)}^2\frac{{\gamma }^2}{{(\gamma +1)}^2(2\gamma +1)}\left\{1-\frac{3(2\gamma +1)}{{(\gamma +2)}^2}\right\}.$$

For the values of (β₂, γ, σ) utilized in the simulation studies, $\text{SNR}(1,.5,1)\approx \frac{1}{450}$, $\mathrm{\hspace{0.17em} }\text{SNR}(1,2,1)\approx \frac{1}{180}$, $\mathrm{\hspace{0.17em} }\text{SNR}(3,.5,1)\approx \frac{1}{50}$, $\mathrm{\hspace{0.17em} }\text{SNR}(3,2,1)\approx \frac{1}{20}$, $\mathrm{\hspace{0.17em} }\text{SNR}(5,.5,1)\approx \frac{1}{18}$, and $(5,2,1)\approx \frac{5}{36}.$ These values explain the ordering of the simulated powers for the ${\widehat{S}}_3^2$-based test. Note in particular that $\text{SNR}(5,.5,1)\approx \frac{1}{18}$ is only slightly larger than $\text{SNR}(3,2,1)\approx \frac{1}{20},$ and this is reflected by the small differences in the observed powers for the ${\widehat{S}}_3^2$-based test for these two sets of values of (β₂,γ).

For the simulation runs concerning violations of assumption (A3), we considered the two models described earlier for generating martingale-type and AR-type structures. In the simulation, we performed runs for ρ ∈ {.5, 1, 2, 5, 10}. The last four rows of Table 2 present the simulated powers of the tests under these dependent error models, with ρ = 5 for the AR-type structure. For the martingale structure, the best directional test is based on ${\widehat{S}}_4^2,$ with the global test surpassing the performance of this best directional test for large n, and also being slightly better than the MaxTest and BonfTest. For the AR(1) structure, the best is also the ${\widehat{S}}_4^2$-test, with the global test’s power also very good, and again the power of the global test is best for large n. The tests based on ${\widehat{S}}_1^2$ and ${\widehat{S}}_2^2$ also have some detection abilities for this violation, but are not competitive with the ${\widehat{S}}_4^2$-based test or the global test. The test based on ${\widehat{S}}_3^2$ possesses no ability to detect this particular type of violation. The global test performs slightly better than the MaxTest and BonfTest for the AR(1) error structure.

Finally, we consider the situation where several of the assumptions are violated simultaneously. We expect that the global test is ideally suited for this situation. Table 3 presents the achieved powers of the tests for four sets of simulation parameters where all four assumptions (A1)-(A4) are violated as in (25). All four directional tests have detection abilities. The performance of the global test is extremely commendable, as its power is generally higher than any of the directional tests as well as the MaxTest and the BonfTest. It is interesting to observe that the BoxCox test sometimes has higher power than the ${\widehat{G}}_4^2$-test for small n, but the rate of increase of its power as n increases is relatively slow compared to the latter test. It is conceivable that the BoxCox test has higher power over the ${\widehat{G}}_4^2$-test when n is small simply because the latter test is highly conservative for small n.

6 Illustrative Examples

Example 1: The first illustration pertains to car mileage data gathered by the first author while commuting from Ann Arbor, Michigan to Bowling Green, Ohio during the period October 20, 1996 through January 27, 1999. There were 205 observations corresponding to gas fill-ups for the following variables: Date, the date of the gas fill-up; NumGallons (denoted Y ), the number of gallons of regular unleaded gasoline pumped into the car; MilesLastFill (denoted X₁), the distance travelled since the last fill-up; NumDaysBetw (denoted X₂), the number of days since last fill-up; and AveMilesGal, the miles per gallon between gas fill-ups. This data set is at the URL: http://www.stat.sc.edu/~pena/DataSets/CarMileage.txt. We fit the multiple linear regression model Y_i = β₀+β₁X_1i+β₂X_2i+σε_i, i= 1,2,…,205, where ε_is are i.i.d. N(0,1) variates. Scatterplots of Y versus X₁ and X₂ are provided in the first two plots of Figure 1, respectively. The Pearson's correlation coefficient between Y and X₁ is .653, between Y and X₂ is − .002, and between X₁ and X₂ is − .378. Other summary statistics for this data set are provided in Table 4.

Figure 1.

Relevant plots for the analysis of the car mileage data. The first plot is a scatterplot of Y versus X₁; the second is that between Y and X₂. The next four scatterplots are $p[\cdot ]\text{\hspace{0.28em}versus\hspace{0.28em}}\mathrm{\Delta }{\widehat{G}}_4^2[\cdot ]$ for successive re-analyses with some observations excluded.

Table 4.

Summary statistics for the car mileage data set, where FQ and TQ are the first and third quartiles, respectively

Statistic	X2 = NumDays Between	Y=Number Gallons	X1 =MilesLast Fill	AveMiles Gal
Min	1	8.199	207.0	18.37
FQ	3	12.459	362.3	28.28
Med	4	12.909	379.3	29.46
TQ	5	13.344	394.5	30.58
Max	26	14.209	447.0	33.47
Mean	4.1	12.823	375.5	29.29
SD	2.7	0.778	31.7	1.89

Table 5 contains the results of the analysis, including the F-value, estimates of regression coefficients, $\widehat{\sigma }$, and the coefficient of determination. The row labeled ‘None’ pertains to the analysis where all 205 observations were used. If the model assumptions are satisfied, the regression coefficients β₁ and β₂ are both found to be significantly different from zero. However, ${\widehat{G}}_4^2=24.26$ has a p-value of zero, and those associated with ${\widehat{S}}_2^2$ and ${\widehat{S}}_3^2$ are also very small, indicating violation of model assumptions.

Table 5.

Results of analyses of the car mileage data using the complete data, and with excluded observations. The F-column contains the F-statistic value for testing that β₁ = β₂ = 0, while the R²-column contains the coefficient of determination.

	Test Statistic / (p-value)						Estimates / (SE)
Excluded Obs #	${\widehat{G}}_4^2$	${\widehat{S}}_1^2$	${\widehat{S}}_2^2$	${\widehat{S}}_3^2$	${\widehat{S}}_4^2$	$\widehat{F}$	b0	b1	b2	$\widehat{\sigma }$	R2
None	24.26 (≈0)	0.03 (.86)	17.24 (≈ 0)	6.90 (.01)	0.08 (.78)	99.34 (0)	5.48 (.53)	.019 (.001)	.08 (.02)	.56	50%
19, 56, 67, 146, 200	13.09 (.01)	5.07 (.02)	0.08 (.78)	7.81 (.01)	0.14 (.71)	105.3 (0)	5.00 (.54)	.019 (.001)	.15 (.02)	.47	52%
19, 56, 67, 146, 164, 200	8.99 (.06)	3.15 (.08)	0.09 (.76)	4.47 (.03)	1.28 (.26)	128.9 (0)	4.64 (.52)	.019 (.001)	.24 (.03)	.45	57%
19, 56, 58, 67, 146, 164, 200	6.80 (.15)	2.81 (.09)	0.08 (.78)	2.84 (.09)	1.07 (.30)	133.2 (0)	4.48 (.52)	.020 (.001)	.24 (.03)	.44	58%

As advocated in Section 4 we examine for unusual observations. The third plot in Figure 1, which is a scatterplot of $\mathrm{\Delta }{\widehat{G}}_4^2[\cdot ]\mathrm{\hspace{0.17em} }\text{versus\hspace{0.28em}}p[\cdot ],$ indicates that the 19th, 56th, 67th, 146th, and 200th observations are highly unusual. Details of these observations and others that were excluded in further analyses are in Table 6. The dates of these observations reveal their unusual nature. The 19th observation was obtained on Christmas Eve just before a long trip. In contrast to usual practice, though the gas tank was still almost half-full, a decision was made to fully fill it, thus lowering fuel efficiency; the 146th observation was obtained during a long trip which mostly covered interstate highway driving; and the 200th observation encompassed a period when the car was driven during a blizzard and was stuck in deep snow, explaining the low fuel efficiency. The 56th and 67th observations showed up among these unusual values primarily because of their X₂-values of 26 (on vacation) and 22 (in repair shop) days, respectively.

Table 6.

Details of the observations that were excluded from the analyses.

Obs # Excluded	X2 =NumDays Between	Y=Number Gallons	X1 =MilesLast Fill	AveMiles Gal	Date Fill-Up
19	4	8.199	207.0	25.25	12/24/96
56	26	12.058	316.2	26.22	6/10/97
58	4	13.043	430.8	33.03	6/19/97
67	22	12.417	346.9	27.94	8/15/97
146	4	9.809	311.8	31.79	5/24/98
164	21	11.937	278.8	23.36	8/17/98
200	10	13.138	241.4	18.37	1/8/99

We refitted the linear model with these five observations excluded from the analysis. The results are summarized in the third row in Table 5, which still indicates violations of model assumptions. More importantly, the fourth plot in Figure 1 reveals the 164th observation (in the original data set) to be highly unusual. Similar to the 56th and 67th observations, it has a large value of X₂ (equal to 21 due to vacation). Observe also the sensitivity of the directional statistics to the presence of unusual observations. In the first analysis, the p-values of ${\widehat{S}}_1^2$ and ${\widehat{S}}_2^2$ were high and low, respectively, but after the exclusion of the unusual observations, this pattern reversed.

Further excluding this 164th observation (see fourth row of Table 5) now yields a global statistic of ${\widehat{G}}_4^2=8.99$ with p-value of .06 indicating that model assumptions appear viable, though the statistic ${\widehat{S}}_3^2$ has p-value of .03. A glimpse at the $\mathrm{\Delta }{\widehat{G}}_4^2[\cdot ]\mathrm{\hspace{0.17em} }\text{versus\hspace{0.28em}}p[\cdot ]$ scatterplot, the fifth plot in Figure 1, reveals the 58th observation (in the original data set) to be unusual, though there was no obvious explanation for this being so, in contrast to the other six values. Thus, though it may not be fully justifiable, in the final analysis we also excluded the 58th observation. The results are provided in the last row of Table 5, which show all validation test statistics have p-values larger than .05, indicating that after the exclusion of the seven unusual observations pinpointed by the deletion statistics, the linear model assumptions appear acceptable. Observe also that the scatterplot of $\mathrm{\Delta }{\widehat{G}}_4^2[\cdot ]\mathrm{\hspace{0.17em} }\text{versus\hspace{0.28em}}p[\cdot ]$, the sixth plot in Figure 1, no longer shows any unusual observations. However, note that the p-values of ${\widehat{S}}_1^2$ and ${\widehat{S}}_3^2$ are both between .05 and .10, which may be indicating mild violations of the normality and link function assumptions. For these reduced data, the correlation coeffient between Y and X₁ is .618; between Y and X₂ is .219, a significant increase from the original correlation of − .002 indicating the impact of the unusual observations; and between X₁ and X₂ is − .323.

Example 2: The second example is a multiple regression analysis of water salinity data (see Table 3 in their paper) which they used to illustrate robust regression techniques, and which was also used for illustrative purposes in Atkinson (1985, pp. 48–52). The data set consisted of 28 observations on the variables Salinity, the water salinity at the specified time period; LagSalinity, the water salinity lagged two weeks; Trend, representing one of the six biweekly periods in March to May; and WaterFlow, the river discharge. The response variable is Salinity, while the predictors are LagSalinity, Trend, and WaterFlow. The first part of the analyses fitted the multiple regression model

$$\text{Salinity}={\beta }_0+{\beta }_1(\text{LagSalinity})+{\beta }_2(\text{Trend})+{\beta }_3(\text{WaterFlow})+\sigma \epsilon .$$ (32)

The fitted model had b₀ = 9.590^, b₁ = .777, b₂ = − .026, and b₃ = − .295. The coefficients β₀, β₁ and β₃ were significantly different from zero. The multiple R² was 82.6%. When the model validation procedures are applied, we obtained ${\widehat{G}}_4^2=.16(p=.997)$, ${\widehat{S}}_1^2=.02(p=.87)$, ${\widehat{S}}_2^2=.005(p=.95)$, ${\widehat{S}}_3^2=7.63\times {10}^{-6}(p=.998)$, and ${\widehat{S}}_4^2=.128(p=.72).$ The global test thus indicates that model assumptions are acceptable though, as noted by a reviewer, the nearness of these p-values to one also raises a ‘too good a fit’ concern.

An examination of the plot of $\mathrm{\Delta }{G}_4^2[\cdot ]\text{\hspace{0.28em}versus\hspace{0.28em}}p[\cdot ]$, the first plot in Figure 2, reveals that the 16th observation is highly unusual. In Atkinson (1985) the unusual nature of this observation was revealed using a half-normal plot of statistic. It was also pointed out that the value of WaterFlow for this observation is the cause for its being unusual and highly leveraged. The fact that the global test did not conclude violation of model assumptions, even with this very unusual observation, may cast doubt on the effectiveness of the validation procedure. Further analyses of the data, however, reveal the reason for this behavior. LagSalinity is an excellent linear predictor of Salinity, with the correlation coefficient between them equal to .872; whereas WaterFlow is not highly correlated with Salinity, with their correlation coeffient equal to − .477. In addition, the correlation coefficient between LagSalinity and WaterFlow is − .261. Consequently, when the variables LagSalinity and WaterFlow are included in the regression model, the effect of WaterFlow is diminished by the presence of LagSalinity. When LagSalinity is used in the linear regression model without WaterFlow, then ${\widehat{G}}_4^2=2.07(p=.72),$ indicating that model assumptions are not violated. However, when WaterFlow is used as sole predictor variable for Salinity, then ${\widehat{G}}_4^2=10.33(p=.035)$ and ${\widehat{S}}_3^2=7.50(p=.006)$ indicating violations of model assumptions and, in addition, the 16th observation is highly unusual. Therefore, when LagSalinity and WaterFlow are both in the linear regression model and the original data is used, then model assumptions are in fact satisfied. Thus the conclusion from the global test that the assumptions are satisfied, even with the unusual observation, is not cause for alarm regarding the effectiveness of the proposed validation procedure.

Figure 2.

Relevant plots for the salinity data example. The first, second, and sixth plots are the scatterplots of $\mathrm{\Delta }{\widehat{G}}_4^2[i]\text{\hspace{0.28em}versus\hspace{0.28em}}p[i]$for the original data, corrected data, and for the corrected data with a quadratic WaterFlow term in the model, respectively. The third and fourth plot depict the resulting values of ${\widehat{G}}_4^2$ and the correlation between Salinity and WaterFlow for different replacement values for WaterFlow in the 16th observation. The fifth is a scatterplot of Salinity and WaterFlow using Atkinson's replacement value of 23.443.

We follow Atkinson (1985, p. 49), by supposing that the value of 33.443 for WaterFlow for this 16th observation was a misprint of 23.443. We re-fitted the model in (32) but using 23.443 in place of 33.443. The resulting analysis yielded b₀ = 18.39, b₁ = .70, b₂ = − .15, and b₃ = − .63, with β₀, β₁, β₃ significantly different from zero. The multiple R² was 89.28%. Applying the model validation procedures, we obtained ${\widehat{G}}_4^2=6.696(p=.15)$, ${\widehat{S}}_1^2=1.41(p=.23)$, ${\widehat{S}}_2^2=.03(p=.86)$, ${\widehat{S}}_3^2=4.21(p=.04)$, and ${\widehat{S}}_4^2=1.04(p=.31).$ Though the global statistic has p-value exceeding 10%, the p-value for ${\widehat{S}}_3^2$ is .04, which seems to indicate a mild problem in the link function. The scatterplot of $\mathrm{\Delta }{\widehat{G}}_4^2[\cdot ]\text{\hspace{0.28em}versus\hspace{0.28em}}p[\cdot ]$, the second plot in Figure 2, indicates no unusual observations, except possibly for the 5th observation.

To gain further insight about these data, we examined the impact of different replacement values for WaterFlow in the 16th observation, specifically on the resulting value of ${\widehat{G}}_4^2.$ The third plot in Figure 2 presents the values of ${\widehat{G}}_4^2$ for different replacement values for WaterFlow. Note that the ${\widehat{G}}_4^2$-value is largest, hence most indicative of violations of model assumptions, when the replacement value is about 23.9, which is close to the value of 23.443 used by Atkinson. The fourth plot in Figure 2 presents the values of the correlation coefficients between Salinity and WaterFlow for different replacement values, and from this plot the largest absolute correlation is at a value very close to Atkinson’s replacement of 23.443. Using this value the correlation coefficient between Salinity and WaterFlow is − .646. Because for replacement values in the interval from 23.3 to 23.9, the correlation coefficients between Salinity and WaterFlow become largest, the impact of WaterFlow in the linear regression model when LagSalinity is also included in the model is not easily diminished, in contrast when using the original data. Consequently, at such replacement values, potential model violations especially with regards to the linearity assumption for WaterFlow materialize. The fifth plot in Figure 2, a scatterplot between Salinity and WaterFlow when using the replacement value of 23.443, partly reveals a curvilinear relationship between Salinity and WaterFlow.

Recognizing the possible problem with the link function, we follow Atkinson’s (1985, p. 51) suggestion of incorporating a quadratic term of WaterFlow and we fitted the model

$$\text{Salinity}={\beta }_0+{\beta }_1(\text{LagSalinity})+{\beta }_2(\text{Trend})+{\beta }_3(\text{WaterFlow})+{\beta }_4{(\text{WaterFlow})}^2+\sigma \epsilon .$$ (33)

The resulting estimates are b₀ = 67.49, b₁ = .68, b₂ = − .25, b₃ = − 4.57, and b₄ = :08, and the multiple R² was 91.65%. Only β₂ did not turn out to be significantly different from zero with p-value of .053. The model validation statistics are ${\widehat{G}}_4^2=1.74(p=.78)$, ${\widehat{S}}_1^2=1.20(p=.22)$, ${\widehat{S}}_2^2=.022(p=.88)$, ${\widehat{S}}_3^2=.176(p=.67)$, and ${\widehat{S}}_4^2=.348(p=.55).$ The Scatterplot of $\mathrm{\Delta }{\widehat{G}}_4^2[\cdot ]\text{\hspace{0.28em}versus\hspace{0.28em}}p[\cdot ]$, which is the sixth plot in figure 2, no longer shows unusual observation

7 Concluding Remarks

In this paper a global test procedure for validating the four assumptions of the linear model is proposed. The global test statistic, which is a function of the model residuals, is formed from four asymptotically independent statistics, each having the potential of detecting a particular violation. The level and powper properties of the tests were examined through simulation studies, which indicate that the global and directional tests possess the ability to detect different types of violations of the model assumptions. As such, the tests provide a formal method for globally assessing the validity of model assumptions. Deletion statistics and graphical methods based on the global statistic can be used to identify unusual observations. The proposed formal procedure may help in eliminating, or at least reducing, the oftentimes subjective assessment of the validity of model assumptions when using existing graphical techniques.

Other issues remain to be addressed. First, there is the problem of developing an adaptive method. From the simulation results, the power of the global test is generally lower than the best directional test when only one assumption is violated. Some of the directional tests have no power for detecting certain types of alternatives, hence they tend to dilute the power when included in this global statistic. We conjecture that it will be possible to have the data dictate which among the four directional test statistics to combine to form a global test statistic, and by doing so we expect that the resulting adaptive global test may acquire increased power. Several approaches to determining which directional test statistics to combine present themselves, such as those using information measures like the Schwartz (1978) Bayesian information criterion (BIC) or the Akaike information criterion (AIC) (Akaike 1973). Second, the chi-square approximation is not satisfactory for small sample sizes, though we point out that except for the ${\widehat{S}}_3^2$-based statistic, the approximation leads to conservative tests. Two possible ways of alleviating this problem are to utilize empirical estimates of the covariance matrices instead of using the theoretical matrices, or to use computationally-intensive methods to determine the critical regions of the tests. Third, the main motivation for choosing the smoothing functions in the density embedding is to recover some commonly used one-dimensional test statistics, with the aim of formally combining them into one global test statistic. This has been achieved in this paper. However, one is not limited in choosing the functions that enter into the embedding. Finally, to make these linear model validation procedures accessible to practitioners, we plan to provide the procedures through a computer package in the R Library.