On a generalization of the test of endogeneity in a two stage least squares estimation

Abstract

In situations that the predictors are correlated with the error term, we propose a bridge estimator in the two-stage least squares estimation. We apply this estimator to overcome the multicollinearity and sparsity of the explanatory variables, when the endogeneity problem is present.The proposed estimator was applied to modify the Durbin-Wu-Hausman (DWH) test of endogeneity in the presence of multicollinearity. To compare our modified test with the existing DWH for detection of an endogenous problem in multi-collinear data, some numerical assessments are carried out. The numerical results showed that the proposed estimators and the suggested test perform better for the multi-collinear data. Finally, a genetical data set is applied for illustration the our results by estimating the coefficients parameters in the presence of endogeneity and multicollinearity.

1. Introduction

Under some certain regularity conditions, the Ordinary Least Squares (OLS) estimator produces the unbiased and consistent estimators. An important assumption is that the predictors in the model are uncorrelated with the error term. When this condition is not satisfied, then the OLS estimator yields biased and inconsistent estimates and referred to as the endogeneity problem (Ebbes [10]; Bowden and Turkington [3]). An approach to overcome the endogeneity problem is applying some instrumental variable estimators or two-stage least squares which assumes that there is a secondary predictor, termed as instrumental variable, that is correlated to the predictor but not with the error term.

Another important assumption in the classical regression model is that the predictors are uncorrelated. When this assumption is violated, the explanatory variables are nearly dependent which refers as multicollinearity problem and the OLS yields poor estimates. In order to combat this problem, several approaches have been considered and these include the ridge regression estimator introduced by Hoerl and Kennard [17], as one. They considered a shrinkage method to overcome the problem of multicollinearity in the estimation of regression parameters; see ([12,21,28,32] and [22]).

Many authors described the connection between endogeneity and multicollinearity in the literature. Recently, Philips and Evans [26] discussed on approximating and reducing bias in an estimator method called Two Stage Least Squares (2SLS) estimation while Hansen and Kuzbur [15] proposed a jackknife instrument variable estimator with regularization at each jackknife iteration that helps alleviate the bias. Interested readers may refer to [5,6,8] for other developments in this regard.

On the other hand, in the context of variable selection, it is well known that the least absolute shrinkage and selection operator (LASSO) of Tibshirani [29] produces sparse solution. In the presence of endogenous regressors, the direct using of the Lasso selector fails as sparsity of coefficients in regression model does not correspond to sparsity of linear projection coefficients [2].

This paper uses the bridge regression method to combat multicollinearity and sparsity in estimation of the instrumental variable models.

It is known that finding valid instrumentals is not easy. Moreover, using instrumental variables estimators for eliminating the inconsistency of the OLS causes the loss of efficiency. Thus, researchers must find a good way of determining when they must apply the instrument variable estimator, and when they do not have to. Wu [31], Hausman [16] and Durbin [9] proposed some tests for detecting endogenous regressors in the instrumental analysis. The secondary purpose of this paper is to modify the Hausman test statistic in multi-collinear models. The main and preliminary results are given in the next section. In section 3, we propose our test statistic that is used for tesing of endogeneity under the multicollinearity situation and provide some asymptotic results. Section 4 is devoted to a simulation study to investigate the performance of the introduced test statistic. Finally, an application of our results in the analysis of a genetic data is presented in Section 5.

2. Main results

Consider the multiple linear regression model

$$\begin{array}{rl}\mathit{y}& =\mathit{X}\mathit{\beta }+\mathit{ϵ}\end{array}$$ (1)

$$\begin{array}{rl}{\mathit{x}}_{\mathit{i}}& ={\mathit{Z}\mathit{\eta }}_i+{\mathit{u}}_{\mathit{i}},i=1,2,\dots ,p,\end{array}$$ (2)

where $\mathit{y}=(y_1,\dots ,y_n{)}^{\mathrm{\top }}$ is the vector of response variables, $\mathit{X}=({\mathit{x}}_{\mathbf{1}},\dots ,{\mathit{x}}_p)$ is an $n\times p$ matrix of endogenous explicative variables, $\mathit{\beta }=({\beta }_1,\dots ,{\beta }_p{)}^{\mathrm{\top }}$ is the vector of parameters, and $\mathit{ϵ}=({ϵ}_1,\dots ,{ϵ}_n{)}^{\mathrm{\top }}$ is the vector of random disturbances with $E(\mathit{ϵ})=\mathbf{0}$ and $E(\mathit{ϵ}{\mathit{ϵ}}^{\mathrm{\top }})={\sigma }_{\epsilon }^2{\mathit{I}}_n$. It is further assumed that $\mathit{ϵ}$ follows the n-variate normal distribution. Moreover, $\mathit{Z}$ is an $n\times k$ matrix of the exogenous instrumental variables, ${\mathit{\eta }}_i$ is its $k\times 1$ corresponding coefficient vector, and ${\mathit{u}}_{\mathit{i}},i=1,2,\dots ,n,$ is an $n\times 1$ vector of $N(\mathbf{0},{\sigma }_u^2{\mathit{I}}_n)$ errors. It is also assumed that the covariance matrix of $({\mathit{\epsilon }}_{\mathit{i}},{\mathit{u}}_{\mathit{i}})$ is $\mathbf{\Sigma }$.It is known that the two stage least squares (2SLS) estimator of $\mathit{\beta }$ is obtained in two stages as follows: in the first stage, we have to estimate the regression coefficient ${\mathit{\eta }}_i$ using the OLS approach as the below:

$${\hat{\mathit{\eta }}}_i={\left({\mathit{Z}}^{\mathrm{\top }}\mathit{Z}\right)}^{-1}{\mathit{Z}}^{\mathrm{\top }}{\mathit{x}}_i.$$

Then, using the predicted values ${\hat{\mathit{x}}}_i=\mathit{Z}{\hat{\mathit{\eta }}}_{\mathit{i}}$ in the second stage, $\mathit{y}$ is regressed on $\hat{\mathit{X}}=({\hat{\mathit{x}}}_1,\dots ,{\hat{\mathit{x}}}_p)$ and the estimator of coefficient vector $\mathit{\beta }$ is of the form

$${\hat{\mathit{\beta }}}_{2SLS}={\left({\hat{\mathit{X}}}^{\mathrm{\top }}\hat{\mathit{X}}\right)}^{-1}{\hat{\mathit{X}}}^{\mathrm{\top }}\mathit{y}=({\mathit{X}}^{\mathrm{\top }}\mathit{P}\mathit{X}{)}^{-1}{\mathit{X}}^{\mathrm{\top }}\mathit{P}\mathit{y},$$ (3)

where $\mathit{P}=\mathit{Z}({\mathit{Z}}^{\mathrm{\top }}\mathit{Z}{)}^{-1}{\mathit{Z}}^{\mathrm{\top }}$ is a symmetric and idempotent matrix.

Suppose now in the model (2) there are some ill-conditioned in the matrix ${\mathit{X}}^{\mathrm{\top }}\mathit{X}$. The most ill-conditioned situations are multicollinearity and sparsity. In such situation, we use the bridge penalty in the second stage of 2SLS method to deal with these problems. The first stage estimation is the same as the traditional 2SLS, but in the second stage of estimation we apply the bridge estimator in the regression equation $\mathit{y}=\mathit{P}\mathit{X}\mathit{\beta }+\mathit{\epsilon }$. Then by shrinking the estimation under $\sum _{j=1}^p|{\beta }_j{|}^{\gamma }<c$ for some $\gamma ,c>0$, the bridge 2SLS estimator of $\mathit{\beta }$ after solving the following optimization problem:

$${\hat{\mathit{\beta }}}_{2\mathrm{SLS}-\mathrm{B}}=\arg \underset{\mathit{\beta }\in {\mathbb{R}}^p}{min}\{(\mathit{y}-\hat{\mathit{X}}\mathit{\beta }{)}^{\mathrm{\top }}(\mathit{y}-\hat{\mathit{X}}\mathit{\beta })+{\lambda }_n{\mathbf{1}}_p^{\mathrm{\top }}|\mathit{\beta }{|}^{\gamma }\},$$

where ${\lambda }_n\ge 0$ is the tuning parameter, $|\mathit{\beta }{|}^{\gamma }=(|{\beta }_1{|}^{\gamma },\dots ,|{\beta }_p{|}^{\gamma }{)}^{\mathrm{\top }}$, and ${\mathbf{1}}_p$ is a p-tuple of 1's. Using local quadratic approximation (LQA) of Fan and Li [11] the penalty term can be approximated around a local vector ${\mathit{\beta }}^o$ and the bridge estimator has the following closed form

$${\hat{\mathit{\beta }}}_{2\mathrm{SLS}-\mathrm{B}}={[\mathit{C}+{\mathbf{\Sigma }}_{\lambda }({\mathit{\beta }}^o)]}^{-1}{\hat{\mathit{X}}}^{\mathrm{\top }}\mathit{y},\mathit{C}={\hat{\mathit{X}}}^{\mathrm{\top }}\hat{\mathit{X}},$$ (4)

where ${\mathbf{\Sigma }}_{\lambda }({\mathit{\beta }}^o)=\frac{\lambda \gamma }{2}\mathrm{Diag}(|{\beta }_1^o{|}^{\gamma -2},\dots ,|{\beta }_p^o{|}^{\gamma -2})$. For more details one may refer to [24],[33], and [25].Obviously, when $\gamma =1$, the LASSO of [30] can be approximated by

$$\begin{array}{rl}{\hat{\mathit{\beta }}}_{2\mathrm{SLS}-\mathrm{L}}& =\arg \underset{\mathit{\beta }\in {\mathbb{R}}^p}{min}\{(\mathit{y}-\hat{\mathit{X}}\mathit{\beta }{)}^{\mathrm{\top }}(\mathit{y}-\hat{\mathit{X}}\mathit{\beta })+{\lambda }_n{\mathbf{1}}_p^{\mathrm{\top }}|\mathit{\beta }|\}\\ & ={[\mathit{C}+\frac{\lambda }{2}\mathrm{Diag}({\left|{\beta }_1^o\right|}^{-1},\dots ,{\left|{\beta }_p^o\right|}^{-1})]}^{-1}{\hat{\mathit{X}}}^{\mathrm{\top }}\mathit{y}.\end{array}$$

However, the ridge estimator is of the following form for $\gamma =2$, without approximation

$$\begin{array}{rl}{\hat{\mathit{\beta }}}_{2\mathrm{SLS}-\mathrm{R}}& =\arg \underset{\mathit{\beta }\in {\mathbb{R}}^p}{min}\{(\mathit{y}-\hat{\mathit{X}}\mathit{\beta }{)}^{\mathrm{\top }}(\mathit{y}-\hat{\mathit{X}}\mathit{\beta })+{\lambda }_n{\mathbf{1}}_p^T{\left|\mathit{\beta }\right|}^2\}\end{array}$$ (5)

$$\begin{array}{rl}& =({\mathit{X}}^{\mathrm{\top }}\mathit{P}\mathit{X}+k\mathit{I}{)}^{-1}{\mathit{X}}^{\mathrm{\top }}\mathit{P}\mathit{y}.\end{array}$$ (6)

In order to study the properties of these estimators, The following regularity conditions are assumed:

(A.1) The $p\times p$ and $k\times k$ matrices ${\mathbf{\Sigma }}_{\mathit{x}\mathit{x}}=E({\mathit{x}}_{\mathit{i}}{\mathit{x}}_{\mathit{i}}^{\mathrm{\top }})$ and ${\mathbf{\Sigma }}_{\mathit{z}\mathit{z}}=E({\mathit{z}}_{\mathit{i}}{\mathit{z}}_{\mathit{i}}^{\mathrm{\top }})$ are nonsingular and finite.

(A.2) $E({\mathit{x}}_{\mathit{i}}{\epsilon }_i)\ne \mathbf{0}$, $E({\mathit{z}}_{\mathit{i}}{\epsilon }_i)=\mathbf{0}$ and the $k\times p$ matrix ${\mathbf{\Sigma }}_{\mathit{x}\mathit{z}}=E({\mathit{z}}_{\mathit{i}}{\mathit{x}}_{\mathit{i}}^{\mathrm{\top }})$ is finite and of full rank.

(A.3) By the law of large numbers, there exist nonnegative definite matrices ${\mathbf{\Sigma }}_{\mathit{x}\mathit{x}}$, ${\mathbf{\Sigma }}_{\mathit{z}\mathit{z}}$ and ${\mathbf{\Sigma }}_{\mathit{x}\mathit{z}}$ such that

$$\frac{1}{n}\sum _{i=1}^n{\mathit{x}}_{\mathit{i}}{\mathit{x}}_{\mathit{i}}^{\mathrm{\top }}⟶{\mathbf{\Sigma }}_{\mathit{x}\mathit{x}},\frac{1}{n}\sum _{i=1}^n{\mathit{z}}_{\mathit{i}}{\mathit{z}}_{\mathit{i}}^{\mathrm{\top }}⟶{\mathbf{\Sigma }}_{\mathit{z}\mathit{z}},\frac{1}{n}\sum _{i=1}^n{\mathit{x}}_{\mathit{i}}{\mathit{z}}_{\mathit{i}}^{\mathrm{\top }}⟶{\mathbf{\Sigma }}_{\mathit{x}\mathit{z}}.$$

The following theorem discusses on the asymptotic behavior of the 2SLS bridge estimator ${\hat{\mathit{\beta }}}_{2SLS-B}$.

Theorem 1

Under the assumptions (A.1)–(A.3), if $\gamma \ge 1$, ${\displaystyle \frac{{\lambda }_n}{\sqrt{n}}}⟶{\lambda }_0\ge 0$, then

$$\sqrt{n}({\hat{\mathit{\beta }}}_{2SLS-B}-\mathit{\beta }{)}^{\mathrm{\top }}⟶\mathcal{D}\arg minV(\mathit{u}),$$

where

$$\begin{array}{rl}V(\mathit{u})& =-2{\mathit{u}}^{\mathrm{\top }}{\mathbf{\Sigma }}_{\mathit{x}\mathit{z}}{\mathbf{\Sigma }}_{\mathit{z}\mathit{z}}^{-1}\mathit{W}+{\mathit{u}}^{\mathrm{\top }}{\mathbf{\Sigma }}_{\mathit{x}\mathit{z}}{\mathbf{\Sigma }}_{\mathit{z}\mathit{z}}^{-1}{\mathbf{\Sigma }}_{\mathit{z}\mathit{x}}\mathit{u}\\ & +{\lambda }_0\sum _{j=1}^p{u}_j\mathrm{sgn}({\beta }_j)\mid {\beta }_j{\mid }^{\gamma -1}\mathrm{if}\gamma >1,\\ V(\mathit{u})& =-2{\mathit{u}}^{\mathrm{\top }}{\mathbf{\Sigma }}_{\mathit{x}\mathit{z}}{\mathbf{\Sigma }}_{\mathit{z}\mathit{z}}^{-1}\mathit{W}+{\mathit{u}}^{\mathrm{\top }}{\mathbf{\Sigma }}_{\mathit{x}\mathit{z}}{\mathbf{\Sigma }}_{\mathit{z}\mathit{z}}^{-1}{\mathbf{\Sigma }}_{\mathit{z}\mathit{x}}\mathit{u}\\ & +{\lambda }_0\sum _{j=1}^p[u_j\mathrm{sgn}({\beta }_j)I({\beta }_j\ne 0)+|u_j|I({\beta }_j=0)],\mathrm{if}\gamma =1,\end{array}$$

and $\mathit{W}$ follows the $N(\mathbf{0},{\sigma }^2{\mathbf{\Sigma }}_{\mathit{z}\mathit{z}})$ distribution.

Proof.

Define

$$V_n(\mathit{u})=\sum _{i=1}^n[{({\epsilon }_i-\frac{{\mathit{u}}^{\mathrm{\top }}{\hat{\mathit{x}}}_i}{\sqrt{n}})}^2-{\epsilon }_i^2]+{\lambda }_n\sum _{j=1}^p[|{\beta }_j+\frac{u_j}{\sqrt{n}}{|}^{\gamma }-|{\beta }_j{|}^{\gamma }],$$

where $\mathit{u}=(u_1,\dots ,u_p{)}^{\mathrm{\top }}$ and ${\hat{\mathit{x}}}_i=\mathit{Z}\hat{\mathit{\eta }}+{\mathit{u}}_{\mathit{i}},i=1,2,\dots ,n$. Here, the convergence of the first term of $V_n(\mathit{u})$ is proven and the convergence of the second term can be similarly found in Knight and Wenjiang [19]. Note that

$$\sum _{i=1}^n[{({\epsilon }_i-\frac{{\mathit{u}}^{\mathrm{\top }}{\hat{\mathit{x}}}_i}{\sqrt{n}})}^2-{\epsilon }_i^2]={\mathit{u}}^{\mathrm{\top }}\sum _{i=1}^n\frac{\widehat{{\mathit{x}}_{\mathit{i}}}{\widehat{{\mathit{x}}_{\mathit{i}}}}^{\mathrm{\top }}}{n}-2{\mathit{u}}^{\mathrm{\top }}\sum _{i=1}^n\frac{\widehat{{\mathit{x}}_{\mathit{i}}}{\epsilon }_i}{\sqrt{n}}=\frac{{\mathit{u}}^{\mathrm{\top }}{\hat{\mathit{x}}}^{\mathrm{\top }}\hat{\mathit{x}}}{n}+\frac{{\hat{\mathit{x}}}^{\mathrm{\top }}\mathit{\epsilon }}{\sqrt{n}}.$$ (7)

Using the central limit theorem and (A.1)-(A.3), the RHS of (7) converges in distribution to $-2{\mathit{u}}^{\mathrm{\top }}{\mathbf{\Sigma }}_{\mathit{z}\mathit{x}}{\mathbf{\Sigma }}_{\mathit{z}\mathit{z}}^{-1}\mathit{W}+{\mathit{u}}^{\mathrm{\top }}{\mathbf{\Sigma }}_{\mathit{x}\mathit{z}}{\mathbf{\Sigma }}_{\mathit{z}\mathit{z}}^{-1}{\mathbf{\Sigma }}_{\mathit{z}\mathit{x}}\mathit{u}$ which completes the proof.

Corollary 1

${\hat{\mathit{\beta }}}_{2SLS-R}$ has asymptotically normal distribution with mean $-{\lambda }_0{\mathbf{\Sigma }}_{\mathit{z}\mathit{x}}^{-1}{\mathbf{\Sigma }}_{\mathit{z}\mathit{z}}$

${\mathbf{\Sigma }}_{\mathit{x}\mathit{z}}^{-1}\mathit{\beta }$ and variance-covariance matrix ${\sigma }^2{\mathbf{\Sigma }}_{\mathit{z}\mathit{x}}^{-1}{\mathbf{\Sigma }}_{\mathit{z}\mathit{z}}{\mathbf{\Sigma }}_{\mathit{x}\mathit{z}}^{-1}.$

Proof.

Setting $\gamma =2$ and ${\hat{\mathit{\beta }}}_{2SLS-R}=({\mathit{X}}^{\mathrm{\top }}\mathit{P}\mathit{X}+k\mathit{I}{)}^{-1}{\mathit{X}}^{\mathrm{\top }}\mathit{P}\mathit{y}$ in Theorem 1, implies that

$$V(\mathit{u})=-2{\mathit{u}}^{\mathrm{\top }}{\mathbf{\Sigma }}_{\mathit{x}\mathit{z}}{\mathbf{\Sigma }}_{\mathit{z}\mathit{z}}^{-1}\mathit{W}+{\mathit{u}}^{\mathrm{\top }}{\mathbf{\Sigma }}_{\mathit{x}\mathit{x}}{\mathbf{\Sigma }}_{\mathit{z}\mathit{z}}^{-1}{\mathbf{\Sigma }}_{\mathit{z}\mathit{x}}\mathit{u}+{\lambda }_0{\mathit{u}}^{\mathrm{\top }}\mathit{\beta },$$

so,

$$\begin{array}{rl}\arg minV(\mathit{u})& =({\mathbf{\Sigma }}_{\mathit{x}\mathit{z}}{\mathbf{\Sigma }}_{\mathit{z}\mathit{z}}^{-1}{\mathbf{\Sigma }}_{\mathit{z}\mathit{x}}{)}^{-1}{\mathbf{\Sigma }}_{\mathit{x}\mathit{z}}{\mathbf{\Sigma }}_{\mathit{z}\mathit{z}}^{-1}\mathit{W}-{\lambda }_0({\mathbf{\Sigma }}_{\mathit{x}\mathit{z}}{\mathbf{\Sigma }}_{\mathit{z}\mathit{z}}^{-1}{\mathbf{\Sigma }}_{\mathit{z}\mathit{x}}{)}^{-1}\mathit{\beta }\\ & ={\mathbf{\Sigma }}_{\mathit{z}\mathit{x}}^{-1}\mathit{W}-{\lambda }_0{\mathbf{\Sigma }}_{\mathit{z}\mathit{x}}^{-1}{\mathbf{\Sigma }}_{\mathit{z}\mathit{z}}{\mathbf{\Sigma }}_{\mathit{x}\mathit{z}}^{-1}\mathit{\beta }\sim N(-{\lambda }_0{\mathbf{\Sigma }}_{\mathit{z}\mathit{x}}^{-1}{\mathbf{\Sigma }}_{\mathit{z}\mathit{z}}{\mathbf{\Sigma }}_{\mathit{x}\mathit{z}}^{-1}\mathit{\beta },{\sigma }^2{\mathbf{\Sigma }}_{\mathit{z}\mathit{x}}^{-1}{\mathbf{\Sigma }}_{\mathit{z}\mathit{z}}{\mathbf{\Sigma }}_{\mathit{x}\mathit{z}}^{-1}).\end{array}$$

The proof is complete.

3. The proposed test

In this section, we propose a test statistic to diagnose endogeneity when the multicollinearity problem exists in the explanatory variables. Suppose that the regression model between the outcome vector $\mathit{y}$ and the predictor matrix $\mathit{X}$ is expressed as $\mathit{y}=\mathit{X}\mathit{\beta }+\mathit{ϵ}$.

Then, for the endogeneity testing hypotheses

$$\begin{array}{l}{H}_0:cov(\mathit{X},\mathit{ϵ})=\mathbf{0}\\ H_1:cov(\mathit{X},\mathit{ϵ})\ne \mathbf{0}\end{array}.$$ (8)

Hausman[20] introduced the following test statistic

$$H=({\hat{\mathit{\beta }}}_{2SLS}-{\hat{\mathit{\beta }}}_{OLS}{)}^{\mathrm{\top }}[Var({\hat{\mathit{\beta }}}_{2SLS})-Var({\hat{\mathit{\beta }}}_{OLS}){]}^{-1}({\hat{\mathit{\beta }}}_{2SLS}-{\hat{\mathit{\beta }}}_{OLS})$$ (9)

Under the null hypothesis $H_0$, it follows a central chi-square distribution with p degrees of freedom $({\chi }^2(p))$ distribution where p is the number of unknown parameters.

Many fields of sciences, such as marketing, genomics, and basic sciences have studied detection of multicollinearity see e.g. Batterham et al. [1]. An important consequence of a high degree of multicollinearity is the high estimated variance of ${\hat{\mathit{\beta }}}_{OLS}$. Therefore, in this case, the Hausman test statistic tends to zero and always leads to accepting the null hypothesis. Therefore, the Hausman test in the presence of the multicollinearity problem in the model may lead to incorrect conclusions about regressors' endogeneity. Guo et al. [14] introduced a test statistic in this context without considering the problem of multicollinearity.

We propose a modified version of the Hausman test statistic as follows:

$$H^{\ast }=({\hat{\mathit{\beta }}}_{2SLS-B}-{\hat{\mathit{\beta }}}_R{)}^{\mathrm{\top }}[Var({\hat{\mathit{\beta }}}_{2SLS-B})-Var({\hat{\mathit{\beta }}}_R){]}^{-1}({\hat{\mathit{\beta }}}_{2SLS-B}-{\hat{\mathit{\beta }}}_R),$$ (10)

where ${\hat{\mathit{\beta }}}_R=({\mathit{X}}^{\mathrm{\top }}\mathit{X}+k\mathit{I}{)}^{-1}{\mathit{X}}^{\mathrm{\top }}\mathit{y}$ is the ordinary ridge estimation in the model (2) and $k>0$ is the ridge parameter. It is expected that the null hypothesis of endogeneity is rejected if the statistic $H^{\ast }$ significantly differs from zero.

To perform the hypothesis test, we need to find the null distribution of the statistic $H^{\ast }$. To this end, we take the following assumptions

(B.1) $\mathit{X}$ and $\mathit{Z}$ are stochastic and nonsingular matrices.

(B.2) The $p\times p$ matrices ${\mathit{M}}_1=plim(\frac{{\mathit{X}}^{\mathrm{\top }}\mathit{X}}{n})$ and ${\mathit{M}}_2=plim(\frac{{\mathit{X}}^{\mathrm{\top }}\mathit{P}\mathit{X}}{n})$ exist and are non-singular.

Theorem 2

Under (B.1)-(B.2), (A.3) and the null hypothesis (8), the statistic $H^{\ast }$ in (10) has asymptotically $({\chi }^2(p))$.

Proof.

Following the approach of Hausman [20], it suffices to prove the following three assertions:

$\mathbf{1}.$ Under the null hypothesis (8), ${\hat{\mathit{\beta }}}_R$ and ${\hat{\mathit{\beta }}}_{2SLS-B}$ are consistent estimators of $\mathit{\beta }$.

$\mathbf{2}.$ ${\hat{\mathit{\beta }}}_R$ is an efficient estimator under the null hypothesis (8).

$\mathbf{3}.$ ${\hat{\mathit{\beta }}}_R$ and ${\hat{\mathit{\beta }}}_{2SLS-B}$ have asymptotically normal distributions.

Under (B.1)–(B.2) we have

$$\begin{array}{rl}plim{\hat{\mathit{\beta }}}_R& =plim[({\mathit{X}}^{\mathrm{\top }}\mathit{X}+k\mathit{I}{)}^{-1}{\mathit{X}}^{\mathrm{\top }}\mathit{y}=plim[({\mathit{X}}^{\mathrm{\top }}\mathit{X}+k\mathit{I}{)}^{-1}{\mathit{X}}^{\mathrm{\top }}(\mathit{X}\mathit{\beta }+\mathit{\epsilon })]\\ & =plim[(\frac{{\mathit{X}}^{\mathrm{\top }}\mathit{X}+k\mathit{I}}{n}{)}^{-1}\frac{{\mathit{X}}^{\mathrm{\top }}\mathit{X}\mathit{\beta }}{n}]+plim[(\frac{{\mathit{X}}^{\mathrm{\top }}\mathit{X}+k\mathit{I}}{n}{)}^{-1}\frac{{\mathit{X}}^{\mathrm{\top }}\mathit{\epsilon }}{n}]\\ & ={\mathit{M}}_1^{-1}{\mathit{M}}_1\mathit{\beta }+\mathbf{0}=\mathit{\beta },\end{array}$$

which proves the consistency of ${\hat{\mathit{\beta }}}_R$. Also, the assumption (A.3) imply that ${\hat{\mathbf{\Sigma }}}_{\mathit{x}\mathit{z}}=\frac{1}{n}{\mathit{X}}^{\mathrm{\top }}\mathit{Z}\to {\mathbf{\Sigma }}_{\mathit{x}\mathit{z}}$ and therefore around a local vector ${\mathit{\beta }}^o$ we have

$$\begin{array}{rl}plim{\hat{\mathit{\beta }}}_{2SLS-B}& =lim[({\mathit{X}}^{\mathrm{\top }}\mathit{P}\mathit{X}+{\mathbf{\Sigma }}_{\lambda }({\mathit{\beta }}^o){)}^{-1}{\mathit{X}}^{\mathrm{\top }}\mathit{P}(\mathit{X}\mathit{\beta }+\mathit{\epsilon })]=({\mathit{M}}_{\mathbf{2}}+\mathbf{0}{)}^{-1}{\mathit{M}}_{\mathbf{2}}\mathit{\beta }\\ & +plim{[\left(\frac{{\mathit{X}}^{\mathrm{\top }}\mathit{Z}}{n}\right){\left(\frac{{\mathit{Z}}^{\mathrm{\top }}\mathit{Z}}{n}\right)}^{-1}\left(\frac{{\mathit{Z}}^{\mathrm{\top }}\mathit{X}}{n}\right)+\frac{{\mathbf{\Sigma }}_{\lambda }({\mathit{\beta }}^o)}{n}]}^{-1}\\ & \left[\left(\frac{{\mathit{X}}^{\mathrm{\top }}\mathit{Z}}{n}\right){\left(\frac{{\mathit{Z}}^{\mathrm{\top }}\mathit{Z}}{n}\right)}^{-1}\left(\frac{{\mathit{Z}}^{\mathrm{\top }}\mathit{\epsilon }}{n}\right)\right]\\ & =\mathit{\beta }+[{\mathbf{\Sigma }}_{xz}{\mathbf{\Sigma }}_{zz}^{-1}{\mathbf{\Sigma }}_{zx}{]}^{-1}[{\mathbf{\Sigma }}_{xz}{\mathbf{\Sigma }}_{zz}^{-1}{\mathbf{\Sigma }}_{z\epsilon }]=\mathit{\beta }.\end{array}$$

To establish the efficiency of ${\hat{\mathit{\beta }}}_R$, the Cramer-Rao inequality implies that

$$Cov({\hat{\mathit{\beta }}}_R)\ge {\left(\frac{\mathrm{\partial }}{\mathrm{\partial }\mathit{\beta }}E{\hat{\mathit{\beta }}}_R\right)}^{\mathrm{\top }}{{\mathit{I}}_n}^{-1}(\mathit{\beta })\left(\frac{\mathrm{\partial }}{\mathrm{\partial }\mathit{\beta }}E{\hat{\mathit{\beta }}}_R\right)$$

where ${\mathit{I}}_n(\mathit{\beta })={\sigma }^2{\mathit{X}}^{\mathrm{\top }}\mathit{X}$ is the Fisher's information matrix. Given that $E({\hat{\mathit{\beta }}}_R)=({\mathit{X}}^{\mathrm{\top }}\mathit{X}+k\mathit{I}{)}^{-1}({\mathit{X}}^{\mathrm{\top }}\mathit{X})\mathit{\beta }$, we conclude that:

$$Cov({\hat{\mathit{\beta }}}_R)\ge {\sigma }^2({\mathit{X}}^{\mathrm{\top }}\mathit{X}+k\mathit{I}{)}^{-1}({\mathit{X}}^{\mathrm{\top }}\mathit{X})({\mathit{X}}^{\mathrm{\top }}\mathit{X}+k\mathit{I}{)}^{-1},$$

and also

$$Var({\hat{\mathit{\beta }}}_R)={\sigma }^2({\mathit{X}}^{\mathrm{\top }}\mathit{X}+k\mathit{I}{)}^{-1}({\mathit{X}}^{\mathrm{\top }}\mathit{X})({\mathit{X}}^{\mathrm{\top }}\mathit{X}+k\mathit{I}{)}^{-1},$$

which means that ${\hat{\mathit{\beta }}}_R$ is an efficient estimator.

The asymptotic normality of ${\hat{\mathit{\beta }}}_R$ is verified by Knight and Wenjiang [19] which they have shown that ${\hat{\mathit{\beta }}}_R$ asymptotically follows a normal distribution with mean $-{\lambda }_0{\mathbf{\Sigma }}_{\mathit{x}\mathit{x}}^{-1}\mathit{\beta }$ and variance-covariance matrix ${\sigma }^2{\mathbf{\Sigma }}_{\mathit{x}\mathit{x}}^{-1}$. Finally, the asymptotic normality of ${\hat{\mathit{\beta }}}_{2SLS-B}$ follows from Theorem (1), and this completes the proof.

4. Simulation study

In this section, we carry out some numerical analyses of the two most applied special cases of the bridge estimator i.e. ${\hat{\mathit{\beta }}}_{2SLS-R}$ and ${\hat{\mathit{\beta }}}_{2SLS-L}$. First we compare the performance of ${\hat{\mathit{\beta }}}_{2SLS-R}$ and ${\hat{\mathit{\beta }}}_{2SLS-L}$ with the traditional estimators ${\hat{\mathit{\beta }}}_{OLS}$, ${\hat{\mathit{\beta }}}_{2SLS}$, ${\hat{\mathit{\beta }}}_R$ and ${\hat{\mathit{\beta }}}_{lasso}$ based on their absolute relative bias (R.Bias) and estimated mean square errors (EMSE) under different degrees of endogeneity. Next, the our proposed test statistic is compared to Durbin- Wu- Hausman test in terms of the estimated power.

textbfWe generate data in the form of the vector $\mathit{W}=(\mathit{X},\mathit{Z},{\mathit{ϵ}\mathbf{)}}^{\mathrm{\top }}$ from a multivariate normal distribution with mean zero and a specific variance covariance matrix. In order to impose multicollinearity to the data, the variance covariance matrix is adopted such that the correlation between the covariates is equal to the constant value 0.9 and there is a slight correlation $(0.4)$ between all instruments and the predictor variables, meaning that all instrument variables are relevant. Moreover, for endogeneity, different levels of $\rho \in \{0,0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8\}$ are considered. Using the approach of principal component analysis, we take the largest eigenvector of ${\mathit{X}}^{\mathrm{\top }}\mathit{X}$ as the vector of coefficients $\mathit{\beta }$ ([18]). Then, the response vector generated following $\mathit{y}=\mathit{X}\mathit{\beta }+\mathit{\epsilon }$. Specifically, in this experiment, we take the number of covariates and the number of instruments as $p_x=2$ and $p_z=3$, respectively. Then by generating new error terms, the experiment is replicated 5000 times. The absolute R.Bias and EMSE for the standardized values of any estimator $\hat{\mathit{\beta }}$ say, are calculated respectively as follows:

$$R.Bias(\hat{\beta })=\frac{1}{5000}\sum _{l=1}^{5000}\sum _{j=1}^{p_x}\mid {\displaystyle \frac{{\hat{\beta }}_{jl}-{\beta }_j}{{\beta }_j}}\mid ,EMSE=\frac{1}{5000}\sum _{l=1}^{5000}\sum _{j=1}^{p_x}({\hat{\beta }}_{jl}-{\beta }_j{)}^2$$

where ${\hat{\beta }}_{jl}$ is the estimation of ${\hat{\beta }}_j$ in the lth replication of the simulation. The value of the ridge parameter k is obtained via the generalized cross-validation technique introduced by Golub et al. [13] to obtain a good estimator. See also Liu and Jiang [20] and references therein. The results are summarized in Table 1, for the values n = 50 and $\rho =0,0.5,0.8$¹.

Table 1. Estimated parameter values, absolute R.Bias and EMSE for $n=50$ and different values of ρ.

$\rho =0$
Coefficients	β	$\widehat{{\beta }_{ols}}$	$\widehat{{\beta }_{2sls}}$	$\widehat{{\beta }_{ridge}}$	$\widehat{{\beta }_{2sls-r}}$
${\beta }_1$	0.7	0.658	0.671	0.678	0.624
${\beta }_2$	0.5	0.509	0.576	0.502	0.5548
R.bias		0.402	0.879	0.486	0.647
EMSE		0.126	1.019	0.116	0.343
$\rho =0.5$
Coefficients	β	$\widehat{{\beta }_{ols}}$	$\widehat{{\beta }_{2sls}}$	$\widehat{{\beta }_{ridge}}$	$\widehat{{\beta }_{2sls-r}}$
${\beta }_1$	0.7	1.400	0.740	1.366	0.737
${\beta }_2$	0.5	0.044	0.484	0.066	0.476
R.bias		1.158	0.825	1.103	0.636
EMSE		0.812	0.777	0.706	0.327
$\rho =0.8$
Coefficients	β	$\widehat{{\beta }_{ols}}$	$\widehat{{\beta }_{2sls}}$	$\widehat{{\beta }_{ridge}}$	$\widehat{{\beta }_{2sls-r}}$
${\beta }_1$	0.7	1.838	0.774	1.815	0.757
${\beta }_2$	0.5	−0.252	0.459	$-0.236$	0.468
R.bias		1.891	0.805	1.851	0.691
EMSE		1.909	0.853	1.831	0.419

Table 1 shows that in the case $\rho =0$ where we only have the multicollinearity problem, the OLS estimator has the lowest R.bias value and the ridge estimator has the lowest EMSE value. While in the presence of a problem of the endogeneity, the EMSE and the absolute R.Bias values of the 2SLS-R estimator are lesser than the other three competitors.

Figure 1 depicts the absolute R.bias and EMSE values of the three estimators for different values of ρ and n.

Figure 2.

Criteria plot of Bias and EMSE for n = 20, 100.

Figure 1.

Plot of absolute R.Bias and EMSE for n = 20, 100.

In view of Figure 1, for enough large sample size, if the endogeneity level is almost greater than 0.2, then the new estimator is always better than the other three estimators.

In order to evaluate the performance of ${\hat{\mathit{\beta }}}_{2SLS-L}$, the sparse and endogenous samples are generated. For this purpose, the vector $\mathit{W}=(\mathit{X},\mathit{Z},{\mathit{ϵ}\mathbf{)}}^{\mathrm{\top }}$ with the multivariate normal distribution with mean zero and the variance covariance matrix is considered such that all instruments are relevant and for the different levels of the endogeneity corresponding to the values $\rho =0,\dots .,1$. The vector $\mathit{\beta }$ is also taken as the form $\mathit{\beta }=(0.6,0.001)$ to create sparsity conditions. Here again, we set the number of covariates and the number of instruments as $p_x=2$ and $p_z=3$ and we compare the performance of new 2SLS-L estimator with the three estimators OLS, 2SLS and Lasso. The results are summarized in Table 2 and plotted in Figure 3.

Table 2. Estimated parameter values, absolute R.Bias and EMSE for $n=50$ and different values of ρ.

$\rho =0$
Coefficients	β	${\hat{\beta }}_{ols}$	${\hat{\beta }}_{2sls}$	${\hat{\beta }}_{lasso}$	${\hat{\beta }}_{2sls-lasso}$
${\beta }_1$	0.6	0.604	0.610	0.572	0.349
${\beta }_2$	0.001	−0.0004	0.0001	0.0002	0.0004
R.bias		0.227	0.507	0.217	0.502
EMSE		0.041	0.354	0.040	0.308
$\rho =0.6$
Coefficients	β	${\hat{\beta }}_{ols}$	${\hat{\beta }}_{2sls}$	${\hat{\beta }}_{lasso}$	${\hat{\beta }}_{2sls-lasso}$
${\beta }_1$	0.6	1.196	0.656	1.137	0.371
${\beta }_2$	0.001	0.003	−0.0001	0.002	0.0003
R.bias		0.684	0.511	0.598	0.467
EMSE		0.386	1.096	0.319	0.261
$\rho =1$
Coefficients	β	${\hat{\beta }}_{ols}$	${\hat{\beta }}_{2sls}$	${\hat{\beta }}_{lasso}$	${\hat{\beta }}_{2sls-lasso}$
${\beta }_1$	0.6	1.589	0.643	1.567	0.524
${\beta }_2$	0.001	0.0003	−0.002	0.0001	0.0002
R.bias		1.042	0.524	0.996	0.454
EMSE		1.005	0.997	0.934	0.266

Table 3. Estimated parameter values and MSPEs for the OLS, the 2SLS, the Ridge and 2SLS-R methods.

Estimation	${\hat{\beta }}_{ols}$	${\hat{\beta }}_{2sls}$	${\hat{\beta }}_{ridge}$	${\hat{\beta }}_{2sls-r}$
Intercept	0.0055	0.0064	0.0055	0.0061
$\widehat{{\beta }_1}$	−0.1747	−5.868	−0.1707	−3.5670
$\widehat{{\beta }_2}$	0.2384	1.5041	0.2280	0.8182
$\widehat{{\beta }_3}$	0.0565	4.1146	0.0502	2.6057
Estimated MSPE	0.9914	0.9882	0.9901	0.9832

As we can see in Table 2, almost having an endogeneity level greater than 0.6, the 2SLS-L is superior to the other three estimators.

At the end of this section, the power of our proposed test of endogeneity for a set of data with multicollinearity and endogeneity ill-conditioned is studied. We compare its power with the power of the existing DWH test. For this purpose we generate samples from model (1) by simulating vector $\mathit{W}=(\mathit{X},\mathit{Z},{\mathit{ϵ}\mathbf{)}}^{\mathrm{\top }}$ which has a multivariate normal distribution with mean zero and consider the variance covariance matrix in such a way that the correlation between the covariates is equal to 0.9 and all instruments are relevant. Also, the number of instruments and covariates are assumed as $p_z=3$ and $p_x=3$, respectively. Further, we consider the beta coefficient vector as $\mathit{\beta }=(2,3,2)$. We have considered 9 different values of the endogeneity levels as $\rho =0,0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8$ and used the exact critical value based on the distribution of statistic at 0.05 significance level test. For each value of the endogeneity level, the power of the proposed test is estimated by N = 1000 simulated samples of size n = 100 from the alternative hypothesis. As shown in Figure 3, these two test statistics have almost the same power under the null hypothesis while under the alternative hypothesis our modified test behaves better, than the DWH test in the presence of the multicollinearity, in terms of the power criterion.

Figure 3.

Comparison of the DWH test statistic and the new modified DWH test statistic $H^{\ast }$.(solid line: DWH test statistic; dashed line: new modified DWH test statistic).

5. Genetic data analysis

In this section, we apply our results to carry out a regression analysis of a genetic data set that suffers from multicollinearity and endogeneity. Several authors have reported the presence or absence of a relationship between the genetic variant(s) and the outcome as the primary analysis result, rather than an instrumental analysis, ( Didelez and Sheehan [7], Burgess and Dylan [4]). We use the our method to overcome endogeneity and multicollinearity in the data set of Health and Retirement Study (HRS), which is supported by the National Institute on Aging (NIA U01AG009740, RC2 AG036495, RC4 AG039029). This data contain genotype and phenotype variants which may effect on some outcomes such as educational attainment. see e. g. Rietveld et al. [27] and Okbay et al. [23].

We are involved in running multiple regression to predict educational attainment (EA) using systolic blood pressure (SBP), main arterial pressure (MAP) and pulse pressure (PP) as predictor variables. These predictors are extremely correlated (with $\hat{\rho }=0.922$ between MAP and PP and VIF=60.79 in the regression equation between these exposure variables). On the other hand, the dependent variable EA and these three predictors are correlated in which correlation between residual of regression equation and EA is about 0.9997, so, we choose another three variables major depressive disorder (MDD), schizophrenia polygenic score (SPS) and subjective well-being (SWB) as instrumental variables since their correlation with EA is 0.030, 0.015 and 0.012, respectively. We first assessed the fitted value at the first stage and observed that the previous multicollinearity in the matrix ${\mathit{X}}^{\mathrm{\top }}\mathit{P}\mathit{X}$ still exists and consequently a cross-validation technique incorporated to obtain the ridge parameter which is equal to k = 1. Hence, using (6) we can estimate the parameters on the model. We also calculate the Mean Squared Prediction Error (MSPE) criterion to evaluate the performance of the estimators. To end this, we divide the real data into two subgroups, training and testing, such that 80 percent of the data set to the training set and the rest in the test group. Table 3 presents the estimated values of the coefficients of SBP, MAP, and PP using the four methods OLS, 2SLS, ridge and regularized 2SLS, along with the MSPE values for the test group.

According to the results of this paper, the proposed new estimator offers smaller estimated MSPE compared to the others. Furthermore, the value of the proposed test statistic $H^{\ast }$, is equal to 5.401 while for the DWH test, it is lesser and equals to $0.144$. Note that both of these values of tests statistics provide evidences in favor of $H_0$.

6. Conclusion

In this study we used the bridge method to combat the multicollinearity and sparsity in a two-stage least squares estimation. In particular, we studied a ridge-2SLS as well as a Lasso-2SLS estimation procedures. Some large sample properties of the proposed estimator investigated and compared with some other estimators using a simulation study. Furthermore, we proposed a modification of the DWH test of endogeneity. Using a simulation study we observed that the our new test has higher power than the power of the existing DWH test in multi-collinear data.

Disclosure statement

No potential conflict of interest was reported by the author(s).