Robust Statistical Inference in Generalized Linear Models Based on Minimum Renyi’s Pseudodistance Estimators

Abstract

Minimum Renyi’s pseudodistance estimators (MRPEs) enjoy good robustness properties without a significant loss of efficiency in general statistical models, and, in particular, for linear regression models (LRMs). In this line, Castilla et al. considered robust Wald-type test statistics in LRMs based on these MRPEs. In this paper, we extend the theory of MRPEs to Generalized Linear Models (GLMs) using independent and nonidentically distributed observations (INIDO). We derive asymptotic properties of the proposed estimators and analyze their influence function to asses their robustness properties. Additionally, we define robust Wald-type test statistics for testing linear hypothesis and theoretically study their asymptotic distribution, as well as their influence function. The performance of the proposed MRPEs and Wald-type test statistics are empirically examined for the Poisson Regression models through a simulation study, focusing on their robustness properties. We finally test the proposed methods in a real dataset related to the treatment of epilepsy, illustrating the superior performance of the robust MRPEs as well as Wald-type tests.

1. Introduction

Generalized linear models (GLMs) were first introduced by Nelder and Wedderburn [1] and later expanded upon by McCullagh and Nelder [2]. The GLMs represent a natural extension of the standard linear regression models, which enclose a large variety of response variable distributions, including distributions of count, binary, or positive values. Let $Y_1,\dots ,Y_n$ be independent response variables. The classical GLM assumes that the density function of each random variable $Y_i$ belongs to the exponential family, having the form

$$f\left(y,{\theta }_i,\varphi \right)=exp\left\{\frac{y{\theta }_i-b\left({\theta }_i\right)}{a\left(\varphi \right)}+c\left(y,\varphi \right)\right\},$$ (1)

for $i=1,\dots ,n,$ where the functions $a\left(\varphi \right),$$b\left({\theta }_i\right)$ and $c\left(y,\varphi \right)$ are known. Therefore, the observations are independent but not identically distributed, depending on a location parameter ${\theta }_i,$$i=1,\dots ,n,$ and a nuisance parameter $\varphi .$ Further, we denote by ${\mu }_i$ the expectation of the random variable $Y_i$ and we assume that there exists a monotone differentiable function, so called link function g, verifying

$$g\left({\mu }_i\right)={\mathit{x}}_i^T\mathit{\beta },$$

with $\mathit{\beta }=({\mathit{\beta }}_1,\dots ,{\mathit{\beta }}_k)\in {\mathbb{R}}^k$$(k<n)$ the regression parameter vector. The $k\times 1$-vector of explanatory variables, ${\mathit{x}}_i,$ is assumed to be nonrandom, i.e., the design matrix is fixed. Correspondingly, the location parameter depends on the explanatory variables $\theta =\theta \left({\mathit{x}}^T\mathit{\beta }\right)$ the density function given in (1) can be written as $f_i\left(y,\mathit{\beta },\varphi \right),$ empathizing its dependence of $\mathit{\beta }$ and ${\mathit{x}}_i$.

The maximum likelihood estimator (MLE) and the quasilikelihood estimators were well studied for the GLMs, and it is well known that they are asymptotically efficient but lack robustness in the presence of outliers, which can result in a significant estimation bias. Jaenada and Pardo [3] revised the different robust estimators in the statistical literature and studied the lack of robustness of the MLE as well. Among others, Stefanski et al. [4] studied optimally bounded score functions for the GLM and generalized the results obtained by Krasker and Welsch [5] for classical LRMs. Künsch et al. [6] introduced the so-called conditionally unbiased bounded-influence estimate, and Morgenthaler [7], Cantoni and Ronchetti [8], Bianco and Yohai [9], Croux and Hesbroeck [10], Bianco et al. [11], and Valdora and Yohai [12] continued the development of robust estimators for the GLMs based on general M-estimators. Later, Ghosh and Basu [13] proposed robust estimators for the GLM, based on the density power divergence (DPD) introduced in Basu et al. [14].

There are not many papers considering robust tests for GLMs. In this sense, Basu et al. [15] considered robust Wald-type tests based on the minimum DPD estimator, but assuming random explanatory variables for the GLM. The main purpose of this paper is to introduce new robust Wald-type tests based on the MRPE under fixed (not random) explanatory variables.

Broniatowski et al. [16] presented robust estimators for the parameters of the linear regression model (LRM) with random explanatory variables and Castilla et al. [17] considered Wald-type test statistics, based on MRPE, for the LRM. Toma and Leoni–Aubin [18] defined new robustness and efficient measures based on the RP and Toma et al. [19] considered the MRPE for general parametric models, and constructed a model selection criterion for regression models. The term “Rényi pseudodistance” (RP) was adopted in Broniatowski et al. [16] because of its similarity with the Rényi divergence (Rényi [20]), although this family of divergences was considered previously in Jones et al. [21]. Fujisawa and S. Eguchi [22] used the RP under the name of $\gamma$-cross entropy, introduced robust estimators obtained by minimizing the empirical estimate of the $\gamma$-cross entropy (or the $\gamma$-divergence associated to the $\gamma$-cross entropy) and studied their properties. Further, Hirose and Masuda [23] considered the $\gamma$ likelihood function to find robust estimation. Using the $\gamma$-divergence, Kawashima and Fujisawa [24,25] presented robust estimators for sparse regression and sparse GLMs with random covariates. The robustness of all the previous estimators is based on density power weight, $f{\left(y,\theta \right)}^l,$ which gives a small weight to outliers observations. This idea was also developed by Basu et al. [15] for the minimum DPD estimator and was considered some years ago by Windham [26]. More concretely, Basu et al. [14] considered the density power function multiplied by the score function.

The outline of the paper is as follows: in Section 2, some results in relation to the MRPEs for GLMs, previously obtained in Jaenada and Pardo [3], are presented. Section 3 introduces and studies Wald-type tests based on the MRPE for testing linear null hypothesis for the GLMs. In Section 4, the influence function of the MRPE as well as the influence functions of the Wald-type tests are derived. Finally, we empirically examine the performance of the proposed robust estimators and Wald-type test statistics for the Poisson regression model through a simulation study in Section 5, and we illustrate its applicability with real data sets for binomial and Poisson regression.

2. Asymptotic Distribution of the MRPEs for the GLMs

In this Section, we revise some of the results presented in Jaenada and Pardo [3] in relation to the MRPE. Let $Y_1,\dots ,Y_n,$ be INIDO random variables with density functions with respect to some common dominating measure, $g_1,\dots ,g_n$ respectively. The true densities $g_i$ are modeled by the density functions given in (1), belonging to the exponential family. Such densities are denoted by $f_i\left(y,\mathit{\beta },\varphi \right)$ highlighting its dependence on the regression vector $\mathit{\beta },$ the nuisance parameter $\varphi$ and the observation $i$, $i=1,\dots ,n.$ In the following, we assume that the explanatory variables ${\mathit{x}}_i,$ are fixed, and therefore the response variables verify the INIDO set up studied in Castilla et al. [27].

For each of the response variables $Y_i$, the RP between the theoretical density function belonging to the exponential family, $f_i(y,\gamma ),$ and the true density underlying the data, $g_i,$ can be defined, for $\alpha >0$ as

$$R_{\alpha }\left(f_i(y,\gamma ),g_i\right)=\frac{1}{\alpha +1}log\left(\int f_i{(y,\gamma )}^{\alpha +1}dy\right)-\frac{1}{\alpha }log\left(\int f_i{(y,\gamma )}^{\alpha }{g}_i\left(y\right)dy\right)+k,$$ (2)

where

$$k=\frac{1}{\alpha \left(\alpha +1\right)}log\left(\int g_i{\left(y\right)}^{\alpha +1}dy\right)$$

does not depend on $\gamma ={({\mathit{\beta }}^T,\varphi )}^T.$

We consider $(y_1,\dots ,y_n)$ a random sample of independent but nonhomogeneous observations of the response variables with fixed predictors $({\mathit{x}}_1,\dots ,{\mathit{x}}_n).$ Since only one observation of each variable $Y_i$ is available, a natural estimate of its true density $g_i$ is the degenerate distribution at the the observation $y_i.$ Consequently, in the following we denote ${\widehat{g}}_i$ the density function of the degenerate variable at the point $y_i.$ Then, substituting in (2) the theoretical and empirical densities, yields to the loss

$$R_{\alpha }\left(f_i(y,\gamma ),{\widehat{g}}_i\right)=\frac{1}{\alpha +1}log\left(\int f_i{(y,\gamma )}^{\alpha +1}dy\right)-\frac{1}{\alpha }log{f}_i{(Y_i,\gamma )}^{\alpha }+k.$$ (3)

If we consider the limit when $\alpha$ tends to zero we get

$$R_0\left(f_i(y,\gamma ),{\widehat{g}}_i\right)=\underset{\alpha \downarrow 0}{lim}{R}_{\alpha }\left(f_i(y,\gamma ),{\widehat{g}}_i\right)=-log{f}_i(Y_i,\gamma )+k.$$ (4)

Last expression coincides with the Kullback–Leibler divergence, except for the constant $k.$ More details about Kullback–Leiber divergence can be seen in Pardo [28].

For the seek of simplicity, let us denote

$$L_{\alpha }^i\left(\gamma \right)={\left(\int f_i{(y,\gamma )}^{\alpha +1}dy\right)}^{\frac{\alpha }{\alpha +1}},$$

and

$$V_i(Y_i,\gamma )=\frac{f_i{(Y_i,\gamma )}^{\alpha }}{L_{\alpha }^i\left(\gamma \right)}.$$

The expression (3) can be rewritten as

$$R_{\alpha }\left(f_i(y,\gamma ),{\widehat{g}}_i\right)=-\frac{1}{\alpha }log\left(\frac{f_i{(Y_i,\gamma )}^{\alpha }}{{\left(\int f_i{(y,\gamma )}^{\alpha +1}dy\right)}^{\frac{\alpha }{\alpha +1}}}\right)+k=-\frac{1}{\alpha }log{V}_i(Y_i,\gamma )+k.$$

Based on the previous idea, we shall define an objective function averaging the RP between all the the RPs. Since minimizing $R_{\alpha }\left(f_i(y,\gamma ),{\widehat{g}}_i\right)$ in $\gamma$ is equivalent to maximizing $log{V}_i(Y_i,\gamma ),$ we define a loss function averaging those quantities as

$$T_n^{\alpha }\left(\gamma \right)=\frac{1}{n}\sum _{i=1}^n\frac{f_i{(Y_i,\gamma )}^{\alpha }}{{\left(\int f_i{(y,\gamma )}^{\alpha +1}dy\right)}^{\frac{\alpha }{\alpha +1}}}=\frac{1}{n}\sum _{i=1}^n\frac{f_i{(Y_i,\gamma )}^{\alpha }}{L_{\alpha }^i\left(\gamma \right)}=\frac{1}{n}\sum _{i=1}^n{V}_i(Y_i,\gamma ).$$ (5)

Based on (5), we can define the MRPE of the unknown parameter $\gamma$, ${\widehat{\gamma }}_{\alpha },$ by

$${\widehat{\gamma }}_{\alpha }=arg\underset{\gamma \in \Gamma }{max}{T}_n^{\alpha }\left(\gamma \right),$$ (6)

with $T_n^{\alpha }\left(\gamma \right)$ defined in (5)

$$T_n^0\left(\gamma \right)=\frac{1}{n}\sum _{i=1}^{n}log{f}_i(y_i,\gamma )$$

at $\alpha =0.$ The MRPE coincides with the MLE at $\alpha =0$, and therefore the proposed family can be considered a natural extension of the classical MLE.

Now, since the MRPE is defined as a maximum, it must annul the first derivatives of the loss function given in (5). The estimating equations of the parameters $\mathit{\beta }$ and $\varphi$ are given by

$$\left\{\begin{array}{c}\frac{1}{n}{\displaystyle \sum _{i=1}^n}\frac{\partial V_i(Y_i,\gamma )}{\partial \mathit{\beta }}={\mathbf{0}}_k\\ \frac{1}{n}{\displaystyle \sum _{i=1}^n}\frac{\partial V_i(Y_i,\gamma )}{\partial \varphi }=0.\end{array}\right.$$ (7)

For the first equation, we have

$$\begin{array}{cc}\hfill \frac{\partial V_i(Y_i,\gamma )}{\partial \mathit{\beta }}& =\frac{1}{L_{\alpha }^i{\left(\gamma \right)}^2}\left\{\alpha f_i{(Y_i,\gamma )}^{\alpha }\frac{\partial log{f}_i(Y_i,\gamma )}{\partial \mathit{\beta }}{L}_{\alpha }^i\left(\gamma \right)\right.\hfill \\ \hfill & \left.-\left[\alpha {\left(\int f_i{(y,\gamma )}^{\alpha +1}dy\right)}^{\frac{\alpha }{\alpha +1}-1}\int f_i{(y,\gamma )}^{\alpha +1}\frac{\partial log{f}_i(y,\gamma )}{\partial \mathit{\beta }}dy\right]f_i{(Y_i,\gamma )}^{\alpha }\right\}.\hfill \end{array}$$

The previous partial derivatives can be simplified as

$$\frac{\partial log{f}_i(Y_i,\gamma )}{\partial \mathit{\beta }}=\frac{Y_i-{\mu }_i}{Var\left(Y_i\right)g^{\prime }\left({\mu }_i\right)}{\mathit{x}}_i=K_{1i}\left(Y_i,\gamma \right){\mathit{x}}_i$$

and

$$\frac{\partial log{f}_i(Y_i,\gamma )}{\partial \varphi }=-\frac{\left(Y_i{\theta }_i-b\left({\theta }_i\right)\right)}{a{\left(\varphi \right)}^2}{a}^{\prime }\left(\varphi \right)+\frac{\partial c\left(Y_i,\varphi \right)}{\partial \varphi }=K_{2i}\left(Y_i,\gamma \right).$$

See Ghosh and Basu [13] for more details. Now using the simplified expressions, we can write the estimating equation for $\mathit{\beta }$ as

$${\displaystyle \sum _{i=1}^n}\frac{{\mathit{x}}_i}{L_{\alpha }^i\left(\gamma \right)}\left\{M_i\left(Y_i,\gamma \right)-N_i\left(Y_i,\gamma \right)\right\}={\mathbf{0}}_k$$ (8)

being

$$M_i\left(Y_i,\gamma \right)=f_i{(Y_i,\gamma )}^{\alpha }{K}_{1i}\left(Y_i,\gamma \right)$$

and

$$N_i\left(Y_i,\gamma \right)=\frac{f_i{(Y_i,\gamma )}^{\alpha }}{\int f_i{(y,\gamma )}^{\alpha +1}dy}\int f_i{(y,\gamma )}^{\alpha +1}{K}_{1i}\left(y,\gamma \right)dy.$$

Subsequently, the estimating equation for $\varphi ,$ is given by

$$\begin{array}{cc}\hfill \frac{\partial V_i(Y_i,\gamma )}{\partial \varphi }& =\frac{1}{L_{\alpha }^i{\left(\gamma \right)}^2}\left\{\alpha f_i{(Y_i,\gamma )}^{\alpha }\frac{\partial log{f}_i(Y_i,\gamma )}{\partial \varphi }{L}_{\alpha }^i\left(\gamma \right)\right.\hfill \\ \hfill & \left.-\left[\alpha {\left(\int f_i{(y,\gamma )}^{\alpha +1}dy\right)}^{\frac{\alpha }{\alpha +1}-1}\int f_i{(y,\gamma )}^{\alpha +1}\frac{\partial log{f}_i(y,\gamma )}{\partial \varphi }dy\right]f_i{(Y_i,\gamma )}^{\alpha }\right\}\hfill \\ \hfill & =\frac{1}{L_{\alpha }^i{\left(\gamma \right)}^2}\left\{\alpha f_i{(Y_i,\gamma )}^{\alpha }\frac{\partial log{f}_i(Y_i,\gamma )}{\partial \varphi }{L}_{\alpha }^i\left(\gamma \right)\right.\hfill \\ \hfill & \left.-\left[\alpha \frac{L_{\alpha }^i\left(\gamma \right)}{\int f_i{(y,\gamma )}^{\alpha +1}dy}\int f_i{(y,\gamma )}^{\alpha +1}\frac{\partial log{f}_i(y,\gamma )}{\partial \varphi }dy\right]f_i{(Y_i,\gamma )}^{\alpha }\right\}.\hfill \end{array}$$

and thus, the estimating equation for $\varphi$ is given by

$${\displaystyle \sum _{i=1}^n}\frac{1}{L_{\alpha }^i\left(\gamma \right)}\left\{M_i^{*}\left(Y_i,\gamma \right)-N_i^{*}\left(Y_i,\gamma \right)\right\}=0$$ (9)

being

$$M_i^{*}\left(Y_i,\gamma \right)=f_i{(Y_i,\gamma )}^{\alpha }{K}_{2i}\left(Y_i,\gamma \right),$$

and

$$N_i^{*}\left(Y_i,\gamma \right)=\frac{f_i{(Y_i,\gamma )}^{\alpha }}{\int f_i{(y,\gamma )}^{\alpha +1}dy}\int f_i{(y,\gamma )}^{\alpha +1}{K}_{2i}\left(y,\gamma \right)dy.$$

Under some regularity conditions, Castilla et al. [27] established the consistency and asymptotic normality of the MRPEs under the INIDO setup. Before stating the consistence and asymptotic distribution of the MRPEs for the GLM, let us introduce some useful notation. We define

$$\begin{array}{cc}\hfill S_{\alpha }^i& =\int f_i{(y,\mathit{\beta },\varphi )}^{\alpha +1}dy\hfill \\ \hfill m_{jli}\left(\gamma \right)& =\frac{1}{\int f_i{(y,\gamma )}^{\alpha +1}dy}\int f_i{(y,\gamma )}^{\alpha +1}{K}_{ji}\left(y,\gamma \right)K_{li}\left(y,\gamma \right)dy,\hfill \\ \hfill m_{ji}\left(\gamma \right)& =\frac{1}{\int f_i{(y,\gamma )}^{\alpha +1}dy}\int f_i{(y,\mathit{\beta },\varphi )}^{\alpha +1}{K}_{ji}\left(y,\gamma \right)dy,\hfill \\ \hfill l_{jli}\left(\gamma \right)& =\int \frac{f_i{(y,\gamma )}^{2\alpha +1}}{L_{\alpha }^i{\left(\gamma \right)}^2}\left(K_{ji}\left(y,\gamma \right)-m_{ji}\left(\gamma \right)\right)\left(K_{li}\left(y,\gamma \right)-m_{li}\left(\gamma \right)\right)dy,\hfill \end{array}$$ (10)

for all $j,l=1,2$ and $i=1,\dots ,n.$

Theorem 1.

Let $Y_1,\dots ,Y_n$ be a random sample from the GLM defined in (1). The MRPE ${\widehat{\gamma }}_{\alpha }={({\widehat{\mathit{\beta }}}_{\alpha }^T,{\widehat{\varphi }}_{\alpha })}^T$ is consistent and its asymptotic distribution is given by

$$\sqrt{n}{\mathsf{\Omega }}_n{\left(\gamma \right)}^{-\frac{1}{2}}{\mathsf{\Psi }}_n\left(\gamma \right)\left(({\widehat{\mathit{\beta }}}_{\alpha },{\widehat{\varphi }}_{\alpha })-(\mathit{\beta },\varphi )\right)\underset{n\to \infty }{\stackrel{L}{\to }}N({\mathbf{0}}_{k+1},{\mathit{I}}_{k+1}),$$

where $\mathit{X}$ denotes the design matrix, ${\mathit{I}}_k$ is the k-dimensional identity matrix and the matrices ${\mathsf{\Psi }}_n$ and ${\mathsf{\Omega }}_n$ are defined by

$${\mathsf{\Omega }}_n\left(\gamma \right)=\frac{1}{n}\left(\begin{array}{cc}{\mathit{X}}^T{\mathit{D}}_{11}\mathit{X}& {\mathit{X}}^T{\mathit{D}}_{12}\mathbf{1}\\ {\mathbf{1}}^{\mathit{T}}{\mathit{D}}_{12}\mathit{X}& {\mathbf{1}}^T{\mathit{D}}_{22}\mathbf{1}\end{array}\right),$$

$${\mathsf{\Psi }}_n\left(\gamma \right)=\frac{1}{n}\left(\begin{array}{cc}{\mathit{X}}^T\left({\mathit{D}}_{11}^{*}-{\left({\mathit{D}}_1^{*}\right)}^T{\mathit{D}}_1^{*}\right)\mathit{X}& {\mathit{X}}^T\left({\mathit{D}}_{12}^{*}-{\left({\mathit{D}}_1^{*}\right)}^T{\mathit{D}}_2^{*}\right)\mathbf{1}\\ {\mathbf{1}}^T\left({\mathit{D}}_{12}^{*}-{\left({\mathit{D}}_1^{*}\right)}^T{\mathit{D}}_2^{*}\right)\mathit{X}& {\mathbf{1}}^T\left({\mathit{D}}_{22}^{*}-{\left({\mathit{D}}_2^{*}\right)}^T{\mathit{D}}_2^{*}\right)\mathbf{1}\end{array}\right),$$

with

$${\mathit{D}}_{jk}=diag{\left(l_{jki}\left(\gamma \right)\right)}_{i=1,\dots ,n,j,k=1,2}{\mathit{D}}_{jk}^{*}=diag{\left(m_{jki}\left(\gamma \right)\right)}_{i=1,\dots ,n}$$

and

$${\mathit{D}}_j^{*}=diag{\left(m_{ji}\left(\gamma \right)\right)}_{i=1,\dots ,n},,j.k=1,2.$$

Proof. 

The consistency is proved for general statistical models in Castilla et al. [27] and the asymptotic distribution of the MRPEs for GLM is derived in Jaenada and Pardo [3]. □

3. Wald Type Tests for the GLMs

In this section, we define Wald-type tests for linear null hypothesis of the form

$$H_0:{\mathit{M}}^T\gamma =\mathit{m}\mathrm{vs}{H}_1:{\mathit{M}}^T\gamma \ne \mathit{m}$$ (11)

being $\gamma ={({\mathit{\beta }}^T,\varphi )}^T,$$\mathit{M}$ a $(k+1)\times r$ full rank matrix and

$$\mathit{m}={\left(m_1,\dots ,m_r\right)}^T$$ (12)

a r-dimensional vector $(r\le k+1)$. If the nuisance parameter $\varphi$ is known, as with logistic and Poisson regression, the matrix $\mathit{M}={\mathit{L}}_{k\times r}.$ Additionally, choosing

$$\mathit{M}=\left({\mathit{L}}_{k\times r},{\mathbf{O}}_{1\times r}\right)$$

gives rise to a null hypothesis defined by a linear combination of the regression coefficients, $\mathit{\beta }$, with $\varphi$ known or unknown. Further, the simple null hypothesis is a particular case when choosing $\mathit{M}$ as the identity matrix of rank k,

$$H_0:\mathit{\beta }={\mathit{\beta }}_0\mathrm{vs}{H}_1:\mathit{\beta }\ne {\mathit{\beta }}_0$$

with $\mathit{m}={\mathit{\beta }}_0={\left({\beta }_1^0,\dots ,{\beta }_k^0\right)}^T.$

In the following we assume that there exist a matrix ${\mathit{A}}_{\alpha }\left(\gamma \right)$ verifying

$$\underset{n\to \infty }{lim}{\mathsf{\Psi }}_n\left(\gamma \right){\mathsf{\Omega }}_n{\left(\gamma \right)}^{-1}{\mathsf{\Psi }}_n\left(\gamma \right)={\mathit{A}}_{\alpha }\left(\gamma \right).$$

Definition 1.

Let ${\widehat{\gamma }}_{\alpha }={({\widehat{\mathit{\beta }}}_{\alpha }^T,{\widehat{\varphi }}_{\alpha })}^T$ be the MRPE of $\gamma ={({\mathit{\beta }}^T,\varphi )}^T$ for the GLM. The Wald-type tests, based on the MRPE, for testing (11) are defined by

$$W_n\left({\widehat{\gamma }}_{\alpha }\right)=n{\left({\mathit{M}}^T{\widehat{\gamma }}_{\alpha }-\mathit{m}\right)}^T{\left({\mathit{M}}^T{\mathsf{\Psi }}_n{\left({\widehat{\gamma }}_{\alpha }\right)}^{-1}{\mathsf{\Omega }}_n\left({\widehat{\gamma }}_{\alpha }\right){\mathsf{\Psi }}_n{\left({\widehat{\gamma }}_{\alpha }\right)}^{-1}\mathit{M}\right)}^{-1}\left({\mathit{M}}^T{\widehat{\gamma }}_{\alpha }-\mathit{m}\right).$$ (13)

The following theorem presents the asymptotic distribution of the Wald-type test statistics, $W_n\left({\widehat{\gamma }}_{\alpha }\right).$

Theorem 2.

The Wald-type test $W_n\left({\widehat{\gamma }}_{\alpha }\right)$ follows asymptotically, under the null hypothesis presented in (11), a chi-square distribution with degrees of freedom equal to the dimension of the vector $\mathit{m}$ in (12)

Under the null hypothesis given in (11) the asymptotic distribution of the Wald-type test statistics is a chi-square distribution with r degrees of freedom.

Proof. 

We know that

$$\sqrt{n}\left({({\widehat{\mathit{\beta }}}_{\alpha }^T,{\widehat{\varphi }}_{\alpha })}^T-{({\mathit{\beta }}^T,\varphi )}^T\right)\underset{n\to \infty }{\stackrel{L}{\to }}N({\mathbf{0}}_{k+1},{\mathit{A}}_{\alpha }{\left(\gamma \right)}^{-1}).$$

Therefore,

$$\sqrt{n}\left({\mathit{M}}^T{\widehat{\gamma }}_{\alpha }-\mathit{m}\right)=\sqrt{n}\mathit{M}\left({\widehat{\gamma }}_{\alpha }-\gamma \right)\underset{n\to \infty }{\stackrel{L}{\to }}N({\mathbf{0}}_{k+1},{\mathit{M}}^T{\mathit{A}}_{\alpha }{\left(\gamma \right)}^{-1}\mathit{M}).$$

Now, the result follows taking into account that ${\widehat{\gamma }}_{\alpha }$ is a consistent estimator of ${\gamma }_0.$ □

Based on the previous convergence, the null hypothesis in (11) is rejected, if

$$W_n\left({\widehat{\gamma }}_{\alpha }\right)>{\chi }_{r,\alpha }^2$$ (14)

being ${\chi }_{r,\alpha }^2$ the $100(1-\alpha )$ percentile of a chi-square distribution with r degrees of freedom.

Finally, let ${\gamma }_1$ be a parameter point verifying ${\mathit{M}}^T{\gamma }_1\ne \mathit{m},$ i.e., ${\gamma }_1$ is not on the null hypothesis. The next result establishes that the Wald-type tests given in (14) are consistent (see Fraser [29]).

Theorem 3.

Let ${\gamma }_1$ be a parameter point verifying ${\mathit{M}}^T{\gamma }_1\ne \mathit{m}.$ Then the Wald-type tests given in (14) are consistent, i.e.,

$$\underset{n\to \infty }{lim}{P}_{{\gamma }_1}\left(W_n\left({\widehat{\gamma }}_{\alpha }\right)>{\chi }_{r,\alpha }^2\right)=1.$$

Proof. 

See Appendix A. □

Remark 1.

In the proof of the previous Theorem was established the approximate power function of the Wald-type tests defined in (13),

$${\pi }_{W_n\left({\widehat{\gamma }}_{\alpha }\right)}\left({\gamma }_1\right)\approx 1-{\varphi }_{\mathcal{N}(0,1)}\left(\frac{1}{\sigma \left({\gamma }_1\right)}\left(\frac{{\chi }_{r,\alpha }^2}{\sqrt{n}}-\frac{W_n\left({\gamma }_1\right)}{\sqrt{n}}\right)\right)$$

where

$${\sigma }^2\left({\gamma }_1\right)={\left(\frac{\partial l_{{\widehat{\gamma }}_{\alpha }}\left(\zeta \right)}{\partial {\zeta }^T}\right)}_{\gamma ={\gamma }_1}{\mathit{A}}_{\alpha }{\left({\gamma }_1\right)}^{-1}{\left(\frac{\partial l_{{\widehat{\gamma }}_{\alpha }}\left(\zeta \right)}{\partial \zeta }\right)}_{\gamma ={\gamma }_1}$$

and

$$l_{{\widehat{\gamma }}_{\alpha }}\left(\zeta \right)={\left({\mathit{M}}^T{\widehat{\gamma }}_{\alpha }-\mathit{m}\right)}^T{\left({\mathit{M}}^T{\mathit{A}}_{\alpha }{\left(\zeta \right)}^{-1}\mathit{M}\right)}^{-1}\left({\mathit{M}}^T{\widehat{\gamma }}_{\alpha }-\mathit{m}\right).$$

From the above expression, the necessary sample size n for the Wald-type tests to have a predetermined power, ${\pi }_0,$ is given by $n=\left[n^{*}\right]+1$, with

$$n^{*}=\frac{A+B+\sqrt{A(A+2B)}}{2l_{{\gamma }_1}^2\left({\gamma }_1\right)}$$

being

$$A={\sigma }^2\left({\gamma }_1\right){\left({\varphi }^{-1}(1-{\pi }_0)\right)}^2,B=2{\chi }_{r,\alpha }^2{l}_{{\gamma }_1}\left({\gamma }_1\right)$$

and $[·]$ the integer part.

In accordance with Maronna et al. [30], the breakdown point of the estimators ${\widehat{\gamma }}_{\alpha }$ of a parameter $\gamma$ is the largest amount of contamination that the data may contain such that ${\widehat{\gamma }}_{\alpha }$ still gives enough information about $\gamma .$ The derivation of a general breakdown points it is in general not easy, so it may deserve a separate paper where it may be jointly considered the replacement finite-sample breakdown point introduced by Donoho and Huber [31]. Although breakdown point is an important theoretical concept in robust statistics, perhaps is more useful the definition of breakdown point associated to a finite sample: replacement finite-sample break down point. More details can be seen in Section 3.2.5 of Maronna et al. [30].

4. Influence Function

We derive in this section the IF of the MRPEs of the parameters $\gamma ={({\mathit{\beta }}^T,\varphi )}^T$ and Wald-type statistics based on these MRPEs, $W_n\left({\widehat{\gamma }}_{\alpha }\right).$ The influence function (IF) of an estimator quantifies the impact of an infinitesimal perturbation in the true distribution of the data on the asymptotic value of the resulting parameter estimate (in terms of the corresponding statistical functional). An estimator is said to be robust if its IF is bounded. If we denote $\mathit{G}=\left(G_1,\dots ,G_n\right)$ the true distributions underlying the data, the functional ${\mathit{T}}_{\alpha }\left(\mathit{G}\right)$ and associated to the MRPE for the parameters $\gamma$ is such that

$$\frac{1}{n}\sum _{i=1}^n{R}_{\alpha }\left(f_i(y,{\mathit{T}}_{\alpha }\left(\mathit{G}\right)),g_i\left(y\right)\right)=\underset{\gamma }{min}\frac{1}{n}\sum _{i=1}^n{R}_{\alpha }\left(f_i(y,\gamma ),g_i\left(y\right)\right).$$

The IF of a estimator is defined as the limiting standardized bias due to infinitesimal contamination. That is, given a contaminated distribution at the point $(y_t,{\mathit{x}}_t)$, ${\mathit{G}}_{\epsilon }=(1-\epsilon )\mathit{G}+\epsilon {\Delta }_{(y_t,{\mathit{x}}_t)}$ with ${\Delta }_{(y_t,{\mathit{x}}_t)}$ the degenerate distribution at $(y_t,{\mathit{x}}_t)$, the IF of the estimator ${\widehat{\gamma }}_{\alpha }$ in terms of its associated functional ${\mathit{T}}_{\alpha }\left(\mathit{G}\right)$ is computed as

$$IF\left((y_t,{\mathit{x}}_t),{\mathit{T}}_{\alpha }\left(\mathit{G}\right)\right)=\underset{\epsilon \to 0}{lim}\frac{{\mathit{T}}_{\alpha }\left({\mathit{G}}_{\epsilon }\right)-{\mathit{T}}_{\alpha }\left(\mathit{G}\right)}{\epsilon }.$$

In the following, let us denote ${\mathit{T}}_{\alpha }\left(\mathit{G}\right)=\left({\mathit{T}}_{\alpha }^{\mathit{\beta }}\left(\mathit{G}\right),T_{\alpha }^{\varphi }\left(\mathit{G}\right)\right),$ where ${\mathit{T}}_{\alpha }^{\mathit{\beta }}\left(\mathit{G}\right)$ and $T_{\alpha }^{\varphi }\left(\mathit{G}\right)$ are the functionals associated the parameters $\mathit{\beta }$ and $\varphi$, respectively. Then, they must satisfy the estimating equations of the MRPE given by

$$\begin{array}{cc}\hfill \sum _{i=1}^n\frac{{\mathit{x}}_i}{L_{\alpha }^i\left(({\mathit{T}}_{\alpha }^{\mathit{\beta }}\left(\mathit{G}\right),{\mathit{T}}_{\alpha }^{\varphi }\left(\mathit{G}\right))\right)}\left\{M_i\left(y_i,({\mathit{T}}_{\alpha }^{\mathit{\beta }}\left(\mathit{G}\right),{\mathit{T}}_{\alpha }^{\varphi }\left(\mathit{G}\right))\right)-N_i\left(y_i,({\mathit{T}}_{\alpha }^{\mathit{\beta }}\left(\mathit{G}\right),{\mathit{T}}_{\alpha }^{\varphi }\left(\mathit{G}\right))\right)\right\}& ={\mathbf{0}}_k\hfill \\ \hfill \sum _{i=1}^n\frac{1}{L_{\alpha }^i\left(({\mathit{T}}_{\alpha }^{\mathit{\beta }}\left(\mathit{G}\right),{\mathit{T}}_{\alpha }^{\varphi }\left(\mathit{G}\right))\right)}\left\{M_i^{*}\left(y_i,({\mathit{T}}_{\alpha }^{\mathit{\beta }}\left(\mathit{G}\right),{\mathit{T}}_{\alpha }^{\varphi }\left(\mathit{G}\right))\right)-N_i^{*}\left(y_i,({\mathit{T}}_{\alpha }^{\mathit{\beta }}\left(\mathit{G}\right),{\mathit{T}}_{\alpha }^{\varphi }\left(\mathit{G}\right))\right)\right\}& =0\hfill \end{array}$$ (15)

where the quantities $L_{{\alpha }^i\left(\gamma \right)},M_i\left(y_i,\gamma \right),N_i\left(y_i,\gamma \right),M_i^{*}\left(y_i,\gamma \right)$ and $N_i^{*}\left(y_i,\mathit{gamma}\right)$ are defined in Section 2. Now, evaluating the previous equation at the contaminated distribution ${\mathit{G}}_{\epsilon }$, implicitly differentiating the estimating equations in $\epsilon$ and evaluating them at $\epsilon =0,$ we can obtain the expression of the IF for the GLM.

We first derive the expression IF of MRPEs at the $i_0-th$ direction. For this purpose, we consider the contaminated distributions

$${\mathit{G}}_{i_0,\epsilon }=\left(G_1,\dots ,G_{i_0-1},G_{i_0,\epsilon },G_{i_0+1},\dots ,G_n\right),$$

with $G_{i_0,\epsilon }=(1-\epsilon )G_{i_0}+\epsilon {\Delta }_{(y_{i_0},{\mathit{x}}_{i_0})}.$ Here, only the $i_0$-th component of the vector of distributions is contaminated. If the true density function $g_i$ of each variable belongs to the exponential model, we have that

$$g_i\left(y\right)=\left\{\begin{array}{cc}{f}_i(y,\gamma )\hfill & i\ne i_0\\ \left(1-\epsilon \right)f_i(y,\gamma )+\epsilon {\Delta }_{(y_{i_0},{\mathit{x}}_{i_0})}\left(y\right)\hfill & i=i_0.\end{array}\right.$$

Accordingly, we define

$${\gamma }_{\epsilon }^{i_0}={\mathit{T}}_{\alpha }\left(G_1,\dots ,G_{i_0-1},G_{i_0,\epsilon },G_{i_0+1},\dots ,G_n\right)$$

the MRPE when the true distribution underlying the data is ${\mathit{G}}_{i_0,\epsilon }.$ Based on Remark 5.2 in Castilla et al. [27] the IF of the MRPE at the $i_0-th$ direction with $(y_{i_0},{\mathit{x}}_{i_0})$ the point of contamination is given by

$$\begin{array}{l}IF((y_{i_0},{\mathit{x}}_{i_0}),{\mathit{T}}_{\alpha },\mathit{G})={\left(\frac{\partial {\mathit{T}}_{\alpha }\left({\mathit{G}}_{i_0,{\gamma }_{\epsilon }}\right)}{\partial \epsilon }\right)}_{\epsilon =0}\hfill \\ ={\mathsf{\Psi }}_n{\left(\gamma \right)}^{-1}\frac{f_{i_0}{(y_{i_0},\gamma )}^{\alpha }}{\int f_{i_0}{(y,\gamma )}^{\alpha +1}dy}\left(\begin{array}{c}{K}_{1i}\left(y_{i_0},\gamma \right)-f_{i_0}{(y_{i_0},\gamma )}^{-\alpha }{N}_{i_0}\left(y_{i_0},\gamma \right)\\ K_{2i}\left(y_{i_0},\gamma \right)-f_{i_0}{(y_{i_0},\gamma )}^{-\alpha }{N}_{i_0}^{*}\left(y_{i_0},\gamma \right)\end{array}\right)\left(\begin{array}{cc}{\mathit{x}}_{i_0}& 0\\ 0& 1\end{array}\right).\hfill \end{array}$$

In a similar manner, the IF in all directions (i.e., all components of the vector of distributions are contaminated) has the following expression

$$\begin{array}{l}IF((y_1,{\mathit{x}}_1),\dots ,(y_n,{\mathit{x}}_n),{\mathit{T}}_{\alpha },\mathit{G})={\left(\frac{\partial {\mathit{T}}_{\alpha }\left({\mathit{G}}_{{\gamma }_{\epsilon }}\right)}{\partial \epsilon }\right)}_{\epsilon =0}\hfill \\ ={\mathsf{\Psi }}_n{\left(\gamma \right)}^{-1}\sum _{i=1}^n\left(\frac{f_i{(y_i,\gamma )}^{\alpha }}{\int f_i{(y,\gamma )}^{\alpha +1}dy}\left(\begin{array}{c}{K}_{1i}\left(y_i,\gamma \right)-f_i{(y_i,\gamma )}^{-\alpha }{N}_i\left(y_i,\gamma \right)\\ K_{2i}\left(y_i,\gamma \right)-f_i{(y_i,\gamma )}^{-\alpha }{N}_i^{*}\left(y_i,\gamma \right)\end{array}\right)\left(\begin{array}{cc}{\mathit{x}}_i& 0\\ 0& 1\end{array}\right)\right),\hfill \end{array}$$

with $(y_1,{\mathit{x}}_1),\dots ,(y_n,{\mathit{x}}_n)$ the point of contamination. We next derive the expression of the IF for the Wald-type tests presented in Section 3. The statistical functional associated with the Wald-type tests for the linear null hypothesis (11) at the distributions $\mathit{G}=\left(G_1,\dots ,G_n\right)$, ignoring the constant $n,$ is given by

$${\mathit{W}}_{\alpha }\left(\mathit{G}\right)={\left({\mathit{M}}^T{\mathit{T}}_{\alpha }\left(\mathit{G}\right)-\mathit{m}\right)}^T{\left({\mathit{M}}^T{\mathit{A}}_{\alpha }{\left({\mathit{T}}_{\alpha }\left(\mathit{G}\right)\right)}^{-1}\mathit{M}\right)}^{-1}\left({\mathit{M}}^T{\mathit{T}}_{\alpha }\left(\mathit{G}\right)-\mathit{m}\right).$$ (16)

Again, evaluating the Wald-type test functionals at the contaminated distribution ${\mathit{G}}_{\epsilon }$ and implicitly differentiating the expression, we can get the expression of it IF. In particular, the IF of the Wald-type test statistics at the $i_0$-th direction and the contamination point $(y_{i_0},{\mathit{x}}_0)$ is given by

$$\begin{array}{cc}\hfill I{F}_1\left((y_{i_0},{\mathit{x}}_0),{\mathit{W}}_{\alpha },\mathit{G}\right)& ={\left(\frac{\partial {\mathit{W}}_{\alpha }\left({\mathit{G}}_{i_0,\epsilon }\right)}{\partial \epsilon }\right)}_{\epsilon =0}\hfill \\ \hfill & =2{\left({\mathit{M}}^T{\mathit{T}}_{\alpha }\left(\mathit{G}\right)-\mathit{m}\right)}^T{\left({\mathit{M}}^T{\mathit{A}}_{\alpha }{\left({\mathit{T}}_{\alpha }\left(\mathit{G}\right)\right)}^{-1}\mathit{M}\right)}^{-1}{\mathit{M}}^{T}IF((y_{i_0},{\mathit{x}}_0),{\mathit{T}}_{\alpha },\mathit{G}).\hfill \end{array}$$

Evaluating the previous expression at the null hypothesis, ${\mathit{M}}^T{\mathit{T}}_{\alpha }\left(\mathit{G}\right)=\mathit{m},$ the IF becomes identically zero,

$$I{F}_1\left((y_{i_0},{\mathit{x}}_{i_0}){\mathit{W}}_{\alpha },\mathit{G}\right)={\mathbf{0}}_{k+1}.$$

Therefore, it is necessary to consider the second order IF of the proposed Wald-type tests. Twice differentiating in ${\mathit{W}}_{\alpha }\left({\mathit{G}}_{\epsilon }\right)$, we get

$$\begin{array}{cc}\hfill I{F}_2\left((y_{i_0},{\mathit{x}}_{i_0}),{\mathit{W}}_{\alpha },\mathit{G}\right)& ={\left(\frac{{\partial }^2{\mathit{W}}_{\alpha }\left({\mathit{G}}_{i_0,\epsilon }\right)}{\partial {\epsilon }^2}\right)}_{\epsilon =0}\hfill \\ \hfill & =2IF{((y_{i_0},{\mathit{x}}_{i_0}),{\mathit{T}}_{\alpha },{\mathbf{F}}_{\mathit{\beta }})}^T\mathit{M}{\left({\mathit{M}}^T{\mathit{A}}_{\alpha }{\left({\mathit{T}}_{\alpha }\left(\mathit{G}\right)\right)}^{-1}\mathit{M}\right)}^{-1}{\mathit{M}}^{T}IF((y_{i_0},{\mathit{x}}_{i_0}),{\mathit{T}}_{\alpha },\mathit{G}).\hfill \end{array}$$

Finally, the second order IF of the Wald-type tests in all directions is given by

$$\begin{array}{cc}\hfill I{F}_2\left((y_1,{\mathit{x}}_1),\dots ,(y_n,{\mathit{x}}_n),{\mathit{W}}_{\alpha },\mathit{G}\right)& ={\left(\frac{{\partial }^2{\mathit{W}}_{\alpha }\left({\mathit{G}}_{\epsilon }\right)}{\partial {\epsilon }^2}\right)}_{\epsilon =0}\hfill \\ \hfill & =2IF{((y_1,{\mathit{x}}_1),\dots ,(y_n,{\mathit{x}}_n),{\mathit{T}}_{\alpha },\mathit{G})}^T\mathit{M}{\left({\mathit{M}}^T{\mathit{A}}_{\alpha }{\left({\mathit{T}}_{\alpha }\left(\mathit{G}\right)\right)}^{-1}\mathit{M}\right)}^{-1}{\mathit{M}}^T\hfill \\ \hfill & ·IF((y_1,{\mathit{x}}_1),\dots ,(y_n,{\mathit{x}}_n),{\mathit{T}}_{\alpha },\mathit{G}).\hfill \end{array}$$

To asses the robustness of the MRPEs and Wald-type test statistics we must discuss the boundedness of the corresponding IF. The boundedness of the second order IF of the Wald-type test statistics is determined by the boundedness of the IF of the MRPEs. Further, the matrix ${\mathsf{\Psi }}_n\left(\gamma \right)$ is assumed to be bounded, so the robustness of the estimators only depend on the second factor of the IF. Most standard GLMs enjoy such properties for positives values of $\alpha$, but the influence function is unbounded at $\alpha =0,$ corresponding with the MLE. As an illustrative example, Figure 1 plots the IF of the MRPEs for the Poisson regression model with different values of $\alpha =0,0.5$ at one direction. The model is fitted with only one covariate, the parameter $\varphi$ is known for Poisson regression ($\varphi =1$) and the true regression vector is fixed $\mathit{\beta }=1.$ As shown, the IF of the MRPEs with positives values of $\alpha$ are bounded, whereas the IF of the MLE is not, indicating it lack of robustness.

Figure 1.

IF of MRPEs with $\alpha =0$ (left) and $\alpha =0.5$ (right) of Poisson regression model.

5. Numerical Analysis: Poisson Regression Model

We illustrate the proposed robust method for the Poisson regression model. As pointed out in Section 1 the Poisson regression model belongs to the GLM with known shape parameter $\varphi =1,$ location parameter ${\theta }_i={\mathit{x}}_i^T\mathit{\beta }$ and known functions $b\left({\theta }_i\right)=exp\left({\mathit{x}}_i^T\mathit{\beta }\right)$ and $c\left(y_i\right)=-log(y_i!)$. Since the nuisance parameter is known, for the seek of simplicity in the following we only use $\mathit{\beta }=\gamma .$ In Poisson regression, the mean of the response variable is linked to the linear predictor through the natural logarithm, i.e., ${\mu }_i=exp\left({\mathit{x}}_i^T\mathit{\beta }\right).$ Thus, we can apply the previous proposed method to estimate the vector of regression parameters $\mathit{\beta }$ with objective function given in Equation (5).

The results provided are computed in the software R. The minimization of the objective function is performed using the implemented optim() function, which applies the Nelder–Mead iterative algorithm (Nelder and Mead [32]). Nelder–Mead optimization algorithm is robust although relatively slow. The corresponding objective function $T_n^{\alpha }\left(\gamma \right)$ given in (5) is highly nonlinear and requires the evaluation of nontrivial quantities. Further, the computation of the Wald-type test statistics defined in (13) requires to evaluate the covariance matrix of the MRPEs, involving nontrivial integrals. Some simplified expressions of the main quantities defined throughout the paper for the Poisson regression model, such as $L_{\alpha }^i\left(\mathit{\beta }\right),K_{1i}(y,\mathit{\beta }),N_i(y,\mathit{\beta }),m_{1i}\left(\mathit{\beta }\right),m_{11i}\left(\mathit{\beta }\right)$ or $l_{11i}\left(\mathit{\beta }\right),$ are given in the Appendix B. There is no closed expression for these quantities, and they need to be approximated numerically. Since the minimization is iteratively performed, computing such expressions at each step of the algorithm and for each observation may entail an increased computational burden. Nonetheless, the complexity is not significant for low-dimensional data. On the other hand, the optimum in (5) need not to be uniquely defined, since the objective function may have several local minima. Then, the choice of the initial value of the iterative algorithm is crucial. Ideally, a good initial point should be consistent and robust. In our results the MLE is used as initial estimate for the algorithm.

We analyze the performance of the proposed methods in Poisson regression through a simulation study. We asses the behavior of the MRPE under the sparse Poisson regression model with $k=12$ covariates but only 3 significant variables. We set the 12-dimensional regression parameter $\mathit{\beta }=(1.8,1,0,0,1.5,0,\dots 0)$ and we generate the explanatory variables, ${\mathit{x}}_i$, from the standard uniform distribution with variance-covariance matrix having Toeplitz structure, with the $(j,l)$-th element being $0.5^{|j-l|},j,l=1,\dots ,p$. The response variables are generated from the Poisson regression model with mean ${\mu }_i={\mathit{x}}_i^T\mathit{\beta },$$Y_i\sim \mathcal{P}\left({\mu }_i\right).$ To evaluate the robustness of the proposed estimators, we contaminate the responses using a perturbed distribution of the form $(1-b)\mathcal{P}\left({\mu }_i\right)+b\mathcal{P}\left(2{\mu }_i\right),$ where b is a realization of a Bernoulli variable with parameter $\epsilon$ so called the contamination level. That is, the distribution of the contaminated responses lies in a small neighbourhood of the assumed model. We repeat the process $R=1000$ for each value of $\alpha$.

Figure 2 presents the mean squared error of the estimate (MSE), $MSE=\left|\right|{\widehat{\mathit{\beta }}}_{\alpha }-\mathit{\beta }{\left|\right|}_2,$ (left) and the MSE on the prediction (right) against contamination level on data for different values of $\alpha =0,0.1,0.3,0.5$ and $0.7$. The sample size is fixed at $n=200$ and the MSE on the prediction is calculated using $n=200$ new observations following the true model. As shown, greater values of $\alpha$ correspond to more robust estimators, revealing the role of the tuning parameter on the robustness gain. Most strikingly, the MSE grows linearly for the MLE, while the proposed estimators manage to maintain a low error in all contaminated scenarios.

Figure 2.

Mean Squared Error (MSE) on estimation (left) and prediction (right) against contamination level on data.

Furthermore, it is to be expected that the error of the estimate decreases with larger samples sizes. In this regard, Figure 3 shows the MSE for different values of $\alpha =0,0.1,0.3,0.5$ and $0.7,$ against the sample size in the absence of contamination (left) and under $5\%$ of contamination. Our proposed estimators are more robust than the classical MLE with almost all contaminated scenarios, since the MSE committed is lower for all positives values of $\alpha$ than for $\alpha =0$ (corresponding to the MLE), except for too small sample sizes. Conversely, the MLE is, as expected, the most efficient estimator in absence of contamination, closely to our proposed estimators with $\alpha =0.1,0.3$, highlighting the importance of $\alpha$ in controlling the trade-off between efficiency and robustness. In this regard, values of $\alpha$ about $0.3$ perform the best taking into account the low loss of efficiency and the gain in robustness. Finally, note that small sample sizes adversely affect to greater values of $\alpha$.

Figure 3.

MSE in estimation of $\mathit{\beta }$ in absence of contamination (left) and under $5\%$ of contamination level in data (right) with different values of $\alpha$ against sample size for Poisson regression model.

On the other hand, one could be interested on testing the significance of the selected variables. For this purpose, we simplify the true model and we examine the performance of the proposed Wald-type test statistics under different true coefficients values. In particular, let us consider a Poisson regression model with only two covariates, generated from the uniform distribution as before, and the linear null hypothesis

$$H_0:{\mathit{\beta }}_2=0.$$ (17)

That is, we are interested in assessing the significance of the second variable. The sample size if fixed at $n=200$ and the true value of the component of the regression vector is set ${\mathit{\beta }}_1=1.$ We study the power of the tests under increasing signal of the second parameter ${\mathit{\beta }}_2$ and increasing contamination level. Here, the model is contaminated by perturbing the true distribution with $(1-b)\mathcal{P}\left({\mu }_i\right)+b\mathcal{P}\left({\tilde{\mu }}_i\right),$ where ${\mu }_i={\mathit{x}}_i^T\mathit{\beta }$ is the mean of the Poisson variable in the absence of contamination, ${\tilde{\mu }}_i={\mathit{x}}_i^T\tilde{\mathit{\beta }}$ is the contaminated mean, with $\tilde{\mathit{\beta }}=(1,0),$ and b is a realization of a Bernoulli variable with probability of success $\epsilon .$ Table 1 presents the rejection rate of the Wald-type test statistics for different true values of ${\mathit{\beta }}_2$ under different contaminated scenarios. As expected, stronger signals produce higher power for all Wald-type test. Moreover, the power of the Wald-type test statistics based on the MLE decreases when increasing the contamination, whereas the power of the statistics based on the MRPEs with positives values of $\alpha$ keeps sufficiently high. Then, our proposed robust estimators are able to detect the significance of the variable even in heavily contaminated scenarios.

Table 1.

Rejection rate of Wald-type test statistics based on MRPEs with different true values of ${\mathit{\beta }}_2$ and contamination levels.

${\mathit{\beta }}_2$	$\mathit{\alpha }$	Contamination Level
		0	$5\%$	$10\%$	$15\%$	$20\%$	$25\%$
$0.3$	0	0.332	0.264	0.227	0.187	0.157	0.141
	0.1	0.435	0.376	0.328	0.285	0.251	0.223
	0.3	0.557	0.511	0.483	0.416	0.390	0.360
	0.5	0.617	0.563	0.533	0.493	0.467	0.427
	0.7	0.638	0.590	0.568	0.536	0.513	0.476
$0.5$	0	0.756	0.730	0.683	0.621	0.551	0.493
	0.1	0.833	0.798	0.775	0.736	0.681	0.622
	0.3	0.885	0.870	0.864	0.829	0.792	0.752
	0.5	0.895	0.891	0.886	0.867	0.842	0.814
	0.7	0.901	0.897	0.893	0.879	0.854	0.832
$0.7$	0	0.971	0.979	0.968	0.948	0.915	0.862
	0.1	0.980	0.988	0.983	0.973	0.962	0.932
	0.3	0.988	0.995	0.992	0.987	0.985	0.969
	0.5	0.989	0.995	0.995	0.992	0.992	0.977
	0.7	0.989	0.995	0.993	0.995	0.990	0.983

6. Real Data Applications

6.1. Example I: Poisson Regression Regression

We finally apply our proposed estimators in a real dataset arising from Crohn’s disease. The data were first studied in Lô and Ronchetti [33] to asses the adverse events of a drug. The clinical study included 117 patients affected by the disease, for whom information was recorded for 7 explanatory variables: BMI (body mass index), HEIGHT, COUNTRY (one of the two countries where the patient lives), SEX, AGE, WEIGHT, and TREAT (the drug taken by the patient in factor form: placebo, Dose 1, Dose 2), in addition to the response variable AE (number of adverse events). Lô and Ronchetti [33] considered a Poisson regression model for the Crohn data and determined that only variables Dose 1, BMI, HEIGHT, SEX, AGE, and COUNTRY may be essentially significant. Further, they flagged observations 23rd, 49th, and 51st to be highly influential on the classical analysis. Table 2 presents the estimated coefficient of the explanatory variable when fitting the Poisson regression model. Robust methods suggest higher coefficients for the variables BMI and AGE, whereas fewer values for the coefficients of the categorical variables COUNTRY, SEX, Dose 1.

Table 2.

Estimated coefficients for Crohn’s disease data for different values of $\alpha$ with original data and clean data (after removing influential observations).

	Intercept	BMI	Height	Age	Country	Sex	Dose 1
Original Data
MLE ($\alpha =$ 0)	6.261	0.026	−0.037	0.012	−0.394	−0.646	−0.533
$\alpha =$ 0.1	5.197	0.037	−0.033	0.014	−0.489	−0.800	−0.469
$\alpha =$ 0.3	4.798	0.058	−0.036	0.021	−0.545	−1.284	−0.832
$\alpha =$ 0.5	4.391	0.067	−0.037	0.028	−0.557	−1.535	−1.036
$\alpha =$ 0.7	5.699	0.067	−0.047	0.036	−0.737	−1.759	−1.157

Following the discussion in Lô and Ronchetti [33], classical tests may not select variable AGE to be significant. Then, we propose testing the significance of that variable using Wald-type test statics based on different values $\alpha$. Table 3 shows the p-values of the corresponding tests with null hypothesis $H_0$: AGE = 0, with the original data and after removing the outlying observations.

Table 3.

p-values of test with null hypothesis $H_0$: AGE = 0 with original and clean data (after removing influential observations).

	Original Data	Clean Data
MLE ($\alpha =$ 0)	0.059	0.011
$\alpha =$ 0.1	0.018	0.004
$\alpha =$ 0.3	0.001	0.000
$\alpha =$ 0.5	0.000	0.000
$\alpha =$ 0.7	0.000	0.000

The MLE rejects the significance of the variable AGE when the original data are used, whereas the Wald-type test statistics with positives values of $\alpha$ indicate strong evidence against the null hypothesis. In contrast, if the influential observations are removed, all Wald-type test statistics agree in the significance of the variable. This example illustrates the robustness of the proposed statistics.

6.2. Example II: Binomial Regression

We finally illustrate the applicability of the MRPE for robust inference in the binomial regression model. We examine the damaged carrots dataset, first studied in Phelps [34] and later discussed by Cantoni and Ronchetti [8] and Ghosh and Basu [13] to illustrate robust procedures for binomial regression. The data contain 24 samples, among which the 14th observation was flagged as an outlier in the y-space but not a leverage point. The data are issued from a soil experiment and give the proportion of carrots showing insect damage in a trial with three blocks and eight dose levels of insecticide. The explanatory variables are the logarithm transform of the dose (Logdose) and two dummy variables for Blocks 1 and 2.

Binomial regression is a natural extension of the logistic regression when the response variable Y does not follow a Bernoulli distribution but a Binomial distribution counting the number of successes in a series of m independent Bernoulli trials. Binomial regression model belongs to the GLM with known shape parameter $\varphi =1,$ location parameter ${\theta }_i={\mathit{x}}_i^T\mathit{\beta }$ and functions $b\left({\theta }_i\right)=mlog\left(1+exp\left({\mathit{x}}_i^T\mathit{\beta }\right)\right)$ and $c\left(y_i\right)=log\left(\left(\genfrac{}{}{0pt}{}{m}{y_i}\right)\right)$. The mean of the response variable is then linked to the linear predictor through the logit function, i.e.,

$$log\left(\frac{{\mu }_i}{m-{\mu }_i}\right)={\mathit{x}}_i^T\mathit{\beta }.$$

Table 4 presents the estimated coefficients of the regression vector for the carrots data using the MLE and robust MRPEs when the model is fitted with the original data and the model fitted without the outlying observation. The results provided are computed in the same manner as in Section 5, adapting the corresponding quantities in Equation (5) for the binomial model. All integrals involved were numerically approximated, and the MLE is used as initial estimate for the optimization algorithm. The influence of observation 14 stands out when using the MLE; the estimated coefficients are remarkably different when fitting the model with and without observation 14. In contrast, all methods estimate similar coefficients after removing the outlying observation, coinciding with the robust estimates for moderately high values of the tuning parameter $\alpha$.

Table 4.

Estimated coefficients for damaged carrots data for different values of $\alpha$ with original data and clean data (after outliers removal).

	Intercept	Logdose	B1	B2
Original Data
MLE ($\alpha =0$)	1.480	−1.817	0.542	0.843
$\alpha =$ 0.1	1.729	−1.949	0.527	0.755
$\alpha =$ 0.3	2.017	−2.100	0.479	0.652
$\alpha =$ 0.5	2.090	−2.134	0.386	0.625
$\alpha =$ 0.7	2.150	−2.161	0.258	0.615
Clean Data
MLE ($\alpha =0$)	2.141	−2.179	0.546	0.636
$\alpha =$ 0.1	2.126	−2.167	0.529	0.633
$\alpha =$ 0.3	2.105	−2.149	0.479	0.627
$\alpha =$ 0.5	2.108	−2.144	0.385	0.621
$\alpha =$ 0.7	2.154	−2.163	0.257	0.614

7. Conclussions

In this paper, we presented the MRPE and Wald-type test statistics for GLMs. The proposed MRPEs and statistics have appealing robustness properties where the data are contaminated due to outliers or leverage points. MRPEs are consistent and asymptotically normal and represent an attractive alternative to the classical nonrobust methods. Additionally, robust Wald-type test statistics, based on the MRPEs, were developed. Through the study of the IFs and the development of an extensive simulation study, we proved their robustness from a theoretical and practical point of view, respectively. In particular, we illustrated the superior performance of the MRPEs and the corresponding Wald-type tests for the Poisson regression model.

Abbreviations

The following abbreviations are used in this manuscript:

DPD	Density Power Divergence
IF	Influence Function
GLM	Genelarized Linear Model
LRM	Linear Regression Model
MLE	Maximum Likelihood Estimator
MRPE	Minimum Rényi Pseudodistance Estimator
RP	Rényi Pseudodistance

Appendix A. Proof of Theorem 3

Let us define

$$l_{\eta }\left(\zeta \right)={\left({\mathit{M}}^T\eta -\mathit{m}\right)}^T{\left({\mathit{M}}^T{\mathit{A}}_{\alpha }{\left(\zeta \right)}^{-1}\mathit{M}\right)}^{-1}\left({\mathit{M}}^T\eta -\mathit{m}\right)$$

so the Wald-type test statistic is such that

$$n{l}_{{\widehat{\gamma }}_{\alpha }}\left({\widehat{\gamma }}_{\alpha }\right)=W_n\left({\widehat{\gamma }}_{\alpha }\right).$$

We know that ${\widehat{\gamma }}_{\alpha }\underset{n\to \infty }{\stackrel{P}{\to }}{\gamma }_1$ and therefore $l_{{\widehat{\gamma }}_{\alpha }}\left({\gamma }_1\right)$ and $l_{{\gamma }_1}\left({\gamma }_1\right)$ have the same asymptotic distribution. A first order Taylor expansion of $g\left(\zeta \right)=l_{{\widehat{\gamma }}_{\alpha }}\left(\zeta \right)$ at ${\widehat{\gamma }}_{\alpha }$ around ${\gamma }_1$ gives,

$$l_{{\widehat{\gamma }}_{\alpha }}\left({\widehat{\gamma }}_{\alpha }\right)=l_{{\widehat{\gamma }}_{\alpha }}\left({\gamma }_1\right)+{\left(\frac{\partial l_{{\widehat{\gamma }}_{\alpha }}\left(\zeta \right)}{\partial {\zeta }^T}\right)}_{\gamma ={\gamma }_1}\left({\widehat{\gamma }}_{\alpha }-{\gamma }_1\right)+o_p\left(∥{\widehat{\gamma }}_{\alpha }-{\gamma }_1∥\right).$$

Based on the asymptotic distribution of ${\widehat{\gamma }}_{\alpha }$ we have

$$\sqrt{n}{o}_p\left(∥{\widehat{\gamma }}_{\alpha }-{\gamma }_1∥\right)=o_p\left(1\right)$$

therefore

$$\sqrt{n}\left(l_{{\widehat{\gamma }}_{\alpha }}\left({\widehat{\gamma }}_{\alpha }\right)-l_{{\gamma }_1}\left({\gamma }_1\right)\right)\mathrm{and}\sqrt{n}{\left(\frac{\partial l_{{\widehat{\gamma }}_{\alpha }}\left(\zeta \right)}{\partial {\zeta }^T}\right)}_{\gamma ={\gamma }_1}\left({\widehat{\gamma }}_{\alpha }-{\gamma }_1\right)$$

have asymptotically the same distribution, i.e.,

$$\sqrt{n}\left(l_{{\widehat{\gamma }}_{\alpha }}\left({\widehat{\gamma }}_{\alpha }\right)-l_{{\gamma }_1}\left({\theta }_1\right)\right)\underset{n\to \infty }{\stackrel{L}{\to }}N\left({\mathbf{0}}_{k+1},{\left(\frac{\partial l_{{\widehat{\gamma }}_{\alpha }}\left(\zeta \right)}{\partial {\zeta }^T}\right)}_{\gamma ={\gamma }_1}{\mathit{A}}_{\alpha }{\left({\gamma }_1\right)}^{-1}{\left(\frac{\partial l_{{\widehat{\gamma }}_{\alpha }}\left(\zeta \right)}{\partial \zeta }\right)}_{\gamma ={\gamma }_1}\right).$$

Now, we shall denote,

$${\sigma }^2\left({\gamma }_1\right)={\left(\frac{\partial l_{{\widehat{\gamma }}_{\alpha }}\left(\zeta \right)}{\partial {\zeta }^T}\right)}_{\gamma ={\gamma }_1}{\mathit{A}}_{\alpha }{\left({\gamma }_1\right)}^{-1}{\left(\frac{\partial l_{{\widehat{\gamma }}_{\alpha }}\left(\zeta \right)}{\partial \zeta }\right)}_{\gamma ={\gamma }_1}.$$

Then, we have,

$$\begin{array}{cc}\hfill P_{{\gamma }_1}\left(W_n\left({\widehat{\gamma }}_{\alpha }\right)>{\chi }_{r,\alpha }^2\right)& =P_{{\gamma }_1}\left(W_n\left({\widehat{\gamma }}_{\alpha }\right)-n{l}_{{\gamma }_1}\left({\gamma }_1\right)>{\chi }_{r,\alpha }^2-n{l}_{{\gamma }_1}\left({\gamma }_1\right)\right)\hfill \\ \hfill & =P_{{\gamma }_1}\left(\frac{\sqrt{n}}{\sigma \left({\gamma }_1\right)}\left(l_{{\widehat{\gamma }}_{\alpha }}\left({\widehat{\gamma }}_{\alpha }\right)-l_{{\gamma }_1}\left({\gamma }_1\right)\right)>\frac{1}{\sigma \left({\gamma }_1\right)}\left(\frac{{\chi }_{r,\alpha }^2}{\sqrt{n}}-\sqrt{n}{l}_{{\gamma }_1}\left({\gamma }_1\right)\right)\right)\hfill \\ \hfill & \approx 1-{\varphi }_{N(0,1)}\left(\frac{1}{\sigma \left({\gamma }_1\right)}\left(\frac{{\chi }_{r,\alpha }^2}{\sqrt{n}}-\sqrt{n}{l}_{{\gamma }_1}\left({\gamma }_1\right)\right)\right),\hfill \end{array}$$

where ${\varphi }_{N(0,1)}\left(t\right)$ represents the distribution function of a standard normal distribution evaluated at $t.$ Finally,

$$\underset{n\to \infty }{lim}{P}_{{\gamma }_1}\left(W_n\left({\widehat{\gamma }}_{\alpha }\right)>{\chi }_{r,\alpha }^2\right)=1.$$

Appendix B. Poisson Regression Model

We derive here some explicit expression for the particular case of the Poisson regression. Following the discussion in Section 5, we denote here $\gamma =\mathit{\beta }$ since the nuisance parameter is known, $\varphi =1.$ The Poisson distribution with parameter $e^{{\mathit{x}}_i^T\mathit{\beta }}$ is given by

$$f_i(y,\mathit{\beta })=\frac{1}{y!}{e}^{-e^{{\mathit{x}}_i^T\mathit{\beta }}}{e}^{y{\mathit{x}}_i^T\mathit{\beta }},y=0,1,\dots .$$

Differentiating its logarithm with respect to the regression vector, we get

$$\frac{\partial log{f}_i(y,\mathit{\beta })}{\partial \mathit{\beta }}=(y-e^{{\mathit{x}}_i^T\mathit{\beta }}){\mathit{x}}_i^T.$$

so we can write

$$K_{1i}(y,\mathit{\beta })=y-e^{{\mathit{x}}_i^T\mathit{\beta }}.$$

Further, we have that

$$\begin{array}{c}\hfill N_i\left(y,\mathit{\beta }\right)=\frac{f_i{(y,\mathit{\beta })}^{\alpha }}{{\displaystyle \sum _{y=0}^{\infty }}{f}_i{(y,\mathit{\beta })}^{\alpha +1}}{\displaystyle \sum _{y=0}^{\infty }}{f}_i{(y,\mathit{\beta })}^{\alpha +1}\left(y-e^{{\mathit{x}}_i^T\mathit{\beta }}\right).\end{array}$$

so the estimating equations of the Poisson regression model are given by

$${\displaystyle \sum _{i=1}^n}\frac{1}{L_{\alpha }^i\left(\mathit{\beta }\right)}\left(f_i{(y_i,\mathit{\beta })}^{\alpha }\left(y_i-e^{{\mathit{x}}_i^T\mathit{\beta }}\right)-N_i\left(y_i,\mathit{\beta }\right)\right){\mathit{x}}_i={\mathbf{0}}_k.$$ (A1)

For $\alpha =0,$ we have

$$N_i\left(y_i,\mathit{\beta }\right)=0\mathrm{and}{L}_{\alpha }^i\left(\mathit{\beta }\right)=1$$

so the estimating equations are given by

$${\displaystyle \sum _{i=1}^n}\left(y_i-e^{{\mathit{x}}_i^T\mathit{\beta }}\right){\mathit{x}}_i={\mathbf{0}}_k,$$

yielding to the maximum likelihood estimating equations.

On the other hand, the asymptotic distribution of ${\widehat{\mathit{\beta }}}_{\alpha }$ is given by

$$\sqrt{n}{\left({\mathit{X}}^T{\mathit{D}}_{11}\mathit{X}\right)}^{-\frac{1}{2}}\frac{1}{n}{\mathit{X}}^T\left({\mathit{D}}_{11}^{*}-{\left({\mathit{D}}_1^{*}\right)}^T{\mathit{D}}_1^{*}\right)\mathit{X}\left({\widehat{\mathit{\beta }}}_{\alpha }-\mathit{\beta }\right)\underset{n\to \infty }{\stackrel{L}{\to }}N({\mathbf{0}}_k,{\mathit{I}}_k)$$

being

$${\mathit{D}}_{11}=diag\left(l_{11i}\left(\mathit{\beta }\right)\right),$$

with

$$l_{11i}\left(\mathit{\beta }\right)=\frac{1}{L_{\alpha }^i{\left(\mathit{\beta }\right)}^2}{\displaystyle \sum _{y=0}^{\infty }}{f}_i{(y,\mathit{\beta })}^{2\alpha +1}\left(K_{1i}(y,\mathit{\beta })-m_{1i}\left(\mathit{\beta }\right)\right)$$

and

$$m_{1i}\left(\mathit{\beta }\right)=\frac{1}{{\displaystyle \sum _{y=0}^{\infty }}{f}_i{(y,\mathit{\beta })}^{\alpha +1}}{\displaystyle \sum _{y=0}^{\infty }}{f}_i{(y,\mathit{\beta })}^{\alpha +1}\left(y-e^{{\mathit{x}}_i^T\mathit{\beta }}\right).$$

Finally

$$\begin{array}{cc}\hfill {\mathit{D}}_{11}^{*}& =diag(m_{11i}\left(\left(\mathit{\beta }\right)\right))=\frac{1}{{\displaystyle \sum _{y=0}^{\infty }}{f}_i{(y,\mathit{\beta })}^{\alpha +1}}{\displaystyle \sum _{y=0}^{\infty }}{f}_i{(y,\mathit{\beta })}^{\alpha +1}{\left(y-e^{{\mathit{x}}_i^T\mathit{\beta }}\right)}^2.\hfill \end{array}$$

Author Contributions

Conceptualization, M.J. and L.P.; methodology, M.J. and L.P.; software, M.J. and L.P.; validation, M.J. and L.P.; formal analysis, M.J. and L.P.; investigation, M.J. and L.P.; resources, M.J. and L.P.; data curation, M.J. and L.P.; writing—original draft preparation, M.J. and L.P.; writing—review and editing, M.J. and L.P.; visualization, M.J. and L.P.; supervision, M.J. and L.P.; project administration, M.J. and L.P.; funding acquisition, M.J. and L.P. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the Spanish Grants PGC2018-095194-B-100 (L. Pardo and M. Jaenada) and FPU/018240 (M. Jaenada).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The real datasets are publicly available on the R package robustbase in CRAN under the names of CrohnD (Poisson regression example) and carrots (binomial regression example).

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.