Heavy or semi-heavy tail, that is the question

Abstract

While there has been considerable research on the analysis of extreme values and outliers by using heavy-tailed distributions, little is known about the semi-heavy-tailed behaviors of data when there are a few suspicious outliers. To address the situation where data are skewed possessing semi-heavy tails, we introduce two new skewed distribution families of the hyperbolic secant with exciting properties. We extend the semi-heavy-tailedness property of data to a linear regression model. In particular, we investigate the asymptotic properties of the ML estimators of the regression parameters when the error term has a semi-heavy-tailed distribution. We conduct simulation studies comparing the ML estimators of the regression parameters under various assumptions for the distribution of the error term. We also provide three real examples to show the priority of the semi-heavy-tailedness of the error term comparing to heavy-tailedness. Online supplementary materials for this article are available. All the new proposed models in this work are implemented by the shs R package, which can be found on the GitHub webpage.

1. Introduction

Practitioners usually use heavy-tailed distributions when robustness to potential outliers is a concern. For symmetric distributions, to use robust estimation techniques is often a standard approach for implementing statistical inference. However, robust methods do not work for asymmetric distributions that lead to strongly biased parameter estimates [32]. Hence, accounting for both skewness and tick tails features of real data is an attractive benefit for statistical modeling and inference, especially in the context of a regression model. Several methods to obtain skewed heavy-tailed distributions have been presented in the literature; e.g. Fernandez and Steel [12], Ma and Genton [24], Azzalini and Capitanio [4], Jones and Faddy [22], Ferreira and Steel [14], and many others.

In a broad variety of applied contexts, such as economics, finance, and hydrology, the thickness of the tail of the distribution is not too heavy, but semi-heavy. We have experienced that the semi-heavy-tailed distributions often fit observed data quite well and better rather than heavy-tailed alternatives when there are a few suspicious outliers. Therefore, we consider a situation where data are skewed possessing semi-heavy tails. Indeed, we address the case where a skewed semi-heavy-tailed distribution for the error term in a linear regression is a better option rather than asymmetric heavy-tailed alternatives. To this end, we propose two new families of skewed distributions that have a semi-heavy tail, in some sense. Furthermore, we carry through a simulation study to validate that the proposed distributions perform over the alternative skewed heavy-tailed distributions concerning model fitting and information criteria, including Akaike information criterion (AIC; [1]) and Bayesian information criterion (BIC; [29]).

There are a few works of literature on symmetric semi-heavy-tailed distributions; e.g. hyperbolic secant (HS) family of distributions that originates from Fisher [18], according to Fischer [15]. Vaughan [34] introduced a skewed version of the HS distribution, named generalized HS (GHS), and Fischer and Vaughan [16] added an alternative one called it as skewed generalized HS (SGSH) distribution. Another skewed version of a semi-heavy-tailed distribution, beta hyperbolic secant (BHS), submitted by Fischer and Vaughan [17].

These skewed versions have complex probability densities and cumulative distribution functions. Also, their moments are usually complicated to calculate due to the existence of an infinite series. Furthermore, the asymptotic behavior of maximum likelihood (ML) estimators of the relevant parameters in a more applicable linear regression model, for these distributions, is not usually discussed. Preferably, our proposed distributions (Section 2) have several advantages, including (1) simple formula for density functions, (2) easy to calculate moments, (3) better fit to the real data, and (4) simple stochastic representations that lead to introduce efficient algorithms for generating random numbers.

In Section 2, we present two new skewed semi-heavy-tailed distributions and their basic properties. We enter them into a linear regression model in Section 3 as the distribution of errors, then verify the consistency and asymptotic normality of the corresponding ML estimators. In Section 4, we provide the results of a simulation study that compares the performance of the ML estimators in a linear regression model when errors are generated with our skewed semi-heavy-tailed distributions and also with a heavy-tailed skew Student-t distribution [2]. Section 5 presents three real data analyses with (i) acoustic comfort evaluation data, (ii) pricing diamond stones data, and (iii) the excess rate of return for a company, named Martin Marietta, and an index of the excess rate of return for the New York stock exchange. The final section contains concluding remarks. All technical details are available online as supplementary materials. All methods presented in this work are implemented in the shs R package [27], which can be found on the GitHub webpage .¹

2. Proposed sampling models

A hyperbolic secant distribution is symmetric, and its density is bell-shaped similar to the Gaussian distribution but has slightly heavier tails. The probability density function of an HS distribution is given by

$$f(x)={\displaystyle \frac{2}{\pi \sigma }}{\displaystyle \frac{\exp \left(\frac{x-\mu }{\sigma }\right)}{1+\exp \left(2\frac{x-\mu }{\sigma }\right)}},\mu \in \mathbb{R},\sigma >0\mathrm{and}x\in \mathbb{R},$$

where μ is the mean and σ is the scale parameter. The variance of the distribution is $\frac{{\pi }^2}{4}{\sigma }^2$.

Fischer [15] showed that if V has a half-Cauchy density function, then $U=\log (V)$ has a HS distribution. Using the fact that Cauchy distribution is the special case of a Student-t distribution with 1 degree of freedom, we can get a generalized HS distribution. Let Y has a half Student-t distribution with ν degrees of freedom, then $Z=\log (Y)$ has a standard new asymmetric HS distribution. We refer to this distribution as the skew HS type 1 ($\mathrm{SH}{\mathrm{S}}_1$) distribution. The probability density function of Z is given by

$$f(z)=c_{\nu }\exp (z){(1+\frac{1}{\nu }\exp (2z))}^{-\frac{\nu +1}{2}},c_{\nu }={\displaystyle \frac{2\mathrm{\Gamma }\left(\frac{\nu +1}{2}\right)}{\sqrt{\nu \pi }\mathrm{\Gamma }\left(\frac{\nu }{2}\right)}}.$$ (1)

Figure 1 shows a variety of $\mathrm{SH}{\mathrm{S}}_1$ densities for several values of ν. Given the density (1) for Z, it is easy to show that $X=\sigma Z+\mu$ follows $\mathrm{SH}{\mathrm{S}}_1(\mu ,\sigma ,\nu )$. When $\nu =1$, $f(x)$ reduces to the HS distribution with location parameter μ and scale parameter σ. When $\nu <1$ or $\nu >1$, $\mathrm{SH}{\mathrm{S}}_1$ distribution is negatively or positively skewed, respectively. Cumulative distribution function of (1) is also given by

$$F(z)={\int }_{-\mathrm{\infty }}^z{c}_{\nu }\exp (w){(1+\frac{1}{\nu }\exp (2w))}^{-\frac{\nu +1}{2}}dw={\mathrm{I}}_{\frac{\exp (2z)}{\nu +\exp (2z)}}(\frac{1}{2},\frac{\nu }{2})$$

where ${\mathrm{I}}_y(a,b)$ is the incomplete beta function.

Figure 1.

$\mathrm{SH}{\mathrm{S}}_1$ densities for different values of ν.

The following expression for the moment generating function of Z can be obtained (see the supplementary materials for the proof).

Proposition 2.1

For an $\mathrm{SH}{\mathrm{S}}_1$ distribution with density (1), the moment generating function is given by

$$\mathrm{M}(t)={\displaystyle \frac{{\nu }^{\frac{t}{2}}\mathrm{\Gamma }\left(\frac{1+t}{2}\right)\mathrm{\Gamma }\left(\frac{\nu -t}{2}\right)}{\sqrt{\pi }\mathrm{\Gamma }\left(\frac{\nu }{2}\right)}}.$$

Therefore, we can easily calculate that

$$\begin{array}{rl}\mathrm{E}\left(Z\right)& =\frac{1}{2}(\log (\nu )+{\psi }_{\nu }),\mathrm{E}\left(Z^2\right)={\left(\mathrm{E}\left(Z\right)\right)}^2+\frac{1}{4}{\psi }_{\nu }^{\prime }\\ \mathrm{E}\left(Z^3\right)& ={\left(\mathrm{E}\left(Z\right)\right)}^3+\frac{3}{8}\mathrm{E}(Z){\psi }_{\nu }^{\prime }+\frac{1}{8}{\psi }_{\nu }^{″}\\ \mathrm{E}\left(Z^4\right)& =(\mathrm{E}(Z){)}^4+\frac{24}{16}(\mathrm{E}(Z){)}^2{\psi }_{\nu }^{\prime }+\frac{8}{16}\mathrm{E}(Z){\psi }_{\nu }^{″}+\frac{3}{16}{\psi }_{\nu }^{\prime 2}+\frac{1}{16}{\psi }_{\nu }^{‴}\end{array}$$

where ${\psi }_{\nu }=\psi (\frac{1}{2})-\psi (\frac{\nu }{2})$, ${\psi }_{\nu }^{\prime }={\psi }^{\prime }(\frac{1}{2})+{\psi }^{\prime }(\frac{\nu }{2})$, ${\psi }_{\nu }^{″}={\psi }^{″}(\frac{1}{2})-{\psi }^{″}(\frac{\nu }{2})$, ${\psi }_{\nu }^{‴}={\psi }^{‴}(\frac{1}{2})+{\psi }^{‴}(\frac{\nu }{2})$, in which $\psi (\cdot )$ is the digamma function and ${\psi }^{\prime }(\cdot )$, ${\psi }^{″}(\cdot )$, and ${\psi }^{‴}(\cdot )$ are its first, second, and third derivatives, respectively. We can check that for $\nu =1$, $\mathrm{E}(Z)=0$ and $\mathrm{Var}(Z)=\frac{{\pi }^2}{4}$, which are the mean and variance of standard HS distribution, respectively.

2.1. Second proposal

It could be attractive to have a skewness parameter whose negative or positive values display a negative or positive skewness of data or vice versa. The $\mathrm{SH}{\mathrm{S}}_1$ distribution has not such a feature. To achieve this, we introduce another generalization for HS distribution and refer to that as SHS type 2 ($\mathrm{SH}{\mathrm{S}}_2$). For this distribution, the probability density function is given by

$$f(x)=\frac{\cos \left(\frac{\pi }{2}\alpha \right)}{\pi \sigma }{\displaystyle \frac{e^{\alpha \left(\frac{x-\mu }{\sigma }\right)}}{\cosh \left(\frac{x-\mu }{\sigma }\right)}},-1<\alpha <1,\sigma >0,\mu \in \mathbb{R},x\in \mathbb{R}.$$ (2)

If the random variable X has the density (2), we denote it as $\mathrm{SH}{\mathrm{S}}_2(\mu ,\sigma ,\alpha )$. The following proposition, confirms that $\mathrm{SH}{\mathrm{S}}_2$ has the symmetry property of skewness parameter.

Proposition 2.2

If $X\sim \mathrm{SH}{\mathrm{S}}_2(\mu ,\sigma ,\alpha )$ then $-X\sim \mathrm{SH}{\mathrm{S}}_2(-\mu ,\sigma ,-\alpha )$.

When $\alpha =0$, the distribution reduces to the HS distribution. When $\alpha <0$ or $\alpha >0$, $\mathrm{SH}{\mathrm{S}}_2$ distribution is positively or negatively skewed, respectively. Figure 2 shows a variety of $\mathrm{SH}{\mathrm{S}}_2$ densities for several values of α. It is also easy to show that the cumulative distribution function of $\mathrm{SH}{\mathrm{S}}_2$ is as follows:

$$F(x)={\mathrm{I}}_{\frac{e^{2x}}{1+e^{2x}}}(\frac{1+\alpha }{2},\frac{1-\alpha }{2}).$$

Furthermore, there is a simple stochastic representation for $\mathrm{SH}{\mathrm{S}}_2$ distribution as well; certainly, if $Y\sim \mathrm{Beta}(\frac{1+\alpha }{2},\frac{1-\alpha }{2})$, then $X=\frac{1}{2}\log (\frac{Y}{1-Y})$ agrees with density (2).

Figure 2.

$\mathrm{SH}{\mathrm{S}}_2$ densities for different values of α.

When $Z\sim \mathrm{SH}{\mathrm{S}}_2(0,1,\alpha )$, the following expressions for characteristic and moment generating functions can be obtained.

Proposition 2.3

Characteristic and moment generating functions of Z is given by

$$\begin{array}{rl}\mathrm{C}(t)& ={\displaystyle \frac{\cos \left(\frac{\pi }{2}\alpha \right)}{\cos (\frac{\pi }{2}(\alpha +it))}}\\ \mathrm{M}(t)& ={\displaystyle \frac{\cos \left(\frac{\pi }{2}\alpha \right)}{\cos (\frac{\pi }{2}(\alpha +t))}}\end{array}$$

respectively.

By applying Proposition 2.3, we can show that, for example

$$\begin{array}{rl}\mathrm{E}(Z)& =\frac{\pi }{2}\tan \left(\frac{\pi }{2}\alpha \right)\\ \mathrm{E}(Z^2)& =\frac{{\pi }^2}{2^2}(1+2{\tan }^2\left(\frac{\pi }{2}\alpha \right))\\ \mathrm{E}(Z^3)& =\frac{{\pi }^3}{2^3}(5\tan \left(\frac{\pi }{2}\alpha \right)+6{\tan }^3\left(\frac{\pi }{2}\alpha \right))\\ \mathrm{E}(Z^4)& =\frac{{\pi }^4}{2^4}(5+28{\tan }^2\left(\frac{\pi }{2}\alpha \right)+24{\tan }^4\left(\frac{\pi }{2}\alpha \right)).\end{array}$$

2.2. Tail behavior

Figure 3 plots the tails of densities in (1) and (2) and compares them with normal and Cauchy densities. The Cauchy distribution has a thick tail, while the normal distribution has a thin tail. Intuitively, our proposed distributions have tails lighter than Cauchy (as a well-known heavy-tailed distribution) but heavier than normal, hence possess semi-heavy tails. This intuition can be formalized by studying the behavior of the tails.

Figure 3.

Comparison of tails for Cauchy (solid), normal (dotted), $\mathrm{SH}{\mathrm{S}}_1$ (dotted-dashed), and $\mathrm{SH}{\mathrm{S}}_2$ (dashed) densities.

The following theorem demonstrates that the SHS distributions have semi-heavy tails. The theorem is proved for the standard case. First, formal definitions of heavy and semi-heavy tails are required. We consider the following definitions provided by Omey et al. [26].

Definition 2.4

The random variable X has a fat tail density if its density function $f(\cdot )$ satisfies

$$\underset{x\to \mathrm{\infty }}{lim}{\displaystyle \frac{f(xy)}{f(x)}}=y^{\gamma }\gamma \in \mathbb{R}.$$

Definition 2.5

A density function $f(x)$ is called a semi-heavy-tailed function if it is of the form $f(x)={\mathrm{e}}^{-\eta x}h(x)$, $\eta >0$, where $h(x)$ is a fat tail density function.

Theorem 2.6

Both $\mathrm{SH}{\mathrm{S}}_1$ and $\mathrm{SH}{\mathrm{S}}_2$ distributions have semi-heavy tails.

3. Regression modeling

In linear regression modeling, the usual normality assumption for error terms is not appropriate in cases where observations may be skewed and may have heavy or semi-heavy tails. Departures from normality of the errors may have adverse effects on inferential results [5]. Many remedies arise throughout the statistics literature on extending the classical normal linear regression model; e.g. Student-t regression [23], scale mixtures of normal [13], skew-normal regression [36], finite mixture of normals [31], and Skew-t components regression [9]. However, none of these models explicitly examine semi-heavy tails distributions for errors. Furthermore, the identifiability and computational burden of the mixture-based approaches could be a challenge. The methods that are based on Student-t regression also have a limitation regarding the estimation of the degrees of freedom parameter, once its estimation is known to be complicated and computationally expensive.

Here, we address the situation where error terms are skewed and have a semi-heavy tail. We extend a likelihood-based approach for inference when the errors follow SHS distributions. We assume that the observations $y_i$, $i=1,\dots ,n$, to be generated from

$$y_i={\mathit{\beta }}^{\mathrm{\prime }}{\mathit{x}}_i+\sigma {ϵ}_i,$$ (3)

where ${\mathit{x}}_i\in {\mathbb{R}}^k$ is a vector of covariates, $\mathit{\beta }=({\beta }_1,\dots ,{\beta }_k{)}^{\mathrm{\prime }}$ is the vector of unknown regression parameters, and σ is a scale parameter. We assume that the error terms ${ϵ}_1,\dots ,{ϵ}_n$ are i.i.d. $\mathrm{SH}{\mathrm{S}}_1(0,1,\nu )$ or $\mathrm{SH}{\mathrm{S}}_2(0,1,\alpha )$. In the following, we study each model separately.

3.1. $\mathrm{SH}{\mathrm{S}}_1$ error term

When the error terms follow the $\mathrm{SH}{\mathrm{S}}_1(0,1,\nu )$ distribution, the log-likelihood of the model (3) is given by

$$\begin{array}{rl}\ell ({\mathit{\beta }}^{\mathrm{\prime }},\sigma ,\nu )& \propto n\log \left(\mathrm{\Gamma }\left(\frac{\nu +1}{2}\right)\right)-\frac{n}{2}\log \left(\nu \right)-n\log \left(\mathrm{\Gamma }\left(\frac{\nu }{2}\right)\right)-n\log \left(\sigma \right)\\ & +\sum _{i=1}^n\left(\frac{y_i-{\mathit{\beta }}^{\mathrm{\prime }}{\mathit{x}}_i}{\sigma }\right)-{\displaystyle \frac{\nu +1}{2}}\sum _{i=1}^n\log (1+\frac{1}{\nu }\exp \left(2\frac{y_i-{\mathit{\beta }}^{\mathrm{\prime }}{\mathit{x}}_i}{\sigma }\right)).\end{array}$$

To maximize ℓ with respect to the parameters of the model, we proceed through a quasi-Newton optimization method, the Broyden–Fletcher–Goldfarb–Shanno (BFGS) algorithm, which was proposed by Broyden [6], Fletcher [19], Goldfarb [20], and Shanno [30], independently. Let $\mathit{\theta }=({\mathit{\beta }}^{\mathrm{\prime }},\sigma ,\nu {)}^{\mathrm{\prime }}$ denote the vector of the parameters. The components of the score vector

$$S_n={({\left\{{\displaystyle \frac{\mathrm{\partial }\ell (\mathit{\theta })}{\mathrm{\partial }{\beta }_j}}\right\}}_{j=1}^k,{\displaystyle \frac{\mathrm{\partial }\ell (\mathit{\theta })}{\mathrm{\partial }\sigma }},{\displaystyle \frac{\mathrm{\partial }\ell (\mathit{\theta })}{\mathrm{\partial }\nu }})}^{\mathrm{\prime }}$$

are

$$\begin{array}{rl}{\displaystyle \frac{\mathrm{\partial }\ell \left(\mathit{\theta }\right)}{\mathrm{\partial }{\beta }_j}}& =\frac{\nu +1}{\sigma }\sum _{i=1}^n{x}_{ij}\frac{\frac{1}{\nu }\exp \left(2\frac{y_i-{\mathit{\beta }}^{\mathrm{\prime }}{\mathit{x}}_i}{\sigma }\right)}{1+\frac{1}{\nu }\exp \left(2\frac{y_i-{\mathit{\beta }}^{\mathrm{\prime }}{\mathit{x}}_i}{\sigma }\right)}-\frac{1}{\sigma }\sum _{i=1}^n{x}_{ij},j=1,\dots ,k\\ {\displaystyle \frac{\mathrm{\partial }\ell \left(\mathit{\theta }\right)}{\mathrm{\partial }\sigma }}& =\frac{\nu +1}{\sigma }\sum _{i=1}^n\left(\frac{y_i-{\mathit{\beta }}^{\mathrm{\prime }}{\mathit{x}}_i}{\sigma }\right)\frac{\frac{1}{\nu }\exp \left(2\frac{y_i-{\mathit{\beta }}^{\mathrm{\prime }}{\mathit{x}}_i}{\sigma }\right)}{1+\frac{1}{\nu }\exp \left(2\frac{y_i-{\mathit{\beta }}^{\mathrm{\prime }}{\mathit{x}}_i}{\sigma }\right)}-\frac{1}{\sigma }\sum _{i=1}^n\left(\frac{y_i-{\mathit{\beta }}^{\mathrm{\prime }}{\mathit{x}}_i}{\sigma }\right)-\frac{n}{\sigma }\\ {\displaystyle \frac{\mathrm{\partial }\ell \left(\mathit{\theta }\right)}{\mathrm{\partial }\nu }}& =\frac{\nu +1}{2\nu }\sum _{i=1}^n\left(\frac{y_i-{\mathit{\beta }}^{\mathrm{\prime }}{\mathit{x}}_i}{\sigma }\right)\frac{\frac{1}{\nu }\exp \left(2\frac{y_i-{\mathit{\beta }}^{\mathrm{\prime }}{\mathit{x}}_i}{\sigma }\right)}{1+\frac{1}{\nu }\exp \left(2\frac{y_i-{\mathit{\beta }}^{\mathrm{\prime }}{\mathit{x}}_i}{\sigma }\right)}\\ & -\frac{1}{2}\sum _{i=1}^n\log (1+\frac{1}{\nu }\exp \left(2\frac{y_i-{\mathit{\beta }}^{\mathrm{\prime }}{\mathit{x}}_i}{\sigma }\right))-\frac{n}{2}\psi \left(\frac{\nu }{2}\right)+\frac{n}{2}\psi \left(\frac{\nu +1}{2}\right)-\frac{n}{2\nu }.\end{array}$$

3.2. $\mathrm{SH}{\mathrm{S}}_2$ error term

For errors with $\mathrm{SH}{\mathrm{S}}_2(0,1,\alpha )$ distribution, the log-likelihood of the model (3) is given by

$$\begin{array}{rl}\ell (\mathit{\zeta })=\ell ({\mathit{\beta }}^{\mathrm{\prime }},\sigma ,\alpha )& =n\log (\cos \left(\frac{\pi }{2}\alpha \right))-n\log \left(\sigma \right)-n\log \left(\pi \right)\\ & -\alpha \sum _{i=1}^n{\displaystyle \frac{y_i-{\mathit{\beta }}^{\mathrm{\prime }}{\mathit{x}}_i}{\sigma }}-\sum _{i=1}^n\log \cosh \left({\displaystyle \frac{y_i-{\mathit{\beta }}^{\mathrm{\prime }}{\mathit{x}}_i}{\sigma }}\right).\end{array}$$

For this log-likelihood function, the components of the score vector

$${\tilde{S}}_n={({\left\{{\displaystyle \frac{\mathrm{\partial }\ell (\mathit{\zeta })}{\mathrm{\partial }{\beta }_j}}\right\}}_{j=1}^k,{\displaystyle \frac{\mathrm{\partial }\ell (\mathit{\zeta })}{\mathrm{\partial }\sigma }},{\displaystyle \frac{\mathrm{\partial }\ell (\mathit{\zeta })}{\mathrm{\partial }\alpha }})}^{\mathrm{\prime }}$$

are given by

$$\begin{array}{rl}{\displaystyle \frac{\mathrm{\partial }\ell \left(\mathit{\zeta }\right)}{\mathrm{\partial }{\beta }_j}}& =\alpha \sum _{i=1}^n{\displaystyle \frac{x_{ij}}{\sigma }}+\sum _{i=1}^n{\displaystyle \frac{x_{ij}}{\sigma }}\tanh \left({\displaystyle \frac{y_i-{\mathit{\beta }}^{\mathrm{\prime }}{\mathit{x}}_i}{\sigma }}\right),j=1,\dots ,k\\ {\displaystyle \frac{\mathrm{\partial }\ell \left(\mathit{\zeta }\right)}{\mathrm{\partial }\sigma }}& =-\frac{n}{\sigma }+\alpha \sum _{i=1}^n{\displaystyle \frac{y_i-{\mathit{\beta }}^{\mathrm{\prime }}{\mathit{x}}_i}{{\sigma }^2}}+\sum _{i=1}^n{\displaystyle \frac{y_i-{\mathit{\beta }}^{\mathrm{\prime }}{\mathit{x}}_i}{{\sigma }^2}}\tanh \left({\displaystyle \frac{y_i-{\mathit{\beta }}^{\mathrm{\prime }}{\mathit{x}}_i}{\sigma }}\right)\\ {\displaystyle \frac{\mathrm{\partial }\ell \left(\mathit{\zeta }\right)}{\mathrm{\partial }\alpha }}& =-n\frac{\pi }{2}\tan \left(\frac{\pi }{2}\alpha \right)-\sum _{i=1}^n{\displaystyle \frac{y_i-{\mathit{\beta }}^{\mathrm{\prime }}{\mathit{x}}_i}{\sigma }}.\end{array}$$

To compute the ML estimators of the parameters in this model, we use a BFGS method as well.

3.3. Asymptotic properties

Let $X_n=({\mathit{x}}_1,\dots ,{\mathit{x}}_n{)}^{\mathrm{\prime }}$ denote an $n\times k$ design matrix. We assume $X_n$ to be rank k. Let Θ be the parameter space for ${\mathrm{SHS}}_1$ model, an open subset of ${\mathbb{R}}^{k+2}$. Also let Ψ be the paramater space for ${\mathrm{SHS}}_2$ model, an open subset of ${\mathbb{R}}^{k+2}$. The second derivative matrix of $\ell (\mathit{\theta })$ can be written as

$${\ell }_n^{(2)}=\left[\begin{array}{ccc}{\displaystyle \frac{{\mathrm{\partial }}^2\ell (\mathit{\theta })}{\mathrm{\partial }{\mathit{\beta }}^{\mathrm{\prime }}\mathrm{\partial }\mathit{\beta }}}& {\displaystyle \frac{{\mathrm{\partial }}^2\ell (\mathit{\theta })}{\mathrm{\partial }{\mathit{\beta }}^{\mathrm{\prime }}\mathrm{\partial }\sigma }}& {\displaystyle \frac{{\mathrm{\partial }}^2\ell (\mathit{\theta })}{\mathrm{\partial }{\mathit{\beta }}^{\mathrm{\prime }}\mathrm{\partial }\nu }}\\ {\displaystyle \frac{{\mathrm{\partial }}^2\ell (\mathit{\theta })}{\mathrm{\partial }{\mathit{\beta }}^{\mathrm{\prime }}\mathrm{\partial }\sigma }}& {\displaystyle \frac{{\mathrm{\partial }}^2\ell (\mathit{\theta })}{\mathrm{\partial }{\sigma }^2}}& {\displaystyle \frac{{\mathrm{\partial }}^2\ell (\mathit{\theta })}{\mathrm{\partial }\sigma \mathrm{\partial }\nu }}\\ {\displaystyle \frac{{\mathrm{\partial }}^2\ell (\mathit{\theta })}{\mathrm{\partial }{\mathit{\beta }}^{\mathrm{\prime }}\mathrm{\partial }\nu }}& {\displaystyle \frac{{\mathrm{\partial }}^2\ell (\mathit{\theta })}{\mathrm{\partial }\sigma \mathrm{\partial }\nu }}& {\displaystyle \frac{{\mathrm{\partial }}^2\ell (\mathit{\theta })}{\mathrm{\partial }{\nu }^2}}\end{array}\right]$$

where its elements are given in the supplementary materials. Similarly, the second derivative matrix of $\ell (\mathit{\zeta })$ (see the supplementary materials) can be written as

$${\tilde{\ell }}_n^{(2)}=\left[\begin{array}{ccc}{\displaystyle \frac{{\mathrm{\partial }}^2\ell (\mathit{\zeta })}{\mathrm{\partial }{\mathit{\beta }}^{\mathrm{\prime }}\mathrm{\partial }\mathit{\beta }}}& {\displaystyle \frac{{\mathrm{\partial }}^2\ell (\mathit{\zeta })}{\mathrm{\partial }{\mathit{\beta }}^{\mathrm{\prime }}\mathrm{\partial }\sigma }}& {\displaystyle \frac{{\mathrm{\partial }}^2\ell (\mathit{\zeta })}{\mathrm{\partial }{\mathit{\beta }}^{\mathrm{\prime }}\mathrm{\partial }\alpha }}\\ {\displaystyle \frac{{\mathrm{\partial }}^2\ell (\mathit{\zeta })}{\mathrm{\partial }{\mathit{\beta }}^{\mathrm{\prime }}\mathrm{\partial }\sigma }}& {\displaystyle \frac{{\mathrm{\partial }}^2\ell (\mathit{\zeta })}{\mathrm{\partial }{\sigma }^2}}& {\displaystyle \frac{{\mathrm{\partial }}^2\ell (\mathit{\zeta })}{\mathrm{\partial }\sigma \mathrm{\partial }\alpha }}\\ {\displaystyle \frac{{\mathrm{\partial }}^2\ell (\mathit{\zeta })}{\mathrm{\partial }{\mathit{\beta }}^{\mathrm{\prime }}\mathrm{\partial }\alpha }}& {\displaystyle \frac{{\mathrm{\partial }}^2\ell (\mathit{\zeta })}{\mathrm{\partial }\sigma \mathrm{\partial }\alpha }}& {\displaystyle \frac{{\mathrm{\partial }}^2\ell (\mathit{\zeta })}{\mathrm{\partial }{\alpha }^2}}\end{array}\right].$$

Using ${\mathrm{SHS}}_1$ model for the error term, we obtain the Fisher information matrix as

$$B_n=-\mathrm{E}({\ell }_n^{(2)})=\left[\begin{array}{ccc}{B}_n(\mathit{\beta })& B_n(\mathit{\beta },\sigma )& B_n(\mathit{\beta },\nu )\\ B_n(\mathit{\beta },\sigma )& B_n(\sigma )& B_n(\sigma ,\nu )\\ B_n(\mathit{\beta },\nu )& B_n(\sigma ,\nu )& B_n(\nu )\end{array}\right]$$

and for ${\mathrm{SHS}}_2$ model, we get

$${\tilde{B}}_n=-\mathrm{E}({\tilde{\ell }}_n^{(2)})=\left[\begin{array}{ccc}{\tilde{B}}_n(\mathit{\beta })& {\tilde{B}}_n(\mathit{\beta },\sigma )& {\tilde{B}}_n(\mathit{\beta },\alpha )\\ {\tilde{B}}_n(\mathit{\beta },\sigma )& {\tilde{B}}_n(\sigma )& {\tilde{B}}_n(\sigma ,\alpha )\\ {\tilde{B}}_n(\mathit{\beta },\alpha )& {\tilde{B}}_n(\sigma ,\alpha )& {\tilde{B}}_n(\alpha )\end{array}\right].$$

See the supplementary materials for the details.

To prove the asymptotic results, we need the following conditions:

1. The true parameter vector ${\mathit{\theta }}_0$, in the ${\mathrm{SHS}}_1$ model, is an interior point of Θ.

2. The true parameter vector ${\mathit{\zeta }}_0$, in the ${\mathrm{SHS}}_2$ model, is an interior point of Ψ.

3. $n^{-1}\sum _{i=1}^n{x}_{ij}\to c_j$ as $n\to \mathrm{\infty }$, for $j=1,\dots ,k$, where $c_j$ is a finite real number.

4. $n^{-1}{X}_n^{\mathrm{\prime }}{X}_n$ converges to a finite and positive definite matrix C, as $n\to \mathrm{\infty }$.

5. $n^{-2}\sum _{i=1}^n{x}_{ij}^2{x}_{ik}^2\to 0$ as $n\to \mathrm{\infty }$, for $j=1,\dots ,k$.

The following two theorems reveal the consistency and asymptotic normality of the maximum likelihood estimators in our two proposed models.

Theorem 3.1

Given the conditions C1, C3, C4, and C5, the maximum likelihood estimators of the linear regression model (3) with the ${\mathrm{SHS}}_1$ model for error term, $\hat{\mathit{\theta }},$ is weakly consistent and

$$\sqrt{n}(\hat{\mathit{\theta }}-{\mathit{\theta }}_0)\sim \mathrm{N}(\mathbf{0},{\mathit{B}}_{\mathit{n}}^{\mathbf{-}\mathbf{1}}).$$

Theorem 3.2

Given the conditions C2, C3, C4, and C5, the maximum likelihood estimators of the linear regression model (3) with the ${\mathrm{SHS}}_2$ model for error term, $\hat{\mathit{\zeta }},$ is weakly consistent and

$$\sqrt{n}(\hat{\mathit{\zeta }}-{\mathit{\zeta }}_0)\sim \mathrm{N}(\mathbf{0},{\stackrel{\mathbf{~}}{\mathit{B}}}_{\mathit{n}}^{\mathbf{-}\mathbf{1}}).$$

4. Simulation study

We conducted a simulation study to assess and compare the performance of our two proposed regression models to one another and to the skew Student-t (SSt) regression, as a heavy-tail model, and the skew-normal (SN; [3]) regression model. We simulated the responses from a linear model:

$$y_i={\beta }_1{x}_{1i}+{\beta }_2{x}_{2i}+\sigma {ϵ}_i,i=1,\dots ,n.$$

In this model, for errors with skew normal distribution, the density of ε is given by

$$f(ϵ)=2\varphi (ϵ)\mathrm{\Phi }(\alpha ϵ),$$

where $\varphi (\cdot )$ and $\mathrm{\Phi }(\cdot )$ are density and cumulative distribution functions of the standard normal distribution, respectively. Here, α is the skewness parameter of the density. For errors with skew Student-t distribution, the density of the error term is given by

$$f(ϵ)=\frac{2}{\sigma }t(\frac{ϵ}{\sigma };\nu )T(\alpha \frac{ϵ}{\sigma }\sqrt{\frac{\nu +1}{\nu +\frac{{ϵ}^2}{{\sigma }^2}}};\nu +1)$$

in which $t(\cdot ;\nu )$ and $T(\cdot ;\nu )$ are density and cumulative distribution functions of standard Student-t distribution with ν degrees of freedom, respectively. Similar to skew-normal model, α controls the skewness of the errors.

In our study, we examined the following configuration options:

• the sample size is n = 100, n = 200, or n = 500.

• the regression covariates $x_1$ and $x_2$ are generated from the standard normal distribution.

• the real values of regression coefficients are set to $({\beta }_1,{\beta }_2)=(1,2)$. The real value for scale parameter is also specified by $\sigma =1$.

• both covariates and response variable are centered on its mean.

Toward developing an understanding of the performance of these models, we considered three simulation scenarios that vary in terms of the true underlying error distribution:

1. ${ϵ}_i\sim \mathrm{SH}{\mathrm{S}}_1(0,1,\nu =4)$

2. ${ϵ}_i\sim \mathrm{SH}{\mathrm{S}}_2(0,1,\alpha =0.5)$

3. ${ϵ}_i\sim \mathrm{SSt}(0,1,\alpha =2,\nu =4)$.

In the first and second scenarios, the error distribution is set to correspond to the SHS distributions so that the observations in the simulated data possess a semi-heavy tail. We chose $\nu =4$ and $\alpha =0.5$ for ${\mathrm{SHS}}_1$ and ${\mathrm{SHS}}_2$, respectively, to have both positive and negative skewness. To evaluate the performance of the model when the errors do follow a heavy-tailed distribution, the third scenario takes it to be a skew Student-t distribution. Specifically, we are curious to assess the sensitivity of the proposed semi-heavy-tailed distributions on the misspecification of the tail properties of errors.

The above configurations imply a total of nine different models, and we simulated and estimated R = 1000 times for each. We also computed bias and the mean squared error (MSE) of the parameter estimates for each model. To compare these measures in a better way and faster, we calculated a relative version of them as a ratio of the measure obtained by estimated misspecifying models and the estimated true model for the error term. Results for scenario (1) are summarized in Table 1, those for scenario (2) in Table 2, and those for scenario (3) in Table 3. In each scenario, for the true model, the pure bias and MSE of the estimates are reported as well.

Table 1. Relative bias and MSE of the parameter estimates under $\mathrm{SH}{\mathrm{S}}_1$ distributed errors.

	n	100		200		500
Model		${\hat{\beta }}_1$	${\hat{\beta }}_2$	${\hat{\beta }}_1$	${\hat{\beta }}_2$	${\hat{\beta }}_1$	${\hat{\beta }}_2$
$\mathrm{SH}{\mathrm{S}}_1$	Bias	0.0002	−0.0012	−0.0076	−0.0038	−0.0039	−0.0000
	MSE	0.0117	0.0117	0.0052	0.0058	0.0025	0.0023
$\mathrm{SH}{\mathrm{S}}_2$	Bias	0.2031	1.1926	1.1041	1.0298	1.0769	4.8573
	MSE	1.0660	1.0659	1.0798	1.0843	1.0803	1.0914
SSt	Bias	1.8930	1.2208	0.9862	1.1240	0.9527	8.9018
	MSE	1.0334	1.0236	1.0202	1.0348	1.0035	1.0250
SN	Bias	−0.3185	0.6290	1.2384	1.9219	0.8533	44.9954
	MSE	1.2076	1.2073	1.2146	1.2215	1.1709	1.2559

Notes: For the ${\mathrm{SHS}}_1$ model, the pure bias and MSE of the estimates are reported.

Table 2. Relative bias and MSE of the parameter estimates under $\mathrm{SH}{\mathrm{S}}_2$ distributed errors.

	n	100		200		500
Model		${\hat{\beta }}_1$	${\hat{\beta }}_2$	${\hat{\beta }}_1$	${\hat{\beta }}_2$	${\hat{\beta }}_1$	${\hat{\beta }}_2$
$\mathrm{SH}{\mathrm{S}}_1$	Bias	0.9419	0.9442	0.9925	1.0840	0.8363	1.0166
	MSE	0.9596	0.9655	0.9738	0.9664	0.9617	0.9612
$\mathrm{SH}{\mathrm{S}}_2$	Bias	0.0038	0.0044	0.0058	0.0023	−0.0010	−0.0022
	MSE	0.0427	0.0410	0.0224	0.0209	0.0079	0.0083
SSt	Bias	1.6202	0.8584	1.0212	1.3580	0.2751	1.0326
	MSE	0.9510	0.9504	0.9379	0.9430	0.9335	0.9282
SN	Bias	1.4444	0.9297	0.5085	0.9448	0.8340	2.0620
	MSE	1.2020	1.1738	1.1808	1.2000	1.2733	1.1664

Note: For the ${\mathrm{SHS}}_2$ model, the pure bias and MSE of the estimates are reported.

Table 3. Relative bias and MSE of the parameters estimates under skew Student-t distributed errors.

	n	100		200		500
Model		${\hat{\beta }}_1$	${\hat{\beta }}_2$	${\hat{\beta }}_1$	${\hat{\beta }}_2$	${\hat{\beta }}_1$	${\hat{\beta }}_2$
$\mathrm{SH}{\mathrm{S}}_1$	Bias	1.1092	1.1489	1.0177	1.0559	−2.4400	1.2461
	MSE	1.0263	1.0112	1.0160	1.0172	1.0454	1.0119
$\mathrm{SH}{\mathrm{S}}_2$	Bias	1.1678	1.1670	1.0147	0.9766	−1.7733	1.4539
	MSE	1.0680	1.0496	1.0515	1.0524	1.0722	1.0502
SSt	Bias	0.0027	−0.0048	−0.0033	−0.0014	0.0001	−0.0007
	MSE	0.0074	0.0084	0.0041	0.0039	0.0016	0.0016
SN	Bias	0.2147	0.9159	1.4184	1.4412	−24.2274	1.9104
	MSE	1.5658	1.5123	1.4132	1.4089	1.5629	1.4771

Note: For the SSt model, the pure bias and MSE of the estimates are reported.

The results show that for all true error distributions, both the bias and MSE tend to zero as the sample size is increased; hence we can conclude the consistency and unbiasedness of the estimators of regression coefficients. Further, for all sample sizes and simulation scenarios, the estimators obtained by assuming the SN model for errors are inefficient. Its loss of efficiency comparing to the SSt model is substantial. For the scenario (3), when the error model is an SSt, the estimators obtained by the true model are uniformly more efficient; however, the difference with ${\mathrm{SHS}}_1$ and ${\mathrm{SHS}}_2$ models are minor. The same results can be derived for the scenario (1). Therefore, we can conclude the robustness of our proposed SHS models to misspecification of a heavy-tailed distribution like the SSt model in estimating regression parameters.

Figure 4 displays the average bias plus and minus the average standard error (SE) for the regression parameters under each scenario. The results in Figure 4 confirm those obtained in Tables 1–3; i.e. both the average bias and standard errors tend to zero as the sample size is increased. The lengths of intervals for all scenarios and sample sizes, under all models but the SN model, are almost the same. For the SN model, the length of the interval is the longest one.

Figure 4.

Average bias and confidence intervals on a standardized scale for different error distributions by sample size and true error distributions: ${\mathrm{SHS}}_1$ (left), $\mathrm{SH}{\mathrm{S}}_2$ (middle), and SSt (right). $\mathrm{SH}{\mathrm{S}}_1$ (solid line), $\mathrm{SH}{\mathrm{S}}_2$ (dashed line), SSt (dotted line) and SN (dotdash line).

For the regression parameters, we also computed empirical coverage rates under different models for errors. Figure 5 displays the results for all scenarios and sample sizes. The results in Figure 5 show that the estimated coverage rates are close to the nominal level of $95\mathrm{\%}$ for all sample sizes and simulation scenarios. It justifies the asymptotic normality of the ML estimators of the regression coefficients.

Figure 5.

Coverage rate for each regression parameter for different error distributions by sample size and true error distributions: SHS₁ (left), ${\mathrm{SHS}}_2$ (middle), and $\mathrm{SSt}$ (right). $\mathrm{SH}{\mathrm{S}}_1$ (solid line), $\mathrm{SH}{\mathrm{S}}_2$ (dashed line), SSt (dotted line) and SN (dotdash line).

It is also of interest to directly compare the goodness of fit of the models to the observed data. To this end, we estimated two model assessment measures: the AIC and the BIC. We do not compare the absolute values of AICs or BICs directly, but considers their difference which is defined (e.g.for AIC) as

$$\mathrm{DAIC}(\mathcal{M})=\mathrm{AIC}({\mathcal{M}}_0)-\mathrm{AIC}(\mathcal{M})$$

in which the ${\mathcal{M}}_0$ is the true model for the error term and $\mathcal{M}$ is the misspecified one. The negative values for DAICs/DBICs are in favor of the correct model. Box plots of DAIC values for all simulation scenarios and sample sizes are shown in Figure 6, and box plots of DBIC values are displayed in Figure 7.

Figure 6.

Box plots of DAIC values for misspecified error models by sample size and true error distributions: ${\mathrm{SHS}}_1$ (left), ${\mathrm{SHS}}_2$ (middle), and SSt (right).

Figure 7.

Box plots of DBIC values for misspecified error models by sample size and true error distributions: ${\mathrm{SHS}}_1$ (left), ${\mathrm{SHS}}_2$ (middle), and SSt (right).

For scenarios (1) and (3), the correct model shows superior performance, mainly when the sample size is increased. However, for scenario (2), using an incorrectly specified SSt or ${\mathrm{SHS}}_1$ model leads to better performance in a model selection based on both AIC and BIC. Also, for all sample sizes and simulation scenarios, the SN model has poor performance.

To some extent, it can be said that for the correct SSt model, the performance of incorrectly specified SHS models is approximately equivalent to the correct one based on AIC/BIC. Again, it shows the robustness of the proposed SHS models in model selection for heavy-tailed data.

5. Real examples

In this section, we present three real data analyses with (i) Acoustic comfort evaluation data, (ii) Pricing diamond stones data, and (iii) Martin Marietta Company data. We compared eight different distributions for error terms, including ${\mathrm{SHS}}_1$, ${\mathrm{SHS}}_2$, SN, SSt, GSH, SGHS, BHS, and normal distributions, having $\mu =0$.

In this paper, for testing goodness of regression fit under different models for error terms, we consider the Kolmogorov–Smirnov test based on the empirical distribution of residuals. Specifically, we assume that the regression model can be written in a location-scale form as

$$Y={\mathit{\beta }}^{\mathrm{\prime }}\mathit{x}+\sigma ϵ$$

with error distribution $F_{ϵ}(y)=\mathrm{P}(\frac{Y-{\mathit{\beta }}^{\mathrm{\prime }}\mathit{x}}{\sigma }\le y)$. If $\hat{\mathit{\theta }}$ denotes the ML estimates of all parameters including both regression coefficients and parameters of error distribution, then the error distribution under this model is built as

$$F_{{ϵ}_0}(y)=\mathrm{P}(\frac{Y-{\hat{\mathit{\beta }}}^{\mathrm{\prime }}\mathit{x}}{\hat{\sigma }}\le y).$$

Therefore, the linear regression model with a specific distribution for error terms is true if and only if the distributions $F_{ϵ}$ and $F_{{ϵ}_0}$ are the same. By this result, a Kolmogorov–Smirnov type test for the error distribution can be constructed by comparing the empirical distribution of the residuals, ${\hat{F}}_{ϵ}$, with the one estimated under the assumed distribution, ${\hat{F}}_{{ϵ}_0}$, as follows:

$$T=\sqrt{n}\underset{y\in \mathcal{R}}{sup}|{\hat{F}}_{ϵ}(y)-{\hat{F}}_{{ϵ}_0}(y)|.$$

5.1. Acoustic comfort evaluation data

Zhang et al. [37] used a linear regression framework with Gaussian error terms to model the acoustic comfort evaluation in the common problems of the engineering vehicles with high environmental noise. This high-level noise pollutes the surrounding environment and endangers the driver's physical and mental health. As Zhang et al. [37] have been noticed, in the future, acoustic vehicle comfort will be an essential topic for research due to its close connection to daily life. Therefore, it is vital to address how to evaluate the acoustic comfort of engineering vehicles.

The most studies of acoustic comfort focus on acoustic subjective and objective evaluations and their mathematical mapping [10,25,35]. The objective assessment of acoustic comfort is based on psychoacoustic parameters, including loudness, sharpness, and so on. For subjective evaluation, annoyance could also be a proper index. Linear regression is a standard model to establish the mapping function of psychoacoustic objective parameters and subjective assessment as well (Zhang et al. [37]).

The data, in this example, include an actual application case of 50 noise samples from forklifts. The subjective evaluation index (annoyance) was divided into ten grades, in which the noise annoyance was further subdivided into five parts from low to high, and each part had two grades, as shown in Table 1 of Zhang et al. [37]. Similar to Zhang et al. [37], we selected nine objective parameters as the objective evaluation indexes: Linear sound pressure level (LSPL), A-weighted sound pressure level (ASPL), loudness, sharpness, roughness, fluctuation, tonality, articulation index (AI), and impulsiveness. The details of the data are given in Table 2 of Zhang et al. [37]. These data include one (or even two) extreme observations (Figure 2 in Zhang et al. [37], not reported here). Hence, it means that it may not be relevant to assume Gaussian errors for the regression model. By this feature, we considered the proposed different families for error distribution.

We first performed multiple covariate analyses by using the ML approach to select the best subset of objective parameters. From our results based on all considered error distributions, not shown here, our analysis will be centered on the model with only loudness, sharpness, and impulsiveness as the covariates. We obtained the ML estimates and their standard errors for the selected covariates and computed AICs and BICs, under all models for errors. The results are reported in Tabel 4. It shows that the normal model returns substantially different regression coefficient estimates in comparison with other models. Generally, there is not a consistent agreement in regression coefficient estimates between the given error models, but for our proposed SHS models. All three covariates have a significant positive effect on the response; i.e. higher amounts of annoyance are associated with higher values of these covariates simultaneously.

Table 4. ML inferences under different models for errors for forklift data.

	${\mathrm{SHS}}_1$	${\mathrm{SHS}}_2$	BHS	GSH	SGHS	SSt	SN	Normal
Regression parameters:
constant	$-1.3863$	$-1.4280$	$-1.0222$	$-0.7977$	$-0.1594$	$-1.1855$	$-1.3091$	$-0.0069$
se	$0.4926$	$0.5229$	$0.9308$	$0.4627$	$1.5459$	$0.4416$	$0.6174$	$0.6901$
loudness	$0.0473$	$0.0479$	$0.0534$	$0.0534$	$0.0477$	$0.0415$	$0.0472$	$0.0474$
se	$0.0065$	$0.0062$	$0.0097$	$0.0054$	$0.0126$	$0.0040$	$0.0065$	$0.0084$
sharpness	$0.7300$	$0.7046$	$0.4902$	$0.5062$	$0.5282$	$0.8471$	$0.7070$	$0.5585$
se	$0.2279$	$0.2225$	$0.3734$	$0.1799$	$0.5466$	$0.1262$	$0.2441$	$0.3026$
impulsiveness	$2.0577$	2.0420	$2.0669$	$1.9013$	$1.3515$	$1.5852$	$1.7040$	$1.2190$
se	$0.2916$	$0.2826$	$0.2868$	$0.2854$	$0.6233$	$0.3781$	0.4005	$0.4682$
Distributional parameters:
Scale	$0.1354$	$0.1906$	$0.1301$	$0.4901$	$0.3880$	$0.2985$	$0.6869$	$0.5082$
se	$0.0468$	$0.0510$	$0.1176$	$0.1131$	$0.0820$	$0.0914$	$0.0831$	$0.0508$
Shape1		$0.5131$	$0.6751$	$-2.7291$	$0.8058$	$2.0511$	$2.6029$
se		$0.1903$	$0.8729$	$0.2777$	$0.3675$	$1.1787$	$0.8832$
Shape2			$0.3376$		$1.0972$
se			$0.3327$		$1.3161$
d.f.	$0.3325$					$0.9563$
se	$0.1552$					$0.2189$
AIC	$\mathbf{57.9143}$	$58.2685$	$60.6665$	59.0161	68.5898	71.7423	74.6952	84.1990
BIC	$\mathbf{69.3865}$	$69.7406$	$74.0507$	70.4882	81.9739	85.1265	84.1673	93.7591

Note: Bold values indicate the best selected one.

Smaller values of AIC and BIC indicate a more appropriate model. Hence, for the models we considered here, the ${\mathrm{SHS}}_1$ model fits the data better according to both AIC and BIC, with the same complexity as others. It also fits the data much better than the normal (Normal) model for the errors. The performance of the ${\mathrm{SHS}}_2$ model is almost equivalent to the ${\mathrm{SHS}}_1$ model. Tabel 5 also shows the results of the Kolmogorov–Smirnov test corresponding to all error models. According to the results, we can verify the goodness of fit of all models, excluding the normal model, when the size of the test is 0.1.

Table 5. The Kolmogorov–Smirnov test for forklift data.

	${\mathrm{SHS}}_1$	${\mathrm{SHS}}_2$	BHS	GSH	SGHS	SSt	SN	Normal
T	0.0577	$0.0547$	0.0628	$0.0584$	$0.0978$	$0.1594$	$0.1427$	$0.1813$
P-value	0.9928	$0.9964$	$0.9822$	$0.9917$	$0.6881$	$0.1411$	$0.2365$	0.0656

Overall, the results of forklift data analysis provide strong arguments in favor of modeling the tail of the errors when it is semi-heavy, not heavy.

5.2. Diamond data

Chu [8] describes the development of a pricing model for diamond stones using data that appeared in an advertisement in Singapore's Business Times edition of 18 February 2000. A total of 308 diamond stones were included in the analysis. This dataset is available in the Ecdat R package [21]. We assumed a linear model with the logarithm of the price of diamond stones, as the response variable, and the covariates, weight (carat), clarity, color, and certification body. The weight of a diamond stone is given in terms of carat units. One carat is equal to 0.2 g.

The last three covariates are categorical. Clarity of a diamond stone is classified in descending order as internally flawless (IF), very very slightly imperfect (VVS1 or VVS2), and very slightly imperfect (VS1 or VS2). The most prized diamonds display color purity. Top color purity invites a grade of D. Subsequent degrees of color purity are graded E, F, G, and so on. For the present data, we are facing six different degrees of color purity. Also, the different certification bodies assay diamond stones and provide each of them with a certificate listing their caratage and their grades of clarity and color. Three certification bodies are GIA, IGI, and HRD [8].

We included the categorical covariates in the regression model as a set of dummy variables. We selected color I as the baseline category for the color purity and compared it to the other five colors. For the clarity, we selected VVS2 as the baseline category. Likewise, for the certification body, the IGI is chosen as the baseline category. According to the results of Chu [8], we also considered an interaction term between carat and certification bodies.

Table 6 shows the ML estimates and their standard errors of the regression coefficients and the parameters of error distribution under different models for error terms. Notably, the result from the normal model is different from the rest, exhibiting more inflation in standard errors of the estimated coefficients compared to others. This inflation results in different conclusions comparing to other models; for example, based on the normal errors, the difference between VS1 and VVS2 is not significant, which is not in agreement with the rest. When the other models for errors are applied, all effects are significant and are consistent in the sign of estimates. Table 6 also reports the corresponding values of AIC and BIC. Both criteria provide strong support for the ${\mathrm{SHS}}_2$ model and indicate that our proposed model is the most appropriate distribution for the error terms. For this example, the performance of the ${\mathrm{SHS}}_2$ model is also almost equivalent to the ${\mathrm{SHS}}_1$ model.

Table 6. ML inferences under different models for errors for diamond data.

	${\mathrm{SHS}}_1$	${\mathrm{SHS}}_2$	BHS	GSH	SGHS	SSt	SN	Normal
Regression parameters:
constant	$5.9500$	$5.7900$	$5.9039$	$5.8063$	$5.7959$	$5.9224$	$5.9725$	$5.5477$
se	$0.0437$	$0.0320$	$0.1153$	$0.0404$	$0.0453$	$0.0409$	$0.0415$	$0.0934$
carat	$3.4638$	$3.8540$	$3.4344$	$3.5462$	$3.6555$	$3.4782$	$3.4462$	$3.7301$
se	$0.0920$	$0.0736$	$0.1680$	$0.1043$	$0.0945$	$0.0942$	$0.0798$	$0.1082$
D	$0.4567$	$0.4609$	$0.4284$	$0.4422$	$0.4123$	$0.4588$	$0.4368$	$0.5852$
se	$0.0276$	$0.0194$	$0.0391$	$0.0324$	$0.0327$	$0.0272$	$0.0323$	$0.0603$
E	$0.3676$	$0.3887$	$0.3513$	$0.3982$	$0.4327$	$0.4353$	$0.4006$	$0.4790$
se	$0.0194$	$0.0145$	$0.0430$	$0.0242$	$0.0269$	$0.0234$	$0.0246$	$0.0420$
F	$0.2868$	$0.3436$	$0.2624$	$0.3277$	$0.3319$	$0.3490$	$0.3223$	$0.4061$
se	$0.0174$	$0.0154$	$0.0355$	$0.0218$	$0.0225$	$0.0199$	$0.0220$	$0.0401$
G	$0.2014$	$0.2399$	$0.1776$	$0.2317$	$0.1969$	$0.2500$	$0.2169$	$0.3115$
se	$0.0182$	$0.0146$	$0.0361$	$0.0229$	$0.0232$	$0.0211$	$0.0228$	$0.0438$
H	$0.1277$	$0.1662$	$0.0722$	$0.1329$	$0.1074$	$0.1521$	$0.1221$	$0.2296$
se	$0.0183$	$0.0147$	$0.0311$	$0.0227$	$0.0228$	$0.0197$	$0.0226$	$0.0450$
IF	$0.1707$	$0.2418$	$0.1893$	$0.1680$	$0.1888$	$0.2122$	$0.1857$	$0.3120$
se	$0.0194$	$0.0174$	$0.0278$	$0.0230$	$0.0254$	$0.0234$	$0.0243$	$0.0499$
VS1	$-0.0879$	$-0.0593$	$-0.0886$	$-0.0892$	$-0.1050$	$-0.0825$	$-0.1191$	$-0.0086$
se	$0.0142$	$0.0111$	$0.0237$	$0.0178$	$0.0201$	$0.0160$	$0.0191$	$0.0379$
VS2	$-0.1549$	$-0.1455$	$-0.1820$	$-0.1845$	$-0.2437$	$-0.1800$	$-0.1968$	$-0.1102$
se	$0.0168$	$0.0137$	$0.0234$	$0.0198$	$0.0256$	$0.0192$	$0.0210$	$0.0375$
VVS1	$0.0697$	$0.0824$	$0.0975$	$0.0884$	$0.0506$	$0.1001$	$0.0976$	$0.1715$
se	$0.0147$	$0.0116$	$0.0279$	$0.0201$	$0.0195$	$0.0191$	$0.0208$	$0.0357$
GIA	$0.6251$	$0.7381$	$0.4466$	$0.4919$	$0.5708$	$0.5229$	$0.4971$	$0.6242$
se	$0.0562$	$0.0433$	$0.1026$	$0.0454$	$0.0572$	$0.0644$	$0.0528$	$0.0669$
HRD	$0.7555$	$0.8444$	$0.8357$	$0.8828$	$0.8750$	$0.8587$	$0.7990$	$1.2225$
se	$0.0620$	$0.0552$	$0.0753$	$0.0690$	$0.0729$	$0.0751$	$0.0720$	0.1298
carat × GIA	$-0.9328$	$-1.2494$	$-0.6261$	$-0.7382$	$-0.8297$	$-0.7152$	$-0.6610$	$-0.9127$
se	$0.1081$	$0.0896$	$0.2274$	$0.1083$	$0.1209$	$0.1265$	$0.0993$	$0.1160$
carat × HRD	$-1.1175$	$-1.3926$	$-1.1072$	$-1.2052$	$-1.2271$	$-1.1528$	$-1.0701$	$-1.6104$
se	$0.1080$	$0.0958$	$0.1595$	$0.1218$	$0.1145$	$0.1192$	$0.1079$	$0.1752$
Distributional parameters:
Scale	0.1052	$0.0338$	$0.1301$	$0.1116$	$0.1212$	$0.1631$	$0.1722$	$0.1564$
se	$0.0060$	0.0073	0.0734	$0.0059$	$0.0117$	$0.0123$	$0.0112$	$0.0198$
Shape1		$-0.7439$	$1.0816$	$3.0670$	$1.2336$	$-3.0234$	$-2.2712$
se	–	0.0582	2.2730	$1.2692$	$0.2120$	$0.5657$	$0.5393$
Shape2			1.8586		$-0.6116$
se	–	–	0.3920		0.5829
d.f.	9.3565					5.8957
se	(3.4098)	–	–			1.2959
AIC	−471.6554	−493.3932	−421.9113	−462.4386	−430.2067	−448.6855	−459.4364	−322.4285
BIC	−408.2437	−429.9815	−354.7695	−399.0269	−363.0649	−381.5437	−398.0247	−262.7469

Tabel 7 shows the results of the Kolmogorov–Smirnov test for different error models, rendering the goodness of fit for all models when the size of the test is 0.05. However, a substantially better fit of the ${\mathrm{SHS}}_2$ model could be seen clearly in Figure 8.

Table 7. The Kolmogorov–Smirnov test for diamond data.

	${\mathrm{SHS}}_1$	${\mathrm{SHS}}_2$	BHS	GSH	SGHS	SSt	SN	Normal
T	0.0725	$0.0656$	$0.0692$	$0.0744$	$0.0669$	$0.0672$	$0.0726$	$0.0688$
P-value	0.0788	$0.1411$	$0.1044$	$0.0663$	$0.1264$	$0.1239$	$.0779$	$0.1082$

Figure 8.

Histograms of the residuals and the corresponding estimated densities for different error models for diamond data.

5.3. Martin Marietta company data

Butler et al. [7] analyzed a set of data on the excess rate of return for the Martin Marietta Company. They considered a linear regression of Y, the excess rate of return for the Martin Marietta Company, on CRSP, an index of the excess rate of return for the New York stock exchange, as follows:

$$Y={\beta }_0+{\beta }_1\mathrm{CRSP}+ϵ.$$ (4)

The data consist of 60 monthly observations collected in 5 years, from January 1982 to December 1986. This dataset has also been analyzed by Azzalini and Capitanio [4], Taylor and Verbyla [33], DiCiccio and Monti [11], and Salazar et al. [28]. These data include one very extreme observation (Figure 9); Moreover, according to DiCiccio and Monti [11], some diagnostic measures show that two additional points are possible outliers. Therefore it indicates that it may not be appropriate to assume Gaussian errors for the regression model.

Figure 9.

Scatterplot of the Martin Marietta data and fitted regression lines with different distributions for error term.

We now proceed to consider the proposed different families for error distributions. The ML inferences are reported in Tabel 6. It shows that the Normal estimates return substantially different regression coefficient estimates in comparison with other models. As Butler et al. [7] have also mentioned, the estimated regression coefficients produced by the Normal method classify Martin Marietta Company into a different risk classification. The Normal estimate of ${\hat{\beta }}_1=1.8024$ means Martin Marietta Company rates as a very aggressive investment relative to the market portfolio. However, the corresponding estimates for other models put the company as moderately aggressive. Further, the corresponding estimates for SSt and SN models are somewhat different compared to semi-heavy-tailed distribution families. The smaller values for ${\hat{\beta }}_1$ in our proposed semi-heavy-tailed models than the SSt and SN models, emphasize more on classifying the company as moderately aggressive.

Table 8. ML inferences under different models for errors for the Martin Marietta Company data.

	${\mathrm{SHS}}_1$	${\mathrm{SHS}}_2$	BHS	GSH	SGHS	SN	SSt	Normal
Regression parameters:
Intercept	−0.0275	−0.0460	−0.0492	−0.0073	−0.0075	−0.0933	−0.0755	0.0011
	(0.0125)	(0.0167)	(0.0332)	(0.0087)	(0.0089)	(0.0125)	(0.0207)	(0.0087)
Slope	1.2482	1.2643	1.2515	1.2456	1.2459	1.3790	1.3391	1.8024
	(0.1937)	(0.1979)	(0.2035)	(0.2029)	(0.1942)	(0.2415)	(0.2087)	(0.2015)
Distributional parameters:
Scale	0.0313	0.0400	0.0368	0.0911	0.0849	0.1362	0.0937	0.0928
	(0.0076)	(0.0072)	(0.0232)	(0.0180)	(0.0155)	(0.0293)	(0.0042)	(0.0085)
Shape1		0.4386	1.2657	−2.5607	0.9019	3.9154	2.4587
		(0.1555)	(1.3370)	(0.3520)	(0.0752)	(1.4049)	(1.4575)
Shape2			0.5108		−2.4090
			(0.3731)		(0.4474)
d.f.	0.4320						6.0660
	(0.1488)						(5.4508)
AIC	−135.1765	−135.3089	−133.176	−132.1232	−124.0341	−134.0729	−124.0341	−108.9898
BIC	−126.7991	−126.9316	−122.7043	−123.7458	−115.6567	−123.6012	−115.6567	−102.7068

For the models we considered here, the ${\mathrm{SHS}}_2$ model fits the data better according to both AIC and BIC. It also fits the data much better than the normal model for the errors. Similar to two previous examples, the performance of the ${\mathrm{SHS}}_1$ model is almost equivalent to the ${\mathrm{SHS}}_2$ model.

Figure 8 shows the fitted regression lines following the supposed distributions for the error term in the model (4). For the Normal fitted model, the regression line is shrunk to upward due to the extreme observations. However, the semi-heavy-tailed models limit the influence of extreme observations and provide a robust alternative when there exist a few outliers. On the other hand, both SN and SSt models give high leverage observations less influence than the semi-heavy-tailed models due to their heavy tail properties, which causes their regression lines to be more shrunk to downward. Therefore, a heavy tail assumption for the distribution of errors for these data results in a poor model fit.

For these data, the Kolmogorov–Smirnov test (Table 9) verifies the goodness of regression fit under all models for errors, as well.

Table 9. The Kolmogorov–Smirnov test for Martin Marietta data.

	${\mathrm{SHS}}_1$	${\mathrm{SHS}}_2$	BHS	GSH	SGHS	SSt	SN	Normal
T	0.0864	0.0852	0.0897	0.0761	0.0759	0.0768	0.1386	0.1474
P-value	0.7289	0.7436	0.6861	0.8513	0.8534	0.8445	0.1816	0.1334

6. Conclusions

To consider the semi-heavy-tailedness of data, in the presence of a few outliers, could provide more efficient inferences in the real applications when they do not possess a heavy tail. We developed a regression methodology based on the semi-heavy-tailedness of the errors in a linear model under two new skewed version of the hyperbolic secant distribution. We fitted our models using ML, which is the standard nowadays. We also examined the asymptotic properties of the ML estimators of the regression parameters in our proposed methodology, including consistency and asymptotic normality.

This article demonstrates the utility of the proposed methodology in applied situations when data are not too heavy, using simulated data as well as data from real applications. Our suggested models are also robust comparing to actual heavy-tailedness of data.

Future work will also include extensions of the models for use with multivariate and spatial data. An extension to multivariate data would consist of not only to consider the spatial dependence of each variable but also to examine the spatial dependence between each pair of variables.

Disclosure statement

No potential conflict of interest was reported by the author(s).