Heteroscedastic partially linear model under skew-normal distribution with application in ragweed pollen concentration

Abstract

We introduce a new class of heteroscedastic partially linear model (PLM) with skew-normal distribution. Maximum likelihood estimation of the model parameters by the ECM algorithm (Expectation/Conditional Maximization) as well as influence diagnostics for the new model are investigated. In addition, a Likelihood Ratio test for assessing the homogeneity of the scale parameter is presented. Simulation studies for assessing the performance of the ECM algorithm and the Likelihood Ratio test statistics for homogeneity of variance are developed. Also, a study for misspecification of the structure function is considered. Finally, an application of the new heteroscedastic PLM to a real data set on ragweed pollen concentration is presented to show that it provides a better fit than the classic homocedastic PLM. We hope that the proposed model may attract applications in different areas of knowledge.

1. Introduction

Partially linear models (PLMs) or semiparametric models have been studied by various authors (see, for instance, [16,30], and references therein). These models add a nonparametric component to the usual linear relation between the response and explanatory variables. When the data set presents heterogeneity of variance, the PLMs can be extended by incorporating, for example, a positive continuously differentiable function involving a subset of explanatory variables, named here as Heteroscedastic Partially Linear Models (HPLM). In this context, Chen and You [5] proposed to estimate the nonparametric component through kernel smoothing and constructed a semiparametric generalized least-squares estimator for the parametric component. Ma et al. [26] studied the HPLMs with an unspecified partial baseline component and a nonparametric variance function, proposing a family of consistent estimators and investigate their asymptotic properties. They showed that the optimal semiparametric efficiency bound can be reached by a semiparametric kernel estimator in this family. You et al. [35] proposed a test of heteroscedasticity, a two-step estimator of the heteroscedastic variance function, semiparametric generalized least-squares estimators of the parametric and nonparametric components of the model, and bootstrap goodness-of-fit test to see whether the nonparametric component can be parametrized. More recently, Keilegom and Wang [21] studied a general class of location-dispersion regression models, in which both the location function and the dispersion function are semiparametrically modeled. Note that these works use the normal (Gaussian) distribution to model data, including the use of the traditional method of least-squares in the models.

However, in some situations, where the data are asymmetric, the above methods may not be appropriate, particularly when the response assumes real values. In this sense, a distribution that accommodates skewness, and includes the normal distribution as a special case, was introduced by Azzalini [2], named skew-normal. This class has been studied by various authors in different contexts (see, for example, [4,23,27], among others). An extensive, but not exhaustive, list of the publications involving skew-normal distribution can be accessed on Azzalini's homepage¹. Ferreira and Paula [12] proposed estimation and diagnostic for homoscedastic PLMs in an asymmetric context using the skew-normal distribution (PLM-SN), by developing the expectation-maximization (EM) algorithm for linear regression models and diagnostic analysis through local influence as well as generalized leverage, following the approach of Zhu and Lee [37]. To our knowledge, there is no research in the scientific literature involving HPLMs with an asymmetric error.

So, a natural extension is to propose a PLM model using SN distribution in the presence of heteroscedasticity and to develop influence diagnostics for detecting influential observations, which is the objective of this work. It is important to note that we implemented the ECM algorithm (Expectation/Conditional Maximization) and obtained closed-form expressions for all the estimators of the parameters of the proposed model, except for the parameter of heterogeneity. Influence analysis is an important and key step in data analysis after parameter estimation. There are two approaches for detecting influential observations: the case-deletion approach [7,38] and the local influence approach [6,37]. Since the estimation of the parameters of the HPLM model using skew-normal distribution will be via the ECM algorithm, then, in this work, the local influence approach will be based on the complete-data technique that uses the conditional expectation of the complete-data log-likelihood function (Q-function), from the ECM algorithm, as proposed by Zhu and Lee [37].

Another interesting aspect is that in the PLM, the standard assumption is that all observations have equal variances, but in some cases these models do not comply with this assumption, affecting the efficiency of the estimators. Therefore, it is important to develop tests that allow us to determine the presence or absence of such homogeneity. In this paper, we propose a Likelihood Ratio test o check the homogeneity of the scale parameter in the HPLM-SN model.

The rest of the work is organized as follows. In Section 3, the heteroscedastic partially linear model, under the assumption that the errors follow the skew-normal distribution, is presented and a penalized log-likelihood function is considered for the parameter estimation. The ECM algorithm to obtain the maximum likelihood estimate of the parameters of the HPLM-SN model is given in Section 4 and the discussion of degrees of freedom estimation and goodness-of-fit is also presented. Residual analysis is given in Section 5 to identify atypical observations and/or model misspecification once residuals are measures of agreement between the data and the fitted model. Section 6 presents local influence measures using the methodology proposed by Zhu and Lee [37]. The Hessian and the corresponding matrices of four perturbation schemes are also derived. In Section 7, we discuss the Likelihood Ratio test to test the homogeneity of a scale parameter. The properties of the Likelihood Ratio test statistics are investigated through Monte Carlo simulations. Section 8 deals with simulation studies to evaluate the efficiency of the ECM algorithm and the Likelihood Ratio test for homogeneity of variance. In addition, a study for misspecification of the structure function is considered. Finally, in Section 9, we illustrate the methodology by considering an application with a real data set, where the main interest is to explain the daily pollen concentration. Section 10 summarizes the contributions of the paper.

2. Motivating example

We illustrate our proposed methods with a data set obtained from [32], which is available, e.g. in the R package SemiPar. This data set contains data on ragweed levels and meteorological variables for 335 days in Kalamazoo, Michigan, USA, from 1991 to 1994. Ferreira and Paula [12] have analyzed this data set using a homoscedastic partially linear model under asymmetric distributions. Now we revisit this data set with the aim of expanding the inferential results to the heteroscedastic partially linear skew-normal model. Following [12], the explanatory variables included in our proposed model are the indicator of significant rain (1: at least 3 hours of steady or brief but intense rain, 0: other), temperature (degrees Fahrenheit), wind speed forecast for the following day (mph) and time (t-days in season). The response variable $(\sqrt{y})$ is the square root of the pollen concentration of ragweed ( $grains/m^3$). First, we fit a PLM-SN model to the data as specified by [12]

$$\sqrt{y_i}={\beta }_{1}rai{n}_i+{\beta }_{2}temperatur{e}_i+{\beta }_{3}win{d}_i+f(t_i)+{ϵ}_i,i=1,\dots ,335,$$ (1)

where ${ϵ}_i$ are iid errors following a skew-normal distribution, i.e. $SN(0,{\sigma }^2,\lambda )$. To detect deviations from the error model, we use the standard residuals $e_i^0,i=1,\dots ,335$, defined in Section 5. Figure 1(a) presents the plot of standard residuals $e_i^0$'s, where we may see a heteroscedastic behaviour of the residuals. Furthermore, Figures 1(b and c) show the scatter plots between these residuals and the explanatory variables of temperature and wind. We can note that the heteroscedasticity of the residuals is due to the temperature.

Figure 1.

Standard residuals of the Homoscedastic PLM-SN model fitted ( $e_i^0$'s) to the ragweed levels data: (a) plot of the residuals, (b) scatter plot between residuals and temperature, and (c) scatter plot between residuals and wind.

Thus, this paper aims to introduce a new class of heteroscedastic PLM under skew-normal distribution. In other words, this work aims to develop current research topics which are of paramount importance for proper data analysis. When skewness and heteroscedasticity problem is a concern in a partially linear regression model this manuscript may be a useful reference to cope with those problems, and hence it is a good contribution to the statistics literature.

3. The proposed model

In this section, we propose the heteroscedastic partially linear model under the assumption that the errors follow the skew-normal distribution and discuss the penalized function method, which is often required to maximize the penalized likelihood function.

3.1. Skew-normal distribution

We start with the definition of the skew-normal (SN) distribution that will be used in this article; see [2] for more details. A random variable $Y\sim \mathrm{SN}(\mu ,{\sigma }^2,\lambda )$ if its probability density function (pdf) is given by

$$f(y|\mu ,{\sigma }^2,\lambda )=2\varphi (y|\mu ,{\sigma }^2)\mathrm{\Phi }\left(\frac{\lambda (y-\mu )}{\sigma }\right),y\in \mathbb{R},$$ (2)

where $\varphi (\cdot ;\mu ,{\sigma }^2)$ stands for the pdf of the normal distribution with mean μ and variance ${\sigma }^2$, $\mathrm{\Phi }(\cdot )$ represents the cumulative distribution function (cdf) of the standard normal distribution. Its stochastic representation is given by

$$Y\stackrel{d}{=}\mu +\sigma (\delta |T_0|+(1-{\delta }^2{)}^{1/2}{T}_1),\mathrm{with}\delta =\frac{\lambda }{\sqrt{1+{\lambda }^2}},$$ (3)

where $|T_0|$ denotes the absolute value of $T_0$, $T_0\sim \mathrm{N}(0,1)$ and $T_1\sim \mathrm{N}(0,1)$ are independent, and ‘ $\stackrel{d}{=}$’ means ‘distributed as’. This convenient hierarchical representation facilitates EM-type implementation for the maximum-likelihood estimation and can be used to simulate data. Note that if $Y\sim \mathrm{SN}(\mu ,{\sigma }^2,\lambda ),$ then $Z=(Y-\mu )/\sigma \sim \mathrm{SN}(0,1,\lambda ).$ A particular case of this distribution is the normal distribution ( $Y\sim N(\mu ,{\sigma }^2)$) when $\lambda =0$. From (3) it follows that the expectation and variance of Y are given, respectively, by

$$\mathrm{E}[Y]=\mu +c\sigma \delta \mathrm{and}\mathrm{Var}[Y]={\sigma }^2(1-c^2{\delta }^2),\mathrm{with}c=\sqrt{\frac{2}{\pi }}.$$ (4)

3.2. Model specification

In this section, we define the heteroscedastic partially linear model under skew-normal distribution. First, consider the homoscedastic partially linear model under skew-normal distribution (PLM-SN model), as defined by Ferreira and Paula [12], given by

$$Y_i={\mathbf{x}}_i^{\mathrm{\top }}\mathit{\beta }+f(t_i)+{ϵ}_i,i=1,\dots ,n,$$ (5)

where $Y_i$ denotes the response of the ith experimental unit, ${\mathbf{x}}_i$ is a known $p\times 1$ vector covariate vector, $\mathit{\beta }$ is a p-dimensional vector of unknown regression coefficients, $t_i$ is a scalar that may represent a value of a continuous covariate, for example, time, $f(\cdot )$ is a smoothing function, and ${ϵ}_i$ are independent random errors such that ${ϵ}_i\sim \mathrm{SN}(0,{\sigma }^2,\lambda )$. However, the actual scale parameter may be related to the ith observation $Y_i$ and thus its variance is nonconstant.

We extend the PLM-SN model defined in (5) with the assumption under a skew-normal structure in the presence of heteroscedasticity. Thus, for the ith experimental unit,

$${ϵ}_i\sim \mathrm{SN}(0,{\sigma }_i^2,\lambda ),{\sigma }_i^2={\sigma }^2{m}_i(\mathit{\rho },{\mathbf{z}}_i),i=1,\dots ,n,$$ (6)

where $m_i=m(\mathit{\rho },{\mathbf{z}}_i)$ is a known continuously differentiable positive function, ${\mathbf{z}}_i$ contains the values of the explanatory variables, which generally constitute, although not necessarily, a subset of ${\mathbf{x}}_i$, and $\mathit{\rho }:p^{\ast }\times 1$ is a vector of unknown parameters. If the variances depend on the values of some explanatory variables ${\mathbf{z}}_i$, for example, a specific form of $m_i$ is the log-linear model given by $m_i(\mathit{\rho },{\mathbf{z}}_i)=\exp (\sum _{j=1}^{p^{\ast }}{\rho }_j{z}_{ij})$; see [11,22,34] and references therein for more details. It is assumed that there is a ${\mathit{\rho }}_0$ such that $m({\mathit{\rho }}_0,{\mathbf{z}}_i)=1$, for all $i=1,\dots ,n$. We call the structure defined by (5) and (6) the HPLM-SN model (heteroscedastic partially linear skew-normal model).

Alternatively, the model (5)–(6) can be written in matrix form as follows

$$\mathbf{Y}=\mathbf{X}\mathit{\beta }+\mathbf{N}\mathbf{f}+\mathit{ϵ},$$ (7)

where $\mathbf{Y}=(Y_1,\dots ,Y_n{)}^{\mathrm{\top }}$, $\mathbf{X}$ is an $n\times p$ design matrix with rows ${\mathbf{x}}_i^{\mathrm{\top }}$, $\mathbf{N}$ is an $n\times q$ incidence matrix, $\mathbf{f}$ is a $q\times 1$ vector (unknown smooth curve) and $\mathit{ϵ}=({ϵ}_i,\dots ,{ϵ}_n{)}^{\mathrm{\top }}$ is an $n\times 1$ vector of random errors.

3.3. Penalized log-likelihood function

From expressions (6)–(7), we have that $Y_i\sim SN({\mu }_i,{\sigma }_i^2,\lambda )$, where ${\mu }_i={\mathbf{x}}_i^{\mathrm{\top }}\mathit{\beta }+{\mathbf{n}}_i^{\mathrm{\top }}\mathbf{f}$ and ${\mathbf{n}}_i^{\mathrm{\top }}$ is the ith row of $\mathbf{N}$, the observed-data log-likelihood function of $\mathit{\theta }=({\mathit{\beta }}^{\mathrm{\top }},{\sigma }^2,\lambda ,{\mathit{\rho }}^{\mathrm{\top }},{\mathbf{f}}^{\mathrm{\top }}{)}^{\mathrm{\top }}$ is given by

$$\begin{array}{r}\ell (\mathit{\theta })\propto -\frac{n}{2}\log {\sigma }^2-\frac{1}{2}\sum _{i=1}^n\log m_i-\frac{1}{2{\sigma }^2}\sum _{i=1}^n\frac{(y_i-{\mu }_i{)}^2}{m_i}+\sum _{i=1}^n\log \mathrm{\Phi }\left(\frac{\lambda (y_i-{\mu }_i)}{\sigma m_i^{1/2}}\right).\end{array}$$ (8)

The direct maximization of (8) is difficult due to the term $\mathrm{\Phi }(\cdot ).$ In addition, maximization of (8) without imposing restrictions on the function $\mathbf{f}(\cdot )$ may cause overfitting and nonidentification of $\mathit{\beta }$ (see, for instance, [14]). A well-known procedure that is based on the idea of log-likelihood penalization consists in incorporating a penalty function in the log-likelihood, such that

$$\begin{array}{r}{\ell }_p(\mathit{\theta },\alpha )=\ell (\mathit{\theta })-\frac{\alpha }{2}J(\mathbf{f}),\end{array}$$ (9)

where $J(\mathbf{f})$ denotes the penalty function over $\mathbf{f}(\cdot )$ and α is a smoothing parameter that controls the tradeoff between the goodness-of-fit, measured by large values of $\ell (\mathit{\theta })$, and the estimated smoothing function, measured by small values of $J(\mathbf{f})$. Therefore, the determination of the parameter α is a crucial part of the estimation process, for which different methods of choice are available in the literature, such as the Akaike information criterion or the Bayesian information criterion. Following [15,18], we use a natural cubic spline to estimate f and the general form of $J(\mathbf{f})={\int }_a^b[f^{(2)}(t){]}^2\mathrm{d}t,$ where $f^{(2)}(t)$ denotes the second derivative of $f(t)$ with $[a,b]$ containing the values $t_1^0,\dots ,t_q^0$ being the distinct and ordered values of $t_i$. Moreover, from Theorem 2.1 in [15], page 13, the penalty function will satisfy ${\int }_a^b[f^{(2)}(t){]}^{2}dt={\mathbf{f}}^{\mathrm{\top }}\mathbf{K}\mathbf{f},$ where $\mathbf{K}\in {\mathbb{R}}^{q\times q}$ is a nonnegative definite matrix that depends only on knots. In this article, we will use the methods of [15]- first method- and [10]- second method- to construct the matrices $\mathbf{N}$ and $\mathbf{K}$. They differ in the choice of knots that will be used to estimate the f curve. In the first method, $\mathbf{f}=(f(t_1^0),\dots ,f(t_q^0){)}^{\mathrm{\top }}$, with $t_1^0,\dots ,t_q^0$ being the distinct and ordered values of $t_i$ and $\mathbf{N}$ is an $n\times q$ incidence matrix whose $(i,j)$th element equals the indicator function $I(t_i=t_j^0)$ for $j=1,\dots ,q$. On the other hand, in the second method, the knots are chosen according to a pre-established interval using B-splines and we estimate $f(\cdot )$ as a B-spline of order 3 [3], i.e. $f(x)=\sum _{j=1}^q{f}_j{B}_j(x)$. In this case, the elements of $\mathbf{N}$ are given by $n_{ij}=B_j(t_i),i=1,\dots ,n$ and j = 1,.., q. A function ‘ $bspline(\mathbf{t},q,df)$’ in R [29] is used, where q is the number of equidistant knots desired by the user and df is an order of the polynomial, that equals 3, in this paper (a B-spline cubic); see Appendix 2 for more details. Thus, the expression of matrix $\mathbf{K}$ is described in Appendix 2.

4. Statistical inference

In this section, we discuss some inferential aspects in the HPLM-SN model as well as the penalized maximum likelihood estimation using the ECM algorithm. The ECM algorithm [28] to obtain the maximum likelihood estimate of $\mathit{\theta }$ and a discussion of degrees of freedom estimation are given in Section 4.1. The standard error estimation of $\hat{\mathit{\theta }}$ is presented in Appendix 1.

4.1. Parameter estimation using the ECM algorithm

In this section, we present an ECM algorithm for the ML estimation of the HPLM-SN model. To explore the ECM algorithm, we present the HPLM-SN model in an incomplete data framework, using the results presented in Section 3. Thus, from Equation (3), the set-up defined above can be written hierarchically as

$$\begin{array}{rl}{Y}_i|W_i=w_i& \sim N({\mu }_i+{\sigma }_i\delta w_i,{\sigma }_i^2(1-{\delta }^2)),\end{array}$$ (10)

$$\begin{array}{rl}{W}_i& \sim TN(0,1;0,+\mathrm{\infty }),\end{array}$$ (11)

for $i=1,\dots ,n$ all independent, where $TN(r,s;(a,b))$ denotes the univariate normal distribution $(N(r,s))$, truncated on the interval $(a,b)$ [20]. Let $\mathbf{y}=(y_1,\dots ,y_n{)}^{\mathrm{\top }}$ and $\mathbf{w}=(w_1,\dots ,w_n{)}^{\mathrm{\top }}.$ Then, under the hierarchical representation (10)–(11), it follows that the complete log-likelihood function associated with ${\mathbf{y}}_c=({\mathbf{y}}^{\mathrm{\top }},{\mathbf{w}}^{\mathrm{\top }}{)}^{\mathrm{\top }}$ is

$$\begin{array}{rl}{\ell }_c(\mathit{\theta }|{\mathbf{y}}_c)& \propto -n\log {\sigma }^2-\sum _{i=1}^n\log m_i-\frac{1}{2{\sigma }^2}\sum _{i=1}^n\frac{1}{m_i}\\ & \times [(1+{\lambda }^2)(y_i-{\mu }_i{)}^2-2\lambda w_i(y_i-{\mu }_i)+w_i^2].\end{array}$$

As in the original proposal of [9], the E-step of our algorithm consists of taking the conditional expectation $Q(\mathit{\theta }|{\hat{\mathit{\theta }}}^{(k)})=\mathrm{E}[{\ell }_c(\mathit{\theta }|{\mathbf{y}}_c)|\mathbf{y},{\hat{\mathit{\theta }}}^{(k)}]$, where ${\hat{\mathit{\theta }}}^{(k)}=({\hat{\mathit{\beta }}}^{(k)\mathrm{\top }},{\hat{\mathbf{f}}}^{(k)\mathrm{\top }},{\widehat{{\sigma }^2}}^{(k)},{\hat{\lambda }}^{(k)},{\hat{\mathit{\rho }}}^{(k)\mathrm{\top }}{)}^{\mathrm{\top }},$ is the current estimate of $\mathit{\theta }$ at the kth iteration. The maximum penalized likelihood estimate (MPLE) of $\mathit{\theta }$ is the value that maximizes the function

$$\begin{array}{r}{Q}_p(\mathit{\theta }|{\hat{\mathit{\theta }}}^{(k)})=Q(\mathit{\theta }|{\hat{\mathit{\theta }}}^{(k)})-\frac{\alpha }{2}J(\mathbf{f}).\end{array}$$ (12)

Given ${\hat{\alpha }}^{(k)}$, the M-step consists of the maximization of $Q_p(\mathit{\theta }|{\hat{\mathit{\theta }}}^{(k)})$ with respect to $\mathit{\theta }$. It follows, after some simple algebra, that the conditional expectation of the complete log-likelihood function has the form

$$\begin{array}{rl}Q(\mathit{\theta }|{\hat{\mathit{\theta }}}^{(k)})& \propto -n\log {\sigma }^2-\sum _{i=1}^n\log m_i-\frac{(1+{\lambda }^2)}{2{\sigma }^2}(\mathbf{y}-\mathbf{X}\mathit{\beta }-\mathbf{N}\mathbf{f}{)}^{\mathrm{\top }}\\ & \times \mathbf{H}(\mathbf{y}-\mathbf{X}\mathit{\beta }-\mathbf{N}\mathbf{f})-\frac{1}{2{\sigma }^2}{\widehat{{\mathbf{w}}^2}}^{(k)\mathrm{\top }}\mathbf{H}{\mathbf{1}}_n\\ & +\frac{\lambda }{{\sigma }^2}(\mathbf{y}-\mathbf{X}\mathit{\beta }-\mathbf{N}\mathbf{f}{)}^{\mathrm{\top }}\mathbf{H}{\hat{\mathbf{w}}}^{(k)},\end{array}$$ (13)

where ${\mathbf{1}}_n$ is a $n\times 1$ vector of 1's, $\mathbf{H}$ is a diagonal matrix of the vector $(m_1^{-1}(\mathit{\rho },{\mathbf{z}}_1),\dots ,m_n^{-1}(\mathit{\rho },{\mathbf{z}}_n))$, ${\hat{\mathbf{w}}}^{(k)}=({\widehat{w_1}}^{(k)},\dots ,{\widehat{w_n}}^{(k)}{)}^{\mathrm{\top }}$ and ${\widehat{{\mathbf{w}}^2}}^{(k)}=({\widehat{w_1^2}}^{(k)},\dots ,{\widehat{w_n^2}}^{(k)}{)}^{\mathrm{\top }}$ are $n\times 1$ vectors, with ${\hat{w}}_i^{(k)}=\mathrm{E}[W_i|y_i,{\hat{\mathit{\theta }}}^{(k)}]$ and ${\widehat{w^2}}_i^{(k)}=\mathrm{E}[W_i^2|y_i,{\hat{\mathit{\theta }}}^{(k)}]$.

4.1.1. Step-by-step instructions for the ECM algorithm

The ECM algorithm for the HPLM-SN model can be summarized in the following steps:

1. E-step: Given the current estimates ${\hat{\mathit{\theta }}}^{(k)}$ and ${\hat{\alpha }}^{(k)}$ at the kth iteration, we obtain the conditional expectation of the complete data log-likelihood function given the observed $\mathbf{y}$, named the Q-function, which is given by (13), such that

   $$\begin{array}{rl}{\hat{w}}_i^{(k)}& ={\hat{\lambda }}^{(k)}{e}_i^{(k)}+{\hat{\sigma }}_i^{(k)}{W}_{\mathrm{\Phi }}\left(\frac{{\hat{\lambda }}^{(k)}{e}_i^{(k)}}{{\hat{\sigma }}_i^{(k)}}\right),\end{array}$$ (14)

   $$\begin{array}{rl}{\widehat{w^2}}_i^{(k)}& ={\left[{\hat{\lambda }}^{(k)}{e}_i^{(k)}\right]}^2+{\widehat{{\sigma }^2}}_i^{(k)}+{\hat{\lambda }}^{(k)}\widehat{{\sigma }_i^{(k)}}{e}_i^{(k)}{W}_{\mathrm{\Phi }}\left(\frac{{\hat{\lambda }}^{(k)}{\mathrm{e}}_i^{(k)}}{{\hat{\sigma }}_i^{(k)}}\right),\end{array}$$ (15)

   where ${\mathrm{e}}_i^{(k)}=y_i-{\mathbf{x}}_i^{\mathrm{\top }}{\hat{\mathit{\beta }}}^{(k)}-{\mathbf{n}}_i^{\mathrm{\top }}{\hat{\mathbf{f}}}^{(k)}$, ${\widehat{{\sigma }^2}}_i^{(k)}={\widehat{{\sigma }^2}}^{(k)}{m}_i({\hat{\mathit{\rho }}}^{(k)},{\mathbf{z}}_i)$, $i=1,\dots ,n,$ and $W_{\mathrm{\Phi }}(u)=\varphi (u)/\mathrm{\Phi }(u)$.

   Conditional maximization steps (CM-steps) are given as follows.

2. CM-step 1: Fix ${\hat{\alpha }}^{(k)}$, update ${\hat{\mathit{\beta }}}^{(k)}$, ${\hat{\mathbf{f}}}^{(k)}$, ${\widehat{{\sigma }^2}}^{(k)}$ and ${\hat{\lambda }}^{(k)}$ as

   $$\begin{array}{rl}{\hat{\mathit{\beta }}}^{(k+1)}& ={\left({\mathbf{X}}^{\mathrm{\top }}{\hat{\mathbf{H}}}^{(k)}\mathbf{X}\right)}^{-1}{\mathbf{X}}^{\mathrm{\top }}{\hat{\mathbf{H}}}^{(k)}[\mathbf{y}-\mathbf{N}{\hat{\mathbf{f}}}^{(k)}-{\hat{\lambda }}^{(k)}/(1+({\hat{\lambda }}^{(k)}{)}^2){\hat{\mathbf{w}}}^{(k)}],\\ {\hat{\mathbf{f}}}^{(k+1)}& ={({\mathbf{N}}^{\mathrm{\top }}{\hat{\mathbf{H}}}^{(k)}\mathbf{N}+\frac{\alpha {\widehat{{\sigma }^2}}^{(k)}}{1+({\hat{\lambda }}^{(k)}{)}^2}\mathbf{K})}^{-1}{\mathbf{N}}^{\mathrm{\top }}\\ & \times {\hat{\mathbf{H}}}^{(k)}(\mathbf{y}-\mathbf{X}{\hat{\mathit{\beta }}}^{(k)}-\frac{{\hat{\lambda }}^{(k)}}{1+({\hat{\lambda }}^{(k)}{)}^2}{\hat{\mathbf{w}}}^{(k)}),\\ {\widehat{{\sigma }^2}}^{(k+1)}& =\{{\mathbf{1}}_n^{\mathrm{\top }}{\hat{\mathbf{H}}}^{(k)}{\widehat{{\mathbf{w}}^2}}^{(k)}-2{\hat{\lambda }}^{(k)}{\hat{\mathbf{w}}}^{(k{)}^{\mathrm{\top }}}{\hat{\mathbf{H}}}^{(k)}(\mathbf{y}-{\hat{\mathit{\mu }}}^{(k)})\\ & +[1+({\hat{\lambda }}^{(k}{)}^2]S({\hat{\mathit{\beta }}}^{(k)},{\hat{\mathbf{f}}}^{(k)},{\hat{\mathit{\rho }}}^{(k)})\}/2n\end{array}$$

   and

   $$\begin{array}{r}{\hat{\lambda }}^{(k+1)}={\hat{\mathbf{w}}}^{(k)\mathrm{\top }}{\hat{\mathbf{H}}}^{(k)}(\mathbf{y}-{\hat{\mathit{\mu }}}^{(k)})/S({\hat{\mathit{\beta }}}^{(k)},{\hat{\mathbf{f}}}^{(k)},{\hat{\mathit{\rho }}}^{(k)}),\end{array}$$ (16)

   where $S(\mathit{\beta },\mathbf{f},\mathit{\rho })=(\mathbf{y}-\mathit{\mu }{)}^{\mathrm{\top }}\mathbf{H}(\mathbf{y}-\mathit{\mu })$ and $\mathit{\mu }=\mathbf{X}\mathit{\beta }+\mathbf{N}\mathbf{f}$.
3. CM-step 2: Fix ${\hat{\alpha }}^{(k)}$ and given ${\hat{\mathit{\beta }}}^{(k+1)},{\hat{\mathbf{f}}}^{(k+1)},{\widehat{{\sigma }^2}}^{(k+1)}$ and ${\hat{\lambda }}^{(k+1)},$ update ${\hat{\mathit{\rho }}}^{(k)}$ as

   $$\begin{array}{r}{\hat{\mathit{\rho }}}^{(k+1)}={\mathrm{argmax}}_{\mathit{\rho }}Q(\mathit{\rho }|{\hat{\mathit{\beta }}}^{(k+1)},{\hat{\mathbf{f}}}^{(k+1)},{\widehat{{\sigma }^2}}^{(k+1)},{\hat{\lambda }}^{(k+1)}).\end{array}$$
4. Given ${\hat{\mathit{\theta }}}^{(k+1)},$ update ${\hat{\alpha }}^{(k)}$ as

   $$\begin{array}{r}{\hat{\alpha }}^{(k+1)}={\mathrm{argmin}}_{\alpha }\mathrm{BIC}(\alpha |{\hat{\mathit{\theta }}}^{(k+1)}).\end{array}$$

   We use the Bayesian information criterion (BIC) to select the better value of the α, given by

   $$\begin{array}{r}\mathrm{BIC}(\alpha )=-2{\ell }_p(\hat{\mathit{\theta }},\alpha )+p(\alpha )\log n,\end{array}$$

   where ${\ell }_p(\hat{\mathit{\theta }},\alpha )$ denotes the penalized log-likelihood function available at $\hat{\mathit{\theta }}$ for a fixed α, defined in (9) and n is the sample size. Note that maximizing the penalized log-likelihood function is equivalent to minimizing the BIC. This procedure requires a one-dimensional search, which can be easily accomplished by using, for example, the ‘optim’ routine in R [29] to estimate α, with α between 0.001 and ${10}^3$. In additive linear models, degrees of freedom are defined as approximately the number of effective parameters involved in modeling the nonparametric effects [17,19]. In our case, using the expression of $\hat{\mathbf{f}}$ in (16), we derive effective degrees of freedom as

   $$\begin{array}{r}df(\alpha )=\mathrm{tr}\left\{\mathbf{N}{({\mathbf{N}}^{\mathrm{\top }}\hat{\mathbf{H}}\mathbf{N}+\frac{\alpha {\hat{\sigma }}^2}{1+{\hat{\lambda }}^2}\mathbf{K})}^{-1}{\mathbf{N}}^{\mathrm{\top }}\hat{\mathbf{H}}\right\},\end{array}$$

   where $\mathbf{H}$ is defined in (13). Therefore, one has a total of $p(\alpha )=p+p^{\ast }+2+df(\alpha )$ parameters to be estimated.

   Notes on implementation

   The iterations of the above algorithm are repeated until a suitable convergence rule is satisfied, e.g. $\Vert {\mathit{\theta }}^{(k+1)}-{\mathit{\theta }}^{(k)}\Vert$ is sufficiently small, say ${10}^{-6}$. A set of reasonable starting values may be achieved by computing ${\hat{\mathit{\beta }}}^{(0)}$ and ${\widehat{{\sigma }^2}}^{(0)}$ as the solution of the least-squares regression model of $\mathbf{y}$ on $\mathbf{X}$. So, ${\hat{\mathbf{f}}}^{(0)}=({\mathbf{N}}^{\mathrm{\top }}\mathbf{N}+\alpha {\widehat{{\sigma }^2}}^{(0)}\mathbf{K}{)}^{-1}{\mathbf{N}}^{\mathrm{\top }}(\mathbf{y}-\mathbf{X}{\hat{\mathit{\beta }}}^{(0)})$, ${\hat{\lambda }}^{(0)}$ can be the sample skewness coefficient of $\mathbf{y}-\mathbf{X}{\hat{\mathit{\beta }}}^{(0)}-\mathbf{N}{\hat{\mathbf{f}}}^{(0)}$. The value of ${\hat{\mathit{\rho }}}^{(0)}$ can be the value ${\mathit{\rho }}_0$ such that $m_i({\mathit{\rho }}_0,{\mathbf{z}}_i)=1$ for all $i=1,\dots ,n$ (homoscedasticity of variance).

4.2. Goodness-of-fit

The Mahalanobis distance $d_i^2=(Y_i-{\mathbf{x}}_i^{\mathrm{\top }}\mathit{\beta }-{\mathbf{n}}_i^{\mathrm{\top }}\mathbf{f}{)}^2/{\sigma }_i^2,i=1,\dots ,n,$ is extremely useful in testing the goodness-of-fit and in detecting outliers. According to [33], it can be shown that the distribution of $d_i^2$ is the same as under normal distribution. So, in the SN distribution, $d_i^2=(Y_i-{\mathbf{x}}_i^{\mathrm{\top }}\mathit{\beta }-{\mathbf{n}}_i^{\mathrm{\top }}\mathbf{f}{)}^2/{\sigma }_i^2\sim {\chi }_1^2$. This result is interesting because it allows evaluating the statistical models in practice. By substituting the maximum likelihood estimates of $\mathit{\beta },\mathbf{f}$ and ${\sigma }^2$ at the distance of Mahalanobis $d_i^2$, we can evaluate the fit of the models by constructing quantile-quantile plots with simulated confidence bands of $100\gamma \mathrm{\%},$ $0<\gamma <1$ [1]. In addition, by plotting the Mahalanobis distance and considering as a benchmark the quantile ν of the quadratic form $d_i^2$, we can identify outliers. For instance, for the skew-normal case, we have that $\nu ={\chi }^2(\epsilon ),$ where $0<\epsilon <1$.

5. Residuals

The residual analysis aims at identifying atypical observations and/or model misspecification once residuals are measures of agreement between the data and the fitted model. Under the heteroscedastic PLM-SN model, we defined the following standardized residual

$$\begin{array}{r}{e}_i^1=\frac{y_i-{\mathbf{x}}_i^{\mathrm{\top }}\hat{\mathit{\beta }}-{\mathbf{n}}_i^{\mathrm{\top }}\hat{\mathbf{f}}}{\sqrt{\widehat{{\sigma }^2}{m}_i(\hat{\mathit{\rho }},{\mathbf{z}}_i)}},i=1,\dots ,n,\end{array}$$ (17)

where $\hat{\mathit{\beta }},\hat{\mathbf{f}},\widehat{{\sigma }^2}$ and $\hat{\mathit{\rho }}$ denote the MPLE of $\mathit{\beta },\mathbf{f}$, ${\sigma }^2$ and ρ, respectively, from the ECM-algorithm described in Section 4.1. Note that when $\rho =0,$ under the homoscedastic PLM-SN model, we get the following standardized residual

$$\begin{array}{r}{e}_i^0=\frac{y_i-{\mathbf{x}}_i^{\mathrm{\top }}\hat{\mathit{\beta }}-{\mathbf{n}}_i^{\mathrm{\top }}\hat{\mathbf{f}}}{\sqrt{\widehat{{\sigma }^2}}},\end{array}$$ (18)

where $\hat{\mathit{\beta }},\hat{\mathbf{f}}$ and $\widehat{{\sigma }^2}$ denote the MPLE of $\mathit{\beta },\mathbf{f}$ and ${\sigma }^2$, respectively, from the EM-algorithm described in Section 4 of Ferreira and Paula [12].

Based on the residuals $e_i^1$ and $e_i^0$, we can detect incorrect specification of the error distribution as well as the presence of outlying observations.

6. Influence diagnostics

Cook [6] proposed a unified approach for the assessment of local influence in minor perturbations of a statistical model, which can be viewed as a generalization of the robustness concept for studying and detecting influential subsets of data. Following [12], a direct application of this approach involves extensive algebraic manipulation for the HPLM-SN model. In this article, we will apply the general approach of Zhu and Lee [37] to achieve diagnostic measures for local influence analysis.

6.1. Description of the local influence approach

The general approach developed by Zhu and Lee [37] for local influence analysis of general statistical models with missing data will be utilized to obtain diagnostic measures for the HPLM-SN model. For completeness and to introduce notation, this approach is briefly outlined here. Consider a perturbation vector $\mathit{\omega }=({\omega }_1,\dots ,{\omega }_g{)}^{\mathrm{\top }}$ varying in an open region $\mathbf{\Omega }\subseteq {\mathbb{R}}^g$, and the following perturbed statistical model $\mathcal{M}=\{f({\mathbf{y}}_c,\mathit{\theta },\mathit{\omega }):\mathit{\omega }\in \mathbf{\Omega }\}$, where $f({\mathbf{y}}_c,\mathit{\theta },\mathit{\omega })$ is the probability density function for the complete-data, ${\mathbf{y}}_c$, perturbed by ω and ${\ell }_{c_p}(\mathit{\theta },\mathit{\omega }|{\mathbf{y}}_c)=\log f({\mathbf{y}}_c,\mathit{\theta },\mathit{\omega }),$ its corresponding complete penalized log-likelihood function. We assume there is a ${\mathit{\omega }}_0$ such that ${\ell }_{c_p}(\mathit{\theta },{\mathit{\omega }}_0|{\mathbf{y}}_c)={\ell }_{c_p}(\mathit{\theta }|{\mathbf{y}}_c)$ for all $\mathit{\theta }$. Let $\hat{\mathit{\theta }}(\mathit{\omega })$ be the maximum of the function $Q_p(\mathit{\theta },\mathit{\omega }|\hat{\mathit{\theta }})=\mathrm{E}[{\ell }_{c_p}(\mathit{\theta },\mathit{\omega }|{\mathbf{y}}_c)|\mathbf{y},\hat{\mathit{\theta }}]$. Then, the influence graph is defined as $\mathit{\alpha }(\mathit{\omega })=({\mathit{\omega }}^{\mathrm{\top }},f_Q(\mathit{\omega }){)}^{\mathrm{\top }}$, where $f_Q(\mathit{\omega })$ is the Q-displacement function defined as follows: $f_Q(\mathit{\omega })=2[Q_p(\hat{\mathit{\theta }}|\hat{\mathit{\theta }})-Q_p(\hat{\mathit{\theta }}(\mathit{\omega })|\hat{\mathit{\theta }})].$ Following the approach developed by [6,37], the normal curvature $C_{f_Q,\mathbf{v}}$ of $\mathit{\alpha }(\mathit{\omega })$ at ${\mathit{\omega }}_0$ in the direction of some unit vector $\mathbf{v}$ is used to summarize the local behavior of the Q-displacement function. It can be shown that $C_{f_Q,\mathbf{v}}=-2{\mathbf{v}}^{\mathrm{\top }}{\ddot{Q}}_{{\mathit{\omega }}_0}\mathbf{v}$ and $-{\ddot{Q}}_{{\mathit{\omega }}_0}={\mathbf{\Delta }}_{{\mathit{\omega }}_0}^{\mathrm{\top }}\{-\frac{{\mathrm{\partial }}^2{Q}_p(\mathit{\theta }|\hat{\mathit{\theta }})}{\mathrm{\partial }\mathit{\theta }\mathrm{\partial }{\mathit{\theta }}^{\mathrm{\top }}}{|}_{\mathit{\theta }=\hat{\mathit{\theta }}}{\}}^{-1}{\mathbf{\Delta }}_{{\mathit{\omega }}_0},$ where ${\mathbf{\Delta }}_{{\mathit{\omega }}_0}={\displaystyle \frac{{\mathrm{\partial }}^2{Q}_p(\mathit{\theta },\mathit{\omega }|\hat{\mathit{\theta }})}{\mathrm{\partial }\mathit{\theta }\mathrm{\partial }{\mathit{\omega }}^{\mathrm{\top }}}{|}_{\mathit{\theta }=\hat{\mathit{\theta }},\mathit{\omega }={\mathit{\omega }}_0}.}$ As in [6], the symmetric matrix $-2{\ddot{Q}}_{{\mathit{\omega }}_0}$ is fundamental for detecting influential observations, and its spectral decomposition is given by $-2{\ddot{Q}}_{{\mathit{\omega }}_0}=\sum _{k=1}^g{\zeta }_k{\mathit{\epsilon }}_k{\mathit{\epsilon }}_k^{\mathrm{\top }},$ where $\{({\zeta }_k,{\mathit{\epsilon }}_k),k=1,\dots ,g\}$ are eigenvalue–eigenvector pairs of $-2{\ddot{Q}}_{{\mathit{\omega }}_0}$ with ${\zeta }_1\ge \cdots \ge {\zeta }_r>{\zeta }_{r+1}=\cdots =0$ and orthonormal eigenvectors $\{{\mathit{\epsilon }}_k,k=1,\dots ,g\}$. Lesaffre and Verbeke [25] and Zhu and Lee [37] proposed inspecting all eigenvectors corresponding to non-zero eigenvalues for more revealing information. Based on Zhu and Lee [37], we consider the following aggregated contribution vector of all eigenvectors corresponding to non-zero eigenvalues. Let ${\tilde{\zeta }}_k={\zeta }_k/({\zeta }_1+\dots +{\zeta }_r),$ ${\mathit{\epsilon }}_k^2=({\mathit{\epsilon }}_{k1}^2,\dots ,{\mathit{\epsilon }}_{kg}^2{)}^{\mathrm{\top }}$ and $M(0)=\sum _{k=1}^r{\tilde{\zeta }}_k{\mathit{\epsilon }}_k^2.$ The jth component of $M(0)$, $M(0{)}_j$, is equal to $\sum _{k=1}^r{\tilde{\zeta }}_k{\mathit{\epsilon }}_{kj}^2$. The evaluation of influential cases is based on the visual inspection of the $\{M(0{)}_j,j=1,\dots ,g\}$ plotted against the index j. The jth case may be regarded as influential if $M(0{)}_j$ is larger than the reference.

The inconvenience involved in the use of the normal curvature consists of deciding the influence of the observations, since $C_{f_Q,\mathbf{v}}(\mathit{\theta })$ may assume any value and is not invariant under a uniform change of scale. Zhu and Lee [37] considered the following conformal normal curvature $B_{f_Q,\mathbf{v}}(\mathit{\theta })=C_{f_Q,\mathbf{v}}(\mathit{\theta })/\mathrm{tr}[-2{\ddot{Q}}_{{\mathit{\omega }}_0}]$, which has an interesting property $0\le B_{f_Q,\mathbf{v}}(\mathit{\theta })\le 1$, for any unitary direction $\mathbf{v}$. Now, let ${\mathbf{v}}_j$ be a basic perturbation vector with the jth entry 1 and zero elsewhere. Zhu and Lee [37] showed that for all j, $M(0{)}_j=B_{f_Q,{\mathbf{v}}_j}$. Hence, $M(0{)}_j$ can be obtained by $B_{f_Q,{\mathbf{v}}_j}.$ The computation of $B_{f_Q,{\mathbf{v}}_j}$ is very simple. We refer the reader to Zhu and Lee [37] for other theoretical properties of $B_{f_Q,{\mathbf{v}}_j}$, such as invariance under reparameterizations of $\mathit{\theta }$. Lee and Xu [24] propose to use $1/m+c^{\ast }SM(0)$ as a benchmark for establishing the jth case as influential, where $c^{\ast }$ is a selected constant that may be chosen suitably and $SM(0)$ is the standard deviation of $\{M(0{)}_j,j=1,\dots ,g\}$. In this paper, we consider $c^{\ast }=3$ unless otherwise indicated.

In the following sections, we derive the Hessian matrix for the proposed HPLM-SN model, including a brief discussion on the perturbation schemes employed for our development.

6.2. The Hessian matrix ${\ddot{Q}}_{\mathit{\theta }}(\hat{\mathit{\theta }})$

To obtain the diagnostic measures for the local influence of a particular perturbation scheme, it is necessary to compute ${\ddot{Q}}_{\mathit{\theta }}(\hat{\mathit{\theta }})=\frac{{\mathrm{\partial }}^2{Q}_p(\mathit{\theta }|\hat{\mathit{\theta }})}{\mathrm{\partial }\mathit{\theta }{\mathit{\theta }}^{\mathrm{\top }}}{|}_{\mathit{\theta }=\hat{\mathit{\theta }}},$ where $\mathit{\theta }=({\mathit{\beta }}^{\mathrm{\top }},{\sigma }^2,\lambda ,{\mathit{\rho }}^{\mathrm{\top }},{\mathbf{f}}^{\mathrm{\top }}{)}^{\mathrm{\top }}$. Hence, the Hessian matrix has elements given by

$$\begin{array}{rl}\frac{{\mathrm{\partial }}^2{Q}_p(\mathit{\theta }|\hat{\mathit{\theta }})}{\mathrm{\partial }\mathit{\beta }\mathrm{\partial }{\mathit{\beta }}^{\mathrm{\top }}}& =-\frac{(1+{\lambda }^2)}{{\sigma }^2}{\mathbf{X}}^{\mathrm{\top }}\mathbf{H}\mathbf{X},\\ \frac{{\mathrm{\partial }}^2{Q}_p(\mathit{\theta }|\hat{\mathit{\theta }})}{\mathrm{\partial }\mathit{\beta }\mathrm{\partial }{\sigma }^2}& =-\frac{1}{{\sigma }^4}{\mathbf{X}}^{\mathrm{\top }}\mathbf{H}[(1+{\lambda }^2)(\mathbf{y}-\mathit{\mu })-\lambda \hat{\mathbf{w}}],\\ \frac{{\mathrm{\partial }}^2{Q}_p(\mathit{\theta }|\hat{\mathit{\theta }})}{\mathrm{\partial }\mathit{\beta }\mathrm{\partial }\lambda }& =\frac{1}{{\sigma }^2}{\mathbf{X}}^{\mathrm{\top }}\mathbf{H}[2\lambda (\mathbf{y}-\mathit{\mu })-\hat{\mathbf{w}}],\\ \frac{{\mathrm{\partial }}^2{Q}_p(\mathit{\theta }|\hat{\mathit{\theta }})}{\mathrm{\partial }\mathit{\rho }\mathrm{\partial }{\mathit{\beta }}^{\mathrm{\top }}}& ={\dot{\mathbf{H}}}^{\mathrm{\top }}[\frac{(1+{\lambda }^2)}{{\sigma }^2}\mathbf{D}(\mathbf{y}-\mathit{\mu })-\frac{\lambda }{{\sigma }^2}\mathbf{D}(\hat{\mathbf{w}})]\mathbf{X},\\ \frac{{\mathrm{\partial }}^2{Q}_p(\mathit{\theta }|\hat{\mathit{\theta }})}{\mathrm{\partial }\mathbf{f}\mathrm{\partial }{\mathit{\beta }}^{\mathrm{\top }}}& =-\frac{1+{\lambda }^2}{{\sigma }^2}{\mathbf{N}}^{\mathrm{\top }}\mathbf{H}\mathbf{X},\\ \frac{{\mathrm{\partial }}^2{Q}_p(\mathit{\theta }|\hat{\mathit{\theta }})}{\mathrm{\partial }\mathit{\rho }\mathrm{\partial }{\sigma }^2}& ={\dot{\mathbf{H}}}^{\mathrm{\top }}[\frac{1}{2{\sigma }^4}\mathbf{D}(\widehat{{\mathbf{w}}^2}){\mathbf{1}}_n-\frac{\lambda }{{\sigma }^4}\mathbf{D}(\hat{\mathbf{w}})(\mathbf{y}-\mathit{\mu })],\\ \frac{{\mathrm{\partial }}^2{Q}_p(\mathit{\theta }|\hat{\mathit{\theta }})}{\mathrm{\partial }{\sigma }^2\mathrm{\partial }{\sigma }^2}& =\frac{n}{{\sigma }^4}-\frac{1}{{\sigma }^6}[{\widehat{{\mathbf{w}}^2}}^{\mathrm{\top }}\mathbf{H}{\mathbf{1}}_n-2\lambda (\mathbf{y}-\mathit{\mu }{)}^{\mathrm{\top }}\mathbf{H}\hat{\mathbf{w}}+(1+{\lambda }^2)S(\mathit{\beta },\mathbf{f})],\\ \frac{{\mathrm{\partial }}^2{Q}_p(\mathit{\theta }|\hat{\mathit{\theta }})}{\mathrm{\partial }{\sigma }^2\mathrm{\partial }\lambda }& =\frac{1}{{\sigma }^4}[\lambda S(\mathit{\beta },\mathbf{f})-(\mathbf{y}-\mathit{\mu }{)}^{\mathrm{\top }}\mathbf{H}\hat{\mathbf{w}}],\\ \frac{{\mathrm{\partial }}^2{Q}_p(\mathit{\theta }|\hat{\mathit{\theta }})}{\mathrm{\partial }\mathbf{f}\mathrm{\partial }{\sigma }^2}& =-\frac{1}{{\sigma }^4}{\mathbf{N}}^{\mathrm{\top }}\mathbf{H}[(1+{\lambda }^2)(\mathbf{y}-\mathit{\mu })-\lambda \hat{\mathbf{w}}],\\ \frac{{\mathrm{\partial }}^2{Q}_p(\mathit{\theta }|\hat{\mathit{\theta }})}{\mathrm{\partial }{\lambda }^2}& =-\frac{1}{{\sigma }^2}S(\mathit{\beta },\mathbf{f}),\\ \frac{{\mathrm{\partial }}^2{Q}_p(\mathit{\theta }|\hat{\mathit{\theta }})}{\mathrm{\partial }\mathbf{f}\mathrm{\partial }\lambda }& =\frac{1}{{\sigma }^2}{\mathbf{N}}^{\mathrm{\top }}\mathbf{H}[2\lambda (\mathbf{y}-\mathit{\mu })-\hat{\mathbf{w}}],\\ \frac{{\mathrm{\partial }}^2{Q}_p(\mathit{\theta }|\hat{\mathit{\theta }})}{\mathrm{\partial }\mathit{\rho }\mathrm{\partial }\lambda }& =-\frac{1}{{\sigma }^2}{\dot{\mathbf{H}}}^{\mathrm{\top }}\mathbf{D}(\mathbf{y}-\mathit{\mu })[\lambda (\mathbf{y}-\mathit{\mu })-\hat{\mathbf{w}}],\\ \frac{{\mathrm{\partial }}^2{Q}_p(\mathit{\theta }|\hat{\mathit{\theta }})}{\mathrm{\partial }\mathit{\rho }\mathrm{\partial }{\mathit{\rho }}^{\mathrm{\top }}}& =\sum _{i=1}^n{\dot{\mathbf{M}}}_i-\frac{1}{2{\sigma }^2}\sum _{i=1}^n{\ddot{\mathbf{H}}}_i[(1+{\lambda }^2)(y_i-{\mu }^2)-2\lambda {\hat{w}}_i(y_i-{\mu }_i)+{\widehat{w^2}}_i],\\ \frac{{\mathrm{\partial }}^2{Q}_p(\mathit{\theta }|\hat{\mathit{\theta }})}{\mathrm{\partial }\mathit{\rho }\mathrm{\partial }{\mathbf{f}}^{\mathrm{\top }}}& =\frac{1}{{\sigma }^2}{\dot{\mathbf{H}}}^{\mathrm{\top }}\mathbf{D}((1+{\lambda }^2)(\mathbf{y}-\mathit{\mu })-\lambda \hat{\mathbf{w}})\mathbf{N},\\ \frac{{\mathrm{\partial }}^2{Q}_p(\mathit{\theta }|\hat{\mathit{\theta }})}{\mathrm{\partial }\mathbf{f}\mathrm{\partial }{\mathbf{f}}^{\mathrm{\top }}}& =-\frac{(1+{\lambda }^2)}{{\sigma }^2}{\mathbf{N}}^{\mathrm{\top }}\mathbf{H}\mathbf{N}-\alpha \mathbf{K},\end{array}$$

where $\mathbf{D}(\mathbf{x})$ is the diagonal matrix of the vector $\mathbf{x},$ ${\dot{\mathbf{H}}}^{\mathrm{\top }}=({\dot{\mathbf{H}}}_1^{\mathrm{\top }},\dots ,{\dot{\mathbf{H}}}_n^{\mathrm{\top }}),$ with ${\dot{\mathbf{H}}}_i=-\frac{1}{m_i^2}\frac{\mathrm{\partial }{m}_i}{\mathrm{\partial }{\mathit{\rho }}^{\mathrm{\top }}},$ ${\dot{\mathbf{M}}}_i=[-\frac{1}{m_i}\frac{{\mathrm{\partial }}^2{m}_i}{\mathrm{\partial }\mathit{\rho }\mathrm{\partial }{\mathit{\rho }}^{\mathrm{\top }}}+\frac{1}{m_i^2}\frac{\mathrm{\partial }{m}_i}{\mathrm{\partial }\mathit{\rho }}\frac{\mathrm{\partial }{m}_i}{\mathrm{\partial }{\mathit{\rho }}^{\mathrm{\top }}}]$ and ${\ddot{\mathbf{H}}}_i=\frac{-1}{m_i^2}\frac{{\mathrm{\partial }}^2{m}_i}{\mathrm{\partial }\mathit{\rho }\mathrm{\partial }{\mathit{\rho }}^{\mathrm{\top }}}+\frac{2}{m_i^3}\frac{\mathrm{\partial }{m}_i}{\mathrm{\partial }\mathit{\rho }}\frac{\mathrm{\partial }{m}_i}{\mathrm{\partial }{\mathit{\rho }}^{\mathrm{\top }}}$.

6.3. Perturbation schemes

In this section, we will consider four different perturbation schemes for the HPLM-SN model. For each perturbation scheme, one has the partitioned form ${\mathbf{\Delta }}_{{\mathit{\omega }}_0}=({\mathbf{\Delta }}_{\mathit{\beta }}^{\mathrm{\top }},{\mathbf{\Delta }}_{{\sigma }^2}^{\mathrm{\top }},{\mathbf{\Delta }}_{\lambda }^{\mathrm{\top }},{\mathbf{\Delta }}_{\mathit{\rho }}^{\mathrm{\top }},{\mathbf{\Delta }}_{\mathbf{f}}^{\mathrm{\top }}{)}^{\mathrm{\top }},$ where ${\mathbf{\Delta }}_{\mathit{\beta }}=\frac{{\mathrm{\partial }}^2{Q}_p(\mathit{\theta },\mathit{\omega }|\hat{\mathit{\theta }})}{\mathrm{\partial }\mathit{\beta }\mathrm{\partial }{\mathit{\omega }}^{\mathrm{\top }}}{|}_{\mathit{\theta }=\hat{\mathit{\theta }},\mathit{\omega }={\mathit{\omega }}_0},$ ${\mathbf{\Delta }}_{{\sigma }^2}=\frac{{\mathrm{\partial }}^2{Q}_p(\mathit{\theta },\mathit{\omega }|\hat{\mathit{\theta }})}{\mathrm{\partial }{\sigma }^2\mathrm{\partial }{\mathit{\omega }}^{\mathrm{\top }}}{|}_{\mathit{\theta }=\hat{\mathit{\theta }},\mathit{\omega }={\mathit{\omega }}_0},$ ${\mathbf{\Delta }}_{\lambda }=\frac{{\mathrm{\partial }}^2{Q}_p(\mathit{\theta },\mathit{\omega }|\hat{\mathit{\theta }})}{\mathrm{\partial }\lambda \mathrm{\partial }{\mathit{\omega }}^{\mathrm{\top }}}{|}_{\mathit{\theta }=\hat{\mathit{\theta }},\mathit{\omega }={\mathit{\omega }}_0},$ ${\mathbf{\Delta }}_{\mathit{\rho }}=\frac{{\mathrm{\partial }}^2{Q}_p(\mathit{\theta },\mathit{\omega }|\hat{\mathit{\theta }})}{\mathrm{\partial }\mathit{\rho }\mathrm{\partial }{\mathit{\omega }}^{\mathrm{\top }}}{|}_{\mathit{\theta }=\hat{\mathit{\theta }},\mathit{\omega }={\mathit{\omega }}_0}$ and ${\mathbf{\Delta }}_{\mathbf{f}}=\frac{{\mathrm{\partial }}^2{Q}_p(\mathit{\theta },\mathit{\omega }|\hat{\mathit{\theta }})}{\mathrm{\partial }\mathbf{f}\mathrm{\partial }{\mathit{\omega }}^{\mathrm{\top }}}{|}_{\mathit{\theta }=\hat{\mathit{\theta }},\mathit{\omega }={\mathit{\omega }}_0}.$

6.3.1. Case-weight perturbation

First, consider the following arbitrary allocation of weights for the expected value of the complete-data penalized log-likelihood function (perturbed Q-function), which may capture departures in general directions, given by

$$\begin{array}{r}{Q}_p(\mathit{\theta },\mathit{\omega }|\hat{\mathit{\theta }})=\sum _{i=1}^n{\omega }_i{Q}_i(\mathit{\theta }|\hat{\mathit{\theta }})-\frac{\alpha }{2}{\mathbf{f}}^{\mathrm{\top }}\mathbf{K}\mathbf{f},\end{array}$$

where the contribution from the ith experimental unity to the Q-function is $Q_i(\mathit{\theta }|\hat{\mathit{\theta }})\propto -\log ({\sigma }^2)-\log (m_i)-\frac{1}{2m_i{\sigma }^2}[(1+{\lambda }^2)(y_i-{\mu }^2)-2\lambda {\hat{w}}_i(y_i-{\mu }_i)+{\widehat{w^2}}_i],$ with $\mathit{\omega }=({\omega }_1,\dots ,{\omega }_n{)}^{\mathrm{\top }}$ an $n\times 1$ vector, $0\le w_i\le 1,$ for $i=1,\dots ,n$ and ${\mathit{\omega }}_0=(1,\dots ,1{)}^{\mathrm{\top }}$. For this perturbation scheme, we find ${\mathbf{\Delta }}_{{\mathit{\omega }}_0}$ with the following elements:

$$\begin{array}{rl}{\mathbf{\Delta }}_{\mathit{\beta }}& =\frac{1}{{\sigma }^2}{\mathbf{X}}^{\mathrm{\top }}\mathbf{H}\mathbf{D}(-\lambda \hat{\mathbf{w}}+(1+{\lambda }^2)(\mathbf{y}-\mathit{\mu })),\\ {\mathbf{\Delta }}_{{\sigma }^2}& =-\frac{1}{{\sigma }^2}{\mathbf{1}}_n^{\mathrm{\top }}+\frac{1}{2{\sigma }^4}\{(\mathbf{y}-\mathit{\mu }{)}^{\mathrm{\top }}\mathbf{H}[(1+{\lambda }^2)\mathbf{D}(\mathbf{y}-\mathit{\mu })-2\lambda \mathbf{D}(\hat{\mathbf{w}})]+{\widehat{{\mathbf{w}}^2}}^{\mathrm{\top }}\mathbf{H}\},\\ {\mathbf{\Delta }}_{\lambda }& =\frac{1}{{\sigma }^2}(\mathbf{y}-\mathit{\mu }{)}^{\mathrm{\top }}\mathbf{H}\mathbf{D}(\hat{\mathbf{w}}-\lambda (\mathbf{y}-\mathit{\mu })),\\ {\mathbf{\Delta }}_{\mathit{\rho }}& ={\mathbf{M}}^{\mathrm{\top }}-\frac{1}{2{\sigma }^2}{\dot{\mathbf{H}}}^{\mathrm{\top }}\{\mathbf{D}(\mathbf{y}-\mathit{\mu })[(1+{\lambda }^2)\mathbf{D}(\mathbf{y}-\mathit{\mu })-2\lambda \mathbf{D}(\hat{\mathbf{w}})]+\mathbf{D}(\widehat{{\mathbf{w}}^2})\},\\ {\mathbf{\Delta }}_{\mathbf{f}}& =\frac{1}{{\sigma }^2}{\mathbf{N}}^{\mathrm{\top }}\mathbf{H}\mathbf{D}(-\lambda \hat{\mathbf{w}}+(1+{\lambda }^2)(\mathbf{y}-\mathit{\mu })),\end{array}$$

where ${\mathbf{M}}^{\mathrm{\top }}=({\mathbf{M}}_1^{\mathrm{\top }},\dots ,{\mathbf{M}}_n^{\mathrm{\top }}),$ with ${\mathbf{M}}_i=-\frac{1}{m_i}\frac{\mathrm{\partial }{m}_i}{\mathrm{\partial }{\mathit{\rho }}^{\mathrm{\top }}}$.

6.3.2. Response variable perturbation

A perturbation of the response variables $\mathbf{y}=(y_1,\dots ,y_n{)}^{\mathrm{\top }}$ is defined as ${\mathbf{y}}_{\mathit{\omega }}=\mathbf{y}+S_y\mathit{\omega },$ where $S_y$ is the standard deviation of $\mathbf{y}$. In this case, ${\mathit{\omega }}_0=\mathbf{0}\in {\mathbb{R}}^n$ and the perturbed Q-function is as in Equations (12)–(13), switching ${\mathbf{y}}_{\mathit{\omega }}$ with $\mathbf{y}$. It follows that the matrix ${\mathbf{\Delta }}_{{\mathit{\omega }}_0}$ has the following elements:

$$\begin{array}{rl}{\mathbf{\Delta }}_{\mathit{\beta }}& =\frac{S_y}{{\sigma }^2}[(1+{\lambda }^2){\mathbf{X}}^{\mathrm{\top }}\mathbf{H}],{\mathbf{\Delta }}_{{\sigma }^2}=\frac{S_y}{{\sigma }^4}[(1+{\lambda }^2)(\mathbf{y}-\mathit{\mu }{)}^{\mathrm{\top }}-\lambda {\hat{\mathbf{w}}}^{\mathrm{\top }}]\mathbf{H},\\ {\mathbf{\Delta }}_{\lambda }& =\frac{S_y}{{\sigma }^2}[{\hat{\mathbf{w}}}^{\mathrm{\top }}-2\lambda (\mathbf{y}-\mathit{\mu }{)}^{\mathrm{\top }}]\mathbf{H},\\ {\mathbf{\Delta }}_{\mathit{\rho }}& =\frac{S_y}{{\sigma }^2}{\dot{\mathbf{H}}}^{\mathrm{\top }}[-(1+{\lambda }^2)\mathbf{D}(\mathbf{y}-\mathit{\mu })+\lambda \mathbf{D}(\hat{\mathbf{w}})],\\ {\mathbf{\Delta }}_{\mathbf{f}}& =\frac{S_y}{{\sigma }^2}[(1+{\lambda }^2){\mathbf{N}}^{\mathrm{\top }}\mathbf{H}].\end{array}$$

6.3.3. Explanatory variable perturbation

In this section, we will consider the influence that perturbation in the specific continuous explanatory variable may produce on the parameter estimates. A perturbation of the explanatory variable $x_r$ is defined as $x_{r\mathit{\omega }}=x_w+S_r\mathit{\omega },r\in 1,\dots ,p,$ where $S_r$ is the standard deviation of the explanatory variable $x_r$. In this case, ${\mathit{\omega }}_0=\mathbf{0}\in {\mathbb{R}}^n$ and the perturbed Q-function is like Equations (12)–(13), switching ${\mathbf{X}}_{\mathit{\omega }}$ with $\mathbf{X}$. Consequently, the matrix ${\mathbf{\Delta }}_{{\mathit{\omega }}_0}$ has the following elements:

$$\begin{array}{rl}{\mathbf{\Delta }}_{\mathit{\beta }}& =\frac{S_r}{{\sigma }^2}\{(1+{\lambda }^2)[{\mathbf{I}}_{r_0}(\mathbf{y}-\mathit{\mu }{)}^{\mathrm{\top }}-{\beta }_r{\mathbf{X}}^{\mathrm{\top }}]-\lambda {\mathbf{I}}_{r_0}{\hat{\mathbf{w}}}^{\mathrm{\top }}\}\mathbf{H},\\ {\mathbf{\Delta }}_{{\sigma }^2}& =\frac{{\beta }_r{S}_r}{{\sigma }^4}[\lambda {\hat{\mathbf{w}}}^{\mathrm{\top }}-(1+{\lambda }^2)(\mathbf{y}-\mathit{\mu }{)}^{\mathrm{\top }}]\mathbf{H},\\ {\mathbf{\Delta }}_{\lambda }& =\frac{{\beta }_r{S}_r}{{\sigma }^2}[-{\hat{\mathbf{w}}}^{\mathrm{\top }}+2\lambda (\mathbf{y}-\mathit{\mu }{)}^{\mathrm{\top }}]\mathbf{H},\\ {\mathbf{\Delta }}_{\mathit{\rho }}& =\frac{{\beta }_r{S}_r}{{\sigma }^2}{\dot{\mathbf{H}}}^{\mathrm{\top }}[(1+{\lambda }^2)\mathbf{D}(\mathbf{y}-\mathit{\mu })-\lambda \mathbf{D}(\hat{\mathbf{w}})],\\ {\mathbf{\Delta }}_{\mathbf{f}}& =\frac{-{\beta }_r{S}_r}{{\sigma }^2}(1+{\lambda }^2){\mathbf{N}}^{\mathrm{\top }}\mathbf{H},\end{array}$$

where ${\mathbf{I}}_{r_0}$ denotes a $p\times 1$ vector of zeros with one in the rth position.

6.3.4. Perturbation of the skewness parameter

Consider the perturbed model $Y_i={\mathbf{x}}_i^{\mathrm{\top }}\mathit{\beta }+f(t_i)+{ϵ}_i,i=1,\dots ,n,$ with ${ϵ}_i\sim \mathrm{SN}(0,{\sigma }_i^2,{\lambda }_i),{\sigma }_i^2={\sigma }^2{m}_i(\mathit{\rho },{\mathbf{z}}_i)$ and ${\lambda }_i=\lambda s_i(\mathit{\omega },{\mathbf{z}}_i),$ where $s_i=s(\mathit{\omega },{\mathbf{z}}_i)$ is a known positive continuously differentiable function, ${\mathbf{z}}_i$ contains values of the explanatory variables, which constitute in general, although not necessary, a subset of ${\mathbf{x}}_i$, and $\mathit{\omega }:l\times 1$ is a perturbation vector. It is assumed that there is a ${\mathit{\omega }}_0$ such that $m({\mathit{\omega }}_0,{\mathbf{z}}_i)=1$, for all $i=1,\dots ,n$. The perturbed Q-function is similar to Equations (12)–(13), switching ${\lambda }_i$ with λ. It follows that the matrix ${\mathbf{\Delta }}_{{\mathit{\omega }}_0}$ has the following elements:

$$\begin{array}{rl}{\mathbf{\Delta }}_{\mathit{\beta }}& =\frac{\lambda }{{\sigma }^2}{\mathbf{X}}^{\mathrm{\top }}\mathbf{H}[2\lambda \mathbf{D}(\mathbf{y}-\mathit{\mu })-\mathbf{D}(\hat{\mathbf{w}})],\\ {\mathbf{\Delta }}_{{\sigma }^2}& =\frac{\lambda }{{\sigma }^4}(\mathbf{y}-\mathit{\mu }{)}^{\mathrm{\top }}\mathbf{H}[\lambda \mathbf{D}(\mathbf{y}-\mathit{\mu })-\mathbf{D}(\hat{\mathbf{w}})],\\ {\mathbf{\Delta }}_{\lambda }& =\frac{1}{{\sigma }^2}(\mathbf{y}-\mathit{\mu }{)}^{\mathrm{\top }}\mathbf{H}[\mathbf{D}(\hat{\mathbf{w}})-2\lambda \mathbf{D}(\mathbf{y}-\mathit{\mu })],\\ {\mathbf{\Delta }}_{\mathit{\rho }}& =-\frac{\lambda }{{\sigma }^2}{\dot{\mathbf{H}}}^{\mathrm{\top }}\mathbf{D}(\mathbf{y}-\mathit{\mu })[\mathbf{D}(\mathbf{y}-\mathit{\mu })-\mathbf{D}(\hat{\mathbf{w}})],\\ {\mathbf{\Delta }}_{\mathbf{f}}& =\frac{\lambda }{{\sigma }^2}{\mathbf{N}}^{\mathrm{\top }}\mathbf{H}[2\lambda \mathbf{D}(\mathbf{y}-\mathit{\mu })-\mathbf{D}(\hat{\mathbf{w}})].\end{array}$$

In the next section, for simplicity, we consider the diagnostics for the scale parameter in the HPLM-SN model. However, the method proposed here can be used to test for homogeneity of any parameter involved in the variance, as discussed by Xie et al. [34] and Zeller et al. [36].

7. Likelihood ratio test for homogeneity in the HPLM-SN model

The HPLM-SN model defined in Equations (5)–(6) supposes that the variance of the model is not constant with the scale parameter given by ${\sigma }_i^2={\sigma }^2{m}_i,$ with $m_i=m(\mathit{\rho },{\mathbf{z}}_i)$. If the variance depends on the quantity of some explanatory variables ${\mathbf{z}}_i$, some specific forms of $m_i$ are usually taken to model the varying dispersion: (i) log-linear model $m_i(\mathit{\rho },{\mathbf{z}}_i)=\exp (\sum _{j=1}^{p^{\ast }}{\rho }_j{z}_{ij})$ and (ii) power product model $m_i(\mathit{\rho },{\mathbf{z}}_i)=\prod _{j=1}^{p^{\ast }}{z}_{ij}^{{\rho }_j}=\exp (\sum _{j=1}^{p^{\ast }}{\rho }_{j}log(z_{ij}))$; see [11,22,34] and references therein for more details. Of course, (ii) requires that the $z_{ij}$ be strictly positive, while no such restriction is needed for (i). Furthermore, it is assumed that there is a unique value $\mathit{\rho }={\mathit{\rho }}_0,$ such that $m_i({\mathit{\rho }}_0,{\mathbf{z}}_i)=1$ for all $i=1,\dots ,n$, then ${\sigma }_i^2={\sigma }^2$ and $Y_i^{\prime }s$ has constant variance. Hence the test for homogeneity of scale parameter is equivalent to testing the following hypothesis

$$\begin{array}{r}{H}_0:\mathit{\rho }={\mathit{\rho }}_0\mathrm{vs}{H}_1:\mathit{\rho }\ne {\mathit{\rho }}_0.\end{array}$$

In this article, we use a Likelihood Ratio (LR) test statistic to check $H_0,$ where $LR=2({\ell }_p(\hat{\theta },\hat{\alpha })-{\ell }_p(\tilde{\theta },\tilde{\alpha })),$ with $(\hat{\theta },\hat{\alpha })$ and $(\tilde{\theta },\tilde{\alpha })$ are the restricted (under $H_0$) and unrestricted ML estimators of $(\mathit{\theta },\alpha )$, respectively. When $H_0$ is true, the statistic LR is asymptotically distributed as ${\chi }_{p\ast }^2.$

8. Simulation studies

In this section, we present four simulation studies. The first study evaluates the performance of the ML estimates of the HPLM-SN model parameters determined from the ECM-algorithm. In the second and third studies, the performance of the asymptotic distribution and the power of the LR test statistic are examined. The fourth simulation study also evaluates the performance of the proposed test by providing evidence regarding his behavior when the underlying structure function is misspecified. In all simulation studies, we used the method of Eilers and Marx [10] to construct matrices N and K, with the number of knots being 14 and 12, for scenarios 1 and 2, described in Section 8.1, respectively.

8.1. Study I: parameter recovery

In this subsection, we consider two scenarios for simulation in order to verify if we can estimate the true parameter values accurately by using the proposed estimation method. This is the first step to ensure that the estimation procedure works satisfactorily. We fit the HPLM-SN model defined in Section 3.2 to data that were artificially generated from model (5)–(6), where $f(t_i)=\cos (t_i)$, $t_i\in (-3\pi ,3\pi )$ (scenario 1) or $f(t_i)=\cos (4\pi t_i)\exp (-t_i^2/2)$, $t_i\in (0.6,1.6)$ (Doppler effect, scenario 2), such that we assume equidistant values for $t_i$ in each specific interval, $x_i\sim U(0.2,2),$ $\mathit{\beta }=({\beta }_0,{\beta }_1{)}^{\mathrm{\top }}=(0,2{)}^{\mathrm{\top }}$, ${\sigma }^2=0.1$, $\lambda =3$ and $\rho =0.1,$ with $m_i={\mathrm{e}}^{\rho x_i}$ or $m_i=x_i^{\rho }$, $i=1,\dots ,n$.

We generated 2000 samples from each scenario, for n = 200, 500 and 1000. The average values (mean) and the corresponding standard deviations (SD) of the estimates made by the ECM algorithm in all samples are presented in Tables 1–4. Moreover, these tables contain approximate standard errors (SE) calculated via the observed information matrix for $({\beta }_1,{\sigma }^2,\lambda ,\rho )$; see Appendix 1 for more details.

Table 2.

Mean and standard deviation (SD) estimates by the ECM algorithm based on 2000 samples from the HPLM-SN in scenario 2 and $m_i={\mathrm{e}}^{\rho x_i}$. SE is the average of estimated standard errors.

		n = 200			n = 500			n = 1000
Parameter	True Value	Mean	SD	SE	Mean	SD	SE	Mean	SD	SE
${\beta }_1$	2.0	1.942	0.948	0.559	1.928	0.554	0.415	1.994	0.382	0.309
${\sigma }^2$	0.6	0.551	0.185	0.096	0.566	0.098	0.069	0.577	0.069	0.054
λ	3.0	3.314	0.953	0.471	3.107	0.470	0.335	3.058	0.307	0.244
ρ	4.6	4.693	0.246	0.126	4.651	0.128	0.091	4.631	0.086	0.069

Table 3.

Mean and standard deviation (SD) estimates by the ECM algorithm based on 2000 samples from the HPLM-SN in scenario 1 and $m_i=x_i^{\rho }$. SE is the average of estimated standard errors.

		n = 200			n = 500			n = 1000
Parameter	True Value	Mean	SD	SE	Mean	SD	SE	Mean	SD	SE
${\beta }_1$	2.0	1.951	0.183	0.285	1.982	0.062	0.059	1.993	0.043	0.042
${\sigma }^2$	1.0	0.962	0.214	0.157	0.984	0.097	0.090	0.991	0.069	0.064
λ	3.0	4.445	2.883	2.558	3.223	0.627	0.558	3.082	0.375	0.358
ρ	1.6	1.822	0.234	0.211	1.672	0.120	0.115	1.623	0.079	0.081

Table 1.

Mean and standard deviation (SD) estimates by the ECM algorithm based on 2000 samples from the HPLM-SN in scenario 1 and $m_i={\mathrm{e}}^{\rho x_i}$. SE is the average of estimated standard errors.

		n = 200			n = 500			n = 1000
Parameter	True Value	Mean	SD	SE	Mean	SD	SE	Mean	SD	SE
${\beta }_1$	2.0	1.816	1.071	0.777	1.882	0.544	0.478	1.963	0.390	0.348
${\sigma }^2$	0.6	0.483	0.180	0.109	0.545	0.093	0.077	0.569	0.067	0.061
λ	3.0	3.572	1.443	0.782	3.156	0.471	0.471	3.075	0.311	0.276
ρ	4.6	4.792	0.260	0.176	4.674	0.125	0.125	4.640	0.086	0.079

Table 4.

Mean and standard deviation (SD) estimates by the ECM algorithm based on 2000 samples from the HPLM-SN in scenario 2 and $m_i=x_i^{\rho }$. SE is the average of estimated standard errors.

		n = 200			n = 500			n = 1000
Parameter	True Value	Mean	SD	SE	Mean	SD	SE	Mean	SD	SE
${\beta }_1$	2.0	1.956	0.190	0.257	1.980	0.065	0.066	1.989	0.042	0.044
${\sigma }^2$	1.0	0.962	0.209	0.166	0.979	0.098	0.096	0.991	0.068	0.068
λ	3.0	4.393	2.871	2.594	3.212	0.626	0.601	3.085	0.376	0.378
ρ	1.6	1.811	0.248	0.231	1.675	0.129	0.131	1.639	0.085	0.086

Note that all the point estimates are quite accurate in all the scenarios considered. Thus, the results suggest that the proposed algorithm produces satisfactory estimates. In addition, the SD of the estimates and the SE are closer to each other and decrease with the sample size, showing that the calculation from the observed information matrix seems to be correct.

Finally, in Figure 2, we plot the 2000 estimated functions $f(\cdot )$ of the nonparametric components from the two considered scenarios and log-linear model as a structure of dispersion. Note that in all scenarios considered, the proposed model presents excellent performance. For structure of the dispersion power product model, the results are the same, so the figure is not shown here to save space. The variability among the estimates of the nonparametric function reduces as the sample size increases, as well as the respective mean estimates, become closer to the true values, for both scenarios and both structures of dispersion. This is an indication of the consistency of the nonparametric estimator.

Figure 2.

Plots of the nonparametric components with 2000 replications. Adjusted curves (gray lines) and true curves (black lines): scenario 1 (first column) and scenario 2 (second column) and $m_i={\mathrm{e}}^{\rho x_i}$. (a and b) for n = 200; (c and d) for n = 500 and (e and f) for n = 1000.

8.2. Study II: the empirical distribution of the LR test statistic

In this subsection, the performance of the asymptotic distribution of the LR test statistic is examined following the procedure described in [13,34]. Therefore, the empirical distribution with the theoretical distribution via Monte Carlo simulations is compared.

The design considered in this simulation study is the same as scenario 1 of Study I, but with $\rho =0$ (under $H_0$), $\mathit{\beta }=({\beta }_0,{\beta }_1{)}^{\mathrm{\top }}=(0,2{)}^{\mathrm{\top }}$, ${\sigma }^2=0.1$, $\lambda =3$ and n = 50, 100, 200, 300, 400, 500, 600 and 700. As suggested by Cook and Weisberg [8], the power function and the exponential function are usually employed in practice. Thus, we assume that $m_i={\mathrm{e}}^{\rho x_i}$ and $m_i=x_i^{\rho }$. Each simulated case was replied 2000 times so that the values of the explanatory variable x were kept fixed throughout the simulations. Under $H_0$, it is expected that the LR test statistic follows a ${\chi }_1^2$ distribution. Then, using the 2000 estimates of the LR statistics we obtained the empirical distribution function (edf). Figure 3 shows comparisons between the edf of the LR statistic and the theoretical distribution of ${\chi }_1^2$ for n = 100, 300 and 700. We can see that when n increases, the edfs are very close to the theoretical distribution for the model considered in our study.

Figure 3.

Simulated comparisons between the empirical distributions of the LR statistic and ${\chi }_1^2$ distribution for n = 100 (a) and (d), n = 300 (b) and (e), n = 700 (c) and (f). In the first row, under the exponential function and in the second line, under power function.

8.3. Study III: the empirical power of the LR test

In our experiment, to gain insight into the performance of the homogeneity LR test in the HPLM-SN model, we performed a simulation study and examined the power functions for various ρ parameter set-ups. For demonstration purposes, we perform this simulation study with the same parameter set options from the previous experiment considered with $\rho =0,0.1,0.2,0.3,0.4$ and 0.5. The sample sizes of $n=50,100,200,300,400,500,600$ and 700 were chosen to evaluate the behavior of the test for small and midsize samples. Each simulated case was replied 2000 times so that the values of the explanatory variable x were kept fixed throughout the simulations.

Tables 5 and 6 provide the empirical type I error probability (under the null hypothesis) and the empirical power of LR test under alternative hypotheses with $\alpha =0.05$ (i.e. the percentage of times that the corresponding statistic exceeds $5\mathrm{\%}$ of the upper points of the reference ${\chi }^2$ distribution). The choice of level of significance is usually somewhat arbitrary. The standard significance level $\alpha =0.05$ was chosen to reflect the usual practice in statistical studies. From Tables 5 and 6, we found that the LR test is usually able to achieve the desired significance level and it is successful in detecting the heteroscedasticity behaviour of the scalar parameter for the model considered in our study. As expected, the performance of the test statistic improves with increasing n. It can be seen that as the size of the sample and ρ increase, the empirical power of the tests increases, approaching 1. As pointed out by Xie et al. [34], the score test statistic is not very sensitive to the functional form in the test for homogeneity of variance parameter. This fact might also be true in our study, in the context of the LR test statistic. Note in Tables 5 and 6 that the results are quite similar.

Table 5.

Empirical type I error probability (when $H_0$) and empirical power of LR under $H_1,$ assuming $m_i={\mathrm{e}}^{\rho x_i}$.

n	$\rho =0.0$	$\rho =0.2$	$\rho =0.4$	$\rho =0.6$	$\rho =0.8$	$\rho =1.0$
50	0.342	0.357	0.387	0.547	0.522	0.716
100	0.141	0.180	0.312	0.570	0.718	0.861
200	0.084	0.194	0.569	0.859	0.971	0.997
300	0.062	0.232	0.637	0.928	0.995	1.000
400	0.069	0.295	0.809	0.983	1.000	1.000
500	0.050	0.357	0.887	0.992	1.000	1.000
600	0.051	0.379	0.929	1.000	1.000	1.000
700	0.057	0.454	0.961	1.000	1.000	1.000

Table 6.

Empirical type I error probability (when $H_0$) and empirical power of LR under $H_1,$ assuming $m_i=x_i^{\rho }$.

n	$\rho =0.0$	$\rho =0.2$	$\rho =0.4$	$\rho =0.6$	$\rho =0.8$	$\rho =1.0$
50	0.378	0.377	0.413	0.561	0.528	0.685
100	0.136	0.184	0.304	0.556	0.684	0.814
200	0.083	0.199	0.544	0.832	0.965	0.992
300	0.058	0.221	0.620	0.921	0.994	1.000
400	0.056	0.336	0.862	0.988	1.000	1.000
500	0.055	0.367	0.909	1.000	1.000	1.000
600	0.055	0.377	0.909	1.000	1.000	1.000
700	0.060	0.427	0.949	1.000	1.000	1.000

8.4. Study IV: misspecification of the structure function

We report a simulation study to analyze the influence of misspecification of the structure function. The design considered in this simulation study is the same as scenario 1 of Study I, with $m_i={\mathrm{e}}^{\rho x_i}$ (case 1) and $m_i=z_i^{\rho }$ (case 2), but varying ${\sigma }^2=\{0.1,1\}$ and $\rho =\{0.1,1,5\}$. We generate 2000 Monte Carlo samples of size n = 1000 and we compute the coverage rates (CR) given by the proportion of estimates that filled in $95\mathrm{\%}$ confidence interval and the bias, given by the difference between the mean of the estimates and the true value of the parameters. In this context, we use the following structure functions for $m_i$: $m_i=1$ (homoscedasticity), $m_i={\mathrm{e}}^{\rho x_i}$ and $m_i=z_i^{\rho }$.

For CR, we expected a value close to $95\mathrm{\%}$ and for the bias a value close to 0. According to Tables 7 and 8, the true structure function attains our expectations in terms of CR bias for all parameters taken into consideration. On the other hand, we note that other specifications of $m_i$, in general, present a relatively large bias and a low CR in at least one parameter.

Table 7.

Coverage rates (CR) at the nominal level of $5\mathrm{\%}$ and bias for different structure functions with $m_i={\mathrm{e}}^{\rho x_i}$ (true values of the parameters are in parentheses).

	$m_i=1$		$m_i={\mathrm{e}}^{\rho x_i}$		$m_i=x_i^{\rho }$
Parameter	CR	bias	CR	bias	CR	bias
$\beta (2)$	93.85	0.005	96.25	−0.061	96.49	−0.127
${\sigma }^2(0.1)$	69.50	0.011	94.05	0.001	73.05	0.008
$\lambda (3)$	94.80	0.086	96.00	−0.035	96.09	−0.141
$\rho (0.1)$	–	–	92.25	0.016	86.27	0.009
$\beta (2)$	2.00	0.096	95.40	−0.055	93.93	−0.125
${\sigma }^2(0.1)$	0.00	0.235	92.45	0.001	8.72	0.204
$\lambda (3)$	93.20	0.079	96.05	−0.017	96.09	−0.086
$\rho (1)$	–	–	95.10	0.006	44.86	−0.165
$\beta (2)$	0.00	5.965	87.00	−0.279	18.16	−1.253
${\sigma }^2(0.1)$	0.00	263.417	91.85	0.010	18.41	36.800
$\lambda (3)$	27.35	0.940	96.15	−0.167	27.80	1.343
$\rho (5)$	–	–	94.30	−0.562	2.01	−1.350
$\beta (2)$	93.00	0.016	95.65	−0.075	95.64	−0.135
${\sigma }^2(1)$	69.35	0.111	93.30	0.039	73.63	0.044
$\lambda (3)$	93.95	0.122	96.34	−0.014	95.99	−0.107
$\rho (0.1)$	–	–	91.75	0.021	87.02	0.016
$\beta (2)$	1.80	0.302	94.80	−0.091	94.49	−0.097
${\sigma }^2(1)$	0.00	2.354	93.00	−0.056	20.46	1.761
$\lambda (3)$	93.30	0.112	95.25	−0.021	87.21	−0.374
$\rho (1)$	–	–	95.70	−0.004	44.43	−0.172
$\beta (2)$	0.00	18.438	73.20	−0.377	27.22	−3.129
${\sigma }^2(1)$	0.00	2620.733	91.35	−0.144	27.07	337.696
$\lambda (3)$	27.15	0.963	95.65	6.413	14.17	38.826
$\rho (5)$	–	–	94.05	−0.695	1.32	−1.570

Table 8.

Coverage rates (CR) at the nominal level of $5\mathrm{\%}$ and bias for different structure functions with $m_i=x_i^{\rho }$ (true values of the parameters are in parentheses).

	$m_i=1$		$m_i={\mathrm{e}}^{\rho x_i}$		$m_i=x_i^{\rho }$
Parameter	CR	bias	CR	bias	CR	bias
$\beta (2)$	89.10	0.008	92.65	0.001	92.24	0.001
${\sigma }^2(0.1)$	94.40	−0.001	72.45	−0.012	91.70	−0.001
$\lambda (3)$	95.75	0.080	94.10	0.052	93.90	0.050
$\rho (0.1)$	–	–	93.25	0.011	92.40	0.003
$\beta (2)$	1.20	0.058	92.50	0.005	92.30	0.001
${\sigma }^2(0.1)$	83.35	0.007	0.00	−0.070	91.70	−0.001
$\lambda (3)$	92.70	0.085	92.75	0.089	92.90	0.043
$\rho (1)$	–	–	89.85	0.070	92.40	0.015
$\beta (2)$	0.00	0.094	89.60	0.011	93.10	0.001
${\sigma }^2(0.1)$	15.25	0.026	0.00	−0.085	91.40	−0.001
$\lambda (3)$	89.75	0.086	93.20	0.096	92.55	0.037
$\rho (5)$	–	–	87.45	0.092	92.25	0.018
$\beta (2)$	91.20	0.019	92.75	−0.001	92.35	−0.001
${\sigma }^2(1)$	93.85	−0.013	70.10	−0.130	91.10	−0.014
$\lambda (3)$	94.80	0.099	92.90	0.071	92.80	0.069
$\rho (0.1)$	–	–	91.60	0.013	91.95	0.005
$\beta (2)$	1.00	0.182	93.25	0.012	93.20	−0.003
${\sigma }^2(1)$	79.45	0.078	0.00	−0.708	91.20	−0.016
$\lambda (3)$	92.65	0.106	92.20	0.125	93.10	0.081
$\rho (1)$	–	–	87.70	0.081	92.45	0.020
$\beta (2)$	0.00	0.293	90.30	0.028	91.20	−0.007
${\sigma }^2(1)$	15.95	0.256	0.00	−0.854	90.70	−0.009
$\lambda (3)$	89.95	0.098	92.45	0.140	92.90	0.083
$\rho (5)$	–	–	84.45	0.102	90.55	0.032

9. Application

As suggested by the analysis presented in Section 2, we generalize the model (1) considering heteroscedastic errors, i.e. ${ϵ}_i\sim \mathrm{SN}(0,{\sigma }_i^2,\lambda )$, where ${\sigma }_i^2={\sigma }^2{m}_i(\rho ,z_i)$, with $z_i=\frac{temperatur{e}_i-a}{b-a}$, $a=min\{temperature\}$ and $b=max\{temperature\}$.

Following [34], to test the homogeneity of the scalar parameter, using the LR statistic given in Section 7, we assume $m_i(\rho ,z_i)={\mathrm{e}}^{\rho z_i}$ for simplicity. It is easily seen that when $\rho =0,$ then $m_i=1,\mathrm{\forall }i$. Hence, we have that the LR=118.687 (p-value $\approx 0$), which indicates there is significant evidence of a varying scalar parameter and consequently of heterogeneity in the ragweed data set. Meanwhile, we assume that $m_i(\rho ,z_i)=z_i^{\rho }$. Then the test is still $H_0:\rho =0$. By a similar computation, we get LR=91.278 (p-value $\approx 0$). Therefore, these results are similar to those when we choose the exponential function above.

The results of the fit in terms of log-likelihood and BIC are provided in Table 9. Looking at the BIC values, we see that the heteroscedastic PLM-SN models fit the data better than the homoscedastic PLM-SN model. In particular, the best fit was the heteroscedastic PLM-SN model assuming $m_i(\rho ,z_i)={\mathrm{e}}^{\rho z_i}$. Table 10 summarizes the MPLE results, including the effective degrees of freedom under skew-normal partially linear models fitted. Furthermore, we construct graphs of the standard residuals $e_i^1,i=1,\dots ,335$, defined in Section 5, under the HPLM-SN model (log-linear dispersion). Figures 4(a–c) show that the residuals $e_i^1$'s of the HPLM-SN (log-linear dispersion) do not present more any tendency, confirming that this model is more appropriate for the data set.

Table 9.

Comparison of penalized log-likelihood maximum and BIC for fitted various models using the ragweed levels data. Best fit indicated by (*1).

		Heteroscedastic
	Homoscedastic	(log-linear dispersion)	(power product dispersion)
${\ell }_p(\hat{\mathit{\theta }},\alpha )$	−730.822	−671.479	−685.183
BIC	1556.033	1473.77 (*1)	1490.496

Table 10.

MPLE results and approximate standard errors (SE) under skew-normal partially linear models fitted to the ragweed levels data.

	Homoscedastic		Heteroscedastic
			(log-linear dispersion)
Effect	Estimate	SE	Estimate	SE
Rain	1.456	0.382	0.627	0.234
Temperature	0.088	0.0183	−0.006	0.013
Wind	0.228	0.037	0.130	0.025
${\sigma }^2$	9.760	1.362	0.601	0.093
λ	2.189	0.482	2.705	0.615
ρ	–	–	4.620	0.1450
α	211.486	–	96.057	–
$df(\alpha )$	11.234		16.500

Figure 4.

Residuals of the Heteroscedastic PLM-SN model (exponential function) fitted ( $e_i^1$'s) to the ragweed levels data: (a) plot of residuals, (b) scatter plot between residuals and temperature and (b) scatter plot between residuals, and wind speed.

In order to detect possible outlying observations and to assess the goodness-of-fit of the models, we constructed quantile-quantile plots with simulated confidence bands of $95\mathrm{\%}$ [1] based on the Mahalanobis distance $d_i^2,i=1,\dots ,335$. In Figure 5(a), the HPLM-SN model (exponential function) does not present observations outside of the confidence bands. We may notice from Figure 5(b) that the fit under heteroscedastic skew-normal errors (exponential function) seems to be accurate for capturing the tendency at the end of the season.

Figure 5.

Quantile-quantile plots for the Mahalanobis distance and the $95\mathrm{\%}$ pointwise confidence bands for f (Days in season) from the HPLM-SN model (exponential function) fitted to the ragweed levels data.

Next, we identify influential cases for the data set using $M(0)$. Figure 6 presents the index graphs of M(0) for the proposed perturbation schemes. From Figure 6(a), it can be seen that cases 87, 94, 239 and 270 are observations with an outstanding contribution on the log-likelihood function and that may exercise a high influence on the maximum-likelihood estimates. Case 87 seems to be the most influential in the ML estimators in the HPLM-SN model (exponential function) under the case-weight scheme. From Figure 6(b) it can be seen that observations 234, 276, 289, 293, 297, 323, 325 and 330 appear as possibly influential in response perturbation, which may indicate observations with a large influence on their own predicted values. Case 297 seems to be the most influential in the MPLE results in the HPLM-SN model (exponential function) under response variable perturbation. We now examine the effects of perturbing in the specific explanatory variable, i.e. temperature and wind speed. Figures 6(c–d) illustrate the index plot for perturbation of the temperature and wind speed, respectively. Using this perturbation scheme, we can examine that the same observations that stand out as influential in the response variable are also for the explanatory variable wind speed. In addition, cases 87, 239, 265, 270, 276, 299 and 315 are identified as influential under perturbation of the temperature, highlighting observations 239, 270 and 87 that were also identified under the case-weight scheme. Finally, from Figure 6(e) it can be seen that observations 88, 96, 115, 276 and 315 appear as possibly influential in skewness perturbation, which may reveal cases that are most influential, in the sense, of the likelihood displacement on the skewness structure and consequently on the λ estimate. Cases 276 and 315 seem to be the most influential in the MPLE results in the HPLM-SN model (exponential function) under skewness perturbation, and such observations were also identified as influential in the perturbation scheme of the explanatory variable temperature.

Figure 6.

Diagnostic graphs from the HPLM-SN model (exponential function) fitted to the ragweed level data with $c^{\ast }=3$: (a) case-weight perturbation; (b) response (square root of ragweed) perturbation; (c) explanatory (temperature) perturbation; (d) explanatory (wind speed) perturbation and (e) skewness perturbation.

Observations 87 and 239 present low levels of ragweed (equal to the first quartile of the response variable values) and low temperature (below the median temperature). However, observation 270 presents a high level of ragweed and temperature (above the median of the values of the respective variables). Observation 297 presents a low level of ragweed and wind speed (below the median of the values of the respective variables). Furthermore, observation 276 presents a high level of ragweed and temperature (above the third quartile of the values of the respective variables), while observation 315 has the lowest level of ragweed and high temperature (above the median temperature). Case 276 is in the right tail of the response variable distribution.

10. Conclusions

In this paper, an extension of the skew-normal partially linear model is developed by considering the case where the error terms are independent and follow a skew-normal distribution in the presence of heteroscedasticity. Our proposed model generalizes the recent work of Ferreira and Paula [12]. We developed the maximum likelihood estimator of the parameters based on the ECM algorithm and we obtained analytic expressions for the E and M-steps, except for the parameter of the heterogeneity that requires a CM-step in the algorithm. Local influence methods were implemented for the HPLM-SN model to evaluate the consequences of model perturbations in situations where different perturbation schemes are investigated. In addition, we discuss the Likelihood Ratio test for homogeneity in the HPLM-SN model. To examine the performances and properties of the LR test, formal simulations studies under several situations were performed, including in the context of misspecication of the structure function. Lastly, the HPLM-SN model was applied to a real data set and compared with the homoscedastic version showing the usefulness of the HPLM-SN to fit data sets with nonparametric components in which the responses are asymmetric and heteroscedastic. Finally, the model proposed in this paper can be extended in context generalized additive models (GAM) as an alternative to relaxing the assumption that the heteroscedasticity function is known. We thank the anonymous referee for valuable suggestions for future research. The authors hope to report these findings in a future paper.

Appendix 1. Approximate standard errors.

In this appendix, we derive the observed information matrix associated with the parameter vector $\mathit{\theta }$. The observed information matrix will be used to calculate the standard errors of the estimate $\hat{\mathit{\theta }}$. Following the same procedure as Segal et al. [31], we derive the variance-covariance matrix of the $\hat{\mathit{\theta }}$ from the inverse of the observed information matrix, which is obtained by treating the penalized likelihood function (8) as a usual likelihood. Given the HPLM-SN model in Equations (5)–(6), the corresponding penalized log-likelihood function of $\mathit{\theta }=({\mathit{\beta }}^{\mathrm{\top }},{\mathbf{f}}^{\mathrm{\top }},{\sigma }^2,\lambda {)}^{\mathrm{\top }}$ is of the form ${\ell }_p(\mathit{\theta })=\sum _{i=1}^n{\ell }_{p_i}(\mathit{\theta })$, with ${\ell }_{p_i}(\mathit{\theta })=\log 2+{\ell }_{1_{p_i}}(\mathit{\theta })+\log \mathrm{\Phi }({\ell }_{2_i}(\mathit{\theta }))$, where ${\ell }_{1_{p_i}}(\mathit{\theta })=\log \varphi (y_i;{\mu }_i,{\sigma }_i^2)-\alpha /(2n){\mathbf{f}}^{\mathrm{\top }}\mathbf{K}\mathbf{f}$ and ${\ell }_{2_i}(\mathit{\theta })=\lambda (y_i-{\mu }_i)/{\sigma }_i$, with ${\mu }_i={\mathbf{x}}_i^{\mathrm{\top }}\mathit{\beta }+{\mathbf{n}}_i^{\mathrm{\top }}\mathbf{f}$ and ${\sigma }_i^2={\sigma }^2{m}_i(\mathit{\rho },{\mathbf{z}}_i)$. Thus, the observed information matrix for $\mathit{\theta }$ can be written as ${\mathbf{I}}_{\mathit{\theta }\mathit{\theta }}=-\sum _{i=1}^n\frac{{\mathrm{\partial }}^2{\ell }_{p_i}(\mathit{\theta })}{\mathrm{\partial }\mathit{\theta }\mathrm{\partial }{\mathit{\theta }}^{\mathrm{\top }}}={\mathbf{I}}^1(\mathit{\theta })+{\mathbf{I}}^2(\mathit{\theta }),$ where ${\mathbf{I}}^1(\mathit{\theta })=-\sum _{i=1}^n\frac{{\mathrm{\partial }}^2{\ell }_{1_{p_i}}(\mathit{\theta })}{\mathrm{\partial }\mathit{\theta }\mathrm{\partial }{\mathit{\theta }}^{\mathrm{\top }}}$ and ${\mathbf{I}}^2(\mathit{\theta })=-\sum _{i=1}^n[W_{\mathrm{\Phi }}({\ell }_{2_i}(\mathit{\theta }))\frac{{\mathrm{\partial }}^2{\ell }_{2_i}(\mathit{\theta })}{\mathrm{\partial }\mathit{\theta }\mathrm{\partial }{\mathit{\theta }}^{\mathrm{\top }}}+W_{\mathrm{\Phi }}^{\prime }({\ell }_{2_i}(\mathit{\theta }))\frac{\mathrm{\partial }{\ell }_{2_i}(\mathit{\theta })}{\mathrm{\partial }\mathit{\theta }}\frac{\mathrm{\partial }{\ell }_{2_i}(\mathit{\theta })}{\mathrm{\partial }{\mathit{\theta }}^{\mathrm{\top }}}],$ with $W_{\mathrm{\Phi }}^{\prime }(x)=-W_{\mathrm{\Phi }}(x)(x+W_{\mathrm{\Phi }}(x))$ and $W_{\mathrm{\Phi }}(x)=\varphi (x)/\mathrm{\Phi }(x)$. The first-order and second-order derivatives of ${\ell }_{1_{p_i}}(\mathit{\theta })$ in relation to $\mathit{\theta }$ can be calculated as follows:

$$\begin{array}{rl}& \frac{\mathrm{\partial }{\ell }_{1_{p_i}}(\mathit{\theta })}{\mathrm{\partial }\mathit{\beta }}=\frac{(y_i-{\mu }_i)}{{\sigma }^2{m}_i}{\mathbf{x}}_i;\frac{\mathrm{\partial }{\ell }_{1_{p_i}}(\mathit{\theta })}{\mathrm{\partial }\mathbf{f}}=\frac{(y_i-{\mu }_i)}{{\sigma }^2{m}_i}{\mathbf{n}}_i-\frac{\alpha }{n}\mathbf{K}\mathbf{f};\\ & \frac{\mathrm{\partial }{\ell }_{1_{p_i}}(\mathit{\theta })}{\mathrm{\partial }{\sigma }^2}=\frac{(d_i-1)}{2{\sigma }^2};\frac{\mathrm{\partial }{\ell }_{1_{p_i}}(\mathit{\theta })}{\mathrm{\partial }\lambda }=0;\\ & \frac{\mathrm{\partial }{\ell }_{1_{p_i}}(\mathit{\theta })}{\mathrm{\partial }\mathit{\rho }}=\frac{(d_i-1)}{2m_i}\frac{\mathrm{\partial }{m}_i}{\mathrm{\partial }\mathit{\rho }};\frac{{\mathrm{\partial }}^2{\ell }_{1_{p_i}}(\mathit{\theta })}{\mathrm{\partial }\mathit{\beta }\mathrm{\partial }{\mathit{\beta }}^{\mathrm{\top }}}=-\frac{1}{{\sigma }^2{m}_i}{\mathbf{x}}_i{\mathbf{x}}_i^{\mathrm{\top }};\frac{{\mathrm{\partial }}^2{\ell }_{1_{p_i}}(\mathit{\theta })}{\mathrm{\partial }\mathbf{f}\mathrm{\partial }{\mathit{\beta }}^{\mathrm{\top }}}=-\frac{1}{{\sigma }^2{m}_i}{\mathbf{n}}_i{\mathbf{x}}_i^{\mathrm{\top }};\\ & \frac{{\mathrm{\partial }}^2{\ell }_{1_{p_i}}(\mathit{\theta })}{\mathrm{\partial }{\sigma }^2\mathrm{\partial }\mathit{\beta }}=-\frac{(y_i-{\mu }_i)}{{\sigma }^4{m}_i}{\mathbf{x}}_i;\frac{{\mathrm{\partial }}^2{\ell }_{1_{p_i}}(\mathit{\theta })}{\mathrm{\partial }\mathit{\rho }\mathrm{\partial }{\mathit{\beta }}^{\mathrm{\top }}}=-\frac{(y_i-{\mu }_i)}{{\sigma }^2{m}_i^2}\frac{\mathrm{\partial }{m}_i}{\mathrm{\partial }\mathit{\rho }}{\mathbf{x}}_i^{\mathrm{\top }};\\ & \frac{{\mathrm{\partial }}^2{\ell }_{1_{p_i}}(\mathit{\theta })}{\mathrm{\partial }\mathbf{f}\mathrm{\partial }{\mathbf{f}}^{\mathrm{\top }}}=-\frac{1}{{\sigma }^2{m}_i}{\mathbf{n}}_i{\mathbf{n}}_i^{\mathrm{\top }}-\frac{\alpha }{n}\mathbf{K};\\ & \frac{{\mathrm{\partial }}^2{\ell }_{1_{p_i}}(\mathit{\theta })}{\mathrm{\partial }{\sigma }^2\mathrm{\partial }\mathbf{f}}=-\frac{(y_i-{\mu }_i)}{{\sigma }^4{m}_i}{\mathbf{n}}_i;\frac{{\mathrm{\partial }}^2{\ell }_{1_{p_i}}(\mathit{\theta })}{\mathrm{\partial }\mathit{\rho }\mathrm{\partial }{\mathbf{f}}^{\mathrm{\top }}}=-\frac{(y_i-{\mu }_i)}{{\sigma }^2{m}_i^2}\frac{\mathrm{\partial }{m}_i}{\mathrm{\partial }\mathit{\rho }}{\mathbf{n}}_i^{\mathrm{\top }};\frac{{\mathrm{\partial }}^2{\ell }_{1_{p_i}}(\mathit{\theta })}{\mathrm{\partial }{\sigma }^4}=\frac{1}{2{\sigma }^4}-\frac{d_i}{{\sigma }^4};\\ & \frac{{\mathrm{\partial }}^2{\ell }_{1_{p_i}}(\mathit{\theta })}{\mathrm{\partial }{\sigma }^2\mathrm{\partial }\mathit{\rho }}=-\frac{d_i}{2{\sigma }^2{m}_i}\frac{\mathrm{\partial }{m}_i}{\mathrm{\partial }\mathit{\rho }};\frac{{\mathrm{\partial }}^2{\ell }_{1_{p_i}}(\mathit{\theta })}{\mathrm{\partial }\mathit{\rho }\mathrm{\partial }{\mathit{\rho }}^{\mathrm{\top }}}=\frac{1-2d_i}{2m_i^2}\frac{\mathrm{\partial }{m}_i}{\mathrm{\partial }\mathit{\rho }}\frac{\mathrm{\partial }{m}_i}{\mathrm{\partial }{\mathit{\rho }}^{\mathrm{\top }}}+\frac{d_i-1}{2m_i}\frac{{\mathrm{\partial }}^2{m}_i}{\mathrm{\partial }\mathit{\rho }\mathrm{\partial }{\mathit{\rho }}^{\mathrm{\top }}};\\ & \frac{{\mathrm{\partial }}^2{\ell }_{1_{p_i}}(\mathit{\theta })}{\mathrm{\partial }\mathit{\theta }\mathrm{\partial }\lambda }=\mathbf{0},\end{array}$$

with $d_i=\frac{(y_i-{\mu }_i{)}^2}{{\sigma }^2{m}_i}$. The first-order and second-order derivatives of ${\ell }_{2_i}(\mathit{\theta })$ in relation to $\mathit{\theta }$ are given by

$$\begin{array}{rl}& \frac{\mathrm{\partial }{\ell }_{2_i}(\mathit{\theta })}{\mathrm{\partial }\mathit{\beta }}=-\frac{\lambda }{\sigma m_i^{1/2}}{\mathbf{x}}_i\frac{\mathrm{\partial }{\ell }_{2_i}(\mathit{\theta })}{\mathrm{\partial }\mathbf{f}}=-\frac{\lambda }{\sigma m_i^{1/2}}{\mathbf{n}}_i;\frac{\mathrm{\partial }{\ell }_{2_i}(\mathit{\theta })}{\mathrm{\partial }{\sigma }^2}=-\frac{\lambda (y_i-{\mu }_i)}{2{\sigma }^3{m}_i^{1/2}};\\ & \frac{\mathrm{\partial }{\ell }_{2_i}(\mathit{\theta })}{\mathrm{\partial }\lambda }=\frac{(y_i-{\mu }_i)}{\sigma m_i^{1/2}};\frac{\mathrm{\partial }{\ell }_{2_i}(\mathit{\theta })}{\mathrm{\partial }\mathit{\rho }}=-\frac{\lambda }{2\sigma }\frac{(y_i-{\mu }_i)}{m_i^{3/2}}\frac{\mathrm{\partial }{m}_i}{\mathrm{\partial }\mathit{\rho }};\\ & \frac{{\mathrm{\partial }}^2{\ell }_{2_i}(\mathit{\theta })}{\mathrm{\partial }\mathit{\beta }\mathrm{\partial }{\mathit{\beta }}^{\mathrm{\top }}}=\frac{{\mathrm{\partial }}^2{\ell }_{2_i}(\mathit{\theta })}{\mathrm{\partial }\mathbf{f}\mathrm{\partial }{\mathbf{f}}^{\mathrm{\top }}}=\frac{{\mathrm{\partial }}^2{\ell }_{2_i}(\mathit{\theta })}{\mathrm{\partial }\mathit{\beta }\mathrm{\partial }{\mathbf{f}}^{\mathrm{\top }}}=\mathbf{0};\frac{{\mathrm{\partial }}^2{\ell }_{2_i}(\mathit{\theta })}{\mathrm{\partial }{\lambda }^2}=0;\\ & \frac{{\mathrm{\partial }}^2{\ell }_{2_i}(\mathit{\theta })}{\mathrm{\partial }{\sigma }^2\mathrm{\partial }\mathit{\beta }}=\frac{\lambda }{2{\sigma }^3{m}_i^{1/2}}{\mathbf{x}}_i;\frac{{\mathrm{\partial }}^2{\ell }_{2_i}(\mathit{\theta })}{\mathrm{\partial }{\sigma }^2\mathrm{\partial }\mathbf{f}}=\frac{\lambda }{2{\sigma }^3{m}_i^{1/2}}{\mathbf{n}}_i;\frac{{\mathrm{\partial }}^2{\ell }_{2_i}(\mathit{\theta })}{\mathrm{\partial }\lambda \mathrm{\partial }\mathit{\beta }}=-\frac{1}{\sigma m_i^{1/2}}{\mathbf{x}}_i;\\ & \frac{{\mathrm{\partial }}^2{\ell }_{2_i}(\mathit{\theta })}{\mathrm{\partial }\lambda \mathrm{\partial }\mathbf{f}}=-\frac{1}{\sigma m_i^{1/2}}{\mathbf{n}}_i;\frac{{\mathrm{\partial }}^2{\ell }_{2_i}(\mathit{\theta })}{\mathrm{\partial }\mathit{\rho }\mathrm{\partial }{\mathit{\beta }}^{\mathrm{\top }}}=\frac{\lambda }{2\sigma m_i^{3/2}}\frac{\mathrm{\partial }{m}_i}{\mathrm{\partial }\mathit{\rho }}{\mathbf{x}}_i^{\mathrm{\top }};\frac{{\mathrm{\partial }}^2{\ell }_{2_i}(\mathit{\theta })}{\mathrm{\partial }\mathit{\rho }\mathrm{\partial }{\mathbf{f}}^{\mathrm{\top }}}=\frac{\lambda }{2\sigma m_i^{3/2}}\frac{\mathrm{\partial }{m}_i}{\mathrm{\partial }\mathit{\rho }}{\mathbf{n}}_i^{\mathrm{\top }};\\ & \frac{{\mathrm{\partial }}^2{\ell }_{2_i}(\mathit{\theta })}{\mathrm{\partial }{\sigma }^4}=\frac{3\lambda }{4{\sigma }^5}\frac{(y_i-{\mu }_i)}{m_i^{1/2}};\frac{{\mathrm{\partial }}^2{\ell }_{2_i}(\mathit{\theta })}{\mathrm{\partial }{\sigma }^2\mathrm{\partial }\lambda }=-\frac{1}{2{\sigma }^3}\frac{(y_i-{\mu }_i)}{m_i^{1/2}};\\ & \frac{{\mathrm{\partial }}^2{\ell }_{2_i}(\mathit{\theta })}{\mathrm{\partial }{\sigma }^2\mathrm{\partial }\mathit{\rho }}=\frac{\lambda }{4{\sigma }^3}\frac{(y_i-{\mu }_i)}{m_i^{3/2}}\frac{\mathrm{\partial }{m}_i}{\mathrm{\partial }\mathit{\rho }};\\ & \frac{{\mathrm{\partial }}^2{\ell }_{2_i}(\mathit{\theta })}{\mathrm{\partial }\lambda \mathrm{\partial }\mathit{\rho }}=-\frac{1}{2\sigma }\frac{(y_i-{\mu }_i)}{m_i^{3/2}}\frac{\mathrm{\partial }{m}_i}{\mathrm{\partial }\mathit{\rho }};\\ & \frac{{\mathrm{\partial }}^2{\ell }_{2_i}(\mathit{\theta })}{\mathrm{\partial }\mathit{\rho }\mathrm{\partial }{\mathit{\rho }}^{\mathrm{\top }}}=-\frac{\lambda }{2\sigma }\frac{(y_i-{\mu }_i)}{m_i^{3/2}}[-\frac{3}{2m_i}\frac{\mathrm{\partial }{m}_i}{\mathrm{\partial }\mathit{\rho }}\frac{\mathrm{\partial }{m}_i}{\mathrm{\partial }{\mathit{\rho }}^{\mathrm{\top }}}+\frac{{\mathrm{\partial }}^2{m}_i}{\mathrm{\partial }\mathit{\rho }\mathrm{\partial }{\mathit{\rho }}^{\mathrm{\top }}}].\end{array}$$

Appendix 2. Calculus of the matrices $\mathbf{N}$ and $\mathbf{K}$.

• Eilers and Marx's method

Let $\mathbf{t}=(t_1,\dots ,t_n)$ and ndx as the number of equidistant knots desired by the user.

Commands in R [29]:

require(splines)

bspline=function(x,ndx,bdeg) $\{$

xl=min(x)

xr=max(x)

dx=(xr-xl)/ndx

knots=seq(xl-bdeg*dx,xr+bdeg*dx,by=dx)

B=splineDesign(knots,x,bdeg+1,0*x,outer.ok=T)

$\}$

B=bspline( $\mathbf{t}$,ndx,3)

D = diag(ncol(B))

for (k in 1:2) D = diff(D)

Thus, $\mathbf{N}=B=bspline(\mathbf{t},ndx,3)$ and $\mathbf{K}={\mathbf{D}}^{\mathrm{\top }}\mathbf{D}$.

• Green $\mathrm{\&}$ Silverman's method:

Let $t_1^0,\dots ,t_q^0$ the q distinct and ordered values of $t_i$, $i=1,\dots ,n$ and $\mathbf{N}$ a $(n-k)\times q$ incidence matrix whose $(i,j)th$ element equals the indicator function $I(t_i=t_j^0),$ $j=1,\dots ,q$. In addition, define $h_i=t_{i+1}^0-t_i^0$, for $i=1,\dots ,q-1$ and $\mathbf{S}$ as being the $q\times (q-2)$ matrix with entries $s_{ij}$, for $i=1,\dots ,q$ and $j=2,\dots ,q-1$, given by

$$\begin{array}{r}{s}_{j-1,j}=h_{j-1}^{-1},s_{jj}=-h_{j-1}^{-1}-h_j^{-1}\text{and}{s}_{j+1,j}=h_j^{-1},\text{for}j=2,\dots ,q-1,\end{array}$$

and $s_{i,j}=0$, for $|i-j|\ge 2$. Now, consider $\mathbf{R}$ as being a $(q-2)\times (q-2)$ matrix with elements $r_{ij}$ given by

$$\begin{array}{r}\begin{array}{c}{r}_{ii}=(h_{i-1}+h_i)/3,\text{for}i=2,\dots ,q-1,\\ r_{i,i+1}=r_{i+1,i}=h_i/6,\text{for}i=2,\dots ,q-2,\end{array}\end{array}$$

and $r_{ij}=0,\text{for}|i-j|\ge 2$. Then, $\mathbf{K}=\mathbf{S}{\mathbf{R}}^{-1}{\mathbf{S}}^{\mathrm{\top }}$.

Funding Statement

This work was supported by CNPq and Fundação de Amparo à Pesquisa do Estado de Minas Gerais (FAPEMIG).

Disclosure statement

No potential conflict of interest was reported by the author(s).