Improvement of mixed predictors in linear mixed models

Abstract

In this paper, we introduce stochastic-restricted Liu predictors which will be defined by combining in a special way the two approaches followed in obtaining the mixed predictors and the Liu predictors in the linear mixed models. Superiorities of the linear combination of the new predictor to the Liu and mixed predictors are done in the sense of mean square error matrix criterion. Finally, numerical examples and a simulation study are done to illustrate the findings. In numerical examples, we took some arbitrary observations from the data as the prior information since we did not have historical data or additional information about the data sets. The results show that this case does the new estimator gain efficiency over the constituent estimators and provide accurate estimation and prediction of the data.

1. Introduction

Many common statistical models can be expressed as linear models that incorporate both fixed effects, which are parameters associated with an entire population or with certain repeatable levels of experimental factors, and random effects, which are associated with individual experimental units drawn at random from a population. A model with both fixed effects and random effects is called a linear mixed model (LMM) [23]. LMMs provide a broad range of structures including longitudinal data, repeated measures data (including cross-over studies), growth and dose-response curve data, clustered (or nested) data, multivariate data and correlated data.

Let us consider the LMM

$$y_i=X_i\beta +Z_i{u}_i+{\epsilon }_i,i=1,\dots ,m$$

where $y_i$ is an $n_i\times 1$ vector of response variables measured on subject i, β is a $p\times 1$ parameter vector of fixed effects, $X_i$ and $Z_i$ are $n_i\times p$ and $n_i\times q$ known design matrices of the fixed and random effects, respectively, $u_i$ is a $q\times 1$ random vector, the components of which are called random effects and ${\epsilon }_i$ is an $n_i\times 1$ random vector of errors.

Usually one assumes that

$$u_i\stackrel{iid}{\sim }{N}_q\left(0,{\sigma }^{2}D\right)\mathrm{a}\mathrm{n}\mathrm{d}{\epsilon }_i\stackrel{iid}{\sim }{N}_{n_i}\left(0,{\sigma }^2{W}_i\right),i=1,\dots ,m$$

where $u_i$ and ${\epsilon }_i$ are independent, D and $W_i$ are $q\times q$ and $n_i\times n_i$ known positive definite (pd) matrices.

Let $y=(y_1^{\mathrm{\prime }},\dots ,y_m^{\mathrm{\prime }}{)}^{\mathrm{\prime }}$, $X=(X_1^{\mathrm{\prime }},\dots ,X_m^{\mathrm{\prime }}{)}^{\mathrm{\prime }}$, $Z={\oplus }_{i=1}^m{Z}_i$, where ⊕ represents the direct sum, $u=(u_1^{\mathrm{\prime }},\dots ,u_m^{\mathrm{\prime }}{)}^{\mathrm{\prime }}$ and $\epsilon =({\epsilon }_1^{\mathrm{\prime }},\dots ,{\epsilon }_m^{\mathrm{\prime }}{)}^{\mathrm{\prime }}$. Then, we can write the model $y_i$ more compactly as

$$y=X\mathrm{\text{β}}+Zu+\epsilon$$ (1)

which implies that

$$\left(\begin{array}{c}u\\ \epsilon \end{array}\right)\sim N_{qm+n}\left(\left(\begin{array}{c}{0}_{qm}\\ 0_n\end{array}\right),\left(\begin{array}{cc}{\sigma }^{2}G& 0\\ 0& {\sigma }^{2}W\end{array}\right)\right)$$

where $n=\sum _{i=1}^m{n}_i$, $G=I_m\otimes D$ and $W={\oplus }_{i=1}^m{W}_i$, with ⊗ denoting the Kronecker product and $I_m$, the identity matrix of order m. Then, under model (1), we get $y\sim N(X\mathrm{\text{β}},{\sigma }^{2}H)$, where $H=ZG{Z}^{\mathrm{\prime }}+W$. For the convenience of the theoretical computations, we assume that the matrices G and W are known. If G and W are unknown, we replace G and W by their maximum likelihood (ML) or restricted maximum likelihood (REML) estimates and update the estimate of β by $\widetilde{\mathrm{\text{β}}}$ where $\widetilde{\mathrm{\text{β}}}$ is any estimator of β in LMM.

$\widehat{\mathrm{\text{β}}}$ and $\hat{u}$ are obtained as

$$\begin{array}{rl}\widehat{\mathrm{\text{β}}}& =(X^{\mathrm{\prime }}{H}^{-1}X{)}^{-1}{X}^{\mathrm{\prime }}{H}^{-1}y\end{array}$$ (2)

$$\begin{array}{rl}\hat{u}& =G{Z}^{\mathrm{\prime }}{H}^{-1}(y-X\widehat{\mathrm{\text{β}}})\end{array}$$ (3)

by Henderson [6] and Henderson et al. [7]. The Henderson's estimator and predictor given by (2) and (3) are called, respectively, as the best linear unbiased estimator (BLUE) and the best linear unbiased predictor (BLUP).

In linear regression model, we usually assume that the variables of fixed effects design matrix are independent. However, in practice, there may be strong or near to strong linear relationships among the variables of fixed effects design matrix. In that case the independence assumptions are no longer valid, which causes the problem of multicollinearity. In the existence of multicollinearity, at least one main diagonal element of $(X^{\mathrm{\prime }}{H}^{-1}X{)}^{-1}$ may be quite large, which in view of $Var\left(\widehat{\mathrm{\text{β}}}\right)={\sigma }^2(X^{\mathrm{\prime }}{H}^{-1}X{)}^{-1}$ means that at least one element of $\widehat{\mathrm{\text{β}}}$ may have a large variance, and $\widehat{\mathrm{\text{β}}}$ may be far from its true value.

In statistical research, there have been many attempts to provide better estimators, for example, by the incorporation of prior information available in the form of exact or stochastic restrictions [24, p. 111]. If prior information comes from the theory and imposes restrictions among parameters that should hold in exact terms, it could be included in the model as a deterministic restriction. On the other hand, if the prior information comes from previous estimations of similar but different models or samples, it could be considered as a hint or a range of values that should contain the value of the parameter with some probability. If this information is not taken into account in the estimation despite being good, then the information will be wasted as well as the chance of improve the efficiency of the estimator. This is the idea behind stochastic restrictions approach, and the stochastic restrictions yield asymptotic efficiency gains under some specific assumptions about the asymptotics of prior information (shown in [27,31] for a linear model under normality of the errors). The collection and use of stochastic restriction also solve the multicollinearity problem (see [1,18]). The mixed estimation method suggested by Theil and Goldberger [31] in the linear regression model is extended to LMMs by Kuran and Özkale [12].

In addition to model (1), Kuran and Özkale [12] used the stochastic linear restriction as

$$r=R\mathrm{\text{β}}+\mathrm{\Phi }$$ (4)

where r is an $s\times 1$ random vector, R is a known $s\times p$ prior information of rank $s\le p$ and Φ is a random vector, independent of ϵ, with $E\left(\mathrm{\Phi }\right)=0$ and $Var\left(\mathrm{\Phi }\right)={\sigma }^{2}V$, where V is a known pd matrix and proposed, respectively, the mixed estimator and the mixed predictor in LMMs as

$$\begin{array}{rl}{\widehat{\mathrm{\text{β}}}}_{me}& =(X^{\mathrm{\prime }}{H}^{-1}X+R^{\mathrm{\prime }}{V}^{-1}R{)}^{-1}(X^{\mathrm{\prime }}{H}^{-1}y+R^{\mathrm{\prime }}{V}^{-1}r)\\ & =\widehat{\mathrm{\text{β}}}+\left(X^{\mathrm{\prime }}{H}^{-1}X{)}^{-1}{R}^{\mathrm{\prime }}\right(V+R\left(X^{\mathrm{\prime }}{H}^{-1}X{)}^{-1}{R}^{\mathrm{\prime }}{)}^{-1}\right(r-R\widehat{\mathrm{\text{β}}})\\ {\hat{u}}_{mp}& =G{Z}^{\mathrm{\prime }}{H}^{-1}(y-X{\widehat{\mathrm{\text{β}}}}_{me}).\end{array}$$ (5)

where $\widehat{\mathrm{\text{β}}}$ is given by Equation (2).

Another attempt to provide better estimation is introduced by Liu [13] called as the Liu estimator. And then, Kaçıranlar et al. [10] improved Liu's approach, and introduced restricted Liu estimator and discussed its stochastic properties. By using Swindel [29] and Özkale and Kaçıranlar's [20] approach in the linear regression models, Liu's approach in the linear regression model is enlarged to LMMs by Özkale and Kuran [21] in view of penalized log-likelihood. Thus, Özkale and Kuran [21] suggested, respectively, the Liu estimator and the Liu predictor in LMMs as

$$\begin{array}{rl}{\widehat{\mathrm{\text{β}}}}_d& =(X^{\mathrm{\prime }}{H}^{-1}X+I_p{)}^{-1}(X^{\mathrm{\prime }}{H}^{-1}y+d\widehat{\mathrm{\text{β}}})=F_d\widehat{\mathrm{\text{β}}}\\ {\hat{u}}_d& =G{Z}^{\mathrm{\prime }}{H}^{-1}(y-X{\widehat{\mathrm{\text{β}}}}_d)\end{array}$$

where $F_d=(X^{\mathrm{\prime }}{H}^{-1}X+I_p{)}^{-1}(X^{\mathrm{\prime }}{H}^{-1}X+d{I}_p)$, 0<d<1 is the Liu biasing parameter and $\widehat{\mathrm{\text{β}}}$ is the BLUE given by Equation (2).

Hubert and Wijekoon [8] introduced another alternative Liu-type estimator which will be defined by combining in a special way the two approaches followed in obtaining the mixed estimator and the Liu estimator in the linear regression models. They called their new biased estimator as the stochastic restricted Liu estimator (SRLE). The mean squared error matrix of SPLR was compared with several other biased estimators, and the conditions needed for the superiority over these biased estimators were derived by Hubert and Wijekoon [8]. Liu estimator is also studied by authors in several models and estimators. The Logistic Liu Estimator (LLE) [14]; the Liu-Type Logistic Estimator (LTLE) [9]; the Almost Unbiased Liu Logistic Estimator (AULLE) [33]; the Restricted Logistic Liu Estimator (RLLE) [30] and the Stochastic Restricted Liu Maximum Likelihood Estimator (SRLMLE) [32] have been proposed in the linear regression literature.

Our primary aim in this article is to widen Hubert and Wijekoon's [8] idea under Özkale and Kaçıranlar's [20] approach in the linear regression models to LMMs and the article is organized as follows. In Section 2, we propose the estimator and predictor in linear form of mixed estimator, abbreviated respectively by LFME and LFMP, in LMMs by using Kuran and Özkale's [12] mixed estimation method and Özkale and Kuran's [21] Liu approach. Superiorities of the linear combinations of the predictors are done in the sense of mean square error (MSE) matrix criterion in Section 3. Numerical examples and a simulation study are done to illustrate the findings in Section 4 and Section 5, respectively. Finally, we give some conclusions in Section 6.

2. Stochastic restricted Liu predictors in linear mixed models

In this section, we estimate the parameter vectors of fixed and random effects via the penalized log-likelihood approach.

Under the assumptions of model (1), u and y are jointly Gaussian distributed as

$$\left(\begin{array}{c}u\\ y\end{array}\right)\sim N\left(\left(\begin{array}{c}0\\ X\mathrm{\text{β}}\end{array}\right),{\sigma }^2\left(\begin{array}{cc}G& G{Z}^{\mathrm{\prime }}\\ ZG& H\end{array}\right)\right).$$ (6)

Then, the conditional distribution of y given u is $y|u\sim N(X\mathrm{\text{β}}+Zu,{\sigma }^{2}W)$. Henderson et al. [7] developed a set of equations that simultaneously yield BLUE and BLUP. For this purpose, they maximize the joint density of y and u which will be for Equation (6) as

$$\begin{array}{rl}f\left(y,u\right)& =f\left(y|u\right)f\left(u\right)\\ & =(2\pi {\sigma }^2{)}^{-(n+qm)/2}{\left|W\right|}^{-1/2}{\left|G\right|}^{-1/2}\\ & \times \exp \left\{-\frac{1}{2{\sigma }^2}[{\left(y-X\mathrm{\text{β}}-Zu\right)}^{\mathrm{\prime }}{W}^{-1}\left(y-X\mathrm{\text{β}}-Zu\right)+u^{\mathrm{\prime }}{G}^{-1}u]\right\}\end{array}$$

where $|.|$ denotes the determinant of a matrix.

After the log-joint distribution of $f(y,u)$ is obtained as

$$\begin{array}{rl}\log f\left(y,u\right)& =\log f\left(y|u\right)+\log f\left(u\right)\\ & =-\frac{1}{2}\left\{\right(n+qm)\log \left(2\pi \right)+(n+qm)\log {\sigma }^2+\log \left|W\right|+\log \left|G\right|\\ & +[y-X\mathrm{\text{β}}-Zu{)}^{\mathrm{\prime }}{W}^{-1}\left(y-X\mathrm{\text{β}}-Zu\right)+u^{\mathrm{\prime }}{G}^{-1}u]/{\sigma }^2\}\end{array}$$

we add a penalization term (that is; a penalization term is corresponding to stochastic linear restriction given by Equation (4)) with regularization parameter $\delta =\frac{1}{2{\sigma }^2}\ge 0$ to $\log f(y,u)$,

$$\log f\left(y,u\right)-\frac{1}{2{\sigma }^2}(\mathrm{\text{β}}-D{\widehat{\mathrm{\text{β}}}}_{me}{)}^{\mathrm{\prime }}(\mathrm{\text{β}}-D{\widehat{\mathrm{\text{β}}}}_{me})$$ (7)

where $D=diag(d_1,\dots ,d_p)$ is a diagonal matrix with elements $0<d_i<1$ $i=1,\dots ,p$ as the Liu biasing parameters and ${\widehat{\mathrm{\text{β}}}}_{me}$ is the mixed estimator given by Equation (5).

The objective function (7) looks for an estimator which maximizes $\log f(y,u)$ in a class of estimators that is close to $D{\widehat{\mathrm{\text{β}}}}_{me}$ than the origin.

As a result of dropping the constant term and taking into consideration of the log function, Equation (7) can also be written as

$$\begin{array}{rl}& -\frac{1}{2{\sigma }^2}\{{\left(y-X\mathrm{\text{β}}\right)}^{\mathrm{\prime }}{W}^{-1}\left(y-X\mathrm{\text{β}}\right)+(\mathrm{\text{β}}-D{\widehat{\mathrm{\text{β}}}}_{me}{)}^{\mathrm{\prime }}(\mathrm{\text{β}}-D{\widehat{\mathrm{\text{β}}}}_{me})\}\\ & -\frac{1}{2{\sigma }^2}\left\{u^{\mathrm{\prime }}\right(Z^{\mathrm{\prime }}{W}^{-1}Z+G^{-1})u-2{\left(y-X\mathrm{\text{β}}\right)}^{\mathrm{\prime }}{W}^{-1}Zu\}\end{array}$$ (8)

where the first two terms carry out the Liu estimation idea in the linear regression models and the second term carries out the estimation procedure through the random effects.

Equating the partial derivatives of Equation (8) with respect to the elements of β and u to zero and using ${\widehat{\mathrm{\text{β}}}}_{lfme}$ and ${\hat{u}}_{lfmp}$ to denote the LFME and LFMP gives

$$\begin{array}{rl}& X^{\mathrm{\prime }}{W}^{-1}(y-X{\widehat{\mathrm{\text{β}}}}_{lfme})+D{\widehat{\mathrm{\text{β}}}}_{me}-{\widehat{\mathrm{\text{β}}}}_{lfme}-X^{\mathrm{\prime }}{W}^{-1}Z{\hat{u}}_{lfmp}=0\end{array}$$ (9)

$$\begin{array}{rl}& Z^{\mathrm{\prime }}{W}^{-1}(y-X{\widehat{\mathrm{\text{β}}}}_{lfme})-(Z^{\mathrm{\prime }}{W}^{-1}Z+G^{-1}){\hat{u}}_{lfmp}=0.\end{array}$$ (10)

Equations (9) and (10) can compactly be written in matrix as

$$\left(\begin{array}{cc}{X}^{\mathrm{\prime }}{W}^{-1}X+I_p& X^{\mathrm{\prime }}{W}^{-1}Z\\ Z^{\mathrm{\prime }}{W}^{-1}X& Z^{\mathrm{\prime }}{W}^{-1}Z+G^{-1}\end{array}\right)\left(\begin{array}{c}{\widehat{\mathrm{\text{β}}}}_{lfme}\\ {\hat{u}}_{lfmp}\end{array}\right)=\left(\begin{array}{c}{X}^{\mathrm{\prime }}{W}^{-1}y+D{\widehat{\mathrm{\text{β}}}}_{me}\\ Z^{\mathrm{\prime }}{W}^{-1}y\end{array}\right).$$ (11)

Using Gilmour et al.'s [5] approach, Equation (11) can be written as

$$C\hat{\mathrm{\Psi }}={\omega }^{\mathrm{\prime }}{W}^{-1}y+\kappa$$ (12)

where $\hat{\mathrm{\Psi }}=({\widehat{\mathrm{\text{β}}}}_{lfme}^{\mathrm{\prime }},{\hat{u}}_{lfmp}^{\mathrm{\prime }}{)}^{\mathrm{\prime }}$, $\omega =(X,Z)$, $\kappa =(D{\widehat{\mathrm{\text{β}}}}_{me}^{\mathrm{\prime }},0^{\mathrm{\prime }}{)}^{\mathrm{\prime }}$and $C={\omega }^{\mathrm{\prime }}{W}^{-1}\omega +$Ģ ${}^{+}$ is full rank with

$$\mathit{G}=\left(\begin{array}{cc}{I}_p& 0\\ 0& G\end{array}\right){\mathrm{a}\mathrm{n}\mathrm{d}\mathit{G}}^{+}=\left(\begin{array}{cc}{I}_p& 0\\ 0& G^{-1}\end{array}\right)$$

where the superscript ‘+’ denotes the Moore-Penrose inverse.

Solving Equation (12), we get

$$\hat{\mathrm{\Psi }}=C^{-1}{\omega }^{\mathrm{\prime }}{W}^{-1}y+C^{-1}\kappa$$ (13)

where $C^{-1}$ is found from the inverse formula of the partitioned matrix [26] as

$$C^{-1}=\left(\begin{array}{cc}\acute{N}& -\acute{N}{X}^{\mathrm{\prime }}{H}^{-1}ZG\\ -G{Z}^{\mathrm{\prime }}{H}^{-1}X\acute{N}& \mathrm{{\rm Y}}+G{Z}^{\mathrm{\prime }}{H}^{-1}X\acute{N}{X}^{\mathrm{\prime }}{H}^{-1}ZG\end{array}\right)$$

where $\acute{N}=(X^{\mathrm{\prime }}{H}^{-1}X+I_p{)}^{-1}$ and $\mathrm{{\rm Y}}=(Z^{\mathrm{\prime }}{W}^{-1}Z+G^{-1}{)}^{-1}$.

After $C^{-1}$ is replaced in Equation (13) and after algebraic simplifications, we get the LFME and the LFMP, respectively, as

$$\begin{array}{rl}{\widehat{\mathrm{\text{β}}}}_{lfme}& =(X^{\mathrm{\prime }}{H}^{-1}X+I_p{)}^{-1}(X^{\mathrm{\prime }}{H}^{-1}y+D{\widehat{\mathrm{\text{β}}}}_{me})\\ {\hat{u}}_{lfmp}& =G{Z}^{\mathrm{\prime }}{H}^{-1}(y-X{\widehat{\mathrm{\text{β}}}}_{lfme}).\end{array}$$

For the special case of $D=d{I}_p$, we rewrite ${\widehat{\mathrm{\text{β}}}}_{lfme}$ as 

$${\widehat{\mathrm{\text{β}}}}_{lfme}=(X^{\mathrm{\prime }}{H}^{-1}X+I_p{)}^{-1}(X^{\mathrm{\prime }}{H}^{-1}X+d{I}_p){\widehat{\mathrm{\text{β}}}}_{me}=F_d{\widehat{\mathrm{\text{β}}}}_{me}$$

where 0<d<1 is the Liu biasing parameter. ${\widehat{\mathrm{\text{β}}}}_{lfme}$ carries the idea of Hubert and Wijekoon [8].

3. The comparisons of the predictors in linear mixed models

Prediction of linear combinations of β and u can be expressed as $\mu =L^{\mathrm{\prime }}\mathrm{\text{β}}+M^{\mathrm{\prime }}u$ for specific matrices L$\in {\mathbb{R}}^{p\times 1}$ and $M\in {\mathbb{R}}^{q\times 1}$. This type of prediction problem was investigated by Yang et al. [34], Pereira and Coelho [22] and Robinson [25].

Under the LFMP, the predictor of μ is expressible as

$$\begin{array}{rl}{\hat{\mu }}_{lfmp}& =L^{\mathrm{\prime }}{\widehat{\mathrm{\text{β}}}}_{lfme}+M^{\mathrm{\prime }}{\hat{u}}_{lfmp}\\ & =\mathbb{Q}{\widehat{\mathrm{\text{β}}}}_{lfme}+M^{\mathrm{\prime }}G{Z}^{\mathrm{\prime }}{H}^{-1}y\end{array}$$

where $\mathbb{Q}=L^{\mathrm{\prime }}-M^{\mathrm{\prime }}G{Z}^{\mathrm{\prime }}{H}^{-1}X$. One of the criteria proposed for measuring the ‘betterness’ of ${\hat{\mu }}_{lfmp}$ is taken to be the MSE matrix criterion.

Following Štulajter [28], we can write the MSE matrix for ${\hat{\mu }}_{lfmp}$ as

$$\begin{array}{rl}MSE\left({\hat{\mu }}_{srlp}\right)& =E\left[\right({\hat{\mu }}_{lfmp}-\mu \left)\right({\hat{\mu }}_{lfmp}-\mu {)}^{\mathrm{\prime }}]\\ & =Var\left({\hat{\mu }}_{lfmp}\right)+Var\left(\mu \right)+bias\left({\hat{\mu }}_{lfmp}\right)bias({\hat{\mu }}_{lfmp}{)}^{\mathrm{\prime }}\\ & -Cov({\hat{\mu }}_{lfmp},\mu )-Cov(\mu ,{\hat{\mu }}_{lfmp}),\end{array}$$ (14)

where $bias\left({\hat{\mu }}_{lfmp}\right)=E({\hat{\mu }}_{lfmp}-\mu )$.

$Var\left({\hat{\mu }}_{lfmp}\right)$, $Var\left(\mu \right)$, $bias\left({\hat{\mu }}_{lfmp}\right)$ and $Cov({\hat{\mu }}_{lfmp},\mu )$ can be derived, respectively, as

$$\begin{array}{rl}Var\left({\hat{\mu }}_{lfmp}\right)& =\mathbb{Q}Var\left({\widehat{\mathrm{\text{β}}}}_{lfme}\right){\mathbb{Q}}^{\mathrm{\prime }}+\mathbb{Q}Cov({\widehat{\mathrm{\text{β}}}}_{lfme},y)H^{-1}ZGM\\ & +M^{\mathrm{\prime }}G{Z}^{\mathrm{\prime }}{H}^{-1}Cov(y,{\widehat{\mathrm{\text{β}}}}_{lfme}){\mathbb{Q}}^{\mathrm{\prime }}+{\sigma }^2{M}^{\mathrm{\prime }}G{Z}^{\mathrm{\prime }}{H}^{-1}ZGM,\end{array}$$ (15)

$$\begin{array}{rl}Var\left(\mu \right)& =Var(L^{\mathrm{\prime }}\mathrm{\text{β}}+M^{\mathrm{\prime }}u)={\sigma }^2{M}^{\mathrm{\prime }}GM,\end{array}$$ (16)

$$\begin{array}{rl}bias\left({\hat{\mu }}_{lfmp}\right)& =E({\hat{\mu }}_{lfmp}-\mu )\\ & =E(\mathbb{Q}{\widehat{\mathrm{\text{β}}}}_{lfme}+M^{\mathrm{\prime }}G{Z}^{\mathrm{\prime }}{H}^{-1}y-L^{\mathrm{\prime }}\mathrm{\text{β}}-M^{\mathrm{\prime }}u)\\ & =\mathbb{Q}bias\left({\widehat{\mathrm{\text{β}}}}_{lfme}\right),\end{array}$$ (17)

and

$$\begin{array}{rl}Cov({\hat{\mu }}_{lfmp},\mu )& =Cov(\mathbb{Q}{\widehat{\mathrm{\text{β}}}}_{lfme}+M^{\mathrm{\prime }}G{Z}^{\mathrm{\prime }}{H}^{-1}y,L^{\mathrm{\prime }}\mathrm{\text{β}}+M^{\mathrm{\prime }}u)\\ & =\mathbb{Q}Cov({\widehat{\mathrm{\text{β}}}}_{lfme},u)M+M^{\mathrm{\prime }}G{Z}^{\mathrm{\prime }}{H}^{-1}Cov(y,u)M\end{array}$$ (18)

where

$$\begin{array}{rl}Var\left({\widehat{\mathrm{\text{β}}}}_{lfme}\right)& ={\sigma }^2{F}_d(X^{\mathrm{\prime }}{H}^{-1}X+R^{\mathrm{\prime }}{V}^{-1}R{)}^{-1}{F}_d^{\mathrm{\prime }},\\ Cov({\widehat{\mathrm{\text{β}}}}_{lfme},y)& ={\sigma }^2{F}_d(X^{\mathrm{\prime }}{H}^{-1}X+R^{\mathrm{\prime }}{V}^{-1}R{)}^{-1}{X}^{\mathrm{\prime }},\\ bias\left({\widehat{\mathrm{\text{β}}}}_{lfme}\right)& =\mathbb{(}{F}_d-I_p)\mathrm{\text{β}},\\ Cov({\widehat{\mathrm{\text{β}}}}_{srle},u)& ={\sigma }^2{F}_d(X^{\mathrm{\prime }}{H}^{-1}X+R^{\mathrm{\prime }}{V}^{-1}R{)}^{-1}{X}^{\mathrm{\prime }}{H}^{-1}ZG\end{array}$$

and $Cov(y,u)={\sigma }^{2}ZG$. Then, Equations (14), (15), (16) and (17) are put in Equation (14) to obtain

$$MSE\left({\hat{\mu }}_{lfmp}\right)=\mathbb{Q}MSE\left({\widehat{\mathrm{\text{β}}}}_{lfme}\right){\mathbb{Q}}^{\mathrm{\prime }}+{\sigma }^2{M}^{\mathrm{\prime }}(G-G{Z}^{\mathrm{\prime }}{H}^{-1}ZG)M$$ (19)

where

$$\begin{array}{rl}MSE\left({\widehat{\mathrm{\text{β}}}}_{lfme}\right)& =Var\left({\widehat{\mathrm{\text{β}}}}_{lfme}\right)+\left[bias\right({\widehat{\mathrm{\text{β}}}}_{lfme}\left)\right]\left[bias\right({\widehat{\mathrm{\text{β}}}}_{lfme}){]}^{\mathrm{\prime }}\\ & ={\sigma }^2{F}_d(X^{\mathrm{\prime }}{H}^{-1}X+R^{\mathrm{\prime }}{V}^{-1}R{)}^{-1}{F}_d^{\mathrm{\prime }}+\mathbb{(}{F}_d-I_p)\mathrm{\text{β}}{\mathrm{\text{β}}}^{\mathrm{\prime }}\mathbb{(}{F}_d-I_p{)}^{\mathrm{\prime }}.\end{array}$$ (20)

In the same manner, we obtain

$$\begin{array}{rl}MSE\left({\hat{\mu }}_d\right)& =\mathbb{Q}MSE\left({\widehat{\mathrm{\text{β}}}}_d\right){\mathbb{Q}}^{\mathrm{\prime }}+{\sigma }^2{M}^{\mathrm{\prime }}(G-G{Z}^{\mathrm{\prime }}{H}^{-1}ZG)M\end{array}$$ (21)

$$\begin{array}{rl}MSE\left({\hat{\mu }}_{mp}\right)& =\mathbb{Q}MSE\left({\widehat{\mathrm{\text{β}}}}_{me}\right){\mathbb{Q}}^{\mathrm{\prime }}+{\sigma }^2{M}^{\mathrm{\prime }}(G-G{Z}^{\mathrm{\prime }}{H}^{-1}ZG)M\end{array}$$ (22)

where

$$\begin{array}{rl}MSE\left({\widehat{\mathrm{\text{β}}}}_d\right)& ={\sigma }^2{F}_d(X^{\mathrm{\prime }}{H}^{-1}X{)}^{-1}{F}_d^{\mathrm{\prime }}+\mathbb{(}{F}_d-I_p)\mathrm{\text{β}}{\mathrm{\text{β}}}^{\mathrm{\prime }}\mathbb{(}{F}_d-I_p{)}^{\mathrm{\prime }},\end{array}$$ (23)

$$\begin{array}{rl}MSE\left({\widehat{\mathrm{\text{β}}}}_{me}\right)& ={\sigma }^2(X^{\mathrm{\prime }}{H}^{-1}X+R^{\mathrm{\prime }}{V}^{-1}R{)}^{-1}.\end{array}$$ (24)

are given, respectively, by Özkale and Kuran [21] and Kuran and Özkale [12]. When we examine Equations (19), (21) and (22), we can say that the superiority of $MSE\left({\hat{\mu }}_{lfmp}\right)$ over $MSE\left({\hat{\mu }}_d\right)$ and $MSE\left({\hat{\mu }}_{mp}\right)$ is equivalent to the superiority of $MSE\left({\widehat{\mathrm{\text{β}}}}_{lfme}\right)$ over $MSE\left({\widehat{\mathrm{\text{β}}}}_d\right)$ and $MSE\left({\widehat{\mathrm{\text{β}}}}_{me}\right)$, respectively.

3.1. The SRLE vs the Liu estimator

Theorem 3.1

The LFME is always superior to the Liu estimator in the MSE matrix criterion.

Proof.

By using Equations (20) and (23), we can write

$$MSE\left({\widehat{\mathrm{\text{β}}}}_d\right)-MSE\left({\widehat{\mathrm{\text{β}}}}_{lfme}\right)={\sigma }^2{F}_d\left(\right(X^{\mathrm{\prime }}{H}^{-1}X{)}^{-1}-(X^{\mathrm{\prime }}{H}^{-1}X+R^{\mathrm{\prime }}{V}^{-1}R{)}^{-1})F_d^{\mathrm{\prime }}.$$

Note that,

$$\left(\right(X^{\mathrm{\prime }}{H}^{-1}X+R^{\mathrm{\prime }}{V}^{-1}R{)}^{-1}{)}^{-1}-\left(X^{\mathrm{\prime }}{H}^{-1}X\right)=R^{\mathrm{\prime }}{V}^{-1}R.$$

Following Theorem A.1 in Appendix 1, we say that since $R^{\mathrm{\prime }}{V}^{-1}R$ is nonnegative definite (nnd), $\left(\right(X^{\mathrm{\prime }}{H}^{-1}X+R^{\mathrm{\prime }}{V}^{-1}R{)}^{-1}{)}^{-1}-\left(X^{\mathrm{\prime }}{H}^{-1}X\right)$ is also nnd. Hence $(X^{\mathrm{\prime }}{H}^{-1}X{)}^{-1}-(X^{\mathrm{\prime }}{H}^{-1}X+R^{\mathrm{\prime }}{V}^{-1}R{)}^{-1}$ is nnd and that impiles the difference $MSE\left({\widehat{\mathrm{\text{β}}}}_d\right)-MSE\left({\widehat{\mathrm{\text{β}}}}_{lfme}\right)$ is a positive semidefinite (psd) matrix. Then, the proof is completed.

3.2. The SRLE vs the mixed estimator

Theorem 3.2

The LFME is superior to the mixed estimator for fixed 0<d<1 in the MSE matrix sense if and only if ${\mathrm{\text{β}}}^{\mathrm{\prime }}{E}^{-1}\mathrm{\text{β}}\le \frac{{\sigma }^2}{1-d}$  is satisfied where $E=X^{\mathrm{\prime }}{H}^{-1}X(X^{\mathrm{\prime }}{H}^{-1}X+R^{\mathrm{\prime }}{V}^{-1}R{)}^{-1}+(X^{\mathrm{\prime }}{H}^{-1}X+R^{\mathrm{\prime }}{V}^{-1}R{)}^{-1}{X}^{\mathrm{\prime }}{H}^{-1}X+(1+d)(X^{\mathrm{\prime }}{H}^{-1}X+R^{\mathrm{\prime }}{V}^{-1}R{)}^{-1}$.

Proof.

By using Equations (20) and (24), we can write

$$MSE\left({\widehat{\mathrm{\text{β}}}}_{me}\right)-MSE\left({\widehat{\mathrm{\text{β}}}}_{lfme}\right)={\sigma }^{2}C-\mathbb{(}{F}_d-I_p)\mathrm{\text{β}}{\mathrm{\text{β}}}^{\mathrm{\prime }}\mathbb{(}{F}_d-I_p{)}^{\mathrm{\prime }}$$

where $C=Var\left({\widehat{\mathrm{\text{β}}}}_{me}\right)-Var\left({\widehat{\mathrm{\text{β}}}}_{lfme}\right)=(X^{\mathrm{\prime }}{H}^{-1}X+R^{\mathrm{\prime }}{V}^{-1}R{)}^{-1}-F_d(X^{\mathrm{\prime }}{H}^{-1}X+R^{\mathrm{\prime }}{V}^{-1}R{)}^{-1}{F}_d^{\mathrm{\prime }}$. By utilizing $F_d$, we write the difference C as

$$\begin{array}{rl}C& =(X^{\mathrm{\prime }}{H}^{-1}X+I{)}^{-1}[(X^{\mathrm{\prime }}{H}^{-1}X+I)(X^{\mathrm{\prime }}{H}^{-1}X+R^{\mathrm{\prime }}{V}^{-1}R{)}^{-1}(X^{\mathrm{\prime }}{H}^{-1}X+I)\\ & -(X^{\mathrm{\prime }}{H}^{-1}X+dI)(X^{\mathrm{\prime }}{H}^{-1}X+R^{\mathrm{\prime }}{V}^{-1}R{)}^{-1}(X^{\mathrm{\prime }}{H}^{-1}X+dI\left)\right](X^{\mathrm{\prime }}{H}^{-1}X+I{)}^{-1}\\ & =(X^{\mathrm{\prime }}{H}^{-1}X+I{)}^{-1}[(1-d)X^{\mathrm{\prime }}{H}^{-1}X(X^{\mathrm{\prime }}{H}^{-1}X+R^{\mathrm{\prime }}{V}^{-1}R{)}^{-1}\\ & +(1-d)(X^{\mathrm{\prime }}{H}^{-1}X+R^{\mathrm{\prime }}{V}^{-1}R{)}^{-1}{X}^{\mathrm{\prime }}{H}^{-1}X\\ & +(1-d^2)(X^{\mathrm{\prime }}{H}^{-1}X+R^{\mathrm{\prime }}{V}^{-1}R{)}^{-1}](X^{\mathrm{\prime }}{H}^{-1}X+I{)}^{-1}\end{array}$$ (25)

Equation (25) shows us that the matrix C is pd for any 0<d<1. Then, from the theorem of Farebrother [4], $MSE\left({\widehat{\mathrm{\text{β}}}}_{me}\right)-MSE\left({\widehat{\mathrm{\text{β}}}}_{lfme}\right)$ is nnd if and only if ${\mathrm{\text{β}}}^{\mathrm{\prime }}\mathbb{(}{F}_d-I_p{)}^{\mathrm{\prime }}{C}^{-1}\mathbb{(}{F}_d-I_p)\mathrm{\text{β}}\le {\sigma }^2$. Since $F_d-I_p=(d-1)(X^{\mathrm{\prime }}{H}^{-1}X+I{)}^{-1}$, we obtain the necessary and sufficient condition as

$$\begin{array}{l}(1-d){\mathrm{\text{β}}}^{\mathrm{\prime }}\left[X^{\mathrm{\prime }}{H}^{-1}X\right(X^{\mathrm{\prime }}{H}^{-1}X+R^{\mathrm{\prime }}{V}^{-1}R{)}^{-1}+(X^{\mathrm{\prime }}{H}^{-1}X+R^{\mathrm{\prime }}{V}^{-1}R{)}^{-1}{X}^{\mathrm{\prime }}{H}^{-1}X\\ +(1+d)(X^{\mathrm{\prime }}{H}^{-1}X+R^{\mathrm{\prime }}{V}^{-1}R{)}^{-1}{]}^{-1}\mathrm{\text{β}}\le {\sigma }^2\end{array}$$

which completes the proof. Here d can be selected anyway such as suggested by Özkale and Kuran [21].

4. Numerical examples

4.1. A hypothetical data analysis

In this analysis, we use the results of Los Alamos study of high efficiency particulate air (HEPA) filter cartridges presented by Kerschner et al. [11]. Such cartridges are used with commercial respirators to provide respiratory protection against dusts, toxic fumes, mists, radionuclides and other particulate matter.

The primary objective of the study was to determine whether the current standard aerosol used to test these filters could be replaced by any of alternate aerosols for quality-assurance testing of HEPA respirator cartridges. HEPA respirator filters fail a quality-assurance test if the challenge penetration is greater than some percent penetration breakpoint otherwise, it is considered to have passed. A secondary objective was to identify those factors that contribute most to the variability in the penetrations of the filters.

In a subset of the aerosol data set [2], two aerosols were crossed with two filter manufacturers. Within each manufacturer, three filters were used to evaluate the penetration of the two aerosols, so that there were six filters in total. By taking a filter nested within the two manufacturers, the following linear mixed model with independent random effects and independent random errors is given as

$$y_{ijhl}=\mu +A_i+M_j+F_{jh}+{\epsilon }_{ijhl}$$ (26)

where $y_{ijhl}$ is the percent penetration, μ is an intercept, $A_i$ is a fixed effect for the ith aerosol type $(i=1,2)$, $M_j$ is a fixed effect for the jth filter manufacturer $(j=1,2)$, $F_{jh}\sim N(0,{\gamma }_{F_{jh}})$ is a random effect for the hth filter nested within the the jth manufacturer $(h=1,2,3)$ and ${\epsilon }_{ijhl}\sim N(0,{\rho }_{{\epsilon }_{ijhl}})$ is the error associated with the lth replication in the ijhlth cell $(l=1,2,3)$, subject to the usual restrictions: $\sum _i{A}_i=\sum _j{M}_j=0$.

Clearly, model (26) may be put into the form of model (1) with the vector of responses and approximate design matrices

$$\begin{array}{rl}& y_{36\times 1}={\left[\begin{array}{ccccc}0.750& 0.770& \dots & 1.520& 1.580\end{array}\right]}^{\mathrm{\prime }}\end{array}$$ (27)

$$\begin{array}{rl}& X_{36\times 3}^{\ast }=\left[\begin{array}{ccc}{1}_9& 1_9& -1_9\\ 1_9& 1_9& 1_9\\ 1_9& -1_9& -1_9\\ 1_9& -1_9& 1_9\end{array}\right],Z_{36\times 6}=\left[\begin{array}{cccccc}{1}_3& 0_3& 0_6& 0_9& 0_{12}& 0_7\\ 0_7& 1_3& 1_3& 1_3& 1_3& 0_8\\ 0_8& 0_{15}& 0_{15}& 0_{15}& 0_{15}& 1_3\\ 1_3& 1_3& 1_3& 1_3& 1_3& 0_{15}\\ 0_{15}& 0_{12}& 0_9& 0_6& 0_3& 1_3\end{array}\right].\end{array}$$ (28)

Here, $1_9^{\mathrm{\prime }}=(1,1,1,1,1,1,1,1,1)$, $0_3^{\mathrm{\prime }}=(0,0,0)$ and $1_3$, $0_6$, $0_7$, $0_8$, $0_9$, $0_{12}$, $0_{15}$ are vectors of ones and zeros where subscripts show the dimensions. y in (27) is a vector of the percent penetration values, ${\mathrm{\text{β}}}^{\mathrm{\prime }}=(\mu ,A_1,M_1)=(2,1.5,9)$ is an arbitrary vector and $u^{\mathrm{\prime }}=(F_{11},F_{12},F_{13},F_{21},F_{22},F_{23})$ with $u\sim N(0,{\sigma }^{2}G({\gamma }_{F_{jh}})={\sigma }^{2}G)$, $\epsilon \sim N(0,{\sigma }^{2}W({\rho }_{{\epsilon }_{ijhl}})={\sigma }^{2}W)$.

It was seen that multicollinearity did not appear at the aerosol data examined by Beckman et al. [2] and Kerschner et al. [11]. But, to serve for our purposes, the data must have multicollinearity. Thus, by using this aerosol data and the model in (26), hypothetical data are produced as follows.

Since multicollinearity must be at the X matrix corresponding to fixed effects as explained in theory, the data corresponding to the fixed effects are generated using the following equation in McDonald and Galarneau [15]

$$x_{ij}={\left(1-{\delta }^2\right)}^{\left(1/2\right)}{x}_{ij}^{\ast }+\delta x_{i4}^{\ast },i=1,\dots ,36,j=1,2,3$$

where $x_{ij}^{\ast }$ are taken from (27) and δ is specified so that correlation between any two fixed effects is given by ${\delta }^2$. These effects are then standardized so that the fixed effects design matrix is in correlation form and for this hypothetical data, ${\delta }^2=0.99$ is considered.

The stochastic linear restrictions are taken as

$$r=R\mathrm{\text{β}}+\mathrm{\Phi },R=\left[\begin{array}{ccc}1.0950& 1.0950& 1.0950\end{array}\right],\mathrm{\Phi }\sim \left(0,{\sigma }^{2}V\left(\upsilon \right)={\sigma }^{2}V\right)$$

where r = 5.000 is the 14th element of y vector in Equation (28), R is the 14th row of generated X matrix and $\mathrm{\text{β}}={\left[\begin{array}{ccc}2& 1.5& 9\end{array}\right]}^{{}^{\mathrm{\prime }}}$ is an arbitrary vector. Because of having the largest percent penetration value, the 14th element of y vector is chosen. But, the other elements of y vector and corresponding elements of the X matrix can also be taken as r and R, respectively.

Generally, since there is not too much difference between the results of ML and REML covariance estimates (see [19]), we choose REML approach in our hypothetical data analysis and we obtain ${\hat{G}}_{REML}=0.2534\times I_7$, ${\hat{W}}_{REML}=9.9353\times I_{37}$ and ${\hat{V}}_{REML}=0.6547\times I_1$. Then, from the formula $H=ZG{Z}^{\mathrm{\prime }}+W$, ${\hat{H}}_{REML}$ is obtained.

The eigenvalues of the matrix $X^{\mathrm{\prime }}{\hat{H}}_{REML}^{-1}X$ are obtained as ${\lambda }_1=11.8812$, ${\lambda }_2=0.0084$ and ${\lambda }_3=0.0258$. Then, the condition number, ${\lambda }_{max}/{\lambda }_{min}$, where ${\lambda }_{min}$ and ${\lambda }_{max}$ indicate the minimum and maximum eigenvalues of $X^{\mathrm{\prime }}{\hat{H}}_{REML}^{-1}X$ is calculated as ${\lambda }_{max}/{\lambda }_{min}=1.4154e+03$. Since the condition number is used to measure the extent of multicollinearity in the data and it is larger than 1000, it shows severe multicollinearity (see for example Montgomery et al. [16], p.298).

The Liu biasing parameter d is computed as $\hat{d}=1-\frac{\sum {\hat{\sigma }}^2/{\lambda }_i({\lambda }_i+1)}{\sum {\hat{\alpha }}_i^2/({\lambda }_i+1{)}^2}=0.9596$ which is defined by Özkale and Kuran [21] where $\hat{\alpha }=P^{\mathrm{\prime }}\widehat{\mathrm{\text{β}}}$ and P is an orthogonal matrix such that $P^{\mathrm{\prime }}\left(X^{\mathrm{\prime }}{H}^{-1}X\right)P=\mathrm{\Delta }$ with a diagonal matrix $\mathrm{\Delta }=diag({\lambda }_1,\dots ,{\lambda }_p)$.

Table 1 presents the parameter estimates and scalar MSE values (trace of matrix MSE) when the variance of y is estimated by REML. From Table 1, we observe that LFME outperforms the other estimators which is followed by the mixed estimator.

Table 1. Parameter estimates and MSE scalar values for $\hat{d}=0.9596$ when the covariance parameters are estimated by REML.

	$\widehat{\mathrm{\text{β}}}$	$\hat{u}$	${\widehat{\mathrm{\text{β}}}}_{me}$	${\hat{u}}_{mp}$	${\widehat{\mathrm{\text{β}}}}_{\hat{d}}$	${\hat{u}}_{\hat{d}}$	${\widehat{\mathrm{\text{β}}}}_{lfme}$	${\hat{u}}_{lfmp}$
μ	10.8672	–	20.0638	–	10.4350	–	19.2684	–
$A_1$	$-15.9017$	–	$-28.3389$	–	$-15.2609$	–	$-27.1943$	–
$M_1$	6.9127	–	11.8321	–	6.6414	–	11.3702	–
$F_{intercept}$	–	$-0.0000$	–	$-0.4277$	–	0.0198	–	$-0.3912$
$F_{11}$	–	0.0736	–	0.0736	–	0.0731	–	0.0732
$F_{12}$	–	$-0.0349$	–	$-0.0349$	–	$-0.0354$	–	$-0.0354$
$F_{13}$	–	$-0.0387$	–	$-0.0387$	–	$-0.0392$	–	$-0.0391$
$F_{21}$	–	$-0.1289$	–	$-0.2595$	–	$-0.1222$	–	$-0.2477$
$F_{22}$	–	0.0516	–	$-0.0790$	–	0.0583	–	$-0.0672$
$F_{23}$	–	0.0023	–	$-0.1283$	–	0.0090	–	$-0.1165$
scalarMSE	16.8331	–	11.1831	–	15.5766	–	10.3698	–

To consider the superiorities of the estimators for other d values, plot of the scalar MSE values of $\widehat{\mathrm{\text{β}}}$, ${\widehat{\mathrm{\text{β}}}}_{me}$, ${\widehat{\mathrm{\text{β}}}}_d$, ${\widehat{\mathrm{\text{β}}}}_{lfme}$ versus d are presented by Figure 1. Figure 1 demonstrates that although ${\widehat{\mathrm{\text{β}}}}_{me}$ is better than $\widehat{\mathrm{\text{β}}}$, ${\widehat{\mathrm{\text{β}}}}_d$ and ${\widehat{\mathrm{\text{β}}}}_{lfme}$ are better than $\widehat{\mathrm{\text{β}}}$ for d values respectively in intervals 0.342<d<1 and 0.362<d<1. And, we can also say that though ${\widehat{\mathrm{\text{β}}}}_{lfme}$ is better than ${\widehat{\mathrm{\text{β}}}}_{me}$ for d values in interval 0.524<d<1, ${\widehat{\mathrm{\text{β}}}}_{me}$ is better than ${\widehat{\mathrm{\text{β}}}}_d$ for d values in interval 0<d<1. For d values in interval 0<d<1, ${\widehat{\mathrm{\text{β}}}}_{lfme}$ has smaller MSE scalar values than ${\widehat{\mathrm{\text{β}}}}_d$ and it can be seen that as d values increase, the difference between MSE scalar values of ${\widehat{\mathrm{\text{β}}}}_{lfme}$ and ${\widehat{\mathrm{\text{β}}}}_d$ increases. ${\widehat{\mathrm{\text{β}}}}_{lfme}$ is the best among the estimators for d values in the interval 0.524<d<1. Therefore, we say that there always exists a d value such that ${\widehat{\mathrm{\text{β}}}}_{lfme}$ is better than $\widehat{\mathrm{\text{β}}}$, ${\widehat{\mathrm{\text{β}}}}_{me}$ or ${\widehat{\mathrm{\text{β}}}}_d$.

Figure 1.

MSE scalar values of BLUE, mixed, Liu and LFME versus d.

The theorems in Section 3 are also examined. And so, under the matrix MSE criterion, ${\widehat{\mathrm{\text{β}}}}_{lfme}$ is always better than ${\widehat{\mathrm{\text{β}}}}_{\hat{d}}$. The condition given by Theorem 3.2 is computed as ${\widehat{\mathrm{\text{β}}}}^{\mathrm{\prime }}{E}^{-1}\widehat{\mathrm{\text{β}}}=10.8241$ which is not smaller than ${\hat{\sigma }}^2/(1-\hat{d})=2.6387$, hence ${\widehat{\mathrm{\text{β}}}}_{lfme}$ is not better than ${\widehat{\mathrm{\text{β}}}}_{me}$ for $\hat{d}=0.9596$ under the matrix MSE criterion though it is better than ${\widehat{\mathrm{\text{β}}}}_{me}$ under the scalar MSE criterion. However, there can be found a d value such that ${\widehat{\mathrm{\text{β}}}}_{lfme}$ dominates ${\widehat{\mathrm{\text{β}}}}_{me}$ in terms of the matrix MSE criterion; these d values for this data set are found to be greater than 0.9898.

The hypothetical data analysis summarizes that the SRLE dominates the BLUE, mixed and Liu estimators in the sense of the scalar MSE criterion for appropriate selected d value under multicollinearity.

4.2. Real data analysis: greenhouse gases data

Greenhouse gases trap heat and make the planet warmer. Human activities are responsible for almost all of the increase in greenhouse gases in the atmosphere over the past 150 years. The largest source of greenhouse gas emissions from human activities in the United States is from burning fossil fuels for electricity, heat, and transportation (see the official United States Environmental Protection Agency (EPA) website [3]). The transportation sector generates the largest share of greenhouse gas emissions in the United States (nearly 28.5 % of 2016 greenhouse gas emissions). Greenhouse gas emissions from transportation come from burning fossil fuel for cars, light/heavy trucks-buses, motorcycles and railways. The remaining greenhouse gas emissions from the transportation sector come from other modes of transportation, including commercial aircraft, ships and boats (see EPA website [3]).

In this analysis, we use data on 297 fuel combustion in transport (in million tonnes) randomly selected from 27 areas (Belgium, Bulgaria, European Union, Denmark, Germany, Estonia, Greece, Spain, France, Croatia, Italy, Ireland, Latvia, Lithuania, Luxembourg, Hungary, Netherlands, Austria, Poland, Portugal, Romania, Slovenia, Slovakia, Switzerland, Sweden, United Kingdom, Norway) which were regularly obtained during the period 2006–2016 inclusive.

To determine fuel combustion in transport (in million tonnes), we get repeated measurements from the list of fuel combustions, namely the cars, the light duty trucks, the heavy duty trucks-buses, the motorcycles and railways. The data are available from the official Eurostat website [3].

We define the response (y) as fuel combustion in transport (in million tonnes) and we get the explanatory variables as fuel combustions in cars ($x_1$), light duty trucks ($x_2$), heavy duty trucks-buses ($x_3$), motorcycles ($x_4$) and railways ($x_5$). That is, we can say that fuel combustions in cars, light duty trucks, heavy duty trucks-buses, motorcycles, railways are expressed as fixed effects and since the 27 areas are randomly selected from the areas, the areas factor effect on the response is expressed as random effect. Then, the random intercept and slope model (RISM) is given as

$$\begin{array}{rl}& y_{ij}={\mathrm{\text{β}}}_1{x}_{ij1}+{\mathrm{\text{β}}}_2{x}_{ij2}+{\mathrm{\text{β}}}_3{x}_{ij3}+{\mathrm{\text{β}}}_4{x}_{ij4}+{\mathrm{\text{β}}}_5{x}_{ij5}+u_1+u_2{t}_{ij}+{\epsilon }_{ij},\\ & i=1,\dots ,27,j=1,\dots ,11\end{array}$$

where $y_{ij}$ denotes the ith observation of the jth area of the response, $x_{ijs}$ denotes the ith observation of the jth area of the explanatory variable $x_s$, $s=1,\dots ,5$ and $t_{ij}$ shows time corresponding to $y_{ij}$.

The stochastic linear restriction is taken as

$$\begin{array}{rl}& r=R\mathrm{\text{β}}+\mathrm{\Phi },R=\left[\begin{array}{ccccc}558.69609& 107.73599& 242.23818& 10.86114& 8.14196\end{array}\right]\\ & \mathrm{\Phi }\sim \left(0,{\sigma }^{2}V\left(\upsilon \right)={\sigma }^{2}V\right)\end{array}$$

where r = 983.37034 is the $y_{31}$th observation, R is the vector of $x_{31s}$th observations for $s=1,\dots ,5$ and $\mathrm{\text{β}}={\left[\begin{array}{ccccc}1& 1& 1& 1& 1\end{array}\right]}^{{}^{\mathrm{\prime }}}$ is an arbitrary vector. Because of having the largest cereal production value, the $y_{31}$th observation is chosen. But, the other observations of $y_{ij}$ vector and corresponding elements of the $x_{ijs}$ observations can also be taken as r and R, respectively.

Generally, since there is not too much difference between the results of ML and REML covariance estimates (see [19]), we prefer ML approach in our real data analysis and we get ${\hat{G}}_{ML}=0.022027\times I_2$, ${\hat{W}}_{ML}=4.146253\times I_{298}$ and ${\hat{V}}_{ML}=1.155446\times I_1$. Then, from the formula $H=ZG{Z}^{\mathrm{\prime }}+W$, ${\hat{H}}_{ML}$ is obtained.

The eigenvalues of the matrix $X^{\mathrm{\prime }}{\hat{H}}_{ML}^{-1}X$ are obtained as   ${\lambda }_1=$ $9.040303e+05$, ${\lambda }_2=$ 9.260924e + 02, ${\lambda }_3=$ $2.903158e+02$, ${\lambda }_4=$ 2.571984 and ${\lambda }_5=$ 15.225476. Then, the condition number is calculated as ${\lambda }_{max}/{\lambda }_{min}=351490>1000$ and it shows severe multicollinearity (see for example Montgomery et al. [16], p.298).

The Liu biasing parameter is computed from Özkale and Kuran [21] as $\hat{d}=0.9812$. Table 2 gives the parameter estimates and scalar MSE values when the variance of y is estimated by ML. From Table 2, we observe that LFME outperforms the mixed estimator, the Liu estimator and the BLUE in the sense of scalar MSE.

Table 2. Parameter estimates and MSE scalar values for $\hat{d}=0.9812$ when the covariance parameters are estimated by ML.

	$\widehat{\mathrm{\text{β}}}$	$\hat{u}$	${\widehat{\mathrm{\text{β}}}}_{me}$	${\hat{u}}_{mp}$	${\widehat{\mathrm{\text{β}}}}_{\hat{d}}$	${\hat{u}}_{\hat{d}}$	${\widehat{\mathrm{\text{β}}}}_{lfme}$	${\hat{u}}_{lfmp}$
${\mathrm{\text{β}}}_1$	1.0204	–	1.0046	–	1.0212	–	1.0055	–
${\mathrm{\text{β}}}_2$	1.0507	–	1.0406	–	1.0511	–	1.0410	–
${\mathrm{\text{β}}}_3$	0.9380	–	0.9637	–	0.9374	–	0.9630	–
${\mathrm{\text{β}}}_4$	3.3953	–	3.5860	–	3.3794	–	3.5688	–
${\mathrm{\text{β}}}_5$	3.7790	–	4.1731	–	3.7579	–	4.1502	–
$u_1$	–	0.1329	–	0.1014	–	0.1332	–	0.1017
$u_2$	–	$-0.0252$	–	$-0.0323$	–	$-0.0252$	–	$-0.0324$
scalarMSE	0.1511	–	0.1472	–	0.1497	–	0.1459	–

To consider the superiorities of the estimators for other d values, plot of the scalar MSE values of $\widehat{\mathrm{\text{β}}}$, ${\widehat{\mathrm{\text{β}}}}_{me}$, ${\widehat{\mathrm{\text{β}}}}_d$, ${\widehat{\mathrm{\text{β}}}}_{lfme}$ versus d are presented by Figure 2. Figure 2 demonstrates that although ${\widehat{\mathrm{\text{β}}}}_{me}$ is better than $\widehat{\mathrm{\text{β}}}$ for d values in interval 0<d<1, ${\widehat{\mathrm{\text{β}}}}_d$ and ${\widehat{\mathrm{\text{β}}}}_{lfme}$ are better than $\widehat{\mathrm{\text{β}}}$ for d values respectively in intervals 0.508<d<1 and 0.482<d<1. And, we can also say that though ${\widehat{\mathrm{\text{β}}}}_{lfme}$ is better than ${\widehat{\mathrm{\text{β}}}}_{me}$ for d values in interval 0.535<d<1, ${\widehat{\mathrm{\text{β}}}}_{me}$ is better than ${\widehat{\mathrm{\text{β}}}}_d$ for d values in interval 0.563<d<1. For d values in interval 0<d<1, ${\widehat{\mathrm{\text{β}}}}_{lfme}$ has smaller scalar MSE values than ${\widehat{\mathrm{\text{β}}}}_d$ and it can be seen that as d values increase, the difference between scalar MSE values of ${\widehat{\mathrm{\text{β}}}}_{lfme}$ and ${\widehat{\mathrm{\text{β}}}}_d$ increases. As compared to Figure 1, we see that the tendencies of the estimators towards each other are same.

Figure 2.

MSE scalar values of BLUE, mixed, Liu and LFME versus d.

In examining Theorem 3.1, we see that ${\widehat{\mathrm{\text{β}}}}_{lfme}$ is always better than ${\widehat{\mathrm{\text{β}}}}_{\hat{d}}$ in the sense of matrix MSE. The condition given by Theorem 3.2 is computed as ${\widehat{\mathrm{\text{β}}}}^{\mathrm{\prime }}{E}^{-1}\widehat{\mathrm{\text{β}}}=1.0240$ which is smaller than ${\hat{\sigma }}^2/(1-\hat{d})=17.5236$, hence ${\widehat{\mathrm{\text{β}}}}_{lfme}$ is better than ${\widehat{\mathrm{\text{β}}}}_{me}$ under the matrix MSE matrix criterion.

Under multicollinearity, this real data analysis summarizes that the LFME dominates the BLUE, mixed and Liu estimators in the sense of both the scalar MSE and matrix MSE criteria for appropriate selected d value.

5. A simulation study

In this section, we will discuss the simulation study to compare the performance of$\widehat{\mathrm{\text{β}}}$, ${\widehat{\mathrm{\text{β}}}}_{me}$, ${\widehat{\mathrm{\text{β}}}}_d$ and ${\widehat{\mathrm{\text{β}}}}_{lfme}$ and the performance of $\hat{u}$, ${\hat{u}}_{me}$, ${\hat{u}}_d$ and ${\hat{u}}_{lfmp}$ in the sense of the estimated mean square error (EMSE) and the predicted mean square error (PMSE).

The fixed effects are computed from McDonald and Galarneau [15] as

$$x_{ijk}=(1-{\gamma }^2{)}^{1/2}{w}_{ijk}+\gamma w_{ijp+1},i=1,\dots ,m,j=1,\dots ,n_i,k=1,\dots ,p$$

where $w_{ijk}$ are independent standard normal pseudo-random numbers and γ is specified so that the correlation between any two fixed effects is given by ${\gamma }^2$. Three different sets of correlations are considered corresponding to ${\gamma }^2=0.90$, 0.95 and 0.99. The number of fixed effects are chosen as p = 2 and p = 4. To fix the generated explanatory variables, 2019 is used as the state number.

We consider m = 30, 60 subjects and $n_i=5$, 10 observations per subject. The stochastic linear restrictions are taken as

$$r=R\mathrm{\text{β}}+\mathrm{\Phi },\mathrm{\Phi }\sim \left(0,{\sigma }^{2}V\left(\upsilon \right)={\sigma }^{2}V\right)$$

where $r=\left[\begin{array}{c}0.4\\ 0.3\end{array}\right]$ and $R=\left[\begin{array}{cc}1& 0.5\\ 0.5& 1\end{array}\right]$ are chosen arbitrary. The parameter vector $\mathrm{\text{β}}=({\mathrm{\text{β}}}_1,\dots ,{\mathrm{\text{β}}}_p{)}^{\mathrm{\prime }}$ is chosen as the normalized eigenvector corresponding to the largest eigenvalue of $X^{\mathrm{\prime }}{H}^{-1}X$ so that ${\mathrm{\text{β}}}^{\mathrm{\prime }}\mathrm{\text{β}}=1$ (see [17]). Then, the underlying model takes the following form with q = 2 random effects

$$\begin{array}{rl}& y_{ij}={\mathrm{\text{β}}}_1{x}_{ij1}+{\mathrm{\text{β}}}_2{x}_{ij2}+u_1+u_2{t}_{ij}+{\epsilon }_{ij},\\ & u_i\stackrel{iid}{\sim }N(0,{\sigma }^{2}D),{\epsilon }_{ij}\stackrel{iid}{\sim }N(0,{\sigma }^2{I}_{n_i})\end{array}$$

where D = V are both taken as $\left[\begin{array}{cc}1& \rho \\ \rho & 1\end{array}\right]$ is the AR(1) process with $\rho =0.90$ and $t_{ij}$ shows time which was taken as the same set of occasions, {$t_{ij}=j$ for $i=1,\dots ,m,j=1,\dots ,n_i$}. Three values of ${\sigma }^2$ are investigated as 0.5, 1 and 10.

By using Özkale and Kuran [21], the Liu biasing parameter is selected as ${\hat{d}}_h$ which is given by Theorem 4.1 where h is determined as multiplying the upper bound defined in Theorem 4.1 by 0.99 if $\sum _{i=1}^{p}1/{\lambda }_i({\lambda }_i+1)>2/\left({\lambda }_p\right({\lambda }_p+1\left)\right)$ and determined arbitrarily as ${\hat{d}}_h=0.8650$ if $\sum _{i=1}^{p}1/{\lambda }_i({\lambda }_i+1)<2/\left({\lambda }_p\right({\lambda }_p+1\left)\right)$. This arbitrarily chosen ${\hat{d}}_h=0.8650$ is determined by inspiring the numerical examples presented in Section 4.

For each choice of m, $n_i$, γ, p and ${\sigma }^2$, the experiment is replicated 500 times by generating response variable and the EMSE for any estimator $\widetilde{\mathrm{\text{β}}}$ of β and the PMSE for any predictor $\tilde{u}$ of u are calculated, respectively, as

$$\begin{array}{rl}EMSE\left(\widetilde{\mathrm{\text{β}}}\right)& =\frac{1}{500}\sum _{\mathrm{r}=1}^{500}({\widetilde{\mathrm{\text{β}}}}_{\left(\mathrm{r}\right)}-\mathrm{\text{β}}{)}^{\mathrm{\prime }}({\widetilde{\mathrm{\text{β}}}}_{\left(\mathrm{r}\right)}-\mathrm{\text{β}})\\ PMSE\left(\tilde{u}\right)& =\frac{1}{500}\sum _{\mathrm{r}=1}^{500}({\tilde{u}}_{\left(\mathrm{r}\right)}-u{)}^{\mathrm{\prime }}({\tilde{u}}_{\left(\mathrm{r}\right)}-u)\end{array}$$

where the subscript $($ŗ$)$ refers to the ŗth replication.

To see the degree of supremacy, after the EMSE and the PMSE results are obtained, the change percentage values are calculated as

$$\begin{array}{rl}& \left(1-\frac{EMSE\left({\widetilde{\mathrm{\text{β}}}}^1\right)}{EMSE\left({\widetilde{\mathrm{\text{β}}}}^2\right)}\right).100\\ & \left(1-\frac{PMSE\left({\tilde{u}}^1\right)}{PMSE\left({\tilde{u}}^2\right)}\right).100\end{array}$$

where ${\widetilde{\mathrm{\text{β}}}}^1$ and ${\widetilde{\mathrm{\text{β}}}}^2$ are any two estimators and ${\tilde{u}}^1$ and ${\tilde{u}}^2$ are any two predictors.

The simulation results are summarized by Tables 3–10. Based on Tables 3–10, we have the following conclusions:

Table 4. Predicted MSE values with p = 2, m = 30 and $n_i=5$.

${\gamma }^2$	${\sigma }^2$	$\hat{u}$	${\hat{u}}_{me}$	${\hat{u}}_{{\hat{d}}_h}$	${\hat{u}}_{lfmp}$	$\hat{u}/{\hat{u}}_{lfmp}$	${\hat{u}}_{me}/{\hat{u}}_{lfmp}$	${\hat{u}}_{{\hat{d}}_h}/{\hat{u}}_{lfmp}$
0.90	$\begin{array}{c}0.5\\ 1\\ 10\end{array}$	$\begin{array}{c}0.012964989\\ 0.021812330\\ 0.269327347\end{array}$	$\begin{array}{c}0.012964029\\ 0.021805162\\ 0.269362811\end{array}$	$\begin{array}{c}0.012965441\\ 0.021814216\\ 0.269370583\end{array}$	$\begin{array}{c}0.012964505\\ 0.021807268\\ 0.269428009\end{array}$	$\begin{array}{c}0.0037\\ 0.0232\\ -0.0373\end{array}$	$\begin{array}{c}-0.0036\\ -0.0096\\ -0.0242\end{array}$	$\begin{array}{c}0.0072\\ 0.0318\\ -0.0213\end{array}$
0.95	$\begin{array}{c}0.5\\ 1\\ 10\end{array}$	$\begin{array}{c}0.012960587\\ 0.021807039\\ 0.269340232\end{array}$	$\begin{array}{c}0.012958417\\ 0.021791976\\ 0.269194164\end{array}$	$\begin{array}{c}0.012961078\\ 0.021808828\\ 0.269363964\end{array}$	$\begin{array}{c}0.012958969\\ 0.021794262\\ 0.269246984\end{array}$	$\begin{array}{c}0.0124\\ 0.0585\\ 0.0346\end{array}$	$\begin{array}{c}-0.0042\\ -0.0104\\ -0.0196\end{array}$	$\begin{array}{c}0.0162\\ 0.0667\\ 0.0434\end{array}$
0.99	$\begin{array}{c}0.5\\ 1\\ 10\end{array}$	$\begin{array}{c}0.012958149\\ 0.021804764\\ 0.269366971\end{array}$	$\begin{array}{c}0.012965361\\ 0.021809979\\ 0.269113379\end{array}$	$\begin{array}{c}0.012959567\\ 0.021809042\\ 0.269366531\end{array}$	$\begin{array}{c}0.012966872\\ 0.021815092\\ 0.269136232\end{array}$	$\begin{array}{c}-0.06732\\ -0.0473\\ 0.0856\end{array}$	$\begin{array}{c}-0.0116\\ -0.0234\\ -0.0084\end{array}$	$\begin{array}{c}-0.0563\\ -0.0277\\ 0.0854\end{array}$

Table 5. Estimated MSE values with p = 2, m = 60 and $n_i=10$.

${\gamma }^2$	${\sigma }^2$	$\widehat{\mathrm{\text{β}}}$	${\widehat{\mathrm{\text{β}}}}_{me}$	${\widehat{\mathrm{\text{β}}}}_{{\hat{d}}_h}$	${\widehat{\mathrm{\text{β}}}}_{lfme}$	$\widehat{\mathrm{\text{β}}}/{\widehat{\mathrm{\text{β}}}}_{lfme}$	${\widehat{\mathrm{\text{β}}}}_{me}/{\widehat{\mathrm{\text{β}}}}_{lfme}$	${\widehat{\mathrm{\text{β}}}}_{{\hat{d}}_h}/{\widehat{\mathrm{\text{β}}}}_{lfme}$
0.90	$\begin{array}{c}0.5\\ 1\\ 10\end{array}$	$\begin{array}{c}0.009385296\\ 0.018769145\\ 0.187545841\end{array}$	$\begin{array}{c}0.009030951\\ 0.017380146\\ 0.097858940\end{array}$	$\begin{array}{c}0.009364604\\ 0.018687026\\ 0.180471159\end{array}$	$\begin{array}{c}0.009011161\\ 0.017304725\\ 0.094349996\end{array}$	$\begin{array}{c}3.9863\\ 7.8022\\ 49.6923\end{array}$	$\begin{array}{c}0.2191\\ 0.4339\\ 3.5857\end{array}$	$\begin{array}{c}3.7742\\ 7.3971\\ 47.7201\end{array}$
0.95	$\begin{array}{c}0.5\\ 1\\ 10\end{array}$	$\begin{array}{c}0.018334197\\ 0.036665830\\ 0.366395276\end{array}$	$\begin{array}{c}0.016944513\\ 0.031391803\\ 0.116000194\end{array}$	$\begin{array}{c}0.018252075\\ 0.036343110\\ 0.341960717\end{array}$	$\begin{array}{c}0.016868842\\ 0.031116905\\ 0.108666493\end{array}$	$\begin{array}{c}7.9924\\ 15.1337\\ 70.3417\end{array}$	$\begin{array}{c}0.4465\\ 0.8756\\ 6.3221\end{array}$	$\begin{array}{c}7.5784\\ 14.3801\\ 68.2225\end{array}$
0.99	$\begin{array}{c}0.5\\ 1\\ 10\end{array}$	$\begin{array}{c}0.090014779\\ 0.180018018\\ 1.798970566\end{array}$	$\begin{array}{c}0.061867067\\ 0.090464268\\ 0.085052218\end{array}$	$\begin{array}{c}0.088099668\\ 0.172955346\\ 1.505693191\end{array}$	$\begin{array}{c}0.060553615\\ 0.086931300\\ 0.072424028\end{array}$	$\begin{array}{c}32.7292\\ 51.7096\\ 95.9741\end{array}$	$\begin{array}{c}2.1230\\ 3.9053\\ 14.8475\end{array}$	$\begin{array}{c}31.2669\\ 49.7377\\ 95.1899\end{array}$

Table 6. Predicted MSE values with p = 2, m = 60 and $n_i=10$.

${\gamma }^2$	${\sigma }^2$	$\hat{u}$	${\hat{u}}_{me}$	${\hat{u}}_{{\hat{d}}_h}$	${\hat{u}}_{lfmp}$	$\hat{u}/{\hat{u}}_{lfmp}$	${\hat{u}}_{me}/{\hat{u}}_{lfmp}$	${\hat{u}}_{{\hat{d}}_h}/{\hat{u}}_{lfmp}$
0.90	$\begin{array}{c}0.5\\ 1\\ 10\end{array}$	$\begin{array}{c}0.002761527\\ 0.005265338\\ 0.042150581\end{array}$	$\begin{array}{c}0.002760935\\ 0.005261510\\ 0.041556416\end{array}$	$\begin{array}{c}0.002761485\\ 0.005265072\\ 0.042107279\end{array}$	$\begin{array}{c}0.002760898\\ 0.005261263\\ 0.041516515\end{array}$	$\begin{array}{c}0.0227\\ 0.0773\\ 1.5042\end{array}$	$\begin{array}{c}0.0013\\ 0.0046\\ 0.0960\end{array}$	$\begin{array}{c}0.0212\\ 0.0723\\ 1.4029\end{array}$
0.95	$\begin{array}{c}0.5\\ 1\\ 10\end{array}$	$\begin{array}{c}0.002761602\\ 0.005265418\\ 0.042152520\end{array}$	$\begin{array}{c}0.002760861\\ 0.005260939\\ 0.041539116\end{array}$	$\begin{array}{c}0.002761553\\ 0.005265126\\ 0.042108122\end{array}$	$\begin{array}{c}0.002760818\\ 0.005260674\\ 0.041500225\end{array}$	$\begin{array}{c}0.0283\\ 0.0900\\ 1.5474\end{array}$	$\begin{array}{c}0.0015\\ 0.0050\\ 0.0936\end{array}$	$\begin{array}{c}0.0266\\ 0.0845\\ 1.4436\end{array}$
0.99	$\begin{array}{c}0.5\\ 1\\ 10\end{array}$	$\begin{array}{c}0.002761417\\ 0.005265025\\ 0.042153788\end{array}$	$\begin{array}{c}0.002759848\\ 0.005257830\\ 0.042309344\end{array}$	$\begin{array}{c}0.002761322\\ 0.005264573\\ 0.042194581\end{array}$	$\begin{array}{c}0.002759776\\ 0.005257512\\ 0.042359572\end{array}$	$\begin{array}{c}0.0594\\ 0.1426\\ -0.4881\end{array}$	$\begin{array}{c}0.0026\\ 0.0060\\ -0.1187\end{array}$	$\begin{array}{c}0.0559\\ 0.1341\\ -0.3910\end{array}$

Table 7. Estimated MSE values with p = 4, m = 30 and $n_i=5$.

${\gamma }^2$	${\sigma }^2$	$\widehat{\mathrm{\text{β}}}$	${\widehat{\mathrm{\text{β}}}}_{me}$	${\widehat{\mathrm{\text{β}}}}_{{\hat{d}}_h}$	${\widehat{\mathrm{\text{β}}}}_{lfme}$	$\widehat{\mathrm{\text{β}}}/{\widehat{\mathrm{\text{β}}}}_{lfme}$	${\widehat{\mathrm{\text{β}}}}_{me}/{\widehat{\mathrm{\text{β}}}}_{lfme}$	${\widehat{\mathrm{\text{β}}}}_{{\hat{d}}_h}/{\widehat{\mathrm{\text{β}}}}_{lfme}$
0.90	$\begin{array}{c}0.5\\ 1\\ 10\end{array}$	$\begin{array}{c}0.103274576\\ 0.206275220\\ 2.053570779\end{array}$	$\begin{array}{c}0.089541375\\ 0.155710224\\ 0.481597125\end{array}$	$\begin{array}{c}0.096547614\\ 0.181207541\\ 1.072799065\end{array}$	$\begin{array}{c}0.083734928\\ 0.136889269\\ 0.258139478\end{array}$	$\begin{array}{c}18.9200\\ 33.6375\\ 87.4297\end{array}$	$\begin{array}{c}6.4846\\ 12.0871\\ 46.3992\end{array}$	$\begin{array}{c}13.2708\\ 24.4571\\ 75.9377\end{array}$
0.95	$\begin{array}{c}0.5\\ 1\\ 10\end{array}$	$\begin{array}{c}0.205620921\\ 0.410698449\\ 4.088803667\end{array}$	$\begin{array}{c}0.157704836\\ 0.257266398\\ 0.564946843\end{array}$	$\begin{array}{c}0.180570168\\ 0.324460555\\ 1.852567988\end{array}$	$\begin{array}{c}0.138575374\\ 0.203606623\\ 0.250745900\end{array}$	$\begin{array}{c}32.6063\\ 50.4243\\ 93.8674\end{array}$	$\begin{array}{c}12.1299\\ 20.8576\\ 55.6160\end{array}$	$\begin{array}{c}23.2567\\ 37.2476\\ 86.4649\end{array}$
0.99	$\begin{array}{c}0.5\\ 1\\ 10\end{array}$	$\begin{array}{c}1.024648630\\ 2.046603141\\ 20.376157839\end{array}$	$\begin{array}{c}0.415690358\\ 0.514295506\\ 0.420561330\end{array}$	$\begin{array}{c}0.668005479\\ 1.152087947\\ 7.927083175\end{array}$	$\begin{array}{c}0.270021440\\ 0.283500506\\ 0.141381774\end{array}$	$\begin{array}{c}73.6474\\ 86.1477\\ 99.3061\end{array}$	$\begin{array}{c}35.0426\\ 44.8759\\ 66.3826\end{array}$	$\begin{array}{c}59.5779\\ 75.3924\\ 98.2164\end{array}$

Table 8. Predicted MSE values with p = 4, m = 30 and $n_i=5$.

${\gamma }^2$	${\sigma }^2$	$\hat{u}$	${\hat{u}}_{me}$	${\hat{u}}_{{\hat{d}}_h}$	${\hat{u}}_{lfmp}$	$\hat{u}/{\hat{u}}_{lfmp}$	${\hat{u}}_{me}/{\hat{u}}_{lfmp}$	${\hat{u}}_{{\hat{d}}_h}/{\hat{u}}_{lfmp}$
0.90	$\begin{array}{c}0.5\\ 1\\ 10\end{array}$	$\begin{array}{c}0.163931121\\ 0.332341245\\ 3.883180462\end{array}$	$\begin{array}{c}0.163380131\\ 0.332787043\\ 3.890053063\end{array}$	$\begin{array}{c}0.163802506\\ 0.332597148\\ 3.886868865\end{array}$	$\begin{array}{c}0.163261649\\ 0.333038510\\ 3.893748122\end{array}$	$\begin{array}{c}0.4083\\ -0.2098\\ -0.2721\end{array}$	$\begin{array}{c}0.0725\\ -0.0755\\ -0.0949\end{array}$	$\begin{array}{c}0.3301\\ -0.1327\\ -0.1769\end{array}$
0.95	$\begin{array}{c}0.5\\ 1\\ 10\end{array}$	$\begin{array}{c}0.163937446\\ 0.332351579\\ 3.883202839\end{array}$	$\begin{array}{c}0.163280824\\ 0.330646811\\ 3.862361232\end{array}$	$\begin{array}{c}0.163795856\\ 0.331956434\\ 3.876337677\end{array}$	$\begin{array}{c}0.163165437\\ 0.330373317\\ 3.858445026\end{array}$	$\begin{array}{c}0.4709\\ 0.5952\\ 0.6375\end{array}$	$\begin{array}{c}0.0706\\ 0.0827\\ 0.1013\end{array}$	$\begin{array}{c}0.3848\\ 0.4769\\ 0.4615\end{array}$
0.99	$\begin{array}{c}0.5\\ 1\\ 10\end{array}$	$\begin{array}{c}0.163944095\\ 0.332362072\\ 3.883208286\end{array}$	$\begin{array}{c}0.163086259\\ 0.330539323\\ 3.863445116\end{array}$	$\begin{array}{c}0.163770323\\ 0.331958161\\ 3.876519827\end{array}$	$\begin{array}{c}0.163056855\\ 0.330508220\\ 3.859643872\end{array}$	$\begin{array}{c}0.5411\\ 0.5577\\ 0.6068\end{array}$	$\begin{array}{c}0.0180\\ 0.0094\\ 0.0983\end{array}$	$\begin{array}{c}0.4356\\ 0.4367\\ 0.4353\end{array}$

Table 9. Estimated MSE values with p = 4, m = 60 and $n_i=10$.

${\gamma }^2$	${\sigma }^2$	$\widehat{\mathrm{\text{β}}}$	${\widehat{\mathrm{\text{β}}}}_{me}$	${\widehat{\mathrm{\text{β}}}}_{{\hat{d}}_h}$	${\widehat{\mathrm{\text{β}}}}_{lfme}$	$\widehat{\mathrm{\text{β}}}/{\widehat{\mathrm{\text{β}}}}_{lfme}$	${\widehat{\mathrm{\text{β}}}}_{me}/{\widehat{\mathrm{\text{β}}}}_{lfme}$	${\widehat{\mathrm{\text{β}}}}_{{\hat{d}}_h}/{\widehat{\mathrm{\text{β}}}}_{lfme}$
0.90	$\begin{array}{c}0.5\\ 1\\ 10\end{array}$	$\begin{array}{c}0.025630228\\ 0.051258721\\ 0.512390711\end{array}$	$\begin{array}{c}0.024679603\\ 0.047590094\\ 0.290100771\end{array}$	$\begin{array}{c}0.025211253\\ 0.049603433\\ 0.382396641\end{array}$	$\begin{array}{c}0.024276568\\ 0.046055591\\ 0.216901368\end{array}$	$\begin{array}{c}5.2814\\ 10.1507\\ 57.6687\end{array}$	$\begin{array}{c}1.6330\\ 3.2244\\ 25.2324\end{array}$	$\begin{array}{c}3.7074\\ 7.1524\\ 43.2784\end{array}$
0.95	$\begin{array}{c}0.5\\ 1\\ 10\end{array}$	$\begin{array}{c}0.050967611\\ 0.101931763\\ 1.018926171\end{array}$	$\begin{array}{c}0.047626129\\ 0.089305499\\ 0.426523475\end{array}$	$\begin{array}{c}0.049311739\\ 0.095470733\\ 0.632346202\end{array}$	$\begin{array}{c}0.046080752\\ 0.083656349\\ 0.263886154\end{array}$	$\begin{array}{c}9.5881\\ 17.9290\\ 74.1015\end{array}$	$\begin{array}{c}3.2448\\ 6.3256\\ 38.1309\end{array}$	$\begin{array}{c}6.5521\\ 12.3748\\ 58.2687\end{array}$
0.99	$\begin{array}{c}0.5\\ 1\\ 10\end{array}$	$\begin{array}{c}0.253720249\\ 0.507423211\\ 5.072276675\end{array}$	$\begin{array}{c}0.187425051\\ 0.298292250\\ 0.548580815\end{array}$	$\begin{array}{c}0.216230856\\ 0.377939317\\ 2.190539699\end{array}$	$\begin{array}{c}0.159799805\\ 0.222153380\\ 0.213480744\end{array}$	$\begin{array}{c}37.0173\\ 56.2193\\ 95.7912\end{array}$	$\begin{array}{c}14.7393\\ 25.5249\\ 61.0849\end{array}$	$\begin{array}{c}26.0975\\ 41.2198\\ 90.2544\end{array}$

Table 3. Estimated MSE values with p = 2, m = 30 and $n_i=5$.

${\gamma }^2$	${\sigma }^2$	$\widehat{\mathrm{\text{β}}}$	${\widehat{\mathrm{\text{β}}}}_{me}$	${\widehat{\mathrm{\text{β}}}}_{{\hat{d}}_h}$	${\widehat{\mathrm{\text{β}}}}_{lfme}$	$\widehat{\mathrm{\text{β}}}/{\widehat{\mathrm{\text{β}}}}_{lfme}$	${\widehat{\mathrm{\text{β}}}}_{me}/{\widehat{\mathrm{\text{β}}}}_{lfme}$	${\widehat{\mathrm{\text{β}}}}_{{\hat{d}}_h}/{\widehat{\mathrm{\text{β}}}}_{lfme}$
0.90	$\begin{array}{c}0.5\\ 1\\ 10\end{array}$	$\begin{array}{c}0.034172260\\ 0.068355304\\ 0.684626643\end{array}$	$\begin{array}{c}0.029546813\\ 0.051635608\\ 0.130726927\end{array}$	$\begin{array}{c}0.033889837\\ 0.067263424\\ 0.616032929\end{array}$	$\begin{array}{c}0.029305454\\ 0.050827202\\ 0.120487228\end{array}$	$\begin{array}{c}14.2419\\ 25.6426\\ 82.4010\end{array}$	$\begin{array}{c}0.8168\\ 1.5655\\ 7.8328\end{array}$	$\begin{array}{c}13.5273\\ 24.4356\\ 80.4414\end{array}$
0.95	$\begin{array}{c}0.5\\ 1\\ 10\end{array}$	$\begin{array}{c}0.066774659\\ 0.133571270\\ 1.337913932\end{array}$	$\begin{array}{c}0.050109297\\ 0.078197834\\ 0.111981552\end{array}$	$\begin{array}{c}0.065685569\\ 0.129484733\\ 1.144462731\end{array}$	$\begin{array}{c}0.049299296\\ 0.075848404\\ 0.100152233\end{array}$	$\begin{array}{c}26.1706\\ 43.2150\\ 92.5142\end{array}$	$\begin{array}{c}1.6164\\ 3.0044\\ 10.5636\end{array}$	$\begin{array}{c}24.9465\\ 41.4228\\ 91.2489\end{array}$
0.99	$\begin{array}{c}0.5\\ 1\\ 10\end{array}$	$\begin{array}{c}0.327740789\\ 0.655589328\\ 6.567047387\end{array}$	$\begin{array}{c}0.104003590\\ 0.104919660\\ 0.069167955\end{array}$	$\begin{array}{c}0.306216438\\ 0.587336309\\ 5.123817483\end{array}$	$\begin{array}{c}0.097243659\\ 0.094271933\\ 0.060878465\end{array}$	$\begin{array}{c}70.3290\\ 85.6202\\ 99.0729\end{array}$	$\begin{array}{c}6.4997\\ 10.1484\\ 11.9845\end{array}$	$\begin{array}{c}68.2434\\ 83.9492\\ 98.8118\end{array}$

Table 10. Predicted MSE values with p = 4, m = 60 and $n_i=10$.

${\gamma }^2$	${\sigma }^2$	$\hat{u}$	${\hat{u}}_{me}$	${\hat{u}}_{{\hat{d}}_h}$	${\hat{u}}_{lfmp}$	$\hat{u}/{\hat{u}}_{lfmp}$	${\hat{u}}_{me}/{\hat{u}}_{lfmp}$	${\hat{u}}_{{\hat{d}}_h}/{\hat{u}}_{lfmp}$
0.90	$\begin{array}{c}0.5\\ 1\\ 10\end{array}$	$\begin{array}{c}0.076301301\\ 0.152691652\\ 1.542243891\end{array}$	$\begin{array}{c}0.076204694\\ 0.152427423\\ 1.537362490\end{array}$	$\begin{array}{c}0.076297466\\ 0.152680644\\ 1.541908579\end{array}$	$\begin{array}{c}0.076201619\\ 0.152420539\\ 1.537675547\end{array}$	$\begin{array}{c}0.1306\\ 0.1775\\ 0.2962\end{array}$	$\begin{array}{c}0.0040\\ 0.0045\\ -0.0203\end{array}$	$\begin{array}{c}0.1256\\ 0.1703\\ 0.2745\end{array}$
0.95	$\begin{array}{c}0.5\\ 1\\ 10\end{array}$	$\begin{array}{c}0.076301976\\ 0.152692990\\ 1.542255152\end{array}$	$\begin{array}{c}0.076139063\\ 0.152257915\\ 1.536095808\end{array}$	$\begin{array}{c}0.076304804\\ 0.152700369\\ 1.542372187\end{array}$	$\begin{array}{c}0.076144539\\ 0.152279318\\ 1.537763255\end{array}$	$\begin{array}{c}0.2063\\ 0.2709\\ 0.2912\end{array}$	$\begin{array}{c}-0.0071\\ -0.0140\\ -0.1085\end{array}$	$\begin{array}{c}0.2100\\ 0.2757\\ 0.2988\end{array}$
0.99	$\begin{array}{c}0.5\\ 1\\ 10\end{array}$	$\begin{array}{c}0.076302790\\ 0.152694603\\ 1.542268935\end{array}$	$\begin{array}{c}0.075987587\\ 0.151961778\\ 1.537643517\end{array}$	$\begin{array}{c}0.076303443\\ 0.152694806\\ 1.542476207\end{array}$	$\begin{array}{c}0.076012102\\ 0.152064955\\ 1.540231580\end{array}$	$\begin{array}{c}0.3809\\ 0.4123\\ 0.1321\end{array}$	$\begin{array}{c}-0.0322\\ -0.0678\\ -0.1683\end{array}$	$\begin{array}{c}0.3818\\ 0.4124\\ 0.1455\end{array}$

1. For fixed value of sample size, correlation and p, the EMSE values of $\widehat{\mathrm{\text{β}}}$, ${\widehat{\mathrm{\text{β}}}}_{me}$, ${\widehat{\mathrm{\text{β}}}}_d$ and ${\widehat{\mathrm{\text{β}}}}_{lfme}$ increase with the increase of ${\sigma }^2$.

2. When sample size, ${\sigma }^2$ and p are fixed, the superiorities of ${\widehat{\mathrm{\text{β}}}}_{lfme}$ over $\widehat{\mathrm{\text{β}}}$, ${\widehat{\mathrm{\text{β}}}}_{me}$ and ${\widehat{\mathrm{\text{β}}}}_d$ increase as ${\gamma }^2$ increases. And, the superiority ${\widehat{\mathrm{\text{β}}}}_{lfme}$ over $\widehat{\mathrm{\text{β}}}$ and ${\widehat{\mathrm{\text{β}}}}_d$ can be seen more clear than the superiority ${\widehat{\mathrm{\text{β}}}}_{lfme}$ over ${\widehat{\mathrm{\text{β}}}}_{me}$.

3. The $EMSE\left(\widehat{\mathrm{\text{β}}}\right)$, $EMSE\left({\widehat{\mathrm{\text{β}}}}_{me}\right)$, $EMSE\left({\widehat{\mathrm{\text{β}}}}_d\right)$ and $EMSE\left({\widehat{\mathrm{\text{β}}}}_{lfme}\right)$ values increase with the increase of p values under fixed value of sample size, correlation and ${\sigma }^2$.

4. As $n=\sum _{i=1}^m{n}_i$ increases from 150 to 600 at fixed ${\gamma }^2$, ${\sigma }^2$ and p, the EMSE values of $\widehat{\mathrm{\text{β}}}$, ${\widehat{\mathrm{\text{β}}}}_{me}$, ${\widehat{\mathrm{\text{β}}}}_d$ and ${\widehat{\mathrm{\text{β}}}}_{lfme}$ decreases.

5. As the degree of correlation increases under fixed values of sample size, ${\sigma }^2$ and p, the $EMSE\left(\widehat{\mathrm{\text{β}}}\right)$, $EMSE\left({\widehat{\mathrm{\text{β}}}}_{me}\right)$, $EMSE\left({\widehat{\mathrm{\text{β}}}}_d\right)$ and $EMSE\left({\widehat{\mathrm{\text{β}}}}_{lfme}\right)$ values increase.

6. The $EMSE\left({\widehat{\mathrm{\text{β}}}}_{lfme}\right)$ values are smaller than the $EMSE\left(\widehat{\mathrm{\text{β}}}\right)$, $EMSE\left({\widehat{\mathrm{\text{β}}}}_{me}\right)$ and $EMSE\left({\widehat{\mathrm{\text{β}}}}_{lfme}\right)$ values in all cases. This case is seen from the change percentages which are all positive. The change percentage indicates that as ${\sigma }^2$ and ${\gamma }^2$ increases at fixed sample size and p, the degrees of supremacies of ${\widehat{\mathrm{\text{β}}}}_{lfme}$ over $\widehat{\mathrm{\text{β}}}$, ${\widehat{\mathrm{\text{β}}}}_{me}$ and ${\widehat{\mathrm{\text{β}}}}_d$ increase. Furthermore, we can say that the degree of supremacies of ${\widehat{\mathrm{\text{β}}}}_{lfme}$ over $\widehat{\mathrm{\text{β}}}$ and ${\widehat{\mathrm{\text{β}}}}_d$ is too much than the degree of supremacy of ${\widehat{\mathrm{\text{β}}}}_{lfme}$ over ${\widehat{\mathrm{\text{β}}}}_{me}$.

7. The $PMSE\left(\hat{u}\right)$, $PMSE\left({\hat{u}}_{me}\right)$, $PMSE\left({\hat{u}}_d\right)$ and $PMSE\left({\hat{u}}_{lfme}\right)$ values increase with the increase of ${\sigma }^2$ for fixed value of sample size, correlation and p. And, this increase can be also seen more clearly as p values increase.

8. With the increase of p values under fixed value of sample size, correlation and ${\sigma }^2$, the PMSE values of $\hat{u}$, ${\hat{u}}_{me}$, ${\hat{u}}_d$ and ${\hat{u}}_{lfme}$ increase.

9. As $n=\sum _{i=1}^m{n}_i$ increases from 150 to 600 at fixed ${\gamma }^2$, ${\sigma }^2$ and p, the $PMSE\left(\hat{u}\right)$, $PMSE\left({\hat{u}}_{me}\right)$, $PMSE\left({\hat{u}}_d\right)$ and $PMSE\left({\hat{u}}_{lfme}\right)$ values decrease.

10. The PMSE values of $\hat{u}$, ${\hat{u}}_{me}$, ${\hat{u}}_d$ and ${\hat{u}}_{lfme}$ are very similar to each other in all cases. Although the $PMSE\left({\hat{u}}_{me}\right)$ values are smaller than the $PMSE\left(\hat{u}\right)$, $PMSE\left({\hat{u}}_d\right)$ and $PMSE\left({\hat{u}}_{lfme}\right)$ values, we see that the $PMSE\left({\hat{u}}_{lfme}\right)$ values are smaller than the $PMSE\left(\hat{u}\right)$, $PMSE\left({\hat{u}}_{me}\right)$ and $PMSE\left({\hat{u}}_d\right)$ values as p values increase. But, consequently, we say that the difference between the PMSE values of $\hat{u}$, ${\hat{u}}_{me}$, ${\hat{u}}_d$ and ${\hat{u}}_{lfme}$ is not too much.

11. The change percentage indicates that as ${\sigma }^2$ and ${\gamma }^2$ increases at fixed sample size and p, the degrees of supremacies of ${\hat{u}}_{lfme}$ over $\hat{u}$, ${\hat{u}}_{me}$ and ${\hat{u}}_d$ increase. Moreover, we can say that the degree of supremacies of ${\hat{u}}_{lfme}$ over $\hat{u}$ and ${\hat{u}}_d$ is too much than the degree of supremacy of ${\hat{u}}_{lfme}$ over ${\hat{u}}_{me}$.

12. In Tables 3–10, ${\widehat{\mathrm{\text{β}}}}_{lfme}$ employs better approximation (that is, ${\widehat{\mathrm{\text{β}}}}_{lfme}$ values are closer) to the true values than $\widehat{\mathrm{\text{β}}}$, ${\widehat{\mathrm{\text{β}}}}_{me}$ and ${\widehat{\mathrm{\text{β}}}}_d$.

6. Discussion

In this article, we propose the LFME and LFMP in LMMs with the help of Kuran and Özkale's [12] mixed estimation method and Özkale and Kuran's [21] Liu approach. Then, matrix MSE comparisons are done and lastly, the study is supported with both numerical examples and a simulation study.

This study has confirmed that, under multicollinearity, the use of the LFME outperforms the Liu estimator for any selected Liu biasing parameter and the mixed estimator for appropriate selected Liu biasing parameter in terms of mean square error. If there exists prior information as in the form of stochastic linear restrictions, one can use the LFME and LFMP in an LMM. The advantage of the LFME and LFMP over the Liu estimator and predictor is that LFME and LFMP consider the prior information and over the mixed estimator and predictor is that they gain efficiency.

Appendix 1. Matrix algebra.

Theorem A.1 Rao and Toutenburg [24], p. 302 —

Let A>0 and B>0. Then, B−A>0 if and only if $A^{-1}-B^{-1}>0$.

Disclosure statement

No potential conflict of interest was reported by the authors.