Minimizing the expected value of the asymmetric loss function and an inequality for the variance of the loss

ABSTRACT

The coefficients of regression are usually estimated for minimization problems with asymmetric loss functions. In this paper, we rather correct predictions so that the prediction error follows a generalized Gaussian distribution. In our method, we not only minimize the expected value of the asymmetric loss, but also lower the variance of the loss. Predictions usually have errors. Therefore, it is necessary to use predictions in consideration of these errors. Our approach takes into account prediction errors. Furthermore, even if we do not understand the prediction method, which is a possible circumstance in, e.g. deep learning, we can use our method if we know the prediction error distribution and asymmetric loss function. Our method can be applied to procurement of electricity from electricity markets.

1. Introduction

In this paper, we treat minimization problems with loss functions as follows: Let $\left\{\right(x_i,y_i)\mid 1\le i\le n\}$ be a dataset, where $x_i$ are $1\times p$ explanatory vectors and $y_i\in \mathbb{R}$ are target variables. We assume that the following linear regression model:

$$\mathit{y}=X\mathrm{\text{β}}+\mathit{\epsilon },$$

where $\mathit{y}={}^t(y_1,\dots ,y_n)$, $\mathit{\epsilon }={}^t({\epsilon }_1,\dots ,{\epsilon }_n)$, and X is an $n\times p$ design matrix having $x_i$ as the ith row. Let L be the loss function for the parameter estimation. The parameter vector $\mathrm{\text{β}}$ can be estimated by

$$\widehat{\mathrm{\text{β}}}:=\arg \underset{\mathrm{\text{β}}}{min}\left\{\sum _{i=1}^{n}L\left({\epsilon }_i\right)\right\}.$$

The case of $L\left(\epsilon \right)={\epsilon }^2$ is known as the quadratic loss function, which is symmetric (see, e.g. Refs. [1,10,12]). In the case of an asymmetric loss function, we refer the reader to, e.g. Refs. [3,5,9,16]. Consider the case that the prediction accuracy is assessed by a risk function based on an asymmetric loss function. Let y and $\hat{y}$ be an observed and a predicted value of y. Suppose that the prediction error $\hat{y}-y$ follows a general Gaussian distribution. Then, the error can be corrected by adding a constant C, i.e. $\hat{y}-y+C$, determined by minimizing the risk function. We make the following assumptions:

1. The prediction error $z:=\hat{y}-y$ is the realized value of a random variable Z, whose density function is a generalized Gaussian distribution function (see, e.g. Refs. [4,11,13]) with mean zero

   $$f_Z\left(z\right):=\frac{1}{2ab\mathrm{\Gamma }\left(a\right)}\exp \left(-{∣\frac{z}{b}∣}^{1/a}\right),$$

   where $\mathrm{\Gamma }\left(a\right)$ is the gamma function and a, $b\in {\mathbb{R}}_{>0}$.
2. Let $k_1$, $k_2\in {\mathbb{R}}_{>0}$. If there is a mismatch between y and $\hat{y}$, we suffer a loss,

   $$L\left(z\right):=\left\{\begin{array}{ll}{k}_{1}z,& z\ge 0,\\ -k_{2}z,& z<0.\end{array}\right.$$

Here, $f_Z\left(z\right)$ could be a Gaussian distribution $\mathcal{N}(0,\frac{1}{2}{b}^2)$ or Laplace distribution $\mathrm{L}\mathrm{a}\mathrm{p}\mathrm{l}\mathrm{a}\mathrm{c}\mathrm{e}(0,b)$ with mean zero (see, e.g. [[7, p. 80], [8, p. 164]]). That is, $f_Z\left(z\right)$ is the probability density function of a Gaussian distribution when $a=\frac{1}{2}$, and $f_Z\left(z\right)$ is the probability density function of a Laplace distribution when a = 1.

Under the assumptions (I) and (II), we will derive the optimized predicted value $y^{\ast }=\hat{y}+C$ minimizing the expected value of the loss (risk). That is, we will solve the following risk minimization problem:

$$C=\arg \underset{c}{min}\left\{\mathrm{E}\left[L(Z+c)\right]\right\}.$$

In our method, we not only minimize the expected value of the asymmetric loss, but also lower the variance of the loss:

$$\mathrm{V}\left[L\right(Z+C\left)\right]\le \mathrm{V}\left[L\right(Z\left)\right].$$

The variance of the loss is reduced by correcting the predicted value y to the optimized predicted value $y^{\ast }$.

The motivation of our research is as follows: (1) Suppose we have a risk minimization problem based on an asymmetric loss function. Then, we can reduce the risk by correcting the predictors. For example, the paper [15] formulates a method for minimizing the expected value of the procurement cost of electricity in two popular spot markets: day-ahead and intra-day, under the assumption that the expected value of the unit prices and the distributions of the prediction errors for the electricity demand traded in two markets are known. The paper showed that if the procurement is increased or decreased from the prediction, in some cases, the expected value of the procurement cost is reduced. Our method is a generalization of the method in [15].

(2) In recent years, prediction methods have been black boxed by big data and machine learning (see, e.g. Ref. [6]). The day will soon come when we must minimize the objective function by using predictions obtained by such black-box methods. In our method, even if we do not know the prediction $\hat{y}$, we can determine the parameter C if we know the prediction error distribution $f_Z$ and asymmetric loss function L.

The remainder of this paper is organized as follows: In Section 2, we introduce the expected value and the variance of $L(Z+c)$ and determine the value c = C that gives the minimum value of $\mathrm{E}\left[L\right(Z+c\left)\right]$. In addition, we give a geometrical interpretation of the parameter C and give the minimized expected value $\mathrm{E}\left[L\right(Z+C\left)\right]$. In Section 3, we prove the inequality for the variance of the loss. In Section 4, we report the results of simulations of our method using actual data.

2. Expected value and variance of the loss

In the following, we assume that (I) and (II) hold. Here, we introduce the expected value and the variance of $L(Z+c)$ and determine the value $c=C$ that gives the minimum value of $\mathrm{E}\left[L\right(Z+c\left)\right]$. In addition, we give a geometrical interpretation of the parameter C and give the minimized expected value $\mathrm{E}\left[L\right(Z+C\left)\right]$.

2.1. Expected value and variance of the loss

Let y be an observation value and $\hat{y}$ be a predicted value of y. Let $\mathrm{\Gamma }\left(a\right)$ be the gamma function, and let $\mathrm{\Gamma }(a,x)$ and $\gamma (a,x)$ be the upper and the lower incomplete gamma functions, respectively (see, e.g. Refs. [2, p. 197, 4, p. 2, 14, p. 93]), defined by

$$\mathrm{\Gamma }\left(a\right):={\int }_0^{+\mathrm{\infty }}{t}^{a-1}{\mathrm{e}}^{-t}\mathrm{d}t,\mathrm{\Gamma }(a,x):={\int }_x^{+\mathrm{\infty }}{t}^{a-1}{\mathrm{e}}^{-t}\mathrm{d}t,\gamma (a,x):={\int }_0^x{t}^{a-1}{\mathrm{e}}^{-t}\mathrm{d}t,$$

where $\mathrm{R}\mathrm{e}\left(a\right)>0$ and $x\ge 0$. Also, for $c\in \mathbb{R}$, let $\mathrm{s}\mathrm{g}\mathrm{n}\left(c\right):=1(c\ge 0);-1(c<0)$. Then, the expected value and variance of $L(Z+c)$ are as follows (see Appendix 1 for the proof): For any $c\in \mathbb{R}$, we have

$$\begin{array}{rl}\mathrm{E}\left[L\right(Z+c\left)\right]& =\frac{(k_1-k_2)c}{2}+\frac{(k_1+k_2)\mid c\mid }{2\mathrm{\Gamma }\left(a\right)}\gamma \left(a,{∣\frac{c}{b}∣}^{1/a}\right)+\frac{(k_1+k_2)b}{2\mathrm{\Gamma }\left(a\right)}\mathrm{\Gamma }\left(2a,{∣\frac{c}{b}∣}^{1/a}\right),\end{array}$$ (1)

$$\begin{array}{rl}\mathrm{V}\left[L\right(Z+c\left)\right]& =\frac{(k_1+k_2{)}^2{c}^2}{4}+\frac{(k_1^2-k_2^2)bc}{2\mathrm{\Gamma }\left(a\right)}\mathrm{\Gamma }\left(2a,{∣\frac{c}{b}∣}^{1/a}\right)\\ & -\frac{(k_1+k_2{)}^{2}b\mid c\mid }{2\mathrm{\Gamma }(a{)}^2}\gamma \left(a,{∣\frac{c}{b}∣}^{1/a}\right)\mathrm{\Gamma }\left(2a,{∣\frac{c}{b}∣}^{1/a}\right)\\ & -\frac{(k_1+k_2{)}^2{c}^2}{4\mathrm{\Gamma }(a{)}^2}\gamma {\left(a,{∣\frac{c}{b}∣}^{1/a}\right)}^2-\frac{(k_1+k_2{)}^2{b}^2}{4\mathrm{\Gamma }(a{)}^2}\mathrm{\Gamma }{\left(2a,{∣\frac{c}{b}∣}^{1/a}\right)}^2\\ & +\frac{(k_1^2+k_2^2)b^2\mathrm{\Gamma }\left(3a\right)}{2\mathrm{\Gamma }\left(a\right)}+\mathrm{s}\mathrm{g}\mathrm{n}\left(c\right)\frac{(k_1^2-k_2^2)b^2}{2\mathrm{\Gamma }\left(a\right)}\gamma \left(3a,{∣\frac{c}{b}∣}^{1/a}\right).\end{array}$$ (2)

From this, we have the following:

$$\begin{array}{rl}\mathrm{E}\left[L\right(Z\left)\right]& =\frac{(k_1+k_2)b}{2\mathrm{\Gamma }\left(a\right)}\mathrm{\Gamma }\left(2a\right),\end{array}$$ (3)

$$\begin{array}{rl}\mathrm{V}\left[L\right(Z\left)\right]& =\frac{(k_1^2+k_2^2)b^2\mathrm{\Gamma }\left(3a\right)}{2\mathrm{\Gamma }\left(a\right)}-\frac{(k_1+k_2{)}^2{b}^2\mathrm{\Gamma }(2a{)}^2}{4\mathrm{\Gamma }(a{)}^2}.\end{array}$$ (4)

Let $\mathrm{e}\mathrm{r}\mathrm{f}\left(x\right)$ be the error function (see, e.g. Refs. [2, p. 196]) defined by

$$\mathrm{e}\mathrm{r}\mathrm{f}\left(x\right):=\frac{2}{\sqrt{\pi }}{\int }_0^x\exp \left(-t^2\right)\mathrm{d}t$$

for any $x\in \mathbb{R}$. We give two examples of $\mathrm{E}\left[L\right(Z+c\left)\right]$ and $\mathrm{V}\left[L\right(Z+c\left)\right]$. In the case of $Z\sim \mathrm{L}\mathrm{a}\mathrm{p}\mathrm{l}\mathrm{a}\mathrm{c}\mathrm{e}(0,b)$, since $a=1$, we have

$$\begin{array}{rl}\mathrm{E}\left[L\right(Z+c\left)\right]& =L\left(c\right)+\frac{(k_1+k_2)b}{2}\exp \left(-∣\frac{c}{b}∣\right),\\ \mathrm{V}\left[L\right(Z+c\left)\right]& =\left\{k_1^2+k_2^2+\mathrm{s}\mathrm{g}\mathrm{n}\left(c\right)(k_1^2-k_2^2)\right\}{b}^2-\mathrm{s}\mathrm{g}\mathrm{n}\left(c\right)(k_1+k_2)\left\{L\left(c\right)+b(k_1-k_2)\right\}\\ & \times b\exp \left(-∣\frac{c}{b}∣\right)-\frac{(k_1+k_2{)}^2{b}^2}{4}\exp \left(-2∣\frac{c}{b}∣\right).\end{array}$$

To derive these expressions, we used the equations $\gamma (1,x)=1-{\mathrm{e}}^{-x},\mathrm{\Gamma }(2,x)=(1+x){\mathrm{e}}^{-x}$, and $\gamma (3,x)=2-(2+2x+x^2){\mathrm{e}}^{-x}$, which are easily obtained from the definitions of the incomplete gamma functions. In the case of $Z\sim \mathcal{N}(0,\frac{1}{2}{b}^2)$, since $a=\frac{1}{2}$, we have

$$\begin{array}{rl}\mathrm{E}\left[L\right(Z+c\left)\right]& =\frac{(k_1-k_2)c}{2}+\frac{(k_1+k_2)c}{2}\mathrm{e}\mathrm{r}\mathrm{f}\left(\frac{c}{b}\right)+\frac{(k_1+k_2)b}{2\sqrt{\pi }}\exp \left(-\frac{c^2}{b^2}\right),\\ \mathrm{V}\left[L\right(Z+c\left)\right]& =\frac{(k_1^2+k_2^2)b^2}{4}+\frac{(k_1+k_2{)}^2{c}^2}{4}+\frac{(k_1^2-k_2^2)b^2}{4}\mathrm{e}\mathrm{r}\mathrm{f}\left(\frac{c}{b}\right)\\ & -\frac{(k_1+k_2{)}^2{c}^2}{4}{\mathrm{e}\mathrm{r}\mathrm{f}}^2\left(\frac{c}{b}\right)-\frac{(k_1+k_2{)}^{2}bc}{2\sqrt{\pi }}\mathrm{e}\mathrm{r}\mathrm{f}\left(\frac{c}{b}\right)\exp \left(-\frac{c^2}{b^2}\right)\\ & -\frac{(k_1+k_2{)}^2{b}^2}{4\pi }\exp \left(-\frac{2c^2}{b^2}\right).\end{array}$$

To derive these expressions, we used the equations $\gamma (\frac{1}{2},x)=\sqrt{\pi }\mathrm{e}\mathrm{r}\mathrm{f}\left(\sqrt{\pi }\right),\mathrm{\Gamma }(1,x)={\mathrm{e}}^{-x}$, and $\gamma (\frac{3}{2},x)=\left(\sqrt{\pi }/2\right)\mathrm{e}\mathrm{r}\mathrm{f}\left(\sqrt{\pi }\right)-\sqrt{\pi }{\mathrm{e}}^{-x}$, which are easily obtained from the definitions of the incomplete gamma functions. For $k_1=50$ and b = 1, we can plot $\mathrm{E}\left[L\right(Z\left)\right]$ and $\mathrm{V}\left[L\right(Z\left)\right]$ for the Laplace and the Gaussian distributions respect to $k_2$ as follows:

In both cases, the graph of the expected value is a straight line (positive slope), and the graph of the variance is a quadratic curve (convex downward) for $k_2$ (Figures 1–2).

Figure 1.

Plots for a Laplace distribution. (a) Expected value of $L\left(Z\right)$. (b) Variance of $L\left(Z\right)$.

Figure 2.

Plots for a Gaussian distribution. (a) Expected value of $L\left(Z\right)$. (b) Variance of $L\left(Z\right)$.

2.2. Parameter value minimizing the expected value

Here, we determine the value c = C that gives the minimum value of $\mathrm{E}\left[L\right(Z+c\left)\right]$. Since

$$\begin{array}{rl}\frac{\mathrm{d}}{\mathrm{d}c}\mathrm{\Gamma }\left(2a,{∣\frac{c}{b}∣}^{1/a}\right)& =-\frac{c}{ab}\exp \left(-{∣\frac{c}{b}∣}^{1/a}\right);\\ \frac{\mathrm{d}}{\mathrm{d}c}\gamma \left(a,{∣\frac{c}{b}∣}^{1/a}\right)& =\mathrm{s}\mathrm{g}\mathrm{n}\left(c\right)\frac{1}{ab}\exp \left(-{∣\frac{c}{b}∣}^{1/a}\right),\end{array}$$

we have

$$\frac{\mathrm{d}}{\mathrm{d}c}\mathrm{E}\left[L\right(Z+c\left)\right]=\frac{k_1-k_2}{2}+\mathrm{s}\mathrm{g}\mathrm{n}\left(c\right)\frac{k_1+k_2}{2\mathrm{\Gamma }\left(a\right)}\gamma \left(a,{∣\frac{c}{b}∣}^{1/a}\right).$$

We will denote the value of c satisfying $\left(\mathrm{d}/\mathrm{d}c\right)\mathrm{E}\left[L\right(Z+c\left)\right]=0$ as C. Then, from the first derivative test, we find that $\mathrm{E}\left[L\right(Z+c\left)\right]$ has a minimum value at c = C.

c	Less than C	C	More than C
$\frac{\mathrm{d}}{\mathrm{d}c}\mathrm{E}\left[L\right(Z+c\left)\right]$	Negative	0	Positive
$\mathrm{E}\left[L\right(Z+c\left)\right]$	Strongly decreasing		Strongly increasing

Also, it follows from

$$\gamma \left(a,{∣\frac{C}{b}∣}^{1/a}\right)=\mathrm{s}\mathrm{g}\mathrm{n}\left(C\right)\frac{k_2-k_1}{k_1+k_2}\mathrm{\Gamma }\left(a\right)$$ (5)

that $\mathrm{s}\mathrm{g}\mathrm{n}\left(C\right)=\mathrm{s}\mathrm{g}\mathrm{n}(k_2-k_1)$ and C = 0 only when $k_1=k_2$. Equation (5) presents us with a geometrical interpretation of C as follows: The ratio of $\mathrm{\Gamma }\left(a\right)$ and $\gamma (a,\mid C/b{\mid }^{1/a})$ is $1:\mid k_2-k_1\mid /(k_1+k_2)$. That is, the vertical axis $t=\mid C/b{\mid }^{1/a}$ divides the area between $t^{a-1}{\mathrm{e}}^{-t}$ and the t-axis into $\mid k_2-k_1\mid /(k_1+k_2):1-\mid k_2-k_1\mid /(k_1+k_2)$.

Let ${\mathrm{e}\mathrm{r}\mathrm{f}}^{-1}\left(x\right)$ be the inverse error function. We give two examples of C. In the case of $Z\sim \mathrm{L}\mathrm{a}\mathrm{p}\mathrm{l}\mathrm{a}\mathrm{c}\mathrm{e}(0,b)$, since a = 1, from $\gamma (1,x)=1-{\mathrm{e}}^{-x}$, we have

$$C=-\mathrm{s}\mathrm{g}\mathrm{n}(k_2-k_1)b\log \left(1-∣\frac{k_2-k_1}{k_1+k_2}∣\right).$$

In the case of $Z\sim \mathcal{N}(0,\frac{1}{2}{b}^2)$, since $a=\frac{1}{2}$, from $\gamma (\frac{1}{2},x)=\sqrt{\pi }\mathrm{e}\mathrm{r}\mathrm{f}\left(\sqrt{\pi }\right)$, we have

$$C=\mathrm{s}\mathrm{g}\mathrm{n}(k_2-k_1)b{\mathrm{e}\mathrm{r}\mathrm{f}}^{-1}\left(∣\frac{k_2-k_1}{k_1+k_2}∣\right).$$ (6)

Figure 3 plots C for Laplace and Gaussian distributions respect to $k_2$ for $k_1=50$ and b = 1.

Figure 3.

Plots of C for Laplace and Gaussian distributions. (a) When $Z\sim \mathrm{L}\mathrm{a}\mathrm{p}\mathrm{l}\mathrm{a}\mathrm{c}\mathrm{e}(0,1)$. (b) When $Z\sim \mathcal{N}(0,\frac{1}{2})$.

2.3. Minimized expected value of the loss

Here, we derive the minimum value of $\mathrm{E}\left[L\right(Z+c\left)\right]$. Substituting $c=C$ in Equation (A1), we have, from Equation (5),

$$\mathrm{E}\left[L\right(Z+C\left)\right]=\frac{(k_1+k_2)b}{2\mathrm{\Gamma }\left(a\right)}\mathrm{\Gamma }\left(2a,{∣\frac{C}{b}∣}^{1/a}\right).$$

This is the minimum value of $\mathrm{E}\left[L\right(Z+c\left)\right]$. From this and Equation (3), we have

$$\begin{array}{rl}\mathrm{E}\left[L\right(Z\left)\right]-\mathrm{E}\left[L\right(Z+C\left)\right]& =\frac{(k_1+k_2)b}{2\mathrm{\Gamma }\left(a\right)}\gamma \left(2a,{∣\frac{C}{b}∣}^{1/a}\right),\\ \frac{\mathrm{E}\left[L\right(Z+C\left)\right]}{\mathrm{E}\left[L\right(Z\left)\right]}& =\frac{1}{\mathrm{\Gamma }\left(2a\right)}\mathrm{\Gamma }\left(2a,{∣\frac{C}{b}∣}^{1/a}\right).\end{array}$$

Figure 4 plots $\mathrm{E}\left[L\right(Z\left)\right]-\mathrm{E}\left[L\right(Z+C\left)\right]$ for the Laplace and Gaussian distributions respect to $k_2$ for $k_1=50$ and b = 1.

Figure 4.

Plots of $\mathrm{E}\left[L\right(Z\left)\right]-\mathrm{E}\left[L\right(Z+C\left)\right]$ for the Laplace and Gaussian distributions. (a) When $Z\sim \mathrm{L}\mathrm{a}\mathrm{p}\mathrm{l}\mathrm{a}\mathrm{c}\mathrm{e}(0,1)$. (b) When $Z\sim \mathcal{N}(0,\frac{1}{2})$.

3. Inequality for the variance of the loss

Here, we derive an inequality for the variance of $L(Z+c)$. Let C be the value of c giving the minimum value of $\mathrm{E}\left[L\right(Z+c\left)\right]$. Then, the following holds:

Theorem 3.1

We have

$$\mathrm{V}\left[L\right(Z+C\left)\right]\le \mathrm{V}\left[L\right(Z\left)\right],$$

where equality holds only when $C=0;$ that is, when $k_1=k_2$.

Proof.

Put $C^{\prime }:=|C/b{|}^{1/a}$. It follows from Equation (5) that

$$\begin{array}{rl}\frac{(k_1^2-k_2^2)bC}{2\mathrm{\Gamma }\left(a\right)}\mathrm{\Gamma }\left(2a,C^{\prime }\right)& =-\mathrm{s}\mathrm{g}\mathrm{n}\left(C\right)\frac{k_2-k_1}{k_1+k_2}\frac{(k_1+k_2{)}^{2}b\mid C\mid }{2\mathrm{\Gamma }\left(a\right)}\mathrm{\Gamma }\left(2a,C^{\prime }\right)\\ & =-\frac{(k_1+k_2{)}^{2}b\mid C\mid }{2\mathrm{\Gamma }(a{)}^2}\gamma \left(a,C^{\prime }\right)\mathrm{\Gamma }\left(2a,C^{\prime }\right),\\ \mathrm{s}\mathrm{g}\mathrm{n}\left(C\right)\frac{(k_1^2-k_2^2)b^2}{2\mathrm{\Gamma }\left(a\right)}\gamma \left(3a,C^{\prime }\right)& =-\mathrm{s}\mathrm{g}\mathrm{n}\left(C\right)\frac{k_2-k_1}{k_1+k_2}\frac{(k_1+k_2{)}^2{b}^2}{2\mathrm{\Gamma }\left(a\right)}\gamma \left(3a,C^{\prime }\right)\\ & =-\frac{(k_1+k_2{)}^2{b}^2}{2\mathrm{\Gamma }(a{)}^2}\gamma \left(a,C^{\prime }\right)\gamma \left(3a,C^{\prime }\right).\end{array}$$

Hence, substituting c = C in Equation (A2), we have

$$\begin{array}{rl}\mathrm{V}\left[L\right(Z+C\left)\right]& =\frac{(k_1+k_2{)}^2{C}^2}{4}-\frac{(k_1+k_2{)}^{2}b\mid C\mid }{\mathrm{\Gamma }(a{)}^2}\gamma \left(a,C^{\prime }\right)\mathrm{\Gamma }\left(2a,C^{\prime }\right)\\ & -\frac{(k_1+k_2{)}^2{C}^2}{4\mathrm{\Gamma }(a{)}^2}\gamma {\left(a,C^{\prime }\right)}^2-\frac{(k_1+k_2{)}^2{b}^2}{4\mathrm{\Gamma }(a{)}^2}\mathrm{\Gamma }{\left(2a,C^{\prime }\right)}^2\\ & +\frac{(k_1^2+k_2^2)b^2\mathrm{\Gamma }\left(3a\right)}{2\mathrm{\Gamma }\left(a\right)}-\frac{(k_1+k_2{)}^2{b}^2}{2\mathrm{\Gamma }(a{)}^2}\gamma \left(a,C^{\prime }\right)\gamma \left(3a,C^{\prime }\right).\end{array}$$

From this and Equation (4), we obtain

$$\begin{array}{rl}\mathrm{V}\left[L\right(Z\left)\right]-\mathrm{V}\left[L\right(Z+C\left)\right]& =-\frac{(k_1+k_2{)}^2{C}^2}{4}+\frac{(k_1+k_2{)}^{2}b\mid C\mid }{\mathrm{\Gamma }(a{)}^2}\gamma \left(a,C^{\prime }\right)\mathrm{\Gamma }\left(2a,C^{\prime }\right)\\ & +\frac{(k_1+k_2{)}^2{C}^2}{4\mathrm{\Gamma }(a{)}^2}\gamma {\left(a,C^{\prime }\right)}^2+\frac{(k_1+k_2{)}^2{b}^2}{4\mathrm{\Gamma }(a{)}^2}\mathrm{\Gamma }{\left(2a,C^{\prime }\right)}^2\\ & -\frac{(k_1+k_2{)}^2{b}^2\mathrm{\Gamma }(2a{)}^2}{4\mathrm{\Gamma }(a{)}^2}+\frac{(k_1+k_2{)}^2{b}^2}{2\mathrm{\Gamma }(a{)}^2}\gamma \left(a,C^{\prime }\right)\gamma \left(3a,C^{\prime }\right)\\ & =\frac{(k_1+k_2{)}^2{b}^2}{4\mathrm{\Gamma }(a{)}^2}f\left(a,C^{\prime }\right),\end{array}$$

where, for a>0 and $x\ge 0$, $f(a,x)$ is defined as

$$\begin{array}{rl}f(a,x)& :=x^{2a}\gamma (a,x{)}^2-x^{2a}\mathrm{\Gamma }(a{)}^2+4x^a\gamma (a,x)\mathrm{\Gamma }(2a,x)\\ & +\mathrm{\Gamma }(2a,x{)}^2-\mathrm{\Gamma }(2a{)}^2+2\gamma (a,x)\gamma (3a,x).\end{array}$$

Here, since

$$\begin{array}{rl}\frac{\mathrm{d}}{\mathrm{d}x}f(a,x)& =2a{x}^{a-1}\left\{x^a\gamma (a,x{)}^2-x^a\mathrm{\Gamma }(a{)}^2+2\gamma (a,x)\mathrm{\Gamma }(2a,x)\right\}\\ & +2x^{a-1}{\mathrm{e}}^{-x}\gamma (3a,x)+2x^{2a-1}{\mathrm{e}}^{-x}\mathrm{\Gamma }(2a,x),\end{array}$$

from Lemma A.1 in Appendix 2, we have $\left(\mathrm{d}/\mathrm{d}x\right)f(a,x)>0$ (a>0, x>0). Also, $f(a,0)=0$ holds for a>0. Therefore, we obtain

$$\mathrm{V}\left[L\right(Z\left)\right]-\mathrm{V}\left[L\right(Z+C\left)\right]\ge 0,$$

where equality holds only when C = 0. Moreover, from Equation (5), we find that C = 0 holds only when $k_1=k_2$.

Figure 5 plots $\mathrm{V}\left[L\right(Z\left)\right]-\mathrm{V}\left[L\right(Z+C\left)\right]$ for the Laplace and Gaussian distributions respect to $k_2$ for $k_1=50$ and b = 1.

Figure 5.

Plots of $\mathrm{V}\left[L\right(Z\left)\right]-\mathrm{V}\left[L\right(Z+C\left)\right]$ for the Laplace and the Gaussian distributions. (a) When $Z\sim \mathrm{L}\mathrm{a}\mathrm{p}\mathrm{l}\mathrm{a}\mathrm{c}\mathrm{e}(0,1)$. (b) When $Z\sim \mathcal{N}(0,\frac{1}{2})$.

4. Simulation

The simulations of our method used actual data, in particular ‘cars’ of the R datasets package. The cars is a dataset of speeds and stopping distances of cars.

We separated the cars data into training and test datasets as follows: The odd-numbered data were selected the cars is the training dataset, and the even-numbered data were the test dataset. The following figure shows the scatter plots of the training and test data. The horizontal axis represents speed, and the vertical axis represents stopping distances of cars (Figure 6).

Figure 6.

Scatter plots of the training and test data. (a) Training data. (b) Verification data.

We used the training dataset to fit the parameters of the regression model $y=ax+b+\epsilon$ and also used it to find the solution to the minimization problem:

$$(\tilde{a},\tilde{b}):=\arg \underset{(a,b)}{min}\left\{\sum _{i=1}^{n}L(a{x}_i+b-y_i)\right\}.$$

The regression coefficients of y = ax + b were estimated to be $\hat{a}=4.09$ and $\hat{b}=-18.61$ by using the least squares method, and the unbiased sample variance of the error was 308.42. We obtained a prediction $\hat{y}=\hat{a}x+\hat{b}$ and $z=\hat{y}-y\sim \mathcal{N}(0,308.42)$. Fix the conditions as $k_1=1$ and $k_2=5$. Then C = 16.99 from Equation (6), and the estimated solution to the minimization problem is $(\tilde{a},\tilde{b})=(5.33,-19.33)$. Figure 7 plots (i)  $\hat{y}=\hat{a}x+\hat{b}$, (ii) $\hat{y}+C$, and (iii) $\tilde{y}=\tilde{a}x+\tilde{b}$.

Figure 7.

Scatter plots and plots of (i), (ii), and (iii). (a) Scatter plot for the training data and plots of (i), (ii), and (iii). (b) Scatter plot for the test data and plots of (i), (ii), and (iii).

The slope of $\tilde{y}$ is steeper than the slope of $\hat{y}$. Therefore, if the model $y\left(x\right)=ax+b+\epsilon$ is true, $a=\hat{a}$, and $b=\hat{b}$, then the value $\tilde{y}\left(x\right)$ is far from $y\left(x\right)$ when x is large. That is, the loss $L\left(\tilde{y}\right(x)-y(x\left)\right)$ is large when x is large. This implies that using $\tilde{y}$ as a prediction is high risk. On the other hand, the slope of $\hat{y}$ is equal to the slope of $\hat{y}+C$. Therefore, using $\hat{y}+C$ as a prediction is low risk.

Table 1 lists the sample means and sample variances of the loss for the training and test data when $k_1=1$ and $k_2=5$.

Table 1. Sample means and sample variances of the loss for the training and the test data when $k_1=1$ and $k_2=5$.

	Mean (training)	Mean (test)	Variance (training)	Variance (test)
(i)	37.31	28.05	3337.93	896.99
(ii)	31.31	21.86	834.35	105.40
(iii)	28.48	25.56	612.18	127.50

Figure 8 is a plot of the sample means of the loss for the training and test data for (i), (ii), and (iii) respect to $k_2$.

Figure 8.

Plots of sample means for the training and the test data. (a) Sample means for the training data. (b) Sample means for the test data.

For the test data, the plot of the loss of (iii) is in the form of steps. This is because $\tilde{a}$ and $\tilde{b}$ are discontinuous with respect to $k_2$. On the other hand, the plot of the loss of (ii) is smooth. This is because C is continuous with respect to $k_2$. Therefore, the loss of (ii) is more stable than the loss of (iii). In addition, deriving $\tilde{a}$ and $\tilde{b}$ is troublesome, but deriving C is easy because C is explicitly a function of $k_1$ and $k_2$.

Figure 9 plots sample variances of the loss for the training and test data for (i), (ii), and (iii) respect to $k_2$.

Figure 9.

Plots of sample variances for the training and the test data. (a) Sample variance for the training data. (b) Sample variances for the training data. (c) Sample variance for the test data. (d) Sample variances for the test data.

Clealy, from Figures 8 and 9, the sample means and sample variances of (ii) and (iii) are lower than those of (i) for any $k_2$. This simulation shows that it is best to use (ii) as the prediction. Other simulation results are listed in Appendix 3.

Appendix 1.

Calculation of the expected value and variance of the loss

A.1. Expected value of the loss

Put $\mathrm{\text{β}}:=\left(2ab\mathrm{\Gamma }\right(a){)}^{-1}$. Then,

$$\begin{array}{rl}\mathrm{E}\left[L\right(Z+c\left)\right]& ={\int }_{-\mathrm{\infty }}^{+\mathrm{\infty }}L(z+c)f_Z\left(z\right)\mathrm{d}z\\ & =k_2\mathrm{\text{β}}{\int }_{-\mathrm{\infty }}^{-c}(-z-c)\exp \left(-{∣\frac{z}{b}∣}^{1/a}\right)\mathrm{d}z+k_1\mathrm{\text{β}}{\int }_{-c}^{+\mathrm{\infty }}(z+c)\exp \left(-{∣\frac{z}{b}∣}^{1/a}\right)\mathrm{d}z.\end{array}$$

Replace z with bz to get

$$\mathrm{E}\left[L\right(Z+c\left)\right]=k_{2}b\mathrm{\text{β}}{\int }_{-\mathrm{\infty }}^{-c/b}(-bz-c)\exp \left(-\mid z{\mid }^{1/a}\right)\mathrm{d}z+k_{1}b\mathrm{\text{β}}{\int }_{-c/b}^{+\mathrm{\infty }}(bz+c)\exp \left(-\mid z{\mid }^{1/a}\right)\mathrm{d}z.$$

When $c\ge 0$, we have

$$\begin{array}{rl}\mathrm{E}\left[L\right(Z+c\left)\right]& =k_{2}b\mathrm{\text{β}}{\int }_{-\mathrm{\infty }}^{-c/b}(-bz-c)\exp \left(-(-z{)}^{1/a}\right)\mathrm{d}z\\ & +k_{1}b\mathrm{\text{β}}{\int }_{-c/b}^0(bz+c)\exp \left(-(-z{)}^{1/a}\right)\mathrm{d}z+k_{1}b\mathrm{\text{β}}{\int }_0^{+\mathrm{\infty }}(bz+c)\exp \left(-z^{1/a}\right)\mathrm{d}z\\ & =k_{2}b\mathrm{\text{β}}{\int }_{c/b}^{+\mathrm{\infty }}(bz-c)\exp \left(-z^{1/a}\right)\mathrm{d}z\\ & +k_{1}b\mathrm{\text{β}}{\int }_0^{c/b}(-bz+c)\exp \left(-z^{1/a}\right)\mathrm{d}z+k_{1}b\mathrm{\text{β}}{\int }_0^{+\mathrm{\infty }}(bz+c)\exp \left(-z^{1/a}\right)\mathrm{d}z\\ & =(k_1+k_2)b^2\mathrm{\text{β}}{\int }_{c/b}^{+\mathrm{\infty }}z\exp \left(-z^{1/a}\right)\mathrm{d}z\\ & +(k_1-k_2)bc\mathrm{\text{β}}{\int }_0^{+\mathrm{\infty }}\exp \left(-z^{1/a}\right)\mathrm{d}z+(k_1+k_2)bc\mathrm{\text{β}}{\int }_0^{c/b}\exp \left(-z^{1/a}\right)\mathrm{d}z.\end{array}$$

When c<0, we have

$$\begin{array}{rl}\mathrm{E}\left[L\right(Z+c\left)\right]& =k_{2}b\mathrm{\text{β}}{\int }_{-\mathrm{\infty }}^0(-bz-c)\exp \left(-(-z{)}^{1/a}\right)\mathrm{d}z+k_{2}b\mathrm{\text{β}}{\int }_0^{-c/b}(-bz-c)\exp \left(-z^{1/a}\right)\mathrm{d}z\\ & +k_{1}b\mathrm{\text{β}}{\int }_{-c/b}^{+\mathrm{\infty }}(bz+c)\exp \left(-z^{1/a}\right)\mathrm{d}z\\ & =k_{2}b\mathrm{\text{β}}{\int }_0^{+\mathrm{\infty }}(bz-c)\exp \left(-z^{1/a}\right)\mathrm{d}z+k_{2}b\mathrm{\text{β}}{\int }_0^{-c/b}(-bz-c)\exp \left(-z^{1/a}\right)\mathrm{d}z\\ & +k_{1}b\mathrm{\text{β}}{\int }_{-c/b}^{+\mathrm{\infty }}(bz+c)\exp \left(-z^{1/a}\right)\mathrm{d}z\\ & =(k_1+k_2)b^2\mathrm{\text{β}}{\int }_{-c/b}^{+\mathrm{\infty }}z\exp \left(-z^{1/a}\right)\mathrm{d}z\\ & +(k_1-k_2)bc\mathrm{\text{β}}{\int }_0^{+\mathrm{\infty }}\exp \left(-z^{1/a}\right)\mathrm{d}z-(k_1+k_2)bc\mathrm{\text{β}}{\int }_0^{-c/b}\exp \left(-z^{1/a}\right)\mathrm{d}z.\end{array}$$

From the above, for any $c\in \mathbb{R}$, we have

$$\begin{array}{rl}\mathrm{E}\left[L\right(Z+c\left)\right]& =(k_1+k_2)b^2\mathrm{\text{β}}{\int }_{\mid c/b\mid }^{+\mathrm{\infty }}z\exp \left(-z^{1/a}\right)\mathrm{d}z\\ & +(k_1-k_2)bc\mathrm{\text{β}}{\int }_0^{+\mathrm{\infty }}\exp \left(-z^{1/a}\right)\mathrm{d}z+(k_1+k_2)b\mid c\mid \mathrm{\text{β}}{\int }_0^{\mid c/b\mid }\exp \left(-z^{1/a}\right)\mathrm{d}z.\end{array}$$

Now set $t:=z^{1/a}$ to get

$$\begin{array}{rl}\mathrm{E}\left[L\right(Z+c\left)\right]& =(k_1+k_2)a{b}^2\mathrm{\text{β}}{\int }_{c^{\prime }}^{+\mathrm{\infty }}{t}^{2a-1}{\mathrm{e}}^{-t}\mathrm{d}t\\ & +(k_1-k_2)abc\mathrm{\text{β}}{\int }_0^{+\mathrm{\infty }}{t}^{a-1}{\mathrm{e}}^{-t}\mathrm{d}t+(k_1+k_2)ab\mid c\mid \mathrm{\text{β}}{\int }_0^{c^{\prime }}{t}^{a-1}{\mathrm{e}}^{-t}\mathrm{d}t\\ & =(k_1+k_2)a{b}^2\mathrm{\text{β}}\mathrm{\Gamma }(2a,c^{\prime })+(k_1-k_2)abc\mathrm{\text{β}}\mathrm{\Gamma }\left(a\right)+(k_1+k_2)ab\mid c\mid \mathrm{\text{β}}\gamma (a,c^{\prime }),\end{array}$$

where $c^{\prime }:=\mid c/b{\mid }^{1/a}$. Therefore, for any $c\in \mathbb{R}$, we have

$$\mathrm{E}\left[L\right(Z+c\left)\right]=\frac{(k_1-k_2)c}{2}+\frac{(k_1+k_2)\mid c\mid }{2\mathrm{\Gamma }\left(a\right)}\gamma \left(a,{∣\frac{c}{b}∣}^{1/a}\right)+\frac{(k_1+k_2)b}{2\mathrm{\Gamma }\left(a\right)}\mathrm{\Gamma }\left(2a,{∣\frac{c}{b}∣}^{1/a}\right).$$

A.2. Variance of the loss

Put $\mathrm{\text{β}}:=\left(2ab\mathrm{\Gamma }\right(a){)}^{-1}$. Then,

$$\begin{array}{rl}\mathrm{E}\left[L\right(Z+c{)}^2]& ={\int }_{-\mathrm{\infty }}^{+\mathrm{\infty }}L(z+c{)}^2{f}_Z(z)\mathrm{d}z\\ & =k_2^2\mathrm{\text{β}}{\int }_{-\mathrm{\infty }}^{-c}(z+c{)}^2\exp \left(-{∣\frac{z}{b}∣}^{1/a}\right)\mathrm{d}z+k_1^2\mathrm{\text{β}}{\int }_{-c}^{+\mathrm{\infty }}(z+c{)}^2\exp \left(-{∣\frac{z}{b}∣}^{1/a}\right)\mathrm{d}z.\end{array}$$

Replace z with bz to get

$$\mathrm{E}\left[L\right(Z+c{)}^2]=k_2^{2}b\mathrm{\text{β}}{\int }_{-\mathrm{\infty }}^{-c/b}(bz+c{)}^2\exp \left(-\mid z{\mid }^{1/a}\right)\mathrm{d}z+k_1^{2}b\mathrm{\text{β}}{\int }_{-c/b}^{+\mathrm{\infty }}(bz+c{)}^2\exp \left(-\mid z{\mid }^{1/a}\right)\mathrm{d}z.$$

When $c\ge 0$, we have

$$\begin{array}{rl}\mathrm{E}\left[L\right(Z+c{)}^2]& =k_2^{2}b\mathrm{\text{β}}{\int }_{-\mathrm{\infty }}^{-c/b}(bz+c{)}^2\exp \left(-(-z{)}^{1/a}\right)\mathrm{d}z\\ & +k_1^{2}b\mathrm{\text{β}}{\int }_{-c/b}^0(bz+c{)}^2\exp \left(-(-z{)}^{1/a}\right)\mathrm{d}z+k_1^{2}b\mathrm{\text{β}}{\int }_0^{+\mathrm{\infty }}(bz+c{)}^2\exp \left(-z^{1/a}\right)\mathrm{d}z\\ & =k_2^{2}b\mathrm{\text{β}}{\int }_{c/b}^{+\mathrm{\infty }}(-bz+c{)}^2\exp \left(-z^{1/a}\right)\mathrm{d}z\\ & +k_1^{2}b\mathrm{\text{β}}{\int }_0^{c/b}(-bz+c{)}^2\exp \left(-z^{1/a}\right)\mathrm{d}z+k_1^{2}b\mathrm{\text{β}}{\int }_0^{+\mathrm{\infty }}(bz+c{)}^2\exp \left(-z^{1/a}\right)\mathrm{d}z\\ & =(k_1^2+k_2^2)b^3\mathrm{\text{β}}{\int }_0^{+\mathrm{\infty }}{z}^2\exp \left(-z^{1/a}\right)\mathrm{d}z+(k_1^2-k_2^2)b^3\mathrm{\text{β}}{\int }_0^{c/b}{z}^2\exp \left(-z^{1/a}\right)\mathrm{d}z\\ & +2(k_1^2-k_2^2)b^{2}c\mathrm{\text{β}}{\int }_{c/b}^{+\mathrm{\infty }}z\exp \left(-z^{1/a}\right)\mathrm{d}z\\ & +(k_1^2+k_2^2)b{c}^2\mathrm{\text{β}}{\int }_0^{+\mathrm{\infty }}\exp \left(-z^{1/a}\right)\mathrm{d}z+(k_1^2-k_2^2)b{c}^2\mathrm{\text{β}}{\int }_0^{c/b}\exp \left(-z^{1/a}\right)\mathrm{d}z.\end{array}$$

When c<0, we have

$$\begin{array}{rl}\mathrm{E}\left[L\right(Z+c{)}^2]& =k_2^{2}b\mathrm{\text{β}}{\int }_{-\mathrm{\infty }}^0(bz+c{)}^2\exp \left(-(-z{)}^{1/a}\right)\mathrm{d}z+k_2^{2}b\mathrm{\text{β}}{\int }_0^{-c/b}(bz+c{)}^2\exp \left(-z^{1/a}\right)\mathrm{d}z\\ & +k_1^{2}b\mathrm{\text{β}}{\int }_{-c/b}^{+\mathrm{\infty }}(bz+c{)}^2\exp \left(-z^{1/a}\right)\mathrm{d}z\\ & =k_2^{2}b\mathrm{\text{β}}{\int }_0^{+\mathrm{\infty }}(-bz+c{)}^2\exp \left(-z^{1/a}\right)\mathrm{d}z+k_2^{2}b\mathrm{\text{β}}{\int }_0^{-c/b}(bz+c{)}^2\exp \left(-z^{1/a}\right)\mathrm{d}z\\ & +k_1^{2}b\mathrm{\text{β}}{\int }_{-c/b}^{+\mathrm{\infty }}(bz+c{)}^2\exp \left(-z^{1/a}\right)\mathrm{d}z\\ & =(k_1^2+k_2^2)b^3\mathrm{\text{β}}{\int }_0^{+\mathrm{\infty }}{z}^2\exp \left(-z^{1/a}\right)\mathrm{d}z-(k_1^2-k_2^2)b^3\mathrm{\text{β}}{\int }_0^{-c/b}{z}^2\exp \left(-z^{1/a}\right)\mathrm{d}z\\ & +2(k_1^2-k_2^2)b^{2}c\mathrm{\text{β}}{\int }_{-c/b}^{+\mathrm{\infty }}z\exp \left(-z^{1/a}\right)\mathrm{d}z\\ & +(k_1^2+k_2^2)b{c}^2\mathrm{\text{β}}{\int }_0^{+\mathrm{\infty }}\exp \left(-z^{1/a}\right)\mathrm{d}z-(k_1^2-k_2^2)b{c}^2\mathrm{\text{β}}{\int }_0^{-c/b}\exp \left(-z^{1/a}\right)\mathrm{d}z.\end{array}$$

From the above, for any $c\in \mathbb{R}$, we have

$$\begin{array}{rl}\mathrm{E}\left[L\right(Z+c{)}^2]& =(k_1^2+k_2^2)b^3\mathrm{\text{β}}{\int }_0^{+\mathrm{\infty }}{z}^2\exp \left(-z^{1/a}\right)\mathrm{d}z\\ & +\mathrm{s}\mathrm{g}\mathrm{n}\left(c\right)(k_1^2-k_2^2)b^3\mathrm{\text{β}}{\int }_0^{\mid c/b\mid }{z}^2\exp \left(-z^{1/a}\right)\mathrm{d}z\\ & +2(k_1^2-k_2^2)b^{2}c\mathrm{\text{β}}{\int }_{\mid c/b\mid }^{+\mathrm{\infty }}z\exp \left(-z^{1/a}\right)\mathrm{d}z\\ & +(k_1^2+k_2^2)b{c}^2\mathrm{\text{β}}{\int }_0^{+\mathrm{\infty }}\exp \left(-z^{1/a}\right)\mathrm{d}z\\ & +\mathrm{s}\mathrm{g}\mathrm{n}\left(c\right)(k_1^2-k_2^2)b{c}^2\mathrm{\text{β}}{\int }_0^{\mid c/b\mid }\exp \left(-z^{1/a}\right)\mathrm{d}z.\end{array}$$

Now set $t:=z^{1/a}$ to get

$$\begin{array}{rl}\mathrm{E}\left[L\right(Z+c{)}^2]& =(k_1^2+k_2^2)a{b}^3\mathrm{\text{β}}{\int }_0^{+\mathrm{\infty }}{t}^{3a-1}{\mathrm{e}}^{-t}\mathrm{d}t+\mathrm{s}\mathrm{g}\mathrm{n}\left(c\right)(k_1^2-k_2^2)a{b}^3\mathrm{\text{β}}{\int }_0^{c^{\prime }}{t}^{3a-1}{\mathrm{e}}^{-t}\mathrm{d}t\\ & +2(k_1^2-k_2^2)a{b}^{2}c\mathrm{\text{β}}{\int }_{c^{\prime }}^{+\mathrm{\infty }}{t}^{2a-1}{\mathrm{e}}^{-t}\mathrm{d}t\\ & +(k_1^2+k_2^2)ab{c}^2\mathrm{\text{β}}{\int }_0^{+\mathrm{\infty }}{t}^{a-1}{\mathrm{e}}^{-t},\mathrm{d}t+\mathrm{s}\mathrm{g}\mathrm{n}\left(c\right)(k_1^2-k_2^2)ab{c}^2\mathrm{\text{β}}{\int }_0^{c^{\prime }}{t}^{a-1}{\mathrm{e}}^{-t}\mathrm{d}t\\ & =(k_1^2+k_2^2)a{b}^3\mathrm{\text{β}}\mathrm{\Gamma }\left(3a\right)+\mathrm{s}\mathrm{g}\mathrm{n}\left(c\right)(k_1^2-k_2^2)a{b}^3\mathrm{\text{β}}\gamma (3a,c^{\prime })\\ & +2(k_1^2-k_2^2)a{b}^{2}c\mathrm{\text{β}}\mathrm{\Gamma }(2a,c^{\prime })\\ & +(k_1^2+k_2^2)ab{c}^2\mathrm{\text{β}}\mathrm{\Gamma }\left(a\right)+\mathrm{s}\mathrm{g}\mathrm{n}\left(c\right)(k_1^2-k_2^2)ab{c}^2\mathrm{\text{β}}\gamma (a,c^{\prime }),\end{array}$$

where $c^{\prime }:=\mid c/b{\mid }^{1/a}$. Therefore, for any $c\in \mathbb{R}$, we have

$$\begin{array}{rl}\mathrm{E}\left[L\right(Z+c{)}^2]& =\frac{(k_1^2+k_2^2)c^2}{2}+\mathrm{s}\mathrm{g}\mathrm{n}\left(c\right)\frac{(k_1^2-k_2^2)c^2}{2\mathrm{\Gamma }\left(a\right)}\gamma \left(a,c^{\prime }\right)+\frac{(k_1^2-k_2^2)bc}{\mathrm{\Gamma }\left(a\right)}\mathrm{\Gamma }\left(2a,c^{\prime }\right)\\ & +\frac{(k_1^2+k_2^2)b^2\mathrm{\Gamma }\left(3a\right)}{2\mathrm{\Gamma }\left(a\right)}+\mathrm{s}\mathrm{g}\mathrm{n}\left(c\right)\frac{(k_1^2-k_2^2)b^2}{2\mathrm{\Gamma }\left(a\right)}\gamma \left(3a,c^{\prime }\right).\end{array}$$

Moreover, from Appendix A.1, we have

$$\begin{array}{rl}\mathrm{E}\left[L\right(Z+c){]}^2& =\frac{(k_1-k_2{)}^2{c}^2}{4}+\frac{(k_1^2-k_2^2)c\mid c\mid }{2\mathrm{\Gamma }\left(a\right)}\gamma \left(a,c^{\prime }\right)+\frac{(k_1^2-k_2^2)bc}{2\mathrm{\Gamma }\left(a\right)}\mathrm{\Gamma }\left(2a,c^{\prime }\right)\\ & +\frac{(k_1+k_2{)}^{2}b\mid c\mid }{2\mathrm{\Gamma }(a{)}^2}\gamma \left(a,c^{\prime }\right)\mathrm{\Gamma }\left(2a,c^{\prime }\right)\\ & +\frac{(k_1+k_2{)}^2{c}^2}{4\mathrm{\Gamma }(a{)}^2}\gamma {\left(a,c^{\prime }\right)}^2+\frac{(k_1+k_2{)}^2{b}^2}{4\mathrm{\Gamma }(a{)}^2}\mathrm{\Gamma }{\left(2a,c^{\prime }\right)}^2.\end{array}$$

Therefore, for any $c\in \mathbb{R}$, we have

$$\begin{array}{rl}\mathrm{V}\left[L\right(Z+c\left)\right]& =\mathrm{E}\left[L\right(Z+c{)}^2]-\mathrm{E}[L(Z+c){]}^2\\ & =\frac{(k_1+k_2{)}^2{c}^2}{4}+\frac{(k_1^2-k_2^2)bc}{2\mathrm{\Gamma }\left(a\right)}\mathrm{\Gamma }\left(2a,c^{\prime }\right)\\ & -\frac{(k_1+k_2{)}^{2}b\mid c\mid }{2\mathrm{\Gamma }(a{)}^2}\gamma \left(a,c^{\prime }\right)\mathrm{\Gamma }\left(2a,c^{\prime }\right)\\ & -\frac{(k_1+k_2{)}^2{c}^2}{4\mathrm{\Gamma }(a{)}^2}\gamma {\left(a,c^{\prime }\right)}^2-\frac{(k_1+k_2{)}^2{b}^2}{4\mathrm{\Gamma }(a{)}^2}\mathrm{\Gamma }{\left(2a,c^{\prime }\right)}^2\\ & +\frac{(k_1^2+k_2^2)b^2\mathrm{\Gamma }\left(3a\right)}{2\mathrm{\Gamma }\left(a\right)}+\mathrm{s}\mathrm{g}\mathrm{n}\left(c\right)\frac{(k_1^2-k_2^2)b^2}{2\mathrm{\Gamma }\left(a\right)}\gamma \left(3a,c^{\prime }\right).\end{array}$$

Appendix 2. Inequalities for the gamma and the incomplete gamma functions

Here, we prove the following lemma used in Theorem 3.1.

Lemma A.1

For a>0 and $x>0,$

$$x^a\gamma (a,x{)}^2-x^a\mathrm{\Gamma }(a{)}^2+2\gamma (a,x)\mathrm{\Gamma }(2a,x)>0.$$

To prove Lemma A.1, we need the following three lemmas:

Lemma A.2

For $a>0,$

$$2\mathrm{\Gamma }\left(2a\right)-a\mathrm{\Gamma }(a{)}^2>0.$$

Lemma A.3

For a>0 and $x\ge 0,$

$$a\gamma (a,x)\ge x^a{\mathrm{e}}^{-x}.$$

Lemma A.4

For a>0 and $b\in \mathbb{R},$

$$\underset{x\to +\mathrm{\infty }}{lim}{x}^b\mathrm{\Gamma }(a,x)=0.$$

Proof Proof of Lemma A.2 —

First, we prove

$$\sum _{n=1}^{\mathrm{\infty }}\frac{1}{n(2n-1)}=2\log 2.$$ (A1)

Let $S_n:=\sum _{k=1}^n\left(1/k(2k-1)\right)$. Accordingly, we have

$$\begin{array}{rl}{S}_n& =\sum _{k=1}^n\left(\frac{2}{2k-1}-\frac{1}{k}\right)\\ & =2\sum _{k=1}^n\frac{1}{2k-1}-\sum _{k=1}^n\frac{1}{k}\\ & =2\sum _{k=1}^n\frac{1}{2k-1}+\left(2\sum _{k=1}^n\frac{1}{2k}-2\sum _{k=1}^n\frac{1}{2k}\right)-\sum _{k=1}^n\frac{1}{k}\\ & =2\left(\sum _{k=1}^n\frac{1}{2k-1}+\sum _{k=1}^n\frac{1}{2k}\right)-2\sum _{k=1}^n\frac{1}{k}\\ & =2\sum _{k=1}^{2n}\frac{1}{k}-2\sum _{k=1}^n\frac{1}{k}\\ & =2\sum _{k=n+1}^{2n}\frac{1}{k}\\ & =2\sum _{k=1}^n\frac{1}{k+n}.\end{array}$$

Therefore, we obtain

$$\begin{array}{rl}\underset{n\to \mathrm{\infty }}{lim}{S}_n& =2\underset{n\to \mathrm{\infty }}{lim}\sum _{k=1}^n\frac{1}{k+n}\\ & =2\underset{n\to \mathrm{\infty }}{lim}\frac{1}{n}\sum _{k=1}^n\frac{1}{1+\frac{k}{n}}\\ & ={\int }_0^1\frac{1}{1+x}\mathrm{d}x\\ & =2\log 2.\end{array}$$

Thus, Equation (A1) is proved.

Next, by using Equation (A1), we prove

$$4^a\mathrm{\Gamma }\left(a+\frac{1}{2}\right)>\sqrt{\pi }\mathrm{\Gamma }(a+1)$$ (A2)

for a>0. Let

$$g\left(a\right):=\frac{4^a\mathrm{\Gamma }\left(a+\frac{1}{2}\right)}{\sqrt{\pi }\mathrm{\Gamma }(a+1)}.$$

To prove $g\left(a\right)>1$ for a>0, we use the following formula [2, p. 13, Theorem 1.2.5]:

$$\frac{\mathrm{d}}{\mathrm{d}x}\log \mathrm{\Gamma }\left(x\right)=\frac{{\mathrm{\Gamma }}^{\mathrm{\prime }}\left(x\right)}{\mathrm{\Gamma }\left(x\right)}=-{\gamma }_0+\sum _{n=1}^{\mathrm{\infty }}\left(\frac{1}{n}-\frac{1}{x+n-1}\right),$$

where ${\gamma }_0$ is Euler's constant. Taking the logarithmic derivative of $g\left(a\right)$, from the above formula, we have

$$\begin{array}{rl}\frac{\mathrm{d}}{\mathrm{d}a}\log g\left(a\right)& =2\log 2+\frac{\mathrm{d}}{\mathrm{d}a}\log \mathrm{\Gamma }\left(a+\frac{1}{2}\right)-\frac{\mathrm{d}}{\mathrm{d}a}\log \mathrm{\Gamma }(a+1)\\ & =2\log 2+\sum _{n=1}^{\mathrm{\infty }}\left(\frac{1}{n}-\frac{1}{a-\frac{1}{2}+n}\right)-\sum _{n=1}^{\mathrm{\infty }}\left(\frac{1}{n}-\frac{1}{a+n}\right)\\ & =2\log 2-\frac{1}{2}\sum _{n=1}^{\mathrm{\infty }}\frac{1}{(a+n)\left(a-\frac{1}{2}+n\right)}\\ & >2\log 2-\frac{1}{2}\sum _{n=1}^{\mathrm{\infty }}\frac{1}{n\left(n-\frac{1}{2}\right)}\\ & =2\log 2-\sum _{n=1}^{\mathrm{\infty }}\frac{1}{n(2n-1)}\end{array}$$

for a>0. Moreover, from Equation (A1), we obtain $\left(\mathrm{d}/\mathrm{d}a\right)\log g\left(a\right)>0$ for a>0. This leads to $\left(\mathrm{d}/\mathrm{d}a\right)g\left(a\right)>0$ for a>0. Equation (A2) follows from this and $g\left(0\right)=1$.

Using Equation (A2) and the formula [2, p. 22, Theorem 6.5.1]

$$\mathrm{\Gamma }\left(2a\right)=\frac{2^{2a-1}}{\sqrt{\pi }}\mathrm{\Gamma }\left(a\right)\mathrm{\Gamma }\left(a+\frac{1}{2}\right),$$

we complete the proof of Lemma A.2 as follows:

$$\begin{array}{rl}2\mathrm{\Gamma }\left(2a\right)-a\mathrm{\Gamma }(a{)}^2& =\frac{2^{2a}}{\sqrt{\pi }}\mathrm{\Gamma }\left(a\right)\mathrm{\Gamma }\left(a+\frac{1}{2}\right)-\mathrm{\Gamma }\left(a\right)\mathrm{\Gamma }(a+1)\\ & =\frac{1}{\sqrt{\pi }}\mathrm{\Gamma }\left(a\right)\left\{4^a\mathrm{\Gamma }\left(a+\frac{1}{2}\right)-\sqrt{\pi }\mathrm{\Gamma }(a+1)\right\}\\ & >0.\end{array}$$

Proof Proof of Lemma A.3 —

For a>0 and $x\ge 0$, we define

$$u(a,x):=a\gamma (a,x)-x^a{\mathrm{e}}^{-x}.$$

Then, we have

$$\frac{\mathrm{d}}{\mathrm{d}x}u(a,x)=x^a{\mathrm{e}}^{-x}\ge 0.$$

The lemma follows from this and $u(a,0)=0$.

Proof Proof of Lemma A.4 —

When $b\le 0$, the statement easily follows from the definition of $\mathrm{\Gamma }(a,x)$. When b>0, we use L'Hôpital's rule to obtain

$$\begin{array}{rl}\underset{x\to +\mathrm{\infty }}{lim}\frac{\mathrm{\Gamma }(a,x)}{x^{-b}}& =\underset{x\to +\mathrm{\infty }}{lim}\frac{x^{a-1}{\mathrm{e}}^{-x}}{b{x}^{-b-1}}\\ & =\underset{x\to +\mathrm{\infty }}{lim}\frac{x^{a+b}{\mathrm{e}}^{-x}}{b}\\ & =0.\end{array}$$

Proof Proof of Lemma A.1 —

For a>0 and $x\ge 0$, we define

$$y_1(a,x):=x^a\gamma (a,x{)}^2-x^a\mathrm{\Gamma }(a{)}^2+2\gamma (a,x)\mathrm{\Gamma }(2a,x).$$

Let us prove $y_1(a,x)>0$ (a>0, x>0). For a>0 and $x\ge 0$, we define

$$\begin{array}{rl}{y}_2(a,x)& :=a\gamma (a,x{)}^2-a\mathrm{\Gamma }(a{)}^2+2{\mathrm{e}}^{-x}\mathrm{\Gamma }(2a,x);\\ y_3(a,x)& :=a{x}^{a-1}\gamma (a,x)-\mathrm{\Gamma }(2a,x)-x^{2a-1}{\mathrm{e}}^{-x};\\ y_4(a,x)& :=a(a-1)\gamma (a,x)+x^a{\mathrm{e}}^{-x}(2x+1-a).\end{array}$$

Then, we have

$$\begin{array}{rl}\frac{\mathrm{d}}{\mathrm{d}x}{y}_1(a,x)& =x^{a-1}{y}_2(a,x);\\ \frac{\mathrm{d}}{\mathrm{d}x}{y}_2(a,x)& =2{\mathrm{e}}^{-x}{y}_3(a,x);\\ \frac{\mathrm{d}}{\mathrm{d}x}{y}_3(a,x)& =x^{a-2}{y}_4(a,x);\\ \frac{\mathrm{d}}{\mathrm{d}x}{y}_4(a,x)& =x^a{\mathrm{e}}^{-x}(3a+1-2x).\end{array}$$

From these relations, we find that the signs of $\left(\mathrm{d}/\mathrm{d}x\right)y_i(a,x)$ and $y_{i+1}(a,x)$ (i = 1, 2, 3) are the same for a>0 and $x>0$. Let $p_i\left(a\right)$ (i = 2, 3, 4) be the value of x satisfying $y_i(a,x)=0$. It is easily verified that $\underset{x\to 0+}{lim}\left(\mathrm{d}/\mathrm{d}x\right)y_4(a,x)=\underset{x\to +\mathrm{\infty }}{lim}\left(\mathrm{d}/\mathrm{d}x\right)y_4(a,x)=\underset{x\to 0+}{lim}{y}_4(a,x)=0$ and $\underset{x\to +\mathrm{\infty }}{lim}{y}_4(a,x)=a(a-1)\mathrm{\Gamma }\left(a\right)$ for a>0. Therefore, from the first derivative test, we obtain Tables 2 and 3. Moreover, using Lemmas A.3 and A.4 and L'Hôpital's rule, we obtain

$$\begin{array}{rl}& \underset{x\to 0+}{lim}\frac{\mathrm{d}}{\mathrm{d}x}{y}_3(a,x)=\left\{\begin{array}{ll}\mathrm{\infty }& (0<a<1),\\ 0& (a\ge 1),\end{array}\right.\underset{x\to +\mathrm{\infty }}{lim}\frac{\mathrm{d}}{\mathrm{d}x}{y}_3(a,x)=\left\{\begin{array}{ll}0& (0<a<2),\\ 2& (a=2),\\ \mathrm{\infty }& (a>2),\end{array}\right.\\ & \underset{x\to 0+}{lim}{y}_3(a,x)=-\mathrm{\Gamma }\left(2a\right)(a>0),\underset{x\to +\mathrm{\infty }}{lim}{y}_3(a,x)=\left\{\begin{array}{ll}0& (0<a<1),\\ 1& (a=1),\\ \mathrm{\infty }& (a>1),\end{array}\right.\\ & \underset{x\to 0+}{lim}\frac{\mathrm{d}}{\mathrm{d}x}{y}_2(a,x)=-2\mathrm{\Gamma }\left(2a\right)(a>0),\underset{x\to +\mathrm{\infty }}{lim}\frac{\mathrm{d}}{\mathrm{d}x}{y}_2(a,x)=0(a>0),\\ & \underset{x\to 0+}{lim}{y}_2(a,x)=2\mathrm{\Gamma }\left(2a\right)-a\mathrm{\Gamma }\left(a{)}^2\right(a>0),\underset{x\to +\mathrm{\infty }}{lim}{y}_2(a,x)=0(a>0),\\ & \underset{x\to 0+}{lim}\frac{\mathrm{d}}{\mathrm{d}x}{y}_1(a,x)=\left\{\begin{array}{ll}\mathrm{\infty }& (0<a<1),\\ 1& (a=1),\\ 0& (a>1),\end{array}\right.\underset{x\to +\mathrm{\infty }}{lim}\frac{\mathrm{d}}{\mathrm{d}x}{y}_1(a,x)=0(a>0),\\ & \underset{x\to 0+}{lim}{y}_1(a,x)=0(a>0),\underset{x\to +\mathrm{\infty }}{lim}{y}_1(a,x)=0(a>0).\end{array}$$

From these results, Lemma A.2, and the fact that the signs of $\left(\mathrm{d}/\mathrm{d}x\right)y_i(a,x)$ and $y_{i+1}(a,x)$ (i = 1, 2, 3) are the same for a>0 and x>0, we obtain Tables 4 and 5. From Tables 4 and 5, we can verify that $y_1(a,x)>0$ holds for $a>0$ and x>0. This completes the proof of the lemma.

Table A1. Case of 0<a<1.

x	0	···	$\frac{3a+1}{2}$	···	$p_4\left(a\right)$	···	$+\mathrm{\infty }$
$\frac{\mathrm{d}}{\mathrm{d}x}{y}_4(a,x)$	0	+	0	−	−	−	0
$y_4(a,x)$	0	+	+	+	0	−	−

Table A2. Case of $a\ge 1$.

x	0	···	$\frac{3a+1}{2}$	···	$+\mathrm{\infty }$
$\frac{\mathrm{d}}{\mathrm{d}x}{y}_4(a,x)$	0	+	0	−	0
$y_4(a,x)$	0	+	+	+	$\begin{array}{c}0(a=1)\\ +(a>1)\end{array}$

Table A3. Case of 0<a<1.

x	0	···	$p_2\left(a\right)$	···	$p_3\left(a\right)$	···	$p_4\left(a\right)$	···	$+\mathrm{\infty }$
$\frac{\mathrm{d}}{\mathrm{d}x}{y}_3(a,x)$	$+\mathrm{\infty }$	+	+	+	+	+	0	−	0
$y_3(a,x)$	−	−	−	−	0	+	+	+	0
$\frac{\mathrm{d}}{\mathrm{d}x}{y}_2(a,x)$	−	−	−	−	0	+	+	+	0
$y_2(a,x)$	+	+	0	−	−	−	−	−	0
$\frac{\mathrm{d}}{\mathrm{d}x}{y}_1(a,x)$	+	+	0	−	−	−	−	−	0
$y_1(a,x)$	0	+	+	+	+	+	+	+	0

Table A4. Case of $a\ge 1$.

x	0	···	$p_2\left(a\right)$	···	$p_3\left(a\right)$	···	$+\mathrm{\infty }$
$\frac{\mathrm{d}}{\mathrm{d}x}{y}_3(a,x)$	0	+	+	+	+	+	$\begin{array}{c}0(a<2)\\ +(a=2)\\ +\mathrm{\infty }(a>2)\end{array}$
$y_3(a,x)$	−	−	−	−	0	+	0
$\frac{\mathrm{d}}{\mathrm{d}x}{y}_2(a,x)$	−	−	−	−	0	+	0
$y_2(a,x)$	+	+	0	−	−	−	0
$\frac{\mathrm{d}}{\mathrm{d}x}{y}_1(a,x)$	+	+	0	−	−	−	0
$y_1(a,x)$	0	+	+	+	+	+	0

Appendix 3. Other simulations

We conducted three other simulations of our method using actual data, i.e. ‘cars’ of the R dataset package. This dataset includes the speeds and stopping distances of automobiles.

We separated the dataset into training and test datasets as follows: The even-numbered data were selected as the training dataset, and the odd-numbered data were the test dataset. The regression coefficients of $y=ax+b$ were estimated as $\hat{a}=3.80$ and $\hat{b}=-16.93$ by using the least squares method, and unbiased sample variance of the error was 159.31. We obtained a prediction $\hat{y}=\hat{a}x+\hat{b}$ and $z=\hat{y}-y\sim \mathcal{N}(0,159.31)$. Fix the conditions as $k_1=1$ and $k_2=5$. Then, C = 16.99 by Equation (6), and the estimated solution to the minimization problem is $(\tilde{a},\tilde{b})=(5.33,-19.33)$. Figure A1 plots (i) $\hat{y}=\hat{a}x+\hat{b}$, (ii) $\hat{y}+C$, and (iii) $\tilde{y}=\tilde{a}x+\tilde{b}$.

Figure A.1.

Scatter plots and plots of (i), (ii), and (iii). (a) Scatter plot for the training data and plots of (i), (ii), and (iii). (b) Scatter plot for the test data and plots of (i), (ii), and (iii).

Table 6 lists the sample means and sample variances of the loss for the training and test data when $k_1=1$ and $k_2=5$.

Table A5. Sample means and sample variances of the loss for the training and test data when $k_1=1$ and $k_2=5$.

	Mean (training)	Mean (test)	Variance (training)	Variance (test)
(i)	31.42	42.54	1307.61	4065.98
(ii)	20.04	31.74	216.34	1836.93
(iii)	19.56	30.40	169.21	1540.48

Figure A2 plots the sample means of the loss for the training and test data for (i), (ii), and (iii) respect to $k_2$.

Figure A.2.

Plots of sample means for the training and the test data. (a) Sample means for the training data. (b) Sample means for the test data.

Moreover Figure A3 plots the sample variances of the loss for the training and test data for (i), (ii), and (iii) respect to $k_2$.

Figure A.3.

Plots of sample variances for the training and the test data. (a) Sample variance for the training data. (b) Sample variances for the training data. (c) Sample variance for the test data. (d) Sample variances for the test data.

Next, we separated the cars dataset into training and test datasets as follows: The 1st to 25th data of the cars dataset were selected as the training dataset (the data of the cars dataset are arranged in ascending order of speed), and the 26th to 50th data were the test dataset. The regression coefficients of y = ax + b were estimated to be $\hat{a}=3.29$ and $\hat{b}=-10.00$ by using the least squares method, and unbiased sample variance of the error was 178.77. Figure A4 plots the sample means of the loss for the training and test data for (i), (ii), and (iii) respect to $k_2$.

Figure A.4.

Plots of sample means for the training and test data. (a) Sample means for the training data. (b) Sample means for the test data.

Moreover, Figure A5 plots the sample variances of the loss for the training and test data for (i), (ii), and (iii) for respect to $k_2$.

Figure A.5.

Plots of sample variances for the training and test data.(a) Sample variance for the training data. (b) Sample variances for the training data. (c) Sample variance for the test data. (d) Sample variances for the test data.

Finally, we separated the cars into training and test datasets as follows: The 26th to 50th data were selected as the training dataset, and the 26th to 50th data were the test dataset. The regression coefficients of $y=ax+b$ were estimated to be $\hat{a}=5.15$ and $\hat{b}=-42.04$ by using the least squares method, and the unbiased sample variance of the error was 277.30. Figure A6 plots the sample means of the loss for the training and test data for (i), (ii), and (iii) respect to $k_2$.

Figure A.6.

Plots of sample means for the training and the test data. (a) Sample means for the training data. (b) Sample means for the test data.

Moreover, Figure A7 plots the sample variances of the loss for the training and test data for (i), (ii), and (iii) respect to $k_2$.

Figure A.7.

Plots of sample variances for the training and test data. (a) Sample variance for the training data. (b) Sample variances for the training data. (c) Sample variance for the test data. (d) Sample variances for the test data.

Disclosure statement

No potential conflict of interest was reported by the author(s).