A Cautionary Note on Beta Families of Distributions and the Aliases Within

Abstract

In this note we examine the four parameter beta family of distributions in the context of the beta-normal and beta-logistic distributions. In the process we highlight the concept of numerical and limiting alias distributions, which in turn relate to numerical instabilities in the numerical maximum likelihood fitting routines for these families of distributions. We conjecture that the numerical issues pertaining to fitting these multiparameter distributions may be more widespread than has originally been reported across several families of distributions.

1. Introduction

There has been a long standing tradition in mathematical statistics of extending classic univariate two parameter families of distributions by adding a 3rd, 4th or even 5th parameter, e.g. the exponentiated Weibull and the epsilon-skew-normal distributions of Mudholkar and Hutson (1996,2000) are oft-cited distributions defined by adding a 3rd parameter to the well-known Weibull and normal distributions, respectively. There are literally hundreds of examples of such papers that are too numerous to cite in one article. The general goal to this approach of extending classical models is to add additional paramters, similar to a nonparametric kernel smoothing type model, while retaining some of the advantages of the parametric approach, e.g. direct inference about the data through the parameters or the ability to link a give parameter to a set of covariates via a regression type frame-work, e.g. see Hutson (2004). Oftentimes, parametric model fit is tested via additional parameters given the model of interest is a subclass of the family, e.g. testing a two parameter Weibull fit as a submodel of the three parameter exponentiated Weibull family, e.g. see Mudholkar and Hutson (1996). The proliferation of these 4 and 5 parameter families of distributions is most likely due to the rapid expansion of easy-to-use statistical maximization software routines. The downside to these types of models are the general subtleties of the underlying numerical methods, such as picking appropriate starting values in the fitting routine relative to non-identifiability issues (in the numerical sense), as well as the sometimes high degree of interdependence of the maximum likelihood estimates relative to large sample approximations for inferential procedures given a finite sample setting.

We turn our focus on one very oft-cited approach for extending classical models through the incorporation of two additional parameters, which is due in part to the work of M.C. Jones (2004), and consists of embedding the cumulative distribution function (c.d.f.) of interest, denoted F, within the beta c.d.f., say B, described in more detail below. There are strong parallels to this approach to the single order statistic distribution functional form. Within the Test volume that contains the Jones (2004) article there is a discussion and rebuttal section relative to the merits of this approach for generating new classes of distributions. The paper also includes several important historical references to prior extensions to classical distributions and serves as good history of this type of approach. Even though we focus on this particular family of distributions this note should serve as a cautionary alarm relative to the general methodology of adding additional parameters to parametric families.

The beta family of distributions, e.g. see Jones (2004), arose from the work of Eugene et al. (2002) who first introduced the specific case of the beta-normal distribution. As background relative to defining specific distributions within this family first let z = (x − μ)/σ,

$$F\text{(}x\text{) = }{F}_{\text{0}}\text{(}z\text{),}$$ (1.1)

$$f(x)=\frac{1}{\sigma }{f}_0(z),$$ (1.2)

$$Q(u)=\mu +\sigma Q_0(u),$$ (1.3)

where μ and σ are the location and scale parameters. The beta-normal and beta-logistic symmetric location-scale base families of distributions, are defined as per Table 1. If we let B_p,q(x) denote the beta c.d.f., b_p,q(x) denote the beta p.d.f. and $B_{p,q}^{-1}(u)$ denote the beta quantile function then the corresponding c.d.f., p.d.f. and quantile function for the beta family of distributions take the form

$$G(x)=B_{p,q}\left[F_0(z)\right],$$ (1.4)

$$g(x)=\frac{1}{\sigma }{b}_{p,q}\left[F_0(z)\right]f_0(z),$$ (1.5)

$$P(u)=\mu +\sigma Q_0\left[B_{p,q}^{-1}(u)\right],$$ (1.6)

respectively, where the form of the beta p.d.f. used in this note is given as $b_{p,q}(t)=\frac{1}{\beta (p,q)}{t}^{p-1}{(1-t)}^{q-1}$ and β(p, q) is the normalizing constant. The c.d.f. and quantile function follow suit. The results in this note extend to several other symmetric base distributions, but for the purpose of illustration we focus on the beta-normal and beta-logistic distributions.

Table 1:

Common Standardized Symmetric Models

Distribution	F0(x)	f0(x)	Q0(u)
Normal	Φ(x)	$\text{exp}\left(-x^2/2\right)/\sqrt{2\pi }$	Φ−1(u)
Logistic	l/(l + exp (−x))	exp(x)/(l + exp(x))2	log(u/(l − u))

A motivation for this note pertains to our own empirical observations and to two passages, one from Jones (2004) and one from Eugene et al. (2002), respectively. Jones states that “Incorporation of four-parameter families of distributions in modelling exercises automatically allows for the effects of skewness and (heavy) tail weights. Of course, for small to moderate sample sizes, there will typically be insufficient information in the data to estimate the parameters a and b [in our notation p and q] well (very large sample sizes will normally be necessary for that). But also this will not usually be very important. Interest would normally be in location and perhaps scale parameters …..” and Eugene et al. (2002) who state the following with respect to beta-normal fitting routines: “A simple procedure for the choice of initial values (α, β, μ, σ) [in our notation p, q, μ, σ] is as follows: Evaluate the likelihood at the points $(\alpha ,\beta ,\mu ,\sigma )=(1,1,\overline{x},s),(\alpha ,\beta ,\mu ,\sigma )=(0.1,0.1,\overline{x},s)$ and start the iteration from the point that has higher likelihood. Our experience indicates that using this technique is sufficiently fast. For most of the data sets, the Newton Raphson algorithm converged in less than 20 iterations.”

We originally noticed numerical fitting issues for values of p and q near 1 in terms of the numerical consistency of the parameter estimates given a symmetric base distribution such as normal or logistic for substantially large sample sizes. This is of particular importance if one is looking to reduce the model via the test H₀ : p = 1, q = 1 or via confidence intervals about p and q. Heuristically, if one looks at the form of the p.d.f. g(x) at (1.5) there are two components of scale at play: 1) the standard scale parameter σ and 2) the beta normalizing constant β(p, q). For near symmetric distributions this lends itself to somewhat of an identifiability issue in terms of the numerical fitting routines relative to scale, which we will illustrate. A general fix to this problem is to constrain the model such that σ is a function of p and/or q with the same relative goodness-of-fit. This appears to be a non-issue for asymmetric base distributions such as the Weibull distribution, which extends to the beta-Weibull distribution, e.g. see Famoye et al. (2005) who introduced the distribution and Cordeiroa et al. (2013) who provide additional mathematical results pertaining to the beta-Weibull distribution.

The other interesting issue, which arose while investigating the beta family of distributions is the concept of distributional aliases from a distance perspective, which we outline in the next section. For a given base distribution parameterization, say standard normal, we found that there were sets of vastly different p and q beta values for which the two distributions were virtually identical from a numerical standpoint, i.e. no matter what the sample size we could finds pairs of distributions that were so close in terms of having virtually the same probability measure that a typical fitting routine would not be able to distinguish between the two sets of parameters, not even for massively large sample sizes. In fact, in the next section we prove the counterintuitive result that if we let p = q and σ go to infinity jointly at a certain rate the limiting distribution is the normal distribution within the beta-normal family of distributions.

The issues outlined above are quite relevant in the context of testing for subclasses of distributions within the beta family of distributions, e.g. testing for normality as a submodel of the beta-normal distribution. In addition, as we will illustrate theoretically, large sample variance approximations of maximum likelihood estimators are vastly different between a given distribution and its respective alias, even when the actual density functions are virtually identical. This in turn impacts inferential procedures around such quantities as E(X) (= μ in the symmetric case). More recently the beta-normal has been incorporated into other research fields such as psychometric testing, crop yield and risk assessment, e.g. see Ranger and Kuhn (2015), Hennessy (2009) and Razzaghi (2009), respectively, without regards to the issues outlined above.

In Section 2 we provide an approach to determining aliases in the beta family of distributions and provide some relevant limiting results. In Section 3 we provide the general form for the likelihood function and its related quantities in terms of a general beta family of distributions. A small simulation study is carried forth to demonstrate the numerical instabilities relative to maximum likelihood estimation between a given distribution and its alias at both moderate to large samples. We then provide some concluding remarks.

2. Aliases

In this section we illustrate how for a fixed set of parameter values for a given beta-normal or beta-logistic distribution there are alias distributions that are virtually identical relative to minimizing the Kulback-Liebler distance

$$D={\int }_{-\infty }^{\infty }\text{log}\left(\frac{f(x)}{g(x)}\right)f(x)dx.$$ (2.1)

The underlying reason why we can find two symmetrical distributions with very distinct parameter values was pointed out in the introduction in that the standard scale parameter σ and the beta-normalizing constant β(p, q) are in a sense both scale factors. Interestingly, the major problem here appears to occur with respect to the most important case of choosing between a normal model and a beta-normal model.

As an example, let us set μ = 0, σ = 1 and p = q = 1 for the beta-normal distribution, i.e. the standard normal density. We can arbitrarily set μ = 0 and p = q = 4 and minimize the Kulback-Liebler distance at (2.1) with respect to σ. In this scenario we arrive at the alias setting σ = 2.180 with D = 0.000015. If we set μ = 0 and p = q = 10 we arrive at the alias to the standard normal distribution by setting σ = 3.518 with D = 0.0000034, even smaller than before, i.e. as p = q move away from 1 the distance D between the standard normal and the beta-normal distributions decreases. The three density functions are plotted together in Figure 1. This observation led to a counterintuitive mathematical result. In fact we can show that in a limiting sense we can prove the convergence of the beta-normal to standard normal if we let p = q go to ∞ in a specific fashion.

Figure 1:

Standard normal distribution overlayed with its alias beta-normal distributions μ = 0, p = q = 4 and σ = 2.180 and μ = 0, p = q = 10 and σ = 3.518.

Towards this end, and without loss of generality with respect to the parameter μ, let p.d.f. f(x) correspond to the beta-normal density function with μ = 0, σ = 1 and p = q = 1 and let the p.d.f. g(x) correspond to the symmetric beta-normal density function with μ = 0, σ > 0 and p = q > 0. An interesting limiting result occurs if for p.d.f. g we let p = q (symmetric beta-normal) and link σ to p as follows:

$$\sigma =\frac{4^{1-p}}{\beta (p,p)}=\frac{2\text{Γ}\left(p+\frac{1}{2}\right)}{\sqrt{\pi }\text{Γ}(p)}.$$ (2.2)

This is the point where the value for σ at (2.2) is given by solving f(0) = g(0).

If we then substitute σ at (2.2) into g then the p.d.f. g is a symmetric beta-normal density function with μ = 0, $\sigma =\frac{4^{1-p}}{\beta (p,p)}=\frac{2\text{Γ}\left(p+\frac{1}{2}\right)}{\sqrt{\pi }\text{Γ}(p)}$ and q = p such that g has the single parameter p. Then the

$$\underset{p\to \infty }{\text{lim}}\text{log}\left(\frac{f(x)}{g(x)}\right)=0,$$ (2.3)

where f denotes the standard normal density function with μ = 0, σ = 1 and p = q = 1. Hence the Kulback-Liebler distance go to 0 for extremely divergent values of p = q different from 1. This results holds for general and equivalent μ ≠ 0 values for the classic normal densities versus symmetric beta normal densities.

The result at (2.3) follows by noting that

$$\text{log}\left(\frac{f(x)}{g(x)}\right)=-\frac{x^2}{2}+\frac{x^2}{8}\pi \gamma {(p)}^2+(1-p)\text{log}\left[1-Erf{\left(x\frac{\sqrt{2\pi }}{4}\gamma (p)\right)}^2\right],$$ (2.4)

where γ(p) = Γ(p)/Γ(p + 1/2) and $Erf(x)=2{\int }_0^x\exp \left(-t^2\right)dt/\sqrt{\pi }$ is the error function. The rest of the steps in the proof are given in Appendix A. In this case we present in Appendix A a detailed proof of that D = O(p⁻¹) → 0, as p → ∞, where D is defined in (2.1).

For the more general symmetric beta-normal case let μ = 0, σ = 1 and p₀ = q₀ be the beta parameters for f for a beta-normal comparison and let g have parameterization μ = 0, σ = 1 and p = q then

$$\sigma =\frac{4^{p_0-p}\beta \left(p_0,p_0\right)}{\beta (p,p)}=\frac{\text{Γ}\left(p+\frac{1}{2}\right)\text{Γ}\left(p_0\right)}{\text{Γ}\left(p_0+\frac{1}{2}\right)\text{Γ}(p)}.$$ (2.5)

Using similar arguments to that of the standard normal case for fixed x we arrive at

$$\text{log}\left(\frac{f(x)}{g(x)}\right)=\frac{4x^2\text{Γ}{\left(\frac{1}{2}+p_0\right)}^2-\pi \text{Γ}{\left(p_0\right)}^2\left(x^2-2\left(p_0-1\right)\text{log}\left(1-Erf{\left(\frac{x}{\sqrt{2}}\right)}^2\right)\right)}{2\pi \text{Γ}{\left(p_0\right)}^2}+O\left(p^{-1}\right),$$ (2.6)

as p → ∞. See Appendix A for remarks pertaining to this result. This results holds for general and equivalent μ ≠ 0 values for symmetric beta-normal density functions versus asymmetric beta normal density functions. We can see from (2.6) this limit does not converge to 0. However, for any given fixed value of x the limit gets quite small for large values of p different from p₀ = q₀, which has implications in the numerical fitting process, which we describe in the next section.

For the more general symmetric beta-logistic case let μ = 0, σ = 1 and p₀ = q₀ be the beta parameters for f for a beta-logistic comparison and let g have parameterization μ = 0, σ = 1 and p = q. Using similar arguments to that of the normal case for fixed x we arrive at the limiting result between two symmetric beta-logistic density functions at fixed value for x in the form

$$\text{log}\left(\frac{f(x)}{g(x)}\right)=\frac{x^2\text{Γ}{\left(\frac{1}{2}+p_0\right)}^2}{4\text{Γ}{\left(p_0\right)}^2}+p_0(x+\text{\hspace{0.17em}log}(4)-2\text{log}[1+\text{\hspace{0.17em}exp}(x)])+O\left(p^{-1}\right).$$ (2.7)

See Appendix A for details. This results holds for general and equivalent μ ≠ 0 values for symmetric beta-logistic distributions versus symmetric beta normal distributions. The standard logistic comparison to the symmetric beta-logistic distribution does not produce the same interesting limiting argument as was presented in the beta-normal case at p₀ = 1.

Numerically we can show similar behavior in the asymmetrical cases. For example if we take a beta-normal density function f with μ = 0, σ = 1 and p = 2 and q = 4 and minimize the Kulback-Liebler distance with respect to σ and q and setting μ = 0 and p = 10 we get for values of σ = 2.147 and q = 14.288 that the distance from (2.3) is 0.00005. The plots of the two density functions are overlaid in Figure 2. In the next section we illustrate the numerical estimation issues caused by the alias phenomena.

Figure 2:

Beta-normal distribution with μ = 0, p = 2, q = 4 and σ = 1 overlayed with its alias beta-normal distributions μ = 0, p = 10, q = 14.288 and σ = 2.147.

3. Maximum Likelihood Estimation

For issues related to maximum likelihood related to the beta family of distributions let us define the vector of the parameter of interest as θ = (μ, σ, p, q) and let z_i = (x_i − μ)/σ. For an i.i.d. observed sample of size n, x₁, x₂, ⋯, x_n the log-likelihood takes the form

$$l_n(\theta )=\sum _{i=1}^{n}l\left(x_i,\mathbf{\theta }\right)$$ (3.1)

$$=\sum _{i=1}^n\left\{\text{log}{b}_{p,q}\left[F_0\left(z_i\right)\right]+\text{\hspace{0.17em}log}{f}_0\left(z_i\right)/\sigma \right\},$$ (3.2)

where

$$\text{log}{b}_{p,q}\left[F_0\left(z_i\right)\right]=\text{\hspace{0.17em}logΓ}(p+q)-\text{\hspace{0.17em}logΓ}(p)-\text{\hspace{0.17em}logΓ}(q)+(p-1)\text{log}{F}_0\left(z_i\right)+(q-1)\text{log}\left[1-F_0\left(z_i\right)\right].$$ (3.3)

The scores per observation for each component of θ, given as s_j(x_i) = ∂l(x_i, θ)/∂θ_j are provided in Appendix B in equations (B.1–B.4). Similarly the sample information per observation for each component of θ, given as i_kj(x_i) = ∂²l(x_i, θ)/∂θ_k∂θ_j are provided in the Appendix B in equations (B.5–B.14).

In Section 2 we illustrated that for the beta-normal and beta-logistic distribution we can find two alias distributions with very divergent values for p and q such that the densities are virtually identical. However, in terms of maximum likelihood estimation it is clear that the information matrix will be quite different for two alias distributions. Note that in the general form the sample information per observation for p and q, i₃₃ and i₄₄ at (B.12) and (B.14), respectively is equivalent to the expected information. As p and q go to infinity these elements tend towards 0 in the limit. The implication being that for alias families the corresponding numerical estimation procedures will become quite unstable in terms of the numerical inversion of the sample information matrix. For example, for true values of μ = 0, σ = 1 and p = q = 1 we have i₃₃ = i₄₄ = 1. For the alias distribution with μ = 0, σ = 3.518 and p = q = 10 we have i₃₃ = i₄₄ = 0.053.

In terms of inference about the parameters, two alias models will be indistinguishable in terms of numerical model fitting procedures and the actual sample data values, yet inferential quantities about the parameters, e.g. large sample confidence intervals will be vastly different. These numerical issues are compounded for beta family based regression models around the standard parameterization μ = Xβ. Similarly, the beta-normal family is used to test for subclasses of models, e.g. testing normality via testing the hypothesis H₀ : p = 1, q = 1. These tests will also be unstable given the numerical issues outlined above.

As an illustration of the numerical instability of these models, in terms of maximum likelihood estimation, we could provide countless examples. We will take a few basic examples to illustrate our point. Let us take our three alias cases from the previous section for the beta-normal distribution with case 1 setting μ = 0, σ = 1 and p = q = 1, case 2 setting μ = 0, σ = 2.18 and p = q = 4 and case 3 setting μ = 0, σ = 3.518 and p = q = 10. The plots of the three densities are given in Figure 1. We use a variation of the starting points suggested by Eugene et al. (2002) and use the true values for μ and σ for the choice of initial values (p = α, q = β, μ, σ) as follows: Evaluate the likelihood at the points (p, q, μ, σ)=(1,1,0,1), (p, q, μ, σ)=(0.1,0.1,0,1) and start the iteration with the highest likelihood.

In Table 2 we utilize Eugene’s starting values and provide the quartiles of the maximum likelihood values from 1000 model fits for sample sizes n = 50 and n = 500. As can be seen the fitted maximum likelihood estimates are vastly different than the true parameter value with the exception of μ in a few cases. Having a large sample size did not improve the instabilities. We repeated this experiment using the actual true parameter values as the starting values. The results are presented in Table 3. As expected, the results are similarly poor. The quartiles themselves were used over the mean since oftentimes there were extreme values for $\widehat{p}$ and $\widehat{q}$, which make sense given consideration our limiting result given in the previous section.

Table 2:

Simulation results for 3 symmetric alias scenarios using Eugene et al. (2002) starting points. Quartiles from the simulated maximum likelihood estimates are given in the table.

n= 50
	Case 1				Case 2				Case 3
	μ = 0, σ = 1, p = q = 1				μ = 0, σ = 2.18, p = q = 4				μ = 0, σ = 3.518, p = q = 10
	$\widehat{\mu }$	$\widehat{\sigma }$	$\widehat{p}$	$\widehat{q}$	$\widehat{\mu }$	$\widehat{\sigma }$	$\widehat{p}$	$\widehat{q}$	$\widehat{\mu }$	$\widehat{\sigma }$	$\widehat{p}$	$\widehat{q}$
Ql	−0.99	0.36	0.14	0.14	−0.96	0.37	0.14	0.14	−0.92	0.38	0.14	0.14
Q2	−0.01	0.49	0.47	0.47	−0.03	0.50	0.57	0.50	0.01	0.51	0.49	0.48
Q3	0.90	1.08	1.54	1.45	1.00	1.06	1.52	1.55	0.98	1.14	1.44	1.56
n = 500
	Case 1				Case 2				Case 3
	μ = 0, σ = 1, p = q = 1				μ = 0, σ = 2.18, p = q = 4				μ = 0, σ = 3.518, p = q = 10
	$\widehat{\mu }$	$\widehat{\sigma }$	$\widehat{p}$	$\widehat{q}$	$\widehat{\mu }$	$\widehat{\sigma }$	$\widehat{p}$	$\widehat{q}$	$\widehat{\mu }$	$\widehat{\sigma }$	$\widehat{p}$	$\widehat{q}$
Ql	−0.41	0.57	0.43	0.43	−0.40	0.59	0.50	0.49	−0.49	0.60	0.47	0.47
Q2	0.01	0.84	0.80	0.81	0.01	0.93	1.01	1.01	−0.01	0.92	0.93	0.93
Q3	0.45	1.43	1.78	1.78	0.41	1.42	1.96	1.98	0.41	1.46	1.87	1.87

Table 3:

Simulation results for 3 symmetric alias scenarios using Case 2 true parameter values as starting points. Quartiles from the simulated maximum likelihood estimates are given in the table.

n= 50
	Case 1				Case 2				Case 3
	μ = 0, σ = 1, p = q = 1				μ = 0, σ = 2.18, p = q = 4				μ = 0, σ = 3.518, p = q = 10
	$\widehat{\mu }$	$\widehat{\sigma }$	$\widehat{p}$	$\widehat{q}$	$\widehat{\mu }$	$\widehat{\sigma }$	$\widehat{p}$	$\widehat{q}$	$\widehat{\mu }$	$\widehat{\sigma }$	$\widehat{p}$	$\widehat{q}$
Ql	−1.88	0.62	0.96	0.97	−1.75	0.59	0.81	0.96	−1.84	0.58	0.92	0.82
Q2	−0.01	1.41	3.14	2.87	0.42	1.46	2.14	3.15	0.00	1.41	3.06	2.94
Q3	1.84	4.29	16.3	16.51	2.00	4.27	12.38	13.60	1.96	4.37	13.52	13.66
n = 500
	Case 1				Case 2				Case 3
	μ = 0, σ = 1, p = q = 1				μ = 0, σ = 2.18, p = q = 4				μ = 0, σ = 3.518, p = q = 10
	$\widehat{\mu }$	$\widehat{\sigma }$	$\widehat{p}$	$\widehat{q}$	$\widehat{\mu }$	$\widehat{\sigma }$	$\widehat{p}$	$\widehat{q}$	$\widehat{\mu }$	$\widehat{\sigma }$	$\widehat{p}$	$\widehat{q}$
Ql	−1.55	1.52	2.16	2.11	−1.18	1.50	2.00	2.02	−1.10	1.49	2.01	2.00
Q2	−0.04	4.23	7.64	7.66	0.03	3.80	6.97	6.70	−0.03	3.88	6.85	7.02
Q3	1.36	4.89	27.28	24.10	1.08	4.89	22.70	23.32	1.26	4.91	23.94	24.32

We repeated our simulation study for an asymmetric case given as case 4 with μ = 0, σ = 1, p = 2, q = 4 with numerical alias given as case 5 μ = 0, σ = 3.518, p = 10, q = 14.28. Figure 2 provides plots of the overlay of the two distributions. As can be seen in Tables 4 we have similar and dramatic instabilities in the model fitting, which are comparable to the symmetric case.

Table 4:

Simulation results for 2 asymmetric alias scenarios using Eugene et al. (2002) starting points. Quartiles from the simulated maximum likelihood estimates are given in the table.

n = 50
	Case 4				Case 5
	μ = 0, σ = 1, p =2, q = 4				μ = 0, σ = 3.518, p = 10, q = 14.28
	$\widehat{\mu }$	$\widehat{\sigma }$	$\widehat{p}$	$\widehat{q}$	$\widehat{\mu }$	$\widehat{\sigma }$	$\widehat{p}$	$\widehat{q}$
Ql	−0.83	0.23	0.13	0.19	−1.04	0.22	0.14	0.14
Q2	−0.20	0.29	0.42	0.96	−0.44	0.28	0.49	0.63
Q3	0.28	0.62	1.35	1.79	0.17	0.60	1.49	1.61
n = 500
	Case 4				Case 5
	μ = 0, σ = 1, p =2, q = 4				μ = 0, σ = 3.518, p = 10, q = 14.28
	$\widehat{\mu }$	$\widehat{\sigma }$	$\widehat{p}$	$\widehat{q}$	$\widehat{\mu }$	$\widehat{\sigma }$	$\widehat{p}$	$\widehat{q}$
Ql	−0.39	0.35	0.37	0.68	−0.64	0.33	0.41	0.52
Q2	−0.16	0.54	0.75	1.27	−0.39	0.49	0.78	0.94
Q3	0.03	0.84	1.59	2.58	−0.17	0.80	1.73	1.95

4. Conclusions

We have given some theoretical and simulation based results to illustrate the beta family of distributions may in fact be an overparameterized way of modeling data relative to the numerical closeness of given distributions and their respective aliases within these families. Mathematically these models make logical sense, however, when combined with numerical fitting issues there is somewhat of an identifiability issue relative to scale and shape parameters. The numerical maximum likelihood fitting issues are non-trivial in nature. Hence, we advise caution against fitting these models to real data relative to interpretation of the parameter estimates and/or any functions derived from these estimates. We conjecture that other 4 and 5 parameter parametric models may have similar underlying issues, e.g. the Kumaraswamy skew-normal distribution by Mameli (2015) or the beta skew-normal distribution by Mameli and Musio (2013). If one wishes to use the beta family of distributions we suggest linking the parameters p or q with the scale parameter σ in a meaningful way that still captures scale, skewness and tail-weight appropriately, but reduces the beta based model down to 3 parameters.

Appendix A: Asymptotic Evaluations

The asymptotic forms involve the ratio of gamma functions. The following general results evaluate the ratio of gamma functions.

Define the ratio γ = Γ(p)/Γ(p + 1/2)

Lemma 1. For $1/2<p\in \Re$, we have

$$\gamma =p^{-1/2}\left[1+\frac{1}{8p}+O\left(p^{-2}\right)\right]\text{ as }p\to \infty$$ (A.1)

and

$${(p+1/4)}^{1/2}<{\gamma }^{-1}<{\left[p-1/2+{(3/4)}^{1/2}\right]}^{1/2}.$$ (A.2)

Proof. The result (A.1) is presented in Tricomi and Erdélyi (1951). To obtain the inequality (A.2) one can use the Kershaw’s Double Inequalities shown, e.g., in Qi (2010) (see, p. 32, where in the context of Qi (2010)’s notations we denote x = p − 1/2 > 0 and s = 1/2), see also Mortici (2010, p. 426).

Consider equation (2.4). Taking into account (A.1), we use the Taylor theorem with $Erf{(x\sqrt{2\pi }\gamma /4)}^2$ around zero to represent

$$\text{log}\left[1-Erf{\left(x\frac{\sqrt{2\pi }}{4}\gamma \right)}^2\right]=-Erf{\left(x\frac{\sqrt{2\pi }}{4}\gamma \right)}^2-\frac{1}{2}Erf{\left(x\frac{\sqrt{2\pi }}{4}\gamma \right)}^4-\frac{1}{3}Erf{\left(x\frac{\sqrt{2\pi }}{4}\gamma \right)}^6{\left(1-\varpi Erf{\left(x\frac{\sqrt{2\pi }}{4}\gamma \right)}^2\right)}^{-3},$$ (A.3)

with $\varpi \in (0,1)$, where the reminder term

$$\begin{gathered} 0\le \frac{1}{3}Erf{\left(x\frac{\sqrt{2\pi }}{4}\gamma \right)}^6{\left(1-\varpi Erf{\left(x\frac{\sqrt{2\pi }}{4}\gamma \right)}^2\right)}^{-3} \\ \le \frac{1}{3}Erf{\left(x\frac{\sqrt{2\pi }}{4}\gamma \right)}^6{\left(1-Erf{\left(x\frac{\sqrt{2\pi }}{4}\gamma \right)}^2\right)}^{-3}. \end{gathered}$$

Here we use the inequality (A.2) that insures that, for arbitrary large p’s, we have $Erf\left(x\frac{\sqrt{2\pi }}{4}\gamma \right)<\text{\hspace{0.17em}1 / 2 + }c$, where 0 < c < 1/4 is a fixed constant. Thus, it is clear that (A.3), the Taylor expansion of $Erf\left(x\frac{\sqrt{2\pi }}{4}\gamma \right)$ with γ around zero and (A.1) provide that

$$\text{log}\left[1-Erf{\left(x\frac{\sqrt{2\pi }}{4}\gamma \right)}^2\right]=-{\left[x\frac{1}{\sqrt{2}}\gamma -x^3\frac{\pi \sqrt{2}}{48}{\gamma }^3+O\left({\gamma }^4\right)\right]}^2-\frac{1}{2}{\left[x\frac{1}{\sqrt{2}}\gamma -x^3\frac{\pi \sqrt{2}}{48}{\gamma }^3+O\left({\gamma }^4\right)\right]}^4=-{\left[x\frac{1}{\sqrt{2}}{p}^{-1/2}+x\frac{1}{8\sqrt{2}}{p}^{-3/2}-x^3\frac{\pi \sqrt{2}}{48}{p}^{-3/2}+O\left(p^{-2}\right)\right]}^2-\frac{1}{2}{\left[x\frac{1}{\sqrt{2}}{p}^{-1/2}\right]}^4+O\left(p^{-3}\right).$$

This and (A.1) applied to equation (2.4) imply

$$\text{log}\left(\frac{f(x)}{g(x)}\right)=-\frac{x^2}{2}+\frac{x^2}{8}\pi {\left(p^{-1/2}+\frac{1}{8}{p}^{-3/2}\right)}^2-(1-p){\left[x\frac{1}{\sqrt{2}}{p}^{-1/2}+x\frac{1}{8\sqrt{2}}{p}^{-3/2}-x^3\frac{\pi \sqrt{2}}{48}{p}^{-3/2}+O\left(p^{-2}\right)\right]}^2-\frac{(1-p)}{2}{\left[x\frac{1}{\sqrt{2}}{p}^{-1/2}\right]}^4+O\left(p^{-2}\right).$$

Then without too much algebra one finds

$$\text{log}\left(\frac{f(x)}{g(x)}\right)=\frac{x^2\left(x^2-3\right)(3-\pi )}{24p}+O\left(p^{-3/2}\right)$$

, where as in the body of the text f(x) denotes the standard normal distribution.

Consider the Kulback-Liebler measure (2.1) based on the form (2.4). In general we note that

$$D=\int \text{log}\left(\frac{f(x)}{g(x)}\right)f(x)dx\ge 0,$$ (A.4)

since

$$D=\int \text{log}\left(\frac{f(x)}{g(x)}\right)f(x)dx=-\int \text{log}\left(\frac{g(x)}{f(x)}\right)f(x)dx\ge -\text{\hspace{0.17em}log}\left(\int \frac{g(x)}{f(x)}f(x)dx\right)=0.$$

Taking into account (2.4), we have, for large values of p > 1,

$$(1-p)\int \text{log}\left[1-Erf{\left(x\frac{\sqrt{2\pi }}{4}\gamma \right)}^2\right]f(x)dx\le (p-1)\int Erf{\left(x\frac{\sqrt{2\pi }}{4}\gamma \right)}^{2}f(x)dx,$$

since, for all 0 < s < 1, log(1 − s) ≥ −s. The Taylor theorem gives

$$Erf\left(x\frac{\sqrt{2\pi }}{4}\gamma \right)=x\frac{1}{\sqrt{2}}\gamma -x^3\frac{\sqrt{2}\pi }{16}{\gamma }^3\varpi \text{exp}\left[-{\left(x\frac{\sqrt{2\pi }}{4}\gamma \varpi \right)}^2\right],\varpi \in (0,1).$$

That is

$$(1-p)\int \text{log}\left[1-Erf{\left(x\frac{\sqrt{2\pi }}{4}\gamma \right)}^2\right]f(x)dx\le (p-1)\int \left\{x^2\frac{1}{2}{\gamma }^2+x^6\frac{{\pi }^2}{128}{\gamma }^6{\varpi }^2\text{exp}\left[-2{\left(x\frac{\sqrt{2\pi }}{4}\gamma \varpi \right)}^2\right]\right\}f(x)dx\le (p-1)\int \left\{x^2\frac{1}{2}{\gamma }^2+x^6\frac{{\pi }^2}{128}{\gamma }^6\right\}f(x)dx=(p-1)\frac{1}{2}{\gamma }^2+(p-1)O\left({\gamma }^6\right).$$

Thus, by virtue of (2.4), we have

$$\begin{gathered} D\le \int \left(-\frac{x^2}{2}+\frac{x^2}{8}\pi {\gamma }^2\right)f(x)dx+(p-1)\frac{1}{2}{\gamma }^2+(p-1)O\left({\gamma }^6\right) \\ =-\frac{1}{2}+(p-1)\frac{1}{2}{\gamma }^2+O\left({\gamma }^2\right)+(p-1)O\left({\gamma }^6\right)=O\left(p^{-1}\right)\to 0, \end{gathered}$$

as p → ∞, since (A.1). This result and (A.4) lead to D = O(p⁻¹) → 0, as p → ∞.

Remark. The asymptotic results (2.6) and (2.7) can be derived in a similar manner to that mentioned above. For example, to evaluate

$$\text{log}\left[1-Erf{\left(x\frac{\sqrt{2\pi }}{4}\frac{\text{Γ}\left(p_0+1/2\right)}{\text{Γ}\left(p_0\right)}\gamma \right)}^2\right]$$

a simple transformation y = xΓ(p₀ + 1/2)/Γ(p₀) can be used for providing the proof scheme presented above.

Appendix B: Scores and Information

Scores per observation for the general beta family of distributions corresponding to the parameter vector θ = (μ, σ, p, q) corresponding to the likelihood at (3.2) are given as:

$$s_1\left(x_i\right)=\frac{-(p-1)f_0\left(z_i\right)}{\sigma F_0\left(z_i\right)}+\frac{(q-1)f_0\left(z_i\right)}{\sigma \left(1-F_0\left(z_i\right)\right)}-\frac{f_0^{\prime }\left(z_i\right)}{\sigma f_0\left(z_i\right)},$$ (B.1)

$$s_2\left(x_i\right)=\frac{-1}{\sigma }-z_i{s}_1\left(x_i\right),$$ (B.2)

$$s_3\left(x_i\right)=\psi (p+q)-\psi (p)+\text{\hspace{0.17em}log}{F}_0\left(z_i\right),$$ (B.3)

$$s_4\left(x_i\right)=\psi (p+q)-\psi (q)+\text{\hspace{0.17em}log}\left[1-F_0\left(z_i\right)\right],$$ (B.4)

where ψ denotes the di-gamma function and the indices j = 1, 2, 3, 4 pertain to the elements of θ.

Sample information per observation for the general beta family of distributions:

$$i_{11}\left(x_i\right)=\frac{(p-1)\left[f_0{\left(z_i\right)}^2-F_0\left(z_i\right)f_0^{\prime }\left(z_i\right)\right]}{{\sigma }^2{F}_0{\left(z_i\right)}^2}-\frac{(q-1)\left[f_0{\left(z_i\right)}^2+\left(1-F_0\left(z_i\right)\right)f_0^{\prime }\left(z_i\right)\right]}{{\sigma }^2[{\left(1-F_0\left(z_i\right)\right]}^2}+\frac{f_0^{\prime }{\left(z_i\right)}^2-f_0\left(z_i\right)f_0^{\prime \prime }\left(z_i\right)}{{\sigma }^2{f}_0{\left(z_i\right)}^2}$$ (B.5)

$$i_{12}\left(x_i\right)=\frac{z_i(p-1)\left[f_0{\left(z_i\right)}^2-F_0\left(z_i\right)f_0^{\prime }\left(z_i\right)\right]}{{\sigma }^2{F}_0{\left(z_i\right)}^2}+\frac{z_i(q-1)\left[f_0{\left(z_i\right)}^2+\left(1-F_0\left(z_i\right)\right)f_0^{\prime }\left(z_i\right)\right]}{{\sigma }^2[{\left(1-F_0\left(z_i\right)\right]}^2}+\frac{z_i\left[f_0^{\prime }{\left(z_i\right)}^2-f_0\left(z_i\right)f_0^{\prime \prime }\left(z_i\right)\right]}{{\sigma }^2{f}_0{\left(z_i\right)}^2}+\frac{s_{11}}{\sigma }$$ (B.6)

$$i_{13}\left(x_i\right)=\frac{f_0\left(z_i\right)}{\sigma F_0\left(z_j\right)}$$ (B.7)

$$i_{14}\left(x_i\right)=\frac{-f_0\left(z_i\right)}{\sigma \left(1-F_0\left(z_i\right)\right)}$$ (B.8)

$$i_{22}\left(x_i\right)=\frac{1}{{\sigma }^2}+z_i^2{i}_{11}+2z_i\left\{\frac{-(p-1)f_0\left(z_i\right)}{{\sigma }^2{F}_0\left(z_i\right)}+\frac{(q-1)f_0\left(z_i\right)}{{\sigma }^2\left[1-F_0\left(z_i\right)\right]}+\frac{f_0^{\prime }\left(z_i\right)}{f_0\left(z_i\right)}\right\}$$ (B.9)

$$i_{23}\left(x_i\right)=-z_i{i}_{13}$$ (B.10)

$$i_{24}\left(x_i\right)=-z_i{i}_{14}$$ (B.11)

$$i_{33}\left(x_i\right)={\psi }^{\prime }(p)-{\psi }^{\prime }(p+q)$$ (B.12)

$$i_{34}\left(x_i\right)=-{\psi }^{\prime }(p+q)$$ (B.13)

$$i_{44}\left(x_i\right)={\psi }^{\prime }(q)-{\psi }^{\prime }(p+q)$$ (B.14)