Skip to main content
Entropy logoLink to Entropy
. 2022 Jan 13;24(1):120. doi: 10.3390/e24010120

Weighted Relative Group Entropies and Associated Fisher Metrics

Iulia-Elena Hirica 1,, Cristina-Liliana Pripoae 2,, Gabriel-Teodor Pripoae 1,, Vasile Preda 1,3,4,*,
Editor: Maria Longobardi
PMCID: PMC8775049  PMID: 35052146

Abstract

A large family of new α-weighted group entropy functionals is defined and associated Fisher-like metrics are considered. All these notions are well-suited semi-Riemannian tools for the geometrization of entropy-related statistical models, where they may act as sensitive controlling invariants. The main result of the paper establishes a link between such a metric and a canonical one. A sufficient condition is found, in order that the two metrics be conformal (or homothetic). In particular, we recover a recent result, established for α=1 and for non-weighted relative group entropies. Our conformality condition is “universal”, in the sense that it does not depend on the group exponential.

Keywords: Fisher metric, group Fisher metric, weighted entropy, relative group entropy, α-weighted group entropy

1. Introduction

1.1. History

The inhabitants of the Universe of Uncertainty are probability distribution functions (PDFs). One can try to understand them through their entropy, a property which provides us a measure of disorder. Since its discovery, in the second part of the 19th Century, entropy was investigated by functional, algebraic or analytical techniques. The first geometric tools arose in the 1920s, through the work of Fisher ([1]), whose information matrix is the germ for what today is known as the Fisher metric. The notion was generalized by Rao ([2,3]) in the 1940’s, who put it in the appropriate context of Riemannian geometry. After a 30 years gap, Efron ([4]) and Amari ([5,6]) reopened the interest in differential geometric invariants associated to statistical models. Since the 1980s, the geometrization of the parameters space of the PDFs evolved into a field with rapidly growing expanse, including (among others) the new topics of statistical manifolds and of the dual connections ([7,8,9,10,11]; see in [12] for a recent review).

Our paper is a piece of Riemannian geometry intended for entropy study. We construct a new family of Fisher-like metrics, canonically associated to some entropy functionals, and we establish a sufficient condition in order for two of these metrics to be conformal. For describing the recipe, we must first sketch the story of its three main ingredients: the weighting procedure of an entropy functional, the relative group entropy and the group Fisher metric associated to it.

Around 1970, Belis and Guiasu ([13,14]) defined and axiomatized the concept of weighted entropy. They modified an entropy formula, by multiplying the integrand (which contained quantitative, objective and probabilistic information) with a “weighting function”, susceptible to model qualitative, subjective and non-stochastic utility data. The notion proved useful in various applications and gave rise to many papers. We quote here but a few: Barbu et al. [15], Batty [16], Das [17], Guiasu [18], Kayal [19], Kelbert et al. [20], Smieja [21], Suhov [22], and Tunnicliffe et al. [23], where the interested reader may find more details.

The notion of relative group entropy is more recent ([24]), but, as a nice coincidence, its algebraic roots are contemporary with the papers of Rao about the Fisher metric and distance, previously quoted. These algebraic foundations are the (Lie) formal groups, defined by Bochner [25] in 1946, as a unifying tool between Lie groups and Lie algebras (see also Hazewinkel [26]). Tempesta used the formal group laws, in order to define universality classes of entropies ([27,28], see also [29,30]). Starting with a formal group logarithm

G(t)=i=0citi+1i+1,(ciQ,c0=1),

he defined ([27]) group entropy functionals of the form

SG(p):=Xp(x)G(lnp(x)1)dx,

where p is a PDF on X. As an application, the relative entropy (also known as the Kullback–Leibler divergence) was generalized ([27]) to the relative group entropy.

Gomez et al. used ([24]) the relative group entropies for defining group Fisher metrics, as “universal” extensions of the classical Fisher metrics arising from the classical Boltzmann-Gibbs entropy. Their main result states that the previous pairs of metrics are homothetic, and the constant of homothety depends on G(0) and G(0).

1.2. Motivation

Our vision about entropy will be naive and formal, without entering into deep interpretation about the physical or informational meanings. Our goal is to filter the details and to see the nude invariants of geometric nature which might enlighten us to the general behavior of the various (and apparently too many) types of entropy avatars. For more details and hints about statistical interpretations, the reader is encouraged to look into the inspirational papers [24,27,31,32].

1.3. Contents of the Paper

In Section 2, we try to systematize and to unify different definitions for the entropy functions associated to PDFs. Two procedures of refinement are recalled: through “weighting” and through “powering” the PDFs. We shall need both in Section 4 and Section 5.

In Section 3, we review, in a creative manner, various methods to obtain semi-Riemannian metrics associated to families of entropy functions or functionals, as part of what we call “the geometrization problem for the entropy”. In particular, we have the Fisher metrics, the Hessian metrics and the group Fisher metrics. Generalizations include “mean” Hessian metrics, with the (“partial”) Hessian appearing under the integral sign.

Section 4 is the main part of the paper and is devoted to a generalization of the result of Gomez et al. [24]. The Theorem 1 provides a formula, linking a canonical weighted and powered Fisher-like metric to another one, which is, moreover, associated to a group exponential. We give a sufficient condition in order the two metrics be conformal or even homothetic. This conformality condition is, in some sense, “universal”, in that it does not depend on the group exponential.

Section 5 contains examples which illustrate the application of the general method from Section 4. We consider a normal PDF on the real line, with two parameters, with particular weighting functions and particular powers. As the compatibility relation is satisfied, it follows that, for any formal group exponential, we can construct a weighted Fisher metric, homothetic with the standard one. In addition to the construction of homothetic metrics in [24], our examples provide an additional control tool, through the weighting function. Moreover, our examples are well suited to be easily adapted and extended in more general frameworks.

1.4. Conventions

All the integrals are supposed to be correctly defined. Partial derivatives are supposed to commute with the integral. All the geometric objects are supposed to be differentiable, even if, in some cases, a weaker assumption would suffice (for example continuity or integrability).

2. Preliminaries

We review in a creative way some entropy-related notions, following mainly those in [14,27,28].

Consider XRm the domain of a real valued random variable x. Let p=p(x) be a (differentiable) probability density function (PDF), with p(x)0 and Xp(x)dx=1. Let f:R×XR be a differentiable controlling function.

Definition 1.

The generalized entropy corresponding to the pair (p,f) is the real number H=H(p,f), where

H:=Xf(p(x),x)dx. (1)

If the controlling function f depends on p and does not depend on x, we write f:RR and we say the entropy

H=Xf(p(x))dx (2)

is classical.

Using the convention in Section 1, we may write (1) and (2) in the equivalent normalized form

H:=Xp(x)h(p(x),x)dx,H=Xp(x)h(p(x))dx, (3)

where f((p(x),x)=p(x)h((p(x),x) and f(p(x))=p(x)h(p(x)), respectively. Normalizing does not change the property of an entropy to be generalized or classical, respectively.

A slightly more general type of normalization is to consider some positive real number α and the entropy functions of the form

H(α):=Xpα(x)h(p(x),x)dx,H(α)=Xpα(x)h(p(x))dx.

This “powering” of the PDF acts as a control tool over the set of entropy functions and enables -sometimes- a useful refinement of the models. However, we must be aware of the fact that, with an appropriate notation, the previous entropy functions may be rewritten in the form (3).

Example 1.

Almost all the known entropies used in statistical mechanics and information geometry are classical. We list but a few, from simple to more sophisticated ones.

(i) The functional f(p):=plog(p) gives the Boltzmann–Gibbs–Shannon (BGS) entropy.

(ii) For any fixed aR\{1}, f(p):=ppa11a1 provides a Tsallis entropy; when a1 we recover the BGS entropy.

(iii) Considering one more PDF q, the functional f(p):=plog(qp) leads to the relative entropy (also known as the Kullback–Leibler divergence [24]) between them, which is the normalized classical entropy

D(pq):=Xp(x)log(p(x)q(x))dx. (4)

We accept (formally) that 0·log(0q)=0 and p·log(p0)=0. More generally, we may start with k1 more PDFs p1, …, pk and a differentiable function F:Rk+1R. Then, the functional f(p):=F(p,p1,,pk) produces a relative entropy of p w.r.t. p1,…, pk, namely, the normalized classical entropy

D(pp1,,pk):=Xp(x)F(p(x),p1(x),,pk(x))dx. (5)

Moreover, this one may be viewed as a generalized entropy associated to the functional piF(p,p1, ,pk), for every i=1,k¯.

(iv) Let G=G(t) be a formal group logarithm, which is a differentiable real valued function with some special algebraic properties, inspired from the formal series linking Lie groups to Lie algebras. (We refer to [24,27,28] for details about these functions). The group entropy functional associated to it is defined by ([24,27])

SG(p):=Xp(x)G(lnp(x)1)dx, (6)

which is a normalized classical entropy of the form (3).

If q is another PDF, then the relative group entropy functional of p and q is defined ([24]) by

DG(pq):=Xp(x)G(lnq(x)p(x)1)dx, (7)

which is another normalized classical entropy of the form (3). Generalization for k PDFs leading to a relative group entropy of the form (5) is also possible, but we leave this step to the reader.

Remark 1.

Consider, moreover, an arbitrary differentiable function w:XR. The weighted generalized entropy corresponding to the triple (p,f,w), associated to the generalized entropy (1), is another generalized entropy, given by

H:=Xw(x)f(p(x),x)dx.

Similar weighted entropies may be defined starting from (2) and (3). For example, the weighted Tsallis entropy writes

H=Xw(x)p(x)(p(x))a11a1dx.

Weighting does not change the property of an entropy to be generalized or normalized, respectively. Instead, weighting may transform a classical entropy into a generalized one.

3. Fisher Metrics for Generalized Entropy Functions

We review in a creative way the main notions related to the Fisher metric derived from a family of generalized entropies, associated to some parameterized PDFs, following mainly the work in [24].

Consider the case when the PDF p in Section 2 depends, moreover, on n real parameters θ1,,θn, that is, p:X×RnR, p=p(x,θ), with θ:=(θ1,,θn). Let f:R×X×RnR be a differentiable controlling function, f=f(p,x,θ). The dependence on the parameter θ provides, via the relation (1), a generalized entropy function (loosely denoted also by) H:RnR, given by

H(θ)=Xf(p(x,θ),x,θ)dx. (8)

Analogously, classical entropy functions and normalized ones arise naturally, from parameterizing with θ the relations (2) and (3), respectively.

The geometrization problem for the entropy functions. Associate a relevant geometric structure to the function H (given by (8) or by any other avatar), whose invariants might provide information about the PDF p. Use f as a control tool in this process.

Remark 2.

The fist idea that comes to mind is to consider the (classical) Hessian tensor field associated to H, with coefficients

(HessH)ij(θ):=X2f(p(x,θ),x,θ)θiθjdx,i,j=1,n¯. (9)

If HessH is non-degenerate, it provides a semi-Riemannian metric (of constant signature) on Rn, called Hessian metric. Its geometry was subject to many papers (see, for example, in [33,34] and the references therein) and is useful in understanding the extremum points of H, as a semi-Riemannian optimization topic.

A simple example is the Euclidian metric gij=δij, arising as Hessian metric from Formula (9), by taking n=2, θ=(θ1,θ2) and f(p(x,θ),x,θ):=(θ1)2+(θ2)22p(x,θ).

Remark 3.

On another hand, consider a controlling function h:R×RnR, with h=h(p,θ), and its associated normalized classical entropy function

H(θ)=Xp(x,θ)h(p(x,θ),θ)dx. (10)

Define

gij(θ):=Xp(x,θ)2h(p(x,θ),θ)θiθjdx,i,j=1,n¯. (11)

If the matrix (gij)i,j=1,n¯ is nowhere vanishing, we obtain the (semi-Riemannian) Fisher metric associated to H (or h) [24]. An example is included in Section 5, for a PDF of exponential type.

Denote h˜(x,θ):=h(p(x,θ),θ). Then,

gij(θ)=Xp(x,θ)2h˜(x,θ)θiθjdx=Xp(x,θ)(Hessh˜)ij(x,θ)dx,i,j=1,n¯. (12)

This formula does not define (in general) a Hessian metric. In fact, this might be called “mean” Hessian metric, as we do not derivate the whole integral, but only a factor of the integrand. (Here we consider the Hessian for h˜ as function depending on θ only.)

However, in some particular cases, the Fisher metrics may (eventually) borrow the appearance of a Hessian metric, but w.r.t. other appropriate function, as in the following case [34]. Consider the PDF of exponential type p:X×RnR, p=p(x,θ), with θ:=(θ1,,θn),

p(x,θ):=exp{C(x)+i=1nFi(x)θiν(θ)},

where C=C(x), F1=F1(x),…, Fn=Fn(x) and ν=ν(θ) are smooth functions. The associated Fischer metric on Rn is g=Hessν, a Hessian metric w.r.t. ν, which is not derived from an entropy function like in Formula (9).

Example 2.

In the case of the BCS-entropy, h=logp, h˜(x,θ)=log(p(x,θ)) and we get the well-known classical Fisher metric

gij(θ)=Xp(x,θ)(Hessh˜)ij(x,θ)dx=Xp(x,θ)log(p(x,θ))θilog(p(x,θ))θjdx.

Its scalar curvature is interpreted as the average statistical uncertainty of a density matrix ([24]). This claim is quite natural, as the scalar curvature is obtained from the curvature tensor by taking the trace (a “mean”) two times successively, while the curvature tensor measures a “force” associated to the “matter” in the “Universe” Rn driven by g.

Denote the entropy function H(θ)=Xp(x,θ)log(p(x,θ))dx. We calculate

(HessH)ij(θ)=Xp(x,θ)(Hessh˜)ij(x,θ)dxX(Hessp)ij(x,θ)log(p(x,θ))dx=
=gij(θ)X2pθiθi(x,θ)log(p(x,θ))dx

This formula expresses the Fischer metric g in terms of the Hessian of H and of the “mean” Hessian of p, “weighted” by h˜.

Remark 4.

(i) Consider now a classical entropy function H:RnR, given by

H(θ)=Xf(p(x,θ))dx

and an associated weighted classical entropy function Hw:RnR, given by

Hw(θ)=Xw(x)f(p(x,θ))dx.

Their Hessians have the components

g^ij(θ):=2Hθiθj(θ)=X2f(p(x,θ))θiθjdx

and

(g^w)ij(θ):=2Hwθiθj(θ)=Xw(x)2f(p(x,θ))θiθjdx.

In case of simultaneous non-degeneracy, the corresponding (semi-Riemannian) Hessian metrics g^ and g^w provide a (geo)metric tool for studying the impact of the weighting function w upon the entropy of the system.

(ii) Suppose the entropy function and the weighted entropy function are given in the normalized form

H(θ)=Xp(x,θ)h(p(x,θ))dx

and

Hw(θ)=Xw(x)p(x,θ)h(p(x,θ))dx.

Denote f(p(x,θ)):=p(x,θ)h(p(x,θ)). Then, we may calculate the components of the respective Hessians g^ and g^w, as in (i). Eventually, in case of non-degeneracy, we get Hessian metrics and follow the strategy from (i).

For H and Hw, we may associate also the “mean” Hessian metrics g and gw, as in (12). Therefore, we have the Fisher metrics on Rn, given by

gij(θ)=Xp(x,θ)2h(p(x,θ))θiθjdx

and

(gw)ij(θ)=Xw(x)p(x,θ)2h(p(x,θ))θiθjdx

(iii) Similar Hessians or “mean” Hessians can be associated to relative entropy functions, as in Example 1,(iii), or to entropy group functionals, as in Example 1, (iv). If non-degenerate, they become semi-Riemannian Hessian metrics or Fisher metrics ([24,27,28]).

(iv) Analogous entropy functions and metrics arise when weighting by functions w=w(x,θ), instead of w=w(x). In fact, this is a more flexible weighting procedure, as it does not impose universal weighting constraints upon the whole family of PDFs.

4. A Step Further: Passing from the Fisher Metric Group to a Weighted One

Consider a PDF p depending on n real parameters θ1,,θn; that is, p:X×RnR, p=p(x,θ), with θ:=(θ1,,θn). Let g be the Fischer (semi)-Riemannian metric on Rn, whose coordinate functions are given by

gij(θ)=Xp(x,θ)log(p(x,θ))θilog(p(x,θ))θjdx.

Let G=G(t) be a formal group logarithm, as in Example 1, (iv). Consider H:Rn×RnR the double parameterized group entropy functional associated to it ([24]), by

H(θ,θ˜):=Xp(x,θ)G(lnp(x,θ)p(x,θ˜))dx. (13)

The Hessian tensor field gG, with components

(gG)ij(θ˜):=2H(θ,θ˜)θiθjθ=θ˜,

is called the group Fisher metric in [24]. The main result in [24] states that the metrics g and gG are homothetic, via the formula

gG=(G(0)+G(0))g. (14)

From it follows the well-known correspondence between the main invariants of g and gG, such as the Christoffel coefficients, the Riemann curvature coefficients and the scalar curvatures.

We prove now that this result remains true, in the more general assumption of α-weighted group entropy functionals.

Theorem 1.

Let α be a positive real number. Let w:X×RnR be a weighting function for (13), providing the double-parameterized weighted α-normalized group entropy functional

Hw,α(θ,θ˜):=Xw(x,θ˜)pα(x,θ)G(lnp(x,θ)p(x,θ˜))dx. (15)

Consider the α-weighted group Fisher metric gGw,α, with components

(gGw,α)ij(θ˜):=2Hw,α(θ,θ˜)θiθjθ=θ˜=Xw(x,θ˜)2θiθj{pα(x,θ)G(lnp(x,θ)p(x,θ˜))}θ=θ˜dx,

and the α-weighted Fisher metric gw,α, with components

(gw,α)ij(θ):=Xw(x,θ)pα(x,θ)log(p(x,θ))θilog(p(x,θ))θjdx.

Then, the metrics gw,α and gGw,α are related, via the formula

(gGw,α)ij(θ)={(2α1)G(0)+G(0)}(gw,α)ij(θ)+G(0)Xw(x,θ)pα1(x,θ)2p(x,θ)θiθjdx. (16)

Proof. 

We begin by deriving

Hw,αθj(θ,θ˜)=Xαw(x,θ˜)pα1(x,θ)p(x,θ)θjG(lnp(x,θ)p(x,θ˜))dx
+Xw(x,θ˜)pα1(x,θ)p(x,θ)θjG(lnp(x,θ)p(x,θ˜))dx,
2Hw,αθiθj(θ,θ˜)=Xα(α1)w(x,θ˜)pα2(x,θ)p(x,θ)θip(x,θ)θjG(lnp(x,θ)p(x,θ˜))dx
+Xαw(x,θ˜)pα1(x,θ)2p(x,θ)θiθjG(lnp(x,θ)p(x,θ˜))dx
+Xαw(x,θ˜)pα2(x,θ)p(x,θ)θip(x,θ)θjG(lnp(x,θ)p(x,θ˜))dx
+X(α1)w(x,θ˜)pα2(x,θ)p(x,θ)θip(x,θ)θjG(lnp(x,θ)p(x,θ˜))dx
+Xw(x,θ˜)pα1(x,θ)2p(x,θ)θiθjG(lnp(x,θ)p(x,θ˜))dx
+Xw(x,θ˜)pα2(x,θ)p(x,θ)θip(x,θ)θjG(lnp(x,θ)p(x,θ˜))dx.

We assign θ:=θ˜ and use the properties: G(0)=0 and Xp(x,θ)dx=1. We get

(gGw,α)ij(θ˜):=[(2α1)G(0)+G(0)]Xw(x,θ˜)pα2(x,θ˜)p(x,θ˜)θip(x,θ˜)θjdx
+ G(0)Xw(x,θ˜)pα1(x,θ˜)2p(x,θ˜)θiθjdx
=[(2α1)G(0)+G(0)]Xw(x,θ˜)pα(x,θ˜)log(p(x,θ˜))θilog(p(x,θ˜))θjdx
+ G(0)Xw(x,θ˜)pα1(x,θ˜)2p(x,θ˜)θiθjdx
=[(2α1)G(0)+G(0)](gw,α)ij(θ˜)+G(0)Xw(x,θ˜)pα1(x,θ˜)2p(x,θ˜)θiθjdx,

which concludes the proof. □

Corollary 1.

In the hypothesis and with the notations from Theorem 1, suppose there exists a function φ:RnR, such that, for every i,j=1,n¯,

Xw(x,θ)pα1(x,θ)2p(x,θ)θiθjdx =φ(θ)Xw(x,θ)pα2(x,θ)p(x,θ)θip(x,θ)θjdx. (17)

Then the metrics gw,α and gGw,α are conformal, via the formula

gGw,α={(2α1)G(0)+φG(0)+G(0)}gw,α. (18)

If, moreover, φ is constant, then the metrics gw,α and gGw,α are homothetic.

Remark 5.

(i) In [24], the notation θ0 is used, instead our notation θ˜. We consider it more appropriate, for not creating the (possible wrong) impression of “constancy” of the variable.

(ii) We used the terminology from [24], when calling gGw,α a group Fisher entropy metric, but, in our opinion, it is but a Hessian-like metric, not Fisher-like (i.e., of a “mean” Hessian type, see Section 2).

(iii) Taking w(x)=1 and α=1 in Corollary 1, we get φ=0 and we recover the result from [24], outlined in the preamble of the section.

(iv) The relation (17) writes also

Xw(x,θ)pα2(x,θ){p(x,θ)2p(x,θ)θiθjφ(θ)p(x,θ)θip(x,θ)θj}dx=0. (19)

A strong sufficient condition for (19) to hold true is the Hessian matrix of p (w.r.t. to θ) is proportional with the matrix (p(x,θ)θip(x,θ)θj)i,j. Relation (19) is satisfied iff the respective proportionality holds as a mean , through the intermediate of the integral, and weighted by wpα2.

(v) We may also understand Corollary 1 in the following way: let p be a parameterized PDF with an entropy function (13); if there exists a weighting function w, a positive integer α, and a function φ, such that (19) is satisfied, then the metrics gGw,α and gw,α are conformal, via (18). Thus, the search for pairs of conformal metrics is controlled by a triple “tool box”. A nontrivial example will be given in the next section.

Moreover, we may conjecture that given a PDF p, there exist w, φ and α such that (19) holds. Preliminary calculations provide hints that this might be true, at least for PDFs “of exponential type”.

(vi) In (18), the conformal factor depends on α, only through the intermediary of the “speed” of G around 0, not (also) through the “acceleration” of G around 0.

(vii) The condition (19) does not depend on G. In this sense, it is an “universal” condition, and we suspect it may hidden some (remarkable?) unraveled family of PDFs.

(viii) We introduce a new function ψ, related to the conformal factor in (18), by

e2ψ:=(2α1)G(0)+φG(0)+G(0).

(We suppose the right side positive.) Denote ρGw,α and ρw,α the scalar curvatures of gGw,α and gw,α, respectively. Then,

e2ψρGw,α=ρw,α2(n1)Δψ(n2)(n1)(dψ)2.

Scenario: suppose p, w, α are given, such that the conformality condition (19) holds. Calculate the scalar curvature ρw,α (together with all the invariants associated to the metric gw,α). Then, the previous formula allows a conformal variation of ρw,α, controlled by the set of formal group logarithms G, which is more flexible than the homothetic variation in [24]. Homothetic transformations of the metrics are quite rigid, as they preserve the qualitative behavior of the main invariants semi-Riemannian (distance, geodesics, curvature). By contrast, conformal non-homothetic transformations produce significant geometric changes, such as passing from a plane to a sphere.

5. Examples

In what follows, consider X=R and the normal PDF p:R×R×(0,)R,

p(x;θ1,θ2)=12πθ2e(xθ1)22(θ2)2. (20)

Denote by p1, p2, p11, p12, p22 the partial derivatives of p, of order one and two, w.r.t. θ1 and θ2. We calculate, successively,

p1=xθ1(θ2)2p,p2={(xθ1)2(θ2)31θ2}p,
p11={(xθ1)2(θ2)41(θ2)2}p,p12={(xθ1)3(θ2)53(xθ1)(θ2)3}p,
p22={(xθ1)4(θ2)65(xθ1)2(θ2)4+2(θ2)2}p.

The classical Fisher metric g has the components g11=(θ2)2, g12=g21=0 and g22=2(θ2)2 (see, for example, [10]).

Let α be a positive real number and consider the particular weighting function w(x,θ):=(xθ1)2. We calculate the coefficients of the α-weighted Fisher metric gw,α:

g11w,α=3(2πθ2)1αα52,g12w,α=0,g22w,α=(α26α+15)(2πθ2)1αα72.

We prove now that, in the particular case of α=3 + 572, with the function φ:=1α3=3  576, the relation (19) is identically satisfied. We evaluate the left side integral in (19), in a case-by-case calculation.

(i) The case i=1,j=2:

(xθ1)2pα(x,θ){(xθ1)33(xθ1)(θ2)2(θ2)5φ(xθ1)3(xθ1)(θ2)2(θ2)5}dx=0,

as pα is an even function and all the binomial integrands are odd functions, w.r.t. (xθ1). (We see that here we did not use the particular form of φ, so the result is valid for any positive α and any function φ=φ(θ).)

(ii) The case i=1,j=1:

(xθ1)2pα(x,θ){(xθ1)2(θ2)2(θ2)4(1α3)(xθ1)2(θ2)4}dx=0.

(We see that here we did not use the particular form of α, so the result is valid for any positive α and for the function φ=1α3.)

(iii) The case i=2,j=2:

(xθ1)2pα(x,θ){(xθ1)45(xθ1)2(θ2)2+2(θ2)4(θ2)6
(1α3)(xθ1)42(xθ1)2(θ2)2+(θ2)4(θ2)6}dx=k(α23α12)=0,

where k is a nowhere vanishing function, depending on α and θ2. (We see that here we used both the particular form of α and of the function φ.)

Let G be any formal group logarithm. As the relation (19) is verified for a constant function φ, it follows that the α-weighted group Fisher metric gGw,α is homothetic to gw,α. Using Formula (18), we obtain

gGw,α={5(3+57)6G(0)+G(0)}gw,α. (21)

Remark 6.

Similar examples may be obtained: for w(x,θ):=(xθ1)a, with even a; for a polynomial function w with variable (xθ1) (the odd powers are irrelevant); for w(x):=xa (or a polynomial function in x), with a significant increase of the complexity of the calculations.

Remark 7.

Consider the PDF (20), a positive real number α and a weighting function w(x,θ)=l((xθ1)22(θ2)2), where l:RR is an arbitrary continuous function (satisfying, eventually, some additional specific hypothesis, in order the following integrals exist). The cases i=1,j=1 and i=1,j=2 of the relation (19) are identically satisfied, for the constant function

φ=1α20l(tα)t12etdt0l(tα)t12etdt.

First, let us remark that this family of examples cannot lead to conformal non-homothetic transformations of the metric, via the relation (18). Second, we must impose that condition (19) holds, for i=2,j=2 also, and this provides a strong additional constraint on φ and α.

Unfortunately, the existence of a suitable value of the power α is not guaranteed. For example, in the particular case of l(u):=1+eu, we get

φ=1α(α+1)2α+1α(α+1).

Then, from the case for i=2,j=2 in (19), it follows that α is non-real or non-positive.

Example 3.

We consider now a particular case of the previous example. Let p=p(x,θ) be the two-variate PDF from (20); the weighting function w(x,θ)=(xθ1)2; α=3 + 572.

(i) Let G be the Thallis group logarithm, given [24] by

G(t)=e(1q)t11q,qR,q1.

When q1, we get G(t)t, i.e. the BGS group logarithm. By abuse, from now on, we shall suppose qR.

Denote

A:=32π1+572(3+572)52,B:=2π1+572453572(3+572)72.

The α-weighted Fisher metric writes

(gijw,α)=A(θ2)1α00B(θ2)1α.

Its Christoffel coefficients are

(Γij1)=01α2θ21α2θ20,(Γij2)=AB1α2θ2001α2θ2.

We calculate successively the Riemann–Christoffel coefficients

R2121=Γ221θ1Γ211θ2+Γ111Γ221+Γ211Γ222Γ121Γ211Γ221Γ212=1α21(θ2)2.
R1212=g11R2121=A(θ2)1α1α21(θ2)2=A(1α)2(θ2)α1.

and the scalar curvature ρ=ρ(θ2),

ρ=2R1212det(gij)=2A(1α)2(θ2)1αAB(θ2)22α=1αB(θ2)α3.

We have G(0)=1 and G(0)=1q. From the relation (21) we calculate the scalar curvature of the associated α-weighted group Fisher metric gGw,α:

ρG(θ2,q)=(21+5576q)1·1αB(θ2)α3.

We restrain the study for q<21 + 5576.

As already pointed out in Example 2, the scalar curvature measures the average statistical uncertainty. In this case, both scalar curvature functions ρ and ρG take strictly negative values, so the uncertainty is proportional to their module. The variation of ρG is depicted in Figure 1.

(ii) If we replace the Tsallis group logarithm with the Kaniakidis group logarithm [24]

G(t)=sinh(1q)t1q,qR,q1,

the scalar curvatures ρ and ρG differ by a constant only. The reason is that G(0)=1 and G(0)=0, and ρG gets no new variables from (21).

(iii) Instead, replacing the Tsallis group logarithm with the Abe group logarithm [24]

G(t)=eatebtab,a,bR,ab,

leads to a behavior of ρG versus ρ similar to (i), because G(0)=1 and G(0)=a+b. In this case, ρG depends on a, b and θ2.

Figure 1.

Figure 1

The variation of ρG, with the notation x:=θ2 and y:=q.

Remark 8.

Let p=p(x,θ) be the two-variate PDF from (20); the weighting function w(x,θ)=(xθ1)2; α an arbitrary positive real number; G an arbitrary group logarithm. In general, the hypothesis of the Corollary 1 is not fulfilled anymore. This implies that the metrics gw,α and gGw,α might not be conformal, and all we deduced previously for their scalar curvatures did not remain true, in general.

However, in this case we can study what can be derived, directly from Theorem 1, via the relation (16). A tedious calculation produces the following formulas:

(gGw,α)11=C·(θ2)1α,(gGw,α)12=0,(gGw,α)22=D·(θ2)1α,

where we denoted

C=[5αG(0)+3G(0)]·2π1α·α52
D=[(2α311α2+21α)·G(0)+(α26α+15)·G(0)]·2π1α·α72

The scalar curvature writes

ρG=1αD(θ2)α3.

Until now, G was arbitrary. Let us particularize it: consider G(t)=t, i.e., the BGS group logarithm. The scalar curvature of the corresponding α-weighted relative group metric is

ρG=1α2α311α2+21α·2πα1·α72·(θ2)α3.

Its “macroscopic” variation with respect to θ2 and α can be seen in Figure 2.

We remark the special case α=1, when the metrics gw,1 and gGw,1 become locally Euclidean and all the related curvature invariants (including the scalar curvature) vanish. For α>1, both scalar curvatures are negative and this behavior is clearly visible in Figure 1. For α(0,1), both scalar curvatures are positive; to see it, we must take a magnified detail, as in Figure 3.

Similar calculations can be made for other group logarithms also, following the same line. For the Tsallis group logarithm, the scalar curvature will depend also on the third variable q; for the Abe group logarithm, the scalar curvature will depend also on two more variables, namely, a and b.

Figure 2.

Figure 2

The variation of ρG, with the notation x:=θ2 and y:=α>0.

Figure 3.

Figure 3

The variation of ρG, with the notation x:=θ2 and y:=α(0,1).

6. Discussion

The main motivation of our research was the discovery of new metrics, able to geometrize properties of statistical phenomena. We succeeded a double-folded generalization of the construction from in [24], extending the “homothetization” of the group Fisher metric to a “conformalization” of the α-weighted ones. These are semi-Riemannian metrics associated to entropy functions depending on arbitrary weighting functions and to powers of the PDFs. However, the generalization has a price: the existence of the conformal transformation requires a compatibility relation between the weighting functions, the powers of the PDFs and the PDFs (the relation (19)). We conjectured that, for a given PDF, there exists a weighting function and a power of the PDF, such that the relation (19) holds. We speculate that the elegance of (19) suggests a deeper property, whose hidden meaning remains to be revealed.

From now on, several directions of study open, hierarchically ordered upon the complexity/difficulty of the calculations involved:

(i) the constructions of new examples, by appropriate α-weighting the PDF from Section 5, as suggested in the Remark 6.

(ii) the constructions of new examples, similar to those in Section 5, for other remarkable PDFs on the real line, depending on two parameters (as the lognormal, the Gamma and the Beta distributions, following the classical approach from [10] ).

(iii) passing from m=1 and n=2 (as in (i) and (ii)) to arbitrary integers m and n, and appropriate α-weighting of classical examples of PDFs.

(iv) deepening the differential geometric study of the previous new metrics, and finding new geometric invariants for the modeling and the control of statistical phenomena. For example, the study of the distance related topics (see [35]), of the geodesics (see [36,37,38]) and of the various types of curvature tensor fields (see [10,39,40,41,42] for samples of the needed geometric tools).

(v) finding conformal, non-homothetic examples for which relation (19) holds. (These examples may also provide hints and support for our previously described conjecture.) Beyond the dreaded complications of the formalism, await the rewards of the much richer conformal geometry.

Acknowledgments

We are grateful to the reviewers for their valuable remarks, which helped us in improving the manuscript.

Author Contributions

Conceptualization, I.-E.H., C.-L.P., G.-T.P. and V.P.; Data curation, not applicable; Formal analysis, I.-E.H., C.-L.P., G.-T.P. and V.P.; Funding acquisition, V.P.; Investigation, I.-E.H., C.-L.P., G.-T.P. and V.P.; Methodology, I.-E.H., C.-L.P., G.-T.P. and V.P.; Project administration, I.-E.H., C.-L.P., G.-T.P. and V.P.; Resources, I.-E.H., C.-L.P., G.-T.P. and V.P.; Software, not applicable; Supervision, I.-E.H., C.-L.P., G.-T.P. and V.P.; Visualization, I.-E.H., C.-L.P., G.-T.P. and V.P. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

Footnotes

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  • 1.Fisher R.A. Theory of Statistical Estimation. Math. Proc. Cambridge Phil. Soc. 1925;22:700–725. doi: 10.1017/S0305004100009580. [DOI] [Google Scholar]
  • 2.Rao C.R. Information and the accuracy attainable in the estimation of statistical parameters. Bull. Calcutta Math. Soc. 1945;37:81–91. [Google Scholar]
  • 3.Rao C.R. On the distance between two populations. Sankhyā. 1945;9:246–248. [Google Scholar]
  • 4.Efron B. Defining the curvature of a statistical problem (with applications to second order efficiency) Ann. Stat. 1975;3:1189–1242. doi: 10.1214/aos/1176343282. [DOI] [Google Scholar]
  • 5.Amari S. A foundation of information geometry. Electron. Commun. Jpn. 1983;66:1–10. doi: 10.1002/ecja.4400660602. [DOI] [Google Scholar]
  • 6.Amari S. Differential Geometric Methods in Statistics. Springer; Berlin, Germany: 1985. Lect. Notes in Statistics, 28. [Google Scholar]
  • 7.Amari S. Information Geometry and Its Applications. Springer; Berlin/Heidelberg, Germany: 2016. [Google Scholar]
  • 8.Ay N., Gibilisco P., Matus F., editors. Information Geometry and Its Applications. Springer; Berlin/Heidelberg, Germany: 2018. [Google Scholar]
  • 9.Morales P.A., Rosas F.E. Generalization of the maximum entropy principle for curved statistical manifolds. Phys. Rev. Res. 2021;3:033216. doi: 10.1103/PhysRevResearch.3.033216. [DOI] [Google Scholar]
  • 10.Calin O., Udriste C. Geometric Modeling in Probability and Statistics. Springer; New York, NY, USA: 2014. [Google Scholar]
  • 11.Tanaka S. Appearance of Thermal Time. Found. Phys. 2021;51:34. doi: 10.1007/s10701-021-00445-w. [DOI] [Google Scholar]
  • 12.Nielsen F. An elementary introduction to Information Geometry. Entropy. 2020;22:1100. doi: 10.3390/e22101100. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Belis M., Guiasu S. A quantitative-qualitative measure of information in cybernetic systems. IEEE Trans. Inf. Theory. 1968;14:593–594. doi: 10.1109/TIT.1968.1054185. [DOI] [Google Scholar]
  • 14.Guiasu S. Weighted entropy. Rep. Math. Phys. 1971;2:165–179. doi: 10.1016/0034-4877(71)90002-4. [DOI] [Google Scholar]
  • 15.Barbu V.S., Karagrigoriou A., Preda V. Entropy and divergence rates for Markov chains: II. The weighted case. Proc. Rom. Acad. 2018;19:3–10. [Google Scholar]
  • 16.Batty M. Cost, accessibility, and weighted entropy. Geogr. Anal. 1983;15:256–267. doi: 10.1111/j.1538-4632.1983.tb00786.x. [DOI] [Google Scholar]
  • 17.Das S. On weighted generalized entropy. Commun. Stat. Theory Methods. 2017;46:5707–5727. doi: 10.1080/03610926.2014.960583. [DOI] [Google Scholar]
  • 18.Guiasu S. Grouping data by using the weighted entropy. J. Stat. Plan. Inference. 1986;15:63–69. doi: 10.1016/0378-3758(86)90085-6. [DOI] [Google Scholar]
  • 19.Kayal S. Weighted Fractional Generalized Cumulative Past Entropy. arXiv. 20212106.10312v1 [Google Scholar]
  • 20.Kelbert M., Stuhl I., Suhov Y. Weighted entropy: Basic inequalities. Mod. Stochastics Theory Appl. 2017;4:233–252. doi: 10.15559/17-VMSTA85. [DOI] [Google Scholar]
  • 21.Smieja M. Weighted approach to general entropy function. IMA J. Math. Control. Inf. 2015;32:329–341. [Google Scholar]
  • 22.Suhov Y., Stuhl I., Sekeh S., Kelbert M. Basic inequalities for weighted entropy. Aequ. Math. 2016;90:817–848. doi: 10.1007/s00010-015-0396-5. [DOI] [Google Scholar]
  • 23.Tunnicliffe M., Hunter G. Dimensionality, Granularity, and Differential Residual Weighted Entropy. Entropy. 2019;21:825. doi: 10.3390/e21090825. [DOI] [Google Scholar]
  • 24.Gomez I., Portesi M., Borges E.P. Universality classes for the Fisher metric derived from relative group entropy. Phys. A. 2019. in print . [DOI]
  • 25.Bochner S. Formal Lie groups. Ann. Math. 1946;47:192–201. doi: 10.2307/1969242. [DOI] [Google Scholar]
  • 26.Hazewinkel M. Formal Groups and Applications. Academic Press; New York, NY, USA: 1978. [Google Scholar]
  • 27.Tempesta P. Group entropies, correlation laws, and zeta functions. Phys. Rev. E. 2011;84:021121. doi: 10.1103/PhysRevE.84.021121. [DOI] [PubMed] [Google Scholar]
  • 28.Tempesta P. Multivariate group entropies, super-exponentially growing complex systems and functional equations. Chaos. 2020;30:123119. doi: 10.1063/5.0009846. [DOI] [PubMed] [Google Scholar]
  • 29.Jensen H.J., Tempesta P. Group Entropies:From Phase Space Geometry to Entropy Functionals via Group Theory. Entropy. 2018;20:804. doi: 10.3390/e20100804. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Rodriguez M.A., Romaniega A., Tempesta P. A new class of entropic information measures, formal group theory and Information Geometry. Proc. Royal Soc. A. 2019;475:20180633. doi: 10.1098/rspa.2018.0633. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Amigo J.M., Balogh S.G., Hernandez H. A Brief Review of Generalized Entropies. Entropy. 2018;20:813. doi: 10.3390/e20110813. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Tsallis C. Beyond Boltzmann-Gibbs-Shannon in Physics and Elsewhere. Entropy. 2019;21:696. doi: 10.3390/e21070696. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Li W. Hessian metric via transport information geometry. J. Math. Phys. 2021;62:033301. doi: 10.1063/5.0012605. [DOI] [Google Scholar]
  • 34.Shima H. The Geometry of Hessian Structures. World Scientific; Singapore: 2007. [Google Scholar]
  • 35.Abbad Z., El Maniani A.D., El Alaoui S.O., El Hassouni M. A Graph-based approach to derive the geodesic distance on Statistical manifolds: Application to Multimedia Information Retrieval. arXiv. 20212106.14060v1 [Google Scholar]
  • 36.Bensadon J. Black-Box Optimization Using Geodesics in Statistical Manifolds. Entropy. 2015;17:304–345. doi: 10.3390/e17010304. [DOI] [Google Scholar]
  • 37.Cafaro C., Ali S.A. Information Geometric Measures of Complexity with Applications to Classical and Quantum Physical Settings. Foundations. 2020;1:45–62. doi: 10.3390/foundations1010006. [DOI] [Google Scholar]
  • 38.Ruscitti C.B., Langoni L.B., Melgarejo A.A. Differential Geometry and Macroscopic Descriptions in Nonequilibrium Process. In: Bulnes F., editor. Advances on Tensor Analysis and Their Applications. IntechOpen; London, UK: 2020. pp. 73–84. Chapter 5. [Google Scholar]
  • 39.Erdmenger J., Grosvenor K.T., Jefferson R. Information geometry in quantum field theory: Lessons from simple examples. SciPost Phys. 2020;8:073. doi: 10.21468/SciPostPhys.8.5.073. [DOI] [Google Scholar]
  • 40.Gibilisco P. Lp Unit Spheres and the α-Geometries: Questions and Perspectives. Entropy. 2020;22:1409. doi: 10.3390/e22121409. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Hirica I.-E., Pripoae C.-L., Pripoae G.-T., Preda V. Affine Differential Geometric Control Tools for Statistical Manifolds. Mathematics. 2021;9:1654. doi: 10.3390/math9141654. [DOI] [Google Scholar]
  • 42.Mageed I.A., Kouvatsos D.D. The Impact of Information Geometry on the Analysis of the Stable M/G/1 Queue Manifold; Proceedings of the 10-th International Conference on Operations Research and Enterprise Systems (ICORES 2021); Setubal, Portugal. 4–6 February 2021; Setúbal, Portugal: SCITEPRESS; pp. 153–160. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

Not applicable.


Articles from Entropy are provided here courtesy of Multidisciplinary Digital Publishing Institute (MDPI)

RESOURCES