Abstract
In this paper, we prove the anisotropic Shannon inequality for the Rényi entropy with the best constant on Folland-Stein homogeneous Lie groups. As a consequence, we also prove the optimal Shannon inequality in the same setting. Using a logarithmic Sobolev inequality in the setting of stratified groups, we prove a Heisenberg-type uncertainty principle in the latter setting.
Mathematics Subject Classification: 94A17, 22E30, 35H10
Introduction
Shannon and Rényi entropies were initially served as characteristics of probability distributions. To begin with, the Shannon entropy was introduced by Shannon in [20], and for a discrete random variable X with distribution p(x) it is given via
In the continuous setting for a continuous random variable X with (probability) density u(x), the natural analog of the previous definition is as follows:
| 1 |
Clearly the Shannon entropy (1) makes sense for a nonnegative such that . In the sequel we will write h[u] instead of h(X), to denote the Shannon entropy of the continuous random variable X with density u. Let us point out that in the continuous case that we study here, p(x) is not a probability, but a probability density, and the latter are not the same. Indeed, the quantity h[X], for X being a discrete random variable, is always non-positive, while if X is the continuous random variable which is uniformly distributed over the interval (a, b); that is
then , and clearly if , see e.g. [17]. Therefore, in the continuous case, one cannot, strictly speaking, talk about the “amount of information” represented by the entropy.
There are other features of the Shannon continuous entropy that make its study a quite involved subject; see for example [13]. Despite these difficulties, continuous informatics is a field of wide interest, c.f. the monograph of Ihara [10], or the work of Conrad [6]. Particularly, in relation to partial differential equations, the conclusion of the Boltzman H-Theorem, see e.g. [25], holds true for the solution to the heat equation on
| 2 |
that is the Shannon entropy (1) is not decreasing in time, since
Analogously, the Rényi entropy, introduced by Rényi in [16], is for nonnegative function with , given by
| 3 |
“corresponds” to the nonlinear diffusion equations
It is easy to check that the Rényi entropy (3) is an extension of the Shannon entropy (1) for different values of in the sense that
for any function u as above. Both Shannon and Rényi entropies, in both discrete and continuous settings, are monotone in the sense that they increase whenever an independent random variable is added, and hence a natural question is whether one can find an upper bound for them; in the Euclidean setting , this is given by Shannon’s inequality [20, 22] which is an upper bound of the form:
| 4 |
Hence inequality (4) shows that the Shannon entropy of a function u is bounded by the second moment of u, and the constant is the best possible. In the case of the Rényi entropy, this upper bound reads as follows:
| 5 |
that is the Rényi entropy of the function u is bounded by the -moment of u, where depends on the range of . Both inequalities (4) and (5) are valid for functions u in some suitable weighted Lebesgue spaces. Inequality (5) is sharp, and the constant in (5) depends on the parameter b on the right-hand side of (5), which in turn depends on the parameter and is given explicitly in [22] where the aforementioned inequality is proved. For the so-called Shannon inequality in the Euclidean setting (4) we refer to the paper [15], while Shannon’s inequality has also been proved in the general setting of homogeneous Lie groups in [3]. Properties of both entropies have been widely studied; see e.g. [1] and references therein.
Applications of Shannon’s inequality (4) and Shannon’s inequality for the Rényi entropy also include the positivity of the relative Lyapunov functional
where is the Lyapunov functional, see e.g. [14], given by
see [22] and [24]. Importantly, the combination of Shannon inequality (4) together with the version of logarithmic Sobolev inequality thanks to Stam [21]
| 6 |
which holds true for non-negative with , gives rise to the Heisenberg uncertainty principle; that is for such u we have
| 7 |
where n is the best possible. Still in the Euclidean setting, Carillo and Toscani [2] proved the logarithmic Sobolev-type inequalities for the Rényi entropy; see also [19].
The setting of the current work is that of homogeneous Lie groups introduced in Section 2. In this setting, in [3] the authors proved the (anisotropic) Shannon inequality, with an explicit constant that is sharp, see the discussion that follows after Remark 3.3. Here we prove the analogue of Shannon’s inequality for the Rényi entropy as in (5) for the setting of homogeneous Lie groups, see Theorem 3. For the proof of the latter, we follow the lines in [22] and we prove the aforementioned inequality with the best constant as it happens in [22], see Corollary 3.2. Later on, in Theorem 3.4, we prove the Shannon inequality as in (5) in the setting of homogeneous Lie groups, using the fact that the Shannon entropy is the limiting case when of the Rényi entropy. The Shannon inequality in the aforesaid setting was also proved in [3] using different methods and the constant there coincides with ours since both are optimal. Our final result is the Heisenberg uncertainty principle in the setting of stratified Lie groups, see Corollary 3.7. The latter is proved by combining Shannon’s inequality with a version of logarithmic Sobolev inequality in the aforesaid setting that is derived by the logarithmic Sobolev inequality as in [4]; see Theorem 3.6.
Preliminaries
In this section, we give a brief description of our setting of homogeneous Lie groups. Such groups were initiated by Folland and Stein, see [8], and later on, they became a subject of study by many authors; see [7] and references therein. The notation that we adopt in the sequel follows the more recent open access monograph on homogeneous Lie groups [7].
A connected, simply connected Lie group whose Lie algebra admits a gradation of the form
| 8 |
where finitely many ’s are nonzero and satisfy relations of the form is called a graded Lie group. Graded Lie groups are naturally homogeneous Lie groups meaning that there exists a dilation mapping denoted as , where , that is an automorphism of , and so also of . Particularly, for , the mapping acts on via
| 9 |
The so-called weights determine the homogeneous dimension, usually denoted by Q, of the homogeneous group in the following way:
A homogeneous group is unimodular and the unique Haar measure denoted by dx on is the Lebesgue measure on the underlying manifold . Hence for being a measurable set in , if stands for the volume of , then for we have
| 10 |
For any homogeneous Lie group , there exists a homogeneous (with respect to the dilations determined by on ) quasi-norm defined on ; that is is a continuous, non-negative function that satisfies the following conditions:
-
(i)
For all , .
-
(ii)
For all and , .
-
(iii)
if and only if .
We note that in the sequel we use the notation to denote both the volume of a measurable set in and the homogeneous quasi-norm of an element , and the meaning of is each appearance will be clear from the context.
Finally, let us introduce the following Lebesgue spaces that are useful for our purposes: For , the weighted Lebesgue space denoted by is defined as follows:
where |x| stands for the quasi-norm of .
Main Results
In this section, we establish the main results of the paper starting with proving the Shannon inequality for the Rényi entropy in the setting of homogeneous Lie groups.
Theorem 3.1
Let be a homogeneous Lie group of homogeneous dimension Q, and let be a homogeneous quasi-norm on . Suppose that , , and
Then, for any nonnegative function with , the inequality
| 11 |
holds, where
and stands for the dimensional surface measure of the unit (quasi-)sphere with respect to
Proof of Theorem 3.1
Note that it is enough to prove (11) for a nonnegative function that is smooth on . The result will then follow by a density argument. We will treat the cases where and separately. Let us first consider the case . For such values of and for we consider the auxiliary function given by
with computed in the Appendix, see (A6). Using Jensen’s inequality for the convex function and the the probability measure we estimate from above the relative entropy of u and as follows
| 12 |
where is computed in Appendix, see (A11), as
where is given in (A6).
Let us now give a lower bound for the relative entropy of u and . By Jensen’s inequality for the same probability measure, and since , we have
| 13 |
where we have used the fact that . A combination of (12) and (13) gives
| 14 |
Now, let us apply inequality (14) to the function . To this end, first we note that
| 15 |
since
Similarly we have
| 16 |
Hence, with the use of (15) and (16), inequality (14) for the function reads as follows
Using the properties of the logarithm, the latter inequality can be rearranged as follows
| 17 |
and since
multiplying (17) by we get
| 18 |
where the last inequality holds true for any . In view of minimising the right-hand side of (18) over , we set
If minimises , then , where is given by
One can easily check that for
we have , and we we can compute
Thus, inequality (18) for becomes
| 19 |
Now, in order to rewrite the equation (19) in the form of (11), observe that (19) can equivalently be written as
where we have denoted . Now, multiplying the latter by we obtain
Observe that the first term of the RHS in the above inequality can be written as
| 20 |
while the second term of the RHS can be written as
| 21 |
Now, using (A11) we see that the (20) can be written as
| 22 |
Now a combination of (22) together with (21) gives that
We now treat the case where . We consider the auxiliary function given by
with given in (A13). Using Jensen’s inequality for the concave function and with the probability measure we estimate
Now, if we denote by the quantity
| 23 |
then, by the above computations, and the definition of the function G, we get
| 24 |
On the other hand, by the properties of the logarithm we have
| 25 |
Another application of Jensen’s inequality to the right-hand side of (25) yields
| 26 |
where has been computed in the Appendix, see (A14), as
| 27 |
A combination of (24) and (27) gives
| 28 |
or equivalently
and since for , the latter implies that
| 29 |
Now, by (23), and since clearly , we have
| 30 |
A combination of (29) and (30) gives
or, using the fact that , the latter can be simplified as follows
| 31 |
For as before, the right-hand side of (31) becomes
Therefore, for any the following inequality holds true
| 32 |
Our next aim is, as we did above, to minimise the quantity over . Now, if we set , then we can write
and we have
It is then not difficult to check that , for
A direct substitution then gives
| 33 |
Observe that since
and
the quantities and are common multiplies of the summands in (33), and we can write
| 34 |
where we have denoted by the following quantity
| 35 |
and for the last equality we have used the expression (A7) for .
where we have substituted the expression for I, and the latter implies that
Hence we get
or after simplifications
| 36 |
and we have obtained (11) for with given by
| 37 |
where we have used the expressions for S and given in (35) and (A13), respectively. This completes the proof of the Theorem 3.1.
The proving procedure that we followed in Theorem 3.1 show that the inequality (11) is sharp. Indeed we have the following result:
Corollary 3.2
The constant in the inequality (11) is sharp. In particular, when , the inequality (11) holds true as as equality for (up to -scaling) where and is given by (A6). Similarly, when , the inequality (11) holds true as an equality for (up to -scaling) , where , with given in (A13).
Remark 3.3
Using simple mathematical arguments, it is easy to check that the left-hand side of the inequality (11), i.e., of the Rényi entropy, when approximates the Shannon entropy, i.e., the quantity
In [3] the authors proved the optimal Shannon inequality on homogeneous Lie groups that reads as follows
| 38 |
where the best constant is given by
| 39 |
and we have , as expected, since the constant is optimal.
In the next result, by taking the limit of the Rényi entropy, we also obtain the so-called Shannon inequality on homogeneous Lie groups. As expected, the sharpness of the constant in the inequality (11) implies an optimal bound for the Shannon entropy in the aforesaid setting which is exactly the quantity on the right-hand side of (38).
Theorem 3.4
Let be a homogeneous Lie group of homogeneous dimension Q, and suppose that is a homogeneous quasi-norm on . Then Shannon inequality reads as follows
| 40 |
where
Proof
We first consider the case where . By Remark 3.3 it is enough to show that , where is given by Theorem 3.1 for , and . If we set , then , when , and we have
To calculate the quantity , when , we will use the Stirling approximation formula:
| 41 |
We have
Since
we have
and this completes the proof for the case where . As expected, the same constant appears when ; Indeed, for in (11) we have
| 42 |
Set
Then the first factor in (42) becomes
By the Stirling formula (41) we have
and so
| 43 |
Next, observe that
Hence, using that
we conclude that
and so also that
| 44 |
For the last factor we have
| 45 |
Combining (43), (44), and (45), we obtain
Summarising, we have shown that
completing the proof of (40) also in the case .
The next result applies to a subclass of graded Lie groups when as in (8) generates the whole of the Lie algebra. Precisely, if is a graded Lie group with Lie algebra that admits a gradation of the form (8) and where the elements , that are such that , generate after iterated commutators the whole of .
In the case of stratified Lie groups the elliptic Laplace operator on is replaced by the (positive) hypoelliptic operator , that is commonly called a sub-Laplacian on , defined via
and we have , where is called the horizontal gradient on ; namely is the vector valued operator acting on f via
Let us recall a version of the logarithmic Sobolev inequality in the setting of stratified groups, as appeared in [4, Corollary 7.1]
Proposition 3.5
Let be a stratified Lie group with homogeneous dimension Q. Then, the following log-Sobolev inequality is satisfied
| 46 |
for every f such that , for some constant A with a formula that depends on the group , and in particular it depends also on its homogeneous dimension Q.
Let us point out that the constant A is related to the Gagliardo-Nirenberg inequalities in our setting, and cannot be yet explicitly computed in the general setting of a stratified Lie group. However, in the particular case of the Heisenberg group we have , see [5, 11], and when we have , see [23], and in these cases A is optimal. To avoid technicalities we do not include the explicit formula for A here, and we refer the interested reader to [4, 9, 18].
Theorem 3.6
Let be a stratified Lie group with homogeneous dimension Q, and let be a nonnegative function such that Then it holds that
| 47 |
where the constant A is the one appearing in (46).
Proof
Let u be a function as in the hypothesis. The proof follows by inequality (46) for u such that .
As it happens in the Euclidean setting, see [12], a combination of the logarithmic Sobolev inequality of the form (47) with the Shannon’s inequality (40) yields the Heisenberg uncertainty principle in the setting of stratified groups.
Corollary 3.7
A version of the Heisenberg uncertainty principle in the setting of stratified groups reads as follows:
| 48 |
for all functions u as in the hypothesis of Theorem 3.6, where and A are given in (39), (46), respectively.
Proof
A combination of (47) and (38) gives
Therefore, we have
and by the properties of the logarithm, we obtain
and the proof is complete.
Acknowledgements
The authors are supported by the FWO Odysseus 1 grant G.0H94.18N: Analysis and Partial Differential Equations and by the Methusalem programme of the Ghent University Special Research Fund (BOF) (Grant number 01M01021). Michael Ruzhansky is also supported by EPSRC grants EP/R003025/2 and EP/V005529. M. Chatzakou is a postdoctoral fellow of the Research Foundation – Flanders (FWO) under the postdoctoral grant No 12B1223N.
Appendix A
For the proof of Theorem 3.1 we used the auxiliary functions , given by
| A1 |
where the constants and that we are computing below are such that , for . The computations that follow, is a homogeneous group adaptation of the analogous computation in the Euclidean setting developed in [22].
Using the polar decomposition for the function we can write
If we set , so that , then we can further compute
| A2 |
To further compute (A2), let us first recall the definition of the Beta function
| A3 |
where , with . Alternatively, one can obtain the following representation of B
| A4 |
For and , we have , so that with the use of (A4) and the property (see e.g. [22]) which relates the Beta function with the Gamma function
| A5 |
Now, the requirement , yields
| A6 |
Next we compute the -norm of . To this end, let us first compute the following integral
where we have used the change of variables implying that . Now, using (A4) the latter can be expressed in the following form
| A7 |
Recall the following property of the Gamma function
We then have
| A8 |
and
| A9 |
Using (A8) and (A9) in (A7), we obtain
| A10 |
where for the last equality we have used the formula for given in (A6).
Finally using (A10) and the fact that , we get
| A11 |
Let us now perform a similar analysis for the function as in (A1) where and . As before, we use polar decomposition and we have
For we get
| A12 |
Arguing as before and using the definition of the Beta function as in (A3) for and , the condition yields
and consequently, we get
| A13 |
Following the same arguments as before one can compute the -norm of and get
| A14 |
Declarations
associated data
All three authors have equally contributed. The manuscript has no associated data.
Footnotes
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Bobkov, S.G., Marsiglietti, A., Melbourne, J.: Concentration functions and entropy bounds for discrete log-concave distributions. Comb. Probab. Comput. 31, 54–72 (2021) [Google Scholar]
- 2.Carrillo, J.A., Toscani, G.: Rényi entropy and improved equilibration rates to self-similarity for nonlinear diffusion equations. Nonlinearity 27(12), 3159 (2014) [Google Scholar]
- 3.Chatzakou, M., Kassymov, A., Ruzhansky, M.: Anisotropic Shannon inequality. Osaka J. Math. 61, 79–89 (2024) [Google Scholar]
- 4.Chatzakou, M., Kassymov, A., Ruzhansky, M.: Logarithmic Sobolev-type inequalities on Lie groups. J. Geom. Anal. 34, 275 (2024) [Google Scholar]
- 5.Chatzakou, M., Kassymov, A., Ruzhansky, M.: Logarithmic Sobolev, Hardy and Poincaré inequalities on the Heisenberg group. ArXiv:2310.00992, (2023)
- 6.Conrad, K.: Probability distributions and maximum entropy. Entropy 6, no. 452 10, (2004)
- 7.Fischer, V., Ruzhansky, M.: Quantization on nilpotent Lie groups. Progress in Mathematics. vol. 314. Basel, Switzerland: Birkhuser, (2016)
- 8.Folland, G., Stein, E.M.: Hardy spaces on homogeneous groups. Math, vol. 28. Princeton University Press, Princeton, N.J, Notes (1982)
- 9.Ghosh, S., Kumar, V., Ruzhansky, M.: Best constants in subelliptic fractional Sobolev and Gagliardo-Nirenberg inequalities and ground states on stratified Lie groups. 10.48550/arXiv.2306.07657, (2023) [DOI] [PMC free article] [PubMed]
- 10.Ihara, S.: Information Theory for Continuous Systems. Vol. 2 World Scientific, (1993)
- 11.Jerison, D., Lee, J.M.: Extremals for the Sobolev inequality on the Heisenberg group and the CR Yamabe problem. J. Amer. Math. Soc. 1(1), 1–13 (1988) [Google Scholar]
- 12.Kubo, H., Ogawa, T., Suguro, T.: Beckner type of the logarithmic Sobolev and a new type of Shannon’s inequalities and an application to the uncertainty principle. Proc. Amer. Math. Soc. 147, 1511–1518 (2019) [Google Scholar]
- 13.Marsh, C.: Introduction to Continuous Entropy; Department of Computer Science. Princeton University, Princeton, NJ, USA (2013) [Google Scholar]
- 14.Newman, W.I.: A Lyapunov functional for the evolution of solutions to the porus medium equation to self-similarity. J. Math. Phys. 25, 3120–3123 (1984) [Google Scholar]
- 15.Ogawa, T., Wakui, H.: Non-uniform bound and finite time blow up for solutions to the drift-diffusion equation in higher dimensions. Anal. Appl. 15, 145–183 (2016) [Google Scholar]
- 16.Rényi, A.: Probability theory. North-Holland Pub. Co., Amsterdam (1970) [Google Scholar]
- 17.Rioul, O.: This is IT: A Primer on Shannon’s Entropy and Information. In: Duplantier, B., Rivasseau, V. (eds) Information Theory. Progress in Mathematical Physics, vol 78. Birkhäuser, Cham., (2021)
- 18.Ruzhansky, M., Tokmagambetov, N., Yessirkegenov, N.: Best constants in Sobolev and Gagliardo-Nirenberg inequalities on graded groups and ground states for higher order nonlinear subelliptic equations. Calc. Var. Partial Differ. Equ. 59, 23pp., (2020)
- 19.Savaré, G., Toscani, G.: The concavity of Rényi entropy power. IEEE Trans. Inf. Theory 60(5), 2687–2693 (2014) [Google Scholar]
- 20.Shannon, C.E.: A mathematical theory of communication. The Bell system technical journal 27(3), 379–423 (1948) [Google Scholar]
- 21.Stam, A.J.: Some inequalities satisfied by the quantities of information of Fisher and Shannon. Inf. Control 2, 102–112 (1959) [Google Scholar]
- 22.Suguro, T.: Shannon’s inequality for the Rényi entropy and an application to the uncertainty principle. J. Funct. Anal. 283(6), 109566 (2022). 10.1016/j.jfa.2022.109566 [Google Scholar]
- 23.Talenti, G.: Best constant in Sobolev inequality. Ann. di Mat. Pura ed Appl. 110(1), 353–372 (1976) [Google Scholar]
- 24.Toscani, G.: Remarks on entropy and equilibrium states. Ann. Mat. Pura Appl. 110, 353–372m (1976) [Google Scholar]
- 25.Villani, C.: A review of mathematical topics in collisional kinetic theory. In Handbook of mathematical fluid dynamics, Vol. I. North-Holland, Amsterdam, (2002), pp. 71–305
