Characterizations of matrix and operator-valued Φ-entropies, and operator Efron–Stein inequalities

Abstract

We derive new characterizations of the matrix Φ-entropy functionals introduced in Chen & Tropp (Chen, Tropp 2014 Electron. J. Prob. 19, 1–30. (doi:10.1214/ejp.v19-2964)). These characterizations help us to better understand the properties of matrix Φ-entropies, and are a powerful tool for establishing matrix concentration inequalities for random matrices. Then, we propose an operator-valued generalization of matrix Φ-entropy functionals, and prove the subadditivity under Löwner partial ordering. Our results demonstrate that the subadditivity of operator-valued Φ-entropies is equivalent to the convexity. As an application, we derive the operator Efron–Stein inequality.

1. Introduction

The introduction of Φ-entropy functionals can be traced back to the early days of information theory [1,2] and convex analysis [3–6], where the notion of ϕ-divergence is defined. Formally, given a non-negative real random variable Z and a smooth convex function Φ, the Φ-entropy functional refers to

$$H_{\mathrm{\Phi }}(Z)=\mathbb{E}\mathrm{\Phi }(Z)-\mathrm{\Phi }(\mathbb{E}Z).$$

By the Jensen inequality, it is not hard to see that the quantity H_Φ(Z) is non-negative. Hence, the Φ-entropy functional can be used as an entropic measure to characterize the uncertainty of the random variable Z.

The investigation of general properties of classical Φ-entropies has enjoyed great success in physics, probability theory, information theory and computer science. Of these, the subadditivity (or the tensorization) property [7–9] has led to the derivations of the logarithmic Sobolev [10], Φ-Sobolev [11] and Poincaré inequalities [12], which, in turn, is a crucial step towards the powerful entropy method in concentration inequalities [13–15] and analysis of Markov semigroups [16].

Let Z=f(X₁,…,X_n) be a random variable defined on n independent random variables (X₁,…,X_n). We say H_Φ(Z) is subadditive if

$$H_{\mathrm{\Phi }}(Z)\le \sum _{i=1}^n\mathbb{E}[{\mathbb{E}}_i\mathrm{\Phi }(Z)-\mathrm{\Phi }({\mathbb{E}}_{i}Z)],$$

where ${\mathbb{E}}_i$ denotes the conditional expectation with respect to X_i. Gross [10] first observed that the ordinary entropy functional $H_{u\log u}(Z)$ is subadditive in his seminal paper. Later on, equivalent characterizations of the subadditive entropy class (see theorem 2.2) were established [11,17,18], which prove to be useful in other contexts such as stochastic processes [17,18].

Parallel to the classical Φ-entropies, Chen & Tropp [19] introduced the notion of matrix Φ-entropy functionals. Namely, for a positive semi-definite random matrix Z, the matrix Φ-entropy functional is defined as

$$H_{\mathrm{\Phi }}(\mathit{\text{Z}})\triangleq \overline{\mathrm{tr}}\hspace{0.17em}[\mathbb{E}\mathrm{\Phi }(\mathit{\text{Z}})-\mathrm{\Phi }(\mathbb{E}\mathit{\text{Z}})],$$

where $\overline{\mathrm{tr}}$ is the normalized trace. The class of subadditive matrix Φ-entropy functionals is characterized in terms of the second derivative of their representing functions. Unlike its classical counterpart, only a few connections between the matrix Φ-entropy functionals and other convex forms of the same functions have been established [20,21] prior to this work.

In this paper, we establish equivalent characterizations of the matrix Φ-entropy functionals defined in [19]. Our results show that matrix Φ-entropy functionals satisfy all known equivalent statements that classical Φ-entropy functions satisfy [15,17,18]. Our results provide additional justification for the original definition of the matrix Φ-entropy functionals (table 1). The equivalences between matrix Φ-entropy functionals and other convex forms of the function Φ advance our understanding of the class of entropy functions. Moreover, it allows us to unify the study of matrix concentration inequalities and matrix Φ-Sobolev inequalities [22,23].

Table 1.

Comparison between the equivalent characterizations of the Φ-entropy functional class (C1) (definition 2.1) and the matrix Φ-entropy functional class (C2) (definition 3.1).

	classical Φ-entropy functional class (C1)	matrix Φ-entropy functional class (C2)
(a)	Φ is affine or Φ′′>0 and 1/Φ′′ is concave	Φ is affine or DΦ′ is invertible and (DΦ′)−1 is concave
(b)	convexity of (u,v)↦Φ(u+v)−Φ(u)−Φ′(u)v	convexity of (u,v)↦Tr[Φ(u+v)−Φ(u)−DΦ[u](v)]
(c)	convexity of (u,v)↦(Φ′(u+v)−Φ′(u))v	convexity of (u,v)↦Tr[(DΦ[u+v](v)−DΦ[u](v)]
(d)	convexity of (u,v)↦Φ′′(u)v2	convexity of (u,v)↦Tr[D2Φ[u](v,v)]
(e)	Φ is affine or Φ′′>0 and Φ′′′′Φ′′≥2Φ′′′2	equation (3.2)
(f)	convexity of (u,v)↦tΦ(u)+(1−t)Φ(v)−Φ(tu+(1−t)v) for any 0≤t≤1	convexity of (u,v)↦ Tr[tΦ(u)+(1−t)Φ(v)−Φ(tu+(1−t)v)] for any 0≤t≤1
(g)	${\mathbb{E}}_1{H}_{\mathrm{\Phi }}(Z|X_1)\ge H_{\mathrm{\Phi }}({\mathbb{E}}_{1}Z)$	${\mathbb{E}}_1{H}_{\mathrm{\Phi }}(\mathit{\text{Z}}|{\mathit{\text{X}}}_1)\ge H_{\mathrm{\Phi }}({\mathbb{E}}_1\mathit{\text{Z}})$
(h)	HΦ(Z) is a convex function of Z	HΦ(Z) is a convex function of Z
(i)	$H_{\mathrm{\Phi }}(Z)=\underset{T>0}{sup}\{\mathbb{E}[{\mathrm{{\rm Y}}}_1(T)\cdot Z+{\mathrm{{\rm Y}}}_2(T)]\}$	$H_{\mathrm{\Phi }}(\mathit{\text{Z}})=\underset{\mathit{\text{T}}\succ \mathbf{\text{0}}}{sup}\{\overline{\mathrm{tr}}\mathbb{E}[{\mathbf{{\rm Y}}}_1(\mathit{\text{T}})\cdot \mathit{\text{Z}}+{\mathbf{{\rm Y}}}_2(\mathit{\text{T}})]\}$
(j)	$H_{\mathrm{\Phi }}(Z)\le \sum _{i=1}^n\mathbb{E}{H}_{\mathrm{\Phi }}^{(i)}(Z)$	$H_{\mathrm{\Phi }}(\mathit{\text{Z}})\le \sum _{i=1}^n\mathbb{E}{H}_{\mathrm{\Phi }}^{(i)}(\mathit{\text{Z}})$

Furthermore, we consider the following operator-valued generalization of matrix Φ-entropy functionals:

$${\mathit{\text{H}}}_{\mathrm{\Phi }}(\mathit{\text{Z}})\triangleq \mathbb{E}\mathrm{\Phi }(\mathit{\text{Z}})-\mathrm{\Phi }(\mathbb{E}\mathit{\text{Z}}).$$

A special case of this operator-valued Φ-entropy functional is the operator-valued variance Var(Z) defined in [24,25], where Φ is the square function. The equivalent conditions for the subadditivity under Löwner partial ordering are derived (theorem 4.5). In particular, we show that subadditivity of the operator-valued Φ-entropies is equivalent to the convexity,

$$\text{subadditvity of}\hspace{0.17em}{\mathit{\text{H}}}_{\mathrm{\Phi }}(\mathit{\text{Z}})\hspace{0.17em}\iff \hspace{0.17em}{\mathit{\text{H}}}_{\mathrm{\Phi }}(\mathit{\text{Z}})\text{ is convex in }\mathit{\text{Z}}.$$

Our result directly yields the operator Efron–Stein inequality, which recovers the well-known Efron–Stein inequality [26,27] when random matrices reduce to real random variables.

(a). Our results

We summarize our results here. First, we derive equivalent characterizations for the matrix Φ-entropy functionals in table 1 (see theorem 3.3). Notably, all known equivalent characterizations for the classical Φ-entropies can be generalized to their matrix correspondences. We emphasize that additional characterizations of the Φ-entropies prove to be useful in many instances. The characterizations (b)–(d) in (C1) are explored by Chafaï [18] to derive several entropic inequalities for M/M/$\mathrm{\infty }$ queueing processes that are not diffusions. With the characterizations (b)–(d), the difficulty of lacking the diffusion property can be circumvented and replaced by convexity. Moreover, as shown in corollary 4.8, item (f) in table 1 can be used to demonstrate an interesting result in quantum information theory: the matrix Φ-entropy functional of a quantum ensemble (i.e. a set of quantum states with some prior distribution) is monotone under any unital quantum channel. This property motivates us to study the dynamical evolution of a quantum ensemble and its mixing time, a fundamentally important problem in quantum computation (see our follow-up work [23] for further details).

Second, we define and derive equivalent characterizations for operator-valued Φ-entropies in table 2 (see theorem 4.5). Note that the only known statement in table 1 that is missing in table 2 is condition (e). In other words, we are not able to generalize (e) in table 1 to the non-commutative case. Finally, we employ the subadditivity of operator-valued Φ-entropies to show the operator Efron–Stein inequality in theorem 5.1.

Table 2.

Equivalent statements of the operator-valued Φ-entropy class (C3) (definition 4.1).

	operator-valued Φ-entropy class (C3)
(a)	the second-order Fréchet derivative D2Φ[u](v,v) is jointly convex in (u,v)
(b)	Φ(u+v)−Φ(u)−DΦ[u](v) is jointly convex in (u,v)
(c)	DΦ[u+v](v)−DΦ[u](v) is jointly convex in (u,v)
(d)	D2Φ[u](v,v) is jointly convex in (u,v)
(e)	tΦ(u)+(1−t)Φ(v)−Φ(tu+(1−t)v) is jointly convex in (u,v) for any 0≤t≤1
(f)	${\mathbb{E}}_1{\mathit{\text{H}}}_{\mathrm{\Phi }}(\mathit{\text{Z}}|{\mathit{\text{X}}}_1)⪰{\mathit{\text{H}}}_{\mathrm{\Phi }}({\mathbb{E}}_1\mathit{\text{Z}})$
(g)	HΦ(Z) is a convex function of Z
(h)	${\mathit{\text{H}}}_{\mathrm{\Phi }}(\mathit{\text{Z}})=\underset{\mathit{\text{T}}\succ \mathbf{\text{0}}}{sup}\{\mathbb{E}[{\mathbf{{\rm Y}}}_1(\mathit{\text{T}})\cdot \mathit{\text{Z}}+{\mathbf{{\rm Y}}}_2(\mathit{\text{T}})]\}$
(i)	${\mathit{\text{H}}}_{\mathrm{\Phi }}(\mathit{\text{Z}})⪯\sum _{i=1}^n\mathbb{E}{\mathit{\text{H}}}_{\mathrm{\Phi }}^{(i)}(\mathit{\text{Z}})$

(b). Prior work

For the history of the equivalent characterizations in the class (C1), we refer to an excellent textbook [15] and papers [17,18].

The original definition of the matrix Φ-entropy class, namely (a) in (C2), was proposed by Chen & Tropp in 2014 [19]. In the same paper, they also established the subadditivity property (j) through (i) and (g): (a)⇒(i)⇒(g)⇒(j) in table 1. Shortly after, the equivalent relation between (a) and the joint convexity of the matrix Brégman divergence (b) was proved in [21]. The equivalent relation between (a) and (d) was almost immediately implied by the result in [20] (see the detailed discussion in the proof of theorem 3.3). The convexity of H_Φ(Z), (h), is noted in [20]. Here, we provide transparent evidence—the joint convexity of (f).

We organize the paper in the following way. We collect the necessary information for the matrix algebra in §2. The equivalent characterizations of the matrix Φ-entropy functionals are provided in §3. We define the operator-valued Φ-entropies and derive their equivalent statements in §4. Section 5 shows an application of the subadditivity—the operator Efron–Stein inequality. The proofs of the main results are collected in §6 and §7, respectively. Finally, we conclude the paper.

2. Preliminaries

We first introduce basic notation.

The set ${\mathbb{M}}^{\mathrm{sa}}$ refers to the subspace of self-adjoint operators on some separable Hilbert space. We denote by ${\mathbb{M}}^{+}$ (resp. ${\mathbb{M}}^{++}$) the set of positive semi-definite (resp. positive-definite) operators in ${\mathbb{M}}^{\mathrm{sa}}$. If the dimension d of a Hilbert space needs special attention, then we highlight it in subscripts, e.g. ${\mathbb{M}}_d$ denotes the Banach space of d×d complex matrices. The trace function $\mathrm{Tr}:{\mathbb{C}}^{d\times d}\to \mathbb{C}$ is defined as the summation of eigenvalues. The normalized trace function $\overline{\mathrm{tr}}$ for every d×d matrix M is denoted by $\overline{\mathrm{tr}}\hspace{0.17em}[\mathit{\text{M}}]\triangleq (1/d)\mathrm{Tr}[\mathit{\text{M}}]$. For $p\in [1,\mathrm{\infty })$, the Schatten p-norm of an operator M is denoted as $\parallel \mathit{\text{M}}{\parallel }_p\triangleq (\sum _i|{\mathrm{\lambda }}_i{(\mathit{\text{M}}){|}^p)}^{1/p}$, where {λ_i(M)} are the singular values of M. The Hilbert–Schmidt inner product is defined as $⟨\mathit{\text{A}},\mathit{\text{B}}⟩\triangleq \mathrm{Tr}\hspace{0.17em}{\mathit{\text{A}}}^{†}\mathit{\text{B}}$. For $\mathit{\text{A}},\mathit{\text{B}}\in {\mathbb{M}}^{\mathrm{sa}}$, $\mathit{A}\underset{\_}{\succ }\mathit{B}$ means that A−B is positive semi-definite. Similarly, A≻B means A−B is positive-definite. Throughout this paper, italic capital letters (e.g. X) are used to denote operators.

Denote a probability space $(\mathrm{\Omega },\mathrm{\Sigma },\mathbb{P})$. A random matrix Z defined on the probability space $(\mathrm{\Omega },\mathrm{\Sigma },\mathbb{P})$ means that it is a matrix-valued random variable defined on Ω. We denote the expectation of Z with respect to $\mathbb{P}$ by

$${\mathbb{E}}_{\mathbb{P}}[\mathit{\text{Z}}]\triangleq {\int }_{\mathrm{\Omega }}\mathit{\text{Z}}\hspace{0.17em}\mathrm{d}\mu ={\int }_{x\in \mathrm{\Omega }}\mathit{\text{Z}}(x)\mathbb{P}(\mathrm{d}x),$$

where the integral is the Bochner integral [28,29]. We note that the results derived in this paper are universal for all probability spaces. Hence, we will omit the subscript $\mathbb{P}$ of the expectation. If we consider a sample space Ω₁×Ω₂ with joint distribution $\mathbb{P}$, then we denote the conditional expectation of Z with respect to the first space Ω₁ by ${\mathbb{E}}_i[\mathit{\text{Z}}]\triangleq {\int }_{x_1\in {\mathrm{\Omega }}_1}\mathit{\text{Z}}(x1,X_2\cdot ){\mathbb{P}}_1(\mathrm{d}{x}_1)$, where ${\mathbb{P}}_1(x_1)={\int }_{x_2\in {\mathrm{\Omega }}_2}\mathbb{P}(x_1,\mathrm{d}{x}_2)$ is the marginal distribution on Ω₁.

Let $\mathcal{U},\mathcal{W}$ be real Banach spaces. The Fréchet derivative of a function $\mathcal{L}:\mathcal{U}\to \mathcal{W}$ at a point $\mathit{\text{X}}\in \mathcal{U}$, if it exists,¹ is a unique linear mapping $\mathsf{D}\mathcal{L}[\mathit{\text{X}}]:\mathcal{U}\to \mathcal{W}$ such that

$$\parallel \mathcal{L}(\mathit{\text{X}}+\mathit{\text{E}})-\mathcal{L}(\mathit{\text{X}})-\mathsf{D}\mathcal{L}[\mathit{\text{X}}](\mathit{\text{E}}){\parallel }_{\mathcal{W}}=o(\parallel \mathit{\text{E}}{\parallel }_{\mathcal{U}}),$$

where $\parallel \cdot {\parallel }_{\mathcal{U}(\mathcal{W})}$ is a norm in $\mathcal{U}$ (resp. $\mathcal{W}$). The notation $\mathsf{D}\mathcal{L}[\mathit{\text{X}}](\mathit{\text{E}})$ then is interpreted as ‘the Fréchet derivative of $\mathcal{L}$ at X in the direction E’. The partial Fréchet derivative of multivariate functions can be defined as follows. Let $\mathcal{U},\mathcal{V}$ and $\mathcal{W}$ be real Banach spaces, $\mathcal{L}:\mathcal{U}\times \mathcal{V}\to \mathcal{W}$. For a fixed ${\mathit{\text{v}}}_0\in \mathcal{V}$, $\mathcal{L}(\mathit{\text{u}},{\mathit{\text{v}}}_0)$ is a function of u whose derivative at u₀, if it exists, is called the partial Fréchet derivative of $\mathcal{L}$ with respect to u, and is denoted by ${\mathsf{D}}_{\mathit{\text{u}}}\mathcal{L}[{\mathit{\text{u}}}_0,{\mathit{\text{v}}}_0]$. The partial Fréchet derivative ${\mathsf{D}}_{\mathit{\text{v}}}\mathcal{L}[{\mathit{\text{u}}}_0,{\mathit{\text{v}}}_0]$ is defined similarly. Likewise, the mth Fréchet derivative ${\mathsf{D}}^m\mathcal{L}[\mathit{\text{X}}]$ is a unique multi-linear map from ${\mathcal{U}}^m\triangleq \mathcal{U}\times \cdots \times \mathcal{U}$ (m times) to $\mathcal{W}$ that satisfies

$$\begin{array}{rl}& \parallel {\mathsf{D}}^{m-1}\mathcal{L}[\mathit{\text{X}}+{\mathit{\text{E}}}_m]({\mathit{\text{E}}}_1,\dots ,{\mathit{\text{E}}}_{m-1})-{\mathsf{D}}^{m-1}\mathcal{L}[\mathit{\text{X}}]({\mathit{\text{E}}}_1,\dots ,{\mathit{\text{E}}}_{m-1})\\ & -{\mathsf{D}}^m\mathcal{L}[\mathit{\text{X}}]({\mathit{\text{E}}}_1,\dots ,{\mathit{\text{E}}}_m){\parallel }_{\mathcal{W}}=o(\parallel {\mathit{\text{E}}}_m{\parallel }_{\mathcal{U}})\end{array}$$

for each ${\mathit{\text{E}}}_i\in \mathcal{U},i=1,\dots ,m$. The Fréchet derivative enjoys several properties as in standard derivatives. We provide these facts in appendix A.

A function $f:I\to \mathbb{R}$ is called operator convex if, for each $\mathit{\text{A}},\mathit{\text{B}}\in {\mathbb{M}}^{\mathrm{sa}}(I)$ and 0≤t≤1,

$$f(t\mathit{\text{A}})+f((1-t)\mathit{\text{B}})⪯f(t\mathit{\text{A}}+(1-t)\mathit{\text{B}}).$$

Similarly, a function $f:I\to \mathbb{R}$ is called operator monotone if, for each $\mathit{\text{A}},\mathit{\text{B}}\in {\mathbb{M}}^{\mathrm{sa}}(I)$,

$$\mathit{\text{A}}⪯\mathit{\text{B}}\hspace{0.17em}\Rightarrow \hspace{0.17em}f(\mathit{\text{A}})⪯f(\mathit{\text{B}}).$$

(a). Classical Φ-entropy functionals

Let (C1) denote the class of functions $\mathrm{\Phi }:[0,\mathrm{\infty })\to \mathbb{R}$ that are continuous, convex on $[0,\mathrm{\infty })$, twice differentiable on $(0,\mathrm{\infty })$, and either Φ is affine or Φ′′ is strictly positive and 1/Φ′′ is concave.

Definition 2.1 (Classical Φ-entropies) —

Let $\mathrm{\Phi }:[0,\mathrm{\infty })\to \mathbb{R}$ be a convex function. For every non-negative integrable random variable Z, so that $\mathbb{E}|Z|<\mathrm{\infty }$ and $\mathbb{E}|\mathrm{\Phi }(Z)|<\mathrm{\infty }$, the classical Φ-entropy H_Φ(Z) is defined as

$$H_{\mathrm{\Phi }}(Z)=\mathbb{E}\mathrm{\Phi }(Z)-\mathrm{\Phi }(\mathbb{E}Z).$$

In particular, we are interested in Z=f(X₁,…,X_n), where X₁,…,X_n are independent random variables and f≥0 is a measurable function.

We say H_Φ(Z) is subadditive [9] if

$$H_{\mathrm{\Phi }}(Z)\le \sum _{i=1}^n\mathbb{E}[H_{\mathrm{\Phi }}^{(i)}(Z)],$$

where $H_{\mathrm{\Phi }}^{(i)}(Z)={\mathbb{E}}_i\mathrm{\Phi }(Z)-\mathrm{\Phi }({\mathbb{E}}_{i}Z)$ is the conditional Φ-entropy, and ${\mathbb{E}}_i$ denotes the conditional expectation conditioned on the n−1 random variables $X_{-i}\triangleq (X_1\dots ,X_{i-1},X_{i+1},\dots ,X_n)$. Sometimes, we also denote $H_{\mathrm{\Phi }}^{(i)}(Z)$ by H_Φ(Z|X_−i).

It is a well-known result that, for any function Φ∈(C1), H_Φ(Z) is subadditive [11, corollary 3] (see also [13, section 3]).

Theorem 2.2 establishes equivalent characterizations of classical Φ-entropies.

Theorem 2.2 [18, theorem 4.4] —

The following statements are equivalent:

• (a) Φ∈(C1): Φ is affine, or Φ′′>0 and 1/Φ′′ is concave;

• (b) Brégman divergence (u,v)↦Φ(u+v)−Φ(u)−Φ′(u)v is convex;

• (c) (u,v)↦(Φ′(u+v)−Φ′(u))v is convex;

• (d) (u,v)↦Φ′′(u)v² is convex;

• (e) Φ is affine or Φ′′>0 and Φ′′′′Φ′′≥2Φ′′′²;

• (f) (u,v)↦tΦ(u)+(1−t)Φ(v)−Φ(tu+(1−t)v) is convex for any 0≤t≤1;

• (g) ${\mathbb{E}}_1{H}_{\mathrm{\Phi }}(Z|X_1)\ge H_{\mathrm{\Phi }}({\mathbb{E}}_{1}Z);$

• (h) {H_Φ(Z)}_Φ∈(C1) forms a convex set;

• (i) $H_{\mathrm{\Phi }}(Z)=\underset{T>0}{sup}(\mathbb{E}[({\mathrm{\Phi }}^{\prime }(T)-{\mathrm{\Phi }}^{\prime }(\mathbb{E}\hspace{0.17em}T))(Z-T)]+H_{\mathrm{\Phi }}(T))$; and

• (j) $H_{\mathrm{\Phi }}(Z)\le \sum _{i=1}^n\mathbb{E}{H}_{\mathrm{\Phi }}^{(i)}(Z)$.

3. Equivalent characterizations of matrix Φ-entropy functionals

Here, we first introduce matrix Φ-entropy functionals, and present the main result (theorem 3.3) of this section, namely new characterizations of the matrix Φ-entropy functionals.

Chen & Tropp [19] introduce the class of matrix Φ-entropies, and prove its subadditivity in 2014. Here, we will show that all equivalent characterizations of classical Φ-entropies in theorem 2.2 have a one-to-one correspondence for the class of matrix Φ-entropies.

Let d be a natural number. The class Φ_d contains each function $\mathrm{\Phi }:(0,\mathrm{\infty })\to \mathbb{R}$ that is either affine or satisfies the following three conditions.

• (i) Φ is convex and continuous at zero.

• (ii) Φ is twice continuously differentiable.

• (iii) Define Ψ(t)=Φ′(t) for t>0. The Fréchet derivative DΨ of the standard matrix function $\mathrm{\Psi }:{\mathbb{M}}_d^{++}\to {\mathbb{M}}_d^{\mathrm{sa}}$ is an invertible linear map on ${\mathbb{M}}_d^{++}$, and the map A↦(DΨ[A])⁻¹ is concave with respect to the Löwner partial ordering on positive definite matrices.

Define $\mathbf{\text{(C2)}}\triangleq {\mathrm{\Phi }}_{\mathrm{\infty }}\equiv \bigcap _{d=1}^{\mathrm{\infty }}{\mathrm{\Phi }}_d$.

Definition 3.1 (Matrix Φ-entropy functional [19]) —

Let $\mathrm{\Phi }:[0,\mathrm{\infty })\to \mathbb{R}$ be a convex function. Consider a random matrix $\mathit{\text{Z}}\in {\mathbb{M}}_d^{+}$ with $\mathbb{E}\parallel \mathit{\text{Z}}{\parallel }_{\mathrm{\infty }}<\mathrm{\infty }$ and $\mathbb{E}\parallel \mathrm{\Phi }(\mathit{\text{Z}}){\parallel }_{\mathrm{\infty }}<\mathrm{\infty }$. The matrix Φ-entropy H_Φ(Z) is defined as

$$H_{\mathrm{\Phi }}(\mathit{\text{Z}})\triangleq \overline{\mathrm{tr}}\hspace{0.17em}[\mathbb{E}\mathrm{\Phi }(\mathit{\text{Z}})-\mathrm{\Phi }(\mathbb{E}\mathit{\text{Z}})].$$

The corresponding conditional matrix Φ-entropy can be defined under the σ-algebra.

Theorem 3.2 (Subadditivity of the matrix Φ-entropy functional [19, theorem 2.5]) —

Let Φ∈(C2), and assume Z is a measurable function of (X₁,…,X_n). We have

$$H_{\mathrm{\Phi }}(\mathit{\text{Z}})\le \sum _{i=1}^n\mathbb{E}[H_{\mathrm{\Phi }}^{(i)}(\mathit{\text{Z}})],$$ 3.1

where $H_{\mathrm{\Phi }}^{(i)}(\mathit{\text{Z}})={\mathbb{E}}_i\mathrm{\Phi }(\mathit{\text{Z}})-\mathrm{\Phi }({\mathbb{E}}_i\mathit{\text{Z}})$ is the conditional entropy, and ${\mathbb{E}}_i$ denotes the conditional expectation conditioned on the n−1 random matrices ${\mathit{\text{X}}}_{-i}\triangleq ({\mathit{\text{X}}}_1,\dots ,{\mathit{\text{X}}}_{i-1},{\mathit{\text{X}}}_{i+1},\dots ,{\mathit{\text{X}}}_n)$.

Theorem 3.3 is the main result of this section. We show that all the equivalent conditions in theorem 2.2 also hold for the class of matrix Φ-entropy functionals. Hence, we have a more comprehensive understanding of the class of matrix Φ-entropy functionals.

Theorem 3.3 —

The following statements are equivalent:

• (a) Φ∈(C2): Φ is affine or DΨ is invertible and A↦(DΨ[A])⁻¹ is operator concave;

• (b) matrix Brégman divergence: (A,B)↦Tr[Φ(A+B)−Φ(A)−DΦ[A](B)] is convex;

• (c) (A,B)↦Tr[DΦ[A+B](B)−DΦ[A](B)] is convex;

• (d) (A,B)↦Tr[D²Φ[A](B,B)] is convex;

• (e) Φ is affine or Φ′′>0 and

  $$\begin{array}{rl}& \mathrm{Tr}[\mathit{\text{h}}\cdot {(\mathsf{D}\mathrm{\Psi }[\mathit{\text{A}}])}^{-1}\circ {\mathsf{D}}^3\mathrm{\Psi }[\mathit{\text{A}}](\mathit{\text{k}},\mathit{\text{k}},{(\mathsf{D}\mathrm{\Psi }[\mathit{\text{A}}])}^{-1}(\mathit{\text{h}}))]\\ & \ge 2\mathrm{Tr}[\mathit{\text{h}}\cdot {(\mathsf{D}\mathrm{\Psi }[\mathit{\text{A}}])}^{-1}\circ {\mathsf{D}}^2\mathrm{\Psi }[\mathit{\text{A}}](\mathit{\text{k}},{(\mathsf{D}\mathrm{\Psi }[\mathit{\text{A}}])}^{-1}({\mathsf{D}}^2\mathrm{\Psi }[\mathit{\text{A}}](\mathit{\text{k}},{(\mathsf{D}\mathrm{\Psi }[\mathit{\text{A}}])}^{-1}(\mathit{\text{h}}))))],\end{array}$$ 3.2

  for each $\mathit{A}\underset{\_}{\succ }\mathbf{0}$ and $\mathit{\text{h}},\mathit{\text{k}}\in {\mathbb{M}}_d^{\mathrm{sa}};$

• (f) (A,B)↦ Tr[tΦ(A)+(1−t)Φ(B)−Φ(tA+(1−t)B)] is convex for any 0≤t≤1;

• (g) ${\mathbb{E}}_1{H}_{\mathrm{\Phi }}(\mathit{\text{Z}}|{\mathit{\text{X}}}_1)\ge H_{\mathrm{\Phi }}({\mathbb{E}}_1\mathit{\text{Z}});$

• (h) {H_Φ(Z)}_Φ∈(C2) forms a convex set of convex functions;

• (i) $H_{\mathrm{\Phi }}(\mathit{\text{Z}})=\underset{\mathit{\text{T}}\succ \mathbf{\text{0}}}{sup}\{\mathrm{Tr}\mathbb{E}[({\mathrm{\Phi }}^{\prime }(\mathit{\text{T}})-{\mathrm{\Phi }}^{\prime }(\mathbb{E}\mathit{\text{T}}))(\mathit{\text{Z}}-\mathit{\text{T}})]+H_{\mathrm{\Phi }}(\mathit{\text{T}})\}$; and

• (j) $H_{\mathrm{\Phi }}(\mathit{\text{Z}})\le \sum _{i=1}^n\mathbb{E}{H}_{\mathrm{\Phi }}^{(i)}(Z)$.

We note that the statements (a)⇒(i)⇒(g)⇒(j) was proved by Chen & Tropp [19]. The equivalence of (a)⇔(b) was shown in [21, theorem 2]. Hansen and Zhang [20] established an equivalence of item (a) and the convexity of the following map:

$$(\mathit{\text{A}},\mathit{\text{X}})\mapsto ⟨\mathit{\text{X}},\mathsf{D}{\mathrm{\Phi }}^{\prime }[\mathit{\text{A}}](\mathit{\text{X}})⟩.$$ 3.3

From lemma A.5, it is not hard to observe that equation (3.3) is equivalent to item (d), i.e.

$$\mathrm{Tr}({\mathsf{D}}^2\mathrm{\Phi }[\mathit{\text{A}}](\mathit{\text{X}},\mathit{\text{X}}))=⟨\mathit{\text{X}},\mathsf{D}{\mathrm{\Phi }}^{\prime }[\mathit{\text{A}}](\mathit{\text{X}})⟩.$$

We provide the detailed proof of the remaining equivalence statements in §6.

4. Operator-valued Φ-entropies

Here, we extend the notion of matrix Φ-entropy functionals (i.e. real-valued) to operator-valued Φ-entropies.

Definition 4.1 (Operator-valued entropy class) —

Let d be a natural number. The class Φ_d contains each function $\mathrm{\Phi }:[0,\mathrm{\infty })\to \mathbb{R}$ such that its second-order Fréchet derivative exists and the following map satisfies the joint convexity (under the Löwner partial ordering) condition:

$$(\mathit{\text{A}},\mathit{\text{X}})\mapsto {\mathsf{D}}^2\mathrm{\Phi }[\mathit{\text{A}}](\mathit{\text{X}},\mathit{\text{X}}),\mathrm{\forall }\hspace{0.17em}\mathit{\text{A}}\in {\mathbb{M}}_d^{\mathrm{sa}},\hspace{0.17em}\mathit{\text{X}}\in {\mathbb{M}}_d.$$ 4.1

We denote the class of operator-valued Φ-entropies ${\mathrm{\Phi }}_{\mathrm{\infty }}\triangleq \bigcap _{d=1}^{\mathrm{\infty }}{\mathrm{\Phi }}_d$ by (C3).

Definition 4.2 (Operator-valued Φ-entropies) —

Let $\mathrm{\Phi }:[0,\mathrm{\infty })\to \mathbb{R}$ be a convex function. Consider a random matrix Z taking values in ${\mathbb{M}}^{+}$, with $\mathbb{E}\parallel \mathit{\text{Z}}{\parallel }_{\mathrm{\infty }}<\mathrm{\infty }$ and $\mathbb{E}\parallel \mathrm{\Phi }(\mathit{\text{Z}}){\parallel }_{\mathrm{\infty }}<\mathrm{\infty }$. That is, the random matrix Z and Φ(Z) are Bochner integrable [28,29] (hence, $\mathbb{E}\mathit{\text{Z}}$ and $\mathbb{E}\mathrm{\Phi }(\mathit{\text{Z}})$ exist and are well defined). The operator-valued Φ-entropy H_Φ is defined as

$${\mathit{\text{H}}}_{\mathrm{\Phi }}(\mathit{\text{Z}})\triangleq \mathbb{E}\mathrm{\Phi }(\mathit{\text{Z}})-\mathrm{\Phi }(\mathbb{E}\mathit{\text{Z}}).$$ 4.2

The corresponding conditional terms can be defined under the σ algebra.

It is worth mentioning that the matrix Φ-entropy functional [19] in §3 is non-negative for every convex function Φ due to the fact that the trace function $\overline{\mathrm{tr}}\mathrm{\Phi }$ is also convex [32] (or see, for example, [33, section 2.2]). However, according to the operator Jensen inequality [34, theorem 3.2], only the operator convex function Φ ensures that the operator-valued Φ-entropy is non-negative.

In the following, we show that the entropy class (C3) is not an empty set.

Proposition 4.3 —

The square function Φ(u)=u² belongs to (C3).

Proof. —

It suffices to verify the joint convexity of the map

$${\mathsf{D}}^2\mathrm{\Phi }[\mathit{\text{A}}](\mathit{\text{X}},\mathit{\text{X}})=2{\mathit{\text{X}}}^2\text{for all }\mathit{\text{A}},\mathit{\text{X}}\in {\mathbb{M}}^{+},$$

where we use the identity of the second-order Fréchet derivative [35, example X.4.6] of the square function. Because the square function is operator convex, Φ(u)=u² belongs to the operator-valued Φ-entropy class (C3). ▪

(a). Subadditivity of operator-valued Φ-entropies

Denote by $\mathit{\text{X}}\triangleq ({\mathit{\text{X}}}_1,\dots ,{\mathit{\text{X}}}_n)$ a series of independent random variables taking values in a Polish space, and let

$${\mathit{\text{X}}}_{-i}\triangleq ({\mathit{\text{X}}}_1,\dots ,{\mathit{\text{X}}}_{i-1},{\mathit{\text{X}}}_{i+1},\dots ,{\mathit{\text{X}}}_n).$$

Let a positive semi-definite matrix Z that depends on the series of random variables X be defined as

$$\mathit{\text{Z}}\triangleq \mathit{\text{Z}}({\mathit{\text{X}}}_1,\dots ,{\mathit{\text{X}}}_n)\in {\mathbb{M}}^{+}.$$

Throughout this paper, we assume that the random matrix Z satisfies the integrability conditions: $|\mathit{\text{Z}}|\triangleq \sqrt{{\mathit{\text{Z}}}^2}$ and |Φ(Z)| is Bochner integrable for Φ∈(C3).

Theorem 4.4 (Subadditivity of the operator-valued Φ-entropy) —

Fix a function Φ∈(C3). Under the prevailing assumptions,

$${\mathit{\text{H}}}_{\mathrm{\Phi }}(\mathit{\text{Z}})⪯\sum _{i=1}^n\mathbb{E}[{\mathit{\text{H}}}_{\mathrm{\Phi }}^{(i)}(\mathit{\text{Z}})],$$ 4.3

where ${\mathit{\text{H}}}_{\mathrm{\Phi }}^{(i)}(\mathit{\text{Z}})={\mathit{\text{H}}}_{\mathrm{\Phi }}(\mathit{\text{Z}}|{\mathit{\text{X}}}_{-i})\triangleq {\mathbb{E}}_i\mathrm{\Phi }(\mathit{\text{Z}})-\mathrm{\Phi }({\mathbb{E}}_i\mathit{\text{Z}})$.

The proof is given in §7.

(b). Equivalent characterizations of operator-valued Φ-entropies

Here, we derive alternative characterizations of the class (C3) in theorem 4.5. As an application of the entropy class, we show that if the function Φ belongs to (C3), then the operator-valued Φ-entropy is monotone under any unital completely positive map.

Theorem 4.5 —

The following statements are equivalent:

• (a) Φ∈(C3): convexity of (A,B)↦D²Φ[A](B,B);

• (b) operator-valued Brégman divergence: (A,B)↦Φ(A+B)−Φ(A)−DΦ[A](B) is convex;

• (c) (A,B)↦DΦ[A+B](B)−DΦ[A](B) is convex;

• (d) (A,B)↦D²Φ[A](B,B) is convex;

• (e) convexity of (A,B)↦ tΦ(A)+(1−t)Φ(B)−Φ(tA+(1−t)B) for any 0≤t≤1;

• (f) ${\mathbb{E}}_1{\mathit{\text{H}}}_{\mathrm{\Phi }}(\mathit{\text{Z}}|{\mathit{\text{X}}}_1)⪰{\mathit{\text{H}}}_{\mathrm{\Phi }}({\mathbb{E}}_1\mathit{\text{Z}})$;

• (g) {H_Φ(Z)}_Φ∈(C3) forms a convex set of convex functions;

• (h) ${\mathit{\text{H}}}_{\mathrm{\Phi }}(\mathit{\text{Z}})=\underset{\mathit{\text{T}}\succ \mathbf{\text{0}}}{sup}\{\mathbb{E}[\mathsf{D}\mathrm{\Phi }[\mathit{\text{T}}](\mathit{\text{Z}}-\mathit{\text{T}})-\mathsf{D}\mathrm{\Phi }[\mathbb{E}\mathit{\text{T}}](\mathit{\text{Z}}-\mathit{\text{T}})]+{\mathit{\text{H}}}_{\mathrm{\Phi }}(\mathit{\text{T}})\}$; and

• (i) ${\mathit{\text{H}}}_{\mathrm{\Phi }}(\mathit{\text{Z}})⪯\sum _{i=1}^n\mathbb{E}{\mathit{\text{H}}}_{\mathrm{\Phi }}^{(i)}(\mathit{\text{Z}})$.

The proof is omitted, because it directly follows from that of theorem 3.3 without taking traces.

Remark 4.6 —

In item (g) of theorem 4.5, we introduce a supremum representation for the operator-valued Φ-entropies. The supremum is defined as the least upper bound (under Löwner partial ordering) among the set of operators. In general, the supremum might not exist owing to matrix partial ordering; however, the supremum in (g) exists and is attained when T≡Z.

In the following, we demonstrate a monotone property of operator-valued Φ-entropies when Φ∈(C3).

Proposition 4.7 (Monotonicity of operator-valued Φ-entropies) —

Fix a convex function Φ∈(C3), then the operator-valued Φ-entropy H_Φ(Z) is monotone under any unital completely positive map N, i.e.

$${\mathit{\text{H}}}_{\mathrm{\Phi }}(\mathsf{N}(\mathit{\text{Z}}))⪯{\mathit{\text{H}}}_{\mathrm{\Phi }}(\mathit{\text{Z}})$$

for any random matrix Z taking values in ${\mathbb{M}}^{+}$.

Proof. —

If Φ∈(C3), by item (e) in theorem 4.5, then we have the joint convexity of the map:

$${\mathit{\text{F}}}_t(\mathit{\text{A}},\mathit{\text{B}})\triangleq t\mathrm{\Phi }(\mathit{\text{A}})+(1-t)\mathrm{\Phi }(\mathit{\text{B}})-\mathrm{\Phi }(t\mathit{\text{A}}+(1-t)\mathit{\text{B}})$$

for any 0≤t≤1. Let X=(A,B) denote the pair of matrices.

For any completely positive unital map N, it can be expressed in the following form [36]:

$$\mathsf{N}(\mathit{\text{A}})=\sum _i{\mathit{\text{K}}}_i\mathit{\text{A}}{\mathit{\text{K}}}_i^{†},$$

where $\sum _i{\mathit{\text{K}}}_i{\mathit{\text{K}}}_i^{†}=\mathit{\text{I}}$ (the identity matrix in ${\mathbb{M}}^{\mathrm{sa}}$), and the dagger denotes the complex conjugate. Hence, by the Jensen operator inequality, proposition A.7 yields

$${\mathit{\text{F}}}_t(\mathsf{N}(\mathit{\text{X}}))⪯{\mathit{\text{F}}}_t(\mathit{\text{X}}),\mathrm{\forall }\hspace{0.17em}0\le t\le 1$$

for any completely positive unital map N, which implies the monotonicity of H_Φ(Z). ▪

Following the same argument, the matrix Φ-entropy functional satisfies the monotone property if Φ∈(C2).

Corollary 4.8 (Monotonicity of matrix Φ-entropy functionals) —

Fix a convex function Φ∈(C2), then the matrix Φ-entropy functional H_Φ(Z) is monotone under any unital completely positive map N: H_Φ(N(Z))≤H_Φ(Z) for any random matrix Z taking values in ${\mathbb{M}}^{+}$.

We remark that the monotonicity of a quantum ensemble is only known for when $\mathrm{\Phi }(x)=x\log x$. This is the famous result in quantum information theory, namely the monotone property of the Holevo quantity [37]. Our corollary 4.8 extends the monotonicity of a quantum ensemble to any function Φ∈(C2).

5. Applications: operator Efron–Stein inequality

Here, we employ the operator subadditivity of H_u→u²(Z) to prove the operator Efron–Stein inequality. For 1≤i≤n, let X₁′,…,X_n′ be independent copies of X₁,…,X_n, and denote ${\widetilde{\mathit{\text{X}}}\hspace{0.17em}}^{(i)}\triangleq ({\mathit{\text{X}}}_1,\dots ,{\mathit{\text{X}}}_{i-1},{\mathit{\text{X}}}_i^{\prime },{\mathit{\text{X}}}_{i+1},\dots ,{\mathit{\text{X}}}_n)$, i.e. replacing the ith component of X by the independent copy X_i′.

Define the quantity²

$$\mathcal{E}(\mathcal{L})\triangleq \frac{1}{2}\mathbb{E}\left[\sum _{i=1}^n{(\mathcal{L}(\mathit{\text{X}})-\mathcal{L}({\widetilde{\mathit{\text{X}}}\hspace{0.17em}}^{(i)}))}^2\right],$$

and denote the operator-valued variance of a random matrix A (taking values in ${\mathbb{M}}^{\mathrm{sa}}$) by

$$\mathbf{\text{Var}}(\mathit{\text{A}})\triangleq \mathbb{E}{(\mathit{\text{A}}-\mathbb{E}\mathit{\text{A}})}^2=\mathbb{E}{\mathit{\text{A}}}^2-{(\mathbb{E}\mathit{\text{A}})}^2.$$

Theorem 5.1 (Operator Efron–Stein inequality) —

With the prevailing assumptions, we have

$$\mathbf{\text{Var}}(\mathit{\text{Z}})⪯\mathcal{E}(\mathit{\text{Z}}).$$

Proof. —

Theorem 5.1 is a direct consequence of the subadditivity of operator-valued Φ-entropies, namely theorem 4.4 with Φ(u)=u².

For two independent and identical random matrices A, B, direct calculation yields

$$\begin{array}{rl}\frac{1}{2}\mathbb{E}[{(\mathit{\text{A}}-\mathit{\text{B}})}^2]& =\frac{1}{2}\mathbb{E}[{\mathit{\text{A}}}^2-\mathit{\text{A}}\mathit{\text{B}}-\mathit{\text{B}}\mathit{\text{A}}+{\mathit{\text{B}}}^2]\\ & =\mathbf{\text{Var}}(\mathit{\text{A}}).\end{array}$$

Observe that Z_i′ is an independent copy of Z conditioned on X_−i. Denote ${\mathbf{\text{Var}}}^{(\mathrm{i})}(\mathit{\text{Z}})\triangleq {\mathbb{E}}_{\mathrm{i}}{(\mathit{\text{Z}}-{\mathbb{E}}_{\mathrm{i}}\mathit{\text{Z}})}^2$ for all i=1,…,n. Then

$${\mathbf{\text{Var}}}^{(\mathrm{i})}(\mathit{\text{Z}})=\frac{1}{2}{\mathbb{E}}_{\mathrm{i}}[{(\mathit{\text{Z}}-{\mathit{\text{Z}}}_{\mathrm{i}}^{\prime })}^2].$$

Finally, theorem 4.4 and proposition 4.3 lead to

$$\begin{array}{rl}\mathbf{\text{Var}}(\mathit{\text{Z}})& ={\mathit{\text{H}}}_{u\mapsto u^2}(\mathit{\text{Z}})\\ & ⪯\sum _{i=1}^n\mathbb{E}{\mathit{\text{H}}}_{u\mapsto u^2}^{(i)}(\mathit{\text{Z}})\\ & =\sum _{i=1}^n{\mathbf{\text{Var}}}^{(\mathrm{i})}(\mathit{\text{Z}})\\ & =\mathcal{E}\mathbf{(}\mathit{\text{Z}}\mathbf{)}.\end{array}$$

 ▪

Note that the established operator Efron–Stein inequality leads directly to a matrix polynomial Efron–Stein inequality.

Corollary 5.2 (Matrix polynomial Efron–Stein) —

With the prevailing assumptions, for each natural number p≥1, we have

$$\parallel \mathbb{E}{(\mathit{\text{Z}}-\mathbb{E}\mathit{\text{Z}})}^2{\parallel }_p^p\le {\parallel \frac{1}{2}\sum _{i=1}^n\mathbb{E}[{(\mathit{\text{Z}}-{\mathit{\text{Z}}}_i^{\prime })}^2]\parallel }_p^p.$$

Corollary 5.2 is a variant of the matrix polynomial Efron–Stein inequality derived in [25, theorem 4.2].

6. Proof of theorem 3.3

Proof. —

(a)⇒(i)⇒(g)⇒(j) This statement is proved by Chen & Tropp in [19].

(a)⇔(b) This equivalent statement is proved in [21, theorem 2].

(a)⇔(d) Theorem 2.1 in [20] proved the equivalence of (a) and the following convexity lemma.

Lemma 6.1 (Convexity lemma [19, lemma 4.2]) —

Fix a function Φ∈(C2), and let Ψ=Φ′. Suppose that A is a random matrix taking values in ${\mathbb{M}}_d^{++},$ and let X be a random matrix taking values in ${\mathbb{M}}_d^{\mathrm{sa}}$. Assume that ∥A∥, ∥X∥ are integrable. Then

$$\mathbb{E}⟨\mathit{\text{X}},\mathsf{D}\mathrm{\Psi }[\mathit{\text{A}}](\mathit{\text{X}})⟩\ge ⟨\mathbb{E}[\mathit{\text{X}}],\mathsf{D}\mathrm{\Psi }[\mathbb{E}\mathit{\text{A}}](\mathbb{E}\hspace{0.17em}\mathit{\text{X}})⟩.$$

What remains is to establish equivalence between the convexity lemma and condition (d). This follows easily from lemma A.5,

$$\mathrm{Tr}({\mathsf{D}}^2\mathrm{\Phi }[\mathit{\text{A}}](\mathit{\text{X}},\mathit{\text{X}}))=⟨\mathit{\text{X}},\mathsf{D}{\mathrm{\Phi }}^{\prime }[\mathit{\text{A}}](\mathit{\text{X}})⟩.$$

Remark 6.2 —

In [19, lemma 4.2], it is shown that the concavity of the map,

$$\mathit{\text{A}}\mapsto ⟨\mathit{\text{X}}{(\mathsf{D}\mathrm{\Psi }[\mathit{\text{A}}])}^{-1}(\mathit{\text{X}})⟩,\mathrm{\forall }\hspace{0.17em}\mathit{\text{X}}\in {\mathbb{M}}_d^{sa},$$

implies the joint convexity of the map (i.e. lemma 6.1),

$$(\mathit{\text{X}},\mathit{\text{A}})\mapsto ⟨\mathit{\text{X}}(\mathsf{D}\mathrm{\Psi }[\mathit{\text{A}}])(\mathit{\text{X}})⟩.$$ 6.1

(b)⇔(c)⇔(d) Define $A_{\mathrm{\Phi }},B_{\mathrm{\Phi }},C_{\mathrm{\Phi }}:{\mathbb{M}}_d^{+}\times {\mathbb{M}}_d^{+}\to \mathbb{R}$ as

$$\begin{array}{rl}{A}_{\mathrm{\Phi }}(\mathit{\text{u}},\mathit{\text{v}})& \triangleq \mathrm{Tr}[\mathrm{\Phi }(\mathit{\text{u}}+\mathit{\text{v}})-\mathrm{\Phi }(\mathit{\text{u}})-\mathsf{D}\mathrm{\Phi }[\mathit{\text{u}}](\mathit{\text{v}})],\\ B_{\mathrm{\Phi }}(\mathit{\text{u}},\mathit{\text{v}})& \triangleq \mathrm{Tr}[\mathsf{D}\mathrm{\Phi }[\mathit{\text{u}}+\mathit{\text{v}}](\mathit{\text{v}})-\mathsf{D}\mathrm{\Phi }[\mathit{\text{u}}](\mathit{\text{v}})],\\ C_{\mathrm{\Phi }}(\mathit{\text{u}},\mathit{\text{v}})& \triangleq \mathrm{Tr}[{\mathsf{D}}^2\mathrm{\Phi }[\mathit{\text{u}}](\mathit{\text{v}},\mathit{\text{v}})].\end{array}$$

Following from [18], we can establish the following relations: for any $(\mathit{\text{u}},\mathit{\text{v}})\in {\mathbb{M}}_d^{+}\times {\mathbb{M}}_d^{+}$,

$$A_{\mathrm{\Phi }}(\mathit{\text{u}},\mathit{\text{v}})={\int }_0^1(1-s)C_{\mathrm{\Phi }}(\mathit{\text{u}}+s\mathit{\text{v}},\mathit{\text{v}})\hspace{0.17em}\mathrm{d}s$$ 6.2

and

$$B_{\mathrm{\Phi }}(\mathit{\text{u}},\mathit{\text{v}})={\int }_0^1{C}_{\mathrm{\Phi }}(\mathit{\text{u}}+s\mathit{\text{v}},\mathit{\text{v}})\hspace{0.17em}\mathrm{d}s,$$ 6.3

and, for small enough ϵ>0,

$$A_{\mathrm{\Phi }}(\mathit{\text{u}},ϵ\mathit{\text{v}})=\frac{1}{2}{C}_{\mathrm{\Phi }}(\mathit{\text{u}},\mathit{\text{v}}){ϵ}^2+o({ϵ}^2)$$ 6.4

and

$$B_{\mathrm{\Phi }}(\mathit{\text{u}},ϵ\mathit{\text{v}})=C_{\mathrm{\Phi }}(\mathit{\text{u}},\mathit{\text{v}}){ϵ}^2+o({ϵ}^2).$$ 6.5

Equation (6.2) is exactly the integral representation for the matrix Brégman divergence proved in [21]. Similarly, equation (6.3) follows from

$$\begin{array}{rl}{B}_{\mathrm{\Phi }}(\mathit{\text{u}},\mathit{\text{v}})& ={\frac{\mathrm{d}}{\mathrm{d}s}\mathrm{Tr}[\mathrm{\Phi }(\mathit{\text{u}}+s\mathit{\text{v}})]|}_{s=1}-{\frac{\mathrm{d}}{\mathrm{d}s}\mathrm{Tr}[\mathrm{\Phi }(\mathit{\text{u}}+s\mathit{\text{v}})]|}_{s=0}\\ & ={\int }_0^1\frac{\mathrm{d}}{\mathrm{d}s}(\frac{\mathrm{d}}{\mathrm{d}s}\mathrm{Tr}[\mathrm{\Phi }(\mathit{\text{u}}+s\mathit{\text{v}})])\mathrm{d}s\\ & ={\int }_0^1{C}_{\mathrm{\Phi }}(\mathit{\text{u}}+s\mathit{\text{v}},\mathit{\text{v}})\mathrm{d}s.\end{array}$$

Equations (6.4) and (6.5) can be obtained by Taylor expansion at (u,0). That is,

$$\begin{array}{rl}{A}_{\mathrm{\Phi }}(\mathit{\text{u}},ϵ\mathit{\text{v}})& =A_{\mathrm{\Phi }}(\mathit{\text{u}},\mathbf{\text{0}})+{\mathsf{D}}_{\mathit{\text{u}}}{A}_{\mathrm{\Phi }}[\mathit{\text{u}},\mathbf{\text{0}}](\mathbf{\text{0}})+{\mathsf{D}}_{\mathit{\text{v}}}{A}_{\mathrm{\Phi }}[\mathit{\text{u}},\mathbf{\text{0}}](ϵ\mathit{\text{v}})\\ & +\frac{1}{2}({\mathsf{D}}_{\mathit{\text{u}}}^2{A}_{\mathrm{\Phi }}[\mathit{\text{u}},\mathbf{\text{0}}](\mathbf{\text{0}},\mathbf{\text{0}})+2{\mathsf{D}}_{\mathit{\text{u}}}{\mathsf{D}}_{\mathit{\text{v}}}{A}_{\mathrm{\Phi }}[\mathit{\text{u}},\mathbf{\text{0}}](\mathbf{\text{0}},ϵ\mathit{\text{v}})+{\mathsf{D}}_{\mathit{\text{v}}}^2{A}_{\mathrm{\Phi }}[\mathit{\text{u}},\mathbf{\text{0}}](ϵ\mathit{\text{v}},ϵ\mathit{\text{v}}))+o({ϵ}^2)\\ & =\mathrm{Tr}[\mathsf{D}\mathrm{\Phi }[\mathit{\text{u}}+\mathbf{\text{0}}](ϵ\mathit{\text{v}})-\mathsf{D}\mathrm{\Phi }[\mathit{\text{u}}](\mathsf{D}[\mathit{\text{v}}](ϵ\mathit{\text{v}}))+\frac{1}{2}{\mathsf{D}}^2\mathrm{\Phi }[\mathit{\text{u}}+\mathbf{\text{0}}](ϵ\mathit{\text{v}},ϵ\mathit{\text{v}})]+o({ϵ}^2)\\ & =\frac{1}{2}{C}_{\mathrm{\Phi }}(\mathit{\text{u}},\mathit{\text{v}}){ϵ}^2+o({ϵ}^2).\end{array}$$

Following the same argument,

$$\begin{array}{rl}& B_{\mathrm{\Phi }}(\mathit{\text{u}},ϵ\mathit{\text{v}})\\ & =B_{\mathrm{\Phi }}(\mathit{\text{u}},\mathbf{\text{0}})+{\mathsf{D}}^2\mathrm{\Phi }[\mathit{\text{u}}+\mathbf{\text{0}}](\mathbf{\text{0}},ϵ\mathit{\text{v}})+\mathsf{D}\mathrm{\Phi }[\mathit{\text{u}}+\mathbf{\text{0}}](\mathsf{D}[\mathit{\text{v}}](ϵ\mathit{\text{v}}))-\mathsf{D}\mathrm{\Phi }[\mathit{\text{u}}](\mathsf{D}[\mathit{\text{v}}](ϵ\mathit{\text{v}}))\\ & +\frac{1}{2}({\mathsf{D}}^3\mathrm{\Phi }[\mathit{\text{u}}+\mathbf{\text{0}}](\mathbf{\text{0}},ϵ\mathit{\text{v}},ϵ\mathit{\text{v}})+2{\mathsf{D}}^2\mathrm{\Phi }[\mathit{\text{u}}+\mathbf{\text{0}}](ϵ\mathit{\text{v}},ϵ\mathit{\text{v}}))+o({ϵ}^2)\\ & =C_{\mathrm{\Phi }}(\mathit{\text{u}},\mathit{\text{v}}){ϵ}^2+o({ϵ}^2).\end{array}$$

We can observe from equations (6.2) and (6.3) that the joint convexity of (u,v)↦A_Φ(u,v) and (u,v)↦B_Φ(u,v) follows from that of (u,v)↦C_Φ(u,v). In other words, we proved that conditions (d)⇒(b) and (d)⇒(c).

Conversely, equations (6.4) and (6.5) show that (b)⇒(d) and condition (c)⇒(d). To be more specific, the joint convexity of (u,v)↦A_Φ(u,ϵv) implies

$$t{A}_{\mathrm{\Phi }}({\mathit{\text{u}}}_1,ϵ{\mathit{\text{v}}}_1)+(1-t)A_{\mathrm{\Phi }}({\mathit{\text{u}}}_2,ϵ{\mathit{\text{v}}}_2)\ge A_{\mathrm{\Phi }}(\mathit{\text{u}},ϵ\mathit{\text{v}}),$$ 6.6

for each ${\mathit{\text{u}}}_1,{\mathit{\text{u}}}_2,{\mathit{\text{v}}}_1,{\mathit{\text{v}}}_2\in {\mathbb{M}}_d^{+}$, t∈[0,1], ϵ>0, and u≡tu₁+(1−t)u₂, v≡tv₁+(1−t)v₂. Invoking equation (6.4) gives

$$t{A}_{\mathrm{\Phi }}({\mathit{\text{u}}}_1,ϵ{\mathit{\text{v}}}_1)+(1-t)A_{\mathrm{\Phi }}({\mathit{\text{u}}}_2,ϵ{\mathit{\text{v}}}_2)=\frac{t{C}_{\mathrm{\Phi }}({\mathit{\text{u}}}_1,{\mathit{\text{v}}}_1)+(1-t)C_{\mathrm{\Phi }}({\mathit{\text{u}}}_2,{\mathit{\text{v}}}_2)}{2}{ϵ}^2+o({ϵ}^2)$$

and

$$A_{\mathrm{\Phi }}(\mathit{\text{u}},ϵ\mathit{\text{v}})=\frac{1}{2}{C}_{\mathrm{\Phi }}(\mathit{\text{u}},ϵ\mathit{\text{v}}){ϵ}^2+o({ϵ}^2).$$

Hence, equation (6.6) is equivalent to

$$t{C}_{\mathrm{\Phi }}({\mathit{\text{u}}}_1,{\mathit{\text{v}}}_1){ϵ}^2+(1-t)C_{\mathrm{\Phi }}({\mathit{\text{u}}}_2,{\mathit{\text{v}}}_2){ϵ}^2+o({ϵ}^2)\ge C_{\mathrm{\Phi }}(\mathit{\text{u}},ϵ\mathit{\text{v}}){ϵ}^2+o({ϵ}^2).$$

The joint convexity of (u,v)↦C_Φ(u,ϵv) follows by dividing by ϵ² on both sides and letting $ϵ\to 0$. The joint convexity of (u,v)↦B_Φ(u,ϵv) can be obtained in a similar way using equation (6.5).

(a)⇔(e) It is trivial if Φ is affine; hence, we assume Φ′′>0. We start from the convexity of the map,

$$\mathit{\text{A}}\mapsto -\mathrm{Tr}[\mathit{\text{h}}{(\mathsf{D}\mathrm{\Psi }[\mathit{\text{A}}])}^{-1}(\mathit{\text{h}})],\text{for all }\mathit{\text{h}}\in {\mathbb{M}}_d^{\mathrm{sa}}.$$ 6.7

To ease the burden of notation, we denote ${\mathsf{T}}_{\mathit{\text{A}}}\triangleq \mathsf{D}\mathrm{\Psi }[\mathit{\text{A}}]\simeq {\mathbb{C}}^{d^2\times d^2}$ and $\widehat{\mathit{\text{h}}}\triangleq \mathit{\text{h}}\simeq {\mathbb{C}}^{d^2\times 1}$ by the isometric isomorphism between super-operators and matrices. Then, equation (6.7) can be rewritten as

$$\mathit{\text{A}}\mapsto -{\widehat{\mathit{\text{h}}}}^{†}\cdot {\mathsf{T}}_{\mathit{\text{A}}}^{-1}\cdot \widehat{\mathit{\text{h}}},\text{for all }\widehat{\mathit{\text{h}}}\in {\mathbb{C}}^{d^2\times 1},$$

which is equivalent to the non-negativity of the second derivative (see proposition A.2),

$$\begin{array}{rl}-{\mathsf{D}}_{\mathit{\text{A}}}^2[{\widehat{\mathit{\text{h}}}}^{†}\cdot {\mathsf{T}}_{\mathit{\text{A}}}^{-1}\cdot \widehat{\mathit{\text{h}}}](\mathit{\text{k}},\mathit{\text{k}})& =-{\widehat{\mathit{\text{h}}}}^{†}\cdot {\mathsf{D}}_{\mathit{\text{A}}}^2[{\mathsf{T}}_{\mathit{\text{A}}}^{-1}](\mathit{\text{k}},\mathit{\text{k}})\cdot \widehat{\mathit{\text{h}}}\\ & \ge 0,\text{for all }\mathit{\text{A}}⪰\mathbf{\text{0}},\widehat{\mathit{\text{h}}}\in {\mathbb{C}}^{d^2\times 1},\hspace{0.17em}\mathit{\text{k}}\in {\mathbb{M}}_d^{\mathrm{sa}}.\end{array}$$

Now, recall the chain rule of the Fréchet derivative in proposition A.1,

$$\begin{array}{rl}\mathsf{D}\mathcal{F}\circ \mathcal{G}[\mathit{\text{A}}](\mathit{\text{u}})& =\mathsf{D}\mathcal{F}[\mathcal{G}(\mathit{\text{A}})](\mathsf{D}\mathcal{G}[\mathit{\text{A}}](\mathit{\text{u}}));\\ {\mathsf{D}}^2\mathcal{F}\circ \mathcal{G}[\mathit{\text{A}}](\mathit{\text{u}},\mathit{\text{v}})& ={\mathsf{D}}^2\mathcal{F}[\mathcal{G}(\mathit{\text{A}})](\mathsf{D}\mathcal{G}[\mathit{\text{A}}](\mathit{\text{v}}),\mathsf{D}\mathcal{G}[\mathit{\text{A}}](\mathit{\text{v}}))\\ & +\mathsf{D}\mathcal{F}[\mathcal{G}(\mathit{\text{A}})]({\mathsf{D}}^2\mathcal{G}[\mathit{\text{A}}](\mathit{\text{u}},\mathit{\text{v}})),\end{array}$$

and the formula of the differentiation of the inverse function (see lemma A.6),

$$\begin{array}{rl}\mathsf{D}\mathcal{G}{[\mathit{\text{A}}]}^{-1}(\mathit{\text{u}})& =-\mathcal{G}{(\mathit{\text{A}})}^{-1}\cdot \mathsf{D}\mathcal{G}[\mathit{\text{A}}](\mathit{\text{u}})\cdot \mathcal{G}{(\mathit{\text{A}})}^{-1};\\ {\mathsf{D}}^2\mathcal{G}{[\mathit{\text{A}}]}^{-1}(\mathit{\text{u}},\mathit{\text{u}})& =2\mathcal{G}{(\mathit{\text{A}})}^{-1}\cdot \mathsf{D}\mathcal{G}[\mathit{\text{A}}](\mathit{\text{u}})\cdot \mathcal{G}{(\mathit{\text{A}})}^{-1}\cdot \mathsf{D}\mathcal{G}[\mathit{\text{A}}](\mathit{\text{u}})\cdot g{(\mathit{\text{A}})}^{-1}\\ & -\mathcal{G}{(\mathit{\text{A}})}^{-1}\cdot {\mathsf{D}}^2\mathcal{G}[\mathit{\text{A}}](\mathit{\text{u}},\mathit{\text{u}})\cdot \mathcal{G}{(\mathit{\text{A}})}^{-1},\end{array}$$

we can compute the following identities by taking $\mathcal{G}[\mathit{\text{A}}]\equiv {\mathsf{T}}_{\mathit{\text{A}}}$ and u≡k:

$${\mathsf{D}}_{\mathit{\text{A}}}[{\mathsf{T}}_{\mathit{\text{A}}}^{-1}](\mathit{\text{k}})=-{\mathsf{T}}_{\mathit{\text{A}}}^{-1}\cdot {\mathsf{D}}_{\mathit{\text{A}}}[{\mathsf{T}}_{\mathit{\text{A}}}](\mathit{\text{k}})\cdot {\mathsf{T}}_{\mathit{\text{A}}}^{-1}$$

and

$$\begin{array}{rl}{\mathsf{D}}_{\mathit{\text{A}}}[{\mathsf{T}}_{\mathit{\text{A}}}^{-1}](\mathit{\text{k}},\mathit{\text{k}})& =2\cdot {\mathsf{T}}_{\mathit{\text{A}}}^{-1}\cdot {\mathsf{D}}_{\mathit{\text{A}}}[{\mathsf{T}}_{\mathit{\text{A}}}](\mathit{\text{k}})\cdot {\mathsf{T}}_{\mathit{\text{A}}}^{-1}\cdot {\mathsf{D}}_{\mathit{\text{A}}}[{\mathsf{T}}_{\mathit{\text{A}}}](\mathit{\text{k}})\cdot {\mathsf{T}}_{\mathit{\text{A}}}^{-1}\\ & -{\mathsf{T}}_{\mathit{\text{A}}}^{-1}\cdot {\mathsf{D}}_{\mathit{\text{A}}}^2[{\mathsf{T}}_{\mathit{\text{A}}}](\mathit{\text{k}},\mathit{\text{k}})\cdot {\mathsf{T}}_{\mathit{\text{A}}}^{-1}.\end{array}$$

Therefore, we reach the expression (3.2), and statement (a) is true if and only if (3.2) holds. Recall that, in the scalar case (i.e. d=1), the Fréchet derivative can be expressed as the product of the differential and the direction [38, theorem 3.11]

$$\mathsf{D}\mathrm{\Psi }[a]h={\mathrm{\Psi }}^{\prime }(a)\cdot h.$$

Hence, equation (3.2) reduces to

$$\begin{array}{rl}& h\cdot ({\mathrm{\Psi }}^{\prime }{(a))}^{-1}\cdot {\mathrm{\Psi }}^{‴}(a)\cdot k^2\cdot {({\mathrm{\Psi }}^{\prime }(a))}^{-1}\cdot h\\ & =\frac{{\mathrm{\Phi }}^{⁗}(a)\cdot {\mathrm{\Phi }}^{″}(a)\cdot k^2{h}^2}{{\mathrm{\Phi }}^{″}{(a)}^2}\\ & \ge 2\cdot h\cdot {({\mathrm{\Psi }}^{\prime }(a))}^{-1}\cdot {\mathrm{\Psi }}^{″}(a)\cdot k\cdot {({\mathrm{\Psi }}^{\prime }(a))}^{-1}\cdot {\mathrm{\Psi }}^{″}(a)\cdot k\cdot {({\mathrm{\Psi }}^{\prime }(a))}^{-1}\cdot h\\ & =\frac{2{\mathrm{\Phi }}^{‴}{(a)}^2\cdot k^2{h}^2}{{\mathrm{\Phi }}^{″}{(a)}^3},\end{array}$$

for all a>0 and $h,k\in \mathbb{R}$. In other words, equation (3.2) can be viewed as a non-commutative generalization of the classical statement: Φ′′′′Φ′′≥2Φ′′′².

(d)⇔(f) For any t∈[0,1], define ${\mathit{\text{F}}}_t:{\mathbb{M}}_d^{+}\times {\mathbb{M}}_d^{+}\to {\mathbb{M}}_d^{\mathrm{sa}}$ as

$${\mathit{\text{F}}}_t(\mathit{\text{X}},\mathit{\text{Y}})\triangleq t\mathrm{\Phi }(\mathit{\text{X}})+(1-t)\mathrm{\Phi }(\mathit{\text{Y}})-\mathrm{\Phi }(t\mathit{\text{X}}+(1-t)\mathit{\text{Y}}).$$

By taking x≡(X,Y) and h≡(h,k) in proposition A.2, the convexity of the twice Fréchet differentiable function F_t is equivalent to

$${\mathsf{D}}^2{\mathit{\text{F}}}_t[\mathit{\text{X}},\mathit{\text{Y}}](\mathit{\text{h}},\mathit{\text{k}})⪰\mathbf{\text{0}}\mathrm{\forall }\hspace{0.17em}\mathit{\text{X}},\mathit{\text{Y}}\in {\mathbb{M}}_d^{+}\text{and}\mathrm{\forall }\hspace{0.17em}\mathit{\text{h}},\mathit{\text{k}}\in {\mathbb{M}}_d^{\mathrm{sa}}.$$

Then, with the help of the partial Fréchet derivative defined in proposition A.3, the second-order Fréchet derivative of F_t(X,Y) can be evaluated as

$$\begin{array}{rl}& {\mathsf{D}}^2{\mathit{\text{F}}}_t[\mathit{\text{X}},\mathit{\text{Y}}](\mathit{\text{h}},\mathit{\text{k}})\\ & ={\mathsf{D}}_{\mathit{\text{X}}}^2{\mathit{\text{F}}}_t[\mathit{\text{X}},\mathit{\text{Y}}](\mathit{\text{h}},\mathit{\text{h}})+{\mathsf{D}}_{\mathit{\text{Y}}}{\mathsf{D}}_{\mathit{\text{X}}}{\mathit{\text{F}}}_t[\mathit{\text{X}},\mathit{\text{Y}}](\mathit{\text{h}},\mathit{\text{k}})\\ & +{\mathsf{D}}_{\mathit{\text{X}}}{\mathsf{D}}_{\mathit{\text{Y}}}{\mathit{\text{F}}}_t[\mathit{\text{X}},\mathit{\text{Y}}](\mathit{\text{k}},\mathit{\text{h}})+{\mathsf{D}}_{\mathit{\text{Y}}}^2{\mathit{\text{F}}}_t[\mathit{\text{X}},\mathit{\text{Y}}](\mathit{\text{k}},\mathit{\text{k}})\\ & =t\cdot {\mathsf{D}}^2\mathrm{\Phi }[\mathit{\text{X}}](\mathit{\text{h}},\mathit{\text{h}})-t^2\cdot {\mathsf{D}}^2\mathrm{\Phi }[t\mathit{\text{X}}+(1-t)\mathit{\text{Y}}](\mathit{\text{h}},\mathit{\text{h}})\\ & -t(1-t)\cdot {\mathsf{D}}^2\mathrm{\Phi }[t\mathit{\text{X}}+(1-t)\mathit{\text{Y}}](\mathit{\text{h}},\mathit{\text{k}})-t(1-t)\cdot {\mathsf{D}}^2\mathrm{\Phi }[t\mathit{\text{X}}+(1-t)\mathit{\text{Y}}](\mathit{\text{k}},\mathit{\text{h}})\\ & +(1-t)\cdot {\mathsf{D}}^2\mathrm{\Phi }[\mathit{\text{Y}}](\mathit{\text{k}},\mathit{\text{k}})-{(1-t)}^2\cdot {\mathsf{D}}^2\mathrm{\Phi }[t\mathit{\text{X}}+(1-t)\mathit{\text{Y}}](\mathit{\text{k}},\mathit{\text{k}}).\end{array}$$ 6.8

Taking trace on both sides of (6.8) and invoking lemma A.5, we have

$$\begin{array}{rl}& \mathrm{Tr}[{\mathsf{D}}^2{\mathit{\text{F}}}_t[\mathit{\text{X}},\mathit{\text{Y}}](\mathit{\text{h}},\mathit{\text{k}})]\\ & =\mathrm{Tr}[t\cdot \mathit{\text{h}}\mathsf{D}\mathrm{\Psi }[\mathit{\text{X}}](\mathit{\text{h}})-t^2\cdot \mathit{\text{h}}\mathsf{D}\mathrm{\Psi }[t\mathit{\text{X}}+(1-t)\mathit{\text{Y}}](\mathit{\text{h}})]\\ & -\mathrm{Tr}[t(1-t)\cdot \mathit{\text{h}}\mathsf{D}\mathrm{\Psi }[t\mathit{\text{X}}+(1-t)\mathit{\text{Y}}](\mathit{\text{k}})+t(1-t)\cdot \mathit{\text{k}}\mathsf{D}\mathrm{\Psi }[t\mathit{\text{X}}+(1-t)\mathit{\text{Y}}](\mathit{\text{h}})]\\ & +\mathrm{Tr}[(1-t)\cdot \mathit{\text{k}}\mathsf{D}\mathrm{\Psi }[\mathit{\text{Y}}](\mathit{\text{k}})-{(1-t)}^2\cdot \mathit{\text{k}}{\mathsf{D}}^2\mathrm{\Psi }[t\mathit{\text{X}}+(1-t)\mathit{\text{Y}}](\mathit{\text{k}})].\end{array}$$ 6.9

Because both the trace and the second-order Fréchet derivative are bilinear, we have the following result:

$$\begin{array}{rl}& \mathrm{Tr}[t^2\cdot \mathit{\text{h}}\mathsf{D}\mathrm{\Psi }[t\mathit{\text{X}}+(1-t)\mathit{\text{Y}}](\mathit{\text{h}})+t(1-t)\cdot \mathit{\text{k}}\mathsf{D}\mathrm{\Psi }[t\mathit{\text{X}}+(1-t)\mathit{\text{Y}}](\mathit{\text{h}})]\\ & =⟨t\mathit{\text{h}},\mathsf{D}\mathrm{\Psi }[t\mathit{\text{X}}+(1-t)\mathit{\text{Y}}](t\mathit{\text{h}})⟩+⟨(1-t)\mathit{\text{k}},\mathsf{D}\mathrm{\Psi }[t\mathit{\text{X}}+(1-t)\mathit{\text{Y}}](t\mathit{\text{h}})⟩\\ & =⟨t\mathit{\text{h}}+(1-t)\mathit{\text{k}},\mathsf{D}\mathrm{\Psi }[t\mathit{\text{X}}+(1-t)\mathit{\text{Y}}](t\mathit{\text{h}})⟩.\end{array}$$ 6.10

Similarly,

$$\begin{array}{rl}& \mathrm{Tr}[t(1-t)\cdot \mathit{\text{h}}\mathsf{D}\mathrm{\Psi }[t\mathit{\text{X}}+(1-t)\mathit{\text{Y}}](\mathit{\text{k}})+{(1-t)}^2\cdot \mathit{\text{k}}\mathsf{D}\mathrm{\Psi }[t\mathit{\text{X}}+(1-t)\mathit{\text{Y}}](\mathit{\text{k}})]\\ & =⟨t\mathit{\text{h}}+(1-t)\mathit{\text{k}},\mathsf{D}\mathrm{\Psi }[t\mathit{\text{X}}+(1-t)\mathit{\text{Y}}]((1-t)\mathit{\text{k}})⟩.\end{array}$$ 6.11

Combining equations (6.10) and (6.11), equation (6.9) can be expressed as

$$\begin{array}{rl}\mathrm{Tr}[{\mathsf{D}}^2{\mathit{\text{F}}}_t[\mathit{\text{X}},\mathit{\text{Y}}](\mathit{\text{h}},\mathit{\text{k}})]& =t\cdot ⟨\mathit{\text{h}},\mathsf{D}\mathrm{\Psi }[\mathit{\text{X}}](\mathit{\text{h}})⟩+(1-t)\cdot ⟨\mathit{\text{k}},\mathsf{D}\mathrm{\Psi }[\mathit{\text{Y}}](\mathit{\text{k}})⟩\\ & -⟨(t\mathit{\text{h}}+(1-t)\mathit{\text{k}}),\mathsf{D}\mathrm{\Psi }[t\mathit{\text{X}}+(1-t)\mathit{\text{Y}}](t\mathit{\text{h}}+(1-t)\mathit{\text{k}})⟩.\end{array}$$

Then, it is not hard to observe that the non-negativity of Tr[D²F_t[X,Y](h,k)] for every $\mathit{\text{X}},\mathit{\text{Y}}\in {\mathbb{M}}_d^{+}$, $\mathit{\text{h}},\mathit{\text{k}}\in {\mathbb{M}}_d^{\mathrm{sa}}$ and t∈[0,1] is equivalent to the joint convexity of the map

$$(\mathit{\text{X}},\mathit{\text{A}})\mapsto ⟨\mathit{\text{X}},\mathsf{D}\mathrm{\Psi }[\mathit{\text{A}}](\mathit{\text{X}})⟩=\mathrm{Tr}[{\mathsf{D}}^2\mathrm{\Phi }[\mathit{\text{A}}](\mathit{\text{X}},\mathit{\text{X}})].$$

(j)⇒(g) Considering n=2, the subadditivity means that

$$H_{\mathrm{\Phi }}(\mathit{\text{Z}})\le {\mathbb{E}}_1{H}_{\mathrm{\Phi }}^{(2)}(\mathit{\text{Z}})+{\mathbb{E}}_2{H}_{\mathrm{\Phi }}^{(1)}(\mathit{\text{Z}}).$$

Then, we have

$$\begin{array}{rl}{\mathbb{E}}_1{H}_{\mathrm{\Phi }}^{(2)}(\mathit{\text{Z}})& \ge H_{\mathrm{\Phi }}(\mathit{\text{Z}})-{\mathbb{E}}_2{H}_{\mathrm{\Phi }}^{(1)}(\mathit{\text{Z}})\\ & =\mathbb{E}\mathrm{\Phi }(\mathit{\text{Z}})-\mathrm{\Phi }(\mathbb{E}\mathit{\text{Z}})-{\mathbb{E}}_2{\mathbb{E}}_1\mathrm{\Phi }(\mathit{\text{Z}})+{\mathbb{E}}_2\mathrm{\Phi }({\mathbb{E}}_1\mathit{\text{Z}})\\ & ={\mathbb{E}}_2\mathrm{\Phi }({\mathbb{E}}_1\mathit{\text{Z}})-\mathrm{\Phi }({\mathbb{E}}_2{\mathbb{E}}_1\mathit{\text{Z}})\\ & =H_{\mathrm{\Phi }}({\mathbb{E}}_1\mathit{\text{Z}}).\end{array}$$

(f)⇔(h) Let s∈[0,1]. Define a pair of positive semi-definite random matrices (X,Y) taking values (x,y) with probability s and (x′,y′) with probability (1−s). Then the convexity of H_Φ implies that

$$H_{\mathrm{\Phi }}(t\mathit{\text{X}}+(1-t)\mathit{\text{Y}})\le t{H}_{\mathrm{\Phi }}(\mathit{\text{X}})+(1-t)H_{\mathrm{\Phi }}(\mathit{\text{Y}})$$ 6.12

for every t∈[0,1]. Now, define $F_t(\mathit{\text{u}},\mathit{\text{v}}):{\mathbb{M}}_d^{+}\times {\mathbb{M}}_d^{+}\to \mathbb{R}$ as

$$F_t(\mathit{\text{u}},\mathit{\text{v}})\triangleq \mathrm{Tr}[t\mathrm{\Phi }(\mathit{\text{u}})+(1-t)\mathrm{\Phi }(\mathit{\text{v}})-\mathrm{\Phi }(t\mathit{\text{u}}+(1-t)\mathit{\text{v}})].$$

Then, it follows that

$$\begin{array}{rl}& s{F}_t(\mathit{\text{x}},\mathit{\text{y}})+(1-s)F_t({\mathit{\text{x}}}^{\prime },{\mathit{\text{y}}}^{\prime })-F_t(s(\mathit{\text{x}},\mathit{\text{y}})+(1-s){\mathit{\text{x}}}^{\prime },{\mathit{\text{y}}}^{\prime })\\ & =t\hspace{0.17em}\mathbb{E}\mathrm{\Phi }(\mathit{\text{X}})-t\mathrm{\Phi }(\mathbb{E}\mathit{\text{X}})+(1-t)\mathbb{E}\mathrm{\Phi }(\mathit{\text{Y}})-(1-t)\mathrm{\Phi }(\mathbb{E}\mathit{\text{Y}})\\ & -\mathbb{E}\mathrm{\Phi }(t\mathit{\text{X}}+(1-t)\mathit{\text{Y}})+\mathrm{\Phi }(t\mathbb{E}\mathit{\text{X}}+(1-t)\mathbb{E}\mathit{\text{Y}})\\ & =t{H}_{\mathrm{\Phi }}(\mathit{\text{X}})+(1-t)H_{\mathrm{\Phi }}(\mathit{\text{Y}})-H_{\mathrm{\Phi }}(t\mathit{\text{X}}+(1-t)\mathit{\text{Y}}),\end{array}$$

which means that the convexity of the pair (u,v)↦F_t(u,v) is equivalent to the convexity of H_Φ, i.e. equation (6.12).

(g)⇔(h) Define a positive semi-definite random matrix $\mathit{\text{Z}}\triangleq f({\mathit{\text{X}}}_1,{\mathit{\text{X}}}_2)$, which depends on two random variables X₁,X₂ on a Polish space. Denote by Z_X1 the random matrix Z conditioned on X₁. According to the convexity of H_Φ, it follows that

$$\begin{array}{rl}{\mathbb{E}}_1{H}_{\mathrm{\Phi }}(\mathit{\text{Z}}|{\mathit{\text{X}}}_1)& ={\mathbb{E}}_1{H}_{\mathrm{\Phi }}({\mathit{\text{Z}}}_{{\mathit{\text{X}}}_1})\\ & ={\mathbb{E}}_1[\overline{\mathrm{tr}}({\mathbb{E}}_2\mathrm{\Phi }({\mathit{\text{Z}}}_{{\mathit{\text{X}}}_1})-\mathrm{\Phi }({\mathbb{E}}_2{\mathit{\text{Z}}}_{{\mathit{\text{X}}}_1}))]\\ & \ge \overline{\mathrm{tr}}{\mathbb{E}}_2\mathrm{\Phi }({\mathbb{E}}_1{\mathit{\text{Z}}}_{{\mathit{\text{X}}}_1})-\overline{\mathrm{tr}}[\mathrm{\Phi }({\mathbb{E}}_1{\mathbb{E}}_2{\mathit{\text{Z}}}_{{\mathit{\text{X}}}_1})]\\ & =H_{\mathrm{\Phi }}({\mathbb{E}}_1\mathit{\text{Z}}).\end{array}$$

Conversely, define a positive semi-definite random matrix $\mathit{\text{Z}}(s,\mathit{\text{X}},\mathit{\text{Y}})\triangleq s\mathit{\text{X}}+(1-s)\mathit{\text{Y}},$ where s is a random variable. Now, let s be Bernoulli distributed with parameter t∈[0,1]. Then, for all t∈[0,1], the inequality ${\mathbb{E}}_1{H}_{\mathrm{\Phi }}(\mathit{\text{Z}}|s)\ge H_{\mathrm{\Phi }}({\mathbb{E}}_1\mathit{\text{Z}})$ coincides,

$$H_{\mathrm{\Phi }}(t\mathit{\text{X}}+(1-t)\mathit{\text{Y}})\le t{H}_{\mathrm{\Phi }}(\mathit{\text{X}})+(1-t)H_{\mathrm{\Phi }}(\mathit{\text{Y}}).$$

 ▪

7. Proof of theorem 4.4

Our approach to proving operator subadditivity (theorem 4.4) parallels [[19, theorem 2.5], [13, section 3.1]. The strategy is as follows. First, we prove the supremum representation for the operator-valued Φ-entropies in §7a. Second, we establish a conditional operator Jensen inequality in §7b. Finally, we arrive at the proof of theorem 4.4 in §7c.

(a). Representation of operator-valued Φ-entropy

Theorem 7.1 (Supremum representation for operator-valued Φ-entropies) —

Fix a function Φ∈(C3). Assume $\mathit{\text{Z}}\in {\mathbb{M}}_d^{++}$ is a random positive definite matrix for which |Z|, |Φ(Z)∥ are Bochner integrable. Then the operator-valued Φ-entropy can be represented as

$${\mathit{\text{H}}}_{\mathrm{\Phi }}(\mathit{\text{Z}})=\underset{\mathit{\text{T}}\succ \mathbf{\text{0}}}{sup}\mathbb{E}[\mathsf{D}\mathrm{\Phi }[\mathit{\text{T}}](\mathit{\text{Z}}-\mathit{\text{T}})-\mathsf{D}\mathrm{\Phi }[\mathbb{E}\mathit{\text{T}}](\mathit{\text{Z}}-\mathit{\text{T}})+\mathrm{\Phi }(\mathit{\text{T}})-\mathrm{\Phi }(\mathbb{E}\mathit{\text{T}})].$$ 7.1

The range of the supremum contains each random positive definite matrix T for which |T| and |Φ(T)| are Bochner integrable. In particular, the normalized matrix Φ-entropy can be written in the dual form

$${\mathit{\text{H}}}_{\mathrm{\Phi }}(\mathit{\text{Z}})=\underset{\mathit{\text{T}}\succ \mathbf{\text{0}}}{sup}\mathbb{E}[{\mathbf{{\rm Y}}}_1(\mathit{\text{T}},\mathit{\text{Z}})+{\mathbf{{\rm Y}}}_2(\mathit{\text{T}})],$$ 7.2

where ${\mathbf{{\rm Y}}}_1(\mathit{\text{T}},\mathit{\text{Z}})=\mathsf{D}\mathrm{\Phi }[\mathit{\text{T}}](\mathit{\text{Z}})-\mathsf{D}\mathrm{\Phi }[\mathbb{E}\hspace{0.17em}\mathit{\text{T}}](\mathit{\text{Z}})$ is a linear map of Z and ${\mathbf{{\rm Y}}}_2(\mathit{\text{T}})=-\mathsf{D}\mathrm{\Phi }[\mathit{\text{T}}](\mathit{\text{T}})+\mathsf{D}\mathrm{\Phi }[\mathbb{E}\mathit{\text{T}}](\mathit{\text{T}})+(\mathrm{\Phi }(\mathit{\text{T}})-\mathrm{\Phi }(\mathbb{E}\hspace{0.17em}\mathit{\text{T}}))$.

Proof. —

Observe that, when T=Z, the right-hand side of equation (7.1) equals H_Φ(Z). Then, it remains to confirm the inequality

$${\mathit{\text{H}}}_{\mathrm{\Phi }}(\mathit{\text{Z}})⪰\mathbb{E}[\mathsf{D}\mathrm{\Phi }[\mathit{\text{T}}](\mathit{\text{Z}}-\mathit{\text{T}})-\mathsf{D}\mathrm{\Phi }[\mathbb{E}\mathit{\text{T}}](\mathit{\text{Z}}-\mathit{\text{T}})+\mathrm{\Phi }(\mathit{\text{T}})-\mathrm{\Phi }(\mathbb{E}\mathit{\text{T}})]$$ 7.3

for each random positive definite matrix T that satisfies the integrability conditions. We follow the interpolation argument as in [19, lemma 4.1]. For s∈[0,1], define the matrix-valued function

$$\mathit{\text{F}}(s)=\mathbb{E}[\mathsf{D}\mathrm{\Phi }[{\mathit{\text{T}}}_s](\mathit{\text{Z}}-{\mathit{\text{T}}}_s)-\mathsf{D}\mathrm{\Phi }[\mathbb{E}\hspace{0.17em}{\mathit{\text{T}}}_s](\mathit{\text{Z}}-{\mathit{\text{T}}}_s)]+{\mathit{\text{H}}}_{\mathrm{\Phi }}({\mathit{\text{T}}}_s),$$

where

$${\mathit{\text{T}}}_s\triangleq (1-s)\cdot \mathit{\text{Z}}+s\cdot \mathit{\text{T}}\text{for }s\in [0,1].$$

Note that F(0)=H_Φ(Z), and F(1) matches the right-hand side of equation (7.3). As a result, it suffices to show that F′(s)≤0 for s∈[0,1] in order to verify equation (7.3). By the replacement Z−T_s=−s⋅(T−Z), the function F(s) can be rephrased as

$$\mathit{\text{F}}(s)=-s\cdot \mathbb{E}[\mathsf{D}\mathrm{\Phi }[{\mathit{\text{T}}}_s](\mathit{\text{T}}-\mathit{\text{Z}})-\mathsf{D}\mathrm{\Phi }[\mathbb{E}{\mathit{\text{T}}}_s](\mathit{\text{T}}-\mathit{\text{Z}})]+\mathbb{E}[\mathrm{\Phi }({\mathit{\text{T}}}_s)-\mathrm{\Phi }(\mathbb{E}\hspace{0.17em}{\mathit{\text{T}}}_s)].$$

Differentiate the above function to arrive at

$$\begin{array}{rl}{\mathit{\text{F}}}^{\prime }(s)& =-s\mathbb{E}[{\mathsf{D}}^2\mathrm{\Phi }[{\mathit{\text{T}}}_s](\mathit{\text{T}}-\mathit{\text{Z}},\mathit{\text{T}}-\mathit{\text{Z}})]+s\mathbb{E}[{\mathsf{D}}^2\mathrm{\Phi }[\mathbb{E}{\mathit{\text{T}}}_s](\mathit{\text{T}}-\mathit{\text{Z}},\mathbb{E}(\mathit{\text{T}}-\mathit{\text{Z}}))]\\ & -\mathbb{E}[\mathsf{D}\mathrm{\Phi }[{\mathit{\text{T}}}_s](\mathit{\text{T}}-\mathit{\text{Z}})-\mathsf{D}\mathrm{\Phi }[\mathbb{E}{\mathit{\text{T}}}_s](\mathit{\text{T}}-\mathit{\text{Z}})]+\mathbb{E}[\mathsf{D}\mathrm{\Phi }[{\mathit{\text{T}}}_s](\mathit{\text{T}}-\mathit{\text{Z}})-\mathsf{D}\mathrm{\Phi }[\mathbb{E}{\mathit{\text{T}}}_s](\mathit{\text{T}}-\mathit{\text{Z}})]\end{array}$$ 7.4a

$$\begin{array}{rl}& =-s\mathbb{E}[{\mathsf{D}}^2\mathrm{\Phi }[T_s](\mathit{\text{T}}-\mathit{\text{Z}},\mathit{\text{T}}-\mathit{\text{Z}})+s{\mathsf{D}}^2\mathrm{\Phi }[\mathbb{E}{\mathit{\text{T}}}_s](\mathbb{E}(\mathit{\text{T}}-\mathit{\text{Z}}),\mathbb{E}(\mathit{\text{T}}-\mathit{\text{Z}}))],\end{array}$$ 7.4b

where we cancel the last two terms in equation (7.4a) and the second equation (7.4b) follows from the bilinearity of the second-order Fréchet differentiation.

Invoke the joint convexity condition of the function D²Φ[T_s](T−Z,T−Z) (see equation (4.1)), we establish the above derivative to be negative semi-definite, i.e. ${\mathit{\text{F}}}^{\prime }(s)⪯\mathbf{\text{0}}$ for s∈[0,1], and thus complete the proof. ▪

(b). A conditional operator Jensen inequality

Lemma 7.2 (Conditional operator Jensen inequality for operator-valued Φ-entropy) —

Suppose that (X₁,X₂) is a pair of independent random matrices taking values in a Polish space, and let Z=Z(X₁,X₂) be a positive definite random matrix for which |Z| and |Φ(Z)| are Bochner integrable. Then

$${\mathit{\text{H}}}_{\mathrm{\Phi }}({\mathbb{E}}_1\mathit{\text{Z}})⪯\mathbb{E}{\mathit{\text{H}}}_{\mathrm{\Phi }}(\mathit{\text{Z}}|{\mathit{\text{X}}}_1),$$

where ${\mathbb{E}}_1$ is the expectation with respect to the first matrix X₁.

Proof. —

Let ${\mathbb{E}}_2$ refer to the expectation with respect to the second matrix X₂. In the following, we use T(X₂) to emphasize that the matrix T depends only on the randomness in X₂. Recall the supremum representation, equation (7.2); we have

$$\begin{array}{rl}{\mathit{\text{H}}}_{\mathrm{\Phi }}({\mathbb{E}}_1\mathit{\text{Z}})& =\underset{\mathit{\text{T}}}{sup}{\mathbb{E}}_2[{\mathbf{{\rm Y}}}_1(\mathit{\text{T}}({\mathit{\text{X}}}_2),{\mathbb{E}}_1\mathit{\text{Z}})+{\mathbf{{\rm Y}}}_2(\mathit{\text{T}}({\mathit{\text{X}}}_2))]\\ & =\underset{\mathit{\text{T}}}{sup}{\mathbb{E}}_1{\mathbb{E}}_2[{\mathbf{{\rm Y}}}_1(\mathit{\text{T}}({\mathit{\text{X}}}_2),\mathit{\text{Z}})+{\mathbf{{\rm Y}}}_2(\mathit{\text{T}}({\mathit{\text{X}}}_2))]\\ & ⪯{\mathbb{E}}_1\underset{\mathit{\text{T}}}{sup}{\mathbb{E}}_2[{\mathbf{{\rm Y}}}_1(\mathit{\text{T}}({\mathit{\text{X}}}_2),\mathit{\text{Z}})+{\mathbf{{\rm Y}}}_2(\mathit{\text{T}}({\mathit{\text{X}}}_2))]\\ & ={\mathbb{E}}_1\underset{\mathit{\text{T}}}{sup}\mathbb{E}[{\mathbf{{\rm Y}}}_1(\mathit{\text{T}}({\mathit{\text{X}}}_2),\mathit{\text{Z}})+{\mathbf{{\rm Y}}}_2(\mathit{\text{T}}({\mathit{\text{X}}}_2))|{\mathit{\text{X}}}_1]\\ & ={\mathbb{E}}_1{\mathit{\text{H}}}_{\mathrm{\Phi }}(\mathit{\text{Z}}|{\mathit{\text{X}}}_1).\end{array}$$

The second relation follows from the Fubini theorem to interchange the order of ${\mathbb{E}}_1$ and ${\mathbb{E}}_2$. In the third line, we use the convexity of the supremum. (Note that it is not always true under partial ordering. However, it holds in our case, because the supremum is attained when $\mathit{\text{T}}\equiv {\mathbb{E}}_1\mathit{\text{Z}}$ in the second line.) The last identity is exactly the supremum representation equation (7.2) in the conditional form. ▪

It is worth emphasizing that the conditional Jensen inequality can also be achieved by item (d) in theorem 4.5 (cf. (f)⇔(g)⇔(h) in theorem 3.3).

(c). Subadditivity of operator-valued Φ-entropies

Now we are in a position to prove the subadditivity of the operator-valued Φ-entropies.

Proof. —

By adding and subtracting the term $\mathrm{\Phi }({\mathbb{E}}_1\mathit{\text{Z}})$, the operator-valued Φ-entropy can be expressed as

$$\begin{array}{rl}{\mathit{\text{H}}}_{\mathrm{\Phi }}(\mathit{\text{Z}})& =\mathbb{E}[\mathrm{\Phi }(\mathit{\text{Z}})-\mathrm{\Phi }({\mathbb{E}}_1\mathit{\text{Z}})+\mathrm{\Phi }({\mathbb{E}}_1\mathit{\text{Z}})-\mathrm{\Phi }(\mathbb{E}\mathit{\text{Z}})]\\ & =\mathbb{E}[{\mathbb{E}}_1\mathrm{\Phi }(\mathit{\text{Z}})-\mathrm{\Phi }({\mathbb{E}}_1\mathit{\text{Z}})]+[\mathbb{E}\mathrm{\Phi }({\mathbb{E}}_1\mathit{\text{Z}})-\mathrm{\Phi }(\mathbb{E}\hspace{0.17em}{\mathbb{E}}_1\mathit{\text{Z}})]\\ & =\mathbb{E}{\mathit{\text{H}}}_{\mathrm{\Phi }}(\mathit{\text{Z}}|{\mathit{\text{X}}}_{-1})+{\mathit{\text{H}}}_{\mathrm{\Phi }}({\mathbb{E}}_1\mathit{\text{Z}})\\ & ⪯\mathbb{E}{\mathit{\text{H}}}_{\mathrm{\Phi }}(\mathit{\text{Z}}|{\mathit{\text{X}}}_{-1})+{\mathbb{E}}_1{\mathit{\text{H}}}_{\mathrm{\Phi }}(\mathit{\text{Z}}|{\mathit{\text{X}}}_1),\end{array}$$ 7.5

where the last inequality results from lemma 7.2, because X₁ is independent from X₋₁.

Following the same reasoning, we obtain the operator-valued Φ-entropy conditioned on X₁,

$${\mathit{\text{H}}}_{\mathrm{\Phi }}(\mathit{\text{Z}}|{\mathit{\text{X}}}_1)⪯\mathbb{E}[{\mathit{\text{H}}}_{\mathrm{\Phi }}(\mathit{\text{Z}}|{\mathit{\text{X}}}_{-2})|{\mathit{\text{X}}}_1]+{\mathbb{E}}_2{\mathit{\text{H}}}_{\mathrm{\Phi }}(\mathit{\text{Z}}|{\mathit{\text{X}}}_1,{\mathit{\text{X}}}_2).$$

By plugging the expression into equation (7.5), we get

$${\mathit{\text{H}}}_{\mathrm{\Phi }}(\mathit{\text{Z}})⪯\sum _{i=1}^2\mathbb{E}{\mathit{\text{H}}}_{\mathrm{\Phi }}(\mathit{\text{Z}}|{\mathit{\text{X}}}_{-i})+{\mathbb{E}}_1{\mathbb{E}}_2{\mathit{\text{H}}}_{\mathrm{\Phi }}(\mathit{\text{Z}}|{\mathit{\text{X}}}_1,{\mathit{\text{X}}}_2).$$

Finally, by repeating this procedure, we achieve the subadditivity of the operator-valued Φ-entropy

$${\mathit{\text{H}}}_{\mathrm{\Phi }}(\mathit{\text{Z}})⪯\sum _{i=1}^n\mathbb{E}[{\mathit{\text{H}}}_{\mathrm{\Phi }}(\mathit{\text{Z}}|{\mathit{\text{X}}}_{-i})],$$

which completes our claim. ▪

8. Conclusion

In this paper, we extend the results of Chen & Tropp [19], Pitrik & Virosztek [21] and Hansen & Zhang [20] to complete the characterizations of the matrix Φ-entropy functionals. Moreover, we generalize the matrix Φ-entropy functionals to the operator-valued Φ-entropies, and show that this generalization preserves the subadditivity property. Additionally, we prove that the set of operator-valued Φ-entropies is not empty and contains at least the square function. Equivalent characterizations of the operator-valued Φ-entropies are also derived. This result demonstrates that the subadditivity of H_Φ(Z) is equivalent to the operator convexity of H_Φ(Z) on the convex cone of positive semi-definite operators. Finally, we exploit the subadditivity to prove the operator Efron–Stein inequality. It is promising that the proposed result can also derive the matrix exponential Efron–Stein (cf. [25, theorem 4.3]) and the moment inequalities for random matrices; see [[13], [15 ch. 15].

The subadditivity of matrix Φ-entropies leads to a series of important inequalities: matrix Poincaré inequalities with respect to binomial and Gaussian distributions, and the related matrix logarithmic Sobolev inequalities [22]. In [23], the subadditivity and the operator Efron–Stein inequality can be exploited to estimate the mixing time of a quantum random graph. It enables us to better understand the dynamics and long-term behaviours of a quantum system undergoing Markovian processes. We believe the proposed results will lead to more matrix functional inequalities, and have a substantial impact on operator algebra and quantum information science.

Finally, we remark that the results of operator-valued Φ-entropies and the operator Efron–Stein inequalities hold in the infinite-dimensional setting. This is not hard to verify, because the tools (such as Fréchet derivatives) employed in the proofs hold in the infinite dimension.

Appendix A. Miscellaneous lemmas

Proposition A.1 ([Properties of Fréchet derivatives [38, theorem 3.4]) —

Let $\mathcal{U},\mathcal{V}$ and $\mathcal{W}$ be real Banach spaces. Let ${\mathcal{L}}_1:\mathcal{U}\to \mathcal{V}$ and ${\mathcal{L}}_2:\mathcal{V}\to \mathcal{W}$ be Fréchet differentiable at $\mathit{\text{A}}\in \mathcal{U}$ and ${\mathcal{L}}_1(\mathit{\text{A}}),$ respectively, and let $\mathcal{L}={\mathcal{L}}_2\circ {\mathcal{L}}_1$ (i.e. $\mathcal{L}(\mathit{\text{A}})={\mathcal{L}}_2({\mathcal{L}}_1(\mathit{\text{A}}))$. Then $\mathcal{L}$ is Fréchet differentiable at A and $\mathsf{D}\mathcal{L}[\mathit{\text{A}}](\mathit{\text{E}})=\mathsf{D}{\mathcal{L}}_2[{\mathcal{L}}_1(\mathit{\text{A}})](\mathsf{D}{\mathcal{L}}_1[\mathit{\text{A}}](\mathit{\text{E}}))$.

Proposition A.2 (Convexity of twice Fréchet differentiable matrix functions [39, proposition 2.2]) —

Let U be an open convex subset of a real Banach space $\mathcal{U},$ and $\mathcal{W}$ is also a real Banach space. Then a twice Fréchet differentiable function $\mathcal{L}:U\to \mathcal{W}$ is convex if and only if ${\mathsf{D}}^2\mathcal{L}(\mathit{\text{X}})(\mathit{\text{h}},\mathit{\text{h}})⪰\mathbf{\text{0}}$ for each X∈U and $\mathit{\text{h}}\in \mathcal{U}$.

Proposition A.3 (Partial Fréchet derivative [40, proposition 5.3.15]) —

If $\mathcal{L}:\mathcal{U}\times \mathcal{V}\to \mathcal{W}$ is Fréchet differentiable at $(\mathit{\text{X}},\mathit{\text{Y}})\in \mathcal{U}\times \mathcal{V},$ then the partial Fréchet derivatives ${\mathsf{D}}_{\mathit{\text{X}}}\mathcal{L}[\mathit{\text{X}},\mathit{\text{Y}}]$ and ${\mathsf{D}}_{\mathit{\text{Y}}}\mathcal{L}[\mathit{\text{X}},\mathit{\text{Y}}]$ exist, and

$$\mathsf{D}\mathcal{L}[\mathit{\text{X}},\mathit{\text{Y}}](\mathit{\text{h}},\mathit{\text{k}})={\mathsf{D}}_{\mathit{\text{X}}}\mathcal{L}[\mathit{\text{X}},\mathit{\text{Y}}](\mathit{\text{h}})+{\mathsf{D}}_{\mathit{\text{Y}}}\mathcal{L}[\mathit{\text{X}},\mathit{\text{Y}}](\mathit{\text{k}}).$$

Proposition A.4 [41, theorem 2.2] —

Let $\mathit{\text{A}},\mathit{\text{X}}\in {\mathbb{M}}^{\mathrm{sa}}$ and $t\in \mathbb{R}$. Assume $f:I\to \mathbb{R}$ is a continuously differentiable function defined on interval I and assume that the eigenvalues of A+tX⊂I. Then

$${\frac{\mathrm{d}}{\mathrm{d}t}\mathrm{Tr}f(\mathit{\text{A}}+t\mathit{\text{X}})|}_{t=t_0}=\mathrm{Tr}[\mathit{\text{X}}{f}^{\prime }(\mathit{\text{A}}+t_0\mathit{\text{X}})].$$

Proposition A.4 directly leads to the following lemma.

Lemma A.5 —

Let $\mathit{\text{A}},\mathit{\text{X}},\mathit{\text{Y}}\in {\mathbb{M}}^{\mathrm{sa}}$ and $t\in \mathbb{R}$. Assume $f:I\to \mathbb{R}$ is a continuously differentiable function defined on interval I, and assume that the eigenvalues of A+tX⊂I. Then

$$\mathrm{Tr}({\mathsf{D}}^{2}f[\mathit{\text{A}}](\mathit{\text{X}},\mathit{\text{Y}}))=⟨\mathit{\text{X}},\mathsf{D}{f}^{\prime }[\mathit{\text{A}}](\mathit{\text{Y}})⟩=⟨\mathit{\text{Y}},\mathsf{D}{f}^{\prime }[\mathit{\text{A}}](\mathit{\text{X}})⟩.$$

Lemma A.6 (Second-order Fréchet derivative of inversion function) —

Let $\mathcal{G}:\mathbb{M}\to \mathbb{M}$ be second-order Fréchet differentiable at $\mathit{\text{A}}\in \mathbb{M},$ and $\mathcal{G}(\mathit{\text{A}})$ be invertible. Then, for each $\mathit{\text{h}},\mathit{\text{k}}\in \mathbb{M},$ we have

$$\begin{array}{rl}\mathsf{D}\mathcal{G}{[\mathit{\text{A}}]}^{-1}(\mathit{\text{h}})& =-\mathcal{G}{(\mathit{\text{A}})}^{-1}\cdot \mathsf{D}\mathcal{G}[\mathit{\text{A}}](\mathit{\text{h}})\cdot \mathcal{G}{(\mathit{\text{A}})}^{-1};\\ {\mathsf{D}}^2\mathcal{G}{[\mathit{\text{A}}]}^{-1}(\mathit{\text{h}},\mathit{\text{k}})& =2\cdot \mathcal{G}{(\mathit{\text{A}})}^{-1}\cdot \mathsf{D}\mathcal{G}[\mathit{\text{A}}](\mathit{\text{k}})\cdot \mathcal{G}{(\mathit{\text{A}})}^{-1}\cdot \mathsf{D}\mathcal{G}[\mathit{\text{A}}](\mathit{\text{k}})\cdot \mathcal{G}{(\mathit{\text{A}})}^{-1}\\ & -\mathcal{G}{(\mathit{\text{A}})}^{-1}\cdot {\mathsf{D}}^2\mathcal{G}[\mathit{\text{A}}](\mathit{\text{h}},\mathit{\text{k}})\cdot \mathcal{G}{(\mathit{\text{A}})}^{-1}.\end{array}$$

Proof. —

Denote $\mathcal{F}:\mathit{\text{A}}\mapsto {\mathit{\text{A}}}^{-1}$ as the inversion function. Recall the chain rule of the Fréchet derivative,

$$\begin{array}{rl}\mathsf{D}\mathcal{F}\circ \mathcal{G}[\mathit{\text{A}}](\mathit{\text{h}})& =\mathsf{D}\mathcal{F}[\mathcal{G}(\mathit{\text{A}})](\mathsf{D}\mathcal{G}[\mathit{\text{A}}](\mathit{\text{h}}));\\ {\mathsf{D}}^2\mathcal{F}\circ \mathcal{G}[\mathit{\text{A}}](\mathit{\text{h}},\mathit{\text{k}})& ={\mathsf{D}}^2\mathcal{F}[\mathcal{G}(\mathit{\text{A}})](\mathsf{D}\mathcal{G}[\mathit{\text{A}}](\mathit{\text{k}}),\mathsf{D}\mathcal{G}[\mathit{\text{A}}](\mathit{\text{k}}))+\mathsf{D}\mathcal{F}[\mathcal{G}(\mathit{\text{A}})]({\mathsf{D}}^2\mathcal{G}[\mathit{\text{A}}](\mathit{\text{h}},\mathit{\text{k}})).\end{array}$$

Applying the formulae of the Fréchet derivative of the inversion function (see, for example, [[35]; example X.4.2; [42, exercise 3.27]),

$$\begin{array}{rl}\mathsf{D}{[\mathit{\text{X}}]}^{-1}(\mathit{\text{Y}})& =-{\mathit{\text{X}}}^{-1}\mathit{\text{Y}}{\mathit{\text{X}}}^{-1},\\ {\mathsf{D}}^2{[\mathit{\text{X}}]}^{-1}({\mathit{\text{Y}}}_1,{\mathit{\text{Y}}}_2)& ={\mathit{\text{X}}}^{-1}{\mathit{\text{Y}}}_1{\mathit{\text{X}}}^{-1}{\mathit{\text{Y}}}_2{\mathit{\text{X}}}^{-1}+{\mathit{\text{X}}}^{-1}{\mathit{\text{Y}}}_2{\mathit{\text{X}}}^{-1}{\mathit{\text{Y}}}_1{\mathit{\text{X}}}^{-1},\end{array}$$

concludes the desired results. ▪

Proposition A.7 (Operator Jensen inequality [34,43–45]) —

Let (Ω,Σ) be a measurable space and suppose that $I\subseteq \mathbb{R}$ is an open interval. Assume that, for every x∈Ω, K(x) is a (finite or infinite dimensional) square matrix and satisfies

$${\int }_{x\in \mathrm{\Omega }}\mathit{\text{K}}(\mathrm{d}x)\mathit{\text{K}}{(\mathrm{d}x)}^{†}=\mathit{\text{I}}$$

(identity matrix in ${\mathbb{M}}^{\mathrm{sa}}$). If $\mathit{\text{f}}:\mathrm{\Omega }\to {\mathbb{M}}^{\mathrm{sa}}$ is a measurable function for which σ( f(x))⊂I, for every x∈Ω, then

$$\varphi ({\int }_{x\in \mathrm{\Omega }}\mathit{\text{K}}(\mathrm{d}x)\mathit{\text{f}}(x)\mathit{\text{K}}{(\mathrm{d}x)}^{†})⪯{\int }_{x\in \mathrm{\Omega }}\mathit{\text{K}}(\mathrm{d}x)\varphi (\mathit{\text{f}}(x))\mathit{\text{K}}{(\mathrm{d}x)}^{†}\mu (\mathrm{d}x)$$

for every operator convex function $\varphi :I\to \mathbb{R}$. Moreover,

$$\mathrm{Tr}\left[\varphi ({\int }_{x\in \mathrm{\Omega }}\mathit{\text{K}}(\mathrm{d}x)\mathit{\text{f}}(x)\mathit{\text{K}}{(\mathrm{d}x)}^{†}\mu (\mathrm{d}x))\right]\le \mathrm{Tr}[{\int }_{x\in \mathrm{\Omega }}\mathit{\text{K}}(\mathrm{d}x)\varphi (\mathit{\text{f}}(x))\mathit{\text{K}}{(\mathrm{d}x)}^{†}\mu (\mathrm{d}x)]$$

for every convex function $\varphi :I\to \mathbb{R}$.

Data accessibility

This work does not have any experimental data.

Authors' contributions

Both authors contributed equally to this paper.

Competing interests

We have no competing interests.

Funding

M.-H.H. is supported by an ARC Future Fellowship under grant no. FT140100574.