On Matrix-Valued Monge–Kantorovich Optimal Mass Transport

Abstract

We present a particular formulation of optimal transport for matrix-valued density functions. Our aim is to devise a geometry which is suitable for comparing power spectral densities of multivariable time series. More specifically, the value of a power spectral density at a given frequency, which in the matricial case encodes power as well as directionality, is thought of as a proxy for a “matrix-valued mass density.” Optimal transport aims at establishing a natural metric in the space of such matrix-valued densities which takes into account differences between power across frequencies as well as misalignment of the corresponding principle axes. Thus, our transportation cost includes a cost of transference of power between frequencies together with a cost of rotating the principle directions of matrix densities. The two endpoint matrix-valued densities can be thought of as marginals of a joint matrix-valued density on a tensor product space. This joint density, very much as in the classical Monge–Kantorovich setting, can be thought to specify the transportation plan. Contrary to the classical setting, the optimal transport plan for matrices is no longer supported on a thin zero-measure set.

I. Introduction

The problem of optimal mass transport (OMT) dates back to the work of G. Monge in 1781 [1] while its modern formulation is due to L. Kantorovich [2]. In recent years the subject is developing rapidly due to its intrinsic significance and range of applications in physics, economics, and probability [3]–[5].

Our motivation for studying “matrix-valued transport” originates in the spectral analysis of multi-variable time series. Just as in scalar time series, spectral content is assessed based on (estimated) statistics of the underlying process, where these are simply moments of the corresponding power spectral density (PSD). Different metrics have been proposed to compare PSD’s for purposes of spectral approximation, estimation, and system modeling (see [6], [7] and the references therein). However, since spectra are estimated based on integrals, weak^* metrics¹ are preferable since they provide continuity of statistics to perturbations in the PSD. Earlier metrics and so called, divergence measures, typically fail in this respect (see [7]). Hence, for this reason, optimal mass transport which endows the space of (scalar) probability/mass/power densities with a natural weak^* metric—the Wasserstein metric, is of particular interest. Our aim in this paper is to develop one possible such generalization of the Wasserstein metric that allows comparison of matrix-valued density functions in a similar spirit.

The scalar OMT theory has been adapted in [8] to model slowly time-varying changes in power spectra of time series and has been used for statistical estimation, data assimilation, and morphing. While in scalar time series, the power spectral content may drift across frequencies over time (e.g., when considering Doppler effects, echolocation of a moving target, etc.), in vector-valued time series the power spectral content may shift principle directions as well. In fact, such a rotation of the power-specral content is typical in general antenna-arrays when a scatterer changes position with respect to array elements. Therefore, a concept of transport between matrix-valued densities requires that we take into account both, the cost of shifting power across frequencies as well as the cost of rotating the corresponding principle axes. Besides our particular formulation of a “non-commutative” Monge–Kantorovich transportation problem and of a correponding metric, the main results in this paper are i) that the optimal transport can be cast as a convex-optimization problem, ii) the geodesics and transport paths can be determined using convex programming, and iii) the optimal transport plan has support which, in contrast to the classical Monge–Kantorovich setting, is no longer contained on a thin zero-measure set. The relevance of the proposed metric is highlighted in examples on spectral morphing and spectral tracking in the final section of the paper.

II. Preliminaries on Optimal Mass Transport

Consider two probability density functions μ₀ and μ₁ supported on ℝ and let ℳ(μ₀, μ₁) be the set of probability measures m on ℝ × ℝ with μ₀ and μ₁ as marginals, i.e.,

$$\underset{ℝ}{\int }m(x,y)dy={\mu }_0(x),\underset{ℝ}{\int }m(x,y)dx={\mu }_1(y),m(x,y)\ge 0.$$

Clearly, ℳ(μ₀, μ₁) is not empty since already the product μ₀(x)μ₁(y) ∈ ℳ(μ₀, μ₁). Probability densities are thought of as distributions of mass and the optimal mass transport problem is to determine

$${\mathcal{T}}_c({\mu }_0,{\mu }_1):=\underset{m\in \mathcal{M}({\mu }_0,{\mu }_1)}{inf}\underset{ℝ\times ℝ}{\int }c(x,y)m(x,y)dxdy$$ (1)

where c(x, y) is the cost of transporting one unit of mass from location x to y. In particular, when c(x, y) = |x − y|², the optimal cost gives rise to the 2-Wasserstein metric

$$W_2({\mu }_0,{\mu }_1)={\mathcal{T}}_2{({\mu }_0,{\mu }_1)}^{\frac{1}{2}}$$

where

$${\mathcal{T}}_2({\mu }_0,{\mu }_1):=\underset{m\in \mathcal{M}({\mu }_0,{\mu }_1)}{inf}\underset{ℝ\times ℝ}{\int }{\mid x-y\mid }^{2}m(x,y)dxdy.$$ (2)

In general, (1) is a linear program with dual

$$\underset{\varphi ,\psi }{sup}\left\{\underset{ℝ}{\int }({\varphi }_0(x){\mu }_0(x)-{\varphi }_1(x){\mu }_1(x))dx\mid {\varphi }_0(x)-{\varphi }_1(y)\le c(x,y)\right\}$$ (3)

where ϕ₀, ϕ₁ are continuous, see [3]. For the quadratic cost c(x, y) = |x − y|² and in one spatial dimension, 𝓣₂(μ₀, μ₁) can also be written explicitly in terms of the cumulative distributions functions

$$M_i(x)=\underset{-\infty }{\overset{x}{\int }}{\mu }_i(x)dx\text{for}i=0,1$$

in the form

$${\mathcal{T}}_2({\mu }_0,{\mu }_1)=\underset{0}{\overset{1}{\int }}{\mid M_0^{-1}(t)-M_1^{-1}(t)\mid }^{2}dt.$$ (4)

In this case, the optimal joint probability density m ∈ ℳ(μ₀, μ₁) has support on (x, T (x)) where T (x) is the sub-differential of a convex lower semi-continuous function, see [3, p. 75]. More specifically, T (x) is uniquely defined by

$$M_0(x)=M_1(T(x)).$$ (5)

Interestingly, a geodesic μ_τ (τ ∈ [0, 1]) between μ₀ and μ₁ can be written explicitly as well in terms of a corresponding cumulative function M_τ, for each τ, defined via

$$M_{\tau }((1-\tau )x+\tau T(x))=M_0(x).$$ (6)

Indeed, it readily follows that:

$$\begin{array}{l}{W}_2({\mu }_0,{\mu }_{\tau })=\tau W_2({\mu }_0,{\mu }_1)\\ W_2({\mu }_{\tau },{\mu }_1)=(1-\tau )W_2({\mu }_0,{\mu }_1)\end{array}$$

and that μ_τ (τ ∈ [0, 1]) is a geodesic.

III. Matrix-Valued Optimal Mass Transport

We consider the one-dimensional family of matrix-valued functions

$$\mathcal{F}:=\left\{\mathit{\mu }(·)\mid \text{for}x\in ℝ,\mathit{\mu }{(x)}^{\ast }=\mathit{\mu }(x)\in {ℂ}^{n\times n},\mathit{\mu }(x)\ge 0,\text{tr}\left(\underset{ℝ}{\int }\mathit{\mu }(x)dx\right)=1\right\}.$$

These are Hermitian, positive semi-definite matrix-valued functions on ℝ normalized so that their trace integrates to 1. They will be referred to as matrix-valued densities and can be thought of as a generalization of probability density functions. The scalar-valued tr(μ) represents mass at location x. Thus, all elements in ℱ have the same total mass over the support. Below, we motivate a particular cost of transportation between such matrix-valued functions and introduce a suitable generalization of the Monge–Kantorovich OMT to matrix-valued densities.

A. Tensor Product and Partial Trace

Consider two n-dimensional real or complex (Hilbert) spaces ℋ₀ and ℋ₁, let ℒ(ℋ₀) and ℒ(ℋ₁) denote the space of linear operators on ℋ₀ and ℋ₁, respectively, and let μ₀ ∈ ℒ(ℋ₀) and μ₁ ∈ ℒ(ℋ₁). Thus, in the present subsection, μ_i (i ∈ {0, 1}) are fixed matrices. We denote their tensor product by μ₀ ⊗ μ₁ ∈ ℒ(ℋ₀ ⊗ ℋ₁) which is formally defined via

$${\mathit{\mu }}_0\otimes {\mathit{\mu }}_1:u\otimes v\mapsto {\mathit{\mu }}_{0}u\otimes {\mathit{\mu }}_{1}v.$$

Since our spaces are finite-dimensional this can be identified with the Kronecker product of the corresponding matrix representation of the two operators. The space ℒ(ℋ₀ ⊗ ℋ₁) is the span of all products μ₀ ⊗ μ₁ with μ_i ∈ ℒ(ℋ_i) for i ∈ {0, 1}.

Consider μ ∈ ℒ(ℋ₀ ⊗ ℋ₁). The partial traces tr_ℋ0 and tr_ℋ1, or tr₀ and tr₁ for brevity, are linear maps

$$\begin{array}{l}{\text{tr}}_1:\mathcal{L}({\mathcal{H}}_0\otimes {\mathcal{H}}_1)\to \mathcal{L}({\mathcal{H}}_0):\mathit{\mu }\mapsto {\text{tr}}_1(\mathit{\mu })\\ {\text{tr}}_0:\mathcal{L}({\mathcal{H}}_0\otimes {\mathcal{H}}_1)\to \mathcal{L}({\mathcal{H}}_1):\mathit{\mu }\mapsto {\text{tr}}_0(\mathit{\mu })\end{array}$$

defined uniquely by the property that on simple products they act as follows:

$${\text{tr}}_1({\mathit{\mu }}_0\otimes {\mathit{\mu }}_1)=\text{tr}({\mathit{\mu }}_1){\mathit{\mu }}_0\text{and}{\text{tr}}_0({\mathit{\mu }}_0\otimes {\mathit{\mu }}_1)=\text{tr}({\mathit{\mu }}_0){\mathit{\mu }}_1$$

for any μ₀ ∈ ℒ(ℋ₀) and μ₁ ∈ ℒ(ℋ₁). Alternatively, μ ∈ ℒ(ℋ₀ ⊗ ℋ₁) can be represented by a matrix [μ_ik,ℓm] of size n² × n² as it maps a basis element u_i ⊗ v_k ∈ ℋ₀ ⊗ ℋ₁ to Σ_ℓ,m μ_ik,ℓmu_ℓ ⊗ v_m. Then, the partial trace e.g., tr₁(μ) is the represented by the n × n matrix with (i, ℓ)-th entry Σ_kμ_ik,ℓk, for 1 ≤ i, ℓ ≤ n. Likewise the (k, m)-th entry of tr₀(μ) is Σ_i μ_ik,im, for 1 ≤ k, m ≤ n. See [9] for the significance of partial trace in the context of quantum mechanics.

B. Joint Matrix-Valued Density

We now return to considering matrix-valued density functions μ₀, μ₁ ∈ ℱ. A naive attempt is to seek a joint density m ≥ 0 having support on ℝ × ℝ and having μ₀, μ₁ as “marginals,” i.e., so that

$$\underset{ℝ}{\int }\mathit{m}(x,y)dy={\mathit{\mu }}_0(x),\underset{ℝ}{\int }\mathit{m}(x,y)dx={\mathit{\mu }}_1(y).$$ (7)

However, in contrast to the scalar case, such an m does not exist in general. To see this, consider the case of matrix valued measures

$$\begin{array}{l}{\mathit{\mu }}_0(x)=\left[\begin{array}{cc}\frac{1}{2}& 0\\ 0& 0\end{array}\right]\delta (x-1)+\left[\begin{array}{cc}0& 0\\ 0& \frac{1}{2}\end{array}\right]\delta (x-2),\\ {\mathit{\mu }}_1(y)=\left[\begin{array}{cc}\frac{1}{4}& -\frac{1}{4}\\ -\frac{1}{4}& \frac{1}{4}\end{array}\right]\delta (y-1)+\left[\begin{array}{cc}\frac{1}{4}& \frac{1}{4}\\ \frac{1}{4}& \frac{1}{4}\end{array}\right]\delta (y-2)\end{array}$$

where δ(·) denotes the Dirac delta. If m(x, y) were to exist, its support would be contained in {(1, 1), (1, 2) (2, 1), (2, 2)}. It is easy to see that there cannot be a consistent selection of four 2 × 2 matrices so that, in pairs, they sum up to the coefficients making up μ₀(x) and μ₁(y).

Thus, any natural definition of a transportation plan requires that the joint density lives in a bigger space. A particular formulation is as follows: we seek

$$\mathit{m}(x,y)\text{being}{n}^2\times n^2\text{positive}\text{semi-definite}\text{matrix}$$ (8a)

for (x, y) ∈ ℝ × ℝ, such that

$${\mathit{m}}_0(x,y):={\text{tr}}_1(\mathit{m}(x,y)),{\mathit{m}}_1(x,y):={\text{tr}}_0(\mathit{m}(x,y))$$ (8b)

$$\underset{ℝ}{\int }{\mathit{m}}_0(x,y)dy={\mathit{\mu }}_0(x),\underset{ℝ}{\int }{\mathit{m}}_1(x,y)dx={\mathit{\mu }}_1(y)$$ (8c)

and denote

$$\mathit{M}({\mathit{\mu }}_0,{\mathit{\mu }}_1):=\{\mathit{m}\mid (8\mathrm{a})-(8\mathrm{c})\text{are}\text{satisfied}\}.$$

Since μ₀ ⊗ μ₁ ∈ M(μ₀, μ₁), the set M(μ₀, μ₁) is clearly not empty.

We next motivate a suitable transportation cost. This is a functional on the joint density M(μ₀, μ₁), just as in the scalar case. However, besides penalizing transport of mass between two points x and y, we also impose a penalty on a corresponding rotation as well.

C. Transportation Cost

As indicated earlier, we interpret tr(m(x, y)) as the total “mass” that is being transferred from x to y. We consider a scalar cost² c(x, y) = (x − y)² and the “mass transference” cost

$$\underset{\mathit{m}\in \mathit{M}({\mathit{\mu }}_0,{\mathit{\mu }}_1)}{min}\underset{ℝ\times ℝ}{\int }c(x,y)\text{tr}(\mathit{m}(x,y))dxdy.$$ (9)

This coincides with the optimal transportation cost between scalar-valued densities tr(μ₀) and tr(μ₁). Thus, if tr(μ₀(x)) = tr(μ₁(x)), the optimal value of (9) is zero since it reduces to optimal transport between identical scalar marginals. Thus, (9) fails to quantify mismatch of directionality between the given matrix-valued marginals. Below, we introduce a term that penalizes directionality missmatch.

We assume throughout that the marginals are positive definite pointwise. Then, for i ∈ {0, 1}, tr(μ_i(x)) represents the total mass at x while μ_i(x)/tr(μ_i(x)), normalized to have trace 1, encapsulates directional information. Likewise, for the joint density m(x, y), assuming that m(x, y) ≠ 0, we define the normalized partial traces

$$\begin{array}{l}{\underset{\_}{\text{tr}}}_0(\mathit{m}(x,y)):={\text{tr}}_0(\mathit{m}(x,y))/\text{tr}(\mathit{m}(x,y))\\ {\underset{\_}{\text{tr}}}_1(\mathit{m}(x,y)):={\text{tr}}_1(\mathit{m}(x,y))/\text{tr}(\mathit{m}(x,y)).\end{array}$$

Their difference captures the directional mismatch between the two partial traces. Hence, we introduce

$$\text{tr}\left({\Vert ({\underset{\_}{\text{tr}}}_0-{\underset{\_}{\text{tr}}}_1)\mathit{m}(x,y)\Vert }_{\mathrm{F}}^2\mathit{m}(x,y)\right)$$

to quantify the rotational mismatch and we consider the cost functional

$$\text{tr}\left(\left(c(x,y)+\lambda {\Vert ({\underset{\_}{\text{tr}}}_0-{\underset{\_}{\text{tr}}}_1)\mathit{m}(x,y)\Vert }_{\mathrm{F}}^2\right)\mathit{m}(x,y)\right)$$

with λ > 0, to weigh in the relative significance of the linear and rotational penalties.

D. Optimal Transportation Problem

In view of the above, we define

$${\mathcal{T}}_{2,\lambda }({\mathit{\mu }}_0,{\mathit{\mu }}_1):=\underset{\mathit{m}\in \mathit{M}({\mathit{\mu }}_0,{\mathit{\mu }}_1)}{min}\underset{ℝ\times ℝ}{\int }\text{tr}\left((c+\lambda {\Vert ({\underset{\_}{\text{tr}}}_0-{\underset{\_}{\text{tr}}}_1)\mathit{m}\Vert }_{\mathrm{F}}^2)\mathit{m}\right)dxdy$$ (10)

with c(x, y) = (x − y)², and show next that (10) is in fact a convex optimization problem.

From the definition

$$\begin{array}{l}{\underset{\_}{\text{tr}}}_0(\mathit{m})\text{tr}(\mathit{m})={\text{tr}}_0(\mathit{m}),\\ {\underset{\_}{\text{tr}}}_1(\mathit{m})\text{tr}(\mathit{m})={\text{tr}}_1(\mathit{m})\end{array}$$

and hence

$$\begin{array}{l}{\Vert ({\underset{\_}{\text{tr}}}_0-{\underset{\_}{\text{tr}}}_1)\mathit{m}\Vert }_{\mathrm{F}}^2\text{tr}(\mathit{m})=\frac{{\Vert ({\underset{\_}{\text{tr}}}_0-{\underset{\_}{\text{tr}}}_1)\mathit{m}\Vert }_{\mathrm{F}}^2\text{tr}{(\mathit{m})}^2}{\text{tr}(\mathit{m})}\\ =\frac{{\Vert ({\text{tr}}_0-{\text{tr}}_1)\mathit{m}\Vert }_{\mathrm{F}}^2}{\text{tr}(\mathit{m})}.\end{array}$$

Now let m(x, y) = tr(m(x, y)) and m₀(x, y), m₁(x, y) be as in (8). The expression for the optimal cost in (10) is lower bounded by

$$\begin{array}{l}\underset{{\mathit{m}}_0,{\mathit{m}}_1,m}{min}\{\int \left(c(x,y)m(x,y)+\lambda \frac{{\Vert {\mathit{m}}_0-{\mathit{m}}_1\Vert }_{\mathrm{F}}^2}{m}\right)dxdy\mid \\ {\mathit{m}}_0(x,y),{\mathit{m}}_1(x,y)\ge 0,\\ \text{tr}({\mathit{m}}_0(x,y))=\text{tr}({\mathit{m}}_1(x,y))=m(x,y)\\ \int {\mathit{m}}_0(x,y)dy={\mathit{\mu }}_0(x),\int {\mathit{m}}_1(x,y)dx={\mathit{\mu }}_1(y)\}.\end{array}$$ (11)

For an optimal triple m̂, m̂₀, m̂₁ of (11), m̂:= m̂₀ ⊗ m̂₁ is a minimizer of (10) that gives the same optimal value as (11). Thus, the optimal cost in (10) is equivalently written as (11).

For x > 0, the expression (y − z)²/x is jointly convex in the arguments x, y, z, see e.g., [10, p. 72]. It readily follows that the integral in (11) is a jointly convex functional of its arguments. All additional constraints in (11) are convex as well and, therefore, so is the optimization problem.

IV. On the Geometry of Optimal Mass Transport

An important result in the (scalar) OMT theory is that the transportation plan is the sub-differential of a convex function and has support on a thin zero-measure set, see e.g., [3, p. 92]. This property is not shared by the optimal transportation plan between matrix-valued density functions as we explain next.

In standard scalar OMT with convex transportation cost, the optimal transportation plan has a certain cyclically monotonic property [3]. More specifically, if (x₁, y₁), (x₂, y₂) are two points where the transportation plan has support (i.e., m(x, y) ≠ 0), then x₂ > x₁ implies y₂ ≥ y₁. The interpretation is that optimal transportation paths of mass elements do not cross. For the case of matrix-valued distributions as in (3), this property may not hold in the same way. However, interestingly, a weaker monotonicity property holds for the supporting set of the optimal matrix transportation plan. The property is defined next and the precise statement is given in Proposition 2 below.

Definition 1

A set 𝓢 ⊂ ℝ² is called a ρ-monotonically nondecreasing, for ρ > 0, if for any two points (x₁, y₁), (x₂, y₂) ∈ 𝓢, it holds that

$$(x_2-x_1)(y_1-y_2)\le \rho .$$

A geometric interpretation for a ρ-monotonically non-decreasing set is that if (x₁, y₁), (x₂, y₂) ∈ 𝓢 and x₂ > x₁, y₁ > y₂, then the area of the rectangle with vertices (x_i, y_j) (i, j ∈ {1, 2}) is not larger than ρ. The transportation plan of the scalar-valued optimal transportation problem with a quadratic cost has support on a 0-monotonically non-decreasing set.

Proposition 2

Given μ₀, μ₁ ∈ ℱ, let m be the optimal transportation plan in (10) with c(x, y) = (x − y)² and λ > 0. Then m has support on at most a (4 · λ)-monotonically nondecreasing set.

Proof

See the Appendix.

Further, the optimal transportation cost 𝓣_{2, λ}(μ₀, μ₁) satisfies:

1. 𝓣_2,λ (μ₀, μ₁) = 𝓣_2,λ(μ₁, μ₀),

2. 𝓣_2,λ (μ₀, μ₁) ≥ 0,

3. 𝓣_2,λ (μ₀, μ₁) = 0 if and only if μ₀ = μ₁.

Thus, although 𝓣_2,λ(μ₀, μ₁) can be used to compare matrix-valued densities, it is not a metric and neither is ${\mathcal{T}}_{2,\lambda }^{1/2}$ since the triangular inequality does not hold in general. We will introduce a slightly different formulation of a transportation problem which does give rise to a metric.

A. Optimal Transport on a Subset

In this subsection, we restrict attention to a certain subset of transport plans M(μ₀, μ₁) and show that the corresponding optimal transportation cost induces a metric. More specifically, let

$${\mathit{M}}_0({\mathit{\mu }}_0,{\mathit{\mu }}_1):=\{\mathit{m}\mid \mathit{m}(x,y)=({\mathit{\mu }}_0(x)\otimes {\mathit{\mu }}_1(y))a(x,y),\mathit{m}\in \mathit{M}\}.$$

For m(x, y) ∈ M₀(μ₀, μ₁),

$$\begin{array}{l}{\underset{\_}{\text{tr}}}_0(\mathit{m}(x,y)):={\mathit{\mu }}_1(y)/\text{tr}({\mathit{\mu }}_1(y))\\ {\underset{\_}{\text{tr}}}_1(\mathit{m}(x,y)):={\mathit{\mu }}_0(x)/\text{tr}({\mathit{\mu }}_0(x)).\end{array}$$

Given μ₀ and μ₁, the “orientation” of the mass of m(x, y) is fixed. Thus, in this case, the optimal transportation cost is

$${\stackrel{\sim }{\mathcal{T}}}_{2,\lambda }({\mathit{\mu }}_0,{\mathit{\mu }}_1):=\underset{\mathit{m}\in {\mathit{M}}_0({\mathit{\mu }}_0,{\mathit{\mu }}_1)}{min}\int \text{tr}\left(\left(c+\lambda {\Vert ({\underset{\_}{\text{tr}}}_0-{\underset{\_}{\text{tr}}}_1)\mathit{m}(x,y)\Vert }_{\mathrm{F}}^2\right)\mathit{m}\right)dxdy.$$ (12)

Proposition 3

For 𝓣_2,λ as in (12) with λ > 0 and μ₀, μ₁ ∈ ℱ,

$$d_{2,\lambda }({\mathit{\mu }}_0,{\mathit{\mu }}_1):={\left({\stackrel{\sim }{\mathcal{T}}}_{2,\lambda }({\mathit{\mu }}_0,{\mathit{\mu }}_1)\right)}^{\frac{1}{2}}$$ (13)

defines a metric on ℱ.

Proof

It is straightforward to see that

$$d_{2,\lambda }({\mathit{\mu }}_0,{\mathit{\mu }}_1)=d_{2,\lambda }({\mathit{\mu }}_1,{\mathit{\mu }}_0)\ge 0$$

and that d_2,λ (μ₀, μ₁) = 0 if and only if μ₀ = μ₁. We now show that the triangle inequality holds as well. For μ₀, μ₁, μ₂ ∈ ℱ, let

$$\begin{array}{l}{\mathit{m}}_{01}(x,y)=\frac{{\mathit{\mu }}_0(x)}{\text{tr}({\mathit{\mu }}_0(x))}\otimes \frac{{\mathit{\mu }}_1(y)}{\text{tr}({\mathit{\mu }}_1(y))}{m}_{01}(x,y)\\ {\mathit{m}}_{12}(y,z)=\frac{{\mathit{\mu }}_1(y)}{\text{tr}({\mathit{\mu }}_1(y))}\otimes \frac{{\mathit{\mu }}_2(z)}{\text{tr}({\mathit{\mu }}_2(z))}{m}_{12}(y,z)\end{array}$$

denote the optimal transportation plan for the pairs (μ₀, μ₁) and (μ₁, μ₂), respectively, where m₀₁ and m₁₂ are two (scalar-valued) joint densities on ℝ² with marginals tr(μ₀), tr(μ₁) and tr(μ₁), tr(μ₂), respectively. Given m₀₁(x, y) and m₁₂(y, z) there is a joint density function m(x, y, z) on ℝ³ with m₀₁ and m₁₂ as the marginals on the corresponding subspaces [3, p. 208]. We set

$$\mathit{m}(x,y,z)=\frac{{\mathit{\mu }}_0(x)}{\text{tr}({\mathit{\mu }}_0(x))}\otimes \frac{{\mathit{\mu }}_1(y)}{\text{tr}({\mathit{\mu }}_1(y))}\otimes \frac{{\mathit{\mu }}_2(z)}{\text{tr}({\mathit{\mu }}_2(z))}m(x,y,z)$$

and note that it has m₀₁ and m₁₂ as matrix-valued marginal distributions. Now, let m₀₂(x, z) = (μ₀(x)/tr μ₀(x)) ⊗ (μ₂(z)/tr μ₂(z))m₀₂(x, z) be the marginal of m(x, y, z) when tracing out the y-component. This m₀₂(x, z) is a possible transportation plan between μ₀ and μ₂. Hence

$$\begin{array}{l}{d}_{2,\lambda }({\mathit{\mu }}_0,{\mathit{\mu }}_2)\le {\left(\underset{{ℝ}^2}{\int }\left({(x-z)}^2+\lambda {\Vert \frac{{\mathit{\mu }}_0(x)}{\text{tr}{\mathit{\mu }}_0(x)}-\frac{{\mathit{\mu }}_2(z)}{\text{tr}{\mathit{\mu }}_2(z)}\Vert }_{\mathrm{F}}^2\right)m_{02}dxdz\right)}^{\frac{1}{2}}\\ ={\left(\underset{{ℝ}^3}{\int }\left({(x-z)}^2+\lambda {\Vert \frac{{\mathit{\mu }}_0(x)}{\text{tr}{\mathit{\mu }}_0(x)}-\frac{{\mathit{\mu }}_2(z)}{\text{tr}{\mathit{\mu }}_2(z)}\Vert }_{\mathrm{F}}^2\right)mdxdydz\right)}^{\frac{1}{2}}\\ ={\left(\underset{{ℝ}^3}{\int }\left({(x-y+y-z)}^2+\lambda {\Vert \frac{{\mathit{\mu }}_0(x)}{\text{tr}{\mathit{\mu }}_0(x)}-\frac{{\mathit{\mu }}_1(y)}{\text{tr}{\mathit{\mu }}_1(y)}+\frac{{\mathit{\mu }}_1(y)}{\text{tr}{\mathit{\mu }}_1(y)}-\frac{{\mathit{\mu }}_2(z)}{\text{tr}{\mathit{\mu }}_2(z)}\Vert }_{\mathrm{F}}^2\right)mdxdydz\right)}^{\frac{1}{2}}\\ \le {\left(\underset{{ℝ}^2}{\int }\left({(x-y)}^2+\lambda {\Vert \frac{{\mathit{\mu }}_0(x)}{\text{tr}{\mathit{\mu }}_0(x)}-\frac{{\mathit{\mu }}_1(y)}{\text{tr}{\mathit{\mu }}_1(y)}\Vert }_{\mathrm{F}}^2\right)m_{01}dxdy\right)}^{\frac{1}{2}}+{\left(\underset{{ℝ}^2}{\int }\left({(y-z)}^2+\lambda {\Vert \frac{{\mathit{\mu }}_1(y)}{\text{tr}{\mathit{\mu }}_1(y)}-\frac{{\mathit{\mu }}_2(z)}{\text{tr}{\mathit{\mu }}_2(z)}\Vert }_{\mathrm{F}}^2\right)m_{12}dydz\right)}^{\frac{1}{2}}\\ =d_{2,\lambda }({\mathit{\mu }}_0,{\mathit{\mu }}_1)+d_{2,\lambda }({\mathit{\mu }}_1,{\mathit{\mu }}_2)\end{array}$$

where the last inequality is due to the metric property of L₂.

If λ = 0, then 𝓣̃_2,0(μ₀, μ₁) is exactly the OMT cost between the scalar-valued densities tr(μ₀) and tr(μ₁) as was explained earlier. In particular, for an optimal transportation plan m(x, y) between tr(μ₀) and tr(μ₁), the matrix-valued transportation plan m(x, y) = (μ₀(x)/tr(μ₀(x))) ⊗ (μ₁(y)/tr(μ₁(y)))m(x, y) is optimal between μ₀ and μ₁ which satisfies that 𝓣̃_2,0(μ₀, μ₁) = 𝓣₂(tr(μ₀), tr(μ₁)). Thus, 𝓣̃_2,0(μ₀, μ₁) = 0 if and only if tr(μ₀) = tr(μ₁). Hence, 𝓣̃_2,0 fails to be a metric. Moreover, since for any λ ≥ 0 it holds that 𝓣_2,λ ≤ 𝓣̃_2,λ, if tr(μ₀) = tr(μ₁) then 𝓣_2,0(μ₀, μ₁) also equals to zero.

Proposition 4

Given μ₀, μ₁ ∈ ℱ, let m be the optimal transportation plan in (13), then m has support on at most a (2 · λ)-monotonically non-decreasing set.

Proof

We need to prove that if m(x₁, y₁) ≠ 0 and m(x₂, y₂) ≠ 0, then x₂ > x₁, y₁ > y₂ implies

$$(y_1-y_2)(x_2-x_1)\le 2\lambda .$$ (14)

Assume that m evaluated at the four points (x_i, y_j) with i, j ∈ {1, 2}, is as follows:

$$\mathit{m}(x_i,y_i)=m_{ij}·A_i\otimes B_j$$

with

$$A_i=\frac{{\mathit{\mu }}_0(x_i)}{\text{tr}({\mathit{\mu }}_1(x_i))},B_i=\frac{{\mathit{\mu }}_0(y_i)}{\text{tr}({\mathit{\mu }}_1(y_i))}$$

and m₁₁, m₂₂ > 0. The steps of the proof are similar to those of Proposition 2 detailed in the Appendix: first, we assume that Proposition 4 fails and that

$$(y_1-y_2)(x_2-x_1)>2\lambda .$$

Then we show that a smaller cost can be incurred by rearranging the “mass.” Consider the situation when m₂₂ ≥ m₁₁ first and let m̂ be a new transportation plan with

$$\begin{array}{l}\widehat{\mathit{m}}(x_1,y_1)=0\\ \widehat{\mathit{m}}(x_1,y_2)=(m_{11}+m_{12})·A_1\otimes B_2\\ \widehat{\mathit{m}}(x_2,y_1)=(m_{11}+m_{21})·A_2\otimes B_1\\ \widehat{\mathit{m}}(x_2,y_2)=(m_{22}-m_{11})·A_2\otimes B_2.\end{array}$$

Then, m̂, m have the same marginals at the four points, the cost incurred by m is

$$\sum _{i=1}^2\sum _{j=1}^2{m}_{ij}\left({(x_i-y_j)}^2+\lambda {\Vert A_i-B_j\Vert }_{\mathrm{F}}^2\right)$$ (15)

and the cost incurred by m̂ is

$$(m_{11}+m_{12})\left({(x_1-y_2)}^2+\lambda {\Vert A_1-B_2\Vert }_{\mathrm{F}}^2\right)+(m_{11}+m_{21})\left({(x_2-y_1)}^2+\lambda {\Vert A_2-B_1\Vert }_{\mathrm{F}}^2\right)+(m_{22}-m_{11})\left({(x_2-y_2)}^2+\lambda {\Vert A_2-B_2\Vert }_{\mathrm{F}}^2\right).$$ (16)

To show that (15) is larger than (16), after canceling common terms, it suffices to show that

$${(y_1-x_1)}^2+{(y_2-x_2)}^2+\lambda {\Vert A_1-B_1\Vert }_{\mathrm{F}}^2+\lambda {\Vert A_2-B_2\Vert }_{\mathrm{F}}^2\ge {(y_2-x_1)}^2+{(y_1-x_2)}^2+\lambda {\Vert A_1-B_2\Vert }_{\mathrm{F}}^2+\lambda {\Vert A_2-B_1\Vert }_{\mathrm{F}}^2.$$

However, the above holds true since

$$\begin{array}{l}{(y_1-x_1)}^2+{(y_2-x_2)}^2+\lambda {\Vert A_1-B_1\Vert }_{\mathrm{F}}^2+\lambda {\Vert A_2-B_2\Vert }_{\mathrm{F}}^2\ge {(y_1-x_1)}^2+{(y_2-x_2)}^2\\ ={(y_1-x_2)}^2+{(y_2-x_1)}^2+2(x_2-x_1)(y_1-y_2)\\ >{(y_1-x_2)}^2+{(y_2-x_1)}^2+4\lambda \\ \ge {(y_1-x_2)}^2+{(y_1-x_2)}^2+\lambda \left({\Vert A_1-B_2\Vert }_{\mathrm{F}}^2+{\Vert A_2-B_1\Vert }_{\mathrm{F}}^2\right).\end{array}$$

The last inequality follows from:

$${\Vert A_1-B_2\Vert }_{\mathrm{F}}^2=\text{tr}\left(A_1^2+B_2^2-2A_1{B}_2\right)\le \text{tr}\left(A_1^2+B_2^2\right)\le 2.$$

The case m₁₁ > m₂₂ proceeds similarly.

V. Examples

We give two different examples where matrix-valued OMT can be directly applied. Both relate to spectral analysis of multivariable time series.³

A. Spectral Morphing

We first highlight the relevance of matrix-valued OMT to spectral analysis with a numerical example on spectral morphing. The idea is to model slowly time-varying changes in the spectral domain by geodesics in a suitable geometry (see e.g., [7], [8]). The use of geodesic interpolation can be thought of as a regularization technique. Indeed, geodesics smoothly shift spectral power across frequencies lessening the possibility of a fade-in fade-out artifacts and OMT, for scalar power spectra, has been used to this end in [7], [8]. Below we exemplify how geodesics appear in matrix-valued OMT.

Starting with μ₀, μ₁ ∈ ℱ we approximate the geodesic between the two by constructing N − 1 points intermediate matrix densities. To this end, we set μ_τ0 = μ₀ and μ_τN = μ₁, and determine μ_τk ∈ ℱ for k = 1, …, N − 1 as the solution to

$$\underset{{\mathit{\mu }}_{{\tau }_k},0<k<N}{min}\sum _{k=0}^{N-1}{\mathcal{T}}_{2,\lambda }({\mathit{\mu }}_{{\tau }_{k+1}},{\mathit{\mu }}_{{\tau }_k}).$$ (17)

As noted in Section III-D, this can be obtained numerically via convex programming. The present example uses

$$\begin{array}{l}{\mathit{\mu }}_0=\left[\begin{array}{cc}1& 0\\ 0.2e^{-\mathrm{j}\theta }& 1\end{array}\right]\left[\begin{array}{cc}\frac{1}{{\mid a_0(e^{\mathrm{j}\theta })\mid }^2}& 0\\ 0& 0.01\end{array}\right]\left[\begin{array}{cc}1& 0.2e^{\mathrm{j}\theta }\\ 0& 1\end{array}\right]\\ {\mathit{\mu }}_1=\left[\begin{array}{cc}1& 0.2\\ 0& 1\end{array}\right]\left[\begin{array}{cc}0.01& 0\\ 0& \frac{1}{{\mid a_1(e^{\mathrm{j}\theta })\mid }^2}\end{array}\right]\left[\begin{array}{cc}1& 0\\ 0.2& 1\end{array}\right]\end{array}$$

with

$$\begin{array}{l}{a}_0(z)=\left(z^2-1.8cos\left(\frac{\pi }{4}\right)z+{0.9}^2\right)\left(z^2-1.4cos\left(\frac{\pi }{3}\right)z+{0.7}^2\right)\\ a_1(z)=\left(z^2-1.8cos\left(\frac{\pi }{6}\right)z+{0.9}^2\right)\left(z^2-1.5cos\left(\frac{2\pi }{15}\right)z+{0.75}^2\right)\end{array}$$

shown in Fig. 1. Since the value of a power spectral density at each point in frequency is a 2 × 2 Hermitian matrix, we have used the (1, 1), (1, 2), and (2, 2) subplots to display the magnitude of the corresponding entries, i.e., |μ(1, 1)|, |μ(1, 2)|, (= |μ(2, 1)|), and |μ(2, 2)|, respectively, and the (2,1) subplot to display the phase ∠μ(1, 2) (= −∠μ(2, 1)).

Fig. 1.

Subplots (1,1), (1,2), and (2,2) show μ_i(1, 1), |μ_i(1, 2)| (same as |μ_i(2, 1)|) and μ_i(2, 2). Subplot (2,1) shows ∠(μ_i(2, 1)) for i ∈ {0, 1} in blue and red, respectively.

The 3-D plots in Fig. 2 refer to (17), with λ = 0.1, for an approximation of a geodesic. The two boundary plots represent the power spectra μ₀ and μ₁ shown in blue and red, respectively, using the same convention about magnitudes and phases explained above. There are in total 7 power spectra μ_τk, k = 1, …, 7 shown along the geodesic between μ₀ and μ₁, and the time-indices correspond to τ_k = k/8. It is interesting to observe the smooth shift of the energy over the geodesic path from the one “channel” to the other while, at the same time, the corresponding peak shifts from one frequency to another. One should bear in mind that the so-constructed geodesic is a non-parametric path interpolating/linking the given spectra.

Fig. 2.

Interpolated results μ_τk for k = 0, …, 8 computed from (17) with μ₀ and μ₁ as the two boundary points: subplots (1, 1), (1, 2), and (2, 2) show μ_τk (1, 1), |μ_τk (1, 2)| (same as |μ_τk (2, 1)|) and μ_τk (2, 2), subplot (2, 1) shows ∠(μ_τk (2, 1)).

B. Regularization Using Geodesics

Consider two time series

$$\begin{array}{l}{x}_1(t)=a_1(t)cos({\theta }_1(t)t+{\varphi }_{1a})+a_2(t)cos({\theta }_2(t)t+{\varphi }_{1b})+w_1(t)\\ x_2(t)=a_2(t)cos({\theta }_1(t)t+{\varphi }_{2a})+a_1(t)cos({\theta }_2(t)t+{\varphi }_{2b})+w_2(t)\end{array}$$

for t = 1, …, 2000, both consisting of sinusoidal signals with time-varying amplitude and frequency (chirp-like) with added white noise w₁(t) and w₂(t). The amplitude a₁(t) decreases from 1.2 to 0.1 while a₂(t) increases from 0.1 to 1.2. Frequency θ₁(t) decreases from (π/4) to (π/4) − (π/30) while θ₂(t) increases from (π/3) to (π/3) + (π/30). Then [w₁(t), w₂(t)]′ is white, with independent components, and sampled from a zero-mean Gaussian distribution with covariance $\left[\begin{array}{cc}3& 1.5\\ 1.5& 3\end{array}\right]$. The initial phases of the sinusoids are randomly selected in [0, 2π].

Since we are dealing with non-stationary time series, we truncate the observed time series, to segments of length equal to 200, to retain resolution. Thus, we let x_i,k(t) := x_i(200k+t) with i = 1, 2, k = 0, …, 9 and process separately the segments {x_i,k(1), x_i,k(2), …, x_i,k(200)}. We obtain matrix-valued sample covariances {R_k,−20, …, R_k,0, …, R_k,20} for each. We then determine autoregressive models based on these sample covariances and, thereby, the corresponding power spectral density functions. More specifically, for ℓ = 0, 1, …, 20, we compute

$$R_{k,\ell }:=\frac{1}{200}\sum _{i=\ell }^{200}\left[\begin{array}{c}{x}_{1,k}(i)\\ x_{2,k}(i)\end{array}\right][x_{1,k}(i-\ell ),x_{2,k}(i-\ell )]$$

and let $R_{k,-\ell }=R_{k,\ell }^{\prime }$. We then solve the Yule-Walker equations

$$R_{k,\ell }=\sum _{i=1}^{20}{A}_{k,i}{R}_{k,\ell -i},\text{for}\ell =1,\dots ,20$$

for the autoregressive (matricial) coefficients A₁, …, A₂₀. We let $\mathrm{\Omega }:=R_0-{\sum }_{i=1}^{20}{A}_{k,i}{R}_{k,-i}$ be the corresponding innovation variance. The estimated power spectral density function for the kth segment, denoted as μ̂_τk, is given as μ̂_τk(θ) = A_k(e^jθ)⁻¹Ω(A_k(e^jθ)^*)⁻¹ with

$$A_k(e^{j\theta })=\left(I-A_{k,1}z-\cdots -A_{k,20}{z}^{20}\right){\mid }_{z=e^{j\theta }}.$$

We scale μ̂_τk so that the integral of its trace is normalized to one. Thus, the observation record is used to obtain 10 PSD’s denoted as μ̂_τk, for k = 1, …, 10. These represent estimates of the spectral power at intermediary points in time. We change the time scale so that τ₁ = 0 and τ₁₀ = 1. The spectrogram is shown in Fig. 3(a).

Fig. 3.

(a) Shows the estimated spectrogram of the observed time series and (b) corresponds to the geodesic-fitted spectrogram.

We construct an OMT-geodesic to regularize the estimated PSD’s. This idea was proposed and carried out in [8] for scalar time series and scalar PSD’s. For the present matrix-valued setting, the geodesic is obtained by solving

$$\underset{{\mathit{\mu }}_{{\tau }_k}}{min}\left\{\sum _{k=1}^{10}{\mathcal{T}}_{2,\lambda }({\mathit{\mu }}_{{\tau }_k},{\widehat{\mathit{\mu }}}_{{\tau }_k})\mid {\mathit{\mu }}_{{\tau }_k}\text{are}\text{on}\text{an}\text{OMT}\text{geodesic}\right\}.$$ (18)

An explicit formula of the OMT geodesic is not available. However, in light of Proposition 2, (18) can be approximated for small λ as follows. Let μ̂_τk = tr(μ̂_τk) for k = 1, …, 10. These are scalar-valued PSD’s. Let M̂_τk denote the corresponding cumulative distribution functions. For λ small, following [8], we compute μ₀ := tr(μ₀) and μ₁ := tr(μ₁) via solving:

$$\underset{{\mu }_0,{\mu }_1}{min}\sum _{k=1}^{10}\underset{0}{\overset{1}{\int }}{\left((1-{\tau }_k)M_0^{-1}(v)+{\tau }_k{M}_1^{-1}(v)-{\widehat{M}}_{{\tau }_k}^{-1}(v)\right)}^{2}dv$$

with M₀ and M₁ representing the cumulative distribution function of μ₀ and μ₁, respectively. Then, as was shown in [8], the μ_τk’s for 1 < k < 10 can be computed via

$$M_{{\tau }_k}^{-1}(v)=(1-{\tau }_k)M_0^{-1}(v)+{\tau }_k{M}_1^{-1}(v).$$

The matrix-valued PSD’s μ₀ and μ₁ are obtained by solving

$$\underset{{\mathit{\mu }}_0,{\mathit{\mu }}_1}{min}\sum _{k=1}^{10}\underset{0}{\overset{1}{\int }}{\Vert (1-{\tau }_k)\frac{{\mathit{\mu }}_0(M_0^{-1}(v))}{{\mu }_0(M_0^{-1}(v))}+{\tau }_k\frac{{\mathit{\mu }}_1(M_1^{-1}(v))}{{\mu }_1(M_1^{-1}(v))}-\frac{{\widehat{\mathit{\mu }}}_{{\tau }_k}({\widehat{M}}_{{\tau }_k}^{-1}(v))}{{\widehat{\mu }}_{{\tau }_k}({\widehat{M}}_{{\tau }_k}^{-1}(v))}\Vert }_{\mathrm{F}}^{2}dv$$

and the μ_τk’s for 1 < k < 10 are computed via

$$\frac{{\mathit{\mu }}_{{\tau }_k}(M_{{\tau }_k}^{-1}(v))}{{\mu }_{{\tau }_k}(M_{{\tau }_k}^{-1}(v))}=(1-{\tau }_k)\frac{{\mathit{\mu }}_0(M_0^{-1}(v))}{{\mu }_0(M_0^{-1}(v))}+{\tau }_k\frac{{\mathit{\mu }}_1(M_1^{-1}(v))}{{\mu }_1(M_1^{-1}(v))}.$$

We display this geodesic-fitted spectrogram in Fig. 3(b). It can be seen that the shift of energy from one channel to another and between resonant frequencies is smoother than that shown in Fig. 3(a) (which is a spectrogram based on matricial auto-regressive models).

In order to compare the resolution between the two techniques (spectrogram based on AR-modeling vs. geodesic regularization), we identify the frequency and directionality of peak power for the two power spectral densities and compare principal direction. This we explain next as the result is quite revealing and suggesting.

For each μ̂_τk (θ), we find two frequencies θ₁ and θ₂ where the power spectral densities (PSD’s) have locally maximal power, i.e., the two frequencies where tr(μ̂_τk (θ)) has the largest peaks. Then we compute the (normalized) eigenvectors corresponding to the dominant eigenvalues of μ_τk (θ₁) and μ_τk (θ₂), respectively. These eigenvectors are shown in Fig. 4(a) using black dashed lines. The red and green plots in Fig. 4(a) represent the path of the two eigenvectors as τ_k increases from 0 to 1. The axes in Fig. 4(a) correspond to the two channels/components of the time series and τ_k. (Should all the power be present in one of the two channels, the eigenvector would line up, accordingly, to one of the axes.) The values of the eigenvector when projected onto the two channels/axes, reflect the energy of the signals in the corresponding channels. Thus, in antenna-array applications, the direction of eigenvectors corresponds to the direction of a scatterer relative to the array. Statistical errors are reflected in the jagged nature of the paths when these are based on a spectrogram as in Fig. 4(a). However, when comparing with the eigenvectors of the OMT regularized spectrogram/AR-models μ_τk, the corresponding paths shown in Fig. 4(b) are smooth. Direct comparison between Fig. 4(a) and (b) highlights the potential advantages of using geodesics as a means to regularize power distribution in non-stationary time series.

Fig. 4.

In (a), the trajectories of the dominant eigenvector of μ̂_τk (θ₁) and μ̂_τk (θ₂) are shown in red and blue, respectively. The corresponding trajectories of μ_τk (θ₁) and μ_τk (θ₂) are shown in (b).

VI. Conclusion

The geometry of Monge–Kantorovich optimal mass transport provides scalar densities with a natural metric structure (see [11], [12]; also [13] for a systems viewpoint and connections to image analysis and power spectra). Our interest has been in extending such a geometric structure to matrix-valued densities. To this end, we formulated one possible matrix-valued version of the Monge–Kantorovich transportation problem. Computations require convex programming and the framework directly extends the scalar case. An alternative generalization of the Monge–Kantorovich theory to a “non-commutative” setting has been given in the context of the theory of free-probabilities [14]. However, this may not be suitable for matrix-valued power distributions as it is not weak^* continuous. Alternative non-commutative generalizations of the Wasserstein metric are given in [15]–[17]. Possible connections between the formulation herein and these alternative viewpoints is the subject of current investigation.

Biographies

Lipeng Ning received the B.S. and M.S. degrees in control science and engineering from Beijing Institute of Technology, Beijing, China, in 2006 and 2008, respectively, and the Ph.D. degree in electrical and computer engineering from the University of Minnesota, Minneapolis, in 2013.

He is currently a Postdoc Research Fellow in Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA. His research interests include system identification, spectral estimation, sparse signal recovery, and diffusion magnetic resonance imaging.

Tryphon T. Georgiou (F’00) received the Diploma in mechanical and electrical engineering from the National Technical University of Athens, Athens, Greece, in 1979 and the Ph.D. degree from the University of Florida, Gainesville, in 1983.

He is a faculty member in the Department of Electrical and Computer Engineering, University of Minnesota, Minneapolis, and the Vincentine Hermes-Luh Chair. He has served as a Co-Director of the Control Science and Dynamical Systems Center at the University of Minnesota (1990–2014), and on the Board of Governors of the Control Systems Society of the IEEE (2002–2005).

Dr. Georgiou has been a recipient of the George S. Axelby Outstanding Paper Award of the IEEE Control Systems Society for the years 1992, 1999, and 2003. He is a Foreign Member of the Royal Swedish Academy of Engineering Sciences (IVA).

Allen Tannenbaum (F’08) is a faculty member in computer science and applied mathematics at Stony Brook University, Stony Brook, NY, USA. He works in control, computer vision, and medical imaging.

Appendix

Appendix: Proof of Proposition 2

We need to show that if m(x₁, y₁) ≠ 0 and m(x₂, y₂) ≠ 0, then x₂ > x₁, y₁ > y₂ implies

$$(x_2-x_1)(y_1-y_2)\le 4\lambda .$$ (19)

Without loss of generality, let

$$\mathit{m}(x_i,y_j)=m_{ij}·A_{ij}\otimes B_{ij}$$ (20)

with A_ij, B_ij ≥ 0, tr(A_ij) = tr(B_ij) = 1 and i, j ∈ {1, 2}. Note that m₁₂ and m₂₁ could be zero if m does not have support on the particular point. We assume that the condition in the proposition fails and that

$$(x_2-x_1)(y_1-y_2)>4\lambda .$$ (21)

We then show that by rearranging the mass, the cost can be reduced.

We first consider the situation when m₂₂ ≥ m₁₁. By rearranging the value of m at the four points (x_i, y_j) with i, j ∈ {1, 2}, we construct a new transportation plan m̃ at these four locations as follows:

$$\stackrel{\sim }{\mathit{m}}(x_1,y_1)=0$$ (22a)

$$\stackrel{\sim }{\mathit{m}}(x_1,y_2)=(m_{11}+m_{12})·{\stackrel{\sim }{A}}_{12}\otimes {\stackrel{\sim }{B}}_{12}$$ (22b)

$$\stackrel{\sim }{\mathit{m}}(x_2,y_1)=(m_{11}+m_{21})·{\stackrel{\sim }{A}}_{21}\otimes {\stackrel{\sim }{B}}_{21}$$ (22c)

$$\stackrel{\sim }{\mathit{m}}(x_2,y_2)=(m_{22}-m_{11})·A_{22}\otimes B_{22}$$ (22d)

where

$$\begin{array}{ll}{\stackrel{\sim }{A}}_{12}=\frac{m_{11}{A}_{11}+m_{12}{A}_{12}}{m_{11}+m_{12}},& {\stackrel{\sim }{B}}_{12}=\frac{m_{11}{B}_{22}+m_{12}{B}_{12}}{m_{11}+m_{12}}\\ {\stackrel{\sim }{A}}_{21}=\frac{m_{11}{A}_{22}+m_{21}{A}_{21}}{m_{11}+m_{21}},& {\stackrel{\sim }{B}}_{21}=\frac{m_{11}{B}_{11}+m_{21}{B}_{21}}{m_{11}+m_{21}}.\end{array}$$

This new transportation plan m̃ has the same marginals as m at x₁, x₂ and y₁, y₂. The original cost incurred by m at these four locations is

$$\sum _{i=1}^2\sum _{j=1}^2{m}_{ij}\left({(x_i-y_j)}^2+\lambda {\Vert A_{ij}-B_{ij}\Vert }_{\mathrm{F}}^2\right)$$ (23)

while the cost incurred by m̃ is

$$(m_{11}+m_{12})\left({(x_1-y_2)}^2+\lambda {\Vert {\stackrel{\sim }{A}}_{12}-{\stackrel{\sim }{B}}_{12}\Vert }_{\mathrm{F}}^2\right)+(m_{11}+m_{21})\left({(x_2-y_1)}^2+\lambda {\Vert {\stackrel{\sim }{A}}_{21}-{\stackrel{\sim }{B}}_{21}\Vert }_{\mathrm{F}}^2\right)+(m_{22}-m_{11})\left({(x_2-y_2)}^2+\lambda {\Vert A_{22}-B_{22}\Vert }_{\mathrm{F}}^2\right).$$ (24)

After simplification, to show that (23) is larger than (24), it suffices to show that

$$2m_{11}(x_2-x_1)(y_1-y_2)$$ (25)

is larger than

$$\lambda m_{11}\left(\sum _{i=1}^2\sum _{j\ne i}{\Vert {\stackrel{\sim }{A}}_{ij}-{\stackrel{\sim }{B}}_{ij}\Vert }_{\mathrm{F}}^2-\sum _{i=1}^2{\Vert A_{ii}-B_{ii}\Vert }_{\mathrm{F}}^2\right)$$ (26a)

$$+\lambda m_{12}\left({\Vert {\stackrel{\sim }{A}}_{12}-{\stackrel{\sim }{B}}_{12}\Vert }_{\mathrm{F}}^2-{\Vert A_{12}-B_{12}\Vert }_{\mathrm{F}}^2\right)$$ (26b)

$$+\lambda m_{21}\left({\Vert {\stackrel{\sim }{A}}_{21}-{\stackrel{\sim }{B}}_{21}\Vert }_{\mathrm{F}}^2-{\Vert A_{21}-B_{21}\Vert }_{\mathrm{F}}^2\right).$$ (26c)

From (21), it follows that the value in (25) is greater than 20λm₁₁. We derive upper bounds for each term in (26). First,

$$(26a)\le \lambda m_{11}\left({\Vert {\stackrel{\sim }{A}}_{12}-{\stackrel{\sim }{B}}_{12}\Vert }_{\mathrm{F}}^2+{\Vert {\stackrel{\sim }{A}}_{21}-{\stackrel{\sim }{B}}_{21}\Vert }_{\mathrm{F}}^2\right)\le 4\lambda m_{11}$$

where the last inequality follows from the fact that:

$${\Vert A-B\Vert }_{\mathrm{F}}^2=\text{tr}(A^2-2AB+B^2)\le \text{tr}(A^2+B^2)\le 2$$

for any A, B ≥ 0 with tr(A) = tr(B) = 1. Now consider

$$\begin{array}{l}{\Vert {\stackrel{\sim }{A}}_{12}-{\stackrel{\sim }{B}}_{12}\Vert }_{\mathrm{F}}^2-{\Vert A_{12}-B_{12}\Vert }_{\mathrm{F}}^2=\text{tr}\left(({\stackrel{\sim }{A}}_{12}-{\stackrel{\sim }{B}}_{12}+A_{12}-B_{12})({\stackrel{\sim }{A}}_{12}-{\stackrel{\sim }{B}}_{12}-A_{12}+B_{12})\right)\\ =\frac{m_{11}}{m_{11}+m_{12}}({\Vert A_{11}-B_{22}\Vert }_{\mathrm{F}}^2-{\Vert A_{12}-B_{12}\Vert }_{\mathrm{F}}^2-\frac{m_{12}}{m_{11}+m_{12}}{\Vert A_{11}-B_{22}-A_{12}+B_{12}\Vert }_{\mathrm{F}}^2)\\ \le \frac{m_{11}}{m_{11}+m_{12}}{\Vert A_{11}-B_{22}\Vert }_{\mathrm{F}}^2\\ \le 2\frac{m_{11}}{m_{11}+m_{12}}\end{array}$$

where the second equality follows from the definitions of Ã₁₂ and B̃₁₂ while the last inequality is obtained by bounding the terms in the trace. Thus, referring to expressions by the respective equation numbering

$$(26b)\le 2\lambda m_{12}\frac{m_{11}}{m_{11}+m_{12}}\le 2\lambda m_{11}.$$

In a similar manner, (26c) ≤ 2λm₁₁. Therefore

$$(26)\le 8\lambda m_{11}<(25)$$

which implies that the cost incurred by m̃ is smaller than the cost incurred by m.

For the case where m₁₁ > m₂₂, we can prove the claim by constructing a new transportation plan m̂ with values

$$\begin{array}{l}\widehat{\mathit{m}}(x_1,y_1)=(m_{11}-m_{22})·A_{11}\otimes B_{11}\\ \widehat{\mathit{m}}(x_1,y_2)=(m_{12}+m_{22})·{\widehat{A}}_{12}\otimes {\widehat{B}}_{12}\\ \widehat{\mathit{m}}(x_2,y_1)=(m_{21}+m_{22})·{\widehat{A}}_{21}\otimes {\widehat{B}}_{21}\\ \widehat{\mathit{m}}(x_2,y_2)=0\end{array}$$

with

$$\begin{array}{ll}{\widehat{A}}_{12}=\frac{m_{12}{A}_{12}+m_{22}{A}_{11}}{m_{12}+m_{22}},& {\widehat{B}}_{12}=\frac{m_{12}{B}_{12}+m_{22}{B}_{22}}{m_{12}+m_{22}}\\ {\widehat{A}}_{21}=\frac{m_{21}{A}_{21}+m_{22}{A}_{22}}{m_{21}+m_{22}},& {\widehat{B}}_{21}=\frac{m_{21}{B}_{21}+m_{22}{B}_{11}}{m_{21}+m_{22}}.\end{array}$$

The rest of the proof is carried out in a similar manner.