Hilbert series, Howe duality, and branching rules

Abstract

Let λ be a partition, with l parts, and let F^λ be the irreducible finite dimensional representation of GL(m) associated to λ when l ≤ m. The Littlewood Restriction Rule describes how F^λ decomposes when restricted to the orthogonal group O(m) or to the symplectic group Sp(m/2) under the condition that l ≤ m/2. In this paper, this result is extended to all partitions λ. Our method combines resolutions of unitary highest weight modules by generalized Verma modules with reciprocity laws from the theory of dual pairs in classical invariant theory. Corollaries include determination of the Gelfand–Kirillov dimension of any unitary highest weight representation occurring in a dual pair setting, and the determination of their Hilbert series (as a graded module for p⁻). Let L be a unitary highest weight representation of sp(n, R), so*(2n), or u(p, q). When the highest weight of L plus ρ satisfies a partial dominance condition called quasi-dominance, we associate to L a reductive Lie algebra g_L and a graded finite dimensional representation B_L of g_L. The representation B_L will have a Hilbert series P(q) that is a polynomial in q with positive integer coefficients. Let δ(L) = δ be the Gelfand–Kirillov dimension of L and set c_L equal to the ratio of the dimensions of the zeroth levels in the gradings of L and B_L. Then the Hilbert series of L may be expressed in the form

In the easiest example of the correspondence L → B_L, the two components of the Weil representation of the symplectic group correspond to the two spin representations of an orthogonal group.

1. Extending the Littlewood Restriction Rule

Let V be a complex vector space of dimension n with a nondegenerate symmetric or skew symmetric form. The group leaving the form invariant is the group G, the orthogonal group O(n), or the symplectic group Sp(n/2) when n is even. The representations F^λ of GL(V) are parameterized by the partitions λ with at most n parts. For any integer partition λ = (λ₁ ≥ ⋯ ≥ λ_l) with at most l parts, let F be the irreducible representation of GL(l) indexed in the usual way by its highest weight (see ref. 1, section 5.2.1). Similarly, for each nonnegative integer partition μ with at most l parts, let V be the irreducible representation of Sp(l) with highest weight μ. Let E denote the irreducible representation of O(l) associated to the nonnegative integer partition ν with at most l parts and having Young diagram whose first two columns have lengths that sum to l or less. The parameters in the O(l) case are somewhat more delicate than the earlier two cases due in part to the disconnectedness of the group. For more details consult ref. 1 (chapter 10).

In 1940 D. E. Littlewood gave a formula for the decomposition of F^λ as a representation of G by restriction.

Theorem 1 [Littlewood Restriction (2, 3)].

Suppose that λ is a partition having at most n/2 (positive) parts.

(i) Suppose n is even and set k = n/2.

Then the multiplicity of the Sp(k) representation V^μ in F^λ equals

1.1

where the sum is over all nonnegative integer partitions ξ with columns of even length.

(ii) Then, for all n the multiplicity of the O(n) representation E^ν in F^λ equals

1.2

where the sum is over all nonnegative integer partitions ξ with rows of even length.

Recently, Gavarini (4) used Brauer algebra methods to re-prove and slightly extend Littlewood's formula. In this announcement, we describe some new results in character theory and an interpretation of these results through Howe duality. This will yield yet another proof of the Littlewood Restriction Rule and, more importantly, a generalization valid for all parameters λ. Our formula is rather more complex in form (see Section 5) but reduces to the above formula when λ is a partition having at most n/2 (positive) parts.

2. Gelfand–Kirillov Dimension

The character theory mentioned above is for unitarizable highest weight representations of a real reductive group. Much is known about these representations. They were classified in 1980 (5, 6). Their characters and cohomology formulas were determined a decade later (7–9). Here we report some refinements to this character theory and focus on a set of coarser invariants that can be determined more precisely. These are the Hilbert series, Gelfand–Kirillov dimension, and Bernstein degree.

Let L denote a unitarizable highest weight representation for g, one of the classical Lie algebras su(p, q), sp(n, ℝ), or so*(2n). These Lie algebras occur as part of the reductive dual pairs (10):

2.1

where n = p + q. Let GKdim denote the Gelfand–Kirillov dimension. Our first main result is as follows.

Theorem 2.

Suppose that L is a unitarizable highest weight representation occurring in one of the dual pair settings (2.1).

1.

If g is so*(2n), then GKdim(L) equals k(2n − 2k − 1) for 1 ≤ k ≤ [n − 2/2] and equals () otherwise.

2.

If g is sp(n), then GKdim(L) equals k/2 (2n − k + 1) for 1 ≤ k ≤ n − 1 and equals () otherwise.

3.

If g is u(p, q), then GKdim(L) equals k(n − k) for 1 ≤ k ≤ min{p, q} and equals pq otherwise.

Note that in all cases the GK dimension depends only on the dual pair setting given by k and n and is independent of λ otherwise. It is of course convenient to compute GK dimension of L directly from the highest weight. Let β denote the maximal root of g.

Corollary 3.

Set s = −(2(λ, β)/(β, β)). Then for so*(2n),

for sp(n),

and for u(p, q) with n = p + q,

The referee has noted that Theorem 2 is known to experts.

3. Hilbert Series

For any Hermitian symmetric pair g, k, let h be a Cartan subalgebra of both k and g and let g = p⁻ ⊕ k ⊕ p⁺ be a k-module decomposition of g. Choose Δ⁺ a set of positive roots for the root system Δ such that p⁺ is the span of the positive noncompact root spaces. Set ρ equal to half the sum of all the positive roots and let 𝒲 (resp. 𝒲_k) denote the Weyl group of g (resp. k).

Suppose L is a unitarizable highest weight representation with highest weight λ. In ref. 9, one associates to λ a subgroup 𝒲_λ of 𝒲, a root system Δ_λ, and a Hermitian symmetric pair g_λ, k_λ as follows. Let Δ denote the roots with root spaces in p⁺. For any root α let α^∨ denote the coroot defined by (α^∨, ξ) = 2(α, ξ)/(α, α). Suppose λ is k-dominant and define 𝒲_λ to be the subgroup of the Weyl group 𝒲 generated by the identity and all the reflections r_α that satisfy the following three conditions:

3.1

When the root system Δ contains only one root length, we call the roots short. Let Δ_λ equal the subset of Δ of elements δ for which r_δ ∈ 𝒲_λ and let Δ_λ,k = Δ_λ ∩ Δ_k, Δ = Δ_λ ∩ Δ⁺ and Δ = Δ_λ,k ∩ Δ⁺. Then in our setting Δ_λ and Δ_λ,k are abstract root systems and we let g_λ (resp. k_λ) denote the reductive Lie algebra with root system Δ_λ (resp. Δ_k,λ) and Cartan subalgebra equal to h. Then the pair (g_λ, k_λ) is a Hermitian symmetric pair [although not necessarily of the same type as (g, k) as is indicated by the example of the Weil representation below in Section 4, where the first pair is (D_n, A_n−1) and the second is (C_n, A_n−1)]. Set 𝒲_λ,k = 𝒲_λ ∩ 𝒲_k and define:

3.2

Let ρ_λ denote half the sum of the positive roots of g_λ and set δ_λ = ρ − ρ_λ. For any k-integral ξ ∈ h*, let ξ⁺⁺ denote the unique element in the 𝒲_k-orbit of ξ which is Δ dominant. For any k-dominant integral weight λ define the generalized Verma module with highest weight λ to be the induced module defined by: N(λ + ρ) = U(g) ⊗_U(k⊕p⁺) E_λ, with E_λ is the irreducible k representation with highest weight λ. For any k-dominant integral weights ξ and ζ, let m denote the multiplicity of the k representation E_ξ in the g generalized Verma module N(ζ + ρ). If S(p⁻) denotes the symmetric tensor algebra of p⁻, then

3.3

We let L(λ + ρ) denote the unique irreducible quotient of N(λ + ρ), which is the irreducible highest weight representation of g with highest weight λ.

Remark 1:

We have used three separate notations that overlap in the case of finite dimensional representations. The predominant notation for irreducible finite dimensional representations is by highest weight. However, in some classical settings, we switch to coordinate versions of highest weights that are then identified with integer partitions. Finally, when we are considering highest weight g-modules we use the generalized Verma module parameters of highest weight plus ρ.

Suppose λ is Δ-dominant and let Σ_λ denote the set of all positive roots α where (α, λ + ρ) = 0. Then Σ_λ is a set of strongly orthogonal noncompact roots. For such λ, we say λ + ρ is quasi-dominant if (α, λ + ρ) > 0 for all positive roots α that are orthogonal to all elements of Σ_λ.

For any Hermitian symmetric pair g, k and irreducible highest weight g-module M, let M₀ denote the k-submodule generated by any highest weight vector. Set M_j = p⁻⋅M_j−1 for j > 0. Define the Hilbert series H_M(q) of M by

3.4

Because the enveloping algebra of p⁻ is Noetherian, there is a unique integer d and a unique polynomial R_M(q) such that

3.5

In this setting, the integer d is the Gelfand–Kirillov dimension, d = GKdimM. Expanding the denominator as a series, we obtain

3.6

The coefficient of q^t can be expressed as a polynomial in t for large values of t. The leading coefficient of this polynomial is 1/(d − 1)!∑_0≤j≤e a_j = R_m(1)/(d − 1)! and is called the Bernstein degree of M and denoted B deg(M).

For any g_λ-dominant integral μ we let B_μ denote the irreducible finite dimensional g_λ-module with highest weight μ. Let p denote the subalgebra of p⁻ spanned by the root spaces for roots in −Δ ∩ Δ_λ and let B denote the grading of B_μ as a p-module. Define the Hilbert series by

3.7

This is of course a polynomial. For any k-dominant integral weight μ, let E_μ (resp. E_kλ,μ) denote the irreducible finite dimensional k (resp. k_λ)-module with highest weight μ.

Theorem 4.

Suppose that L = L(λ + ρ) is a unitarizable representation occurring in one of the dual pair settings (2.1) and that λ + ρ is quasi-dominant. Set d equal to the GK dimension of L as given by Theorem 2. Then the Hilbert series of L is

3.8

In particular, the Bernstein degree of L is given by

3.9

To illustrate through an example all the parameters, roots, Hermitian symmetric pairs, and finite dimensional representations involved, we consider a special set of representations, the Wallach representations (see ref. 11). Let r equal the split rank of g, and let ζ be the fundamental weight of g that is orthogonal to all the roots of k. Suppose g is isomorphic to so*(2n), sp(n), or su(p, q) and set c = 2, 1/2, or 1, depending on which of the three cases we are in. For 1 ≤ j < r define the jth Wallach representation W_j to be the unitarizable highest weight representation with highest weight −jcζ.

Suppose L is a Wallach representation for sp(n) having highest weight λ = −(j/2)(1, 1, … , 1) for some j, 1 ≤ j ≤ n − 1. Then

the set of strongly orthogonal roots

and Δ_λ is the root system of type D_n+1−j with simple roots

and the Hermitian symmetric pair g_λ, k_λ is so*(2n + 2 − 2j). Here Σ_λ contains a long root precisely when j is even and for all j, λ + ρ is quasi-dominant. The first n + 1 − j coordinates of λ + ρ − ρ_λ all equal j/2 and so this weight λ + ρ − ρ_λ equals j times the last fundamental weight of D_n+1−j on [g_λ, g_λ] ∩ h plus a central character of g_λ. Also in this example the dimensions of E_λ and E_kλ,λ+δλ are both one.

4. Examples

We next apply Theorem 4 to some well known representations to determine their GKdim, Hilbert series and Bernstein degree. We begin with the Wallach representations. For so*(2n) the Hilbert series for the first Wallach representation is

4.1

4.2

For sp(n) the Hilbert series for the first Wallach representation is

4.3

This is the Hilbert series for the half of the Weil representation generated by a one dimensional representation of k. The other part of the Weil representation has Hilbert series

4.4

For the components of the Weil representation of sp(n) there is an elementary derivation of the Hilbert series that we now give for comparison. Let L_e and L_o denote the even and odd components of the Weil representation acting respectively on the even and odd degree polynomials in n variables. Then

4.5

Breaking the numerator (1 + q)ⁿ into its even and odd degree parts in q gives the expressions 4.2 and 4.3.

For U(p, q) the Hilbert series for the first Wallach representation is:

4.6

These examples are obtained from Corollary 3 and Theorem 4 by writing out, respectively, the Hilbert series of the n − 3rd exterior power of the standard representation of so*(2n − 4), the two components of the spin representation of so*(2n), and the p − 1st fundamental representation of U(p − 1, q − 1). Combinatorial expressions for the Hilbert series of other Wallach representations can be given as well.

We say a highest weight representation L is positive if all the nonzero coefficients of the polynomial R_L(q) in 3.5 are positive. All Cohen–Macaulay S(p⁻)-modules including the Wallach representations are positive but many unitary highest weight representations are not. So Theorem 4 introduces a large class of positive highest weight representations that include all the examples in the previous paragraph. This theorem asserts that if λ + ρ is quasi-dominant then L(λ + ρ) is positive.

We conjecture that among positive highest weight modules occurring in the dual pair setting that are nontrivial quotients of generalized Verma modules, the modules with quasi-dominant highest weights prevail. However, the conditions of positivity and quasi-dominance are not equivalent. The former properly contain the latter. There are irreducible positive generalized Verma modules with highest weights which are not quasi-dominant. We would like to know to what extent the conditions of positivity and quasi-dominance are related.

5. Reciprocity Laws and Resolutions

The refinements in the character theory for unitarizable highest weight representations discussed above lead to new finite dimensional branching formulas extending the work of Littlewood (2). Let 𝒮 = 𝒫(M_2k×n) or 𝒫(M_k×n) as in 2.1. We consider the action of two dual pairs on 𝒮. The first is GL(m) × GL(n) with m = 2k or k and the second is G₁ × g₂, one of the two pairs (i) or (ii) in 2.1. In this setting G₁ is contained in GL(m) while GL(n) is closely related to the maximal compact subgroup of g₂. We can calculate the multiplicity of an irreducible G₁ × GL(n) representation in 𝒮 in two ways. The resulting identity is the branching formula. These reciprocity laws have been studied by Howe (12) where the two dual pairs are called see-saw pairs in the sense of Kudla.

The theory of dual pairs gives a decompositions of 𝒮 for each of the three actions.

As a GL(m) × GL(n) representation,

5.1

where the sum is over all nonnegative integer partitions having min{m, n} or fewer parts.

As a Sp(k) × so*(2n) representation,

5.2

where the sum is over all nonnegative integer partitions μ having min{k, n} or fewer parts.

And, as a O(k) × sp(n) representation,

5.3

where the sum is over all nonnegative integer partitions ν having min{k, n} or fewer parts and having a Young diagram whose first two columns sum to k or less.

As remarked above computing the multiplicity of V^μ⊗F in 𝒮 and E^ν⊗F in 𝒮 we obtain the following.

Theorem 5.

(i) The multiplicity of the Sp(k) representation V in F equals the multiplicity of F in the unitarizable highest weight representation V of so*(2n).

(ii) The multiplicity of the O(k) representation E in F equals the multiplicity of F in the unitarizable highest weight representation E of sp(n).

In the cases where the unitarizable highest weight representation is the full generalized Verma module, we call the parameter a generic point. A short calculation shows that the Littlewood hypothesis is included in the generic set.

Corollary 6.

(i) Suppose μ is a generic point. Then the multiplicity of the Sp(k) representation V in F equals

5.4

where the sum is over all nonnegative integer partitions ξ with columns of even length.

(ii) Suppose ν is a generic point. Then the multiplicity of the O(k) representation E in F equals

5.5

where the sum is over all nonnegative integer partitions ξ with rows of even length.

The formulas 5.4 and 5.5 are those of the Littlewood Restriction Theorem (ref. 2, p. 240). Littlewood's hypotheses correspond to the unitarizable highest weight representations lying in the discrete series or in some cases the limits of the discrete series.

For a general unitarizable highest weight representation L from ref. 9 we know the p⁺-cohomology formulas for L. From this data we obtain explicit resolutions for L where the terms of this resolution are sums of generalized Verma modules.

Theorem 7.

Suppose L = L(λ + ρ) is a unitarizable highest weight module. Then L admits a resolution in terms of generalized Verma modules. Specifically, for 1 ≤ i ≤ r_λ = card(Δ_λ ∩ Δ_n⁺), set C = Then there is a resolution of L:

5.6

Combining Theorems 5 and 7 we obtain the following.

Corollary 8.

Suppose that F^σ ⊗ F^σ occurs in 𝒮 as in 5.1 and that V and E, respectively, occur as Sp(k) and O(k) subrepresentations of F^σ. Then (i) the multiplicity of the Sp(k) representation V in F equals the multiplicity of F in V, which is given by

5.7

where μ^⧣ (resp. σ^⧣) is the highest weight of V (resp. F), (ii) and the multiplicity of the O(k) representation E in F equals the multiplicity of F in E, which is given by

5.8

where ν^⧣ (resp. σ^⧣) is the highest weight of E (resp. F).

The above corollary provides an algorithm for decomposing finite dimensional GL(n) representations into irreducible orthogonal or symplectic representations. Such algorithms have been implemented using the maple software package.

As an example of this multiplicity formula we list some cases for the orthogonal group.

Example 9:

Let ν be a partition such that,

with: d ≥ 2 and 2 + 2a + b ≤ k.

Then for all σ,

where ν₁ is the k tuple given by

6. Final Remarks

Theorem 7 is the technical backbone of the article. In addition to giving the extension to all parameters of Littlewood Restriction, the proof of Theorem 4 begins with two applications of Theorem 7, one for g and one for g_λ. The argument then passes to the computation of a ratio using the hypothesis of quasi-dominance.

During the time this announcement has been refereed, some related research has appeared (13). In this work, the authors begin with a highest weight module L and then consider the associated variety 𝒱(L) as defined by Vogan. This variety is the union of K_ℂ-orbits and equals the closure of a single orbit. In ref. 13, the GK dimension and the Bernstein degree of L are recovered from the corresponding objects for the variety 𝒱(L). As an example of their technique, they obtain the GK dimension and the degree of the Wallach representations (ref. 13, pp. 149–150). Our results in this setting obtain these two invariants as well as the full Hilbert series because all of the highest weights are quasi-dominant. The results of these two very different approaches have substantial overlap although neither subsumes the other.