Advanced Studies in Pure Mathematics 26, 1998 Analysis on Homogeneous Spaces and Representations of Lie Groups pp. 1–26

A Langlands classification for unitary representations

David A. Vogan, Jr.

Abstract.

The Langlands classification theorem describes all admissible representations of a reductive group G in terms of the tempered representations of Levi subgroups of G. I will describe work with Susana Salamanca-Riba that provides (conjecturally) a similar description of the unitary representations of G in terms of certain very special unitary representations of Levi subgroups.

§1. Introduction

Suppose G is a real reductive Lie group in Harish-Chandra’s class (see [HC], section 3). There are two powerful general techniques for constructing irreducible unitary representations of G. Parabolic induction is based on real analysis and geometry on certain compact homogeneous spaces G/P. Cohomological induction is based on complex analysis on certain indefinite Kähler homogeneous spaces G/L. When G is SL(2, ℝ), parabolic induction gives rise to Bargmann’s unitary principal series representations, and cohomological induction to Bargmann’s discrete series. We are concerned here not with the details of these constructions, but rather with the question of classification: which unitary representations can be found (and which cannot be found) by these methods. In the case of SL(2, ℝ), what is missing are the complementary series representations, the two “limits of discrete series representations,” and the trivial representation.

In general a precise answer is difficult to obtain, and from some perspectives even undesirable. By deformation arguments, one can push either construction to yield larger sets of unitary representations. Thus for example the complementary series representations of SL(2, ℝ) may be regarded as parabolically induced, and the limits of discrete series as cohomologically induced. The trivial representation emerges from either construction as a kind of very singular limiting case.

In this paper we will avoid such arguments entirely, using only the simplest versions of parabolic and cohomological induction. We seek especially a simple understanding of the unitary representations not obtained from a smaller group by either kind of induction. For cohomological induction such a result (partly conjectural) appears in [SV]. For parabolic induction there is a result in [Knp], Theorem 16.10. Combining them, we will find

Theorem 1.1. Suppose G is a semisimple Lie group in Harish-Chandra’s class. Define Π_{u,small}(G) to be the set of equivalence classes of irreducible unitary representations (π, H_{π}) of G with the property that the infinitesimal character of π (in the Harish-Chandra parametrization) belongs to the convex hull of the Weyl group orbit of the half-sum of positive roots. The representations in Π_{u,small}(G) are called unitarily small.

Fix a Cartan decomposition G = K exp(𝔰_{0}) of G, and a maximal torus T ⊂ K. Let R(∧𝔰) denote the convex hull of the weights of T in the exterior algebra of 𝔰. An irreducible representation of K is called small if its weights all belong to R(∧𝔰).

(1) Assume that Conjecture 0.6 of [SV] holds for G. Then any irreducible unitary representation of G not belonging to Π_{u,small}(G) may be constructed by parabolic or cohomological induction from an irreducible unitary representation of a reductive subgroup of G of lower dimension.

(2) The set Π_{u,small}(G) is compact in the Fell topology.

(3) The “extremal points” of Π_{u,small}(G), the unitary representations of infinitesimal character equal to the half-sum of positive roots, are those with non-vanishing continuous cohomology described in [VZ].

(4) Any unitarily small representation of G must contain a small representation of K. Conversely, assume that Conjecture 0.6 of [SV] holds for G. Then any unitary representation of G containing a small representation of K and having real infinitesimal character must be unitarily small.

The notion of “real infinitesimal character” is taken for example from [Grn], Definition 5.4.11; we will recall it in section 2 below.

Theorem 1.1 says that the unitarily small representations provide building blocks from which other unitary representations may be constructed by (parabolic or cohomological) induction. When G is SL(2, ℝ), the unitarily small representations are the spherical complementary series (parametrized by a half-open interval [0, 1)), the first two discrete series representations π_{±1}^{±}, the trivial representation ρ, and the two limits of discrete series π_{0}^{±}. The Fell topology on this set makes the complementary series converge to the three points π_{±1}^{±} and ρ as the parameter approaches 1; it is not a Hausdorff topology, but the pathology is not very serious. Essentially we have a closed interval with one of the endpoints tripled, and two isolated points.

There are some conjectural descriptions of large families of unitary representations of reductive groups, and it is interesting to see how these descriptions relate to Theorem 1.1. One of the most famous is the method of coadjoint orbits, which suggests that unitary representations of G should be related to orbits of G on 𝔤_{0}^{∗}. There are constructions of orbits parallel to parabolic induction and to cohomological induction; these are described in some detail in [Vorb]. There is also an analogue of Theorem 1.1, describing precisely a family of orbits from which all others may constructed. (This is just the Jordan decomposition, Proposition 4 of [Vorb]). They are the nilpotent coadjoint orbits, of which there are finitely many for each reductive group. This suggests

Conjecture 1.2. Suppose G is a semisimple Lie group in Harish-Chandra’s class. Then the unitary representations associated to nilpotent coadjoint orbits should be unitarily small in the sense of Theorem 1.1.

In the absence of a definition of the phrase “unitary representations associated to nilpotent coadjoint orbits,” this should really not be called a conjecture; it might be thought of as a desideratum for the missing definition. Nevertheless, we can offer a little evidence for it.

Example 1.3. One-dimensional representations. Suppose G is a semisimple Lie group in Harish-Chandra’s class. One nilpotent coadjoint orbit is the point {0}. The corresponding unitary representations are precisely the irreducible representations of the group of connected components G/G_{0}. (This is a well-established desideratum for any correspondence from orbits to representations.) Any such representation has infinitesimal character equal to the half-sum of a set of positive roots in Harish-Chandra’s parametrization, and so is unitarily small.

Example 1.4. Infinitesimal character zero. Suppose G is a semisimple Lie group in Harish-Chandra’s class. Let us consider the collection Π^{0}(G) of all irreducible admissible representations of G of infinitesimal character 0 in Harish-Chandra’s parametrization. This is a finite set (as it would be with 0 replaced by any fixed infinitesimal character). It follows from Harish-Chandra’s work on asymptotics of matrix coefficients that the representations in Π^{0}(G) are automatically tempered, and therefore unitarizable. We therefore have Π^{0}(G) ⊂ Π_{u,small}(G).

It turns out (again from Harish-Chandra’s work on tempered representations) that Π^{0}(G) is non-empty if and only if G is quasisplit; that is, if and only if the minimal parabolic subgroup P = MAN of G has M_{0} abelian. We assume this for the rest of the example. Then the representations in Π^{0}(G) are precisely the irreducible constituents of Ind_{P_{0}}^{G}(1).

The assumption that G is quasisplit is equivalent to the existence of nilpotent coadjoint orbits of dimension equal to the number of roots. (This is the largest possible dimension for any coadjoint orbit.) Such nilpotent orbits are called principal. According to some standard desiderata for an orbit correspondence, the unitary representations attached to principal nilpotent orbits should include all of those in Π^{0}(G) (see [Orng], Definitions 12.1 and 12.4). Because these are unitarily small, this is at least consistent with Conjecture 1.2.

It is a fairly simple matter to extend the discussion in Examples 1.3 and 1.4 to cover all special unipotent representations (see [Orng], Definition 12.4).

Our goal in this paper is to explain carefully the statement of Theorem 1.1. Section 2 recalls Harish-Chandra’s theory of infinitesimal characters, the notion of “real infinitesimal character,” and some related structure theory in the Lie algebra. Section 3 uses those ideas to formulate the part of Theorem 1.1 concerning parabolic induction. Section 4 reviews briefly the notion of cohomological induction. Section 5 outlines the part of Theorem 1.1 concerning cohomological induction. Section 6 gives a more precise version of the statement in Theorem 1.1(1).

Here are some references for the rest of Theorem 1.1. The assertion (2) is elementary, although the necessary basic facts about the Fell topology (many of which are due to Miličić) are a little difficult to find in the literature. Some of them are discussed in the last section of [SV]. The result in (3) is a special case of the main theorem of [Sal]. Assertion (4) is contained in [SV], mostly in Theorem 6.7 there.

§2. Infinitesimal character

We continue to work with a reductive group G in Harish-Chandra’s class. In general Lie groups will be denoted by upper case Roman letters, Lie algebras by the corresponding lower case German letters with a subscript 0, and complexifications by dropping the subscript 0. For example,

𝔤_{0} = Lie(G),  𝔤 = 𝔤_{0} ⊗_{ℝ} ℂ.  (2.1)(a)

We will often make use of a Cartan decomposition

G = K exp 𝔰_{0},  (2.1)(b)

with corresponding Cartan involution θ. Occasionally we will make use of a non-degenerate Ad(G)-invariant symmetric bilinear form ⟨, ⟩ on 𝔤_{0}. We may arrange for this form to be preserved also by the Cartan involution, and to be positive definite on 𝔰_{0} and negative definite on 𝔨_{0}.

Recall that a Cartan subgroup of G is by definition the centralizer in G of a Cartan subalgebra of 𝔤_{0}. Any such subgroup is conjugate by G to one preserved by θ. If H is a θ-stable Cartan subgroup, then the Cartan decomposition becomes a direct product

H = T A,  T = H ∩ K,  A = exp(𝔥_{0} ∩ 𝔭_{0}).  (2.1)(c)

Here the first factor is compact and the second is a vector group. A similar argument applies to the center Z(G) of the group G: there is a direct product decomposition

Z(G) = T^{z}(G)A^{z}(G), T^{z}(G) = Z(G) ∩ K, A^{z}(G) = exp(𝔷(𝔤_{0}) ∩ 𝔰_{0}). (2.1)(d)

Definition 2.2. The center (as an abstract algebra) of the universal enveloping algebra U(𝔤) is written ℨ(𝔤). Because U(𝔤) is generated by 𝔤, this is

ℨ(𝔤) = {u ∈ U(𝔤) | Xu − uX = 0 (X ∈ 𝔤)}.

By definition of the adjoint action of 𝔤 on U(𝔤), this is

ℨ(𝔤) = {u ∈ U(𝔤) | ad(X)(u) = 0 (X ∈ 𝔤)}.

Because the adjoint action of 𝔤_{0} is the differential of the adjoint action of the group G, this is equivalent to

ℨ(𝔤) = {u ∈ U(𝔤) | Ad(g)(u) = u (g ∈ G)}.

This last equivalence uses in an essential way the assumption that G is in Harish-Chandra’s class; without it, we would have to replace G by its identity component G_{0}.

Theorem 2.3. Suppose 𝔥 is a Cartan subalgebra of the reductive Lie algebra 𝔤, and W = W (𝔤, 𝔥) is the corresponding Weyl group. Then there is an algebra isomorphism (the Harish-Chandra homomorphism)

ξ_{𝔥}: ℨ(𝔤) → S(𝔥)^{W} .

For any λ ∈ 𝔥^{∗}, evaluation at λ is an algebra homomorphism from S(𝔥) to ℂ. Composition of this homomorphism with the Harish-Chandra homomorphism gives an algebra homomorphism

ξ_{𝔥}(λ): ℨ(𝔤) → ℂ.

(1) Two such homomorphisms ξ_{𝔥}(λ) and ξ_{𝔥}(λ′) are equal if and only if λ′ = wλ for some w ∈ W (𝔤, 𝔥).

(2) Every homomorphism from ℨ(𝔤) to ℂ is of the form ξ_{𝔥}(λ) for some λ ∈ 𝔥^{∗}.

A proof of this result of Harish-Chandra may be found for example in [Hmp], section 23.

Definition 2.4. Suppose G is a reductive group in Harish-Chandra’s class. The admissible dual of G is the set Π(G) of equivalence classes of irreducible (𝔤, K)-modules. It is the same thing to consider infinitesimal equivalence classes of irreducible admissible representations of G. The unitary dual of G is the subset Π_{u}(G) of irreducible (𝔤, K)- modules admitting a positive definite invariant Hermitian form. These may be identified with unitary equivalence classes of irreducible unitary representations of G.

Suppose 𝔥 is a Cartan subalgebra of 𝔤, and λ ∈ 𝔥^{∗}. A representation of 𝔤 is said to have infinitesimal character λ if ℨ(𝔤) acts through the homomorphism ξ_{𝔥}(λ) (Definitition 2.2). Any irreducible representation of 𝔤, or any irreducible (𝔤, K)-module, has some infinitesimal character.

We now come to the central notions of real and imaginary infinitesimal character.

Definition 2.5. Suppose H ⊂ G is a θ-stable Cartan subgroup; write H = T A as in (2.1)(c). This gives a decomposition of the Lie algebra

𝔥_{0} = 𝔱_{0} + 𝔞_{0}.  (2.5)(a)

The canonical real form of 𝔥 is the subspace

RE 𝔥 = i𝔱_{0} + 𝔞_{0}.  (2.5)(b)

Obviously this subspace is in fact a real form of 𝔥. It is clear from the discussion at (2.1)(b) that the bilinear form ⟨, ⟩ is positive definite on RE 𝔥. Now 𝔥 has a decomposition into semisimple and central parts

𝔥_{0} = ([𝔤_{0}, 𝔤_{0}] ∩ 𝔥_{0}) + 𝔷(𝔤_{0}) = 𝔥_{0}^{s}(𝔤) + 𝔥_{0}^{z}(𝔤).  (2.5)(c)

This decomposition is compatible with that of (2.5)(a), so we may write

𝔥_{0}^{s}(𝔤) = 𝔱_{0}^{s}(𝔤) + 𝔞_{0}^{s}(𝔤),  𝔥_{0}^{z}(𝔤) = 𝔱_{0}^{z}(𝔤) + 𝔞_{0}^{z}(𝔤).  (2.5)(d)

(The last two summands are the Lie algebras of the groups T^{z}(G) and A^{z}(G) of (2.1)(d).) It follows that

RE 𝔥 = RE 𝔥^{s}(𝔤) + RE 𝔥^{z}(𝔤)  (2.5)(e)

Any linear functional λ ∈ 𝔥^{∗} has a unique decomposition

λ = RE λ + i IM λ,  (2.5)(f)

with RE λ and IM λ taking real values on RE 𝔥. We call these the canonical real and imaginary parts of λ.

Lemma 2.6. Suppose H and H′ are θ-stable Cartan subgroups of a real reductive group G. Use the notation of Definition 2.5.

(1) The canonical real part RE 𝔥^{s}(𝔤) of the semisimple part of 𝔥 may be characterized as the subspace on which all the roots of 𝔥 in 𝔤 take real values.

(2) Suppose g ∈ Ad(𝔤) is an inner automorphism of the complex Lie algebra carrying 𝔥 to 𝔥′. Then g carries RE 𝔥 to RE 𝔥′.

(3) The action of the Weyl group W (𝔤, 𝔥) preserves RE 𝔥.

Sketch of proof. Part (1) is standard. (The roots are differentials of characters of H, and so must take imaginary values on 𝔱_{0}. That they take real values on 𝔞_{0} can be deduced for example from the existence of a compact form of 𝔤 with Lie algebra 𝔨_{0} + i𝔰_{0}.) For (2), part (1) guarantees that g carries RE 𝔥^{s} to (RE 𝔥′)^{s}, and g acts trivially on 𝔷(𝔤). Part (3) is a special case of (2). Q.E.D.

Corollary 2.7. Suppose G is a real reductive group, and ξ: ℨ(𝔤) → ℂ is an infinitesimal character. Choose a θ-stable Cartan subgroup H of G and a weight λ ∈ 𝔥^{∗} so that ξ = ξ_{𝔥}(λ) (Definition 2.4). Then the infinitesimal characters ξ_{𝔥}(RE λ) and ξ_{𝔥}(IM λ) depend only on ξ, and not on the choices of H and λ. In particular, the statement that ξ is real (meaning λ = RE λ) is independent of choices.

We will see in section 3 that the imaginary part of the infinitesimal character of a unitary representation reveals how the representation is parabolically induced. A similar but weaker statement for the real part appears in section 5: the real part of the infinitesimal character of a unitary representation reveals how the representation is cohomologically induced.

§3. Parabolic induction and reduction to real infinitesimal character

We continue to work with a real reductive group G in Harish-Chandra’s class, using especially the notation of (2.1). Suppose now that P is a real parabolic subgroup of G (see for example [Knp], section V.5). Let N_{P} be the unipotent radical of P, a connected normal nilpotent subgroup. The intersection L_{P} = P ∩ θP is a reductive group in Harish-Chandra’s class, having the restriction of θ as a Cartan involution. There is a Levi decomposition

P = L_{P} N_{P} ,  (3.1)(a)

a semidirect product with N_{P} normal. The Levi subgroup has a direct product decomposition

L_{P} = M_{P} A_{P}  (3.1)(b)

respected by θ. The subgroup A_{P} is just A^{z}(L_{P}), the split factor of the center of L_{P}. Any irreducible admissible representation of L_{P} is of the form δ ⊗ ν, with δ ∈ Π(M_{P}) and ν ∈ Π(A_{P}) ≃ 𝔞_{P}^{∗}.

Lemma 3.2. In the setting (3.1), the functor Ind_{P}^{G} of normalized parabolic induction carries irreducible admissible representations of L_{P} to finite length admissible representations of G, preserving unitarity and infinitesimal character. More precisely, fix a θ-stable Cartan subgroup H of L_{P}; necessarily H is a product

H = (H ∩ M_{P})A_{P} = H_{1}A_{P} .

Suppose that δ ⊗ ν is an irreducible representation of L_{P}, and that δ has infinitesimal character λ ∈ 𝔥_{1}^{∗} (Definition 2.4), so that δ ⊗ ν has infinitesimal character (λ, ν) ∈ 𝔥^{∗}.

(1) If δ is unitary and ν is purely imaginary (so that it defines a unitary character of A_{P}) then Ind_{P}^{G}(δ ⊗ ν) is a unitary representation of G.

(2) The induced representation Ind_{P}^{G}(δ ⊗ ν) has infinitesimal character (λ, ν) ∈ 𝔥^{∗}.

(3) The canonical real and imaginary parts of the infinitesimal character (Corollary 2.7) are given by (RE λ, RE ν) and (IM λ, IM ν) respectively.

(4) Suppose that δ has real infinitesimal character and ν is purely imaginary. Then the canonical real and imaginary parts of the infinitesimal character are given by λ and ν respectively.

(5) Suppose that δ has real infinitesimal character and ν is purely imaginary. Assume in addition that for every non-zero weight α ∈ 𝔞_{P}^{∗} of the adjoint representation, the inner product ⟨ν, α⟩ is non-zero. Then the induced representation Ind_{P}^{G}(δ ⊗ ν) is irreducible.

Proof. Part (1) may be found for example in [Knp], section VII.2, and part (2) in [Knp], Proposition 8.22. Part (3) is immediate from (2) and Definition 2.5, and part (4) is a special case of (3). Part (5) is a fairly easy consequence of the results of [SpV], although it is not stated explicitly there. (The bilinear form appearing in (5) is the one described after (2.1)(b).) Q.E.D.

The setting of part (4) is in some sense the general explanation for the canonical real and imaginary parts of an infinitesimal character. Here is a statement.

Theorem 3.3 [Knp], Theorem 16.10. Suppose π is an irreducible unitary representation of G. Then there is a parabolic subgroup P = M_{P} A_{P} N_{P} of G, a unitary representation δ of M_{P} of real infinitesimal character, and a unitary character ν of A_{P}, so that π is equivalent to Ind_{P}^{G}(δ ⊗ ν). This may be done so that the non-degeneracy condition of Lemma 3.2(4) on ν is satisified.

There is a similar statement for Hermitian representations, but we will not need it.

Theorem 3.3 evidently reduces the study of unitary representations to the case of real infinitesimal character. We conclude this section with some remarks on the organization of that reduction.

Lemma 3.4. Suppose 𝔞_{min,0} is a maximal abelian subalgebra of 𝔰_{0} (the −1 eigenspace of θ). Define M_{min} to be the centralizer of A_{min} = exp(𝔞_{min,0}) in K, M′_{min} to be the normalizer, and W_{res} = M′_{min}/M_{min} the restricted Weyl group of A_{min} in G. Write R(𝔤, 𝔞_{min}) ⊂ 𝔞_{min,0}^{∗} for the system of restricted roots.

(1) The restricted Weyl group W_{res} is isomorphic to the Weyl group of the restricted root system R(𝔤, 𝔞_{min}).

(2) Any group A as in (2.1)(c) or A_{P} as in (3.1) is conjugate by K to a subgroup of A_{min}.

(3) Two elements of 𝔞_{min,0} are conjugate by Ad 𝔤 if and only if they are conjugate by W_{res}.

(4) Suppose ν ∈ 𝔞_{min}^{∗}. Regard 𝔞_{min}^{∗} as a subspace of 𝔤^{∗} by using 𝔪_{min} and the root spaces as a complement to 𝔞. Define

L^{ν} = {g ∈ G | Ad^{∗}(g)(ν) = ν} ⊃ M_{min}A_{min},

the stabilizer of ν in the coadjoint action. Then L^{ν} is the θ-stable Levi factor of a real parabolic subgroup of G; write L^{ν} = M^{ν}A^{ν} for its Langlands decomposition. This subgroup is characterized by its restricted root system

R(𝔩^{ν}, 𝔞_{min}) = {α ∈ R(𝔤, 𝔞_{min}) | ⟨ν, α⟩ = 0}.

The subscript min on A_{min} may be a bit confusing, since A_{min} is maximal as a subalgebra of 𝔰_{0}. It refers to a minimal parabolic subgroup of G, which may be taken to have Langlands decomposition with split component A_{min}. Alternatively, one can think of it as meaning “minimally compact.”

Proof. Parts (1) and (3) can be reformulated in a compact form of the symmetric space G/K; the references we give are for these reformulations. Part (1) is [Hel], Corollary VII.2.13. Part (2) is [Hel], Lemma V.6.3. Part (3) is [Hel], Corollary VII.8.9. Part (4) is standard and straightforward. (That L^{ν} contains M_{min}A_{min} is obvious. It follows easily that the Lie algebra 𝔩^{ν} is spanned by 𝔪_{min} + 𝔞_{min} and the indicated restricted root spaces. The characterization of L^{ν} as a real Levi subgroup can be deduced for example from the Bruhat decomposition of [Hel], Corollary IX.1.8. Q.E.D.

Corollary 3.5. Suppose as in Lemma 3.4 that 𝔞_{min,0} is a maximal abelian subalgebra of 𝔰_{0}; use the other notation defined there. Fix a maximal torus T_{min} ⊂ M_{min}, so that

𝔥_{min,0} = 𝔱_{min,0} + 𝔞_{min,0}

is a Cartan subalgebra of 𝔤_{0}.

(1) The imaginary part of the infinitesimal character of any unitary representation of G is represented by a weight ν ∈ i𝔞_{min,0}^{∗}, unique up to the action of the restricted Weyl group W_{res}. From now on we fix such a weight, and L^{ν} = M^{ν}A^{ν} as in Lemma 3.4(4).

(2) Fix a parabolic subgroup P^{ν} = L^{ν}N^{ν} with Levi factor L^{ν}. Then unitary induction from P^{ν} provides a bijection from irreducible unitary representations of M^{ν} with real infinitesimal character onto irreducible unitary representations of G whose infinitesimal character has canonical imaginary part ν. Explicitly, this bijection carries δ to Ind_{P^{ν}}^{G} (δ ⊗ ν).

This result is more or less immediate from Theorem 3.3 and Lemma 3.4. It is possible to avoid the restricted Weyl group ambiguity in the choice of ν by first choosing a positive system R^{+}(𝔤, 𝔞_{min}) of restricted roots; equivalently, a minimal parabolic subgroup

P_{min} = M_{min}A_{min}N_{min}.

We can then make ν unique by requiring it to be dominant for R^{+}. The parabolic subgroup P^{ν} may then be taken to be one of the 2^{ℓ} standard parabolics containing P_{min}; here ℓ is the rank of the restricted root system. The possible ν attached to each such parabolic constitute the corresponding face of the closed positive Weyl chamber in i𝔞_{min}^{∗}.

§4. Cohomological induction

The central idea of section 3 was that canonical imaginary parts of infinitesimal characters of unitary representations arise from parabolic induction. More precisely, one can begin with any unitary character of the split part of a Cartan subgroup (the weight ν in Theorem 3.3 or Corollary 3.5) and use it construct a parabolic; then unitary representations with imaginary part ν arise nicely by induction from that parabolic.

In this section we will lay the foundations for a similar analysis of the canonical real part of an infinitesimal character. By analogy with Corollary 3.5, one might hope that such real parts arise from a character of the compact part of a Cartan subgroup (compare Definition 2.5). It is therefore natural to begin by choosing a maximal torus

T_{max,0} ⊂ K  (4.1)(a)

This torus will play a rôle analogous to that of A_{min} in section 3. Although we will be concerned mostly with questions on the Lie algebra, we may as well record the definition

T_{max} = Z_{K}(T_{max,0}).  (4.1)(b)

The centralizer of T_{max,0} in G is a Cartan subgroup

H_{max} = T_{max}A_{max} ⊂ G,  (4.1)(c)

a fundamental Cartan subgroup of G. We will need the Weyl groups

W (K, T_{max}) = N_{K}(T_{max})/T_{max},  (4.1)(d)
W (G, H_{max}) = N_{G}(H_{max})/H_{max}.  (4.1)(e)

We call these the Weyl group of T_{max} in K and the Weyl group of H_{max} in G. Here is the analogue of Lemma 3.4.

Lemma 4.2. Write R(𝔤, 𝔱_{max}) ⊂ 𝔱_{max}^{∗} for the set of non-zero weights of 𝔱_{max} on 𝔤.

(1) The set R(𝔤, 𝔱_{max}) is a root system. The corresponding Weyl group W (𝔤, 𝔱_{max}) may be identified with the restrictions to 𝔱_{max} of elements of W (𝔤, 𝔥_{max}) commuting with the Cartan involution θ. That is,

W (𝔤, 𝔱_{max}) ≃ W (𝔤, 𝔥_{max})^{θ},

with the isomorphism arising by restriction of linear transformations from 𝔥_{max} to 𝔱_{max}.

(2) The Weyl groups W (G, H_{max}) and W (K, T_{max}) are isomorphic, the isomorphism arising by restriction of group automorphisms from H_{max} to T_{max}.

(3) Regard W (K, T_{max}) as acting on 𝔱_{max} by differentiation. Then we have inclusions

W (𝔨, 𝔱_{max}) ⊂ W (K, T_{max}) ⊂ W (𝔤, 𝔱_{max}).

If G (or equivalently K) is connected, then the first inclusion is an equality.

(4) Suppose T is one of the groups in (2.1)(c). Then T_{0} is conjugate by K to a subgroup of T_{max,0}.

(5) Two elements of 𝔱_{max} are conjugate by Ad 𝔤 if and only if they are conjugate by W (𝔤, 𝔱_{max}).

Notice that the statement here is significantly weaker than in Lemma 3.4, because of the failure of the second inclusion in (3) to be an equality. The weights in i𝔱_{max}^{∗} are going to represent real parts of infinitesimal characters (at least approximately); but each such infinitesimal character will correspond to several different W (K, T_{max}) orbits of weights.

Proof. These facts are all fairly standard; here are some references. Part (1) may be found in [IC4], Proposition 3.12 (particularly the last isomorphism of part (c) there). For part (2), the main point is to show that every element x of N_{G}(H_{max}) has a representative (modulo A_{max}) belonging to K. To see this, consider the Cartan decomposition (2.1)(b) x = k exp(Z), with Z ∈ 𝔰_{0}. The fact that x normalizes T_{max,0} implies that for every t ∈ T_{max,0} there is a t′ ∈ T_{max,0} so that

k exp(Z)t = t′k exp(Z).

That is,

kt exp(Ad(t^{−1}Z) = t′k exp(Z).

Now the uniqueness of the Cartan decomposition implies that Ad(t) fixes Z; that is, that Z ∈ 𝔞_{max,0}, as we wished to show. For part (3), the inclusions are clear. That the first is an equality when K is connected is standard (see for example [Hel], pages 256–7). Part (4) is just the conjugacy of maximal tori in a compact Lie group. Part (5) is analogous to Lemma 3.4(3), and can in fact be reduced to it. (For this one needs to construct an involutive automorphism of 𝔤 whose restriction to 𝔥_{max} is −θ.) We omit the details. Q.E.D.

Definition 4.3. Fix a maximal torus T_{max,0} in K as in (4.1), and a weight λ ∈ i𝔱_{max,0}^{∗}. Identify 𝔱_{max}^{∗} with a subspace of 𝔤^{∗} by using 𝔞_{max} and the restricted weight spaces as a complement for 𝔱_{max}. The θ-stable Levi subgroup associated to λ is then the stabilizer of λ in the coadjoint action:

L^{λ} = {g ∈ G | Ad^{∗}(g)(λ) = λ} ⊃ H_{max}.  (4.3)(a)

This is a θ-stable reductive subgroup of G in Harish-Chandra’s class. It will play the same rôle for cohomological induction as the Levi subgroup L_{P} did for parabolic induction. In place of the split component A_{P}, we will use

T^{z}(L^{λ}) = Z(L^{λ}) ∩ K,  (4.3)(b)

(cf. (2.1)(d)), a subgroup of T_{max}. For generic λ, L^{λ} = H_{max}, and T_{0}^{z}(L^{λ}) = T_{max,0}.

The restricted root system for L^{λ} is

R(𝔩^{λ}, 𝔱_{max}) = {α ∈ R(𝔤, 𝔱_{max}) | ⟨α, λ⟩ = 0}.  (4.3)(c)

As usual, the bilinear form is the one introduced after (2.1)(b). If we use this form to identify λ with an element X_{λ} ∈ i𝔱_{max,0}, then these roots are just the ones vanishing on X_{λ}.

We will also need a θ-stable nilpotent subalgebra 𝔲^{λ}, characterized by

R(𝔲^{λ}, 𝔱_{max}) = {α ∈ R(𝔤, 𝔱_{max}) | ⟨α, λ⟩ > 0}.  (4.3)(d)

In terms of a corresponding Lie algebra element X_{λ} as above, 𝔲^{λ} is the sum of the positive eigenspaces of ad(X_{λ}). The θ-stable parabolic subalgebra associated to λ is by definition

𝔮^{λ} = 𝔩^{λ} + 𝔲^{λ};  (4.3)(e)

this is a parabolic subalgebra of 𝔤. Because the weights of 𝔱_{max,0} take purely imaginary values, complex conjugation with respect to the real form 𝔤_{0} carries 𝔮^{λ} to an opposite parabolic subalgebra; we have a triangular decomposition

𝔤 = \overline{𝔲}^{λ} + 𝔩^{λ} + 𝔲^{λ}.  (4.3)(f)

The next result establishes for cohomological induction some of what was done in Lemma 3.2 for parabolic induction.

Lemma 4.4. Suppose we are in the setting of Definition 4.3; use the notation there.

(1) The group L^{λ} is precisely the normalizer in G of 𝔮^{λ}. In particular, Ad(L^{λ}) preserves 𝔲^{λ}.

(2) There is a G-invariant complex structure on G/L^{λ}, having 𝔮^{λ}/𝔩^{λ} as holomorphic tangent space at the identity coset. We may identify G/L^{λ} as an open G-orbit on the projective variety of parabolic subalgebras of 𝔤 conjugate to 𝔮^{λ}. This identification sends gL^{λ} to the parabolic Ad(g)(𝔮^{λ}).

(3) Write 2ρ(𝔲^{λ}) for the one-dimensional character of L^{λ} on the top exterior power of 𝔲^{λ}. Suppose H ⊂ L^{λ} is a θ-stable Cartan subgroup; write

𝔥 = 𝔱^{z}(𝔩^{λ}) + (𝔞^{z}(𝔩^{λ}) + 𝔥^{s}(𝔩^{λ}))  (4.4)(a)

as in Definition 2.5. Then the differential of 2ρ(𝔲) vanishes on 𝔞^{z}(𝔩^{λ}) + 𝔥^{s}(𝔩^{λ}). It is therefore natural to write also 2ρ(𝔲^{λ}) for the restriction to 𝔱^{z}(𝔩^{λ}) of the differential of this character.

(4) Suppose Z is any irreducible (𝔩^{λ}, L^{λ} ∩ K)-module. Then the compact center T^{z}(L^{λ}) acts on Z by a unitary character Λ′, having differential λ′ ∈ i(𝔱_{0}^{z}(𝔩^{λ}))^{∗}. For H as in (3), suppose that the infinitesimal character of Z is given by a weight γ′ ∈ 𝔥^{∗}. Then with respect to the decomposition (4.4)(a) of 𝔥, we have

γ′ = (λ′, ν′),  ν′ ∈ (𝔞^{z}(𝔩^{λ}) + 𝔥^{s}(𝔩^{λ}))^{∗}  (4.4)(b)

(5) With notation as in (4), write Z^{#} for the tensor product of Z with the top exterior power of 𝔲^{λ}. Then Z^{#} has infinitesimal character γ′ + 2ρ(𝔲^{λ}). The induced (𝔤, L^{λ} ∩ K)-module

W = ind_{\overline{𝔮}^{λ}}^{𝔤}(Z^{#}) = U(𝔤) ⊗\overline{𝔮}_{λ} Z^{#}  (4.4)(c)

has infinitesimal character

γ′ + ρ(𝔲^{λ}) = (λ′ + ρ(𝔲^{λ}), ν′) ∈ 𝔥^{∗}.  (4.4)(d)

The same is true of the produced module

W′ = pro_{𝔮^{λ}}^{𝔤}(Z^{#}).

The real and imaginary parts of this infinitesimal character are

RE(γ′ + ρ(𝔲^{λ})) = (λ′ + ρ(𝔲^{λ}), RE ν′), IM(γ′ + ρ(𝔲^{λ})) = (0, IM ν′). (4.4)(e)

A definition of the induced and produced modules may be found in [Grn], Chapter 6, or in [KV], page 105.

Proof. For (1), that L^{λ} normalizes 𝔮^{λ} follows easily from Definition 4.3. Write N^{λ} ⊃ L^{λ} for the full normalizer. Since a parabolic subalgebra of a reductive Lie algebra is self-normalizing, we must have 𝔫^{λ} ⊂ 𝔮^{λ} ∩ \overline{𝔮}^{λ} = 𝔩^{λ}. From this it follows that N^{λ} is generated by L^{λ} and representatives for the normalizer of 𝔮^{λ} in the Weyl group W (K, T_{max}) of (4.1)(d). It is a straightforward exercise to show that this normalizer is precisely W (L^{λ}, T_{max}).

For (2), the existence of the complex structure follows from (1) and general principles (see for example [Orng], Proposition 1.19). The indicated map evidently embeds G/L^{λ} in the variety of parabolics; that the image is open follows from the triangular decomposition (4.3)(f).

Parts (3) and (4) are elementary, as is the first formula in (5). The infinitesimal characters of the induced and produced modules may be found in [KV], Theorem 5.24. The assertion about real and imaginary parts is then elementary. Q.E.D.

We do not propose to attempt here a detailed explanation of cohomological induction. For this we refer to the introduction of [KV]. The idea is that a representation Z of L^{λ} gives rise to an (infinitedimensional) holomorphic vector bundle V(Z) on G/L^{λ}, having Z^{#} as its fiber over the identity coset. The produced module W′ of Lemma 4.4(5) may be identified with certain formal power series sections of V(Z) at the identity coset. Cohomological induction provides a family of representations of G (more precisely, a family of (𝔤, K)-modules) that are formally analogous to the Dolbeault cohomology of G/L^{λ} with coefficients in V(Z). Explicitly,

(L_{\overline{𝔮}^{λ},L^{λ}∩K}^{𝔤,K})_{j} (Z) = (Π_{𝔤,L^{λ}∩K}^{𝔤,K})_{j} (ind_{\overline{𝔮}^{λ}}^{𝔤}(Z^{#})) .  (4.5)

Here the functors L_{j} on the left carry (𝔩^{λ}, L^{λ} ∩ K) modules of finite length to (𝔤, K)-modules of finite length. They are defined by first twisting Z by the one-dimensional character 2ρ(𝔲^{λ}); extending to 𝔮^{λ} (by 0 on \overline{𝔲}^{λ}); inducing to 𝔤; and finally applying a Bernstein-Zuckerman derived functor to get a K-finite representation. This last step does not affect infinitesimal character, so (with notation as in Lemma 4.4) we see that the cohomological induction functors add ρ(𝔲^{λ}) to the infinitesimal character.

This “ρ-shift” is annoying after the simple situation for parabolic induction recorded in Lemma 3.2(2). The reason for the difference is that parabolic induction involves a shift by a square root of the character 2ρ(𝔫_{P}) (of L_{P} on the top exterior power of 𝔫_{P}. (More precisely, one uses a positive character of L_{P} that provides a square root of 2ρ(𝔫_{P}) on the identity component of L_{P}.) In the case of the θ-stable parabolic 𝔮^{λ}, the character 2ρ(𝔲^{λ}) need not admit a square root; we used a shift by the character itself rather than a square root, leading inevitably to the shift in the infinitesimal character formula. It is possible to avoid this shift at the expense of introducing a double cover of L^{λ}. For a description of this we refer to [Orng], chapter 6.

The following theorem complements Lemma 4.4, providing a fairly complete analogue of Lemma 3.2 for cohomological induction.

Theorem 4.6. Suppose we are in the setting of Definition 4.3, so that 𝔮^{λ} = 𝔩^{λ} + 𝔲^{λ} is a θ-stable parabolic subalgebra of G. Then the functors L_{j} of cohomological induction (see (4.5)) carry (𝔩^{λ}, L^{λ} ∩ K) modules of finite length to (𝔤, K)-modules of finite length. They are zero unless 0 ≤ j ≤ S = dim(𝔲^{λ} ∩ 𝔨).

Suppose now that Z is an irreducible (𝔩^{λ}, L^{λ} ∩ K)-module of infinitesimal character γ′ = (λ′, ν′) (notation as in Lemma 4.4(4)).

(1) The cohomologically induced representations L_{j}(Z) have infinitesimal character γ′ + ρ(𝔲^{λ}).

(2) Assume that the infinitesimal character satisfies the positivity condition

Re⟨γ′ + ρ(𝔲^{λ}), α⟩ ≥ 0,  (α ∈ Δ(𝔲^{λ}, 𝔥)).  (4.6)

Then L_{j}(Z) = 0 for j < S. The module L_{S}(Z) is irreducible or zero; it is unitary if Z is.

(3) Assume that the inequalities in (4.6) are all strict. Then L_{S}(Z) is non-zero (and irreducible). It is unitary if and only Z is unitary.

Proof. The assertions through (1) are fairly easy consequences of the definitions. The vanishing of L_{j} for j > S is [KV], Theorem 5.35. The calculation of infinitesimal character, already mentioned after (4.5), follows from Lemma 4.4 and [KV], Theorem 5.21. The irreducibility in (2) and (3) is [KV], Corollary 8.28. The unitarity for L_{S}(Z) is [KV], Theorem 9.1. The converse assertion in (3) (deducing unitarity for Z from L_{S}(Z) is [VUn], Theorem 1.3(b). Q.E.D.

Cohomological induction in certain respects requires more care than parabolic induction, as a tool for constructing unitary representations. Lemma 3.2 allowed us to begin with any unitary representation of L_{P}; Theorem 4.6 requires a rather restrictive positivity hypothesis on the representation Z of L^{λ}. We will conclude this section with a discussion of a simple but important way that this positivity condition can be fulfilled. We will also take the opportunity to recast the notation without the weight λ, which we used for the original construction of a θ-stable parabolic but not subsequently.

Suppose therefore that

𝔮 = 𝔩 + 𝔲  (4.7)(a)

is a Levi decomposition of a θ-stable parabolic subalgebra of 𝔤. Write

L = normalizer in G of 𝔮  (4.7)(b)

for the corresponding Levi subgroup, and H ⊂ L for a θ-stable Cartan subgroup. Recall from (2.1)(d) the compact and split components T^{z}(L) and A^{z}(L) of the center of L, and then from Definition 2.5 the direct sum decomposition

𝔥 = 𝔱^{z}(𝔩) + (𝔞^{z}(𝔩) + 𝔥^{s}(𝔩)) .  (4.7)(c)

Now let Z be an irreducible (𝔩, L ∩ K)-module of infinitesimal character

γ = (λ, ν) = (λ, ν^{z}, ν^{s}) ∈ 𝔥^{∗};  (4.7)(d)

here we use the decomposition (4.7)(c) of 𝔥. According to Lemma 4.4(4),

λ ∈ i𝔱_{0}^{z}(𝔩)^{∗}  (4.7)(e)

is the differential of the (necessarily unitary) character by which T^{z}(L) acts on Z. In the same way, we see that ν^{z} ∈ (𝔞^{z}(𝔩))^{∗} is the differential of the character by which A^{z}(L) acts on Z. In particular,

ν^{z} ∈ i𝔞_{0}^{z}(𝔩)^{∗}  (Z unitary).  (4.7)(f)

Finally, ν^{s} is just the infinitesimal character of Z restricted to the semisimple derived group of L. Just as in Lemma 4.4, we can decompose the infinitesimal character γ + ρ(𝔲) into its canonical real and imaginary parts; these are

RE(γ + ρ(𝔲)) = (λ + ρ(𝔲), 0, RE ν^{s}), IM(γ + ρ(𝔲)) = (0, −iν^{z}, IM ν^{s})

(4.7)(g)

(still assuming that Z is unitary).

Here is the special setting for the positivity condition in Theorem 4.6.

Proposition 4.8. Suppose 𝔮 = 𝔩 + 𝔲 is a θ-stable parabolic subalgebra with Levi subgroup L. Let Z be an irreducible unitary (𝔩, L ∩ K)- module of infinitesimal character γ = (λ, ν) (notation as in (4.7)). Assume that

(a) every weight α of 𝔱^{z}(𝔩) in 𝔲 is non-negative on λ: ⟨λ, α⟩ ≥ 0; and

(b) the weight RE ν belongs to the convex hull of the W (𝔩, 𝔥) orbit of ρ(𝔩), half the sum of a set of positive roots for 𝔥 in 𝔩.

Then the strict positivity hypothesis (4.6) is satisfied:

Re⟨γ + ρ(𝔲), β⟩ > 0, (β ∈ Δ(𝔲, 𝔥)).

Consequently L_{S}(Z) is an irreducible unitary (𝔤, K)-module.

Proof. The real part of an inner product with a root is the same as the inner product of the canonical real part with the root; so what we are trying to prove is that

⟨(λ + ρ(𝔲), RE ν), β⟩ > 0,  (β ∈ Δ(𝔲, 𝔥)).  (4.9)(a)

By hypothesis RE ν belongs to the convex hull of various half sums of positive roots; so it suffices to prove the inequality with RE ν replaced by one of these half sums. That is, it suffices to fix a set of positive roots Δ^{+}(𝔩, 𝔥), to write ρ(𝔩) for half the sum of these roots, and prove

⟨(λ + ρ(𝔲), ρ(𝔩)), β⟩ > 0,  (β ∈ Δ(𝔲, 𝔥)).  (4.9)(b)

Because 𝔮 is a parabolic subalgebra, the roots

Δ^{+}(𝔤, 𝔥) = Δ(𝔲, 𝔥) ∪ Δ^{+}(𝔩, 𝔥)  (4.9)(c)

constitute a positive system for 𝔥 in 𝔤. The corresponding half sum of positive roots is

ρ = (ρ(𝔲), ρ(𝔩)).  (4.9)(d)

Combining (4.9)(b) and (4.9)(c), we see that we need to prove

⟨λ + ρ, β⟩ > 0,  (β ∈ Δ(𝔲, 𝔥)).  (4.9)(e)

In this last inner product, the term ⟨λ, β⟩ is non-negative by hypothesis (1) on Z. The term ⟨ρ, β⟩ is strictly positive since β is a positive root by the construction of Δ^{+}. So the sum of these two terms is strictly positive, as we wished to show. That L_{S}(Z) is irreducible unitary follows from Theorem 4.6. Q.E.D.

§5. Reduction to unitarily small representations

Proposition 4.8 allows us to construct unitary representations of G having certain kinds of real infinitesimal character from unitary representations of θ-stable Levi subgroups. This is more or less parallel to the construction of representations with partly imaginary infinitesimal character in Lemma 3.2. In the earlier setting we had a converse as well: Theorem 3.3 guaranteed that any unitary representation could be constructed. The point of [SV] is to try to prove a parallel result for cohomological induction. To formulate what is expected, we need a little preliminary geometry.

Definition 5.1. Suppose H is a θ-stable Cartan subgroup of the reductive group G, and V = RE 𝔥^{∗} is the dual of the canonical real part. Write Δ = Δ(𝔤, 𝔥) ⊂ V for the roots of 𝔤, and W = W (𝔤, 𝔥) ⊂ Aut V for the Weyl group. To each system of positive roots Δ^{+} ⊂ Δ we attach the closed positive Weyl chamber

C = C(Δ^{+}) = {v ∈ V | ⟨v, α⟩ ≥ 0 (α ∈ Δ^{+})}.  (5.1)(a)

Define P_{C} to be the projection from V onto C: P_{C}(w) is by definition the element of C closest to w.

Suppose γ ∈ V is an arbitrary element. We define a “truncation operator” T_{γ}: V → V as follows. Given v ∈ V, choose a positive root system Δ^{+} making v dominant; that is, so that v ∈ C(Δ^{+}). Next, let γ′ be the unique dominant weight conjugate to γ: that is, γ′ ∈ W · γ ∩ C(Δ^{+}). Finally, define

T_{γ}(v) = P_{C}(v − γ′).  (5.1)(b)

Because the positive root system Δ^{+} is not uniquely specified by v, the first issue is to show that T_{γ}(v) does not depend on its choice. This fact and some other basic properties of T_{γ} are recorded in the next proposition.

Proposition 5.2. In the setting of Definition 5.1, the truncation operator T_{γ} is well-defined. It depends only on the Weyl group orbit W · γ, and has the following additional properties.

(1) The operator T_{γ} is continuous and in fact a contraction: if v, w ∈ V, then

| T_{γ}(v) − T_{γ}(w) | ≤ | v − w | .

(2) The operator T_{γ} commutes with the action of W: T_{γ}(w · v) = w · T_{γ}(v).

(3) The inverse image of 0 under T_{γ} is the convex hull of the Weyl group orbit of γ. More generally, suppose v_{0} ∈ V, and let W_{0} be the stabilizer of v_{0} in W. Fix γ_{0} ∈ W · γ belonging to a common positive Weyl chamber with v_{0}. Then

T_{γ}^{−1}(v_{0}) = v_{0} + (convex hull of W_{0} · γ_{0}).

Proof. These facts are more or less elementary; proofs of most of them may be found in section 1 of [SV]. The idea of applying them to representation theory is taken from [Car]. Q.E.D.

Corollary 5.3. Suppose G is a real reductive group, H is a θstable Cartan subgroup of G, and ξ: ℨ(𝔤) → ℂ is an infinitesimal character. Choose a weight ϕ ∈ 𝔥^{∗} defining the infinitesimal character (Definition 2.4), and use the notation of Definition 5.1, so that RE λ ∈ V. Write ρ ∈ V for half the sum of a set of positive roots. Then the Weyl group orbit of T_{ρ}(RE ϕ) depends only on the infinitesimal character ξ.

Corollary 5.4. Suppose we are in the setting of Proposition 4.8, so that L_{S}(Z) has infinitesimal character γ + ρ(𝔲). Then T_{ρ}(RE(γ + ρ(𝔲)) = λ, the weight by which the compact center 𝔱^{z}(𝔩) acts on Z.

Proof. Choose a positive root system Δ^{+}(𝔤, 𝔥) as in (4.9)(c). By hypothesis (1) of Proposition 4.8, λ is non-negative on the roots of 𝔲.

Because 𝔱^{z}(𝔩) is central in 𝔩, λ is perpendicular to the roots of 𝔩. Therefore λ is dominant for Δ^{+}, and fixed by the Weyl group W (𝔩, 𝔥). According to Proposition 5.2(3), the preimage under T_{ρ} of λ includes

λ + (convex hull of W (𝔩, 𝔥) · ρ).

Using the decomposition (4.9)(d), we can rewrite this as

λ + ρ(𝔲) + (convex hull of W (𝔩, 𝔥) · ρ(𝔩)).

According to hypothesis (2) of Proposition 4.8, this convex hull contains RE ν; so the real part RE((λ, ν) + ρ(𝔲)) of the infinitesimal character belongs to T_{ρ}^{−1}(λ), as we wished to show. Q.E.D.

Recall that we are seeking an analogue of Theorem 3.3 for cohomological induction. Roughly speaking, it should say that any unitary representation for which the real part of the infinitesimal character truncates (by T_{ρ}) to a weight λ, ought to be cohomologically induced from a Levi subgroup defined by λ. In order to make sense of such a statement, we need to get the real part of the infinitesimal character (at least after truncation) into i𝔱_{max,0}^{∗}. This at least is possible.

Lemma 5.5. Suppose H_{max} is a maximally compact Cartan subgroup of G as in (4.1); use the notation there. Then the canonical real part of the infinitesimal character of any unitary representation of G is represented by a weight in i𝔱_{max,0}^{∗}, unique up to the action of W (𝔤, 𝔱_{max}) (Lemma 4.2). Consequently the same is true of its truncation by T_{ρ}.

Sketch of proof. That the representative (if it exists) has the desired uniqueness property follows from Lemma 4.2(5). That truncation by ρ preserves i𝔱_{max,0}^{∗} follows fairly easily from the definition of T_{ρ}. The main point is therefore the existence of a representative. This we can prove not just for unitary representations, but for arbitrary Hermitian representations. The advantage of the generalization is that Hermitian representations behave very simply in the classification by lowest Ktypes described in [Grn] (see for example [SV], Theorem 2.9). Using this reduction, Lemma 5.5 may be reduced to the following fact.

Lemma 5.6. Suppose G is a quasisplit real reductive group in Harish-Chandra’s class, and H_{max} and H_{min} are maximally and minimally compact Cartan subgroups; use the notation of Corollary 3.5 and (4.1) accordingly. Suppose ν ∈ 𝔞_{max,0}^{∗}, w ∈ W_{res}, and wν = −ν. Then ν is conjugate by Ad 𝔤 to an element of i𝔱_{max,0}^{∗}.

For this fact we omit the elementary (although not trivial) proof. Here is a suggestive special case. Suppose G is complex. In this case we can choose H_{max} = H_{min} = H = T A, and the real Cartan subalgebra 𝔥_{0} is actually complex. Its complexification 𝔥 is isomorphic to a sum of two copies of 𝔥_{0} (corresponding to the +i and −i eigenspaces of the complex structure from G). The canonical real part respects this decomposition; it is the product of two copies of 𝔞_{0}.

𝔥 = 𝔥_{0}^{L} + 𝔥_{0}^{R},  RE 𝔥 = 𝔞_{0}^{L} + 𝔞_{0}^{R}.  (5.7)(a)

On the other hand, we have H = T A, so 𝔥 = 𝔱 + 𝔞. The identifications in (5.7)(a) may be arranged so that

𝔞 = {(X, X) | X ∈ 𝔥_{0}}, 𝔱 = {(X, −X) | X ∈ 𝔥_{0}}  (5.7)(b)

. Finally, the Weyl group W (𝔤, 𝔥) is just a product of two copies of the Weyl group W_{0} of 𝔥_{0} in 𝔤_{0}, and W_{res} ≃ W_{0} is the diagonal subgroup. The weights ν as in Lemma 5.6 are precisely

{ν = (ν_{0}, ν_{0}) | ν_{0} ∈ 𝔞_{0}, wν_{0} = −ν_{0} (some w ∈ W_{0})}.  (5.7)(c)

Such a weight ν is obviously conjugate (by (1, w)) to the weight (ν_{0}, −ν_{0}) in i𝔱_{0}^{∗}. This proves Lemma 5.6 for complex groups.

We omit the remaining details in the proof of Lemma 5.5. Q.E.D.

Here at last is a (partly conjectural) converse to Proposition 4.8.

Theorem 5.8 [SV], Theorem 5.8. Suppose G is a real reductive group in Harish-Chandra’s class, and that Conjecture 0.6 of [SV] is true. Suppose X is an irreducible unitary (𝔤, K)-module. Then there is a θstable parabolic subalgebra 𝔮 = 𝔩+𝔲 of 𝔤, with Levi subgroup L (Definition 4.3), and an irreducible unitary (𝔩, L ∩ K)-module Z of infinitesimal character γ = (λ, ν) (cf. (4.7)), with the following properties.

(1) Every weight α of 𝔱^{z}(𝔩) in 𝔲 is positive on λ: ⟨λ, α⟩ > 0.

(2) The weight RE ν belongs to the convex hull of the W (𝔩, 𝔥) orbit of ρ(𝔩), half the sum of a set of positive roots for 𝔥 in 𝔩.

(3) The (𝔤, K)-module X is isomorphic to L_{S}(Z).

Just as we unwound Theorem 3.3 into Corollary 3.5, we can unwind Theorem 5.8 into the following statement.

Corollary 5.9. Suppose H_{max} is a maximally compact Cartan subgroup of G as in (4.1); use the notation there. Fix a weight λ ∈ i𝔱_{max,0}^{∗}, and representatives λ_{1}, …, λ_{r} for the orbits of W (K, T_{max}) on W (𝔤, 𝔱_{max}) · λ. To each λ_{i} associate a θ-stable parabolic subalgebra

𝔮_{i} = 𝔮^{λ_{i}}

as in Definition 4.3, with Levi subgroup

L_{i} = L^{λ_{i}}.

For each i, define Π_{u}^{λ_{i}}(L_{i}) to be the set of irreducible unitary (𝔩_{i}, L_{i} ∩K)- modules Z having the following two properties.

(a) The compact center 𝔱^{z}(𝔩_{i}) acts on Z by the weight λ_{i}.

(b) Write the infinitesimal character of Z as (λ_{i}, ν) in accordance with (4.7). Then RE ν belongs to the convex hull of W (𝔩_{i}, 𝔥) · ρ(𝔩_{i}).

Then

(1) The cohomological induction functor L_{S_{i}} is a bijection from Π_{u}^{λ_{i}}(L_{i}) to a set Π_{u}^{λ_{i}}(G) of irreducible unitary representations of G.

(2) As i varies, the sets Π_{u}^{λ_{i}}(G) are disjoint.

(3) Suppose X is an irreducible unitary representation of G of infinitesimal character ϕ. Then X belongs one of the sets Π_{u}^{λ_{i}}(G) if and only if T_{ρ}(RE ϕ) is conjugate by Ad 𝔤 to λ.

Actually conclusions (1) and (2) do not depend on the conjecture of [SV]. Part (3) shows how to recover the W (𝔤, 𝔱_{max}) orbit of λ_{i} from (the infinitesimal character of) any representation X in Π_{u}^{λ_{i}}(G). The formulation of (2) suggests that we should actually be able to recover λ_{i} (or at least its W (K, T_{max}) orbit) from X. This is indeed possible, and is one of the main points of [SV] (see Definition 0.2 there).

§6. Reduction to general Levi subgroups

By combining Corollary 3.5 and Corollary 5.9, we can get a description of all unitary representations of G in terms of unitarily small representations of Levi subgroups. In this section we will make such a reduction explicit.

Definition 6.1. A Levi subgroup of G is one of the form

L = centralizer in G of 𝔠_{0},

with 𝔠_{0} a subspace of a θ-stable Cartan subalgebra 𝔥 of 𝔤. (We do not require 𝔠_{0} to be θ-stable.) This is a reductive group in Harish-Chandra’s class, with Cartan involution θ. Decompose the Cartan subalgebra 𝔥_{0} as

𝔥_{0} = 𝔥_{0}^{s}(𝔩) + 𝔱_{0}^{z}(𝔩) + 𝔞_{0}^{z}(𝔩)

as in Definition 2.5. Suppose Z is an irreducible unitary representation of L, of infinitesimal character

ϕ = (ϕ^{s}, ϕ^{z}) = (ϕ^{s}, ϕ^{z,c}, ϕ^{z,n}) ∈ 𝔥^{s}(𝔩)^{∗} + 𝔱^{z}(𝔩)^{∗} + 𝔞^{z}(𝔩)^{∗}

(Here the additional superscripts on ϕ^{z} stand for “compact” and “noncompact.”) Necessarily

ϕ^{z,c} ∈ i𝔱_{0}^{z}(𝔩)^{∗} ⊂ RE 𝔥^{∗},

and

ϕ^{z,n} ∈ i𝔞_{0}^{z}(𝔩)^{∗} ⊂ IM 𝔥^{∗}.

We say that Z is unitarily small if

ϕ^{s} ∈ convex hull of W (𝔩, 𝔥) · ρ ⊂ RE 𝔥.

This is consistent with the definition made in Theorem 1.1 in the semisimple case. We write Π_{u,small}^{ϕ^{z}}(L) for the set of unitarily small representations of L with the indicated action of the center of 𝔩.

Finally, we say that ϕ^{z} is maximally regular if it is orthogonal only to the roots of 𝔥 in 𝔩. That is, we require

⟨ϕ^{z}, α⟩ ≠ 0

for every non-zero weight of 𝔥^{z}(𝔩) in 𝔤. Suppose ϕ^{z} is maximally regular. Then the centralizer L(ϕ^{z,n}) is a Levi factor of a real parabolic subgroup P (ϕ^{z,n}) in G. The weight ϕ^{z,c} defines a θ-stable parabolic subalgebra

𝔮(ϕ^{z,c}, ϕ^{z,n}) = 𝔩(ϕ^{z,c}, ϕ^{z,n}) + 𝔲(ϕ^{z,c}, ϕ^{z,n})

for L(ϕ^{z,n}) as in Definition 4.3; its Levi subgroup L(ϕ^{z,c}, ϕ^{z,n}) is just the original L.

Theorem 6.2. Suppose L is a Levi subgroup of G, and ϕ^{z} ∈ 𝔥^{z}(𝔩)^{∗} is a maximally regular weight (Definition 6.1). Define P (ϕ^{z,n}) and 𝔮(ϕ^{z,c}, ϕ^{z,n}) as in Definition 6.1.

(1) Cohomological induction from L to L(ϕ^{z,n}), followed by parabolic induction from L(ϕ^{z,n}) to G, carries the unitarily small representations in Π_{u,small}^{ϕ^{z}}(L) bijectively onto a set Π_{u}^{ϕ^{z}}(G) of irreducible unitary representations of G.

(2) Two such sets of unitary representations of G can overlap if and only if the corresponding pairs (L, ϕ^{z}) are conjugate by K, in which case they coincide.

(3) Suppose that Conjecture 0.6 of [SV] holds for G. Then any unitary representation of G belongs to one of the sets Π_{u}^{ϕ^{z}}(G).

This is a combination of Corollary 3.5 and Corollary 5.9. It provides a rather precise form of the claim in Theorem 1.1(1).

References

[Car] J. Carmona, Sur la classification des modules admissibles irréductibles, in “Non-commutative Harmonic Analysis and Lie Groups”, (J. Carmona and M. Vergne, eds.), Lecture Notes in Mathematics, Springer-Verlag, Berlin-Heidelberg-New York-Tokyo, 1983.

[HC] Harish-Chandra, Harmonic analysis on reductive groups I. The theory of the constant term, J. Func. Anal., 19 (1975), 104–204.

[Hel] S. Helgason, “Differential Geometry, Lie Groups, and Symmetric Spaces”, Academic Press, New York, San Francisco, London, 1978.

[Hmp] J.E. Humphreys, “Introduction to Lie Algebras and Representation Theory”, Springer-Verlag, Berlin-Heidelberg-New York, 1972.

[Knp] A. Knapp, “Representation Theory of Real Semisimple Groups: an Overview Based on Examples”, Princeton University Press, Princeton, New Jersey, 1986.

[Sal] S. Salamanca-Riba, On the unitary dual of real semisimple Lie groups and the A_{𝔮}(λ) modules. The regular case, to appear in Duke Math. J.

[SV] S. Salamanca-Riba and D. Vogan, On the classification of unitary representations of reductive Lie groups, to appear in Ann. of Math.

[SpV] B. Speh and D. Vogan, Reducibility of generalized principal series representations, Acta Math., 145 (1980), 227–299.

[Grn] D. Vogan, “Representations of Real Reductive Lie Groups”, Birkhäuser, Boston-Basel-Stuttgart, 1981.

[IC4] D. Vogan, Irreducible characters of semisimple Lie groups IV. Character-multiplicity duality, Duke Math. J., 49 (1982), 943–1073.

[VUn] D. Vogan, Unitarizability of certain series of representations, Ann. of Math., 120 (1984), 141–187.

[Orng] D. Vogan, “Unitary Representations of Reductive Lie Groups”, Annals of Mathematics Studies, Princeton University Press, Princeton, New Jersey, 1987.

[Vorb] D. Vogan, Unitary representations of reductive Lie groups and the orbit method, in “New Developments in Lie Theory and Their Applications”, (J. Tirao and N. Wallach, eds.), Birkhäuser, Boston-Basel-Berlin, pp. 87–114.

[KV] A. Knapp and D. Vogan, “Cohomological Induction and Unitary Representations”, Princeton University Press, Princeton, New Jersey, 1995.

[VZ] D. Vogan and G. Zuckerman, Unitary representations with non-zero cohomology, Compositio Math., 53 (1984), 51–90.

Department of Mathematics Massachusetts Institute of Technology Cambridge, Massachusetts 02139-4307