The method of coadjoint orbits for real reductive groups

David A. Vogan, Jr.

IAS/Park City Mathematics Series Volume 6, 1998

The method of coadjoint orbits for real reductive groups

David A. Vogan, Jr.

Introduction

The method of coadjoint orbits suggests that irreducible unitary representations of a Lie group G are something like quantum mechanical systems, and that their classical analogues are (roughly speaking) symplectic homogeneous spaces. (Actually one imposes on the symplectic homogeneous space two additional structures: that of a Hamiltonian G-space, and an admissible orbit datum. We will ignore these for the purposes of the introduction.) The problem of quantization in mathematical physics is to attach a quantum mechanical model to a classical physical system. The problem of quantization in representation theory is to attach a unitary group representation to a symplectic homogeneous space.

The miraculous aspect of the orbit method is twofold. First, the relevant symplectic homogeneous spaces can easily be classified: they are the covering spaces of the orbits of G on the dual of its own Lie algebra. (This is the source of the terminology “coadjoint orbits.”) Second, the method works: it has successfully suggested where to look for large families of very different unitary representations of very different Lie groups.

Both of these claims of miraculousness require some justification. For the possibility of classifying symplectic homogeneous spaces, one can think of the superficially similar problem of classifying Riemannian homogeneous spaces. Even for as simple a group as SO(3), the answer to this problem is quite complicated (see Exercise 1). For a group like GL(n, ℝ), I do not know any reasonable description of the Riemannian homogeneous spaces. Yet we will see that the symplectic homogeneous spaces for any semisimple Lie group can be parametrized quite precisely in fairly elementary terms.

That the effectiveness of the method is miraculous—that is, that we should be surprised at a close connection between irreducible unitary representations and symplectic homogeneous spaces—requires even more justification. Mathematics

\begin{matrix}c \\ ◯ \end{matrix}1998 American Mathematical Society

1Department of Mathematics, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139 E-mail address: dav@math.mit.edu

is full of bijections between sets that at first glance appear unrelated (like commutative Banach algebras and compact topological spaces, for example; or vector spaces and cardinal numbers). We learn to expect that there is a clever idea that builds a bridge, and that once we have mastered the bridge, we can move back and forth freely and easily. In the case of the orbit method, there seems to be no such bridge. Relatively simple groups like SL(2, ℝ) have irreducible unitary representations (the complementary series) that do not correspond to any symplectic homogeneous space. Conversely, Torasso found in [Tor] that the double cover of SL(3, ℝ) has a symplectic homogeneous space corresponding to no unitary representation. (The space in Torasso’s counterexample actually carries the more refined structure of Hamiltonian G-space and admissible orbit datum mentioned earlier.)

If there is no big theorem at the end of the orbit method’s rainbow, why should we be chasing it? The point is that unitary representations remain extraordinarily mysterious. The orbit method provides some light in a very dark room. Many large families of coadjoint orbits do correspond in comprehensible ways to unitary representations, and provide a clear geometric picture of these representations. Many ideas about orbits (like the Jordan decomposition in the case of reductive groups) suggest corresponding properties of unitary representations; and these properties can sometimes be proved. Best of all, coadjoint orbits can tell us where to look for unitary representations that we haven’t yet thought of.

The first goal of these notes is to describe what is known about the quantization problem for reductive Lie groups, and the extent to which its resolution can lead to all unitary irreducible representations. We will find that the quantization problem comes down to a finite set of coadjoint orbits for each reductive group: the nilpotent orbits.

The problem of quantizing nilpotent orbits has a long and colorful history. Two themes recur: quantization of very particular classes of orbits, and incomplete schemes for quantization of general orbits. I will mention one favorite example of each: Torasso’s paper [Tor2] on quantization of minimal orbits, and a joint paper [GV] with Graham suggesting how to guess at the representations attached to general orbits.

The second goal of the notes is to describe a new method for studying the quantization of nilpotent coadjoint orbits in terms of restriction to a maximal compact subgroup. This falls into the second “recurring theme,” and therefore the results will be very incomplete. What I intend to show is that the problem of quantizing a particular nilpotent orbit can often be reduced to a finite amount of linear algebra; and that there may be hope for understanding this linear algebra in general.

I am grateful to the graduate students at the 1998 PCMI for their interest and for their critical comments on these notes. Most of all I would like to thank Monica Nevins, who served as teaching assistant for the summer course and read the notes with great care. Her mathematical advice is responsible for many pockets of clarity in the exposition. (After her last corrections, I sprinkled a number of typographical and mathematical absurdities through the text, in order to restore it to my usual standards. For these she bears no responsibility.)

LECTURE 1

Some ideas from mathematical physics

The method of coadjoint orbits has its origins in mathematical physics. As explained for example in [Arn], a classical mechanical system can often be modelled by a symplectic manifold X, called the phase space. A point of the manifold records something like the positions and momenta of all the particles in the system. The evolution of the system in time defines a path γ in X. Newton’s laws for the evolution of the system say that this path is an integral curve for a Hamiltonian vector field ξ_{E} on X:

dγ/dt = ξ_{E}(γ(t)).  (1.1)(a)

The function E on X corresponds to the energy of the mechanical system. Other physical observables correspond to other functions f on X. Along an integral curve γ for ξ_{E}, the observable f evolves according to the differential equation

d(f ∘ γ)/dt = {E, f} ∘ γ;  (1.1)(b)

here {E, f} is the Poisson bracket of E and f. In particular, the observable f is conserved (that is, constant in time) if and only if it Poisson commutes with the energy.

A quantum mechanical system, on the other hand, is typically modelled by a Hilbert space H. Each state of the system corresponds to a line in the Hilbert space. The evolution of the system in time therefore corresponds to a path in the projective space ℙH. The physical laws for the system are encoded by a oneparameter group of unitary operators, whose orbits on ℙH are the possible time histories of the system. The generator of this one-parameter group is a skew-adjoint operator A, again corresponding to the energy of the system. (We will be careless about questions of unbounded operators, domains, and so on; these questions seem not to affect the philosophical ideas we want to emphasize.) If v(t) is a vector in H representing the state of the system at time t, then the analogue of (1.1)(a) is the Schrödinger equation

dv/dt = Av(t).  (1.2)(a)

(This will look more like a classical Schrödinger equation in the setting of (1.3)(b) below.) In general a physical observable corresponds to an operator B on H. If the system is in a state corresponding to a line L in ℙH, then the outcome of the observation corresponding to B cannot be predicted exactly. There is a probability distribution of possible outcomes, with expectation ⟨Bv, v⟩; here v is any unit vector in L. Along an orbit of our one-parameter group e^{tA} of unitary operators, this expectation becomes

⟨e^{−tA}Be^{tA}v, v⟩.  (1.2)(b)

Consequently B is conserved exactly when it commutes with the one-parameter group. A more precise analogue of (1.1)(b) is the differential equation

d/dt(⟨Bv, v⟩) = ⟨(BA − AB)v(t), v(t)⟩.  (1.2)(c)

Again the conserved operators are those commuting with the energy.

These two kinds of mathematical model are fundamentally different. Asking for any close correspondence between them seems to fly in the face of all the painfully learned lessons of physics in this century. Nevertheless, such correspondences sometimes exist. That is, certain classical mechanical systems correspond formally to quantum mechanical ones. Because such correspondences are at the heart of the orbit philosophy, we need to understand them in some detail.

A simple case (arising for example from the motion of point particles) is a system attached to a smooth Riemannian manifold M of “states” (typically the possible positions of the particles in the system). A tangent vector to M at a state m specifies the velocities of the particles; the Riemannian metric gives kinetic energy. The phase space X of such a mechanical system may be identified with the cotangent bundle T^{∗}(M), endowed with its natural symplectic structure. The energy function E on T^{∗}(M) may be the sum of a kinetic energy term (the Riemannian length in the cotangent direction) and a potential energy term depending only on M (that is, on the positions of the particles). Explicitly,

E(m, ξ) = ⟨ξ, ξ⟩ + V (m)  (m ∈ M, ξ ∈ T_{m}^{∗}(M)).  (1.3)(a)

There is a simple and natural family of observables in this system. If q is any smooth function on M, then q defines a function on the phase space T^{∗}(M); the value of this function depends only on the positions of the particles in the system, and not on their velocities. One sees easily that any two such functions on T^{∗}(M) must Poisson commute with each other:

{q_{1}, q_{2}} = 0  (q_{i} ∈ C^{∞}(M)).  (1.3)(b)

Such a system has a natural quantization. The Hilbert space H is the complex Hilbert space L^{2}(M). If v is a unit vector in this Hilbert space, then |v|^{2} is a non-negative function on M of integral 1; that is, a probability density on M. One thinks of this density as describing the probability of finding the particles of the system in a particular position. The quantum-mechanical energy operator is the Laplace-Beltrami operator L on M, plus the potential V. More precisely,

A = iℏL + \frac{i}{ℏ}V.  (1.3)(c)

(The factor ℏ is Planck’s constant divided by 2π.) There is a natural family of quantum observables corresponding to the classical position observables. If q is any smooth function on M, then multiplication by q defines an operator A_{q} on L^{2}(M). Any two of these operators commute with each other:

[A_{q_{1}}, A_{q_{2}}] = 0  (q_{i} ∈ C^{∞}(M)).  (1.3)(d)

(A little thought shows that the identification of these observables as “positions” is more or less equivalent to our earlier assertion that the probability density |v|^{2} describes the distribution across M of the quantum state.)

Physically the idea of quantization is to replace a classical mechanical model by a quantum mechanical model of the “same” system. This idea is a little difficult to pin down, since part of the point of quantum mechanics is that there never was a physically real classical system in the first place. Mathematically the idea is to pass from a symplectic manifold X and an interesting collection of real-valued functions {f_{i} | i ∈ I} on X (the classical observables), to a Hilbert space H and an interesting collection of skew-adjoint operators {A_{i} | i ∈ I} on H (the quantum observables). The shape of the equations (1.1)(b) and (1.2)(c) for the time evolution of observables suggests that we might ask for quantization to carry Poisson bracket of functions to commutator of operators. That is,

{f_{i}, f_{j}} = f_{k} ⇒ [A_{i}, A_{j}] = A_{k}.  (1.4)

The requirement (1.4) turns out to be too stringent to impose on all observables. Roughly speaking, the reason is that functions under Poisson bracket are more nearly commutative than operators under commutator. To understand this statement, it is helpful to look at an example. Let us take for X the 2n-dimensional vector space ℝ^{2n}, endowed with the standard symplectic structure

ω((x, λ), (y, ξ)) = ⟨x, ξ⟩ − ⟨y, λ⟩  (x, y, ξ, λ ∈ ℝ^{n}).  (1.5)(a)

An interesting family of observables consists of the affine functions on X. As a basis of these, we can take the constant function 1 together with the linear functions

p_{i}(x, λ) = x_{i},  q_{j}(x, λ) = λ_{j}.  (1.5)(b)

The constant function Poisson commutes with all functions, and the linear functions satisfy

{p_{i}, p_{i′}} = 0,  {q_{j}, q_{j′}} = 0,  {p_{i}, q_{j}} = δ_{ij} · 1.  (1.5)(c)

The quantization problem as formulated at (1.4) asks us to realize these same relations as commutators of self-adjoint operators on a Hilbert space. There is a very natural way to do that. We define

H = L^{2}(ℝ^{n}),  (1.6)(a)

thinking of this as the ℝ^{n} in the last n coordinates of X. The functions q_{j} are then naturally identified with the coordinate functions on ℝ^{n}. Multiplication by such a coordinate function is a self-adjoint operator on L^{2}, so it is reasonable to define

Q_{j} = multiplication by \sqrt{−1}q_{j},  (1.6)(b)

a skew-adjoint operator on L^{2}(ℝ^{n}). Similarly, we put

P_{i} = ∂/∂q_{i},  (1.6)(c)

a skew-adjoint operator on L^{2}. These operators satisfy commutation relations

[P_{i}, P_{i′}] = 0,  [Q_{j}, Q_{j′}] = 0,  [P_{i}, Q_{j}] = δ_{ij} · \sqrt{−1}.  (1.6)(d)

If we make the constant function 1 correspond to the scalar quantum operator \sqrt{−1}, then the requirement of (1.4) is satisfied.

The problem with (1.4) becomes apparent when we try to quantize more functions. A natural extension of the class of affine functions on X is the class of polynomial functions. This is just the polynomial algebra with generators p_{i} and q_{j}:

Poly(X) = ℝ[p_{1}, …, p_{n}, q_{1}, …, q_{n}].  (1.7)(a)

Similarly, the span of the operators P_{i}, Q_{j}, and the imaginary scalars extends naturally to the skew-adjoint polynomial coefficient differential operators:

Diff(ℝ^{n}) = {D ∈ ℂ[∂/∂q_{1}, …, ∂/∂q_{n}, q_{1}, …, q_{n}] | D^{∗} = −D}.  (1.7)(b)

There are many ways to arrange vector space isomorphisms between Poly(X) and Diff(ℝ^{n}), but none that respect the two kinds of commutator (see Exercise 3).

One lesson that might be drawn from the details of Exercise 3 is that finitedimensional Lie algebras of classical observables have a better chance to quantize nicely. This restriction does not interfere with our particular interest in representation theory of finite-dimensional Lie groups. As a first approximation to a more mathematical notion of quantization, we may think of a symplectic manifold X endowed with a finite-dimensional Lie algebra 𝔤 of smooth functions. A quantization would then be a Hilbert space endowed with a finite-dimensional Lie algebra (also isomorphic to 𝔤) of skew-adjoint operators. This is close to where we want to be; but we need to face some of the problems about unbounded operators that we have ignored until now. On X, the Lie algebra of functions defines a Lie algebra of (Hamiltonian) vector fields; the question there is whether these vector fields may be integrated to a group action on X. On the Hilbert space, the question is whether the family of unbounded skew-adjoint operators can be integrated to a group representation. Both questions are subtle and difficult, but to some extent irrelevant: we are concerned finally only with cases when the integration is possible. Here is the classical setting.

Definition 1.8. Suppose X is a symplectic manifold, and f ∈ C^{∞}(X) is a smooth function. The Hamiltonian vector field ξ_{f} of f is defined in terms of the Poisson bracket by

ξ_{f}(g) = {f, g}.

Suppose G is a Lie group endowed with a smooth action on X by symplectomorphisms. We say that X is a Hamiltonian G-space if there is a linear map

μ̃: 𝔤 → C^{∞}(X)

with the following properties. First, μ̃ intertwines the adjoint action of G on 𝔤 with its action on C^{∞}(X). Second, for each Y ∈ 𝔤, the vector field by which Y acts on X is ξ_{μ̃(Y)}. Third, μ̃ is a Lie algebra homomorphism.

This definition may sound circular. In most developments of symplectic manifolds the Hamiltonian vector field ξ_{f} is defined first (by using the symplectic form to identify the 1-form df with a vector field). Then the formula in Definition 1.8 is used as the definition of the Poisson bracket. But for the theory of quantization the Poisson bracket is more fundamental than the symplectic form, so it is not unreasonable to define other ideas in terms of the Poisson bracket. One consequence is that Definition 1.8 can be formulated in the category of Poisson manifolds, or even of (possibly singular) Poisson algebraic varieties. The definition is due essentially to Kirillov and to Kostant independendently (see [Kir, 15.2] and [Kos, 5.4.1]).

The natural quantum analogue of a Hamiltonian G-space is simply a unitary representation of G.

Definition 1.9. Suppose G is a Lie group. A unitary representation of G is a pair (π, H) with H a Hilbert space, and

π: G → U(H)

a homomorphism from G to the group of unitary operators on H. That is, we require a continuous action map

G × H → H

that preserves lengths in the Hilbert space.

To make a stronger connection with the formalism of Definition 1.8, we should introduce the Lie algebra representation dπ of 𝔤; the operators of dπ are defined on a dense subspace H^{∞} of the Hilbert space, and they are formally skew-adjoint. But making correct statements about the relationship between these operators and π is difficult and unnecessary; so we have simply dropped them. The reader may like to consider whether Definition 1.8 might similarly be modified to avoid any mention of the Lie algebra.

The idea that we want from mathematical physics is that there should be a notion of “quantization” passing from Definition 1.8 to Definition 1.9: that is, from Hamiltonian G-spaces to unitary representations. We will sharpen this idea in various ways as we go along.

One sharpening is available immediately. Definitions 1.8 and 1.9 both admit simplest cases from which more general ones might be built. In the case of Definition 1.8, these are the homogeneous Hamiltonian G-spaces; that is, those for which the action on X is transitive. In the case of Definition 1.9, it is the slightly more subtle notion of irreducibility: that H should admit exactly two closed subspaces invariant under the action of G. We would like quantization to carry homogeneous Hamiltonian G-spaces to irreducible unitary representations.

LECTURE 2

The Jordan decomposition and three kinds of quantization

As explained in the introduction, the first miraculous aspect of the orbit method is that the homogeneous Hamiltonian G-spaces are easy to classify. The result is due to Kirillov and to Kostant. Here is the basic construction.

Lemma 2.1 ([Kir], 15.2, [Kos], Theorem 5.4.1). Suppose G is a Lie group with Lie algebra 𝔤, and X ⊂ 𝔤^{∗} is an orbit of the coadjoint action of G. For λ ∈ X, write G^{λ} for the stabilizer of λ; then X ≃ G/G^{λ}.

1. The tangent space to X at λ is

T_{λ}(X) ≃ 𝔤/𝔤^{λ}.

2. The skew-symmetric bilinear form

ω_{λ}(u, v) = λ([u, v]) (u, v ∈ 𝔤)

on 𝔤 has radical exactly 𝔤^{λ}, and so defines a symplectic form on T_{λ}(X).

3. The forms ω_{λ} make X into a symplectic manifold.

4. For u ∈ 𝔤, define μ̃(u) ∈ C^{∞}(X) to be the restriction to X of the linear functional u on 𝔤^{∗}. Then the mapping μ̃ makes X a Hamiltonian G-space (Definition 1.8).

Lemma 2.2. Suppose that X is a Hamiltonian G-space (Definition 1.8). Suppose X̃ is another smooth manifold with a G action, and that π: X̃ → X is a G-equivariant local diffeomorphism. Then X̃ inherits from X a natural structure of Hamiltonian G-space.

This is entirely elementary, and we leave it to the reader.

Theorem 2.3 ([Kir], 15.2, [Kos], Theorem 5.4.1). The homogeneous Hamiltonian G-spaces for a Lie group G are the covering spaces of coadjoint orbits. More precisely, suppose X is such a space, with moment map μ̃: 𝔤 → C^{∞}(X). Reinterpret μ̃ as a map

μ: X → 𝔤^{∗}, μ(x)(u) = μ̃(u)(x) (x ∈ X, u ∈ 𝔤).

Then μ is a G-equivariant local diffeomorphism onto a coadjoint orbit, and the Hamiltonian G-space structure on X is pulled back from that on the orbit (Lemma 2.1) by the map μ (Lemma 2.2).

To get a typical homogeneous Hamiltonian G-space, we must therefore begin with an element λ ∈ 𝔤^{∗}, and its stabilizer G^{λ}. A covering of the orbit is a homogeneous space G/G_{1}^{λ}, with

G_{0}^{λ} ⊂ G_{1}^{λ} ⊂ G^{λ}.

The subgroup G_{1}^{λ} is determined by the space up to conjugation in G^{λ}. These observations are summarized in

Corollary 2.4. The homogeneous Hamiltonian G-spaces covering the coadjoint orbit G · λ are in one-to-one correspondence with conjugacy classes of subgroups of the discrete group G^{λ}/G_{0}^{λ} of connected components of G^{λ}.

In order to study quantization for a Lie group G, we must therefore first understand the orbits of G on 𝔤^{∗}.

We are particularly interested in reductive Lie groups. Once it was traditional to look at connected semisimple Lie groups with finite center, but now it is clear that inductive arguments require a more flexible class: some abelian factors should be allowed, and some disconnectedness. Exactly which groups should be allowed is a surprisingly delicate question. The many pleasant properties of connected semisimple groups disappear not all at once but one or two at a time as the hypotheses are weakened. Harish-Chandra introduced a class of reductive groups that works well for many purposes. On the other hand his class excludes even some disconnected compact groups, like the orthogonal groups, that we would like to allow.

That is perhaps enough in the way of excuses and apologies to cover almost any definition. The one we will use is suggested by [Kna].

Definition 2.5. Write GL(n) for the group of real or complex n×n matrices. The Cartan involution of GL(n) is the automorphism conjugate transpose inverse:

θ(g) = ^{t}\overline{g}^{−1}.

A linear reductive group is a closed subgroup G of some GL(n), preserved by θ and having finitely many connected components. A reductive group is a Lie group G̃ endowed with a homomorphism π: G̃ → G onto a linear reductive group, so that the kernel of π is finite.

The group of fixed points of θ on GL(n) is the orthogonal group O(n) (in the case of ℝ) or the unitary group U(n) (in the case of ℂ). The easiest examples of linear reductive groups are the closed subgroups of O(n) or U(n). All compact Lie groups are therefore linear reductive groups. Some additional examples are provided by Exercise 4.

The great advantage of Definition 2.5 is that it makes the most important structure theorem fairly easy to prove. Here it is.

Theorem 2.6 (Cartan decomposition). Suppose G is a linear reductive group. Write

K = G^{θ}, 𝔰 = −1 eigenspace of θ on 𝔤.

Then the map

K × 𝔰 → G, (k, X) ↦ k exp(X)

is a diffeomorphism from K × 𝔰 onto G. In particular K is maximal among the compact subgroups of G.

Suppose G̃ is a reductive group, with π: G̃ → G as in Definition 2.5. Write K̃ = π^{−1}(K), a compact subgroup of G̃, and use dπ to identify the Lie algebras of G̃ and G. Then the map

K̃ × 𝔰 → G̃, (k̃, X) ↦ k̃ exp(X)

is a diffeomorphism from K̃ × 𝔰 onto G̃. In particular K̃ is maximal among the compact subgroups of G̃.

Suppose G̃ is a reductive group. Define a map θ from G̃ to itself by

θ(k̃ exp(X)) = k̃ exp(−X) (k̃ ∈ K̃, X ∈ 𝔰).

Then θ is an involution of order 2, the Cartan involution of G̃. The group of fixed points is K̃.

For the case of GL(n) this theorem is just the polar decomposition for matrices. The version for linear reductive groups follows fairly easily. The version for general reductive groups is a consequence.

The Lie algebra version of Theorem 2.6 is entirely trivial, but is worth stating explicitly. Notice that the Lie algebra 𝔨 of K is equal to the +1 eigenspace of the (involutive linear) automorphism θ of 𝔤. Since θ^{2} = 1, the Lie algebra must be the direct sum of the +1 and −1 eigenspaces of θ:

𝔤 = 𝔨 + 𝔰.

It is often helpful to be able to identify the Lie algebra of a reductive group with its dual space. This we accomplish by means of the next lemma.

Proposition 2.7. Suppose G is a reductive group. Identify 𝔤 with a Lie algebra of n × n matrices in accordance with Definition 2.5. Define a real-valued symmetric bilinear form on 𝔤 by

⟨X, Y ⟩ = Re tr XY.

1. The form is invariant under Ad(G) and the Cartan involution θ:

⟨Ad(g)X, Ad(g)Y ⟩ = ⟨X, Y ⟩ = ⟨θX, θY ⟩.

2. The Cartan decomposition 𝔤 = 𝔨 + 𝔰 is orthogonal for the form.

3. The form is positive definite on 𝔰 and negative definite on 𝔨, and (as a consequence) non-degenerate on 𝔤.

4. There is a G-equivariant linear isomorphism

𝔤^{∗} ≃ 𝔤, λ ↦ X_{λ}

characterized by

λ(Y) = ⟨X_{λ}, Y ⟩ (Y ∈ 𝔤).

The assertions about invariance are elementary, and the orthogonality follows from θ-invariance. The assertions about positivity can be proved easily for GL(n) by explicit calculation with standard bases for matrices; the general cases follow. The last assertion is an easy formal consequence.

It follows from Proposition 2.7 that for G reductive, G orbits on 𝔤^{∗} may be identified with G orbits on 𝔤. In addition, these latter orbits are related to (more precisely, they are subsets of) conjugacy classes of matrices under GL(n). The following definitions for matrices therefore make sense for 𝔤.

Definition 2.8. Suppose G is a reductive Lie group, with Lie algebra 𝔤 consisting of n × n matrices (Definition 2.5). An element X ∈ 𝔤 is called nilpotent if it is nilpotent as a matrix; that is, if X^{N} = 0 for N large enough. An equivalent requirement is that every eigenvalue of X be equal to zero.

An element X ∈ 𝔤 is called semisimple if the corresponding complex matrix is diagonalizable. (By this we mean that X is regarded as an n × n complex matrix, even if 𝔤 consists of real matrices. For example, the real matrix

X = ( \begin{matrix}0 & 1 \\ −1 & 0 \end{matrix} )

is semisimple, since over ℂ it is conjugate to ( \begin{matrix}i & 0 \\ 0 & −i \end{matrix} ).)

An element X ∈ 𝔤 is called hyperbolic if it is semisimple and all eigenvalues are real. Typical examples are the self-adjoint matrices in 𝔰.

An element X ∈ 𝔤 is called elliptic if it is semisimple and all eigenvalues are purely imaginary. Typical examples are the skew-adjoint matrices in 𝔨, like the 2 × 2 example above.

It is more difficult to produce examples of nilpotent elements related to the Cartan involution and the reductive group structure. Here is one way. Suppose G is a linear reductive group, and we have three elements H, E, and F of 𝔤, satisfying

θE = −F, θH = −H, [H, E] = 2E, [E, F] = H.  (2.9)(a)

Then necessarily E is nilpotent; for the matrix E^{N} belongs to the 2N eigenspace of ad(H) on 𝔤𝔩(n), and this must be zero for large N. On the other hand, there is a group homomorphism

ϕ: SL(2, ℝ) → G,  (2.9)(b)

characterized by the requirements

dϕ ( \begin{matrix}1 & 0 \\ 0 & −1 \end{matrix} ) = H, dϕ ( \begin{matrix}0 & 1 \\ 0 & 0 \end{matrix} ) = E, dϕ ( \begin{matrix}0 & 0 \\ 1 & 0 \end{matrix} ) = F.  (2.9)(c)

This homomorphism respects the Cartan involutions:

ϕ(^{t}g^{−1}) = θ(ϕ(g)).  (2.9)(d)

Exercise 6 asks you to think about why the homomorphism ϕ exists.

Evidently a homomorphism ϕ satisfying (2.9)(d) defines by (2.9)(c) elements H, E, and F satisfying (2.9)(a). We call such elements a standard 𝔰𝔩(2) triple. The element E alone determines F = −θE and H = [E, F]. Not every nilpotent element E belongs to a standard triple, however.

Proposition 2.10 (Jordan decomposition). Suppose G is a reductive group with Lie algebra 𝔤 (Definition 2.5) and Cartan decomposition G = K · exp(𝔰) (Theorem 2.6).

1. Any element X ∈ 𝔤 has a unique decomposition

X = X_{h} + X_{e} + X_{n}

characterized by the requirements that X_{h} is hyperbolic (Definition 2.8); X_{e} is elliptic; X_{n} is nilpotent; and X_{h}, X_{e}, and X_{n} commute with each other.

2. After replacing X by a conjugate under Ad(G), we may assume that X_{h} ∈ 𝔰, X_{e} ∈ 𝔨, and that X_{n} = E belongs to a standard 𝔰𝔩(2) triple (see (2.9)(a)).

3. The Ad(G) orbits of hyperbolic elements in 𝔤 are in one-to-one correspondence with the Ad(K) orbits on 𝔰.

4. The Ad(G) orbits of elliptic elements in 𝔤 are in one-to-one correspondence with the Ad(K) orbits on 𝔨.

5. The Ad(G) orbits of nilpotent elements in 𝔤 are in one-to-one correspondence with the Ad(K) orbits of standard 𝔰𝔩(2) triples in 𝔤.

Proposition 2.10 provides a systematic way to list the orbits of a reductive group G acting on its Lie algebra. First, list the possible hyperbolic parts X_{h}. According to the proposition, this is equivalent to listing the orbits of K on 𝔰. (By standard structure theory, this in turn is equivalent to listing the orbits of the “small Weyl group” of G on a Cartan subspace 𝔞; but we will not use this fact.)

Proposition 2.11. Suppose G = K · exp(𝔰) is the Cartan decomposition of a reductive group, and X_{h} ∈ 𝔰. Write G^{X_{h}} for the stabilizer of X_{h} in the adjoint action. Then G^{X_{h}} is a reductive group with Cartan decomposition K^{X_{h}} · exp(𝔰^{X_{h}}). The orbits of G on 𝔤 with hyperbolic part X_{h} are in one-to-one correspondence with the orbits of G^{X_{h}} on 𝔤^{X_{h}} having hyperbolic part 0. The correspondence sends the orbit of X_{e} + X_{n} for G^{X_{h}} to the orbit of X_{h} + X_{e} + X_{n}.

The only part of this proposition that does not follow at once from Proposition 2.10 is the fact that G^{X_{h}} is a reductive group. This may be reduced at once to the case that G is linear reductive. Then the fact that G^{X_{h}} is closed and θ-stable is easy; what requires proof is the finiteness of the group of connected components. There are some hints in Exercise 7.

So now we are reduced to the problem of listing orbits with hyperbolic part 0, in a smaller reductive group. To keep the notation simple, we will just discuss this problem for G itself. We proceed by listing all possible elliptic parts X_{e} for such an orbit. According to Proposition 2.10, this is the same as listing the orbits of K on its own Lie algebra 𝔨. (Again standard structure theory reduces this to the orbits of the Weyl group of K on the Lie algebra of a maximal torus, but again we do not need this.)

Proposition 2.12. Suppose G = K · exp(𝔰) is the Cartan decomposition of a reductive group, and X_{e} ∈ 𝔨. Write G^{X_{e}} for the stabilizer of X_{e} in the adjoint action. Then G^{X_{e}} is a reductive group with Cartan decomposition K^{X_{e}} · exp(𝔰^{X_{e}}). The orbits of G on 𝔤 with hyperbolic part 0 and elliptic part X_{e} are in one-to-one correspondence with the nilpotent orbits of G^{X_{e}} on 𝔤^{X_{e}}. The correspondence sends the orbit of X_{n} for G^{X_{e}} to the orbit of X_{e} + X_{n}.

This is an easy consequence of Proposition 2.10.

Finally we are reduced to listing nilpotent orbits in a still smaller reductive group. (It is worth remarking that the class of smaller reductive groups arising in these reductions from a fixed G is finite.) In order to complete the picture in Propositions 2.11 and 2.12, we restate the description of nilpotent orbits in Proposition 2.10.

Proposition 2.13. Suppose G is a linear reductive group. The nilpotent orbits of G on 𝔤 are in one-to-one correspondence with the K-conjugacy classes of homomorphisms

ϕ: SL(2, ℝ) → G

respecting the Cartan involutions: ϕ(^{t}g^{−1}) = θ(ϕ(g)). The correspondence sends ϕ

to the orbit of dϕ ( \begin{matrix}0 & 1 \\ 0 & 0 \end{matrix} ).

For any reductive G there is the trivial homomorphism ϕ, corresponding to the nilpotent orbit 0. There will be other classes if and only if the derived group of G_{0} is noncompact. In any case the total number of classes (the number of nilpotent orbits in the Lie algebra) is finite. This is not particularly obvious or easy to prove; a nice introduction to the theory is in [CM].

Recall that we are really concerned with representation theory. Our interest in orbits arises from the formal relationship suggested by mathematical physics between Definitions 1.8 and 1.9: that there should be a method of “quantization” to pass from coadjoint orbits for G to irreducible unitary representations of G. According to Proposition 2.7, we can identify coadjoint orbits with adjoint orbits (for reductive groups). The last few propositions have described the adjoint orbits in three steps. The idea of quantization correspondingly suggests that unitary representations of a reductive group G should be constructed in three steps; even more, that an irreducible unitary representation ought to have a kind of “Jordan decomposition” along the lines of Proposition 2.10. We will not consider this problem very seriously, but here is a brief outline of what is known.

The first step in the Jordan decomposition is finding the hyperbolic part of an orbit. For this step there is a perfect and complete analogue in unitary representation theory, given in [Kna], Theorem 16.10. The second step is finding the elliptic part of an orbit. Here the representation-theoretic analogue is known (see [VoCI]) but its behavior is more complicated. A partly conjectural connection with the behavior of orbits is described in [SV]; the geometric nature of the results is emphasized in [VLC].

The final step in the Jordan decomposition is a nilpotent coadjoint orbit. The representation-theoretic analogue is the theory of unipotent representations, which is still in its infancy. Two more or less expository discussions of the theory may be found in [VO] and [VAV]. As indicated in the introduction, we will return to the problem of quantizing nilpotent coadjoint orbits in the latter part of these lectures.

The title of this section suggests that the three steps for constructing a unitary representation from a coadjoint orbit should be different from each other, even though they share a common motivation. In a sense accepting such differences is an admission of defeat, and contrary to the spirit of the orbit method. Nevertheless it is of tremendous value technically. To conclude this section, we will outline the construction of representations parallel to the hyperbolic part of a coadjoint orbit: parabolic induction. This construction appeared in a number of special cases before Gelfand and his collaborators began to emphasize its general importance in the early 1950s. The history of the subject from that point on is colorful and impressive, but my scholarship is not deep enough, nor my hide thick enough, to allow a discussion of it here.

Let us therefore fix a hyperbolic element

λ_{h} ∈ 𝔤^{∗}  (2.14)(a)

in the dual of the Lie algebra for our reductive Lie group G. Recall from Proposition 2.7 that this means that the element X_{h} ∈ 𝔤 defined by

λ_{h}(Y) = ⟨X_{h}, Y ⟩  (Y ∈ 𝔤)  (2.14)(b)

is hyperbolic. After replacing λ_{h} by a conjugate under G, we may therefore assume that

X_{h} ∈ 𝔰.  (2.14)(c)

The isotropy group for the coadjoint action at λ_{h} is

G^{λ_{h}} = {g ∈ G | Ad(g)(X_{h}) = X_{h}} = L,  (2.14)(d)

a reductive group with Cartan involution θ|_{L} (Proposition 2.11). Now a hyperbolic element of the Lie algebra acts in any algebraic representation by a diagonalizable operator with real eigenvalues. (Exercise 8 outlines a proof of this fact for the adjoint representation.) Consequently ad(X_{h}) is diagonalizable with real eigenvalues. Explicitly,

𝔤 = ∑_{r∈ℝ} 𝔤_{r},  (2.14)(e)

with

𝔤_{r} = {Y ∈ 𝔤 | [X_{h}, Y] = rY }.  (2.14)(f)

Two immediate consequences are

[𝔤_{r}, 𝔤_{s}] ⊂ 𝔤_{r+s},  Ad(L)(𝔤_{r}) = 𝔤_{r},  𝔤_{0} = Lie(L).  (2.14)(g)

Proposition 2.15. With notation as above, define

𝔲 = ∑_{r>0} 𝔤_{r}.  (2.15)(a)

Then 𝔲 is an ad-nilpotent Lie subalgebra of 𝔤, normalized by the adjoint action of L. The exponential map is a diffeomorphism of 𝔲 onto a closed subgroup U of G, also normalized by L and meeting it trivially. The semidirect product group Q = LU ⊂ G is a closed subgroup of G. The homogeneous space G/Q is compact; in fact it is homeomorphic to K/L ∩ K.

There is a unitary character χ(λ_{h}) of L defined by

χ(λ_{h})(k · exp(Z)) = exp(iλ_{h}(Z))  (k ∈ L ∩ K, Z ∈ 𝔩 ∩ 𝔰).  (2.15)(b)

Equivalently, χ(λ_{h}) may be defined as the unique character of L that is trivial on L ∩ K, and has differential iλ_{h}.

There is a character ρ_{Q} of Q taking positive real values, defined by

ρ_{Q}(q) = | det (Ad(q)|_{𝔲}) |^{1/2}.  (2.15)(c)

Many of the assertions in this proposition are very easy to prove, and none is particularly deep. We will not discuss the proof, however.

The character ρ_{Q} is included because it is used in the definition of Mackey’s unitary induction from Q to G. Geometrically, it is the character of Q corresponding to the (Q-equivariant) bundle of half-densities on G/Q. To see that, it is better to write

ρ_{Q}(q) = | det (Ad^{∗}(q)|_{(𝔤/𝔮)^{∗}}) |^{1/2}.

But the equivalence of this definition with (2.15)(c) is easy to prove.

Before describing how this structure is used to construct unitary representations, we will say a little about its geometric content. We should look at the geometry of arbitrary coadjoint orbits with hyperbolic part λ_{h}, but to simplify the discussion we consider only the coadjoint orbit of λ_{h} itself. This is

X = Ad^{∗}(G) · λ_{h} ≃ G/G^{λ_{h}} = G/L.  (2.16)(a)

Because L is contained in Q, there is a natural fibration

X ≃ G/L → G/Q;  (2.16)(b)

the fiber over the identity coset is Q/L ≃ U. Using the definition of the symplectic structure on X in Lemma 2.1, together with the structural information in (2.15), one sees easily that Q/L is a Lagrangian submanifold of X; that is, that Q/L has half the dimension of X = G/L, and that the symplectic form ω_{λ} vanishes on 𝔮/𝔩. We summarize this by saying that

X ≃ G/L → G/Q is a G-equivariant Lagrangian fibration.  (2.16)(c)

Definition 2.17. Suppose G is a reductive Lie group, λ_{h} is a hyperbolic element of 𝔤^{∗} as in (2.14), L is the stabilizer of λ_{h} in G, and Q = LU is the semidirect product subgroup constructed in Proposition 2.15. The functor of parabolic induction carries unitary representations (π_{L}, H_{L}) of L to unitary representations (π_{G}, H_{G}) of G, as follows. First extend π_{L} to a representation π_{Q} of Q on the same Hilbert space, by making U act trivially. Next, define H_{G}^{c} to be the vector space of continuous functions from G to H_{L} satisfying

ϕ(gq) = ρ_{Q}(q^{−1})π_{Q}(q^{−1})ϕ(g)  (g ∈ G, q ∈ Q).  (2.17)(a)

The group G acts on H_{G}^{c} by left translation:

[π_{G}^{c}(g)ϕ](x) = ϕ(g^{−1}x).  (2.17)(b)

We make H_{G}^{c} into a pre-Hilbert space as follows. Suppose ϕ and ψ belong to H_{G}^{c}. Define a complex-valued function f_{⟨ϕ,ψ⟩} on G by

f_{⟨ϕ,ψ⟩}(g) = ⟨ϕ(g), ψ(g)⟩;  (2.17)(c)

the inner product on the right is in H_{L}. From the transformation property (2.17)(a) and the unitarity of π_{Q} we deduce

f_{⟨ϕ,ψ⟩}(gq) = ρ_{Q}^{2}(q^{−1})f_{⟨ϕ,ψ⟩}(g).  (2.17)(d)

Up to a choice of measure on 𝔤/𝔮, this means that f_{⟨ϕ,ψ⟩} may be regarded as a continuous density on G/Q. Since G/Q is compact, such a density has finite total volume; so we can define

⟨ϕ, ψ⟩ = ∫_{G/Q} f_{⟨ϕ,ψ⟩}.  (2.17)(e)

That this is a pre-Hilbert structure on H_{G}^{c} is clear. Define H_{G} to be the completion of H_{G}^{c}. Since the operators π_{G}^{c}(g) preserve the pre-Hilbert structure, they extend to unitary operators π_{G}(g) on H_{G}. It is straightforward to show that

(π_{G}, H_{G}) = Ind_{Q}^{G}(π_{L} ⊗ 1)  (2.17)(f)

is a (continuous) unitary representation of G.

The simplest example of Definition 2.17 has π_{L} equal to the trivial representation of L. In that case π_{G} is the representation of G on the space of square-integrable half-densities on G/Q. Accordingly to the philosophy of geometric quantization, the corresponding Hamiltonian G-space (in the correspondence we seek between Definitions 1.8 and 1.9) is just the cotangent bundle T^{∗}(G/Q). This space is not homogeneous for G; the moment map to 𝔤^{∗} has its image in the cone of nilpotent elements. (This means that Definition 2.17 has something to teach us about quantizing nilpotent coadjoint orbits. We will not pursue this knowledge, however.)

The next simplest example of Definition 2.17 has π_{L} equal to the one-dimen-sional character χ(λ_{h}) of Proposition 2.15. In that case π_{G} is a space of sections of the bundle of half-densities on G/Q, twisted by a Hermitian line bundle defined by χ(λ_{h}). It should be thought of as a quantization of the hyperbolic coadjoint orbit X = Ad^{∗}(G) · λ_{h}.

Parabolic induction turns out to be a nice analogue for unitary representations of the geometric correspondence described in Proposition 2.11. There is not really a theorem to state here, but we can make a definition.

Definition 2.18. Suppose G is a reductive Lie group, λ_{h} ∈ 𝔤^{∗} is a hyperbolic element, L is the stabilizer of λ_{h} in G, and Q = LU is the group constructed in Proposition 2.15. Recall from Proposition 2.11 that there is a bijection between coadjoint orbits for G with hyperbolic part conjugate to λ_{h}, and coadjoint orbits for L with hyperbolic part 0.

Let X_{L} ⊂ 𝔩^{∗} be any coadjoint orbit for L with hyperbolic part 0, and suppose π_{L} is any unitary representation of L attached to X_{L}. (Here “attached” refers to some unspecified quantization scheme for L; it does not need to have any particular mathematical meaning.) Write X_{G} = Ad^{∗}(G) · (λ_{h} + X_{L}) for the corresponding coadjoint orbit for G. Then we declare that the unitary representation of G

π_{G} = Ind_{Q}^{G}(π_{L} ⊗ χ(λ_{h}))

(Definition 2.17) is attached to X_{G}.

If we want to make the orbit method into a scheme for classifying unitary representations, then there are theorems to be proven here: that (for appropriate classes of π_{L}) the representations π_{G} are irreducible, and that they exhaust the irreducible unitary representations of G. That is the nature of the results in [Kna], Theorem 16.10 (compare also the formulation in [VLC], section 3). Since we are concerned mostly with defining a method of quantization, we will be content with Definitions 2.17 and 2.18.

LECTURE 3

Complex polarizations

Our goal in this section is to introduce briefly the replacement for parabolic induction related to elliptic coadjoint orbits. Because these ideas will be at the center of the lectures of Roger Zierau, we will give few details. We want only to formulate the main ideas in such a way as to suggest a transition from the hyperbolic case to the nilpotent case.

We begin by considering again the geometric content of parabolic induction; specifically, how the parabolically induced representation can be described in terms of the coadjoint orbit to which it is formally attached. To simplify, we will work only with the hyperbolic orbit X = G · λ_{h} ≃ G/L, and not with a more general orbit having hyperbolic part λ_{h}. We first defined a unitary character χ(λ_{h}) of L, and thus a hermitian line bundle on G/L. Next, we twisted this line bundle by a half-density bundle attached to a character ρ_{Q} of L. Finally, we considered sections of the twisted line bundle on G/Q; that is, sections on G/L that are constant along the fibers of the projection G/L → G/Q. Because these fibers are connected, the constancy can be expressed in terms of the vanishing of certain derivatives. Here is a slightly more precise statement.

Proposition 3.1. In the setting of Definition 2.17, the space of smooth vectors in the representation Ind_{Q}^{G}(χ(λ_{h})) may be identified with smooth functions ϕ on G satisfying

ϕ(gl) = ρ_{Q}(l^{−1})χ(λ_{h})(l^{−1})ϕ(g) (g ∈ G, l ∈ L),

and

Z · ϕ = 0 (Z ∈ 𝔲).

Here in the second condition we identify 𝔤 with left-invariant vector fields on G; that is, with right derivatives.

The advantage of this formulation over Definition 2.17 is the almost complete disappearance of the group Q (which has no analogue in the case of elliptic coadjoint orbits).

We begin now a discussion of quantizing elliptic coadjoint orbits that is parallel to that for hyperbolic orbits in (2.14). We fix therefore an elliptic element

λ_{e} ∈ 𝔤^{∗},  (3.2)(a)

and write X_{e} ∈ 𝔤 for its image under the isomorphism of Proposition 2.7. After conjugating λ_{e} by G, we may assume that θλ_{e} = λ_{e}, or equivalently that X_{e} ∈ 𝔨.

The isotropy group for the coadjoint action at λ_{e} is

G^{λ_{e}} = {g ∈ G | Ad(g)(X_{e}) = X_{e}} = L,  (3.2)(b)

a closed reductive subgroup of G with Cartan involution θ|_{L}. Elements of the Lie algebra of the compact group K must act in any finite-dimensional representation of K by diagonalizable operators with purely imaginary eigenvalues. Consequently i ad(X_{e})_{ℂ} is diagonalizable with real eigenvalues. Explicitly,

𝔤_{ℂ} = ∑_{r∈ℝ} 𝔤_{ℂ,r},  (3.2)(c)

with

𝔤_{ℂ,r} = {Y ∈ 𝔤_{ℂ} | [iX_{e}, Y] = rY }.  (3.2)(d)

Just as in (2.14), we have

[𝔤_{ℂ,r}, 𝔤_{ℂ,s}] ⊂ 𝔤_{ℂ,r+s},  Ad(L)(𝔤_{ℂ,r}) = 𝔤_{ℂ,r},  𝔤_{ℂ,0} = Lie(L)_{ℂ}.  (3.2)(e)

Just as in Proposition 2.15, we can define

𝔲 = ∑_{r>0} 𝔤_{ℂ,r}.  (3.2)(f)

Then 𝔲 is an ad-nilpotent Lie subalgebra of 𝔤_{ℂ}, normalized by the adjoint action of L. The semidirect product Lie algebra

𝔮 = 𝔩_{ℂ} + 𝔲 ⊂ 𝔤_{ℂ}  (3.2)(g)

is a parabolic subalgebra of 𝔤_{ℂ}. What is different from the hyperbolic case is that these are not complexifications of subalgebras of 𝔤. To see that more precisely, we consider complex conjugation acting on 𝔤_{ℂ}:

\overline{U + iV} = U − iV (U, V ∈ 𝔤).

Then \overline{𝔤_{ℂ,r}} = 𝔤_{ℂ,−r}. Therefore

\overline{𝔲} = ∑_{r>0} 𝔤_{ℂ,−r}, \overline{𝔮} = 𝔩_{ℂ} + \overline{𝔲}.  (3.2)(h)

It follows that

𝔮 + \overline{𝔮} = 𝔤_{ℂ}, 𝔮 ∩ \overline{𝔮} = 𝔩_{ℂ}, 𝔤_{ℂ} = \overline{𝔲} ⊕ 𝔩_{ℂ} ⊕ 𝔲.  (3.2)(i)

Proposition 3.3. Suppose X = G · λ_{e} ≃ G/L is an elliptic coadjoint orbit for the reductive group G; use the notation above. Then there is a distinguished G-invariant complex structure on X, characterized by the requirement that 𝔮/𝔩_{ℂ} ⊂ 𝔤_{ℂ}/𝔩_{ℂ} is the antiholomorphic tangent space to X at the point λ_{e} (corresponding to the identity coset eL).

There is a unitary character 2ρ_{𝔮} of L defined by

2ρ_{𝔮}(l) = det(Ad(l)|_{𝔲}).  (3.3)(a)

An equivalent definition is

2ρ_{𝔮}(l) = det(Ad^{∗}(l)|_{(𝔤/𝔮)^{∗}}).  (3.3)(b)

The differential of 2ρ_{𝔮} is a purely imaginary-valued one-dimensional character of the Lie algebra 𝔩, which we still denote by 2ρ_{𝔮}. Explicitly,

2ρ_{𝔮}(Z) = tr ad(Z)|_{𝔲}  (Z ∈ 𝔩).  (3.3)(c)

The linear functional iλ_{e} restricted to 𝔩 is a purely imaginary-valued onedimensional character of the Lie algebra.

We now have in hand most of the ingredients we need to formulate a definition along the lines of Proposition 3.1 for elliptic orbits. There are two problems. First, instead of the character ρ_{Q} of L we have available only 2ρ_{𝔮}, which is like the square of ρ_{Q}. We defined ρ_{Q} by taking a square root (of an absolute value, losing some plus or minus one factors in the process). The character 2ρ_{𝔮} is unitary, so its absolute value is identically one (and uninteresting). If we do not take an absolute value, the square root may or may not exist (as a unitary character of L). (Exercise 9 looks at an example.)

The second problem is that the linear functional iλ_{e} may not exponentiate to a unitary character of L. This problem is of fundamental importance: it’s not a bug, it’s a feature. The “elliptic unitary representations” parametrized by elliptic coadjoint orbits come not in continuous families (as the orbits do, by Proposition 2.10(4)) but in discrete ones. For representation theory we are interested only in the elliptic orbits to which we can attach something like a unitary character of L. The precise requirement addresses both problems (the square root of 2ρ_{𝔮} and the exponential of iλ_{e}) at the same time.

Definition 3.4. Suppose G is a reductive Lie group, and λ_{e} ∈ 𝔤^{∗} is an elliptic element. Use the notation of (3.2) and (3.3). An admissible datum at λ_{e} is an irreducible unitary representation (τ, V_{τ}) of L so that

dτ = (iλ_{e} + ρ_{𝔮}) · I_{τ};

here I_{τ} is the identity operator on the vector space V_{τ}.

The representation V_{τ} gives rise to a G-equivariant vector bundle V_{τ} over G/L; the smooth sections of V_{τ} may be identified with smooth functions ϕ from G to V_{τ} satisfying

ϕ(gl) = τ(l^{−1})ϕ(g)  (g ∈ G, l ∈ L).  (3.4)(a)

This identification makes sense locally on G/L: if U is an open set in G/L and Ũ its preimage in G, then smooth sections of V_{τ} over U correspond to functions on Ũ satisfying (3.4)(a) for all g ∈ Ũ.

Recall now the holomorphic structure on G/L introduced in Proposition 3.3. The vector bundle V_{τ} is naturally a G-equivariant holomorphic bundle. The holomorphic sections of V_{τ} over an open set U may be identified with smooth functions ϕ from Ũ to V_{τ} satisfying

ϕ(gl) = τ(l^{−1})ϕ(g)  (g ∈ Ũ, l ∈ L),  (3.4)(b)

and

Z · ϕ = 0  (Z ∈ 𝔲).  (3.4)(c)

Here in the second condition we identify 𝔤_{ℂ} with left-invariant complex vector fields on G; these are just the Cauchy-Riemann equations for the corresponding section. The group G acts by left translation on holomorphic sections of V_{τ}.

This definition of admissible orbit datum does not look much like Duflo’s (see [D] or [VAV], Definition 7.2). Nevertheless it is equivalent (for elliptic orbits in reductive groups). A reformulation that is a little closer to Duflo’s definition appears at the end of this section.

Given λ_{e}, there are just finitely many possibilities for τ (possibly none). All of them are finite-dimensional. If G is connected, then so is L, so there is at most one possible τ in that case.

Definition 3.4 provides us with a natural analogue of the representation attached to the hyperbolic orbit, as described in Proposition 3.1. It is the space Γ(G/L, V_{τ}) of all holomorphic sections of V_{τ}. This is a reasonable topological vector space, endowed with a natural continuous representation of G (by left translation). There are problems, however. One is that G/L is usually not a Stein manifold, so there is no reason to expect random holomorphic vector bundles to have any global sections at all. Indeed V_{τ} rarely has holomorphic sections (see Exercise 10).

In the absence of holomorphic sections, it is natural to look at higher Dolbeault cohomology of G/L with coefficients in V_{τ}. In many cases this leads to nice topological representations of G, but it is always difficult (and often impossible) to find G-invariant pre-Hilbert space structures on these representations. The problem appears even when G/L is Stein. The natural pre-Hilbert space structure is given by integrating the Hermitian inner product on sections over G/L. This integral does not converge for all holomorphic sections, and in some interesting cases it does not converge except for the zero section. For the representations on higher Dolbeault cohomology the problems are worse, and they are far from completely understood; Zierau’s lectures will explain what is known.

For the purpose of abstract quantization—of formally attaching unitary representations to coadjoint orbits—many of these problems can be circumvented. This is the subject of [KV], which I will not attempt to summarize here. There is in particular a definition like Definition 2.18, saying how to attach unitary representations to “elliptic plus nilpotent” coadjoint orbits once we know how to attach them to nilpotent orbits. An exposition appears in [VUR]. We will be content with the following special case.

Proposition 3.5 (see [Wng]). Suppose G is a reductive Lie group, and λ_{e} ∈ 𝔤^{∗} is an elliptic element. Use the notation of Definition 3.4. Suppose that (τ, V_{τ}) is an admissible datum at λ_{e}. Then there is attached to the G-orbit of (λ_{e}, τ) a unitary representation (π(λ_{e}, τ), H) of G, characterized by the following properties. Write V_{τ} for the holomorphic vector bundle on X = G/L associated to τ, and s = dim_{ℂ}(K/L ∩ K).

1. The Dolbeault cohomology groups H^{0,p}(X, V_{τ}) vanish for p ≠ s.

2. The \overline{∂} operator computing Dolbeault cohomology has closed range. Consequently the cohomology space H^{0,s}(X, V_{τ}) inherits a nice complete Hausdorff topological vector space structure, on which G acts continuously by left translation.

3. There is a G-equivariant embedding with dense image of the Hilbert space representation H in H^{0,s}(X, V_{τ}).

The representation π(λ_{e}, τ) is always a finite sum of irreducible representations; but it may vanish, and it may be reducible. The second possibility in particular sounds at first like a bug in the orbit method. For an attempt to make it a feature, see [VDK], Theorem 10.12(e).

To conclude this section, we recast the definition of admissible datum (Definition 3.4). We first construct a square root of 2ρ_{𝔮}. For that, define the metaplectic double cover of L by

L̃ = {(l, z) ∈ L × ℂ^{×} | 2ρ_{𝔮}(l) = z^{2}}.  (3.6)(a)

Projection on the first factor is a surjective homomorphism

p: L̃ → L,  p(l, z) = l.  (3.6)(b)

Evidently the kernel of p is the two-element central subgroup {1, ϵ}, with ϵ = (1, −1) ∈ L̃. Projection on the second factor defines a unitary character ρ_{𝔮} of L̃:

ρ_{𝔮}(l, z) = z.  (3.6)(c)

The notation is chosen because the square of this character is just 2ρ_{𝔮}: explicitly,

[ρ_{𝔮}(x)]^{2} = 2ρ_{𝔮}(p(x))  (x ∈ L̃).  (3.6)(d)

We call a representation of L̃ genuine if the central element ϵ acts by -1. A metaplectic admissible datum at λ_{e} is a genuine irreducible unitary representation (τ̃, V_{τ}) of L̃, with the property that

dτ̃ = iλ_{e} · I_{τ}.  (3.6)(e)

Proposition 3.7. Suppose λ_{e} ∈ 𝔤^{∗} is an elliptic element; use the notation of Definition 3.4 and (3.6). There is a one-to-one correspondence between admissible data (τ, V_{τ}) at λ_{e}, and metaplectic admissible data (τ̃, V_{τ}), characterized by

τ̃ ⊗ ρ_{𝔮} = τ ∘ p.

This is very easy. The new definition of admissible datum still differs from Duflo’s in that Duflo’s construction of the covering L̃ is different; but the two constructions may be shown to be equivalent.

LECTURE 4

The Kostant-Sekiguchi correspondence

We have now seen more or less how to carry out the first two steps of the quantization program described in section 2. What remains is to quantize nilpotent coadjoint orbits. This we will not be able to do completely, but the effort will occupy the rest of the notes. Our first goal is to understand more precisely the structure and classification of these orbits.

We begin by sharpening and recasting Proposition 2.13. To begin, we introduce some notation for SL(2, ℝ) and its complexification SL(2, ℂ). Write θ_{0} for the inverse transpose automorphism:

θ_{0}(g) = ^{t}g^{−1}  (g ∈ SL(2, ℂ)).  (4.1)(a)

We denote its differential by the same letter:

θ_{0}(Z) = −^{t}Z  (Z ∈ 𝔰𝔩(2, ℂ)).  (4.1)(b)

The complex conjugation σ_{0} defining the real form SL(2, ℝ) is just complex conjugation of matrices.

The standard basis of 𝔰𝔩(2, ℝ) is

H_{0} = ( \begin{matrix}1 & 0 \\ 0 & −1 \end{matrix} ) , E_{0} = ( \begin{matrix}0 & 1 \\ 0 & 0 \end{matrix} ) , F_{0} = ( \begin{matrix}0 & 0 \\ 1 & 0 \end{matrix} ) .  (4.1)(c)

These matrices satisfy

[H_{0}, E_{0}] = 2E_{0},  [H_{0}, F_{0}] = −2F_{0}, [E_{0}, F_{0}] = H_{0};  (4.1)(d)
θ_{0}(H_{0}) = −H_{0},  θ_{0}(E_{0}) = −F_{0},  θ_{0}(F_{0}) = −E_{0}.  (4.1)(e)

All are fixed by σ_{0}. We will also need a different basis of 𝔰𝔩(2, ℂ) diagonalizing the action of θ_{0}:

h_{0} = ( \begin{matrix}0 & i \\ −i & 0 \end{matrix} ) , x_{0} = \frac{1}{2} ( \begin{matrix}1 & −i \\ −i & −1 \end{matrix} ) , y_{0} = \frac{1}{2} ( \begin{matrix}1 & i \\ i & −1 \end{matrix} ) .  (4.1)(f)

These matrices satisfy

[h_{0}, x_{0}] = 2x_{0},  [h_{0}, y_{0}] = −2y_{0},  [x_{0}, y_{0}] = h_{0};  (4.1)(g)
θ_{0}(h_{0}) = h_{0},  θ_{0}(x_{0}) = −x_{0},  θ_{0}(y_{0}) = −y_{0};  (4.1)(h)
σ_{0}(h_{0}) = −h_{0},  σ_{0}(x_{0}) = y_{0},  σ_{0}(x_{0}) = y_{0}.  (4.1)(i)

Here is the improvement of Proposition 2.13 that we want.

Proposition 4.2. Suppose G is a real reductive group with Cartan involution θ and corresponding maximal compact subgroup K. Write 𝔤_{ℂ} and K_{ℂ} for the complexifications of 𝔤 and K, and σ for the complex conjugation on 𝔤_{ℂ}. Then the following sets are in natural one-to-one correspondence.

1. Orbits of G on the cone N_{ℝ} of nilpotent elements of 𝔤.

2. Conjugacy classes under G of Lie algebra homomorphisms ϕ_{ℝ} from 𝔰𝔩(2, ℂ) to 𝔤_{ℂ}, intertwining σ_{0} with σ. (Equivalently, we require that ϕ_{ℝ}(𝔰𝔩(2, ℝ)) ⊂ 𝔤).

3. Conjugacy classes under K of Lie algebra homomorphisms ϕ_{ℝ,θ} from 𝔰𝔩(2, ℂ) to 𝔤_{ℂ}, intertwining σ_{0} with σ and θ_{0} with θ.

4. Conjugacy classes under K_{ℂ} of Lie algebra homomorphisms ϕ_{θ} from 𝔰𝔩(2, ℂ) to 𝔤_{ℂ}, intertwining θ_{0} with θ.

5. Orbits of K_{ℂ} on the cone N_{θ} of nilpotent elements in 𝔰_{ℂ} (the -1 eigenspace

of θ on 𝔤_{ℂ}). Here the bijections from (3) to (2) and (4) are given by the obvious inclusions; that from (2) to (1) sends ϕ_{ℝ} to ϕ_{ℝ}(E_{0}) (notation (4.1)(c)); and that from (4) to (5) sends ϕ_{θ} to ϕ_{θ}(x_{0}).

This formulation incorporates ideas of Jacobson-Morozov, Kostant-Rallis, and Sekiguchi (among others); there is a more detailed discussion of the proof in [VAV], Theorem 6.4. The correspondence between (1) and (5) is the Kostant-Sekiguchi correspondence between real nilpotent orbits of G and nilpotent orbits of K_{ℂ} on 𝔰_{ℂ}. If ϕ_{ℝ,θ} is a homomorphism as in (3), then the correspondence is

E = ϕ_{ℝ,θ}(E_{0}) ↔ x = ϕ_{ℝ,θ}(x_{0})

(notation as in (4.1)(c,f)).

We now examine what Proposition 4.2 can tell us about the structure of isotropy groups and orbits.

Proposition 4.3. Suppose ϕ is a Lie algebra homomorphism from 𝔰𝔩(2, ℂ) to a complex reductive Lie algebra 𝔤_{ℂ}. Write

E = ϕ(E_{0}), H = ϕ(H_{0}), F = ϕ(F_{0})

(notation as in (4.1)(c)).

1. The operator ad(H) has integer eigenvalues; so if we write

𝔤_{ℂ}(p) = {X ∈ 𝔤_{ℂ} | [H, X] = pX},

then

𝔤_{ℂ} = ∑_{p∈ℤ} 𝔤_{ℂ}(p).

2. Write

𝔩 = 𝔤_{ℂ}(0), 𝔲 = ∑_{p>0} 𝔤_{ℂ}(p).

Then 𝔮 = 𝔩 + 𝔲 is a Levi decomposition of a parabolic subalgebra of 𝔤_{ℂ}.

3. The centralizer of E is graded by the decomposition in (1). More precisely,

𝔤_{ℂ}^{E} = 𝔩^{E} + ∑_{p>0} 𝔤_{ℂ}(p)^{E} = 𝔩^{E} + 𝔲^{E}.

4. The subalgebra 𝔩^{E} = 𝔤_{ℂ}^{H,E} is equal to 𝔤_{ℂ}^{ϕ}, the centralizer in 𝔤_{ℂ} of the image of ϕ. It is a reductive subalgebra of 𝔤_{ℂ}. Consequently the decomposition in (3) is a Levi decomposition of the algebraic Lie algebra 𝔤_{ℂ}^{E}.

Parallel results hold if (E, H, F) are replaced by

x = ϕ(x_{0}), h = ϕ(h_{0}), y = ϕ(y_{0})

(notation as in (4.1)(f)).

Most of the assertions in this proposition follow from the representation theory of 𝔰𝔩(2), applied to the representation ad ∘ϕ on 𝔤_{ℂ}. The rest are fairly easy; we omit the arguments.

Now we want analogous facts on the group level.

Proposition 4.4. Suppose G is a real reductive Lie group, and ϕ_{ℝ} is a Lie algebra homomorphism from 𝔰𝔩(2, ℂ) to 𝔤_{ℂ}, intertwining σ_{0} with σ. Define E, H, and F in 𝔤 as in Proposition 4.3.

1. The elements E and F are nilpotent, and H is hyperbolic.

2. Define L = G^{H} to be the isotropy group of the adjoint action at H, and U = exp(𝔲 ∩ 𝔤). Then Q = LU is the parabolic subgroup of G associated to H (Proposition 2.15).

3. The stabilizer of E in the adjoint action is contained in Q, and respects the

Levi decomposition:

G^{E} = (L^{E})(U^{E}).

4. The subgroup L^{E} = G^{H,E} is equal to G^{ϕ_{ℝ}}, the centralizer in G of the image of ϕ_{ℝ}. It is a reductive subgroup of G. The subgroup U^{E} is simply connected unipotent.

5. Suppose that ϕ_{ℝ} = ϕ_{ℝ,θ} also intertwines θ_{0} with θ. Then G^{ϕ_{ℝ,θ}} is preserved by θ, and we may take θ as a Cartan involution on this reductive group. In particular, G^{E} and G^{ϕ_{ℝ,θ}} have a common maximal compact subgroup

K^{ϕ_{ℝ,θ}} = (Ł ∩ K)^{E}.

This proposition provides good information about the action of G on nilpotent elements in 𝔤. Eventually we will want the same information about the action of K_{ℂ} on nilpotent elements in 𝔰_{ℂ}. Here it is.

Proposition 4.5. Suppose G is a real reductive Lie group with Cartan decomposition G = K exp(𝔰). Write K_{ℂ} for the complexification of K. Suppose that ϕ_{θ} is a Lie algebra homomorphism from 𝔰𝔩(2, ℂ) to 𝔤_{ℂ}, intertwining θ_{0} with θ. Define x, h, and y in 𝔤 as in Proposition 4.3.

1. The elements x and y are in 𝔰_{ℂ} are nilpotent, h ∈ 𝔨_{ℂ} is hyperbolic, and ih ∈ 𝔨_{ℂ} is elliptic.

2. The parabolic subalgebra 𝔮 = 𝔩 + 𝔲 of 𝔤 constructed as in Proposition 4.3 using h, is preserved by θ.

3. Define L_{K} = (K_{ℂ})^{h} to be the isotropy group of the adjoint action of K_{ℂ} at h, and U_{K} = exp(𝔲 ∩ 𝔨_{ℂ}). Then Q_{K} = L_{K}U_{K} is the parabolic subgroup of K_{ℂ} associated to h (Proposition 2.15).

4. The stabilizer of x in the adjoint action of K_{ℂ} is contained in Q_{K}, and respects the Levi decomposition:

K_{ℂ}^{x} = (L_{K}^{x})(U_{K}^{x}).

5. The subgroup L_{K}^{x} = K_{ℂ}^{h,x} is equal to K_{ℂ}^{ϕ_{θ}}, the centralizer in K_{ℂ} of the image of ϕ_{θ}. It is a reductive algebraic subgroup of K_{ℂ}. The subgroup U_{K}^{x} is simply connected unipotent. In particular, the decomposition of (4) is a Levi decomposition of the algebraic group K_{ℂ}^{x}.

6. Suppose that ϕ_{θ} = ϕ_{ℝ,θ} also intertwines σ_{0} with σ. Then K_{ℂ}^{ϕ_{ℝ,θ}} is preserved by σ, and we may take σ as complex conjugation for a compact real form of this reductive algebraic group. In particular, K_{ℂ}^{x} and K_{ℂ}^{ϕ_{ℝ,θ}} have a common maximal compact subgroup

K^{ϕ_{ℝ,θ}} = Ł_{K}^{x} ∩ K.

These are rather convoluted and technical statements, not easy to absorb at first. One way to think about them is this. The Cartan decomposition says that a reductive group G is topologically a product of a maximal compact subgroup K and a vector space. A consequence of this decomposition is that many nice homogeneous spaces G/H are isomorphic to vector bundles over K/K ∩ H. Said a little more abstractly: a nice homogeneous space for G is a vector bundle over

(maximal compact for G)/(maximal compact for isotropy group).

In order to understand a homogeneous space topologically, we must therefore understand maximal compact subgroups of isotropy groups.

Proposition 4.4 computes a maximal compact subgroup of an isotropy group for a nilpotent adjoint orbit. The natural inference (which is actually true) is that this orbit G · E is isomorphic to a vector bundle over K/K^{ϕ_{ℝ,θ}}; the isomorphism respects the action of K.

Proposition 4.5 analyzes K_{ℂ} · x ⊂ 𝔰_{ℂ}. Notice that K_{ℂ} has exactly the same maximal compact subgroup K as G; and the Proposition says that the maximal compact subgroups of isotropy groups also look the same. The inference (again true) is that K_{ℂ} ·x is isomorphic to a vector bundle over K/K^{ϕ_{ℝ,θ}}; the isomorphism again respects the action of K.

It is not impossibly difficult to calculate the vector bundles appearing in these two settings, and to see that they are isomorphic. The conclusion is that the orbits G · E and K_{ℂ} · x are diffeomorphic as manifolds with K action (assuming that they correspond under the Kostant-Sekiguchi correspondence). Vergne has given a more natural and direct proof of this diffeomorphism. Here is a statement.

Theorem 4.6 ([Vrg]). Suppose G = K · exp(𝔰) is a Cartan decomposition of a reductive group, and E ∈ 𝔤 and x ∈ 𝔰_{ℂ} are nilpotent elements. Assume that the orbits G · E and K_{ℂ} · x correspond under the Kostant-Sekiguchi correspondence (Proposition 4.2). Then there is a K-equivariant diffeomorphism from G · E onto K_{ℂ} · x.

LECTURE 5

Quantizing the action of K

We would like to be able to attach to a nilpotent coadjoint orbit a unitary representation of G. The construction of parabolically induced representations (on hyperbolic orbits) suggests that we might look for a pre-Hilbert space endowed with a representation of G preserving the inner product. The precise construction used for cohomological induction (in the case of elliptic orbits) produces even less: a pre-Hilbert space endowed with a (𝔤, K)-module structure. In the case of nilpotent orbits, we will be able to produce only a space with a representation of K (and even that only under an additional technical hypothesis). The action of 𝔤 must be added later (and we will usually not know how to do it).

So let us fix a nilpotent element

λ_{n} ∈ 𝔤^{∗},  (5.1)(a)

and write E ∈ 𝔤 for its image under the isomorphism of Proposition 2.7. According to Proposition 4.2, we can find a Lie algebra homomorphism ϕ_{ℝ} from 𝔰𝔩(2, ℝ) to 𝔤 with ϕ_{ℝ}(E_{0}) = E (notation (4.1)(c)). After replacing λ_{n} by a conjugate under G, we may assume that

ϕ_{ℝ} = ϕ_{ℝ,θ} intertwines θ_{0} and θ.  (5.1)(b)

Following the prescription of Proposition 4.2, we then define

x = ϕ_{ℝ,θ}(x_{0}) ∈ 𝔰_{ℂ}.  (5.1)(c)

The isomorphism of Proposition 2.7 associates to x a linear functional

λ_{θ} ∈ 𝔰_{ℂ}^{∗},  λ_{θ}(Y) = ⟨x, Y ⟩  (Y ∈ 𝔤).  (5.1)(d)

The element λ_{θ} is not uniquely determined by λ_{n}, but the orbit K_{ℂ}·λ_{θ} is determined by G · λ_{n}. Vergne’s Theorem 4.6 provides a K-equivariant diffeomorphism

G · λ_{n} ≃ K_{ℂ} · λ_{θ}.  (5.1)(e)

The discussion of elliptic orbits in section 3 showed that we should expect to need a little more than the orbit to construct a representation. Specifically, we should have also an “admissible orbit datum at λ_{n}” in the sense of Duflo. For the purposes of these lectures, defining admissible orbit data would be an unnecessary digression. What we need to understand is what becomes of admissible orbit data on G · λ_{n} after they are transferred to K_{ℂ} · λ_{θ}. Here is the key definition.

Definition 5.2. Suppose λ_{θ} ∈ 𝔰_{ℂ}^{∗} is a nilpotent element. Write K_{ℂ}^{λ_{θ}} for the isotropy group at λ_{θ}. Define 2ρ to be the algebraic character of K_{ℂ}^{λ_{θ}} by which it acts on the top exterior power of the cotangent space at λ_{θ} to the orbit:

2ρ(k) = det (Ad^{∗}(k)|_{(𝔨_{ℂ}/𝔨_{ℂ}^{λ_{θ}})^{∗}}) (k ∈ K_{ℂ}^{λ_{θ}}).

The differential of 2ρ is a one-dimensional representation of the Lie algebra 𝔨_{ℂ}^{λ_{θ}}, which we denote also by 2ρ ∈ (𝔨_{ℂ}^{λ_{θ}})^{∗}. We define ρ ∈ (𝔨_{ℂ}^{λ_{θ}})^{∗} to be half of 2ρ. Explicitly,

ρ(Z) = \frac{1}{2} tr(ad^{∗}(Z)|_{(𝔨_{ℂ}/𝔨_{ℂ}^{λ_{θ}})^{∗}}) (Z ∈ 𝔨_{ℂ}^{λ_{θ}}).

An admissible K_{ℂ}-orbit datum at λ_{θ} is an irreducible algebraic representation (τ, V_{τ}) of K_{ℂ}^{λ_{θ}} whose differential is equal to ρ (times the identity on V_{τ}). The element λ_{θ} is called admissible if an admissible orbit datum exists.

Just as in the case of elliptic orbits, admissible orbit data may or may not exist. When they do exist, there is often a one-dimensional admissible datum (τ_{0}, V_{τ_{0}}). In that case it is easy to see that all admissible data are in one-to-one correspondence with irreducible representations of the group of connected components of K_{ℂ}^{λ_{θ}}; the correspondence arises by tensoring with τ_{0}. If G is connected and simply connected, then this component group is just the fundamental group of the orbit K_{ℂ} · λ_{θ}.

Theorem 5.3 (Schwartz; see [VAV], Theorem 7.14). In the setting of (5.1), there is a natural bijection from Duflo’s admissible orbit data at λ_{n} to admissible K_{ℂ}- orbit data at λ_{θ}. In particular, the orbit G · λ_{n} is admissible in Duflo’s sense if and only if the orbit K_{ℂ} · λ_{θ} is admissible.

We now have in hand the ingredients necessary to imitate the construction of a quantization made for elliptic orbits: a complex structure on the orbit, and a holomorphic vector bundle. The difficulty is that the complex structure is preserved only by K, and not by the whole group G; so the space of holomorphic sections of the bundle is a representation only of K. Here is the definition. (The definition would still make sense without the codimension condition imposed here, but it is no longer a reasonable one.)

Definition 5.4. Suppose λ_{n} ∈ 𝔤^{∗} is a nilpotent element, and λ_{θ} ∈ 𝔰_{ℂ}^{∗} corresponds to it under the Kostant-Sekiguchi correspondence. Fix an admissible orbit datum (τ, V_{τ}) at λ_{θ}, and let

V_{τ} = K_{ℂ} ×_{K_{ℂ}^{λ_{θ}}} V_{τ}

be the corresponding algebraic vector bundle on K_{ℂ}·λ_{θ}. Assume that the boundary of \overline{K_{ℂ} · λ_{θ}} (that is, the complement of the open orbit) has complex codimension at least two. Define

X_{K}(λ_{n}, τ) = space of algebraic sections of V_{τ},

an algebraic representation of K_{ℂ}. We call this the quantization of the K action on G · λ_{n} (for the admissible orbit datum τ).

What this definition amounts to is a desideratum for the quantization of the G action on G · λ_{n}: that is, that whatever unitary representation π_{G}(λ_{n}, τ) we associate to these data, we should have

K-finite part of π_{G}(λ_{n}, τ) ≃ X_{K}(λ_{n}, τ).  (5.5)

LECTURE 5. QUANTIZING THE ACTION OF K 33

One might justify this requirement just by analogy with the quantization of elliptic orbits in Proposition 3.5. One of the main points of [VAV] is to provide a stronger justification: roughly speaking, to show that any representation of G satisfying certain rather weak and natural requirements related to G · λ_{n} must come close to satisfying (5.5). (A more precise statement is at (12.2) in [VAV].) There is also a growing collection of examples that support (5.5); a recent one is in [AHV].

In the absence of the codimension condition in Definition 5.4, I do not know in general how to formulate a requirement like (5.5); that is, how to quantize the action of K. For the moment it is enough to notice that the condition is satisfied in a great many interesting examples (see Exercise 13). When G is complex, the coadjoint orbits are complex symplectic manifolds, and therefore of real dimension 4m. Consequently the codimension condition is automatically satisfied in this case.

LECTURE 6

Associated graded modules

In section 5 we have attached to some admissible nilpotent coadjoint orbits certain candidates for the restrictions to K of corresponding unitary representations. In this section we will study the restrictions to K of arbitrary admissible representations, finding a description of more or less the same nature as the conjectural one from the orbit method. Our goal is to understand this general description so well that we can sometimes prove that it coincides with the conjectural one.

Definition 6.1. Suppose G = K exp(𝔰) is a Cartan decomposition of a reductive group. Write

M(𝔤, K) = category of (𝔤, K)-modules of finite length.

The set of equivalence classes of irreducible (𝔤, K)-modules is denoted Ĝ. Similarly, write

A(K) = category of admissible representations of K.

These are the locally finite representations of K (every vector belongs to a finitedimensional K-invariant subspace on which K acts continuously) with the property that each irreducible representation of K occurs finitely often. One of Harish-Chandra’s basic theorems about (𝔤, K)-modules says that any (𝔤, K)-module of finite length belongs to A(K); that is, that there is a forgetful functor

M(𝔤, K) → A(K).

Recall that the Grothendieck group of an abelian category A is the abelian group KA generated by the objects of the category, modulo relations

[B] = [A] + [C]

whenever there is a short exact sequence

0 → A → B → C → 0.

Here and in general we write [A] ∈ KA for the class of the object A in the Grothendieck group.

Both of the module categories above are abelian, so they have Grothendieck groups. Every object in M(𝔤, K) has finite length; it follows easily that KM(𝔤, K) is a free abelian group with generators the irreducible (𝔤, K)-modules. Here is another way to say the same thing: if M is any (𝔤, K)-module of finite length, then

[M] = ∑_{π∈_{G}} m_{π}(M)[V_{π}].

Here m_{π}(M) is a non-negative integer, the number of times that π appears in a Jordan-Hölder series for M. This sum has only finitely many non-zero terms.

The situation for A(K) is a bit more subtle, since the objects do not have finite length. In this case the Grothendieck group is a direct product of copies of ℤ, one for each irreducible representation of K. Explicitly, if N is any admissible representation of K, then

[N] = ∑_{μ∈_{K}} m_{μ}(N)[V_{μ}].

Here m_{μ}(N) is the multiplicity of μ in N, a non-negative integer; but the sum need not be finite.

Finally, write K_{ℂ} for the complexification of K; this is a complex reductive algebraic group. The locally finite representations of K may be identified with the algebraic representations of K_{ℂ}. (Formally, the forgetful functor from the latter to the former is an equivalence of categories.) We may therefore replace K by K_{ℂ} at will in these definitions without changing anything.

We are going to need one more abelian category and its Grothendieck group. We could introduce the category directly, but perhaps it is preferable to begin with the construction that motivates it. So suppose M is a (𝔤, K)-module of finite length. An easy induction on the length shows that M is generated as a (𝔤, K)-module by some finite-dimensional subspace S; in fact we can choose S to be of dimension at most the length of M. Next, the definition of local finiteness for the action of K ensures that S is contained in a finite-dimensional K-invariant subspace M_{0} of M. The compatibility of the K and 𝔤 actions ensures that U(𝔤)M_{0} is K-invariant; since it is also 𝔤 invariant, it must be a (𝔤, K)-submodule of M. Since it contains S, it must be all of M:

M = U(𝔤)M_{0}.  (6.2)(a)

Now the enveloping algebra U(𝔤) has an increasing filtration

ℂ = U_{0}(𝔤) ⊂ ℂ + 𝔤_{ℂ} = U_{1}(𝔤) ⊂ U_{2}(𝔤) ⊂ ⋯;  (6.2)(b)

here U_{n}(𝔤) is the span of products of at most n elements of 𝔤. The subspaces U_{n}(𝔤) are finite-dimensional and Ad(K)-invariant. Now define

M_{n} = U_{n}(𝔤)M_{0}.  (6.2)(c)

This is a finite-dimensional K-invariant subspace of M. Because U_{p}(𝔤)U_{q}(𝔤) ⊂ U_{p+q}(𝔤), we have

U_{p}(𝔤)M_{q} ⊂ M_{p+q}.  (6.2)(d)

The most important fact about the universal enveloping algebra is the Poincaré-Birkhoff-Witt theorem. There are many equivalent formulations (often involving a basis for U(𝔤)) but for us the following coordinate-free version is best.

Theorem 6.3 ([Hum], Theorem 17.3). If U(𝔤) is given the structure of filtered algebra described above, then the associated graded algebra is naturally isomorphic to S(𝔤), the symmetric algebra on the complex vector space 𝔤_{ℂ}. We write

σ_{n}: U_{n}(𝔤)/U_{n−1}(𝔤) → S^{n}(𝔤)

for the isomorphism; here S^{n} denotes homogeneous polynomials of degree n.

The notation is chosen because σ_{n} behaves like a symbol map for differential operators. (This is no accident: if U(𝔤) is identified with left-invariant differential operators on G, then the filtration we have is precisely the degree filtration for operators, and σ can be identified with the symbol of the differential operator.) One can interpret the theorem as saying that multiplication in U(𝔤) is like multiplication of polynomials, up to lower order terms.

Now we want to understand not U(𝔤) itself, but the U(𝔤)-module M. We will see that the module structure is like that of a module for the polynomial ring S(𝔤), modulo lower order terms. Here is the result.

Lemma 6.4. Suppose M is a (𝔤, K)-module of finite length. Construct a filtration of M as in (6.2) above. Write gr M for the associated graded vector space:

(gr M)_{n} = M_{n}/M_{n−1} (n ≥ 0);

here we define M_{−1} = 0. Each of these spaces inherits from M a representation of the group K; so gr M is an admissible locally finite representation of K. Because of (6.2)(d), there are well-defined maps

(gr U)_{p} ⊗ (gr M)_{q} → (gr M)_{p+q},
(u + U_{p−1}) ⊗ (m + M_{q−1}) ↦ um + M_{p+q−1} (u ∈ U_{p}(𝔤), m ∈ M_{q}).

In light of Theorem 6.3, these may be interpreted as maps

S^{p}(𝔤) ⊗ (gr M)_{q} → (gr M)_{p+q}.

These maps make gr M into a finitely generated graded module for S(𝔤), generated by the degree 0 subspace M_{0}. The action of S(𝔤) and the representation of K on gr M satisfy the compatibility condition

k · (p · m) = (Ad(k)p) · (k · m) (k ∈ K, p ∈ S(𝔤), m ∈ gr M).

This lemma is easier to prove than it is to state; so having assumed the burden of stating it, we may safely leave the proof to the reader.

At this point we begin to take advantage of the geometric aspects of algebraic geometry. Recall first of all that the maximal ideals in the polynomial ring S(𝔤) correspond precisely to the complex-linear functionals on 𝔤. We will write a little informally

Spec S(𝔤) = 𝔤_{ℂ}^{∗},  (6.5)(a)

having in mind the identification of prime ideals with irreducible subvarieties of 𝔤_{ℂ}^{∗}. If X is any subset of S(𝔤), we write

V(X) = {λ ∈ 𝔤_{ℂ}^{∗} | p(λ) = 0, all p ∈ X},  (6.5)(b)

the associated variety of X. This is the same as the associated variety of the ideal generated by X. If N is a finitely generated S(𝔤)-module, we can define

V(N) = V(Ann N),  (6.5)(c)

the associated variety of N. A standard argument from commutative algebra shows that V(N) coincides with the support of N, the set of all prime ideals for which the localization N_{P} is non-zero. (For details we refer to [VAV], section 2.) Our next goal is to say something about the associated variety of gr M in the setting of Lemma 6.4. By definition, that means we need to find some elements in Ann gr M.

Because K preserves M_{n}, the differentiated action of 𝔨 also preserves M_{n}. By the definition of (𝔤, K)-module, this is just the restriction to 𝔨 of the 𝔤 action. By the definition of the S(𝔤) action in Lemma 6.4, it follows that 𝔨 acts by 0 on gr M. (Elements of 𝔤 are supposed to raise degree by 1 in M.) Consequently

gr M is a finitely generated graded S(𝔤/𝔨) module.  (6.6)(a)

In terms of associated varieties, this says

V(gr M) ⊂ (𝔤_{ℂ}/𝔨_{ℂ})^{∗} ⊂ 𝔤_{ℂ}^{∗}.  (6.6)(b)

Write ℨ(𝔤) for the center of the universal enveloping algebra U(𝔤). This is a polynomial algebra on rank 𝔤 generators. It inherits from U(𝔤) a filtration; the associated graded algebra is

gr ℨ(𝔤) ≃ S(𝔤)^{G_{0}},  (6.6)(c)

the algebra of polynomials invariant under the adjoint action of G_{0}. Because M is a U(𝔤)-module of finite length, it must be annihilated by an ideal I ⊂ ℨ(𝔤) of finite codimension. (The codimension of I may be bounded by the length of M as a U(𝔤)-module.) Since I · M = 0, it follows at once that

(gr I) · (gr M) = 0.  (6.6)(d)

Now gr I is a graded ideal of finite codimension in S(𝔤)^{G_{0}}. By an elementary argument, gr I must contain some power of the augmentation ideal J = ∑_{n>0} S^{n}(𝔤)^{G_{0}}:

gr I ⊃ J^{N},  some N > 0.  (6.6)(e)

In terms of associated varieties, this says

V(gr M) ⊂ V(J^{N}) = V(J).  (6.6)(f)

Now the associated variety of J is not so obvious as that of 𝔨. Fortunately Kostant has computed it; here is his result.

Theorem 6.7 ([VAV], Theorem 5.7). Suppose G is a reductive Lie group, and J ⊂ S(𝔤) is the collection of G_{0}-invariant polynomials without constant term. Then the associated variety of J is the cone N^{∗} of nilpotent elements in 𝔤_{ℂ}^{∗}.

Suppose in particular that M is a (𝔤, K)-module of finite length, endowed with a filtration as in (6.2). Then the associated variety of gr M is contained in the cone N_{θ}^{∗} of nilpotent elements in (𝔤_{ℂ}/𝔨_{ℂ})^{∗}.

Now we can define the new abelian category mentioned after Definition 6.1.

Definition 6.8. Let C(𝔤, K) be the category of finitely generated S(𝔤/𝔨)-modules N carrying locally finite representations of K, and subject to the following requirements: first, the compatibility requirement

k · (p · n) = (Ad(k)p) · (k · m) (k ∈ K, p ∈ S(𝔤), n ∈ N);

and second, the support requirement

V(N) ⊂ N_{θ}^{∗}

(the cone of nilpotent elements in (𝔤_{ℂ}/𝔨_{ℂ})^{∗}). This is an abelian category. In light of Kostant’s Theorem 6.7, the support requirement is equivalent to requiring J (the set of G_{0}-invariant polynomials without constant term) to act nilpotently on N.

Because of the remarks at the end of Definition 6.1, it is equivalent to consider the corresponding category C(𝔤, K_{ℂ}) of modules with an algebraic action of K_{ℂ}.

Proposition 6.9. Suppose N is an object in C(𝔤, K). Then N is admissible (finite multiplicities) as a representation of K. In particular, there is a forgetful functor

C(𝔤, K) → A(K).

Sketch of proof. An easy argument reduces the lemma to the case G connected and

N = S(𝔤)/(𝔨 + J)S(𝔤).

This case was treated by Kostant and Rallis in [KR], who showed that the representation μ of K appears with multiplicity equal to the dimension of the space of M-fixed vectors in μ. (Here M ⊂ K comes from the Langlands decomposition MAN of a minimal parabolic subgroup of G.) □

We would like the operation “associated graded” to define something like a functor from M(𝔤, K) (Definition 6.1) to C(𝔤, K). The difficulty is that our definition of gr M required the construction of a filtration of M, and the construction required at least one non-functorial choice (of the finite-dimensional generating subspace S of M). Fortunately this choice does not affect gr M too much.

Proposition 6.10 ([VAV], Proposition 2.2). Suppose M is a (𝔤, K)-module of finite length, and {M_{n}} and {M′_{n}} are two filtrations constructed as in (6.2). Then the corresponding associated graded modules gr M and gr′ M belong to C(𝔤, K). They define the same class in the Grothendieck group KC(𝔤, K):

[gr M] = [gr′ M].

More precisely, each associated graded module has a finite filtration so that the corresponding subquotients (after rearrangement) are pairwise isomorphic in C(𝔤, K). Consequently there is a well-defined group homomorphism

[gr]: KM(𝔤, K) → KC(𝔤, K).

The corresponding forgetful functors to KA(K) make a commutative triangle. That is, the K-multiplicities in M and gr M coincide.

That gr M belongs to C(𝔤, K) follows from Lemma 6.4, Theorem 6.7, and (6.6)(a). The last statement about K-multiplicities is easy. For the rest of the proof, we refer to [VAV].

LECTURE 7

A good basis for associated graded modules

Proposition 6.10 provides a very useful way of working with the Grothendieck group of finite-length Harish-Chandra modules. Here is why.

Theorem 7.1. The homomorphism [gr] of Proposition 6.10 is a surjection from the Grothendieck group of Harish-Chandra modules onto the Grothendieck group of finitely generated compatible (S(𝔤/𝔨), K)-modules supported on the nilpotent cone. Its kernel is precisely the group of virtual Harish-Chandra modules with K-multi-plicities equal to zero; that is, the kernel of the forgetful functor

KM(𝔤, K) → KA(K).

Corollary 7.2. The possible K-multiplicities for virtual Harish-Chandra modules (that is, the image of the forgetful functor in Theorem 7.1) are precisely the Kmultiplicities in virtual (S(𝔤/𝔨), K)-modules supported on the nilpotent cone.

We are not going to give a detailed proof of Theorem 7.1. For the moment we are concerned mostly with Corollary 7.2. In order to extract interesting information about Harish-Chandra modules from this Corollary, we need to understand better the Grothendieck group KC(𝔤, K). This is a subtle matter, because the objects of C(𝔤, K) (unless they are finite-dimensional) do not have finite length. (The Noetherian property guarantees that ascending chains of submodules must be finite, but infinite descending chains are easy to construct.) We cannot hope to use irreducible objects as a basis for the Grothendieck group. Instead we will begin with a theorem of Kostant and Rallis.

Theorem 7.3 (Kostant-Rallis; see [VAV], Corollary 5.22). Suppose G is a real reductive Lie group with maximal compact subgroup K, and K_{ℂ} is the complexification of K. Write N_{θ}^{∗} for the cone of nilpotent elements in (𝔤_{ℂ}/𝔨_{ℂ})^{∗}. Then K_{ℂ} acts on N_{θ}^{∗} with finitely many orbits.

Choose representatives λ_{1}, …, λ_{r} for these orbits, and define

H_{i} = K_{ℂ}^{λ_{i}}  (7.4)(a)

to be the isotropy group at λ_{i}. We would like to disassemble the category C(𝔤, K) using these orbits. If the orbits were closed, there would be no difficulty: the category would be a direct sum of subcategories consisting of modules supported on a single orbit. Each of these categories in turn would be equivalent to the category of finite-dimensional representations of an isotropy group H_{i}.

However, the orbits are not closed, and no such direct sum decomposition is possible. What we can do instead is filter the category C(𝔤, K) using supports of modules. If an orbit K_{ℂ} · λ_{j} appears in the support of a module, and λ_{i} is in the closure of K_{ℂ} · λ_{j}, then λ_{i} must also appear in the support. The support of any non-zero module N in C(𝔤, K) may therefore be written uniquely as a union of orbit closures \overline{K_{ℂ} · λ_{i}}, where λ_{i} is not in the closure of any other orbit in the support. These orbit closures are the components of V(N) as an algebraic variety.

This discussion is intended to motivate the following definition. For each i, define a subcategory of C(𝔤, K) as follows.

C(𝔤, K)_{i} = \{N ∈ C(𝔤, K) | λ_{i} ∈ (\overline{K_{ℂ} · λ_{j}} − K_{ℂ} · λ_{j}) ⇒ λ_{j} ∉ V(N)\}. (7.4)(b)

These are the modules for which \overline{K_{ℂ} · λ_{i}} is a component of V(N), possibly with multiplicity zero. Next

C(𝔤, K)_{i}^{0} = {N ∈ C(𝔤, K) | λ_{i} ∉ V(N)}.  (7.4)(c)

These are the modules (automatically in C(𝔤, K)_{i}) for which K_{ℂ} · λ_{i} does not meet the associated variety.

Now any finitely generated module (say for a finitely generated commutative algebra over an algebraically closed field) is something like a vector bundle over a dense open set in its support. We should therefore expect modules in C(𝔤, K)_{i} to look like K_{ℂ}-equivariant vector bundles on K_{ℂ} · λ_{i} ≃ K_{ℂ}/H_{i}. Such a vector bundle is the same thing as an algebraic representation of H_{i}. The conclusion is that modules in C(𝔤, K)_{i} should correspond to representations of H_{i}, with modules in C(𝔤, K)_{i}^{0} corresponding to the zero representation. Making these ideas precise requires some passage to associated graded objects. (For example, a module is not necessarily annihilated by the ideal of its support, but only by some power of that ideal. We need a finite filtration of the module so that the subquotients are actually annihilated by the ideal of the support.) As a result, we get not a representation of H_{i} but a rather a class in some Grothendieck group. Here is some formalism useful for making this precise.

Lemma 7.5. Suppose H is a complex algebraic group, with Levi decomposition H = LU. (Here U is the unipotent radical of H, a connected normal subgroup, and L is reductive.) Write F(H) for the category of finite-dimensional algebraic representations of H, and Ĥ for the set of equivalence classes of irreducible algebraic representations of H.

1. Every irreducible representation of H is trivial on U; so restriction to L defines a bijection Ĥ ≃ L̂.

2. The irreducible representations of H constitute a basis for the Grothendieck group KF(H). In particular, the functor of restriction to L defines an isomorphism KF(H) ≃ KF(L).

The class in KF(H) of a representation of H is called a genuine virtual representation. According to the lemma, this is the same thing as a formal finite sum of irreducible representations.

Proposition 7.6 ([VAV], Theorem 2.13). In the setting of (7.4), attached to any module N ∈ C(𝔤, K)_{i} there is a genuine virtual representation χ(λ_{i}, N) ∈ KF(H_{i}). This correspondence descends to an isomorphism of Grothendieck groups

KC(𝔤, K)_{i}/KC(𝔤, K)_{i}^{0} ≃ KF(H_{i}).

There are various ways to build this into a statement about the full Grothendieck group KC(𝔤, K). One simple one is to define

C(𝔤, K)_{[m]} = {N ∈ C(𝔤, K) | dim V(N) ≤ m}.  (7.7)

This is a finite increasing family of subcategories of C(𝔤, K); if we define the empty set to have dimension -1, then C(𝔤, K)_{[−1]} is zero. These subcategories induce a finite increasing filtration of the Grothendieck group, and we can describe the subquotients.

Corollary 7.8. If dim K_{ℂ} · λ_{i} = m, then C(𝔤, K)_{[m]} ⊂ C(𝔤, K)_{i}. The maps of Proposition 7.6 induce isomorphisms of Grothendieck groups

KC(𝔤, K)_{[m]}/KC(𝔤, K)_{[m−1]} ≃ ∑_{{i \atop dim K_{ℂ}·λ_{i}=m}} KF(H_{i}).

The Grothendieck groups KF(H_{i}) all have natural bases, parametrized by irreducible representations of H_{i}. We would like to lift these back to a basis for the Grothendieck group KC(𝔤, K)_{i}/KC(𝔤, K)_{i}^{0}. The following proposition says that this is always possible, and sometimes can be done canonically.

Proposition 7.9. In the setting of (7.4), suppose that (τ, V_{τ}) is an irreducible representation of the stabilizer H_{i} of λ_{i}. Then there is an object N(λ_{i}, τ) ∈ C(𝔤, K) with the following properties. First, V(N(λ_{i}, τ)) = \overline{K_{ℂ} · λ_{i}}. In particular, N(λ_{i}, τ) belongs to C(𝔤, K)_{i}. Second, the map of Proposition 7.6 sends N(λ_{i}, τ) to χ(λ_{i}, N(λ_{i}, τ)) = [τ].

If we choose such an object N(λ_{i}, τ) for every irreducible representation of every H_{i}, then the classes [N(λ_{i}, τ)] constitute a basis for the Grothendieck group KC(𝔤, K).

Suppose that the boundary of \overline{K_{ℂ} · λ_{i}} (that is, the complement of the orbit in its closure) has codimension at least two. Then there is a canonical choice for N(λ_{i}, τ). As representations of K_{ℂ}, we have

N(λ_{i}, τ) ≃ Ind_{H_{i}}^{K_{ℂ}}(τ)

in this case.

Proof. We begin with the algebraic vector bundle

V_{τ} = K_{ℂ} ×_{H_{i}} V_{τ}  (7.10)(a)

over the orbit K_{ℂ} · λ_{i} ≃ K_{ℂ}/H_{i}. Define

M(λ_{i}, τ) = space of algebraic sections of V_{τ} .  (7.10)(b)

This space carries an algebraic representation of K_{ℂ}; equivalently, a locally finite representation of K. At the same time, it is a module for S(𝔤/𝔨): a polynomial p acts by multiplication by the restriction of p to the orbit. (This statement is supposed to be elementary and clear, and from a geometric point of view it should be; but for a more group-oriented person there is a possibility of confusion. Such a person may prefer to identify a section m of V_{τ} with an algebraic function

μ: K_{ℂ} → V_{τ} ,  μ(kh) = τ(h^{−1})μ(k)  (k ∈ K_{ℂ}, h ∈ K_{ℂ}^{λ_{i}}).  (7.10)(c)

How does p act on such a function? To see that, we define from p a complex-valued function

π: K_{ℂ} → ℂ,  π(k) = p(k · λ_{i}).  (7.10)(d)

The function π is right-invariant under K_{ℂ}^{λ_{i}}. Consequently πμ is a function from K_{ℂ} to V_{τ} satisfying the same transformation law as μ; that is, πμ may be regarded as a section of V_{τ}. This is the section p · m.)

The compatibility condition of Definition 6.8 is very easy to check. Because the orbit is quasiaffine (open in an affine algebraic variety) the bundle has many sections. In particular, the sections span the fiber at each point:

{m(λ_{i}) | m ∈ M(λ_{i}, τ)} = V_{τ} .  (7.10)(e)

(In the more group-theoretic description (7.10)(c) of sections of V_{τ}, the left side here is

{μ(e) | μ: K_{ℂ} → V_{τ} as in (7.10)(c)}.

This description does not make it clear that we get all of V_{τ} from algebraic functions μ; for that the algebraic geometry point of view is essential.)

The only thing wrong with M(λ_{i}, τ) is that it need not be finitely generated as a module for S(𝔤/𝔨). To get around this, we choose a finite-dimensional K-invariant subspace M_{0} of M(λ_{i}, τ) with the property that

{m(λ_{i}) | m ∈ M_{0}} = V_{τ} .  (7.10)(f)

Such a subspace exists by (7.10)(e). Define

N(λ_{i}, τ) = S(𝔤/𝔨) · M_{0},  (7.10)(g)

a finitely generated K-invariant submodule of M(λ_{i}, τ). It is not difficult to show that N(λ_{i}, τ) satisfies the requirements of the proposition. If we identify N(λ_{i}, τ) with a coherent sheaf of modules over \overline{K_{ℂ} · λ_{i}}, then M(λ_{i}, τ) may be identified with the sections over the open set K_{ℂ} · λ_{i}:

M(λ_{i}, τ) ≃ N(λ_{i}, τ)(K_{ℂ} · λ_{i}).  (7.10)(h)

This description and some commutative algebra (see [EGA], Proposition 5.11.1) imply that M(λ_{i}, τ) is finitely generated if and only if the boundary of \overline{K_{ℂ} · λ_{i}} has codimension at least 2. When that condition is satisfied, we may therefore take N(λ_{i}, τ) = M(λ_{i}, τ). This choice is in a certain sense maximal, if we require also that N(λ_{i}, τ) have no embedded associated primes; that is, no submodules with much larger annihilators. □

Exercise 14 looks at the simplest example of this construction.

LECTURE 8 Proving unitarity

In section 7, we saw how to use the geometry of the nilpotent cone N_{θ}^{∗} to control the possible K-types of Harish-Chandra modules. In fact the work of Harish-Chandra and Langlands already provided a large and powerful array of tools for controlling the possible K-types of Harish-Chandra modules. In this section we will describe some of that classical theory, and see how comparing the two approaches can lead to constructions of unitary representations.

To begin, we need the notion of representations with real infinitesimal character. A precise definition appears in Definition 8.5 below. For most purposes it is enough to understand some properties of real infinitesimal character. First, discrete series representations have real infinitesimal character. Second, suppose that P = MAN is a parabolic subgroup of G, δ is a representation of M with real infinitesimal character, and ν is a character of A. Then Ind_{P}^{G}(δ ⊗ ν ⊗ 1) has real infinitesimal character if and only if ν is real-valued.

Theorem 8.1. Suppose G is a reductive Lie group, and M is an irreducible tempered Harish-Chandra module with real infinitesimal character. Then M has a unique lowest K-type μ(M), which occurs with multiplicity one. This defines a bijection from the set of equivalence classes of irreducible tempered Harish-Chandra modules with real infinitesimal character onto K̂. We may write M(μ) for the representation corresponding to μ ∈ K̂.

For linear groups in Harish-Chandra’s class, this theorem is essentially proved in [VG]. The extension to general reductive groups is straightforward; what is perhaps surprising for experts is that nothing goes wrong.

Theorem 8.2. The classes of irreducible tempered representations with real infinitesimal character form a basis for the quotient group

KM(𝔤, K)/(kernel of restriction to K).

By Theorem 7.1, the images

{[gr M] | M tempered irreducible with real infinitesimal character}

form a basis of KC(𝔤, K).

Even in the simplest cases, the bases provided by Proposition 7.9 and Theorem 8.2 are completely different. Exercise 15 looks at an example.

We want to use properties of the change-of-basis matrix from the basis of Proposition 7.9 to the one of Theorem 8.2 (for the Grothendieck group KC(𝔤, K)) in order to make deductions about unitary representations. (Of course it is not yet clear how we can hope to do that.) Because one basis is parametrized by irreducible representations of K, it is convenient to make use of highest weight theory for K. We therefore fix a Cartan subalgebra

𝔱 ⊂ 𝔨,  (8.3)(a)

and define

T = {t ∈ K | Ad(t)|_{𝔱} = 1},  (8.3)(b)
T′ = {n ∈ K | Ad(n)(𝔱) = 𝔱} = N_{K}(T),  (8.3)(c)
W (K, T) = T′/T.  (8.3)(d)

We may regard this Weyl group as a group of automorphisms of 𝔱 or of T. The identity component of T is a compact torus, but T may be non-abelian. We can also construct from 𝔱 a fundamental Cartan subalgebra and subgroup of G:

𝔥^{f} = {Z ∈ 𝔤 | [Z, 𝔱] = 0},  H^{f} = {h ∈ G | Ad(h)|_{𝔥^{f}} = 1}.  (8.3)(e)

Proposition 8.4. Suppose μ ∈ K̂. The restriction of μ to T (cf. (8.3)) contains a unique W (K, T) orbit of extremal representations. If μ_{0} ∈ T̂ is such an extremal representation, write μ_{1} ∈ i𝔱^{∗} for its differential. These weights form a single W (K, T) orbit in i𝔱^{∗}. The corresponding set of weights λ(μ_{1}) ∈ i𝔱^{∗} constructed in [VG] is therefore a single W (K, T) orbit. The infinitesimal character of M(μ) is represented by the weights in this orbit, extended to be θ-fixed complex linear functionals on 𝔥^{f}.

We will not recall here the construction of λ(μ_{1}) in [VG]. One case is very simple, however: if G is a complex reductive group, then λ(μ_{1}) = μ_{1}.

We will be comparing “lengths” of infinitesimal characters, using the corresponding weights in (𝔥^{f})^{∗}. A minor technical difficulty is that the bilinear form of Proposition 2.7 is not (after complexification) positive definite on (𝔥^{f})^{∗}. The form is positive definite on the real span of the roots; so it is convenient to use something like that restriction to measure length. Here is the relevant definition, taken from [VG], Definition 5.4.11.

Definition 8.5. Suppose 𝔥^{f} is a fundamental Cartan subalgebra as in (8.3). Write

𝔱 = 𝔥^{f} ∩ 𝔨, 𝔞^{f} = 𝔥^{f} ∩ 𝔰

for its decomposition into the +1 and −1 eigenspaces of the Cartan involution θ. The canonical real part of 𝔥_{ℂ}^{f} is the subspace

RE 𝔥_{ℂ}^{f} = i𝔱 + 𝔞^{f}.

(Notice that the roots of 𝔥_{ℂ}^{f} take real values on this subspace, and that the form of Proposition 2.7 is real-valued and positive definite there.) The canonical real part of ξ ∈ (𝔥_{ℂ}^{f})^{∗} is the element RE ξ characterized by the requirement that RE ξ takes real values on RE 𝔥_{ℂ}^{f}, and ξ − RE ξ takes purely imaginary values there.

An infinitesimal character is called real if it is represented by a weight ξ equal to its canonical real part; that is, if ξ takes real values on RE 𝔥_{ℂ}^{f}.

Definition 8.6. Suppose that [N] ∈ KC(𝔤, K). Rewrite [N] in the basis of Theorem 8.2 as

[N] = ∑_{μ∈_{K}} m_{μ}(N)[M(μ)].

Attach to [N] a finite collection of W (K, T) orbits in i𝔱^{∗} by

Λ([N]) = {λ(μ) | μ ∈ K̂, m_{μ}(N) ≠ 0}.

(Recall that λ(μ) represents the infinitesimal character of M(μ).) This collection is non-empty if and only if [N] ≠ 0. If [N] ≠ 0, we can therefore define

∥ [N] ∥ = max_{λ∈Λ([N])} ∥λ∥,

the infinitesimal character size of [N].

The definition is interesting (and the terminology is justified) because of the following result.

Proposition 8.7. Suppose [M] ∈ KM(𝔤, K) is a virtual Harish-Chandra module such that [gr M] = [N] is a non-zero element of KC(𝔤, K). Write

[M] = ∑_{π∈_{G}} m_{π}(M)[V_{π}].

Then there is an irreducible π ∈ Ĝ with m_{π}(M) ≠ 0 and the following additional property. Pick a representative ξ ∈ (𝔥_{ℂ}^{f})^{∗} for the infinitesimal character of π. Then the canonical real part of ξ (Definition 8.5) is at least as large as ∥ [N] ∥:

∥ RE ξ∥ ≥ ∥ [N] ∥.

The kind of property that we want to prove is this: if the support of N is small, then ∥ [N] ∥ must be large. Here is the formal setting.

Definition 8.8. Suppose B is a closed K_{ℂ}-invariant subset of N_{θ}^{∗}. (The notation is chosen because we will most often want B to be the boundary \overline{S} − S of a single orbit S.) Consider the subcategory of C(𝔤, K) consisting of modules supported on B:

C(𝔤, K)(B) = {M ∈ C(𝔤, K) | V(M) ⊂ B}.

The Grothendieck group of this category is the subgroup of KC(𝔤, K) spanned by the various [N(λ_{i}, τ)] (Proposition 7.9) with λ_{i} ∈ B.

Set

∥B∥ = min_{0≠[N]∈KC(𝔤,K)(B)} ∥ [N] ∥,

the infinitesimal character size of B.

Corollary 8.9. Suppose M is an irreducible Harish-Chandra module for G, and ξ ∈ (𝔥_{ℂ}^{f})^{∗} represents the infinitesimal character of M. Then the canonical real part of ξ is at least as large as the infinitesimal character size of V(gr M).

Notice that the infinitesimal character size of B is defined in terms of the change of basis matrix from Proposition 7.9 to Theorem 8.2: it is really a commutative algebra object. The Corollary gets interesting representation-theoretic information about M (a lower bound on its infinitesimal character) from this commutative algebra calculation.

In order to say something about unitarity, we need some ideas from [VoCI].

Definition 8.10. Suppose M ∈ M(𝔤, K) has an invariant Hermitian form ⟨, ⟩_{M}. Write m_{μ}(M) for the multiplicity in M of μ ∈ K̂ (Definition 6.1). As is explained in [VoCI], the signature of the form gives rise to three non-negative integers

p_{μ}(M), q_{μ}(M), z_{μ}(M), p_{μ}(M) + q_{μ}(M) + z_{μ}(M) = m_{μ}(M).

The form is non-degenerate if and only if z(M) = 0, positive semidefinite if and only if q(M) = 0, and so on. We can therefore define genuine virtual representations

[M]^{+}, [M]^{−}, [M]^{0} ∈ KA(K),

by

[M]^{+} = ∑_{μ∈_{K}} p_{μ}(M)[V_{μ}], [M]^{−} = ∑_{μ∈_{K}} q_{μ}(M)[V_{μ}],

and so on.

Theorem 8.11 ([VoCI]). Suppose M ∈ M(𝔤, K) has an invariant Hermitian form; use the notation of Definition 8.10. The virtual K representations [M]^{±} and [M]^{0} are (restrictions to K of) virtual tempered representations of G of real infinitesimal character. According to Theorem 8.2, we may therefore regard them as well-defined virtual equivariant modules on the nilpotent cone:

[M]^{+}, [M]^{−}, [M]^{0} ∈ KC(𝔤, K).

Each infinitesimal character for a tempered representation appearing in [M]^{±} or [M]^{0} is bounded above by the canonical real part of the infinitesimal character of an irreducible constituent of M. In particular, if M has infinitesimal character ξ, then

∥ RE ξ ∥ ≥ ∥ [M]^{±}∥.

Notice that the virtual modules [M]^{±} and [M]^{0} all have non-negative Kmultiplicities, and their sum is the genuine virtual module [gr M]. It is therefore natural to expect that [M]^{±} and [M]^{0} are themselves genuine virtual modules. It is very likely that this follows from the proof of Theorem 8.11, but I have not yet checked carefully. I will therefore be conservative and label it as a conjecture.

Conjecture 8.12. In the setting of Theorem 8.11, the virtual modules [M]^{±} and [M]^{0} are represented by actual modules N^{±} and N^{0} in C(𝔤, K).

At last we can use these ideas to say something about unitary representations.

Theorem 8.13. Suppose M is a (𝔤, K) module of finite length and infinitesimal character ξ, carrying a non-degenerate invariant Hermitian form ⟨, ⟩_{M}. Assume that

1. The associated variety V(gr M) is the closure of single orbit S = K_{ℂ} · λ.

Write

B = \overline{S} − S

for the boundary of this orbit.

2. The genuine virtual representation τ = χ(λ, gr M) defined by Proposition 7.6 is irreducible.

3. The canonical real part of the infinitesimal character of M is strictly smaller than the infinitesimal character size of B (Definition 8.8):

∥ RE ξ∥ < ∥B∥.

4. Conjecture 8.12 holds for M.

Then M is irreducible and the Hermitian form on M is definite, so M is the Harish-Chandra module of an irreducible unitary representation of G.

Suppose in addition that the module N(λ, τ) of Proposition 7.9 has infinitesimal character size less than ∥B∥:

∥ [N(λ, τ)] ∥ < ∥B∥.

Then [gr M] = [N(λ, τ)]. In particular, if τ is an admissible orbit datum at λ (Definition 5.2), then M satisfies the desideratum at (5.5) to be a quantization of the corresponding real nilpotent coadjoint orbit.

Proof. Fix modules N^{±} as in Conjecture 8.12. (Since the form is non-degenerate, the radical is zero, so N^{0} = 0.) Because these sum to gr M in the Grothendieck group, it follows that V(N^{±}) ⊂ V(gr M). We therefore have well-defined genuine virtual representations χ(λ, N^{±}), and

χ(λ, gr M) = χ(λ, N^{+}) + χ(λ, N^{−}).

On the other hand, assumption (2) in the theorem says that the left side is irreducible. It follows that one of the terms on the right is zero. Possibly after replacing ⟨, ⟩_{M} by its negative (which interchanges N^{+} and N^{−}), we conclude that

χ(λ, N^{+}) = χ(λ, gr M), χ(λ, N^{−}) = 0.

By Proposition 7.6, it follows that V(N^{−}) ⊂ B.

We are trying to prove that ⟨, ⟩_{M} is positive definite. Suppose not; that is, that N^{−} ≠ 0. By Definition 8.8 and what we have just seen about the support of N^{−},

∥ [N^{−}] ∥ ≥ ∥B∥.

According to Theorem 8.11,

∥ RE ξ ∥ ≥ ∥ [N^{−}] ∥.

These two inequalites together contradict the assumption in (3) of the theorem. The contradiction proves the positivity of the Hermitian form. A similar argument proves that M must be irreducible.

Finally, we consider the claim that [gr M] = [N(λ, τ)]. According to Propositions 7.6 and 7.9, the difference [gr M] − [N(λ, τ)] must be in the Grothendieck group of modules supported on B. Suppose it is non-zero. Assumption (3) in the theorem and the assumption about the infinitesimal character size of N(λ, τ) imply that

∥[gr M] − [N(λ, τ)]∥ < ∥B∥,

contradicting Definition 8.8. The virtual module must therefore be zero, as we wished to show. □

Theorem 8.13 produces irreducible unitary representations attached to a nilpotent coadjoint orbit; so we should consider how to go about verifying its hypotheses.

Arranging condition (1) is relatively straightforward; here is a very brief sketch. Begin with the corresponding complex group orbit

S_{ℂ} = Ad(𝔤_{ℂ}) · λ ⊂ 𝔤_{ℂ}^{∗};  (8.14)(a)

here Ad(𝔤_{ℂ}) is the identity component of the automorphism group of 𝔤_{ℂ}. Choose an infinitesimal character ξ (cleverly!) in such a way that the maximal primitive ideal J_{ξ}^{max} of infinitesimal character ξ has associated variety \overline{S_{ℂ}}. (Methods for choosing such a ξ may be found in [BV].) Finally let M be any irreducible Harish-Chandra module with annihilator J_{ξ}^{max}. (Methods for constructing such modules in the case of complex groups may also be found in [BV]; similar ideas apply to real groups.) Then V(gr M) will be the closure of the union of some of the nilpotent K_{ℂ} orbits in S_{ℂ} ∩ (𝔤/𝔨)^{∗}. This is close to (1).

The irreducibility condition in (2) of Theorem 8.13 is a subtle point, and I will not try to say more about it here; but the methods described below for studying (3) are helpful.

For condition (3), we already know ξ explicitly from arranging (1); so the whole problem is to find an explicit lower bound ∥B∥ ≥ C (Definition 8.8). This problem has two parts. The modules involved in the definition of ∥B∥ are integer combinations of various N(λ′, τ′) (Proposition 7.9) with λ′ ∈ B. The first part of the problem is to find more or less explicit formulas

[N(λ′, τ′)] = ∑_{μ∈_{K}} m_{μ}(λ′, τ′)[M(μ)].  (8.14)(b)

This is an algebraic geometry problem; it can be approached by the technique McGovern introduced for studying functions on nilpotent orbits. One introduction to this technique is in [AHV]. In particular, one hopes to find in each of these formulas a term M(μ) with m_{μ}(λ′, τ′) ≠ 0 and

∥λ(μ)∥ ≥ C.

The second part of the problem is to check that some of these large terms M(μ) do not cancel when we form integer combinations

∑_{λ′∈B, τ′∈_{K^{λ′}}} n_{λ′,τ′}[N(λ′, τ′)].  (8.14)(c)

Once we know the formulas in (8.14)(b), this is a question of linear algebra. Finally, I hope that verification of (4) is just a matter of examining [VoCI]. To conclude, here is the simplest example of a computation of ∥B∥. This result

controls the infinitesimal character of a virtual Harish-Chandra module with finite K-multiplicities. The most obvious way to produce such a virtual module is as an integer combination of finite-dimensional Harish-Chandra modules. There is an obvious lower bound for the infinitesimal character of such a module. The theorem says that more exotic methods of producing the modules cannot give smaller infinitesimal character.

Theorem 8.15. Suppose B = {0} ⊂ (𝔤/𝔨)^{∗}. Then the infinitesimal character size of B (Definition 8.7) is equal to the length of ρ, half the sum of a set of positive roots.

The trivial representation M of G has infinitesimal character ρ and associated variety {0}, so the inequality B ≤ ∥ρ∥ follows from Corollary 8.9. We consider therefore the opposite inequality.

Proof for G complex connected. The objects [N] appearing in Definition 8.8 are integer combinations of N(0, τ), with τ an irreducible representation of the isotropy group K_{ℂ} of 0. By inspection of Proposition 7.9, N(0, τ) is just the irreducible representation (τ, V_{τ}) of K, equipped with the module structure in which a polynomial p ∈ S(𝔤/𝔨) acts by its value at 0. We need to express N(0, τ) (as a representation of K) as an integer combination of the tempered representations M(μ).

In Theorem 8.1 those representations are parametrized by irreducible representations μ of K; but now it will be more convenient to parametrize them by (W (K, T) group orbits of) elements of T̂ (Proposition 8.4). Write H^{f} = T A for the corresponding Cartan in G, and B = T AN for a Borel subgroup. Then the tempered representation corresponding to γ ∈ T̂ is

M(γ) = Ind_{B}^{G}(γ ⊗ 1 ⊗ 1);  M(γ)|_{K} = Ind_{T}^{K}(γ).  (8.16)(a)

Write

Δ^{+}(𝔨_{ℂ}, 𝔱_{ℂ}) ⊂ T̂  (8.16)(b)

for a set of positive roots of T in 𝔨_{ℂ}, and

2ρ_{c} = ∑_{α∈Δ^{+}(𝔨_{ℂ},𝔱_{ℂ})} α.  (8.16)(c)

Then the differential of 2ρ_{c} is equal to ρ (for a certain choice of positive roots of 𝔥_{ℂ}^{f}); so the inequality we are trying to prove may be written as

∥B∥ ≥ ∥2ρ_{c}∥.  (8.16)(d)

Write γ_{τ} ∈ T̂ for the highest weight of τ. A version of the Weyl character formula is

τ = ∑_{S⊂Δ^{+}(𝔨_{ℂ},𝔱_{ℂ})}(−1)^{|S|} Ind_{T}^{K}(γ_{τ} + 2ρ(S)).  (8.16)(e)

Here 2ρ(S) is the sum of the roots in S. Combining this with (8.16)(a) gives

[N(0, τ)] = ∑_{S⊂Δ^{+}(𝔨_{ℂ},𝔱_{ℂ})}(−1)^{|S|}M(γ_{τ} + 2ρ(S)).  (8.16)(f)

Recall from the remark after Proposition 8.4 that λ(γ) is just the differential of γ. It is a standard and elementary fact about compact group representations that the longest of the weights appearing in (8.16)(f) is γ_{τ} + 2ρ_{c}:

[N(0, τ)] = ±M(γ_{τ} + 2ρ_{c}) + terms M(γ′) with λ(γ′) strictly shorter. (8.16)(g)

Suppose now that N = ∑_{τ} m_{τ} [N(0, τ)] is a non-zero finite integer combination as in Definition 8.8. Choose τ_{0} so that ∥γ_{τ_{0}} + 2ρ_{c}∥ is maximal subject to m_{τ} ≠ 0. It follows from (8.16)(g) that

∥ [N] ∥ = ∥γ_{τ_{0}} + 2ρ_{c}∥ ≥ ∥2ρ_{c}∥.  (8.16)(h)

This proves (8.16)(d), which is what we wished to show. □

EXERCISES

Exercise 1. Find all the Riemannian homogeneous spaces for SO(3). (Decide first of all what it should mean for two such spaces to be isomorphic.)

Exercise 2. Show that every symplectic homogeneous space for SO(3) is either a point, or a two-dimensional sphere of radius r > 0. Consequently these spaces are parametrized by the non-negative real numbers.

Exercise 3. This exercise concerns the non-existence of a nice simultaneous quantization of a large family of classical observables. The results come more or less from [Gr], with a more mathematical account in [vH]. Perhaps the best place to read about it is in [AM], Theorem 5.4.9. We take as symplectic manifold X = ℝ^{2}, with coordinate functions p and q. The Poisson bracket on smooth functions on X is

\{f, g\} = \frac{∂f}{∂p} \frac{∂g}{∂q} − \frac{∂f}{∂q} \frac{∂g}{∂p}.  E3(a)

For the quantization, we use the Hilbert space H = L^{2}(ℝ); the coordinate on ℝ is called q. We begin the quantization as in (1.6), defining two skew-adjoint operators

A(q) = multiplication by \sqrt{−1}q,  A(p) = ∂/∂q.  E3(b)

Recall that the quantum interpretation of position suggests defining

A(f) = multiplication by \sqrt{−1}f,  f ∈ ℝ[q].  E3(c)

As in (1.7), we try to quantize other polynomial functions on ℝ^{2} using polynomial coefficient differential operators on ℝ. The requirement (1.4) is

[A(f), A(g)] = A({f, g})  (f, g ∈ ℝ[p, q]).  E3(d)

It is easy to check that this is satisfied for those functions where A has so far been defined: polynomials in q, and linear functions of p. On quadratic polynomials, it is natural to define

A(p^{2}) = \sqrt{−1}\frac{∂^{2}}{∂q^{2}} , A(pq) = −\frac{1}{2}(q \frac{∂}{∂q} + \frac{∂}{∂q} q), A(q^{2}) = \sqrt{−1}q^{2}.  E3(e)

(We have already explained A(q^{2}). The requirement (E3)(d) means that A(p^{2}) should commute with A(p) = ∂/∂q, so it should be a constant coefficient differential operator. This particular choice seems reasonable, and is consistent with the standard construction of the quantum harmonic oscillator. Finally the choice of A(pq) is forced by (E3)(d), since {p^{2}, q^{2}} = 4pq.)

Here at last is the exercise. Show first that the span of p^{2}, q^{2}, and pq is a Lie algebra 𝔤 under Poisson bracket, isomorphic to SL(2, ℝ). Next, show that the Poisson bracket defines a representation of 𝔤 on the space S_{n} of homogeneous polynomials of degree n in p and q; and that this is the irreducible n+1-dimensional representation of 𝔤.

Now write T_{n} for the space of skew-symmetric polynomial coefficient differential operators on ℝ, of total degree at most n. (The total degree is the sum of the differential operator degree and the polynomial coefficient degree.) Show that T_{n} is invariant under operator bracket by A(𝔤) (the operators of (E3)(e)), and that this defines a representation of 𝔤 on T_{n}. Show further that T_{n} is the sum of the irreducible representations of 𝔤 of dimensions 1, 2, …, n+1. Show that A(q^{n}) may be regarded as a highest weight vector for the representation of dimension n + 1.

Conclude that there is a unique linear “quantization map” A carrying ℝ[p, q] to skew-symmetric polynomial coefficient differential operators on ℝ, subject to the requirements (E3)(e); A(q^{n}) = \sqrt{−1}q^{n}; and

A({f, g}) = [A(f), A(g)]  (f homogeneous of degree 2).  E3(f)

What remains is to show that A fails to satisfy (E3)(d) in general. To do that, show that {p^{2}, {p^{2}, q^{4}}} is proportional to {p^{3}, q^{3}}, but that [A(p^{2}), [A(p^{2}), A(q^{4})]] is not proportional to [A(p^{3}), A(q^{3})].

Exercise 4. Suppose ϕ is a non-degenerate bilinear form on ℝ^{n}, either symmetric or skew-symmetric. There is a unique invertible matrix J so that

ϕ(u, v) = ⟨Ju, v⟩ (u, v ∈ ℝ^{n}).

Assume that J^{2} = ±I. Show that the group of the form

G(ϕ) = {g ∈ GL(n, ℝ) | ϕ(gu, gv) = ϕ(u, v) (u, v ∈ ℝ^{n})}

is a linear reductive group. Check that this condition allows the indefinite orthogonal groups O(p, q) (p + q = n) and the symplectic group Sp(2n, ℝ) as linear reductive groups.

Exercise 5. Find a closed subgroup of GL(1, ℝ) that is stable under θ but is not a reductive group. (Hint: it follows from Theorem 2.6 that a reductive group has a finite number of connected components.) Can you find more interesting examples in GL(2, ℝ)?

Exercise 6. Suppose G is a linear reductive Lie group (Definition 2.5), and we are given elements H, E, and F of the Lie algebra as in (2.9)(a). Why does the group homomorphism ϕ of (2.9)(b–c) exist? (Hint: the group SL(2, ℝ) is not simply connected, so the problem is not completely trivial.) What can you say if G is only assumed to be reductive?

Exercise 7. Write GL(n, ℂ) = K · exp(𝔰) for the polar decomposition of n × n complex matrices; here K = U(n) and 𝔰 is the space of self-adjoint matrices. Fix X ∈ 𝔨. Show that the set of fixed points of the one-parameter group Ad(exp(tX)) on 𝔰 is precisely 𝔰^{X}. Conclude that the stabilizer of X in the adjoint action of GL(n, ℂ) is K^{X} · exp(𝔰^{X}).

Suppose that Z ∈ 𝔰. Show that the stabilizer of Z in the adjoint action is K^{Z} · exp(𝔰^{Z}). (Hint: look at iZ.)

Suppose G = K · exp(𝔰) is the Cartan decomposition of a reductive group, and X ∈ 𝔰. Show that G^{X} = K^{X} · exp(𝔰^{X}), and conclude that G^{X} is a reductive group.

Exercise 8. Suppose G is a reductive group with Cartan involution θ. 1. Show that the bilinear form Q on 𝔤 defined by

Q(X, Y) = −⟨X, θY ⟩

(notation as in Proposition 2.7) is positive definite.

2. If X ∈ 𝔰, show that the operator ad(X) on 𝔤 is self-adjoint with respect to Q. Conclude that ad(X) is diagonalizable with real eigenvalues.

3. If Y ∈ 𝔨, show that the operator ad(Y) on 𝔤 is skew-adjoint with respect to Q. Conclude that ad(Y)_{ℂ} is diagonalizable with purely imaginary eigenvalues.

4. If k ∈ K, show that the operator Ad(k) on 𝔤 is orthogonal with respect to Q. Conclude that Ad(k)_{ℂ} is diagonalizable with eigenvalues in the unit circle.

Exercise 9. The point of this exercise is to understand the constructions of (3.2) in some examples, and in particular the character 2ρ_{𝔮} of Proposition 3.3. You can pick your own favorite examples; certainly even compact groups are interesting in this context. If you would like to test your grasp of general structure theory, you can try to prove that if G is a complex reductive group (and λ_{e} is elliptic, and so on) then the character 2ρ_{𝔮} always has a distinguished square root.

If you don’t have a favorite example to try, here is one. Begin with the complex vector space ℂ^{n}. Using a standard identification ℂ ≃ ℝ^{2}, we get ℂ^{n} ≃ ℝ^{2n}. Let G = GL(2n, ℝ), so that 𝔤 consists of all ℝ-linear transformations of ℝ^{2n}. Define

J = multiplication by i on ℂ^{n} ≃ ℝ^{2n}.

This is an ℝ-linear transformation, and so an element of 𝔤; it is elliptic. (Why?) Write λ_{e} ∈ 𝔤^{∗} for the corresponding linear functional:

λ_{e}(Z) = tr JZ.

Use the notation of (3.2). In particular L consists of invertible real-linear transformations of ℂ^{n} that commute with multiplication by i; so

L = GL(n, ℂ).

Having come this far, you should try to identify the coadjoint orbit G · λ_{e} with the space of all complex structures on the vector space ℝ^{2n}.

Now we want to investigate the eigenspaces of i ad(J). Let B (for “bar”) be the complex-conjugation linear transformation of ℂ^{n}. Then JB = −BJ. Show that B + iJB is in the +2 eigenspace of i ad(J), and in fact that

𝔲 = {T (B + iJB) | T a complex-linear transformation of ℂ^{n}}.

Deduce that the adjoint action of L on 𝔲 is equivalent to the action of GL(n, ℂ) on n × n complex matrices, by

g · T = gT \overline{g}^{−1} (g ∈ GL(n, ℂ), T ∈ 𝔤𝔩(n, ℂ)).

Finally conclude that

2ρ_{𝔮}(g) = (det(g)/\overline{det(g)})^{n} (g ∈ GL(n, ℂ)).

Does this unitary character of GL(n, ℂ) have a square root?

Exercise 10. One of the pleasant properties of a Stein manifold X is that a compact complex submanifold of X must be finite. In the setting of (3.2), show that K/L ∩ K is a compact complex submanifold of X. Deduce that X can be Stein only if the Lie algebra element X_{e} belongs to the center of 𝔨. (In that case it turns out that X actually is Stein.)

If you know something about the Bott-Borel-Weil theorem, show that the holomorphic vector bundle V_{τ} has non-zero holomorphic sections over K/L ∩ K if and only if K/L ∩ K is finite. Deduce that V_{τ} has non-zero holomorphic sections over X only if K/L ∩ K is finite. (This statement is also if and only if.)

Exercise 11. Use the notation of Definition 5.2. Find a definition of a metaplectic double cover of K_{ℂ}^{λ_{θ}} analogous to (3.6), and a character ρ of K̃_{ℂ}^{λ_{θ}}. Show that admissible orbit data at λ_{θ} are in one-to-one correspondence with irreducible genuine representations (τ̃, V_{τ}) of this metaplectic cover that are trivial on the identity component (that is, having differential 0). To complete the analogy with Proposition 3.7, show that the restriction of λ_{θ} to 𝔨_{ℂ}^{λ_{θ}} is zero.

Exercise 12. Suppose G = Sp(2n, ℝ) is the standard real symplectic group. We want to look at some examples of nilpotent admissible orbit data. The facts stated in this first paragraph can just be assumed, although verifying them makes a reasonable exercise in structure theory. The maximal compact subgroup of G is isomorphic to U(n), so its complexification K_{ℂ} is isomorphic to GL(n, ℂ). There are two representations of K_{ℂ} on the space S of symmetric n × n complex matrices: we write them as (π^{+}, S^{+}) and (π^{−}, S^{−}). Explicitly,

π^{+}(g)U = gU ^{t}g (g ∈ GL(n, ℂ), U ∈ S)

and

π^{−}(g)V = ^{t}g^{−1}V g^{−1} (g ∈ GL(n, ℂ), V ∈ S)

Then there is a K_{ℂ}-equivariant isomorphism

𝔰_{ℂ}^{∗} ≃ S^{+} ⊕ S^{−}.

The nilpotent elements of 𝔰_{ℂ}^{∗} are the pairs (U, V) so that UV and V U are both nilpotent. (A complete classification of the K_{ℂ} orbits of such pairs is an entertaining linear algebra problem, which the reader might also like to consider.) We will confine our attention to certain relatively small orbits: specifically, to the case UV = V U = 0. There are two obvious invariants to attach to such an orbit: the rank p of U and the rank q of V. The vanishing of UV forces p + q ≤ n. It turns out that the pair (p, q) determines the K_{ℂ} orbit of (U, V).

So let us fix (p, q) with p + q ≤ n. Define

U = ( \begin{matrix}I_{p} & 0 \\ 0 & 0_{n−p} \end{matrix} ) ,  V = ( \begin{matrix}0_{n−q} & 0 \\ 0 & I_{q} \end{matrix} ) ,  λ_{θ} = (U, V).

Then λ_{θ} is a nilpotent element of 𝔰_{ℂ}^{∗} in the orbit parametrized by (p, q). We want to compute the admissible orbit data. Show first of all that the stabilizer of λ_{θ} in K_{ℂ} is

K_{ℂ}^{λ_{θ}} = ( \begin{matrix}A & ∗ & ∗ \\ 0 & B & ∗ \\ 0 & 0 & C \end{matrix} ) (A ∈ O(p, ℂ), B ∈ GL(n − p − q, ℂ), C ∈ O(q, ℂ)).

Next, show that the character 2ρ of Definition 5.2 is

2ρ(A, B, C) = (det A)^{n−1}(det B)^{q−p}(det C)^{n−1}.

(Here we write (A, B, C) as shorthand for the matrix from the preceding display.) Conclude that admissible orbit data exist if and only if either q − p is even, or p + q = n. If one of these conditions is satisfied, then the possible admissible orbit data are

τ(A, B, C) = (det A)^{ϵ}(det B)^{(q−p)/2}(det B)^{δ}.

Here ϵ and δ both belong to ℤ/2ℤ; the factor with ϵ (respectively δ) actually appears only if p (respectively q) is non-zero. Therefore (still assuming that q − p is even, or that p + q = n) there are four admissible orbit data if p and q are both non-zero; two if exactly one of p and q is zero; and one if p = q = 0.

Exercise 13. This is a continuation of Exercise 12. First show that the orbit with parameters (p, q) has complex dimension (p+q)(p+q+1)/2+(p+q)(n−p−q). The orbits in the boundary are those with parameters (p′, q′), with p′ ≤ p, q′ ≤ q, and p′ +q′ < p+q. Conclude that the boundary has complex codimension n−(p+q)+1 (if p + q > 0; the boundary is empty if p + q = 0). Therefore Definition 5.4 applies if and only if p + q < n. If you know a little about the representations of U(n), you might try to calculate X_{K}(λ_{n}, τ) in this case. It turns out that the representations of U(n) appear with multiplicity one; those appearing are the ones of highest weight

μ = (μ_{1}, …, μ_{q}, (q − p)/2, …, (q − p)/2, μ_{n−p+1}, …, μ_{n}),

μ_{i} ≡ δ (mod 2) (1 ≤ i ≤ q), μ_{j} ≡ ϵ (mod 2) (n − p + 1 ≤ j ≤ n).

Of course μ must also be a highest weight: that is, the coordinates of μ must be weakly decreasing integers. (Hint: the most difficult part is to understand induction from O(p) to U(p). This is a compact symmetric space, so the induction is computed by a theorem of Helgason. It is the sum of all representations of U(p) whose highest weights have all coordinates even. Given this fact and its twist by the determinant character, you just need the Borel-Weil theorem to finish the calculation.)

Exercise 14. This exercise looks at the construction of Proposition 7.9 in the case of G = U(1, 1). Then K = U(1) × U(1), and K_{ℂ} = ℂ^{×} × ℂ^{×}. We have

(𝔤_{ℂ}/𝔨_{ℂ})^{∗} ≃ \{( \begin{matrix}0 & u \\ v & 0 \end{matrix} ) | u, v ∈ ℂ\} ≃ ℂ^{2};

the action of K_{ℂ} is

(z, w) · (u, v) = (zw^{−1}u, z^{−1}wv).

The nilpotent cone is

N_{θ}^{∗} = {(u, v) | uv = 0}.

There are three orbits of K_{ℂ}, represented by

λ_{1} = (1, 0),  λ_{2} = (0, 1),  λ_{3} = (0, 0).

The first two isotropy subgroups are

H_{1} = H_{2} = ℂ_{Δ}^{×} ⊂ ℂ^{×} × ℂ^{×}.

The third is H_{3} = K_{ℂ}.

Fix an integer m, and let τ_{m} be the character

τ_{m}(z) = z^{m}

of H_{1}. Then M(λ_{1}, τ_{m}) may be identified with the space of functions on K_{ℂ} transforming according to the character τ_{m} of H_{1}. As a basis for M(λ_{1}, τ_{m}), we can take the functions

f_{p}(z, w) = z^{p}w^{m−p}.

The function f_{p} transforms under K according to the character (p, m − p) (in the standard identification of K̂ with ℤ^{2}).

Now M(λ_{1}, τ_{m}) is also a module for S(𝔤/𝔨) ≃ ℂ[u, v]. Show that in this module structure, v acts by zero, and u·f_{p} = f_{p+1}. Conclude that M(λ_{1}, τ_{m}) is not finitely generated. Show that the module N(λ_{1}, τ_{m}) constructed in the proof of Proposition 7.9 must be

N(λ_{1}, τ_{m}) = span of {f_{p} | p ≥ p_{0}}

for some integer p_{0}. Conclude that

K-types of Ind_{H_{1}}^{K_{ℂ}}(τ_{m}) = {(r, s) | r + s = m}
K-types of N(λ_{1}, τ_{m}) = {(r, s) | r + s = m, r − s ≥ 2p_{0} − m}.

If you are feeling very ambitious, much of this exericise can be generalized to the setting of Exercise 12, with λ_{1} one of the elements with p + q = n.

Exercise 15. This exercise looks at the bases provided by Theorem 8.2 and Proposition 7.9 in case G = SL(2, ℂ) (viewed as a real group). The maximal compact subgroup is K = SU(2). Let T be the standard maximal torus in K (consisting of diagonal matrices); T is isomorphic to U(1), so T̂ ≃ ℤ; we write δ_{m} for the character corresponding to m. The representations of K are parametrized (by their highest weights) by non-negative integers; we write μ_{m} for the (m + 1-dimensional) representation of K of highest weight δ_{m}.

Let A be the group of diagonal matrices in G with positive real entries, and N the group of upper triangular matrices with ones on the diagonal. Then B = T AN is a Borel subgroup of G. If δ ∈ T̂ and ν ∈ Â, then we can construct a principal series representation

I(δ ⊗ ν) = Ind_{B}^{G}(δ ⊗ ν ⊗ 1).

This representation is tempered exactly when ν is unitary, and has real infinitesimal character exactly when ν is real-valued. We will often think of Â as identified with 𝔞_{ℂ}^{∗}, and so write −ν for the inverse of the character ν. The principal series I(δ ⊗ ν) and I(δ^{−1}⊗−ν) have the same irreducible composition factors, so for many purposes we can confine our attention to characters δ = δ_{m} of T with m ≥ 0.

Because of the Iwasawa decomposition G = KAN, we find

I(δ ⊗ ν)|_{K} ≃ Ind_{T}^{K}(δ),

and this in turn is the sum of all the irreducible representations of K containing the weight δ. For m ≥ 0, this is

I(δ_{m} ⊗ ν)|_{K} ≃ ∑_{k≥0} μ_{m+2k}.

The tempered irreducible representations of real infinitesimal character are the various principal series

π_{m} = I(δ_{m} ⊗ 0) (m ≥ 0).

Their restrictions to K are what we just computed:

π_{m}|_{K} ≃ ∑_{k≥0} μ_{m+2k}.

These representations are the basis of Theorem 8.2 for (restrictions to K of) virtual Harish-Chandra modules. That is, any finite length Harish-Chandra module for G has the same restriction to K as a unique integer combination of the various π_{m}. The first part of the exercise is essentially to verify this assertion. There are just two kinds of irreducible Harish-Chandra modules for G: the irreducible principal series, and the finite-dimensional representations. For the principal series, we have

I(δ_{m} ⊗ ν)|_{K} ≃ π_{|m|}|_{K}.

Show that if F is any irreducible finite-dimensional representation of G, then

F |_{K} ≃ μ_{p} + μ_{p+2} + ⋯ + μ_{p+2q},

for some non-negative integers p and q. Conclude that

F |_{K} ≃ (π_{p} − π_{p+2q+2})|_{K}.

Now we examine the basis of Proposition 7.9. The complexification of K is G, and the action of K_{ℂ} on (𝔤_{ℂ}/𝔨_{ℂ})^{∗} is equivalent to the action of G on 𝔤. In

particular there are exactly two nilpotent orbits, through λ_{1} = 0 and λ_{2} = ( \begin{matrix}0 & 1 \\ 0 & 0 \end{matrix} ).

The isotropy groups are H_{1} = G and H_{2} = Z(G)N. The orbit dimensions are 0 and 2, so the codimension condition in Proposition 7.9 is always satisfied. The irreducible (algebraic) representations of G may be identified with the irreducible representations μ_{m} of K. The irreducible representations of H_{2} are the irreducible characters of Z(G) ≃ ℤ/2ℤ; there is the trivial character τ_{e} and the non-trivial character τ_{o}. The restrictions to K of the various standard modules are computed by Proposition 7.9; check that they are

N(λ_{1}, μ_{m})|_{K} ≃ μ_{m},

N(λ_{2}, τ_{e})|_{K} ≃ ∑_{k≥0} μ_{2k},

N(λ_{2}, τ_{o})|_{K} ≃ ∑_{k≥0} μ_{2k+1}.

Proposition 7.9 says that any Harish-Chandra module for G has the same restriction to K as a unique integer combination of the various N(λ, τ). The next part of the exercise is to verify that assertion. By the first part, it is enough to consider the various π_{m}. Show that if m is even and non-negative,

π_{m}|_{K} ≃ (N(λ_{2}, τ_{e}) − N(λ_{1}, μ_{0}) − N(λ_{1}, μ_{2}) − ⋯ − N(λ_{1}, μ_{m−2}))|_{K};

and similarly for m odd.

The point of section 8 is that the other change of basis is more interesting. Show that

N(λ_{1}, μ_{m})|_{K} = (π_{m} − π_{m+2})|_{K},
N(λ_{2}, τ_{e})|_{K} = π_{0}|_{K}, N(λ_{2}, τ_{o})|_{K} = π_{1}|_{K}.

Exercise 16. This exercise concerns the notion of “infinitesimal character size” in Definition 8.8, for the group SL(2, ℂ). In Exercise 15 we identified the characters of the compact torus T with ℤ. This identification extends to i𝔱^{∗} ≃ ℝ. Let us also normalize the invariant bilinear form on 𝔤 so that (after restriction, complexification, and dualization to i𝔱^{∗}) it agrees with the standard inner product on ℝ. Thus the representation μ_{m} of K has highest weight δ_{m}, which (according to the remark after Proposition 8.4) is mapped by the algorithm of [VG] to the weight m ∈ ℝ ≃ i𝔱^{∗}, which has norm m. Using these facts, and the calculation in Exercise 15, prove that

∥K_{ℂ} · λ_{1}∥ = ∥2∥ = 2, ∥K_{ℂ} · λ_{2}∥ = ∥0∥ = 0.

Next, show that 2 ∈ i𝔱^{∗} is equal to ρ, half the sum of a certain system of positive roots for the fundamental Cartan. This is the infinitesimal character of the trivial representation of G. The conclusion (by Proposition 8.7) is that any virtual Harish-Chandra module [M] with [gr M] finite-dimensional and non-zero must have an irreducible constituent of infinitesimal character at least as big as ρ.

BIBLIOGRAPHY

[AM] R. Abraham and J. Marsden, Foundations of Mechanics, Benjamin/Cummings, Reading, Massachusetts, 1978.

[AHV] J. Adams, J.-S. Huang, and D. Vogan, Functions on the model orbit in E_{8}, Represent. Theory 2 (1998), 224–263.

[Arn] V. Arnold, Mathematical Methods of Classical Mechanics, Springer-Verlag, New York-Heidelberg-Berlin, 1978.

[BV] D. Barbasch and D. Vogan, Unipotent representations of complex semisimple Lie groups, Ann. of Math. 121 (1985), 41–110.

[CM] D. Collingwood and W. McGovern, Nilpotent Orbits in Semisimple Lie Algebras, Van Nostrand Reinhold, New York, 1993.

[D] M. Duflo, Théorie de Mackey pour les groupes de Lie algébriques, Acta Math. 149 (1982), 153–213.

[GV] W. Graham and D. Vogan, Geometric quantization for nilpotent coadjoint orbits, Geometry and representation theory of real and p-adic groups (J. Tirao, D. Vogan, and J. Wolf, eds.), Birkhäuser, Boston-Basel-Berlin, 1998, pp. 69–137.

[Gr] H. J. Groenewold, On the principles of elementary quantum mechanics, Physica 12 (1946), 405–460.

[EGA] A. Grothendieck (rédigés avec la collaboration de J. Dieudonné), Eléments de Géométrie Algébrique IV. Etude locale des schémas et des morphismes de schémas (Seconde Partie), Publications Mathématiques, vol. 24, Institut des Hautes Etudes Scientifiques, Le Bois-Marie, Bures-sur-Yvette, France, 1965.

[vH] L. van Hove, Sur certaines représentations unitaires d’un groupe infini de transformations, Acad. Roy. Belgique. Cl. Sci. Mém. Coll. in 8^{∘} 26, fasc. 6 (1951), 1–102.

[Hum] J.E. Humphreys, Introduction to Lie Algebras and Representation Theory, Springer-Verlag, Berlin-Heidelberg-New York, 1972.

[Kir] A. Kirillov, Elements of the Theory of Representations, translated by E. Hewitt, Springer-Verlag, Berlin-Heidelberg-New York, 1976.

[Kna] A. Knapp, Representation Theory of Real Semisimple Groups: an Overview Based on Examples, Princeton University Press, Princeton, New Jersey, 1986.

[KV] A. Knapp and D. Vogan, Cohomological Induction and Unitary Representations, Princeton University Press, Princeton, New Jersey, 1995.

[Kos] B. Kostant, Quantization and unitary representations, Lectures in Modern Analysis and Applications, (C. Taam, ed.), Lecture Notes in Mathematics, vol. 170, Springer-Verlag, Berlin-Heidelberg-New York, 1970.

[KR] B. Kostant and S. Rallis, Orbits and representations associated with symmetric spaces, Amer. J. Math. 93 (1971), 753–809.

[Mac] G. Mackey, Mathematical Foundations of Quantum Mechanics, W. A. Benjamin, Inc., New York, 1963.

[SV] S. Salamanca-Riba and D. Vogan, On the classification of unitary representations of reductive Lie groups, to appear in Ann. of Math.

[Tor] P. Torasso, Quantification géométrique, opérateurs d’entrelacement et représentations unitaires de S̃L_{3}(ℝ), Acta Math. 150 (1983), 153–242.

[Tor2] P. Torasso, Méthode des orbites de Kirillov-Duflo et représentations minimales des groupes simples sur un corps local de caractéristique nulle, Duke Math. J. 90 (1997), 261–377.

[Vrg] M. Vergne, Instantons et correspondance de Kostant-Sekiguchi, C. R. Acad. Sci. Paris Sér. I Math. 320 (1995), 901–906.

[VG] D. Vogan, Representations of Real Reductive Lie Groups, Birkhäuser, Boston-Basel-Stuttgart, 1981.

[VoCI] D. Vogan, Unitarizability of certain series of representations, Ann. of Math. 120 (1984), 141–187.

[VO] D. Vogan, Unitary Representations of Reductive Lie Groups, Annals of Mathematics Studies, Princeton University Press, Princeton, New Jersey, 1987.

[VAV] D. Vogan, Associated varieties and unipotent representations, Harmonic Analysis on Reductive Groups (W. Barker and P. Sally, eds.), Birkhäuser, Boston-Basel-Berlin, 1991, pp. 315–388.

[VUR] D. Vogan, Unipotent representations and cohomological induction, The Penrose Transform and Analytic Cohomology in Representation Theory (M. Eastwood, J. Wolf, and R. Zierau, eds.), Contemporary Mathematics 154, American Mathematical Society, Providence, Rhode Island, 1993, pp. 47–70.

[VDK] D. Vogan, The orbit method and unitary representations of reductive Lie groups, Algebraic and Analytic Methods in Representation Theory (Sønderborg, 1994), Perspectives in Mathematics, vol. 17, Academic Press, San Diego, 1997, pp. 243–339.

[VLC] D. Vogan, A Langlands classification for unitary representations, Analysis on Homogeneous spaces and Representations of Lie groups, Advanced Studies in Pure Mathematics, volume 26, Kinokuniya Company, Ltd., Tokyo, 1998.

[Wey] H. Weyl, The Theory of Groups and Quantum Mechanics, Dover, 1931.

[Wng] H. Wong, Dolbeault cohomological realization of Zuckerman modules associated with finite rank representations, J. Funct. Anal. 129 (1995), 428–