Geometric quantization for nilpotent coadjoint orbits William Graham * Department of Mathematics University of Chicago Chicago, Illinois 60637 David A. Vogan, Jr. † Department of Mathematics Massachusetts Institute of Technology Cambridge, Massachusetts 02139

Contents

1. Introduction.

2. Symplectic and Poisson manifolds.

3. Hamiltonian G-spaces and moment maps.

4. Lagrangian subspaces.

5. The metaplectic representation.

6. Admissible orbit data.

7. Symplectic spinors and the Lagrangian Grassmannian.

8. Existence of Lagrangian coverings.

9. Construction of representations.

1. Introduction. Suppose G is a Lie group, 𝔤 its Lie algebra, and 𝔤^{∗} the dual vector space. It is a classical idea of Kirillov and Kostant (see [10] and [11]) that irreducible unitary representations of G are related to the orbits of G on 𝔤^{∗}. This idea finds its purest form in

Theorem 1.1 (Kirillov [10]). Suppose G is a connected and simply connected nilpotent Lie group. Then there is a bijection from the set 𝔤^{∗}/G of coadjoint orbits of G to the set Ĝ of equivalence classes of irreducible unitary representations of G.

For other groups there are complications even with regard to what is true (never mind what one can prove). We recall very briefly a few of these. First, not every coadjoint orbit can correspond to a representation: one has to impose an appropriate “integrality” requirement on the orbit. This complication occurs already for G the circle group.

Second, one needs a little more information beyond the orbit itself: very roughly speaking, something like a local coefficient system on the orbit. This complication occurs in semidirect product groups K × V (with K compact and V a vector group on which K acts) whenever the isotropy groups of the K action on V can be disconnected. More serious problems in this direction were found by Rothschild and Wolf in [17]. They gave an example in the split real group of type G_{2} in which two representations of different infinitesimal characters could be attached to the same coadjoint orbit.

Third, the same unitary representation may arise from each of several coadjoint orbits. Exactly when this complication occurs depends on exactly how the correspondence from orbits to representations is defined. In the approach we will follow (due mostly to Duflo) the simplest example has G = SU(2); the trivial representation is attached to the orbit {0} and also to the orbit of the half-sum of positive roots.

Fourth, some unitary representations are not attached to any coadjoint orbit. This complication appears first for SL(2, ℝ), where the complementary series representations are not attached to orbits.

Fifth, the unitary representations attached to some coadjoint orbits are not irreducible. The simplest way that this happens is that the representation is zero. With Duflo’s version of the correspondence, this happens for G = U(3), in the following way. Coadjoint orbits for U(3) are parametrized by decreasing sequences of three real numbers. Consider the orbit O_{b} parametrized by (b, −1/2, −1/2), with b a real number greater than −1/2. These orbits are all isomorphic to ℂℙ^{2}, the complex projective plane. Duflo’s integrality condition (admissibility, as defined in section 6) amounts to b ∈ ℤ. The representation Duflo attaches to O_{b} is the space of holomorphic sections of the b − 1 tensor power of a standard line bundle. For

* Supported in part by an NSF postdoctoral fellowship † Supported in part by NSF grant DMS-9402994 b − 1 ≥ 0, this space is S^{b−1}(ℂ^{3}), a symmetric power of the standard representation of U(3); but for b = 0 there are no holomorphic sections, so the representation is zero. Of course a more interesting possibility is that the representation is actually reducible. The simplest example I know of this phenomenon occurs in the complex form of the exceptional group G_{2}, for a singular elliptic coadjoint orbit. The representation attached to such an orbit is always a unitary degenerate series representation (induced from a non-trivial one-dimensional character of a parabolic subgroup). One such induced representation is reducible.

Finally, when G is not of type I there are terrible complications, because irreducible unitary representations of G are no longer such a natural class of objects to consider. This is first apparent in the work of Auslander and Kostant [3], who established a version of Theorem 1.1 for solvable Lie groups.

In light of all these complications, what one might hope for along the lines of Theorem 1.1 for general Lie groups is something like this.

Problem 1.2. Suppose G is a type I Lie group. Find a construction attaching to each pair (X, τ) a unitary representation π(X, τ) of G. Here X is an orbit of G on 𝔤^{∗} satisfying an appropriate integrality hypothesis, and τ is some appropriate additional structure. The representations π(X, τ) should be close to irreducible, and they should include most of the interesting irreducible unitary representations of G.

We will eventually refine the statement of this problem substantially (see Problem 6.3). But we can discuss ideas for the solution of the problem without a clearer statement, and this we do next. The classical strategy, known as geometric quantization, is this. A coadjoint orbit X carries a natural G-invariant symplectic structure. (The definition will be recalled in Corollary 2.13.) There are various standard constructions of unitary representations of G, beginning with some data D and leading to a unitary representation π(D). Attached to such a construction it is often possible to find a symplectic manifold Y (D) with a G action. The idea of geometric quantization can be phrased in this way: given a coadjoint orbit X, one tries to find data D so that X ≃ Y (D) (as symplectic manifolds with G action). If this can be done, then one says that π(D) is the unitary representation associated to X. To use this idea to solve Problem 1.2, one must show that every (appropriately integral) coadjoint orbit is isomorphic to some Y (D), and that π(D) is independent of the choice of D (subject to the condition X ≃ Y (D)).

Here is an example. One standard construction of a unitary representation begins with an action of G on a smooth manifold M. (A good example to keep in mind is the action of G = SL(2, ℝ) on the real projective space M = ℝℙ^{1} of lines through the origin in ℝ^{2}. Thus M is just a circle, but the action of G is interesting and complicated.) To get a unitary representation we need a Hilbert space, and it is natural to consider something like L^{2}(M), the space of square-integrable functions on M. To define this space we must choose a measure on M. In order to get a natural action of G on L^{2}(M) by unitary operators, the measure must be preserved by the action of G. In many examples (including the action of SL(2, ℝ) on the circle described above) there is no nice G-invariant measure. To circumvent this problem, one can introduce the real line bundle D^{1/2}(M) of half-densities on M. (The precise definition will be recalled in Definition 5.5.) For us the central fact is that the tensor product of this bundle with itself is the density bundle D^{1}(M), whose sections are the (signed) smooth measures on M:

D^{1/2}(M) ⊗ D^{1/2}(M) ≃ D^{1}(M)  (1.3)(a)

Here the tensor product is of line bundles on M.

Consider now the space S of compactly supported smooth sections of D_{ℂ}^{1/2}(M). (The subscript ℂ denotes complexification.) If s_{1} and s_{2} belong to S, then it follows from (1.3)(a) that s_{1}\overline{s_{2}} is a compactly supported section of D_{ℂ}^{1}(M); that is, it is a compactly supported complex-valued density on M. We may therefore define a pre-Hilbert space structure on S by

⟨s_{1}, s_{2}⟩ = ∫_{M} s_{1}\overline{s_{2}}  (1.3)(b)

The completion of this pre-Hilbert space is written L^{2}(M, D^{1/2}), the space of square-integrable half-densities on M. Each element g of G (and indeed each diffeomorphism of M) acts on the space S and preserves the inner product; so we get a unitary representation

π(M): G → U(L^{2}(M, D^{1/2}))  (1.3)(c)

of G. (We write U(H) for the group of unitary operators on a Hilbert space H.) Now geometric quantization asks that we find attached to M a symplectic manifold with a G action. The candidate we choose is the cotangent bundle:

Y (M) = T^{∗}(M).  (1.3)(d)

Geometric quantization says that if a coadjoint orbit X is isomorphic to the cotangent bundle T^{∗}(M) of a G-manifold M, then we should attach to X the unitary representation of G on half-densities on M. This is reasonable statement as far as it goes, but it doesn’t go very far. The action of G on T^{∗}(M) preserves the zero section M ⊂ T^{∗}(M), so it is never transitive unless M is discrete. Since a coadjoint orbit is by definition a homogeneous space, it can be isomorphic to a cotangent bundle only if it is discrete.

Fortunately it is possible to generalize this construction in many ways. The simplest is to consider in addition to M a Hermitian line bundle L over M, equipped with an action of G preserving the metric. Then L ⊗ D^{1/2}(M) is a complex line bundle on M; write S(M, L) for the space of compactly supported smooth sections. If σ_{1} and σ_{2} belong to S(M, L), then we can write σ_{i} = l_{i} ⊗ s_{i}; here l_{i} is a compactly supported section of L and s_{i} a nowhere vanishing section of D^{1/2}(M). Then ⟨l_{1}, l_{2}⟩_{L} is a compactly supported complexvalued function on M and s_{1}s_{2} is a smooth density; so the product is a compactly supported complex-valued density. We may therefore define a pre-Hilbert space structure on S(M, L) by

⟨σ_{1}, σ_{2}⟩ = ∫_{M}⟨l_{1}, l_{2}⟩_{L}s_{1}s_{2}.  (1.4)(a)

The completion of this pre-Hilbert space is written L^{2}(M, L ⊗ D^{1/2}). We get a unitary representation

π(M, L): G → U(L^{2}(M, L ⊗ D^{1/2}))  (1.4)(b)

The philosophy of geometric quantization requires also a symplectic manifold attached to M and L. This is provided by a construction of Kostant (see [13] or [21], Proposition 4.6). From a Hermitian line bundle L on a real manifold M one can construct the twisted cotangent bundle

Y (M, L) = T^{∗}(M, L).  (1.4)(c)

This is a fiber bundle over M with a natural symplectic structure ω(L); it is an affine bundle over the cotangent bundle. In particular, the fiber T_{m}^{∗}(M, L) over m in M is an affine space for the vector space T_{m}^{∗}(M); that is, it is a copy of T_{m}^{∗}(M) with the origin forgotten. Sections of T^{∗}(M, L) are certain special connections on L. We can now formulate

Philosophy of Geometric Quantization (first form). Suppose that a coadjoint orbit X for a Lie group G is isomorphic to a twisted cotangent bundle T^{∗}(M, L) (with M a smooth G-manifold and L a Hermitian line bundle on M). Then the unitary representation π(M, L) is attached to X.

This philosophy has some content: the proof of Kirillov’s Theorem 1.1 shows that every coadjoint orbit for a connected nilpotent Lie group is a twisted cotangent bundle. For more complicated groups the full power of geometric quantization requires considering not just the real analysis construction of (1.4)(b) but also some complex-analytic analogues; but for the purposes of this introduction the present statement will suffice.

The geometric part of the geometric quantization approach to Problem 1.2 is therefore this: given a coadjoint orbit X, find a twisted cotangent bundle T^{∗}(M, L) to which X is isomorphic (as a symplectic G-space). To understand how to do that, we need to know a little more about the geometry of T^{∗}(M, L). The main point is that the fibers T_{m}^{∗}(M, L) are Lagrangian submanifolds; that is, the tangent space to a fiber is always a maximal isotropic subspace for the symplectic form. In this way the symplectic manifold T^{∗}(M, L) has a foliation with Lagrangian leaves; the base manifold M may be identified with the space of leaves.

The idea now is to find a parallel structure in our coadjoint orbit X. Let us fix a base point f ∈ X, with isotropy group G_{f}; then

X ≃ G/G_{f}.  (1.5)(a)

We would like to find a G-invariant Lagrangian foliation of X. Since X is homogeneous, the space of leaves must be homogeneous. The whole foliation will therefore be determined as soon as we know the leaf Λ_{f} through the base point f; the other leaves will just be the translates g · Λ_{f}. The purely set-theoretic requirement that X be the disjoint union of these translates imposes a very strong constraint on Λ_{f}: it implies that there must be a subgroup H ⊃ G_{f} with

Λ_{f} = H · f ≃ H/G_{f} ⊂ G/G_{f} ≃ X.  (1.5)(b)

The requirement that Λ_{f} be an isotropic submanifold is a further condition on H: it must be a Lie group, and

f|_{[𝔥,𝔥]} = 0.  (1.5)(c)

That is, f must define a one-dimensional representation of the Lie algebra of H. In the presence of (1.5)(c), the assumption that Λ_{f} is Lagrangian is equivalent to

dim H/G_{f} = \frac{1}{2} dim G/G_{f}.  (1.5)(d)

In order for the space of leaves to be a nice manifold M, we need H to be a closed subgroup of G; and in order for M to carry an appropriate G-equivariant Hermitian line bundle, we need a one-dimensional unitary representation

τ ∈ Ĥ,  dτ = 2πif.  (1.5)(e).

(Notice that if H is connected and simply connected, then (1.5)(c) guarantees the existence of a unique τ with differential 2πif.)

Conversely, suppose that X ≃ G/G_{f} is given, and that we can find a closed subgroup H ⊃ G_{f} together with a unitary character τ of H, satisfying (1.5)(c)–(e). Then τ defines a Hermitian line bundle L on M = G/H, and one can show that some open set in T^{∗}(M, L) is G-equivariantly symplectomorphic to a covering space of X. We will say that X is locally isomorphic to T^{∗}(M, L). In this way the geometric problem of relating coadjoint orbits to twisted cotangent bundles is reduced to the group-theoretic problem of finding appropriate subgroups H.

When G is a nilpotent group, the family of subgroups of G is rather rich; that is why one can find a group H making each coadjoint orbit a twisted cotangent bundle. As G becomes more complicated, the supply of subgroups dwindles. For reductive groups, one has the following remarkable result.

Theorem 1.6 (Ozeki and Wakimoto [16]) Suppose G is a reductive Lie group and f ∈ 𝔤^{∗}; write G_{f} for the isotropy group. Suppose 𝔥 ⊃ 𝔤_{f} is a Lie subalgebra of 𝔤 such that

i) the linear functional f vanishes on [𝔥, 𝔥]; and ii) dim 𝔥/𝔤(f) = \frac{1}{2} dim 𝔤/𝔤_{f}.

Then 𝔥 must be a parabolic subalgebra of 𝔤. In particular, the dimension of the coadjoint orbit G · f must be exactly twice the codimension of a parabolic subalgebra.

Ozeki and Wakimoto are actually much more precise about the relationship between f and 𝔥.

Corollary 1.7. Suppose G is a reductive Lie group, and X is a coadjoint orbit for G. Assume that the dimension of X is not equal to twice the codimension of any parabolic subalgebra of 𝔤. Then X is not locally isomorphic to a twisted cotangent bundle T^{∗}(M, L) for G.

The dimensions appearing in Corollary 1.7 are easy to compute. One finds, for example

Corollary 1.8. Suppose G is a split simple group over ℝ or ℂ, not of type A; and suppose X is a coadjoint orbit of minimal non-zero dimension. Then X is not locally isomorphic to a twisted cotangent bundle for G.

For these coadjoint orbits, the philosophy of geometric quantization as described above does not suggest a representation to attach to X. Our goal in this paper is to find an appropriate extension of that philosophy. The main idea, taken from [8] and [7], is to replace the Lagrangian foliation considered in (1.5) by a family of Lagarangian submanifolds which are allowed to overlap. Here is a formal definition.

Definition 1.9 (Guillemin-Sternberg and Ginsburg; see [8], Definition 2.1, and [7], A.1). Suppose X is a symplectic manifold. A Lagrangian covering of X is a diagram of smooth manifolds and smooth maps

Z
π ↙  ↘ ρ
X  M

subject to the two conditions below.

a) The diagram is a double fibration ([9], page 340). That is, the maps π and ρ are fibrations, and π × ρ is an embedding of Z in X × M.

This condition allows us to identify each fiber L_{m} = ρ^{−1}(m) with a subset of X, and each fiber M_{x} = π^{−1}(x) with a subset of M.

b) Each fiber L_{m} is a Lagrangian submanifold of X. If X carries a symplectic action of G, then we say that the Lagrangian covering is equivariant if Z and

M carry actions of G making π and ρ G-maps. It is called homogeneous if Z is a homogeneous space (in which case X and M must be as well).

Just as in the setting of (1.4) and (1.5), the manifold M is indexing a collection of Lagrangian submanifolds that cover X. The first observation, due essentially to Ginzburg, is that nice Lagrangian coverings often exist.

Theorem 1.10 (see [7], end of Appendix A). Suppose G is a complex reductive Lie group, and X = G · f ⊂ 𝔤^{∗} is a coadjoint orbit. Then there is an equivariant Lagrangian covering of X (Definition 1.9) with M = G/Q a partial flag variety for G.

A proof will be given in section 8; the main ingredient is a dimension estimate due to Spaltenstein in [19].

We turn now to a discussion of representation theory. In the setting of (1.5), the space of the representation was (roughly speaking) a space of sections of a line bundle L_{M} on M = G/H. We want to describe this space in terms of the symplectic manifold X = G/G_{f}. To do that, we first pull back the line bundle to a line bundle L_{X} on X; L_{X} is induced by the character τ_{X} = τ|_{G_{f}} of G_{f}. Because L_{X} is pulled back from M by the projection ρ: X → M, it makes sense to speak of sections of L_{X} that are “constant along the fibers of ρ.” These fibers are just the leaves of the Lagrangian foliation of X. Sections of L_{M} may be identified with sections of L_{X} constant along the leaves of our specified Lagrangian foliation.

Suppose now that we are in the setting of Definition 1.9, and that G is acting compatibly on X, Z, and M. In order to have a parallel construction in the setting of Definition 1.9, we need first of all a (G-equivariant) line bundle

L_{M} → M.  (1.11)(a)

(Henceforth we will omit mention of the assumed G-equivariance of the structures introduced.) We can then define L_{Z} to be the pullback of L_{M} to Z by the fibration ρ:

L_{Z} = ρ^{∗}(L_{M}).  (1.11)(b)

The representation we want will be on a space of sections of L_{M}; equivalently, on a space of sections of L_{Z} that are constant along the fibers of ρ. Recall that these fibers may be identified with Lagrangian submanifolds of X; so already we have a construction reminiscent of (1.5).

The full space of sections of L_{M} is too large to carry the representation we want, however. In Definition 1.9, suppose that X has dimension 2n, and that the fibers of π have dimension d. Then Z has dimension 2n + d. Since the fibers of ρ are Lagrangian in X, they have dimension n; so M has dimension n + d. The philosophy of geometric quantization says that X should correspond to a representation of “functional dimension” n; that is, to something like a space of sections of a line bundle on an n-dimensional manifold. If d is not zero, M is too large. (If d = 0, then Z is a covering of X, and so inherits from it a symplectic structure. The map ρ provides a nice Lagrangian foliation of Z, and we are (at least in the homogeneous case) very close to the setting (1.5).) So we need a way to pick out a nice subspace of sections of L_{M}. Carrying this out in detail will occupy most of this paper; for the moment we offer only a brief sketch.

There is a fiber bundle B = B(X) over X for which the fiber B_{x} over x is the variety of Lagrangian subspaces of T_{x}(X). (Thus B_{x} is a compact manifold of dimension n(n + 1)/2.) We define a bundle map over X

τ: Z → B  (1.11)(c)

as follows. Suppose z ∈ Z; write x = π(z) ∈ X and m = ρ(m) ∈ M. Then the fiber L_{m} of ρ over m is a Lagrangian submanifold of X containing x; so its tangent space T_{x}(L_{m}) is a Lagrangian subspace of T_{x}(X). We set τ(z) = T_{x}(L_{m}).

Since each point of B is an n-dimensional real vector space, there is a tautological n-dimensional real vector bundle over B. Taking its top exterior power and complexifying provides a line bundle D_{B} → B. (The D stands for determinant.) Roughly speaking, we need a square root L_{B} of D_{B}. A little more precisely, we need a twisted version of Kostant’s “symplectic spinors” on the base symplectic manifold X (see [12]). This is an infinite-dimensional vector bundle

S_{X} → X;  (1.11)(d)

the fiber over x may be identified with the smooth part of the metaplectic representation attached to the symplectic vector space T_{x}(X), tensored with a one-dimensional twist. When X is a coadjoint orbit, a Gequivariant family of twisted symplectic spinors will be specified by an “admissible orbit datum” in the sense of Duflo (Definition 6.2). The reducibility of the metaplectic representation gives a natural decomposition S = S^{even} ⊕ S^{odd}. From these twisted spinors we will construct the line bundle L_{B}, and an embedding

smooth sections of S^{even} ↪ smooth sections of L_{B}  (1.11)(e)

Now we can use the map τ: Z → B to pull L_{B} back to Z. The last ingredient we will need is an isomorphism

τ^{∗}(L_{B}) ≃ L_{Z}.  (1.11)(f)

Using the embedding (1.11)(e) we can identify S^{even} with a space of sections of L_{B}. These sections may be pulled back using τ to sections of τ^{∗}(L_{B}) over Z, and then identified with sections of L_{Z} using (1.11)(f). Write W_{Z}^{even} for the resulting space of sections of L_{Z}. Then the (smooth) representation of G we want to consider is on the space

W_{Z}^{even} ∩ (smooth sections of L_{M})  (1.11)(g)

inside sections of L_{Z}. The problem of constructing a unitary structure on this space we will leave to a future paper. (Even to guarantee the existence of an invariant Hermitian form requires an additional assumptions on L_{M}.)

2. Symplectic and Poisson manifolds. We begin by recalling the definition of symplectic manifold.

Definition 2.1. A symplectic manifold is a smooth manifold X endowed with a 2-form ω_{X}, subject to the following conditions. Recall first of all that a 2-form may be regarded as a smoothly varying family of skew-symmetric bilinear forms ω_{x} on the various tangent spaces T_{x}(X). We impose two conditions on ω_{X}.

a) Each form ω_{x} is non-degenerate; that is, for every non-zero tangent vector v ∈ T_{x}(X) there is a vector w ∈ T_{x}(X) so that ω_{x}(v, w) ≠ 0.

b) The 2-form ω_{X} is closed. Condition (a) is equivalent to the assumption that dim X = 2n is even, and that the nth exterior power

ω_{X}^{n} is a nowhere-vanishing volume form on X.

This definition may be made equally well in several other categories. We can define a complex symplectic manifold, which is a complex manifold endowed with a holomorphic 2-form satisfying analogues of (a) and (b); or a complex symplectic algebraic variety, which is a smooth complex algebraic variety endowed with an algebraic 2-form. We will invoke these generalizations as needed.

Any bilinear form on a vector space may be interpreted as a linear map from the vector space to its dual. On a symplectic manifold we therefore have maps from tangent spaces to cotangent spaces

τ_{x}: T_{x}(X) → T_{x}^{∗}(X),  τ_{x}(v)(w) = ω_{x}(w, v).  (2.2)(a)

The non-degeneracy hypothesis on ω_{X} means that these maps are all isomorphisms, so they define a bundle isomorphism

τ_{X}: T (X) → T^{∗}(X)  (2.2)(b)

from the tangent bundle to the cotangent bundle. If f ∈ C^{∞}(X) is a (real-valued) smooth function, then df is a 1-form; that is, a smooth section of the cotangent bundle. We may therefore define

ξ_{f} = τ^{−1}(df),  (2.2)(c)

a smooth section of the tangent bundle; that is, ξ_{f} is a vector field on X. It is called the Hamiltonian vector field of f. By inspection of the definitions, we see that ξ_{f} has the following characteristic property: if γ is any vector field on X, then

ω_{X}(γ, ξ_{f}) = γ · f;  (2.2)(d)

both sides are smooth functions on X.

Using the Hamiltonian vector fields, we can now define the Poisson bracket of smooth functions f and g on the symplectic manifold X:

{f, g} = ξ_{f} · g = dg(ξ_{f}) = ω_{X}(ξ_{f}, ξ_{g}) = −ξ_{g} · f.  (2.2)(e)

Here the second expression may be taken as the definition; the remainining equalities are the definition of dg and (2.2)(d).

Proposition 2.3. Suppose (X, ω_{X}) is a symplectic manifold.

i) The Poisson bracket (2.2)(e) defines a Lie algebra structure on C^{∞}(X). ii) The map f ↦ ξ_{f} is a Lie algebra homomorphism from C^{∞}(X) to the Lie algebra of vector fields on X.

Its kernel consists of the locally constant functions on X. iii) For each f ∈ C^{∞}(X), the endomorphism g ↦ {f, g} is a derivation of C^{∞}(X). That is, it is linear in

g, and

{f, gh} = {f, g}h + g{f, h}.

iv) If we identify derivations of C^{∞}(X) with vector fields on X, then bracket with f corresponds to the

Hamiltonian vector field ξ_{f}:

{f, g} = ξ_{f} · g.

The elementary proof may be found in [2], Chapter 8 or [1], Chapter 3; both references use a different sign convention from ours. The Jacobi identity for the Lie algebra structure amounts to the fact that ω_{X} is closed. Of course (iv) is just the definition of the Poisson bracket given in (2.2)(e). One may also interpret it as defining the Hamiltonian vector field ξ_{f} in terms of the Poisson bracket.

For most of what we do, the Poisson bracket is more fundamental than the symplectic structure. At the same time, a Poisson bracket can be defined even on some singular spaces (like closures of nilpotent coadjoint orbits) where a symplectic structure does not make sense. We therefore recall a few highlights from the theory of Poisson spaces.

Definition 2.4. A Poisson algebra A over a field F is a commutative algebra (with 1) over F endowed with a Poisson bracket

{, }: A × A → A,

subject to the following conditions.

a) The Poisson bracket makes A a Lie algebra over F. That is, it is bilinear, skew-symmetric, and satisfies

the Jacobi identity

{f, {g, h}} = {{f, g}, h} + {g, {f, h}} (f, g, h ∈ A).

b) For each f ∈ A, the endomorphism ξ_{f} of A defined by

ξ_{f} · g = {f, g}

is a derivation:

{f, gh} = {f, g}h + g{f, h}.

We call ξ_{f} the Hamiltonian vector field of f. The Jacobi identity for the Poisson bracket says that the map

A → Der A, f ↦ ξ_{f}

is a Lie algebra homomorphism. (Recall that the commutator of two derivations of A is a derivation; this defines the Lie algebra structure on Der A.)

A Poisson manifold is a smooth manifold X endowed with a real Poisson algebra structure on C^{∞}(X). An affine Poisson algebraic variety (over F) is an affine algebraic variety X endowed with a Poisson algebra structure on the algebra R(X) of regular functions on X. A Poisson algebraic variety is a an algebraic variety for which the sheaf of rings is a sheaf of Poisson algebras. More generally, a Poisson space is a ringed space in which the sheaf of rings is a sheaf of Poisson algebras.

There is a small consistency problem to worry about in the last definitions: whether an affine Poisson algebraic variety X is a Poisson algebraic variety. In fact it is, essentially because Poisson structures localize well: after inverting an element h, one can (and is forced to) define

{f/h, g/h} = (h{f, g} − f{h, g} − g{f, h}) /h^{3}.

In this way a Poisson structure on an algebra gives rise to Poisson structures on all localizations; so the Poisson structure on R(X) makes the sheaf of rings on X into a sheaf of Poisson algebras. Similar remarks apply in the setting of manifolds: a Poisson structure on C^{∞}(X) gives rise to one on the sheaf of germs of smooth functions. In this case the key fact is that the Poisson bracket is local: that if f vanishes near x, then all Poisson brackets {f, h} also vanish near x.

Let us see how to recover something close to a symplectic structure from a Poisson structure. To fix ideas we will concentrate on the case of algebraic varieties, but it is easy to give a parallel discussion for manifolds. What we will see is that a Poisson structure on X provides a foliation of “most” of X by symplectic manifolds; Poisson brackets are computed by restricting to the leaves and using the symplectic Poisson brackets there.

Suppose therefore that A is a Poisson algebra over F. Write X = Spec A; then A can be thought of as an algebra of functions on X. A closed point x of X is a maximal ideal 𝔪_{x} ⊂ A. Write

F_{x} = A/𝔪_{x},  (2.6)(a)

an extension field of F. Recall that the Zariski cotangent space of X at x is the F_{x} vector space

T_{x}^{∗}(X) = 𝔪_{x}/𝔪_{x}^{2}.  (2.6)(b)

It is clear from (b) in Definition 2.4 that

{𝔪_{x}, 𝔪_{x}^{2}} ⊂ 𝔪_{x}.  (2.6)(c)

The Poisson bracket therefore descends to a skew-symmetric F_{x}-bilinear pairing

{, }_{x}: T_{x}^{∗}(X) × T_{x}^{∗}(X) → F_{x},  {f + 𝔪_{x}^{2}, g + 𝔪_{x}^{2}} = {f, g} + 𝔪_{x}  (2.6)(d)

using the identifications (2.6)(a) and (2.6)(b). Write R_{x} for the radical of {, }_{x}; the form {, }_{x} descends to a non-degenerate skew-symmetric form on T_{x}^{∗}/R_{x}. The Zariski tangent space at x is the dual space

T_{x}(X) = (T_{x}^{∗}(X))^{∗}  (2.6)(e)

Using the form {, }_{x} each cotangent vector v defines a tangent vector δ(v):

δ(v)(w) = {w, v}_{x}  (v, w ∈ T_{x}^{∗}(X)).  (2.6)(f)

Write S_{x} ⊂ T_{x}(X) for the image of δ, and R_{x}^{⊥} ⊂ T_{x}(X) for the annihilator of R_{x} in T_{x}. Then

S_{x} ⊂ R_{x}^{⊥} ≃ (T_{x}^{∗}(X)/R_{x})^{∗}  (2.6)(g)

On the other hand, the map δ factors to an isomorphism

\overline{δ}: T_{x}^{∗}(X)/R_{x} ≃ S_{x}.  (2.6)(h)

If S_{x} is finite-dimensional, it follows that S_{x} = R_{x}^{⊥}; so we can use \overline{δ} to transfer the skew-symmetric form \{, \}_{x} to a non-degenerate skew-symmetric form ω_{x} on S_{x}. (In the case of a symplectic manifold, the map \overline{δ} is just the inverse of the map τ of (2.2)(b).)

Suppose now that X is an irreducible complex affine algebraic variety, with A the algebra of regular functions. Then all the Zariski cotangent spaces are finite-dimensional, of dimension at least equal to the dimension n of X. Equality holds exactly on the smooth part X_{s} of X, which is a dense open subset. Write 2r(x) for the rank of the bilinear form {, }_{x}. It is easy to check that each point x has an open neighborhood U_{x} with r(y) ≥ r(x) for all y ∈ U_{x}. Consequently the rank function assumes its maximum value r on a dense open subset U_{s} of X_{s}. On U_{s}, the subspaces R_{x} define a subbundle of the cotangent bundle of rank n − 2r. The annihilators of these spaces define a subbundle R^{⊥} = S of the tangent bundle T U_{s}, of rank 2r. It is not difficult to see that the fiber S_{x} consists precisely of the values at x of all Hamiltonian vector fields ξ_{f}. Recall that we have defined non-degenerate skew-symmetric forms ω_{x} on each fiber S_{x}.

We now do a little holomorphic differential geometry on X. Since the Hamiltonian vector fields are closed under Lie bracket, it follows easily that the distribution S is integrable; that is, that the family of smooth vector fields with values in S is closed under Lie bracket. The Frobenius theorem therefore provides a smooth foliation of U_{s} by complex submanifolds of dimension 2r. Writing S for the submanifold through x, we have T_{x}S = S_{x}. It is not difficult to check that the forms ω_{x} define a holomorphic symplectic structure on S, and that the corresponding Poisson brackets fit together to give the original Poisson bracket on U_{s}, as we wished to show.

Poisson manifolds have sometimes been defined in such a way that the nice open set U_{s} is all of X. This approach excludes interesting behavior that we want to consider, however.

We record one consequence of these ideas: a characterization of symplectic manifolds among Poisson manifolds.

Proposition 2.7. Suppose X is a smooth Poisson manifold. The following conditions are equivalent.

a) The Poisson bracket arises from a symplectic structure ω_{X} on X.

b) For every x ∈ X, the radical R_{x} ⊂ T_{x}^{∗}(X) of {, }_{x} is zero (cf. (2.6)(d)).

c) For every x ∈ X, the subspace S_{x} ⊂ T_{x}(X) is all of T_{x}(X).

d) For every x ∈ X, the collection {ξ_{f}(x)|f ∈ C^{∞}(X)} of values at x of Hamiltonian vector fields is all of T_{x}(X).

We leave the proof to the reader. Of course there is a parallel statement for algebraic varieties.

Here is a fundamental example. Suppose 𝔤 is a finite-dimensional real Lie algebra. Let {X_{1}, …, X_{n}} be a basis of 𝔤, and {λ_{1}, …, λ_{n}} the dual basis of the dual vector space 𝔤^{∗}. Each λ_{i} gives rise to a vector field \frac{∂}{∂λ_{i}} on 𝔤^{∗}. We define a Poisson bracket on C^{∞}(𝔤^{∗}) by

\{f, g\}(λ) = ∑_{i,j} λ([X_{i}, X_{j}])\frac{∂f}{∂λ_{i}} \frac{∂g}{∂λ_{j}} .  (2.8)(a)

(Notice that this bracket preserves the subspace S(𝔤) of polynomial functions on 𝔤^{∗}. It therefore makes 𝔤^{∗} into a real affine Poisson algebraic variety.) Among the axioms for a Poisson algebra, the only difficult one to check is the Jacobi identity. To compute the iterated Poisson brackets appearing there, we need to compute the derivative

\frac{∂λ([X_{i}, X_{j}])}{∂λ_{q}.}

The function being differentiated is linear, so the derivative is constant; it is λ_{q}([X_{i}, X_{j}]). Therefore

\{f, \{g, h\}\} = ∑_{p,q,i,j} λ([X_{p}, X_{q}])\frac{∂f}{∂λ_{p}} (λ_{q}([X_{i}, X_{j}])\frac{∂g}{∂λ_{i}} \frac{∂h}{∂λ_{j}} + λ([X_{i}, X_{j}]) (\frac{∂^{2}g}{∂λ_{q}∂λ_{i}} \frac{∂h}{∂λ_{j}} + \frac{∂g}{∂λ_{i}} \frac{∂^{2}h}{∂λ_{q}∂λ_{j}} )) .

The other iterated brackets in the Jacobi identity are similar. The terms involving second derivatives cancel for easy reasons. The remaining term is

∑_{p,q,i,j} λ([X_{p}, X_{q}])λ_{q}([X_{i}, X_{j}])\frac{∂f}{∂λ_{p}} \frac{∂g}{∂λ_{i}} \frac{∂h}{∂λ_{j}} .

Now for any Y ∈ 𝔤 we have

Y = ∑_{q} λ_{q}(Y)X_{q}.

Inserting this above gives

∑_{p,i,j} λ([X_{p}, [X_{i}, X_{j}]])\frac{∂f}{∂λ_{p}} \frac{∂g}{∂λ_{i}} \frac{∂h}{∂λ_{j}} .

Now it is more or less obvious that the contribution of these terms to the Jacobi identity for the Poisson bracket vanishes because of the Jacobi identity for 𝔤.

Although the expression (2.8)(a) for the Poisson bracket is very attractive, there is a less symmetric formulation that offers more information. The Lie algebra 𝔤 acts on 𝔤^{∗} by the coadjoint action:

(ad^{∗}(Y) · τ)(X) = −τ(ad(Y)(X)) = τ([X, Y]).

In this way every element Y of 𝔤 defines a vector field on 𝔤^{∗}. By abuse of notation, we will still call this vector field Y. One computes easily that

Y (λ) = ∑_{j} λ([Y, X_{j})])\frac{∂}{∂λ_{j}} .

Using this formula, we can rewrite (2.8)(a) as

\begin{aligned}\{f, g\} & = ∑_{i} \frac{∂f}{∂λ_{i}} (X_{i} · g) \\ & = − ∑_{i}(X_{i} · f)\frac{∂g}{∂λ_{i}} . \end{aligned}  (2.8)(b)

Here the action of X_{i} is the coadjoint action.

Suppose G is a Lie group with Lie algebra 𝔤. The conclusion we want to draw from (2.8)(b) is this. Suppose S ⊂ 𝔤^{∗} is preserved by the coadjoint action of G. Let g be any smooth function on 𝔤^{∗} vanishing on S, and f any smooth function. Then we claim that the Poisson bracket {f, g} also vanishes on S. The reason is that X_{i} ·g may be computed at λ by differentiating g along the path Ad^{∗}(exp(tX_{i}))·λ. If λ belongs to S, then this path is contained in S; so g vanishes on the path, and the derivative is zero. The first formula in (2.8)(b) therefore shows that {f, g} vanishes at λ.

What follows is that the Poisson bracket on C^{∞}(𝔤^{∗}) descends to the algebra of restrictions to S of smooth functions on 𝔤^{∗}. Here is a formal setting for this fact.

Definition 2.9. Suppose A is a Poisson algebra. An ideal J ⊂ A is called a Poisson ideal if {A, J} ⊂ J. In this case the quotient algebra A/J inherits a Poisson algebra structure.

There is a minor technical point about restricting smooth functions that should be addressed as well.

Definition 2.10. Suppose M is a smooth manifold and S ⊂ M. A function f on S is said to be smooth if for every s ∈ S there is a neighborhood U_{s} of s in M and a smooth function f_{s} ∈ C^{∞}(U_{s}) with the property that f_{s}|_{U_{s}∩S} = f|_{U_{s}∩S}. In this way S becomes a ringed space: the value of the sheaf on an open set V ⊂ S is C^{∞}(V)

If S is closed then a partition of unity argument shows that C^{∞}(S) = C^{∞}(M)|_{S}.

Proposition 2.11. Suppose G is a Lie group, S ⊂ 𝔤^{∗} is an Ad^{∗}(G)-stable subset, and U ⊂ 𝔤^{∗} is open. Then the ideal J_{S}(U) of smooth functions on U vanishing on S ∩ U is a Poisson ideal in C^{∞}(U). Consequently C^{∞}(S) is a Poisson algebra; and the ringed space (S, C^{∞}) is a Poisson space.

Suppose now that G is an algebraic group (over ℝ or ℂ) and X ⊂ 𝔤^{∗} is an Ad^{∗}(G)-stable subvariety. Then X is an affine Poisson algebraic variety, with algebra of functions

R(X) = S(𝔤)/(J_{X} ∩ S(𝔤)).

This is clear from the discussion after (2.8)(b).

Fix λ ∈ 𝔤^{∗}. Using (2.8)(b) it is easy to check that the space S_{λ} defined in (2.6)(g) is just the set of values at λ of the coadjoint action vector fields. This is precisely the tangent space to the orbit through λ:

S_{λ} = {ad^{∗}(Y) | Y ∈ 𝔤} = T_{λ}(G · λ).  (2.12)(a)

The formula (2.8)(b) also shows that if Y is regarded as a (linear) function on 𝔤^{∗}, then the derivation (Poisson bracket with Y) is just the coadjoint action of Y:

ξ_{Y} = ad^{∗}(Y).  (2.12)(b)

(This is clearest for the basis vectors X_{i}, but it follows at once for all Y ∈ 𝔤.)

Corollary 2.13. Suppose G is a Lie group, and X is an orbit of the coadjoint action of G on 𝔤^{∗}. Then the Poisson structure on X provided by Proposition 2.11 is actually a symplectic structure. At a point λ ∈ X, the symplectic form on T_{λ}(X) is given by

ω_{λ}(ξ_{Y} , ξ_{Z}) = λ([Y, Z]) (Y, Z ∈ 𝔤).

Sketch of proof. We give X its topology as a homogeneous space for G, to make it a manifold. In order to apply Proposition 2.11 we need to assume that this agrees with the subspace topology from 𝔤^{∗}. This is automatic if G is reductive, so we omit a discussion of the minor modification of Proposition 2.11 needed for the general case. At any rate, we find on X the structure of a Poisson manifold. By the construction in Proposition 2.11, the Hamiltonian vector fields on X are just the restrictions to X of Hamiltonian vector fields on 𝔤^{∗}. By (2.12), these span all the tangent spaces to X. By Proposition 2.7, it follows that X is symplectic. To compute the form explicitly, we use the ideas in (2.6). They show that

ω_{x}(ξ_{f}(x), ξ_{g}(x)) = {f, g}(x)  (2.14)(a)

whenever f and g belong to the maximal ideal 𝔪_{x} of functions vanishing at x. Poisson bracket with a constant is zero; so both sides of (2.14) are unchanged if we add constants to f and g. Therefore (2.14) is true for all f and g. In the case of a coadjoint orbit, we take Y and Z to be linear functions on 𝔤^{∗} corresponding to elements of 𝔤; then the conclusion is that

ω_{λ}(ξ_{Y} , ξ_{Z}) = {Y, Z}(λ).  (2.14)(b)

Finally, notice that the Poisson bracket of Y and Z is just the linear functional [Y, Z]:

{Y, Z} = [Y, Z]  (2.14)(c)

This is clear from (2.7)(a) if Y and Z belong to the basis of 𝔤, and the general case is immediate from bilinearity. Now the formula for ω_{λ} in the proposition follows from (2.14)(b) and (2.14)(c). Q.E.D.

3. Hamiltonian G-spaces and moment maps. In this section we recall a little of the general theory of Poisson spaces with Lie group actions. Since Lie groups are assembled from one-parameter subgroups, it is helpful to begin by examining one-parameter groups of automorphisms. We are mostly interested in motivating the definition of Hamiltonian actions, so there is no need to strive for maximum generality. We begin therefore with a Poisson manifold X and a one-parameter group {U_{t}|t ∈ ℝ} of automorphisms of X. This means first of all that each U_{t} is a smooth automorphism of X, and that U_{t}U_{s} = U_{t+s}. We can make U_{t} an algebra automorphism of C^{∞}(X) by (U_{t}f)(x) = f(U_{−t}x); then the last requirement is that these automorphisms preserve the Poisson bracket.

We now have a homomorphism ℝ → End(C^{∞}(X)). A sensible requirement to impose on the original automorphisms U_{t} is that this should be differentiable. This is equivalent to the differentiability of the map

ℝ × X → X, (t, x) ↦ U_{t}(x).

The derivative at 0 of U_{t} is an endomorphism ξ(U) of C^{∞}(X): the defining relation is

\frac{d}{dt}(U_{t}f)|_{t=0} = ξ(U)(f).  (3.1)(a)

By differentiating the requirement that U_{t} is an algebra automorphism, we find that ξ(U) is a derivation:

ξ(U)(fg) = ξ(U)(f)g + fξ(U)(g).  (3.1)(b)

This means that ξ(U) is a vector field on X. Differentiating the requirement that U_{t} preserve Poisson brackets shows that ξ(U) is a derivation of the Poisson bracket:

ξ(U)({f, g}) = {ξ(U)(f), g} + {f, ξ(U)(g)}.  (3.1)(c)

We can recover U_{t} from ξ(U) by solving the differential equation

\frac{d}{dt}(U_{t}(f)) = ξ(U)(U_{t}(f)),  U_{0}(f) = f.  (3.1)(d)

This follows immediately from (3.1)(a) and the composition law for U_{t}.) Conversely, suppose ξ(U) is an endomorphism of C^{∞}(X) satisfying (3.1)(b) and (3.1)(c), and U_{t} is a family of endomorphisms of C^{∞}(X) satisfying (3.1)(d). Then U_{t} is a one-parameter group of Poisson algebra automorphisms of C^{∞}(X). This identifies smooth one-parameter groups U of automorphisms of the Poisson algebra C^{∞}(X) with certain vector fields ξ(U) satisfying (3.1)(c). (Not all vector fields appear, because the differential equation (3.1)(d) may not have a solution for all t.) But it certainly suggests that we should look for interesting vector fields satisfying (3.1)(c).

If f ∈ C^{∞}(X), then the Hamiltonian vector field ξ_{f} of Definition 2.4 satisfies (3.1)(c). If there is a corresponding one-parameter group U(f)_{t} of automorphisms of X, then the automorphisms must preserve each leaf of the symplectic foliation (of an open subset of X) discussed in (2.6). More general Poisson automorphisms may permute these leaves. It is natural to regard the automorphisms attached to Hamiltonian vector fields as “inner.” Here is a definition.

Definition 3.2. Suppose X is a Poisson manifold (Definition 2.4) endowed with a smooth action of a Lie group G by Poisson automorphisms. We say that the action is Hamiltonian (or that X is a Hamiltonian G-space) if we are given a linear map

μ̃: 𝔤 → C^{∞}(X)

with the following properties.

a) The map μ̃ intertwines the adjoint action of G with its action on C^{∞}(X). b) For every Y ∈ 𝔤, write Y_{t} for the one-parameter group of automorphisms of X given by the action of

exp(tY). Then the corresponding vector field ξ(Y) on X (cf. (3.1)) is the Hamiltonian vector field attached to the function μ̃(Y) on X.

The assumption is slightly stronger than the requirement that each one-parameter subgroup in G be generated by a Hamiltonian vector field. By differentiating the requirement in (a) and using (b), we find that μ̃ must be a Lie algebra homomorphism.

A linear map from a finite-dimensional vector space into C^{∞}(X) is precisely the same thing as a smooth map from X to the dual vector space. In the setting of Definition 3.2, we define the moment map for the Hamiltonian G-space to be

μ: X → 𝔤^{∗},  μ(x)(Y) = μ̃(Y)(x)  (Y ∈ 𝔤, x ∈ X).  (3.3)(a)

The same formula shows how to define μ̃ in terms of μ. The first condition in Definition 3.2 may be immediately reformulated in terms of μ: it just says

μ intertwines the action of G on X with the coadjoint action on 𝔤^{∗}.  (3.3)(b)

For the second condition, recall that the smooth map μ gives a pullback map on smooth functions:

μ^{∗}: C^{∞}(𝔤^{∗}) → C^{∞}(X).

We know that μ̃ is a Lie algebra homomorphism. This turns out to be equivalent to the requirement that μ^{∗} preserve Poisson brackets:

{μ^{∗}f, μ^{∗}g} = μ^{∗}{f, g}  (f, g ∈ C^{∞}(𝔤^{∗}))  (3.3)(c)

(That (3.3)(c) implies that μ̃ is a Lie algebra homomorphism is obvious from (2.14)(c). For the other direction, we try to prove (3.3)(c) at a point x ∈ X, say with μ(x) = λ. We can find “Taylor” expansions for f and g at λ, as

f = a + ∑_{i} a_{i}(X_{i} − λ(X_{i})) + ∑_{i≤j}(X_{i} − λ(X_{i}))(X_{j} − λ(X_{j}))f_{ij};

here a and a_{i} are constants, and f_{ij} are smooth functions. Of course there is a similar expansion for g, and we can pull them all back to X by μ^{∗}. Now use these expansions to compute Poisson brackets at x. The constants like a and λ(X_{i}) drop out. The linear terms satisfy

{μ^{∗}(X_{i}), μ^{∗}(X_{j})} = μ^{∗}{X_{i}, X_{j}}

because μ^{∗}(X_{i}) = μ̃(X_{i}), and we are assuming that μ̃ is a Lie algebra homomorphism. The quadratic terms belong to the square of the maximal ideal of functions vanishing at x, and so don’t contribute to the Poisson bracket at x (compare (2.6)(c)). The conclusion is that (3.3)(c) holds at the point x.) Finally, we can obviously write condition (b) of Definition 3.2 as

for every Y ∈ 𝔤, the vector field on X induced by Y is the Hamiltonian vector field ξ_{μ^{∗}Y}.  (3.3)(d)

We have shown that Definition 3.2 is equivalent to

Definition 3.4. Suppose X is a Poisson manifold endowed with a smooth action of a Lie group G by Poisson automorphisms. We say that the action is Hamiltonian if we are given a smooth G-equivariant Poisson mapping (the moment map)

μ: X → 𝔤^{∗}

with the following property: for every Y ∈ 𝔤, write ξ(Y) for the vector field on X induced by the oneparameter group of automorphisms given by the action of exp(tY). Then

ξ(Y) = ξ_{μ^{∗}Y} .

Here on the right we regard Y as a linear functional on 𝔤^{∗}, pull it back to a smooth function on X via μ, and then form the corresponding Hamiltonian vector field.

In light of (3.1)(d), this condition on μ may be reformulated as follows: for every smooth function f on X, we have

\frac{d}{dt}(f(exp(−tY) · x))|_{t=0} = \{μ^{∗}Y, f\}.

The notion of Hamiltonian G-space extends readily to some singular spaces. The best general theory of such spaces is in the algebraic setting, so we begin with that. Suppose X is a complex affine Poisson algebraic variety, with R(X) its ring of regular functions. (This means that R(X) is a complex Poisson algebra that is finitely generated as a commutative algebra.) If G is an algebraic group, then an algebraic action of G on X is just an algebraic action of G on R(X) by Poisson algebra automorphisms:

G × R(X) → R(X).  (3.5)(a)

To say that this action is algebraic means that every element of R(X) belongs to a finite-dimensional Ginvariant subspace V ⊂ R(X), and that the corresponding homomorphism from G to GL(V) is algebraic. Such a homomorphism has a differential, which is a Lie algebra homomorphism from 𝔤 to End(V). (The Lie algebra structure on End(V) is commutator of linear transformations.) The action of G on R(X) may therefore be differentiated to get a Lie algebra homomorphism

ξ: 𝔤 → End(R(X)).  (3.5)(b)

Because G acts by algebra automorphisms, ξ acts by derivations:

ξ(Y)(fg) = (ξ(Y)f)g + f(ξ(Y)g)  (3.5)(c)

Similarly, ξ acts by Poisson algebra derivations:

ξ(Y)({f, g}) = {ξ(Y)f, g} + {f, ξ(Y)g}.  (3.5)(d)

Definition 3.6. Suppose X is a complex affine Poisson algebraic variety, and G is an algebraic group acting algebraically on X by Poisson automorphisms. Write

ξ: 𝔤 → End(R(X))

for the differential of this action. We say that the action is Hamiltonian if we are given a linear map

μ̃: 𝔤 → R(X)

with the following properties.

a) The map μ̃ intertwines the adjoint action of G with its action on R(X).

b) For every Y ∈ 𝔤, the endomorphism ξ(Y) of R(X) is the Hamiltonian vector field attached to μ̃(Y):

ξ(Y) = ξ_{μ̃(Y)}.

Just as in the setting of manifolds, this definition may conveniently be recast in terms of moment maps.

Definition 3.7. Suppose X is a complex affine Poisson algebraic variety, and G is an algebraic group acting algebraically on X by Poisson automorphisms. We say that the action is Hamiltonian if we are given an algebraic G-equivariant Poisson mapping (the moment map)

μ: X → 𝔤^{∗}

with the following property: the differential ξ of the G action on X is given by

ξ(Y) = ξ_{μ^{∗}Y} .

Here on the right we regard Y as a linear function on 𝔤^{∗}, pull it back to a regular function on X via μ, then form the corresponding Hamiltonian vector field.

Similarly we can define Hamiltonian actions on general algebraic varieties. To define a Hamiltonian action on a general Poisson space, we need to know how to differentiate the group action on the sheaf of rings, in order to write a condition like (b) in Definition 3.6. As usual we avoid trying to write the most general natural definition, since we do not know what that might be.

In the philosophy of geometric quantization, Hamiltonian G-spaces are the (crudest) classical analogues of unitary representations of G. Corresponding to irreducible representations are the homogeneous Hamiltonian G-spaces. These were classified by Kostant.

Proposition 3.8 ([11], Theorem 5.4.1.) Suppose X is a homogeneous Hamiltonian Poisson G-manifold (Definition 3.4). Then the moment map μ exhibits X as a covering of a coadjoint orbit G · f ⊂ 𝔤^{∗}; so X is actually a symplectic manifold.

Proof. Fix x ∈ X, and write G_{x} for the isotropy group at x; so X ≃ G/G_{x}. Define f = μ(x) ∈ 𝔤^{∗}, and write G_{f} for the isotropy group; then G_{x} ⊂ G_{f} since μ is G-equivariant. It remains to show that G_{x} is open in G_{f}; that is, that the two groups have the same Lie algebra. For Y ∈ 𝔤, we regard Y as a linear function on 𝔤^{∗}, and pull it back to a smooth function μ^{∗}Y on X. By the definition of Hamiltonian action, this map from 𝔤 to C^{∞}(X) is a Lie algebra homomorphism. Exactly as in the proof of Corollary 2.13, we may therefore compute

ω_{x}(ξ_{μ^{∗}Y} , ξ_{μ^{∗}Z}) = μ^{∗}([Y, Z])(x)
= [Y, Z](μ(x)) = [Y, Z](f)
= ω_{f}(ξ_{Y} , ξ_{Z}).

Suppose now that Y ∈ 𝔤_{f}. Then ξ_{Y} (f) belongs to the radical of the form ω_{f}, so the preceding equation implies that ξ_{μ^{∗}Y} (x) belongs to the radical of ω_{x}. Because ω_{x} is non-degenerate on the values at x of Hamiltonian vector fields (see the discussion after (2.6)), it follows that ξ_{μ^{∗}Y} (x) = 0. By Definition 3.2(b), it follows that the vector field ξ(Y) vanishes at x; that is, that Y ∈ 𝔤_{x}. The other containment 𝔤_{x} ⊂ 𝔤_{f} is clear. Q.E.D.

Here is a standard way to construct Hamiltonian G-spaces. Suppose M is a manifold with a smooth action of G. This means in particular that we are given a Lie algebra homomorphism ξ from 𝔤 to vector fields on M. Now let X = T^{∗}M be the cotangent bundle of M with its standard symplectic structure. The action of G by diffeomorphisms of M automatically lifts to an action of G by symplectomorphisms of X. Now each vector field τ on M may be identified with a smooth function f_{τ} on X; the value of f_{τ} at the cotangent vector (m, v) is given by evaluating the tangent vector τ(m) on the covector v. This mapping sends Lie bracket of vector fields to Poisson bracket of functions on X. The map μ̃ for the Hamiltonian G-space X (Definition 3.2) sends Y ∈ 𝔤 to the function f_{ξ(Y)}. One can check that this map makes X a Hamiltonian G-space; of course the difficult part is condition (b) of Definition 3.2. The corresponding moment map μ sends the cotangent vector (m, v) to the linear functional Y ↦ ⟨ξ(Y)(m), v⟩ on 𝔤. Here the pairing on the right is between tangent vectors and covectors at m.

4. Lagrangian subspaces. A central notion in symplectic geometry is that of Lagrangian submanifold. In this section we will consider how to extend that notion to Poisson spaces. We begin with the simplest linear setting, and gradually generalize it.

Definition 4.1. Suppose F is a field, W is a finite-dimensional F -vector space, and ω is a nondegenerate skew-symmetric bilinear form on W. (We say that the pair (W, ω) is a symplectic vector space over F.) The symplectic group of W is the group Sp(W) of F -linear transformations of W preserving ω. In analogy with (2.2), we define a linear isomorphism

τ: W → W^{∗},  τ(v)(w) = ω(w, v).  (4.1)(a)

Suppose V ⊂ W is any subspace. Set

V^{⊥} = {w ∈ W | ω(w, V) = 0},  (4.1)(b)

the preimage under τ of the annihilator of V in W^{∗}. If we need to emphasize the dependence on ω, we may write V^{⊥,ω}. Then τ factors to a linear isomorphism

τ_{V}: W/V^{⊥} → V^{∗}.  (4.1)(c)

From this it follows that

dim V + dim V^{⊥} = dim W.  (4.1)(d)

Evidently (V^{⊥})^{⊥} ⊃ V; because (4.1)(d) shows that these spaces have the same dimension, we get

(V^{⊥})^{⊥} = V.  (4.1)(e)

Dually, the restriction of τ to V provides an isomorphism

δ_{V}: V → (W/V^{⊥})^{∗}.  (4.1)(f)

This map is the transpose of τ_{V}.

We say that a subspace I ⊂ W is isotropic if I ⊂ I^{⊥}; that is, if ω|_{I} = 0. We say C is co-isotropic if C^{⊥} ⊂ C; that is, if C^{⊥} is isotropic. We say L is Lagrangian if it is both isotropic and co-isotropic; that is, if L^{⊥} = L.

Definition 4.2. Suppose (W, ω) is a symplectic vector space over F of dimension 2n. The Lagrangian Grassmannian is the collection B(W) of Lagrangian subspaces of W; it is a subset of the Grassmannian of all n-dimensional subspaces L of W, defined by the algebraic condition ω|_{L} = 0.

Fix a Lagrangian subspace L of W. The Siegel parabolic subgroup defined by L is the isotropy group

P (L) = {g ∈ Sp(W) | gL ⊂ L}.  (4.2)(a)

Its unipotent radical is the normal subgroup

U(L) = {g ∈ P (L) | g|_{L} = Id_{L}}.  (4.2)(b)

Obviously restriction to L defines an inclusion

ρ(L): P (L)/U(L) ↪ GL(L).  (4.2)(c)

We will see in Proposition 4.4 that ρ(L) is an isomorphism. We define the determinant character of P (L) by

χ(L): P (L) → F^{×},  χ(L)(p) = det(p|_{L}) = det(ρ(L)(p)).  (4.2)(d)

Recall from (4.1)(f) the isomorphism δ_{L}: L → (W/L)^{∗}. If u ∈ U(L), w ∈ W, and v ∈ L, then ω((u−1)w, v) = ω(w, (u^{−1}−1)v) = ω(w, 0) = 0. Consequently (u−1)w ∈ L^{⊥} = L, so u−1 ∈ Hom_{F} (W/L, L). In light of the isomorphism δ_{L}, this gives

u − 1 ∈ Hom_{F} (W/L, L) ≃ Hom_{F} (W/L, (W/L)^{∗}).  (4.2)(e)

That is, u − 1 gives rise to a bilinear form on the vector space W/L. By inspection of the definition of δ_{L}, the form is

B_{u}(v + L, w + L) = ω(w, (u − 1)v).  (4.2)(f)

Proposition 4.3. Suppose (W, ω) is a finite-dimensional symplectic vector space over F, L is a Lagrangian subspace, and u ∈ U(L) (cf. (4.2)(b)). Then the bilinear form B_{u} on W/L defined by (4.2)(f) is symmetric.

Conversely, suppose B is any symmetric bilinear form on W/L. Identify B with a linear map

T_{B} ∈ Hom_{F} (W/L, (W/L)^{∗}), T_{B}(v + L)(w + L) = B(v + L, w + L);

and further identify T_{B} as an element of

Hom_{F} (W/L, L) ⊂ Hom_{F} (W, W)

using (4.2)(e). Explicitly, T_{B} is characterized as an endomorphism of W by the property

ω(w, T_{B}v) = B(v + L, w + L) (v, w ∈ W).

Then u_{B} = 1 + T_{B} belongs to the subgroup U(L) of Sp(W), and the corresponding bilinear form is B. In this way U(L) is naturally isomorphic with the (additive) group of symmetric bilinear forms on W/L.

Proof. To see that the form B_{u} is symmetric, we use (4.2)(f) and the fact that u preserves ω to compute

\begin{aligned}B_{u}(v + L, w + L) & = ω(w, (u − 1)v) = −ω(w, v) + ω(w, uv) \\ & = −ω(uw, uv) + ω(w, uv) = −ω((u − 1)w, uv) = −ω(uv, (u − 1)w). \end{aligned}

Now (4.2)(e) says that (u − 1)w ∈ L, and that uv − v ∈ L. Since ω is zero on L, we can replace uv by v in the last formula without affecting the value. It is then precisely B_{u}(w + L, v + L), as we wished to show.

For the converse, the last formula certainly defines the endomorphism T_{B} of W; T_{B} is zero on L and carries W into L. The symmetry of B implies that ω(w, T_{B}v) = ω(v, T_{B}w) = −ω(T_{B}w, v). Since T_{B} takes values in L, where ω vanishes, we also have ω(T_{B}w, T_{B}v) = 0. Together these properties imply that ω((1 + T_{B})w, (1 + T_{B})v) = ω(w, v), and therefore that u = 1 + T_{B} ∈ Sp(W). Since T_{B} annihilates L, u acts trivially there; so u ∈ U(L). The remaining assertions are easy. Q.E.D.

Proposition 4.4. Suppose (W, ω) is a finite-dimensional symplectic vector space over F. Then the Lagrangian Grassmannian B(W) is a homogeneous space for Sp(W). The isotropy group at a Lagrangian subspace L is the Siegel parabolic subgroup P (L) (Definition 4.2). The map ρ(L) of (4.2)(c) is surjective. More precisely, we can find a second Lagrangian subspace L′ ⊂ W so that L ∩ L′ = 0. In this case the intersection of the two Siegel parabolics is naturally isomorphic to GL(L), and the isomorphism is implemented by restriction to L.

Proof. It is convenient to begin near the end and work backwards. So fix L; we seek a second Lagrangian subspace L′ with L ∩ L′ = 0. Here is a way to construct one. First choose a basis {p_{1}, …, p_{n}} of L. We want to choose elements {q_{1}, …, q_{n}} of W so that

ω(p_{i}, q_{j}) = δ_{i,j}  (4.5)(a)

and

ω(q_{i}, q_{j}) = 0,  i ≤ j.  (4.5)(b)

We will do this by induction on j; that is, we suppose that q_{1}, …, q_{j−1} have been chosen satisfying (4.5), and we try to choose q_{j}. Now (4.1)(f) guarantees that (4.5)(a) will be satisfied for some element q′_{j} of W; in fact it says that q′_{j} is uniquely determined modulo L. That means that we may modify q′_{j} by an element of L without affecting (4.5)(a). We therefore define

q_{j} = q′_{j} + ∑_{i<j} ω(q_{i}, q′_{j})p_{i}.  (4.5)(c)

Then it is easy to see that (4.5)(b) is satisfied for j.

Define L′ to be the span of {q_{1}, …, q_{n}}. By (4.5)(b), L′ is isotropic; so it is Lagrangian by dimension. It is clear from (4.5)(a) that τ_{L} (Definition 4.1) maps L′ isomorphically onto L^{∗}; that is, that the restriction of ω to L × L′ defines

L′ ≃ L^{∗}  (4.5)(d)

Because the pairing of L with itself is zero, it follows that L ∩ L′ = 0. By dimension counting, we get

W ≃ L ⊕ L′ ≃ L ⊕ L^{∗}.  (4.5)(e)

In this last picture, the symplectic form on W is given by

ω((v, λ), (v′, λ′)) = λ′(v) − λ(v′)  (v, v′ ∈ L, λ, λ′ ∈ L^{∗})  (4.5)(f)

We want to compute P (L) ∩ P (L′). The stabilizer of L and L′ in GL(W) is GL(L) × GL(L′). Using (4.5)(d), this can be written as GL(L) × GL(L^{∗}). The question is which elements of the product preserve the symplectic form ω. Using (4.5)(f), we see immediately that if T ∈ GL(L) and S ∈ GL(L^{∗}), then (T, S) preserves ω if and only if

(Sλ′)(T v) = λ′(v) (λ′ ∈ L^{∗}, v ∈ V).

This in turn means that S must be the inverse of the transpose of T. That is,

P (L) ∩ P (L′) = {(T,^{t}T^{−1}) | T ∈ GL(L)}.  (4.5)(g)

This proves the last claim in the proposition, and also the surjectivity of (4.2)(c).

Finally, we must show that B(W) is homogeneous for Sp(W). Suppose L_{1} and L_{2} are any two Lagrangian subspaces. For each of them choose a Lagrangian complement L′_{i} as in (4.5), so that we have natural isomorphisms

L′_{i} ≃ L_{i}^{∗}

and so on as above. Let T be any linear isomorphism from L_{1} onto L_{2}; such a T exists since L_{1} and L_{2} have the same dimension. Let S be the induced (inverse transpose) isomorphism from L_{1}^{∗} to L_{2}^{∗}; equivalently, from L′_{1} to L′_{2}. Then (T, S) defines a linear isomorphism from W = L_{1} ⊕ L′_{1} onto W = L_{2} ⊕ L′_{2}; that is, an element g ∈ GL(W). Using the descriptions (4.5)(f) for ω, we see that g ∈ Sp(W). By construction g · L_{1} = L_{2}, as we wished to show. Q.E.D.

Corollary 4.6. Suppose (W, ω) is a 2n-dimensional symplectic vector space over F, and L ⊂ W is Lagrangian. Then the Siegel parabolic P (L) acts transitively on the set

B(W)_{0,L} = {L′ ∈ B(W) | L′ ∩ L = 0}

of Lagrangian complements to L. The stabilizer in P (L) of one such Lagrangian L′ is a Levi subgroup GL(L) of P (L); this is a complement for the normal subgroup U(L) of Definition 4.2. Consequently U(L) acts simply transitively on B(W)_{0,L}, which is therefore (algebraically) isomorphic to an F vector space of dimension n(n + 1)/2.

Proof. Suppose L′_{1} and L′_{2} are Lagrangian complements to L. In the argument at the end of the proof of Proposition 4.4, take L_{1} = L_{2} = L; then the element g ∈ Sp(W) constructed there preserves L, and carries L_{1} to L_{2}. It is even clear that we can choose g to be the identity on L; that is, g ∈ U(L). The remaining assertions are now clear from Propositions 4.4 and 4.3. Q.E.D.

Although we will make no use of it, we mention in passing a description of the other orbits of P (L) on B(W).

Proposition 4.7. In the setting of Corollary 4.6, define

B(W)_{r,L} = {L′ ∈ B(W) | dim(L ∩ L′) = r}.

Similarly, for each subspace S of L, define

B(W)_{S,L} = {L′ ∈ B(W) | L ∩ L′ = S}.

a) For S and L as above, the quotient W_{S} = S^{⊥}/S inherits from W a natural non-degenerate symplectic form. The subspace L_{S} = L/S of W_{S} is Lagrangian. If S has dimension r, then W_{S} has dimension 2(n − r). b) Each L′ ∈ B(W)_{S,L} defines a Lagrangian complement L′_{S} = L′/S for L_{S} in W_{S}. This correspondence

provides an algebraic isomorphism

B(W)_{S,L} ≃ B(W_{S})_{0,L_{S}}.

c) The set B(W)_{S,L} is an orbit of U(L). It is algebraically isomorphic to an F vector space of dimension (n − r)(n − r + 1)/2.

d) The set B(W)_{r,L} is an orbit of P (L). It is algebraically isomorphic to a vector bundle of dimension (n − r)(n − r + 1)/2 over the Grassmannian of r-dimensional subspaces of L. The base space is a projective algebraic variety of dimension r(n − r), so B(W)_{r,L} has dimension (n − r)(n + r + 1)/2. It is of codimension (r^{2} + r)/2 in B(W).

Because we will not use the result, we leave the elementary proof to the reader.

The preceding linear algebra can be thought of as infinitesimal symplectic geometry; we will apply it to tangent and cotangent spaces of symplectic manifolds (or algebraic varieties). In order to treat simultaneously the case of Poisson manifolds, we need to weaken the assumptions in Definition 4.1. The most obvious way to do that is to drop the assumption that ω is non-degenerate. Perhaps surprisingly, it is also important to examine the case when ω is non-degenerate, but is defined only on a subspace. The two possibilities are related by duality (of vector spaces). We begin with the second.

Definition 4.8. Suppose F is a field. A degenerate symplectic vector space over F is a triple (V, W, ω) subject to the following conditions.

a) The space V is a finite-dimensional F -vector space, and W is a subspace.

b) The form ω is a non-degenerate skew-symmetric bilinear form on W. The symplectic group of V is the group Sp(V) of F -linear automorphisms of V preserving the subspace

W and the form ω. It maps (by restriction of linear transformations from V to W) surjectively to Sp(W). Suppose S ⊂ V is any subspace. Set

S^{⊥} = {w ∈ W | ω(w, S ∩ W) = 0}.

Notice that S^{⊥} ⊂ W. We say that a subspace I ⊂ V is isotropic if I ∩W ⊂ I^{⊥}; that is, if I ∩W is isotropic in W in the sense of Definition 4.1. We say that C is co-isotropic if C^{⊥} ⊂ C; that is, if C ∩ W is a co-isotropic subspace of W in the sense of Definition 4.1. We say L is Lagrangian if it is both isotropic and co-isotropic; that is, if L ∩ W is a Lagrangian subspace of W in the sense of Definition 4.1.

The definition of isotropic is not entirely an obvious one; it might seem natural to consider instead the condition I ⊂ I^{⊥}, meaning that I is an isotropic subspace of W. We might call this strongly isotropic, since it is more restrictive than the condition in Definition 4.8. The definition of Lagrangian would then change as well; a strongly Lagrangian subspace of V is a Lagrangian subspace of W. But the definition we have given seems well suited to representation theory.

Definition 4.9. Suppose F is a field. A degenerate cosymplectic vector space over F is a pair (U, η) subject to the following conditions.

a) The space U is a finite-dimensional F -vector space.

b) The form η is a (possibly degenerate) skew-symmetric bilinear form on U. The symplectic group of U is the group Sp(U) of F -linear automorphisms of U preserving the form η.

Define R ⊂ U to be the radical of η:

R = {u ∈ U | η(u, U) = 0}.

Then η defines a non-degenerate symplectic form on the quotient space W = U/R. Every element of Sp(U) preserves the subspace R, and so we get a quotient map

Sp(U) → Sp(W).

This map is surjective.

Suppose T ⊂ U is any subspace. Set

T^{⊥} = {u ∈ U | η(u, T) = 0}.

Notice that T^{⊥} ⊃ R. We say that a subspace I ⊂ U is isotropic if I ⊂ I^{⊥}; that is, if I/(I ∩ R) is isotropic in U/R in the sense of Definition 4.1. We say that C is co-isotropic if C^{⊥}/R ⊂ C/(R ∩ C); that is, if C/(R ∩ C) is a co-isotropic subspace of U/R in the sense of Definition 4.1. We say L is Lagrangian if it is both isotropic and co-isotropic; that is, if L/(R ∩ L) is a Lagrangian subspace of U/R in the sense of Definition 4.1.

Again we could have changed the definition of co-isotropic by requiring C^{⊥} ⊂ C; that is, that C ⊃ R, and C/R be co-isotropic in U/R. We could call this requirement strongly co-isotropic, and get a corresponding notion of strongly Lagrangian.

We have said that Definitions 4.8 and 4.9 differ by duality of vector spaces. Here is a more precise formulation. Suppose (V, W, ω) is a degenerate symplectic vector space. Define U = V^{∗} to be the dual vector space. There is an order-reversing bijection from subspaces of V to subspaces of U, sending a subspace S ⊂ V to

S^{⊥} = {λ ∈ V^{∗} | λ(S) = 0}.  (4.10)(a)

Then restriction of linear functionals to S defines an isomorphism

S^{∗} ≃ U/S^{⊥}.  (4.10)(b)

Now let R = W^{⊥} be the subspace of U corresponding to W ⊂ V, so that

W^{∗} ≃ U/R.  (4.10)(c)

The non-degenerate symplectic form ω on W defines via the duality isomorphism τ_{W}: W → W^{∗} (cf. (4.1)(a)) a non-degenerate symplectic form η on U/R. We regard η as a symplectic form on U with radical R. Then (U, η) is a degenerate cosymplectic vector space. It is easy to construct the inverse correspondence, and we leave that to the reader. The duality correspondence S ↦ S^{⊥} carries isotropic to co-isotropic, coisotropic to isotropic, and Lagrangian to Lagrangian. Taking inverse transpose defines a natural isomorphism GL(V) ≃ GL(U); this isomorphism restricts to an isomorphism of Sp(V) onto Sp(U).

We can now explain the sense in which degenerate symplectic and cosymplectic vector spaces are infinitesimal versions of Poisson spaces. To fix ideas we discuss affine Poisson algebraic varieties; as usual it is a simple matter to modify the discussion for other nice Poisson spaces.

Proposition 4.11. Suppose X is an affine Poisson algebraic variety over F, with ring of regular functions R(X). Suppose x ∈ X is a closed point; that is, a maximal ideal 𝔪_{x} ⊂ R(X). Set F_{x} = R(X)/𝔪_{x}, a finite extension field of F.

a) The bilinear form {, }_{x} of (2.6)(d) makes the Zariski cotangent space T_{x}^{∗}(X) into a degenerate cosymplectic vector space over F_{x}.

b) Define S_{x} ⊂ T_{x}(X) to be the space of values at x of Hamiltonian vector fields defined by functions in 𝔪_{x} (cf. (2.6)); and define a non-degenerate symplectic form ω_{x} on S_{x} as in (2.6). Then (T_{x}(X), S_{x}, ω_{x}) is a degenerate symplectic vector space over F_{x}. It is dual to the degenerate cosymplectic vector space T_{x}^{∗}(X) of (a).

This is more or less obvious from the definitions (see also the discussion at (2.6)).

We can now begin to consider non-linear versions of the notion of co-isotropic subspace. An algebraic subvariety Y of X is specified by an ideal

J(Y) ⊂ R(X),  R(Y) = R(X)/J(Y).  (4.12)(a)

(We are ignoring the Poisson structure for the moment.) Now suppose we are in the setting of Proposition 4.11, and that x is also a point of Y; that is,

J(Y) ⊂ 𝔪_{x}.  (4.12)(b)

Then J(Y) defines a natural subspace of T_{x}^{∗}(X) by

T_{Y,x}^{∗}(X) = J(Y)/(J(Y) ∩ 𝔪_{x}^{2}) ↪ T_{x}^{∗}(X).  (4.12)(c)

Its annihilator is a subspace of the tangent space at x:

T_{x}(Y) = {λ ∈ T_{x}(X) | λ(T_{Y,x}^{∗}(X)) = 0}.  (4.12)(d)

As the notation indicates, this subspace may be naturally identified with the tangent space at x to Y. One way to think of the reason is in terms of the short exact sequence

0 → T_{Y,x}^{∗}(X) → T_{x}^{∗}(X) → T_{x}^{∗}(Y) → 0.  (4.12)(e)

Proposition 4.13. In the setting of Proposition 4.11, suppose Y is an algebraic subvariety of X defined by an ideal J(Y) ⊂ R(X). Assume that J(Y) ⊂ 𝔪_{x}, and define subspaces of tangent and cotangent spaces as in (4.12) above. Then the following conditions are equivalent.

a) {J(Y), J(Y)} ⊂ 𝔪_{x}.

b) T_{Y,x}^{∗}(X) is an isotropic subspace of the degenerate cosymplectic vector space T_{x}^{∗}(X) (Definition 4.9).

c) T_{x}(Y) is a co-isotropic subspace of the degenerate symplectic vector space T_{x}(X) (Definition 4.8).

Proof. The equivalence of (a) and (b) is clear from the definition of the degenerate symplectic form {, }_{x} in (2.6)(d). The equivalence of (b) and (c) is a general feature of the duality relationship between Definitions 4.8 and 4.9, as explained in (4.10): a subspace S of a degenerate symplectic vector space V is co-isotropic if and only if its annihilator S^{⊥} ⊂ V^{∗} is isotropic in V^{∗}. Q.E.D.

Definition 4.14. Suppose X is an affine Poisson algebraic variety over F, and Y is a subvariety corresponding to an ideal J(Y) ⊂ R(X). We say that Y (or J(Y)) is co-isotropic if {J(Y), J(Y)} ⊂ J(Y). (Similar definitions apply to Poisson structures on more general ringed spaces.)

The point of Proposition 4.13 is that this terminology is reasonable. Here is a precise statement.

Proposition 4.15. Suppose X is an affine Poisson algebraic variety over F, and Y is a co-isotropic subvariety corresponding to an ideal J(Y). If 𝔪_{x} is any maximal ideal containing J(Y), then T_{x}(Y) is a co-isotropic subspace of the degenerate symplectic vector space T_{x}(X).

Conversely, suppose that Y is any subvariety of X, corresponding to an ideal J(Y); and assume that T_{x}(Y) is co-isotropic in T_{x}(X) whenever J(Y) ⊂ 𝔪_{x}. If J(Y) is the intersection of the maximal ideals containing it (that is, if J(Y) is a radical ideal) then Y is co-isotropic.

Proof. This is immediate from Proposition 4.13 and Definition 4.14. Q.E.D.

Example 4.16. If r, s, and f belong to a Poisson algebra R(X), then

{rf, sf} = (r{f, s} + s{r, f} + f{r, s})f.

From this formula it follows that the ideal ⟨f⟩ generated by f is always co-isotropic. A similar argument shows that if J has a collection of generators that is closed under Poisson bracket, then J is co-isotropic.

Suppose V is a degenerate symplectic vector space over F. We make V into an algebraic variety with algebra of functions R(V) = S(V^{∗}) (the symmetric algebra of V^{∗}). The construction of (4.10) provides a degenerate symplectic form η on V^{∗}. It turns out that there is a unique Poisson algebra structure on S(V^{∗}) characterized by

{λ, μ} = η(λ, μ);

the function on the right is the constant function. If T ⊂ V is any linear subspace, then we can regard T as a subvariety defined by the ideal

J(T) = ⟨λ ∈ V^{∗} | λ(T) = 0⟩.

Then it is easy to check that T is a co-isotropic subvariety (Definition 4.14) if and only if it is a co-isotropic subspace (Definition 4.8).

Finally, we turn to the “dual” notion of isotropic subvariety. Here is the classical definition.

Definition 4.17. In the setting and notation of (4.12), assume that Y is reduced. Recall from (2.6) the symplectic vector space S_{x} ⊂ T_{x}(X) inside each tangent space to X. We say that Y is isotropic if at every smooth point x of Y, T_{x}Y is an isotropic subspace of the degenerate symplectic vector space T_{x}X (Defintion 4.8). That is, we require that the intersection T_{x}(Y) ∩ S_{x} is an isotropic subspace of S_{x}.

We say that Y is Lagrangian if it is both isotropic and co-isotropic; that is, if T_{x}(Y) ∩ S_{x} is Lagrangian in S_{x} at every smooth point x of Y.

Even for a symplectic manifold X, the tangent space condition of Definition 4.17 can fail at singular points of Y (where T_{x}Y has larger dimension). For this reason, we would like to have a definition more along the lines of Definition 4.14. We have not done all the checking needed to verify the equivalence with Definition 4.17, but here is a possibility. In the setting and notation of (4.12), define

J(Y)^{c} = {f ∈ R(X) | {f, J(Y)} ⊂ J(Y)}  (4.18)(a)

If f ∈ J(Y)^{c} and x ∈ Y, then the value ξ_{f}(x) at x of the Hamiltonian vector field ξ_{f} belongs to T_{x}(Y). (Here is the reason. By(4.12), we must show that ξ_{f}(x) annihilates the subspace of T_{x}^{∗}(X) spanned by functions in J. If g is any function in 𝔪_{x}, then the value of ξ_{f}(x) on the corresponding tangent vector is {f, g} + 𝔪_{x} ∈ R(X)/𝔪_{x}. If g ∈ J, then the first bracket is in J ⊂ 𝔪_{x}, so we get zero as required.) Recalling from (2.6) that S_{x} denotes the space of Hamiltonian tangent vectors at x, we have therefore shown that

{ξ_{f}(x) | f ∈ J(Y)^{c}} ⊂ S_{x} ∩ T_{x}(Y).  (4.18)(b)

What we have not verified is a partial converse:

if Y is reduced, then equality holds in (4.18)(b) at every smooth point of Y.  (4.18)(c)

Assuming this to be the case, an argument along the lines of Proposition 4.13 proves that Definition 4.17 is equivalent to

Definition 4.19. Suppose X is an affine Poisson algebraic variety over F, and Y is a subvariety corresponding to an ideal J(Y). Define J(Y)^{c} as in (4.18) above. We say that Y (or J(Y)) is isotropic if {J(Y)^{c}, J(Y)^{c}} ⊂ J(Y).

In any case the requirement in Definition 4.17 certainly implies the one in Definition 4.19. In this paper we will use only Definition 4.17.

5. The metaplectic representation. In this section we recall the construction of the metaplectic or oscillator representation. A convenient reference for most of this material is [14]; original sources include [18] and [23].

Definition 5.1. Suppose F is a field of characteristic not equal to 2, and (W, ω) is a finite-dimensional symplectic vector space over F. The Heisenberg group of W is the set H(W) = W × F with group law

(w, s) · (v, t) = (w + v, s + t + ω(w, v)/2))  (5.1)(a)

The center of H(W) is the subgroup F, and this is also the commutator subgroup; so H(W) is a twostep unipotent algebraic group over F, with H(W)/F ≃ W. The group Sp(W) (Definition 4.1) acts by automorphisms on H(W), by acting trivially on F.

If V is any subspace of W, then H(V) = V × F is a subgroup of H(V), and its centralizer is H(V^{⊥}). (Taking V = 0 or V = W, we get the previous claim that F is the center of H(W).) If I is isotropic, then H(I) is abelian; and if L is Lagrangian, then H(L) is a maximal abelian subgroup of H(W).

Assume now that F is a finite or local field. Then H(W) has a natural locally compact topology. Fix once and for all a non-trivial additive unitary character χ of F. (It is traditional and convenient to assume that χ(t) = exp(2πit) if F = ℝ, but we will carry χ along in the notation as a reminder of the importance of this choice.) If L is any Lagrangian subspace of W, then χ extends to a character of the abelian group H(L) by

χ(L)(w, s) = χ(s)  (w ∈ L, s ∈ F).  (5.1)(b)

The Schrödinger representation of H(W) attached to L is the unitarily induced representation

σ_{χ,L} = Ind_{H(L)}^{H(W)}χ(L).

We may omit the subscript χ when no confusion can result.

We will return in a moment to a careful discussion of the space of σ_{χ,L}. Essentially it is the Hilbert space of square-integrable sections of the line bundle on H(W)/H(L) ≃ W/L induced by the character χ(L). The first thing to observe is that the center F of H(W) acts in a Schrödinger representation by the scalar χ. This property turns out to be characteristic.

Theorem 5.2 (Stone and von Neumann; see [14], Theorem 1.3.3, or [15], Théorème 2.I.2 and Lemme 2.I.8). Suppose W is a symplectic vector space over a finite or local field F of characteristic not 2, and χ is a non-trivial additive unitary character of F.

a) The representations σ_{χ,L} of H(W) (Definition 5.1) are all irreducible and equivalent.

b) Suppose σ_{χ} is any unitary representation of H(W) with the property that σ_{χ}(t) = χ(t)Id for t ∈ F. Then σ_{χ} is unitarily equivalent to a multiple of σ_{χ,L}.

Corollary 5.3 (Segal-Shale-Weil) In the setting of Theorem 5.2, fix a Schrödinger representation (σ_{χ}, H) of H(W) (associated to the character χ of F). Suppose g ∈ Sp(W). Then there is a unitary automorphism T (g) of H satisfying

σ_{χ}(g · h) = T (g)σ_{χ}(h)T (g)^{−1} (h ∈ H(W)).

Here g acts on H(W) as in Definition 5.1. The operator T (g) is determined by this condition up to multiplication by a scalar of absolute value one.

Define GMp(W) to be the group of unitary operators on H generated by the various T (g) and the scalar multiplications. Sending T (g) to g defines an exact sequence of groups

1 → 𝕋 → GMp(W) → Sp(W) → 1.

The group GMp(W) carries a natural locally compact topology making these maps continuous. There is a closed subgroup Mp(W) ⊂ GMp(W) so that Mp(W) ∩ 𝕋 = {±1}, and we have an exact sequence

1 → {±1} → Mp(W) → Sp(W) → 1.

Proof. The map h ↦ σ(g · h) is an irreducible unitary representation of H(W) on H. By Theorem 5.2, it is unitarily equivalent to σ. The existence of the operator T (g) follows, and the exact sequence for GMp(W) follows easily. The reduction of the covering to {±1} was carried out by Shale in the real case (see [14]) and by Weil in general ([23], section 43). Q.E.D.

Definition 5.4. Suppose W is a symplectic vector space over a finite or local field F of characteristic not 2, and χ is a non-trivial character of F. Let (σ, H) be a Schrödinger representation of H(W). The group Mp(W) of Corollary 5.3 is called the metaplectic group of W, and its (tautological) representation τ on H is the metaplectic representation.

We want to realize the metaplectic representation more concretely. Although much of the discussion below applies to any finite or local field, it is convenient from now on to assume that F = ℝ. We begin with a more precise description of the space of a Schrödinger representation; and for that we need half densities.

Definition 5.5. Suppose V is a finite-dimensional real vector space. The space D(V) of densities on V consists of all real multiples of Lebesgue measure on V. This is a one-dimensional real vector space. If dv is a Lebesgue measure, then

D(V) = {c dv | c ∈ ℝ}.  (5.5)(a)

If d′v is another Lebesgue measure, then necessarily d′v = a dv for some positive scalar a; and in the coordinates of (5.5)(a),

c dv = c′ d′v if and only if c′ = c/a.  (5.5)(b)

The space D_{ℂ}(V) of complex densities on V consists of all complex multiples of Lebesgue measure on V; it is the complexification of D(V).

Suppose now that t is any real number. A t-density on V is a formal symbol c(dv)^{t}, with dv a Lebesgue measure on V and c a real number. We define equality of such formal symbols by analogy with (5.5)(b): if d′v = a dv is another Lebesgue measure on V, then

c(dv)^{t} = c′(d′v)^{t} if and only if c′ = c/a^{t}.  (5.5)(c)

The space of t-densities on V is written D^{t}(V); it is a one-dimensional real vector space. There is a welldefined multiplication

D^{t}(V) ⊗ D^{s}(V) → D^{t+s}(V),  (c(dv)^{t}) ⊗ (c′(dv)^{s}) ↦ cc′(dv)^{t+s}.  (5.5)(d)

This defines an isomorphism D^{t}(V)⊗D^{s}(V) ≃ D^{t+s}(V). Similarly we can define a one-dimensional complex vector space D_{ℂ}^{z}(V) of z-densities on V for any complex number z.

Suppose now that M is a real manifold. The real line bundle of t-densities on M is the bundle D^{t}(M) whose fiber at m is D^{t}(T_{m}M), the t-densities on the tangent space at m. The isomorphisms of (5.5)(d) give bundle isomorphisms

D^{t}(M) ⊗ D^{s}(M) ≃ D^{t+s}(M).  (5.5)(e)

Similarly we can define D_{ℂ}^{z}(M), the complex line bundle of z-densities on M.

Proposition 5.6. In the setting of Definition 5.5, the space of smooth sections of D^{1}(M) may be identified with the space of smooth densities on M; that is, with signed measures on M that are given by a smooth function times Lebesgue measure in every coordinate chart. In particular, every compactly supported smooth section of D^{1}(M) has a well-defined integral over M (a real number). Similarly, there is a complexvalued integral for sections of D_{ℂ}^{1}(M).

Using Proposition 5.6, we can define on the space of compactly supported smooth sections of D_{ℂ}^{1/2}(M) a natural pre-Hilbert space structure, as follows. Suppose σ_{1} and σ_{2} are such sections. Because D_{ℂ}^{1/2}(M) is the complexification of D^{1/2}(M), the complex conjugate \overline{σ_{2}} is a well-defined section of D_{ℂ}^{1/2}(M). Using the multiplication (5.5)(e), we can regard σ_{1}\overline{σ_{2}} as a compactly supported smooth section of D_{ℂ}^{1}(M). Such a section has an integral, by Proposition 5.6; so we define

⟨σ_{1}, σ_{2}⟩ = ∫_{M} σ_{1}(m)\overline{σ_{2}(m)}  (5.7)(a)

It is easy to check that this is a pre-Hilbert space structure. The corresponding Hilbert space is called L^{2}(M, D_{ℂ}^{1/2}), the space of square-integrable half-densities on M.

The great advantage of this Hilbert space over the (isomorphic) one L^{2}(M, dμ) of square-integrable functions on M with respect to a chosen measure dμ is this. Any diffeomorphism T of M acts on compactly supported smooth sections of D_{ℂ}^{1/2}(M), preserving the pre-Hilbert structure. Consequently T defines a unitary operator ρ(T) on L^{2}(M, D_{ℂ}^{1/2}), and ρ is a unitary representation of Diff(M).

A little more generally, suppose H → M is a Hilbert space bundle over M. Then we can form the tensor product H ⊗ D_{ℂ}^{1/2}(M); topologically its fibers are Hilbert spaces, but there is no longer a distinguished inner product. More precisely, the fiber over m comes equipped with a positive sesquilinear pairing not into ℂ, but into the space D_{ℂ,m}^{1}(M) of densities at m. In this way the space of compactly supported continuous sections of H ⊗ D_{ℂ}^{1/2}(M) acquires a pre-Hilbert space structure. Explicitly, suppose h_{1} and h_{2} are compactly supported continuous sections of H, and σ_{1} and σ_{2} are compactly supported continuous sections of D_{ℂ}^{1/2}(M). Then we define

⟨h_{1} ⊗ σ_{1}, h_{2} ⊗ σ_{2}⟩ = ∫_{M}⟨h_{1}(m), h_{2}(m)⟩_{H_{m}}σ_{1}(m)\overline{σ_{2}(m)}  (5.7)(b)

The completion of this Hilbert space is called L^{2}(M, H ⊗ D_{ℂ}^{1/2}), the space of square-integrable half-density sections of H. (Of course the space M is implicit in the bundle H, and we may sometimes omit it from the notation.) Sometimes it is helpful to think of this space in terms of real half-densities, using the isomorphism

H ⊗_{ℂ} D_{ℂ}^{1/2}(M) ≃ H ⊗_{ℝ} D^{1/2}(M).  (5.7)(c)

Example 5.8. Suppose G is a Lie group, H is a closed subgroup, and (τ, H_{τ}) is a unitary representation of H. Then τ defines a G-equivariant Hilbert bundle H = G×_{H} H_{τ} over G/H. The total space of the bundle is G × H_{τ} modulo the equivalence relation

(gh, v) ∼ (g, τ(h)v) (g ∈ G, h ∈ H, v ∈ H_{τ}).

The action of G on the total space is by the left action in the first variable; this respects the equivalence relation. At the same time the action of G on G/H defines an action on half-densities on G/H; so G acts on the compactly supported continuous half-density sections of H. This action respects the pre-Hilbert space structure, and so gives a unitary action of G on L^{2}(G/H, H ⊗ D_{ℂ}^{1/2}). This is nothing but the induced representation:

Ind_{H}^{G}(H_{τ}) = L^{2}((G ×_{H} H_{τ}) ⊗ D_{ℂ}^{1/2}(G/H)).

The twist by half-densities corresponds to the “ρ shift” appearing in more purely group-theoretic descriptions of induction. There the space of the representation is described as a space of functions f on G with values in H_{τ}, satisfying a transformation law

f(gh) = ρ_{G/H}(h^{−1})τ(h^{−1})f(g) (g ∈ G, h ∈ H).

Here ρ_{G/H} is a certain real-valued character of H. The point is that the half-density bundle on G/H is also an induced bundle, and so is characterized by the action of H on the fiber at the identity coset eH; that is, by the action of H on half-densities on T_{eH}(G/H) = 𝔤/𝔥. This action is by one over the square root of the absolute value of the determinant of the adjoint action of H on 𝔤/𝔥; and that is precisely the character ρ_{G/H} of H.

This entire discussion depends only on the notion of Lebesgue measure on finite-dimensional vector spaces, and so works equally well over any local or finite field. (All t-density bundles are trivial in the case of a finite field.)

With this machinery in hand, we can return to the problem of describing the space of the Schrödinger representation. According to Example 5.8, we can take the Hilbert space of σ_{χ,L} to be the space of squareintegrable half-density sections of the line bundle on H(W)/H(L) induced by the character χ(L). In order to understand this space, we will trivialize the line bundle. For that, we need to pick a second Lagrangian subspace L′ of W so that L ∩ L′ = 0; this is possible by Proposition 4.4. In that case L′ is an abelian subgroup of H(W), and

L′ ≃ H(W)/H(L);  (5.9)(a)

this follows from (4.5)(e). Because the isotropy group of the L′ action here is trivial, the line bundle is trivial (as an L′-equivariant bundle). The Hilbert space is therefore

H(L) = L^{2}(L′, D_{ℂ}^{1/2}),  (5.9)(b)

the space of square-integrable half-densities on the vector space L′. If dx′ is a Lebesgue measure on L′, then a typical element of this space is a symbol ϕ(x′)(dx′)^{1/2}; here ϕ is an L^{2} function on L′. This isomorphism respects the action of L′; that is, L′ acts by the left regular representation (translation of half densities). Explicitly,

σ_{χ,L}(l′)[ϕ(x′)(dx′)^{1/2}] = ϕ(x′ − l′)(dx′)^{1/2}.  (5.9)(c)

It is also fairly easy to calculate the action of the subgroup L in this picture. Recall from (4.5)(d) that each element l ∈ L gives a well-defined linear functional x′ ↦ ω(l, x′) on L′, and therefore a well-defined unitary character x′ ↦ χ(ω(l, x′)) of L′. The action of σ_{χ,L}(l) is multiplication by this unitary character:

σ_{χ,L}(l)[ϕ(x′)(dx′)^{1/2}] = χ(ω(l, x′))ϕ(x′)(dx′)^{1/2}.  (5.9)(d)

To see this, recall that the isomorphism (5.9)(b) identifies the half density ϕ(x′)(dx′)^{1/2} with a function Φ on H(W) transforming according to the character χ(L) of H(L). We now compute

σ_{χ,L}(l)[ϕ(x′)(dx′)^{1/2}] = Φ((l, 0)^{−1}(x′, 0));

here the multiplication on the right takes place in the group H(W). Now

(l, 0)^{−1}(x′, 0) = (−l, 0)(x′, 0) = (x′ − l, ω(−l, x′)/2) = (x′, 0)(−l, −ω(l, x′)).

By the transformation law for Φ under H(L), we get

Φ((l, 0)^{−1}(x′, 0)) = Φ(x′, 0)χ(L)(−l, −ω(l, x′))^{−1} = ϕ(x′)(dx′)^{1/2}χ(ω(l, x′)).

This is (5.9)(d).

So far this discussion applies equally well over any local or finite field. Now we turn to something special to ℝ: the differentiated representation. We begin with the Lie algebra 𝔥(W) = W × ℝ of H(W). (The identification of H(W) with the vector space W ×ℝ naturally identifies the Lie algebra with this vector space as well.) The Lie bracket is

[(w, s), (v, t)] = (0, ω(w, v)).  (5.10)(a)

Obviously the center of 𝔥(W) is equal to the commutator subalgebra, which is ℝ. In the setting of (5.9), we get

𝔥(W) = L ⊕ L′ ⊕ ℝ.  (5.10)(b)

In order to differentiate the Schrödinger representation, we must first understand the differential of the character χ of ℝ. This is a map from the Lie algebra ℝ of ℝ to the Lie algebra iℝ of the unit circle; so dχ(1) is a purely imaginary number. With the standard choice χ(t) = exp(2πit) made in Definition 5.1, we get dχ(1) = 2πi.

Proposition 5.11. Suppose W is a real symplectic vector space, and L and L′ are Lagrangian subspaces with L ∩ L′ = 0; realize the Schrodinger representation σ_{χ,L} (Definition 5.1) as in (5.9). Then the differentiated representation of 𝔥(W) (cf. (5.10)) may be calculated as follows.

a) Suppose t ∈ ℝ. Then dσ_{χ,L}(0, t) = tdχ(1) = 2πit.

b) Suppose l ∈ L. Identify l with the linear functional τ ′L(l) on L′ (Definition 4.1), defined by τ_{L′}(l)(l′) =

ω(l′, l). Then

dσ_{χ,L}(l, 0) = multiplication by − dχ(1)τ_{L′}(l) = −2πiτ_{L′}(l).

c) Suppose l′ ∈ L′. Then dσ_{χ,L}(l′, 0) is the directional derivative in the direction −l′.

Proof. Part (a) follows from the fact that the center ℝ of H(W) acts in σ_{χ,L} by the character χ. Parts (b) and (c) are differentiated versions of (5.9)(d) and (c) respectively. For (b), for example, we calculate

dσ_{χ,L}(l, 0)[ϕ(x′)(dx′)^{1/2}] = d/dt(σ_{χ,L}(tl, 0)[ϕ(x′)(dx′)^{1/2}])|_{t=0}
= d/dt(χ(ω(tl, x′))ϕ(x′)(dx′)^{1/2})|_{t=0}
= d/dt(χ(−tτ_{L′}(l)(x′)))|_{t=0}ϕ(x′)(dx′)^{1/2}
= −dχ(1)τ_{L′}(l)(x′)ϕ(x′)(dx′)^{1/2}

This is (b). Q.E.D.

Corollary 5.12. In the setting of Definition 5.1 and (5.9), the space H(L)^{∞} of smooth vectors in the Schrödinger representation is the Schwartz space S(L′, D_{ℂ}^{1/2}): the space of all half-densities ϕ(x′)(dx′)^{1/2} such that any derivative of ϕ times any polynomial in x′ is bounded.

Proof. If we replace the phrase “is bounded” by “belongs to L^{2},” then the statement is immediate from Proposition 5.11 and the definition of the space of smooth vectors. So what we must show is that if ϕ is a function on ℝ^{n} with the property that x^{α} \frac{∂^{β}ϕ}{∂x^{β}} belongs to L^{2} for all multiindices α and β, then ϕ belongs to the Schwartz space. We write L^{2}S(ℝ^{n}) temporarily to distinguish the function space defined by these L^{2} conditions. Since polynomials times derivatives of ϕ again belong to L^{2}S, it suffices to show that ϕ is bounded. This is a consequence of the Sobolev lemma. We sketch a direct argument. For compactly supported smooth ϕ, we have

ϕ(a) = ∫_{x_{i}≤a_{i}} \frac{∂^{n}ϕ}{∂x_{1} ⋯ ∂x_{n}} dx_{1} ⋯ dx_{n}.

We need to estimate the right side using L^{2} norms. For that, consider the polynomial p(x) = (1 + |x|^{2})^{n}. Then 1/p ∈ L^{2}(ℝ^{n}), so

ϕ(a) = ∫_{x_{i}≤a_{i}}(1/p(x))(p(x)\frac{∂^{n}ϕ}{∂x_{1} ⋯ ∂x_{n}})dx_{1} ⋯ dx_{n}.

This integral may be interpreted as the L^{2} inner product of p\frac{∂^{n}ϕ}{∂x_{1}···∂x_{n}} with a cutoff of 1/p. By the Cauchy-Schwarz inequality, we get

|ϕ(a)| ≤ ∥1/p∥_{2}∥p\frac{∂^{n}ϕ}{∂x_{1} ⋯ ∂x_{n}} ∥_{2}.

This gives the estimate we need:

∥ϕ∥_{∞} ≤ ∥1/p∥_{2}∥p\frac{∂^{n}ϕ}{∂x_{1} ⋯ ∂x_{n}} ∥_{2}.

Once it is established for compactly supported smooth functions, the estimate follows for general ϕ ∈ L^{2}S by continuity (more precisely, by the density of C_{c}^{∞} in L^{2}S). Q.E.D.

Proposition 5.11 and Corollary 5.12 show clearly the relevance to the Schrödinger representation of the algebra of polynomial coefficient differential operators on L′. Here is an abstract definition of it.

Definition 5.13. Suppose W is a real symplectic vector space and χ is a non-trivial additive character of ℝ; as usual we will generally assume χ(t) = exp(2πit). The Weyl algebra of W is the complex associative algebra A_{χ}(W) with unit generated by W, subject to the relations

vw − wv = dχ(1)ω(v, w) = 2πiω(v, w) (v, w ∈ W).

That is, A_{χ}(W) is the complex tensor algebra T (W_{ℂ}) divided by the ideal generated by elements v ⊗ w − w ⊗ v − 2πiω(v, w). Since we have specified a choice of χ, we may sometimes omit it from the notation. As a quotient of a graded algebra, A_{χ}(W) inherits an increasing filtration; A_{χ}^{p}(W) is spanned by the images of elements w_{1} ⊗ ⋯ ⊗ w_{q} with q ≤ p. Thus

A_{χ}^{0}(W) = ℂ,  A_{χ}^{1}(W) = ℂ + W_{ℂ},  A^{p}A^{q} ⊂ A^{p+q}.

The generators by which we divide are all sums of tensors of even degree; so A_{χ}(W) inherits from T (W_{ℂ}) a ℤ/2ℤ grading

A_{χ}(W) = A_{χ}^{even}(W) ⊕ A_{χ}^{odd}(W).

Finally, there is a complex conjugate-linear antiautomorphism r ↦ r^{∗} of A_{χ}(W), characterized by the property v^{∗} = −v (v ∈ W). (The existence of this map is a formal consequence of the definition of A_{χ}(W). One defines A_{χ}^{∗}(W) to be the opposite algebra of A_{χ}(W), with complex multiplication by z given by the old multiplication by \overline{z}. Then the elements \{−v | v ∈ W \} of A_{χ}^{∗}(W) satisfy the same relations as the generators W of A_{χ}(W).)

Proposition 5.14. In the setting of Proposition 5.11, the following three algebras are naturally isomorphic.

a) The Weyl algebra A_{χ}(W) of Definition 5.13.

b) The quotient U(𝔥(W))/I_{χ} of the universal enveloping algebra of the Heisenberg Lie algebra (cf. (5.10)) by the ideal I_{χ} generated by the element (0, 1) − dχ(1) = (0, 1) − 2πi. Here (0,1) is the central element of 𝔥(W).

c) The algebra D(L′) of polynomial coefficient differential operators on the Lagrangian subspace L′ of W.

The isomorphism of A_{χ}(W) with D(L′) sends an element l ∈ L to multiplication by the purely imaginary linear functional −dχ(1)τ′_{L}(l) = −2πiτ′_{L}(l) ; and it sends an element l′ ∈ L′ to the directional derivative in the direction −l′. The involution r ↦ r^{∗} is given in D(L′) by the formal adjoint of differential operators.

Proof. By definition the universal enveloping algebra U(𝔥(W)) is the complex associative algebra generated by W + ℝ, subject to the certain relations. To describe them it is convenient to write z for the central element (0, 1) ∈ 𝔥(W). Then the relations are vw −wv = ω(v, w)z and zv = vz for v, w ∈ W. Dividing by I_{χ} amounts to identifying z with dχ(1) = 2πi. It follows that U(𝔥)/I_{χ} may be described as the complex associative algebra generated by W, subject to the relations vw − wv = ω(v, w) · dχ(1), or vw − wv = 2πiω(v, w). This is just A_{χ}(W) by definition. The filtration defined on A_{χ}(W) in Definition 5.13 is clearly the one inherited from the standard filtration on U(𝔥(W)). It follows easily from the Poincaré-Birkhoff-Witt theorem that the associated graded algebra is commutative; specifically, that

gr A_{χ}(W) ≃ S(W_{ℂ}),  (5.15)

the complexified symmetric algebra of W.

To get the isomorphism with D(L′), we consider the action of U(𝔥(W)) on H(L)^{∞} by dσ_{χ,L}. Clearly the element z acts by dχ(1), so the ideal I_{χ} acts by 0. Proposition 5.11 shows that the image of the enveloping algebra is precisely D(L′); so we have a surjective homomorphism U(𝔥(W))/I_{χ} → D(L′). The Poincaré-Birkhoff-Witt theorem has already shown us how to find a basis in the domain. More explicitly, we can take a basis {x_{i}} of L, followed by a dual basis {y_{j}} of L′. (This means that ω(y_{j}, x_{i}) = δ_{ij}.) Then the elements x^{α}y^{β} (for multiindices α and β) form a basis of U(𝔥(W))/I_{χ}. Proposition 5.11 shows that their images in D(L′) are (up to scale factors) the standard basis elements x^{α} \frac{∂^{β}}{∂x^{β}} of D(L′). Our homomorphism is therefore an isomorphism. Q.E.D.

We return now to the metaplectic representation. Corollary 5.3 provides a global description of it; we want an infinitesimal description, giving the action of the Lie algebra of the symplectic group. This Lie algebra is

𝔰𝔭(W) = {M ∈ End W | ω(Mv, w) + ω(v, Mw) = 0}.  (5.16)

We will realize this Lie algebra using the Weyl algebra A_{χ}(W). To do that, we want to take advantage of the non-commutative nature of the Weyl algebra. Suppose r ∈ A_{χ}^{p}(W) and s ∈ A_{χ}^{q}(W). Then the products rs and sr both belong to A_{χ}^{p+q}(W). In fact their images in A_{χ}^{p+q}(W)/A_{χ}^{p+q−1}(W) are the same; this is the content of the assertion in (5.15) that the associated graded algebra is commutative. That is, rs − sr ∈ A_{χ}^{p+q−1}(W). But actually even more is true.

Proposition 5.17. Suppose W is a real symplectic vector space, and χ(t) = exp(2πit) is our standard non-trivial character of ℝ. Define the Weyl algebra A_{χ}(W) as in Definition 5.13, and use the isomorphism (5.15) of the associated graded algebra with the complex symmetric algebra on W.

a) Suppose r ∈ A_{χ}^{p}(W) and s ∈ A_{χ}^{q}(W). Then rs − sr ∈ A_{χ}^{p+q−2}(W).

b) There is a Poisson bracket {, }_{χ} on S(W_{ℂ}), homogeneous of degree -2, defined as follows. Suppose R ∈ S^{p}(W) and S ∈ S^{q}(W) are homogeneous polynomials of degrees p and q. Choose representatives r ∈ A_{χ}^{p}(W) and s ∈ A_{χ}^{q}(W) for R and S, under the isomorphisms S^{m}(W) ≃ A_{χ}^{m}(W)/A_{χ}^{m−1}(W). Then rs − sr ∈ A_{χ}^{p+q−2}(W); and we define {R, S}_{χ} to be the polynomial represented by rs − sr. This bracket makes S(W_{ℂ}) into a complex Poisson algebra (Definition 2.4).

c) The Poisson bracket of (b) is characterized by the property {v, w}_{χ} = dχ(1)ω(v, w) = 2πiω(v, w).

Proof. We use the ℤ/2ℤ grading of A_{χ}(W) from Definition 5.13. It is clear that the image of this grading in S(W_{ℂ}) is the usual grading into even and odd polynomials. To prove the claim in (a), it is enough to replace r and s by any other representatives of their classes in gr A_{χ}(W). Say for definiteness that p is even and q is odd; then we may choose these new representatives so that r ∈ A_{χ}^{even}(W) and s ∈ A_{χ}^{odd}(W). Then rs and sr both belong to A_{χ}^{odd}(W), so rs − sr does as well. But p + q − 1 is even, so the class of rs − sr in gr A_{χ}(W) is an odd polynomial of even degree; so it is zero.

For (b), that the bracket is well-defined is immediate from (a); and the axioms of Definition 2.4 follow easily. (For example, the Jacobi identity is a consequence of the identity [r, [s, t]] = [[r, s], t] + [s, [r, t]] for commutators in an associative algebra.) Finally (c) is immediate from the defining relations of A_{χ}(W) in Definition 5.13. Q.E.D.

Of course essentially this same Poisson structure has appeared earlier. The symplectic structure on W defines a symplectic structure on the dual vector space W^{∗}, by means of the isomorphism τ of W with W^{∗} (Definition 4.1). We may regard S(W_{ℂ}) as the space of complex-valued polynomial functions on the symplectic manifold W^{∗}. The Poisson structure of Proposition 2.3 on C^{∞}(W^{∗}) preserves S(W_{ℂ}), and (after multiplication by dχ(1) = 2πi) it is precisely the structure of Proposition 5.17. For a less obvious identification, one can look inside the dual 𝔥(W)^{∗} of the Heisenberg Lie algebra at the hyperplane

M_{χ} = {λ ∈ 𝔥(W)^{∗} | λ(0, 1) = dχ(1)/2πi = 1}.

Because (0, 1) is central in 𝔥(W), M_{χ} is preserved by the coadjoint action of H(W). (Actually it is a single coadjoint orbit.) By Proposition 2.11, it follows that M_{χ} is a Poisson manifold. Restriction of linear functionals to W identifies M_{χ} with W^{∗}, and then τ^{−1} provides an identification with W. Again the Poisson structure of Proposition 2.11 differs from that of Proposition 5.17 by the factor 2πi. We will make no use of these identifications, so we leave the verifications to the reader.

Corollary 5.18. In the setting of Definition 5.13, the subspace A_{χ}^{2}(W) of the Weyl algebra is closed under commutator. It is therefore a finite-dimensional complex Lie algebra. There is a natural real form

A_{χ}^{2}(W)_{ℝ} = {r ∈ A_{χ}^{2}(W) | r^{∗} = −r}

(cf. Definition 5.13).

a) The subspace A_{χ}^{1}(W)_{ℝ} is an ideal naturally isomorphic to the Heisenberg Lie algebra 𝔥(W) (under the map U(𝔥(W)) → A_{χ}(W) of Proposition 5.14).

b) The subspace A_{χ}^{1,odd}(W)_{ℝ} is naturally isomorphic to W. Its Lie bracket into A_{χ}^{0}(W)_{ℝ} ≃ iℝ is given by dχ(1)ω = 2πiω.

c) The adjoint action of A_{χ}^{2,even}(W)_{ℝ} on A_{χ}^{1,odd}(W)_{ℝ} defines a Lie algebra homomorphism

τ: A_{χ}^{2,even}(W)_{ℝ} → 𝔰𝔭(W).

This map is surjective, and its kernel is A_{χ}^{0}(W)_{ℝ} ≃ iℝ.

d) The short exact sequence of (c) splits uniquely. That is, there is a unique Lie subalgebra 𝔪𝔭(W) ⊂ A_{χ}^{2,even}(W)_{ℝ} with the property that

A_{χ}^{2,even}(W)_{ℝ} ≃ iℝ ⊕ 𝔪𝔭(W).

The subalgebra 𝔪𝔭(W) is mapped isomorphically onto 𝔰𝔭(W) by the adjoint action in (c).

Proof. The first claim is immediate from Proposition 5.17(a). That A_{χ}^{2}(W)_{ℝ} is a real Lie algebra follows from the formula [r, s]^{∗} = [s^{∗}, r^{∗}], which in turn follows from the fact that r ↦ r^{∗} is an antiautomorphism. That it is a real form follows from the fact that r ↦ r^{∗} is conjugate linear. The isomorphism in (b) comes from (5.15), and then the description of the bracket from Definition 5.13.

For (c), we must check that if r ∈ A_{χ}^{2,even}(W)_{ℝ} and v, w ∈ W, then τ(r) satisfies the condition in (5.16) to belong to the symplectic Lie algebra. Multiplying by dχ(1) = 2πi and using (b), this condition becomes [[r, v], w] + [v, [r, w]] = 0. By the Jacobi identity, the left side is [r, [v, w]] = [r, dχ(1)ω(v, w)]. The second term on the right is a scalar, so the bracket is indeed zero. Because W generates A_{χ}(W), the kernel of τ is the intersection of its domain with the center of A_{χ}(W), which is A_{χ}^{0}(W) = ℂ. So the kernel of τ is indeed A_{χ}^{0}(W)_{ℝ}. Now (5.15) shows that the image of τ is isomorphic to maps A_{χ}^{2,even}(W)_{ℝ}/A_{χ}^{0}(W)_{ℝ} ≃ iS^{2}(W). To show that τ is an isomorphism, it remains only to show that 𝔰𝔭(W) has the same dimension as S^{2}(W). Say W has dimension 2n; then S^{2}(W) has dimension n(2n + 1). The dimension of Sp(W), on the other hand, is equal to the dimension of the Lagrangian Grassmannian B(W) ≃ Sp(W)/P (L) plus the dimension of a Siegel parabolic P (L). The first number is n(n + 1)/2 by Corollary 4.6, and the second is n^{2} plus the dimension of U(L) by Proposition 4.4. Finally, the dimension of U(L) is n(n + 1)/2 by Proposition 4.3; so the total dimension of Sp(W) is n(n + 1)/2 + n^{2} + n(n + 1)/2 = n(2n + 1), as we wished to show.

For (d), one knows that 𝔰𝔭(W) is a semisimple Lie algebra. The short exact sequence of (c) is a central extension of that Lie algebra. But every central extension of a semisimple Lie algebra is trivial. We can take for 𝔪𝔭(W) the commutator subalgebra of A_{χ}^{2,even}(W)_{ℝ}. Q.E.D.

Theorem 5.19. Suppose W is a real symplectic vector space, and L and L′ are Lagrangian subspaces with L∩L′ = 0. Realize the Schrödinger representation σ_{χ,L} on H(L) as in (5.9), and define groups GMp(W) and Mp(W) of unitary operators as in Corollary 5.3.

a) The space of smooth vectors for the representation of GMp(W) coincides with the smooth vectors H(L)^{∞} of the Schrödinger representation.

b) Use Proposition 5.14 to identify the Weyl algebra A_{χ}(W) with an algebra of operators on H(L)^{∞}. Then the Lie algebra of GMp(W) is precisely

𝔤𝔪𝔭(W) = A_{χ}^{2,even}(W)_{ℝ}.

In terms of differential operators, these are the even skew-adjoint polynomial coefficient differential operators of total degree (both polynomial and differentiation) at most 2. The short exact sequence of groups

1 → 𝕋 → GMp(W) → Sp(W) → 1

in Corollary 5.3 gives rise to the short exact sequence of Lie algebras

1 → iℝ → 𝔤𝔪𝔭(W) → 𝔰𝔭(W) → 1

of Corollary 5.18(c).

c) The Lie algebra of the subgroup Mp(W) is precisely the subalgebra 𝔪𝔭(W) of Corollary 5.18(d). The double covering Mp(W) → Sp(W) of Corollary 5.3 gives rise to the Lie algebra isomorphism 𝔪𝔭(W) ≃ 𝔰𝔭(W) of Corollary 5.18(d).

Sketch of proof. Suppose X ∈ 𝔤𝔪𝔭(W). This means that first of all that X is a densely defined selfadjoint operator on H(L), and that exp(tX) is a unitary operator in GMp(W) for every real t. This means in turn that for every t there is an element A(t) ∈ Sp(W) so that

σ_{χ,L}(A(t) · h) = exp(tX)σ_{χ,L}(h) exp(−tX)  (h ∈ H(W)).  (5.20)(a)

It is not difficult to see that this condition determines A(t) uniquely. Because the map GMp(W) → Sp(W) is continuous, it follows that A(t) = exp(tY) for some Y ∈ 𝔰𝔭(W).

Now Sp(W) acts by algebra automorphisms on the Weyl algebra A_{χ}(W) of Definition 5.13, by its action on the generating subspace W. Recall that Proposition 5.14 allows us to identify A_{χ}(W) with certain densely defined operators on H(L). If we differentiate (5.20)(a) with respect to h and use Proposition 5.11, we find that the operators exp(tX) preserve H(L)^{∞}, and define (by conjugation) algebra automorphisms of D(L′). More explicitly, for any v ∈ A_{χ}^{1,odd}(W), we have

exp(tY) · v = exp(tX)v exp(−tX).  (5.20)(b)

If we now (formally) differentiate both sides with respect to t, we get

Y · v = [X, v].  (5.20)(c)

This equation is the infinitesimal version of the definition of the Weil representation in Corollary 5.3. It begins with the family of operators v ∈ A_{χ}^{1,odd}(W), and the linear transformation Y of that family; and it seeks a new operator X satisfying (5.20)(c). Corollary 5.18(c) provides solutions to (5.20)(c). The problem is essentially to show that they really arise by differentiating solutions to (5.20)(a). Perhaps the easiest way to do that is simply to exhibit some solutions to (5.20)(a).

For that, let P (L) ⊂ Sp(W) be the Siegel parabolic subgroup preserving the Lagrangian subspace L (Definition 4.2). The action of P (L) on the Heisenberg group H(W) preserves H(L), and so descends to an action of P (L) by diffeomorphisms on the homogeneous space H(W)/H(L) ≃ W/L. The action of P (L) on H(L) fixes the character χ(L), so P (L) acts by automorphisms on the Hermitian line bundle L_{χ} → H(W)/H(L) induced by χ(L). Recall from Example 5.8 that the Hilbert space H(L) of the Schrödinger representation may be identified with the space L^{2}(H(W)/H(L), L_{χ} ⊗D_{ℂ}^{1/2}) of square-integrable half density sections of L. From what we have just said, there is a natural unitary representation τ(L) of P (L) on H(L) compatible with its action by automorphisms of H(W). Explicitly,

τ_{L}(p)σ_{χ,L}(h)τ_{L}(p)^{−1} = σ_{χ,L}(p · h)  (p ∈ P (L), h ∈ H(W)).  (5.21)(a)

This is the defining relation for the metaplectic representation (Corollary 5.3). Consequently τ_{L} may be regarded as a homomorphism from P (L) into GMp(W), a section (over P (L)) of the natural projection from GMp(W) onto Sp(W) (Corollary 5.3). That is, we have a commutative diagram

\begin{aligned}& & GMp(W) \\ & τ^{L}↗ & ↓ \\ P (L) & ⟶ & Sp(W) \end{aligned}  (5.21)(b)

It is now clear that any one-parameter subgroup A(t) of P (L) gives a solution of (5.20)(a): we take for X a generator of the one-parameter group τ_{L}(A(t)) of unitary operators. So we would like to compute τ_{L} explicitly. As in (5.9) we begin by identifying the Hilbert space H(L) with square-integrable half-densities on L′. As in Proposition 4.4, identify GL(L) with the subgroup of P (L) preserving L′; recall that the symplectic form identifies L′ with the dual of L, so GL(L) ≃ GL(L′) (the isomorphism sending g to ^{t}g^{−1}). Now GL(L′) is a group of diffeomorphisms of L′, so there is a natural action of GL(L′) on square integrable half-densities. This is precisely τ_{L}; so we get

τ_{L}(g)[ϕ(x′)(dx′)^{1/2}] = | det g|^{1/2}ϕ(^{t}gx′)(dx′)^{1/2}  (g ∈ GL(L) ⊂ P (L)).  (5.21)(c)

(The determinant factor arises from the action of g on the half-density (dx′)^{1/2}.) It is a simple matter to differentiate this representation. If we choose coordinates (x′_{1}, …, x′_{n}) on L′, then we get at the same time an identification of GL(L) with GL(n, ℝ). The Lie algebra may be identified with n × n real matrices, and the standard basis matrices act by

dτ_{L}(e_{pq}) = x_{p} \frac{∂}{∂x_{q}} + \frac{1}{2}δ_{pq}.  (5.21)(d)

Next we compute the action of the unipotent radical U(L). Recall from Proposition 4.3 that each element u ∈ U(L) corresponds to a symmetric bilinear form B on W/L ≃ L′. This correspondence uses the linear map u − 1 = T, which carries L′ to L. To compute the action of u on ϕ(x′)(dx′)^{1/2}, recall from (5.9) that ϕ corresponds to a function Φ on H(W) transforming according to χ(L) under H(L). We have

τ_{L}(u)[ϕ(x′)(dx′)^{1/2}] = Φ(u^{−1} · (x′, 0)) = Φ((x′ − T x′, 0)).

Now we use the multiplication law in H(W) given in (5.1)(a) to write

(x′ − T x′, 0) = (x′, 0) · (−T x′, ω(x′, T x′)/2).

In light of the identifications in Proposition 4.3, the second factor is (−T x′, B(x′, x′)/2). This term belongs to L, and the character χ_{L} takes the value χ(B(x′, x′)/2) on it. Because of the transformation property of Φ under H(L), we get

Φ(u^{−1} · (x′, 0)) = Φ((x′, 0)χ(−B(x′, x′)/2).

That is,

τ_{L}(u)[ϕ(x′)(dx′)^{1/2}] = ϕ(x′)(dx′)^{1/2}χ(−B(x′, x′)/2)  (u ∈ U(L) ⊂ P (L)).  (5.21)(e)

The Lie algebra of U(L) may also be identified with symmetric bilinear forms on L′, and we compute

dτ_{L}(B) is multiplication by − dχ(1)B/2.  (5.21)(f)

That is, dτ_{L} carries the Lie algebra of U(L) onto multiplication operators by purely imaginary quadratic polynomial functions on L′. In terms of the coordinates chosen in (5.21)(d), this is the span of the multiplication operators ix′_{p}x′_{q}.

These calculations establish the isomorphism in (b) of the proposition for that part of 𝔤𝔪𝔭(W) lying over the parabolic subalgebra 𝔭(L) of 𝔰𝔭(W). We just sketch the rest of the proof. Because 𝔭(L) and 𝔭(L′) together span 𝔰𝔭(W), part (b) follows. At the same time we see that the operators in the enveloping algebra of 𝔤𝔪𝔭(W) are contained in the enveloping algebra of the Heisenberg Lie algebra; so the smooth vectors for the Schrödinger representation are contained in the smooth vectors for the metaplectic representation. On the other hand, if ϕ ∈ L^{2}(L′) is a smooth vector for the metaplectic representation, then it follows from (b) that Dϕ must belong to L^{2} for every even polynomial coefficient differential operator D. This forces ϕ to belong to the Schwartz space, proving (a). The rest of the proposition is formal. Q.E.D.

Proposition 5.22. In the setting of Theorem 5.19, write

H(L) = H^{even}(L) ⊕ H^{odd}(L)

for the decomposition into even and odd half-densities on L′. Then these spaces are invariant under the metaplectic representation τ of Definition 5.4; they are inequivalent irreducible representations τ^{even} and τ^{odd} of Mp(W), independent of the choice of Lagrangian subspaces L and L′.

Proof. Proposition 5.19 allows us to identify H(L) with L^{2}(ℝ^{n}), in such a way that the smooth vectors of τ correspond to the Schwartz space. The Lie algebra of 𝔰𝔭(W) is spanned by the operators

ix_{p}x_{q}, x_{p} \frac{∂}{∂x_{q}} + \frac{1}{2}δ_{pq}, i\frac{∂^{2}}{∂x_{p}∂x_{q}}  (1 ≤ p, q ≤ n)  (5.23)(a)

The invariance of the subspaces H^{even}(L) and H^{odd}(L) is immediate. For the irreducibility, we use the theory of Harish-Chandra modules. Write F for the standard Fourier transform on L^{2}(ℝ^{n}):

(Ff)(ξ) = ∫_{ℝ^{n}} f(x)e^{ix·ξ} dx.  (5.23)(b)

The Fourier transform induces an automorphism θ of order 4 on the Weyl algebra (of polynomial coefficient differential operators) by the requirement

F(Df) = (θD)F(f).  (5.23)(c)

This is given on generators by the familiar formulas

θ(x_{p}) = −i\frac{∂}{∂x_{p}} ,  θ(\frac{∂}{∂x_{p}}) = −ix_{p}.  (5.23)(d)

Obviously θ restricts to 𝔰𝔭(W) as an automorphism of order 2. Explicitly,

θ(ix_{p}x_{q}) = −i\frac{∂^{2}}{∂x_{p}∂x_{q}} , θ(x_{p} \frac{∂}{∂x_{q}} + \frac{1}{2}δ_{pq}) = x_{q} \frac{∂}{∂x_{p}} + \frac{1}{2}δ_{pq}), θ(i\frac{∂^{2}}{∂x_{p}∂x_{q}}) = −ix_{p}x_{q}.  (5.23)(e)

We want to check that θ is a Cartan involution of 𝔰𝔭(W). Because θ is an involution, it is equivalent to show that the fixed subalgebra 𝔨 is compact. Because Mp(W) is by definition a group of unitary operators, this in turn is equivalent to showing that 𝔨 preserves a family of finite-dimensional subspaces of H(L) whose union is dense. (Then the group K will be embedded in the direct product of the corresponding finitedimensional unitary groups.)

Consider now the function e(x) = exp(− ∑ x_{p}^{2}/2). Then \frac{∂e}{∂x_{p}} = −x_{p}e. Consequently

i\frac{∂^{2}e}{∂x_{p}∂x_{q}} = ix_{p}x_{q}e − iδ_{pq}e,

and therefore

i(\frac{∂^{2}}{∂x_{p}∂x_{q}} − x_{p}x_{q})e = −iδ_{pq}e.

By (5.23)(e), the operator on the left side is of the form X + θX for some X ∈ 𝔰𝔭(W). Combining this with similar formulas for x_{p} \frac{∂}{∂x_{q}}, we find a character λ: 𝔨 → iℝ so that

X · e = λ(X)e  (X ∈ 𝔨 = 𝔰𝔭(W)^{θ}).  (5.23)(f)

Recall the filtration of the Weyl algebra introduced in Definition 5.13, and the action of the Weyl algebra on H(L) from Proposition 5.14. Define

H^{p}(L) = A_{χ}^{p}(W) · e.  (5.23)(g)

This is a finite-dimensional subspace of H(L); clearly it consists of polynomials of degree at most p multiplied by e. By Proposition 5.17 and (5.23)(f), we have

the action of 𝔨 preserves H^{p}(L).  (5.23)(h)

Now the remarks preceding the definition of e show that K is compact. At the same time we have computed the Harish-Chandra module of K-finite vectors in τ: it is

H^{K}(L) = ∪_{p}H^{p}(L),

the space of polynomials times the Gaussian e. The irreducibility we want is equivalent to the algebraic irreducibility of H^{K,even}(L) and H^{K,odd}(L) under the enveloping algebra of 𝔰𝔭(W); that is, under the even part A^{even}(W) of the Weyl algebra.

Write P for the space of polynomial functions on ℝ^{n}. There is an obvious linear isomorphism m from P onto H^{K}(L), sending f to f · e. Of course m does not respect the actions of the Weyl algebra: for the generators we have

\frac{∂fe}{∂x_{p}} = (\frac{∂f}{∂x_{p}} − x_{p}f)e,  x_{q} · fe = (x_{q}f) · e.  (5.24)(a)

Now the linear transformation of the generators of the Weyl algebra defined by

j(\frac{∂}{∂x_{p}}) = \frac{∂}{∂x_{p}} − x_{p},  j(x_{q}) = x_{q}  (5.24)(b)

respects the defining relations; so it extends uniquely to an automorphism j of A(W). Then (5.24)(a) gives

a · m(f) = m(j(a)f)  (a ∈ A(W), f ∈ P)  (5.24)(c)

So our irreducibility problem for H^{K}(L) is equivalent to the irreducibility of P^{even} and P^{odd} under the even polynomial coefficient differential operators; and this is very easy to prove.

It remains to establish the inequivalence of τ^{even} and τ^{odd}. For that, we need to understand the Lie algebra 𝔨 and its representation in τ. We have described 𝔨 fairly explicitly in (5.23)(e), and the map j in (5.24)(b). A straightforward calculation shows that j(𝔨) has a basis of elements

\begin{aligned}E_{pq}^{+} & = i (x_{p} \frac{∂}{∂x_{q}} + x_{q} \frac{∂}{∂x_{p}} + δ_{pq} − \frac{∂^{2}}{∂x_{p}∂x_{q}} )  (1 ≤ p ≤ q ≤ n) \\ E_{pq}^{−} & = x_{p} \frac{∂}{∂x_{q}} − x_{q} \frac{∂}{∂x_{p}}  (1 ≤ p < q ≤ n). \end{aligned}  (5.25)

Notice that the second derivative terms act to lower degree by two, and all other terms preserve degree. It follows that j(𝔨) preserves the filtration of P by degree, and that the action in the associated graded space gr P is given by the same formulas without the second derivative.

Lemma 5.26. Consider the natural action π of G = GL(n, ℂ) on the space P of complex polynomials in n variables. Identify the Lie algebra 𝔤 of G with complex n × n matrices. Then on the standard basis matrices the differential of π is given by

dπ(e_{pq}) = x_{p} \frac{∂}{∂x_{q}} .

The Lie algebra of U(n) (consisting of n × n skew-Hermitian matrices) has a basis consisting of elements e_{pq}^{+} = i(e_{pq} + e_{qp}) and e_{pq}^{−} = e_{pq} − e_{qp}. These act by the operators

dπ(e_{pq}^{+}) = i (x_{p} \frac{∂}{∂x_{q}} + x_{q} \frac{∂}{∂x_{p}} ) , dπ(e_{pq}^{−}) = x_{p} \frac{∂}{∂x_{q}} − x_{q} \frac{∂}{∂x_{p}} .

Write det_{ℂ} for the determinant character of U(n). Then

ddet_{ℂ}(e_{pq}^{+}) = 2iδ_{pq}, ddet_{ℂ}(e_{pq}^{−}) = 0.

This is elementary and standard.

Corollary 5.27. The Lie algebra 𝔨 is isomorphic to 𝔲(n). Write ℂ_{1/2} for the one-dimensional space on which 𝔲(n) acts by half the differential of the determinant character. Then the action of j(𝔨) (cf. (5.24)) on gr P is naturally isomorphic to the action of 𝔲(n) on P ⊗ ℂ_{1/2} The isomorphism sends the basis elements E_{pq}^{±} of j(𝔨) to e_{pq}^{±}.

In particular, the irreducible representations of 𝔨 ≃ 𝔲(n) appearing in τ^{even} (respectively τ^{odd}) are S^{k}(ℂ^{n}) ⊗ ℂ_{1/2} with k even (respectively odd).

Proof. The assertions in the first paragraph are clear from (5.25) and Lemma 5.26. Those in the second follow at once (since the natural representation of U(n) on homogeneous polynomials of degree k is irreducible. Q.E.D.

It follows at once from Corollary 5.27 that τ^{even} and τ^{odd} are inequivalent as representations of the maximal compact subgroup K of Mp(W), which completes the proof of Proposition 5.22. Q.E.D.

As a corollary of the proof, we get a description of the group K.

Proposition 5.28. In the setting of Theorem 5.19, fix a positive-definite bilinear form B on L′, and use it to introduce a Fourier transform F as a unitary operator on H(L) ≃ L^{2}(L′, D_{ℂ}^{1/2}) (cf. (5.23)(b)). As in Proposition 4.3, identify L′ with W/L and L^{∗}, and so identify B with an isomorphism T_{B}: L′ → L. Define a linear transformation on W = L ⊕ L′ by

σ_{B} = ( \begin{matrix}0 & −T_{B}^{−1} \\ T_{B} & 0 \end{matrix} ) .

a) The element σ_{B} belongs to Sp(W). We have σ_{B}^{2} = −1, so σ_{B} is a complex structure on W.

b) Conjugation by σ_{B} defines an involutive automorphism θ_{B} of Sp(W).

c) The symplectic form ω is the imaginary part of a unique positive definite Hermitian form h_{B} on the complex vector space (W, σ_{B}).

d) Write U(W, h_{B}) for the unitary group of the Hermitian form. Then U(W, h_{B}) is the group of fixed points of θ_{B}; it is a maximal compact subgroup of Sp(W). The complex-valued determinant of an automorphism of (W, σ_{B}) defines a unitary character

det_{ℂ}: U(W, h_{B}) → ℂ^{×}.

e) Define

Ũ(W, h_{B}) = {(u, z) ∈ U(W, h_{B}) × ℂ^{×} | det_{ℂ}(u) = z^{2}},

the square root of the determinant cover of U(W, h_{B}). Then projection on the first factor is a two-fold covering

Ũ(W, h_{B}) → U(W, h_{B}),

and projection on the second factor is a unitary character

det_{ℂ}^{1/2}: Ũ(W, h_{B}) → ℂ^{×}.

f) The Fourier transform F is a preimage of σ_{B} in the metaplectic group (Corollary 5.3). The corresponding maximal compact subgroup K of Mp(W) is naturally isomorphic to Ũ(W, h_{B}).

Proof. The proof that σ_{B} ∈ Sp(W) is similar to the proof in Proposition 4.3 that u_{B} ∈ Sp(W); we leave it to the reader. That σ_{B}^{2} = −1 is obvious, and then (b) follows. For (c), the Hermitian form must be

h_{B}(v, w) = ω(σ_{B}(v), w) + iω(v, w).

Conversely, this formula is easily seen to define a Hermitian form (compare the proof of Proposition 4.3). For (d), the fixed points of θ_{B} consists of the complex-linear elements of Sp(W); and this in turn is obviously the unitary group of h_{B}. Since Sp(W) is a noncompact simple real group, any compact group of fixed points of an involution must be a maximal compact subgroup. The assertions in (e) are elementary. Finally for (f), we have seen in Proposition 5.22 that a maximal compact subgroup K of Mp(W) may be constructed as a double cover of the maximal compact subgroup of U(W, h_{B}) of Sp(W); and that K admits a one-dimensional character δ (its action on the Gaussian function e described before (5.23)) whose differential is one half the differential of the determinant character of U(W, h_{B}). Write π: K → U(W, h_{B}) for the covering map; then (π, δ): K → U(W, h_{B}) × ℂ^{×} is an isomorphism from K onto Ũ(W, h_{B}), as we wished to show. Q.E.D.

6. Admissible orbit data. Suppose G is a Lie group. As explained in the introduction, we need a little more than a coadjoint orbit to hope to construct a unitary representation. With the discussion of the metaplectic representation in the last section, we now have in place all the ideas needed to describe Duflo’s version of what that “little more” should be. We begin with an element f ∈ 𝔤^{∗}, and form the coadjoint orbit

X = G · f ≃ G/G_{f};  (6.1)(a)

here of course G_{f} is the isotropy group for the coadjoint action of G at f, a closed subgroup of G. From the formula for the differentiated coadjoint action given before (2.8)(b), we find

𝔤_{f} = {Y ∈ 𝔤 | f([Y, 𝔤]) = 0}.  (6.1)(b)

Recall from Corollary 2.13 that X carries a G-invariant symplectic structure. On the tangent space at f, the symplectic form is given by

T_{f}(X) ≃ 𝔤/𝔤_{f},  ω_{f}(Y + 𝔤_{f}, Z + 𝔤_{f}) = f([Y, Z]).  (6.1)(c)

Of course the tangent vector Y + 𝔤_{f} is just the value at f of the vector field ξ_{Y}, the coadjoint action of Y (see (2.12)(b)). It is clear that the isotropy action of G_{f} on T_{f}(X) preserves the form ω_{f}; so we get a Lie group homomorphism

j_{f}: G_{f} → Sp(𝔤/𝔤_{f}, ω_{f}).  (6.1)(d)

In Corollary 5.3 we constructed a natural double covering

p: Mp(𝔤/𝔤_{f}, ω_{f}) → Sp(𝔤/𝔤_{f}, ω_{f}), ker p = {1, ϵ}.

of the symplectic group. We can use the homomorphism j_{f} to pull this back to a double cover of G_{f}. Explicitly, we define

G̃_{f} = {(g, m) ∈ G_{f} × Mp(𝔤/𝔤_{f}, ω_{f}) | j_{f}(g) = p(m)}.  (6.1)(e)

Then projection on the first factor defines a double covering

p_{f}: G̃_{f} → G_{f},  ker p_{f} = {1, ϵ}.  (6.1)(f)

That is, p_{f}(g, m) = g. Similarly, projection on the second factor defines a Lie group homomorphism

j̃_{f}: G̃_{f} → Mp(𝔤/𝔤_{f}, ω_{f}).  (6.1)(g)

Finally, recall that the metaplectic group was defined as a group of unitary operators on a Hilbert space. The homomorphism j̃_{f} therefore gives rise to (or may be interpreted as) a unitary representation

τ_{f}: G̃_{f} → U(H_{f}),  τ_{f}(ϵ) = −1.  (6.1)(h)

We call τ_{f} the metaplectic representation of G̃_{f}. By Proposition 5.22, this representation decomposes as τ_{f} = τ_{f}^{even} ⊕ τ_{f}^{odd}. Various descriptions of the Hilbert space H_{f} are given in section 5. (For example, if we write 𝔤/𝔤_{f} as a direct sum of Lagrangians L and L′, then H_{f} may be identified with square-integrable half-densities on L′. This is most useful if G_{f} preserves L and L′; but we will not always be able to arrange that.)

Definition 6.2 (see [6].) Suppose G is a Lie group and f ∈ 𝔤^{∗}. Use the notation of (6.1); recall also that we have fixed a non-trivial character χ of ℝ (see (5.1)). An admissible orbit datum at f is an irreducible unitary representation (π, H_{π}) of G̃_{f} with the following two properties:

π(ϵ) = −1  (6.2)(a)

(cf. (6.1)(f)), and

dπ(Y) = dχ(1)f(Y)  (Y ∈ 𝔤_{f}).  (6.2)(b)

(Here the scalars all mean the corresponding multiples of the identity operator on H_{π}.) An equivalent formulation is

π(exp Y) = χ(f(Y))  (Y ∈ 𝔤_{f}).  (6.2)(b′)

The pair (f, π) will be called an admissible orbit datum. If there is an admissible orbit datum at f, we say that the orbit G · f is admissible.

The first of the defining properties says that π should be a “genuine” representation, not descending to G_{f}. To understand the second, notice that (6.1)(b) implies that f: 𝔤_{f} → ℝ is a Lie algebra homomorphism. Therefore dχ(1) · f is a Lie algebra homomorphism to iℝ, the Lie algebra of the unit circle in ℂ^{×}. The second condition therefore says that the restriction of π to the identity component G̃_{f,0} should be a multiple of a (specified) unitary character.

Notice that the group G acts on admissible orbit data, as follows. Suppose (f, π) is an admissible orbit datum. If g ∈ G, then g · f ∈ 𝔤^{∗}. Conjugation by g defines an isomorphism c_{g} from G_{f} to G_{g·f}. The covering G̃_{f} pushes forward under this isomorphism to a double cover of G_{g·f}. A little more explicitly, the covering group is just G̃_{f}, and the covering map is c_{g} ∘ p_{f} (notation as in (6.1)(f)). It is easy to check that this covering is naturally isomorphic to the one G̃_{g·f} defined in (6.1)(e). Under this isomorphism, the representation π of G̃_{f} is identified with a representation that we call g · π of G̃_{g·f}. Again it is easy to check that g · π is an admissible orbit datum at g · f; so it makes sense to define g · (f, π) = (g · f, g · π). Because inner automorphisms act trivially on representations (up to equivalence), the stabilizer of (f, π) is precisely G_{f}.

For our purposes Duflo’s definition of admissible orbit datum is always exactly the right “integrality hypothesis” required in Problem 1.2. Here is our promised refinement of that problem.

Problem 6.3. Suppose G is a type I Lie group, and (f, π) is an admissible orbit datum. Find a construction attaching to (f, π) a unitary representation γ(f, π) of G. This representation should be close to irreducible, and should depend only on the G orbit of (f, π): that is, γ(f, π) should be unitarily equivalent to γ(g · (f, π)) for every g in G.

Even in this form the problem is still not perfectly formulated. If G is the double cover of SL(3, ℝ) and f is a nilpotent element with Jordan blocks of sizes 2 and 1, then there are exactly four admissible orbit data at f. Only three of these have associated unitary representations (see [20] and [22], Example 12.4). The example of [17] mentioned in the introduction is also not completely explained. We refer to [22] for a more extensive discussion of the shortcomings of Problem 6.3.

The work of Kirillov and Kostant emphasizes a condition different from admissibility, which is still widely used in work on geometric quantization. We recall this condition, partly for the light it sheds on Definition 6.2.

Definition 6.4 (see [11] or [9], Chapter V.) Suppose G is a Lie group and f ∈ 𝔤^{∗}. Use the notation of (6.1); recall also that we have fixed a non-trivial character χ of ℝ (see (5.1)). An integral orbit datum at f is an irreducible unitary representation (p, V_{p}) of G_{f} with the following property:

dp(Y) = dχ(1)f(Y)  (Y ∈ 𝔤_{f}).  (6.4)(a)

An equivalent formulation is

p(exp Y) = χ(f(Y))  (Y ∈ 𝔤_{f}).  (6.4)(a′)

(As in Definition 6.2, the scalars mean multiples of the identity operator on V_{p}.) If there is an integral orbit datum at f, we say that the orbit G · f is integral.

The unitary representation p is very often forced by (6.4)(a) to be one-dimensional; obviously this is true if G_{f} is connected, for example. In any case we can use it to define an equivariant Hermitian vector bundle G ×_{G_{f}} V_{p} over the orbit G/G_{f}. Many descriptions of geometric quantization appear to depend heavily on this vector bundle, and the lack of any obvious analogue of it in the admissible case is at first disconcerting. Ultimately we will argue that the structure provided by Definition 6.2 is more natural. For the moment, we can at least explain why the notions of integral and admissible orbit sometimes coincide.

Proposition 6.5. In the setting of (6.1), suppose L ⊂ 𝔤/𝔤_{f} is a Lagrangian subspace. Define H to be the subgroup of G_{f} preserving L. Each h ∈ H defines a linear transformation of L, which has a non-zero determinant det_{L}(h) Taking the sign of this determinant, we get a character

sgn_{L}: H → {±1}.

Taking the square root of this character defines a double cover

1 → {1, ϵ} → H̃ → H → 1

(cf. Proposition 5.28(e)); it is equipped with a character sgn_{L}^{1/2}: H̃ → {±1, ±i} which acts by −1 on ϵ.

This covering of H is naturally isomorphic to the metaplectic covering of H induced by G̃_{f}. If H = G_{f}— that is, if G_{f} preserves the Lagrangian subspace L—then tensoring with sgn_{L}^{1/2} defines a bijection from admissible orbit data at f to integral orbit data at f. In particular, the orbit G · f is admissible if and only if it is integral in this case.

If G is nilpotent, then G_{f} always preserves some Lagrangian subspace of 𝔤/𝔤_{f}; so the notions of integral and admissible coincide.

We postpone the proof of Proposition 6.5 to section 7 (see the remarks after the proof of Proposition 7.2).

Here is the first geometric structure we can get from an admissible orbit datum.

Definition 6.6. Suppose G is a Lie group, f ∈ 𝔤^{∗}, and (π, H_{π}) is an admissible orbit datum at f. Recall from (6.1) the metaplectic representation (τ_{f}, H_{f}) of G̃_{f}. Form the tensor product representation (π ⊗ τ_{f}, H_{π} ⊗ H_{f}). By (6.2)(a) and (6.1)(h), this representation is trivial on the kernel {1, ϵ} of the covering; so we may regard it as a representation of G_{f}. We may therefore define

S_{π} = G ×_{G_{f}} H_{π} ⊗ H_{f},  (6.6)(a)

a Hilbert bundle over the orbit G · f. One might call this the bundle of twisted symplectic spinors on G · f. The decomposition of H_{f} as a direct sum of even and odd parts passes to the bundle:

S_{π} = S_{π}^{even} ⊕ S_{π}^{odd}.  (6.6)(b)

We will also want to consider the (Fréchet) subbundles corresponding to the smooth vectors in the metaplectic representation, such as

S_{π}^{even,∞} = G ×_{G_{f}} H_{π} ⊗ H_{f}^{even,∞}.  (6.6)(c)

Here we mean vectors smooth with respect to the action of the full metaplectic group Mp(𝔤/𝔤_{f}). This is the Schwartz space described in Corollary 5.12 and Theorem 5.19(a). (In every case we consider seriously, the orbit datum π will be finite-dimensional; so H_{π} consists entirely of smooth vectors.) Similarly, we can enlarge H_{f} to the corresponding space of distribution vectors (the continuous dual of the smooth vectors, obtaining bundles like

S_{π}^{even,−∞} = G ×_{G_{f}} H_{π} ⊗ H_{f}^{even,−∞}.  (6.6)(d)

The space H_{f}^{−∞} may be identified as in Corollary 5.12 with a space of tempered distributions on ℝ^{n}.

The representations we want to associate to G·f will be related to spaces of sections of these symplectic spinor bundles. It will be convenient to interpret these sections (of infinite-dimensional bundles) as sections of finite-dimensional bundles over a larger space. We conclude this section by introducing this larger space.

Definition 6.7. Suppose (X, ω_{X}) is a symplectic manifold of dimension 2n (Definition 2.1). The bundle of infinitesimal Lagrangians on X is a fiber bundle B(X) over X. The fiber over a point x ∈ X is B(T_{x}(X)) (Definition 4.2), the Lagrangian Grassmannian of Lagrangian subspaces of the tangent space at x to X.

7. Symplectic spinors and the Lagrangian Grassmannian. In this section we will describe a realization of the even half τ^{even} of the metaplectic representation as a space of sections of a line bundle on the Lagrangian Grassmannian. In more traditional representation-theoretic language, we are realizing τ^{even} as a subrepresentation of a degenerate principal series representation, induced from a non-unitary one-dimensional character of a Siegel parabolic subgroup. These results are known to many people; one reference is [12], section 5.

We begin as in Definition 4.2 with a finite-dimensional real symplectic vector space (W, ω) and a Lagrangian subspace L. Write P (L) for the stabilizer of L in Sp(W), and χ(L): P (L) → ℝ^{×} for the determinant character (the determinant of the action of P (L) on L). In analogy with Proposition 5.28(e), we define the square root of the determinant cover of P (L) by

P̃(L) = {(p, z) ∈ P (L) × ℂ^{×} | χ(L)(p) = z^{2}}.  (7.1)(a)

Just as in Proposition 5.28(e), projection on the second factor defines a character

χ(L)^{1/2}: P̃(L) → ℂ^{×};  (7.1)(b)

it takes values in ℝ^{×} ∪ iℝ^{×}.

Proposition 7.2. The covering P̃(L) defined by (7.1) is naturally isomorphic to the covering of P (L) induced by the double cover Mp(W) of Sp(W) (Corollary 5.3). A little more precisely, let H(L) be the realization of the Schrödinger representation of the Heisenberg group in (5.9), and let τ_{L} be the representation of P (L) on H(L) constructed in (5.21). Then the metaplectic representation τ of Mp(W) on H(L) is given by

τ(x) = (χ(L)^{1/2}(x)/|χ(L)(\overline{x})|^{1/2})τ_{L}(\overline{x}).

Here x ∈ P̃(L), and we write \overline{x} for its image in P (L).

The factor in front on the right is a character of P̃(L) taking values in {±1, ±i}. It is trivial on the identity component of P̃(L).

Proof. Write MP (L) for the preimage of P (L) in Mp(W). This is a double cover of P (L). By the definition of the metaplectic representation in Corollary 5.3, and the construction of τ_{L} in (5.21), we find that there is a genuine character ϕ of MP (L) (that is, ϕ(ϵ) = −1 for ϵ the non-trivial element of the kernel of the covering map) with the property that τ(y) = ϕ(y)τ_{L}(\overline{y}). (Here y ∈ MP (L) and \overline{y} is its image in P (L).) What we propose to show is

ϕ(y)^{2} = sgn χ(L)(\overline{y})  (y ∈ MP (L))  (7.3)(a)

It follows that

(ϕ(y)|χ(L)(\overline{y})|^{1/2})^{2} = χ(L)(\overline{y})  (y ∈ MP (L))  (7.3)(b)

The character ϕ(y)|χ(L)(\overline{y})|^{1/2} therefore provides the isomorphism we want from MP (L) to P̃(L): explicitly, it sends y to the pair (\overline{y}, ϕ(y)|χ(L)(\overline{y})|^{1/2}). The formula for τ in the proposition also follows immediately.

So we need only prove (7.3)(a). The differential of τ_{L} is computed explicitly in (5.21). It takes values in the commutator subalgebra of A_{χ}^{2,even}(W)_{ℝ}, just as the differential of τ does. Consequently the differential of ϕ is zero; that is, ϕ is trivial on the identity component of MP (L). Now (7.3)(a) follows for y in the identity component. Since ϕ^{2} does descend to a character of P (L), it remains only to prove (7.3)(a) for some element y such that χ(L)(\overline{y}) < 0.

Now suppose we are in the setting of Theorem 5.19 and Proposition 5.28. The representation space H(L) for τ is the space of square-integrable half-densities on L′, which is an inner product space. We therefore have a well-defined orthogonal group O(L′) ⊂ GL(L′) ≃ GL(L) ⊂ P (L). Choose an element \overline{y} ∈ O(L′) of determinant −1, and a preimage y in MP (L). Recall from (5.23) the Gaussian e ∈ H(L). By (5.21)(c), τ_{L}(\overline{y})(e) = e. On the other hand, \overline{y} belongs to the unitary group U(W, h_{B}) described in Proposition 5.28; so Corollary 5.27 implies that τ(y)(e) = ae, for a scalar a which is a square root of det_{ℂ}(\overline{y}). Now this complex determinant character on the unitary group restricts to the real determinant on the orthogonal subgroup; so det_{ℂ}(\overline{y}) = −1, and a = ±i. Consequently ϕ(y) = ±i, and (7.3)(a) follows. Q.E.D.

Proposition 6.4 is an immediate corollary of Proposition 7.2: the subgroup H of G_{f} is just the preimage in G_{f} of the parabolic subgroup P (L) ⊂ Sp(𝔤/𝔤_{f}).

Suppose (τ, H) is a unitary representation of a Lie group G, and H^{∞} is the subspace of smooth vectors; this is a Fréchet subrepresentation of τ. We want to define a corresponding “superrepresentation” H^{−∞} of distribution vectors. Roughly speaking this should be the dual space of H^{∞}. The difficulty is that the dual space doesn’t contain H. If w ∈ H, then the linear functional λ_{w} on H^{∞} defined by

λ_{w}(v) = ⟨v, w⟩

does indeed belong to the dual space (H^{∞})^{∗}, but the map sending w to λ_{w} is conjugate-linear. We therefore define H^{−∞} to be the Hermitian dual of H^{∞}. This means that as a real vector space, H^{−∞} = (H^{∞})^{∗}, but complex multiplication is defined by

(z · λ)(v) = λ(\overline{z} · v).

With this definition the map w ↦ λ_{w} above provides an inclusion of H in H^{−∞}. The transpose of the representation τ^{∞} defines an algebraic representation τ^{−∞} of G on H^{−∞} by continuous operators. Without further assumptions on τ, it need not be a continuous representation, however.

Proposition 7.4. Suppose (τ, H) is a metaplectic representation of Mp(W) (Corollary 5.3). Write H = H^{even}⊕H^{odd} as in Proposition 5.22, and H^{∞} for the subspace of smooth vectors. Finally write H^{−∞} for the Hermitian dual of H^{∞}, the space of distribution vectors of τ. Suppose L ⊂ W is a Lagrangian subspace, P (L) is its stabilizer in Sp(W), and U(L) is the unipotent radical of P (L). Write also U(L) for the identity component of the inverse image of U(L) in Mp(W). Identify the preimage P̃(L) of P (L) in Mp(W) as in Proposition 7.2.

a) For each Lagrangian subspace L, the space L^{∗}(L) of U(L)-fixed even distribution vectors has dimension

1:

L^{∗}(L) = {λ ∈ H^{even,−∞} | τ(u)(λ) = λ (u ∈ U(L)}.

b) The representation of P̃(L) on H^{−∞} preserves L^{∗}(L), and acts there by the character (\overline{χ(L)^{1/2}})^{−1}.

c) Write L(L) for the Hermitian dual space of L^{∗}(L). These lines may be assembled into a smooth Mp(W)- equivariant line bundle L on the Lagrangian Grassmannian

B(W) ≃ Sp(W)/P (L) ≃ Mp(W)/P̃(L).

It is isomorphic to the line bundle induced by the character χ(L)^{1/2} of P̃(L).

d) There is a canonical Mp(W)-equivariant embedding

γ: H^{even,∞} → C^{∞}(B(W), L),

defined as follows. Suppose v ∈ H^{even,∞} and L ∈ B(W). We need to specify the value γ(v)(L) of the section γ(v) at the point L. This is an element of L(L), and therefore a linear functional on L^{∗}(L). Its value at λ ∈ L^{∗}(L) is

γ(v)(L)(λ) = λ(v).

Here we are using the description of λ from (a) as a distribution vector in H^{even,−∞}.

Proof. For (a), we use the realization of H^{∞} in Corollary 5.12, as the Schwartz space S(L′, D_{ℂ}^{1/2}) of rapidly decreasing half-densities on L′. The space H^{−∞} of is then identified with tempered distributions. The action of P̃(L) is computed in (5.21) and Proposition 7.2. Since the Lie algebra of U(L) acts by multiplication by purely imaginary quadratic polynomials, it is easy to see that the U(L)-fixed distribution vectors are spanned by evaluation and first derivatives at the origin. Only evaluation is even; so L^{∗}(L) is spanned by the linear functional λ defined by

λ(ϕ(x′)(dx′)^{1/2}) = ϕ(0).

(In this definition we have implicitly chosen a half-density (dx′)^{1/2}.) Because P̃(L) normalizes U(L), it automatically preserves L^{∗}(L); the formula for the action on λ follows from (5.21)(c) and Proposition 7.2. (The complex conjugate arises because of the twist in the complex structure on H^{−∞}.) Part (c) is immediate from (b). The mapping defined in (d) is non-zero and Mp(W)-equivariant to the space of arbitrary sections of L. The first thing that requires proof is that γ(v) is a smooth section, and that γ is a continuous map. To see this, choose a coordinate neighborhood X of L in B(W) ≃ Mp(W)/P̃(L) that lifts to t(X) ⊂ Mp(W). This means that t(X) is a smooth submanifold of Mp(W), and that group multiplication identifies t(X) × P̃(L) with an open subset of Mp(W). Now the smooth structure on L^{∗} arises from its identification with an induced bundle. It follows that the section x ↦ t(x) · λ is a smooth local trivialization of L^{∗} over X. To say that γ(v) is smooth at L therefore means precisely that the function x ↦ (t(x) · λ)(v) is smooth in x. But this may be written as λ(τ^{∞}(t(x)^{−1})(v)), which is smooth in x because τ^{∞} is a smooth representation. This argument also shows that the map γ is continuous (from the Fréchet space of smooth vectors to that of smooth sections of L). Q.E.D.

Proposition 7.4 is our promised realization of τ^{even} in a degenerate principal series representation. Here are some useful technical facts about it.

Proposition 7.5. Suppose we are in the setting of Proposition 7.4.

a) The map γ is an isomorphism from H^{even,∞} onto a closed subspace of C^{∞}(B(W), L).

b) Write D_{ℂ} for the complex line bundle of densities on B(W). Then D_{ℂ} is isomorphic to the line bundle induced by the character |χ(L)|^{n+1} of P (L).

c) There is a natural Hermitian pairing between smooth sections of L and smooth sections of L^{∗} ⊗ D_{ℂ} (compare (5.7)). For this reason we can define C^{−∞}(B(W), L^{∗} ⊗D_{ℂ}) (the space of distribution sections) to be the continuous Hermitian dual of C^{∞}(B(W), L).

d) The transpose of γ is an Mp(W)-equivariant continuous surjection

γ^{∗}: → C^{−∞}(B(W), L^{∗} ⊗ D_{ℂ}) → H^{even,−∞}.

e) The map γ^{∗} of (d) restricts to a continuous surjection

γ^{∗,∞}: → C^{∞}(B(W), L^{∗} ⊗ D_{ℂ}) → H^{even,∞}.

f) The map γ of Proposition 7.4 extends to a continuous embedding with closed range

γ^{−∞}: H^{even,−∞} → C^{−∞}(B(W), L).

Sketch of proof. Part (a) follows from the general theory of smooth globalizations of Casselman and Wallach (see [5]): the smooth globalization of a finite length Harish-Chandra module may be realized as the space of smooth vectors in any reflexive Banach space globalization. In this case we compare the two globalizations of the metaplectic Harish-Chandra module given by the metaplectic Hilbert space and by the degenerate principal series. The description of the density bundle amounts to a calculation of the character by which P (L) acts on the top exterior power of 𝔤/𝔭(L). Part (c) is essentially a definition, and (d) follows from (a) and the Hahn-Banach theorem. Finally (e) and (f) follow from (d) and Proposition 7.4 (respectively), again by the general results in [5] on uniqueness of distribution globalizations. Q.E.D.

Definition 7.6. Suppose we are in the setting of Definition 6.6. Write X = G · f for the coadjoint orbit of f, a symplectic manifold. Write B(X) for the bundle of infinitesimal Lagrangians (Definition 6.6). Then the admissible orbit datum gives rise to a G-equivariant vector bundle on B(X), as follows. The fiber at a Lagrangian L in 𝔤/𝔤_{f} is by definition H_{π} ⊗ L(L). Here H_{π} is the representation space of the admissible orbit datum, and L(L) is the line defined in Proposition 7.4(c) (using the metaplectic representation H_{f}). We write this vector bundle as V_{π}.

8. Existence of Lagrangian coverings. In this section we will prove Theorem 1.10. We therefore fix a complex reductive algebraic group G, and a coadjoint orbit

X = G · f ⊂ 𝔤^{∗}.  (8.1)(a)

We fix also a Borel subgroup B of G, with unipotent radical N. This defines an Ad^{∗}(B)-invariant linear subspace

𝔫^{⊥} = {ϕ ∈ 𝔤^{∗} | ϕ|_{𝔫} = 0} ⊂ 𝔤^{∗}.  (8.1)(b)

Because G is reductive, we may use an invariant symmetric form to identify 𝔤^{∗} with 𝔤. Under such an identification, 𝔫^{⊥} is sent to 𝔟. We will also mention the B-invariant subspace

𝔟^{⊥} = {ϕ ∈ 𝔤^{∗} | ϕ|_{𝔟} = 0} ⊂ 𝔫^{⊥} ⊂ 𝔤^{∗}.  (8.1)(c)

In the identification of 𝔤^{∗} with 𝔤, 𝔟^{⊥} corresponds to 𝔫. Here is the easy part of what we want to prove.

Lemma 8.2. In the setting (8.1), the intersection

X_{𝔟} = X ∩ 𝔫^{⊥}

is a non-empty B-invariant closed subset of X. If X is semisimple, then X_{𝔟} is the union of finitely many closed B orbits. If X is nilpotent, then X_{𝔟} ⊂ 𝔟^{⊥}.

Proof. We identify 𝔤^{∗} with 𝔤 using an invariant symmetric bilinear form as above. For the first assertion, only the non-emptiness requires proof. The element f of X corresponds to an element Z ∈ 𝔤. The subspace ℂZ is a solvable subalgebra of 𝔤, and is therefore contained in a maximal solvable subalgebra 𝔟′. By definition 𝔟′ is a Borel subalgebra, so it is conjugate by G to 𝔟: Ad(g)(𝔟′) = 𝔟 for some g ∈ G. Since Z ∈ 𝔟′, it follows that Ad(g)(Z) ∈ 𝔟. Expressed in terms of X, this says that Ad^{∗}(g)(f) ∈ 𝔫^{⊥}, and therefore that X_{𝔟} is non-empty.

For the second assertion, fix a maximal torus H ⊂ B; this is a Cartan subgroup of G. Each semisimple conjugacy class for B in 𝔟 meets 𝔥 exactly once (see for example [4], Theorem III.10.6). By [4], Theorem III.9.2, each B orbit on X_{𝔟} is closed; so we need only show that these orbits are finite in number. This amounts to the fact that a semisimple orbit in 𝔤 meets 𝔥 finitely often (in fact in a single orbit of the Weyl group of H). This is well known.

The last assertion says that the nilpotent elements in 𝔟 are exactly those in 𝔫. This is [4], Theorem III.10.6(4). Q.E.D.

As an orbit for an algebraic group action, X is a locally closed algebraic subvariety of 𝔤^{∗}. Consequently X_{𝔟} is a locally closed algebraic subvariety of the vector space 𝔫^{⊥}. We may therefore write X_{𝔟} as the union of irreducible components:

X_{𝔟} = X_{𝔟}^{1} ∪ ⋯ ∪ X_{𝔟}^{r}.  (8.3)

Here each X_{𝔟}^{i} is an irreducible locally closed B-stable algebraic subvariety of 𝔫^{⊥}. The intersection of any two components is a proper subvariety of each, and hence of lower dimension than either.

Lemma 8.4. In the setting (8.3), suppose ϕ ∈ X_{𝔟}. Then the linear functional ψ = ϕ|_{𝔟} ∈ 𝔟^{∗} vanishes on 𝔫 = [𝔟, 𝔟]. It takes a constant value ψ^{i} on each component X_{𝔟}^{i}.

Proof. As in the proof of Lemma 8.1, we use an invariant bilinear form to translate into statements about conjugacy classes of G in 𝔤. So we are fixing a conjugacy class X, and X_{𝔟} is its intersection with 𝔟. That each ϕ vanishes on 𝔫 follows from the remark after (8.1)(b). The operation of restricting linear functionals to 𝔟 amounts to projecting from 𝔟 to 𝔟/𝔫 ≃ 𝔥. For Z in 𝔟, the element of 𝔥 obtained in this way represents the conjugacy class of the semisimple part of Z; so there are only finitely many possibilities for ψ as ϕ varies over X_{𝔟}. It follows at once that ψ is constant on components of X_{𝔟}. Q.E.D.

Proposition 8.5 (Spaltenstein [19]). In the setting (8.3), each component X_{𝔟}^{i} has dimension equal to half the dimension of X.

Sketch of proof. For X nilpotent, this is the main theorem in [19]. The general case may easily be reduced to that, using the Jordan decomposition. We omit the details. Q.E.D.

Theorem 8.6 (Ginsburg [7], Theorem 4.1). Suppose X is a complex Poisson algebraic variety endowed with a Hamiltonian action of the solvable algebraic group B (Definition 3.7) with moment map μ_{B}: X → 𝔟^{∗}, and Ω ⊂ 𝔟^{∗} is a coadjoint orbit. Then μ_{B}^{−1}(Ω) is a co-isotropic subvariety of X (Definition 4.14).

Corollary 8.7 (Ginsburg [7], Proposition 4.3). In the setting of (8.1) and (8.3), each component X_{𝔟}^{i} is Lagrangian in X.

Proof. We will apply Ginsburg’s theorem to the symplectic variety X = G · f of (8.1)(a), and the solvable group B. The action of G on X is Hamiltonian, with tautological moment map μ_{G} the inclusion of X in 𝔤^{∗}. It follows that the action of B on X is Hamiltonian, with moment map μ_{B} given by μ_{G} composed with the projection 𝔤^{∗} → 𝔟^{∗} (restriction of linear functionals). Fix a component X_{𝔟}^{i} of X_{𝔟} (cf. (8.3)), and define ψ^{i} ∈ 𝔟^{∗} to be the constant value of μ_{B} on X_{𝔟}^{i} (Lemma 8.4). Because X_{𝔟}^{i} is B-stable, Ω^{i} = {ψ^{i}} is an orbit of B. (This is also a consequence of the fact that ψ^{i} vanishes on [𝔟, 𝔟] = 𝔫.) The inverse image of Ω^{i} under μ_{B} is contained in X_{𝔟} (by Lemma 8.2), and so must be a union of components:

μ_{B}^{−1}(Ω^{i}) = ⋃_{{j | ψ^{j}=ψ^{i}}} X_{𝔟}^{j}.

By Theorem 8.6, this union, and in particular its irreducible component X_{𝔟}^{i}, is a co-isotropic subvariety of X. By Proposition 4.13, every tangent space T_{x}(X_{𝔟}^{i}) is a co-isotropic subspace of the symplectic vector space T_{x}(X). If x is a smooth point of X_{𝔟}^{i}, this tangent space has dimension exactly half the dimension of T_{x}(X) (Proposition 8.5), and is therefore Lagrangian (cf. (4.1)). By Definition 4.17, X_{𝔟}^{i} is a Lagrangian subvariety, as we wished to show. Q.E.D.

(The proof in [7] that X_{𝔟}^{i} is isotropic requires a little elucidation; we prefer to deduce it from Spaltenstein’s Proposition 8.5, which Ginsburg claims as a corollary.)

Proof of Theorem 1.10. In the setting of Corollary 8.7, fix a component X_{𝔟}^{i}. Write L for the smooth part of this component, a smooth Lagrangian subvariety of X. (We have dropped the superscript i since it will be fixed henceforth.) Because X_{𝔟}^{i} is B-stable, L must be as well. Define

Q = {q ∈ G | q · L = L},  (8.8)(a)

a subgroup of G containing B. Any subgroup containing B is parabolic, so

M = {g · L | g ∈ G} ≃ G/Q  (8.8)(b)

is a partial flag variety for G. Now L is a smooth algebraic variety with an algebraic action of Q; so we can form a fiber bundle

Z = G ×_{Q} L →^{ρ} G/Q ≃ M.  (8.8)(c)

A point of Z is an equivalence class in G × L, with (gq, l) equivalent to (g, q · l) whenever g ∈ G, q ∈ Q, and l ∈ L. The action of G on X gives a natural map G × L → X, (g, l) ↦ g · l. It is now clear that this map is constant on the equivalence classes defining Z; so it descends to an algebraic map

π: Z → X,  π(g, l) = g · l.  (8.8)(d)

We have now constructed all the spaces and maps required for a Lagrangian covering (Definition 1.9). By construction ρ is a fibration, and the fact that π × ρ is injective is trivial. Define M_{f} to be the fiber of π over the base point f of X. (The notation is chosen because M is the space of translates of L in X, and M_{f} may be identified with the subvariety of Lagrangians in M containing f.) Because π is a submersion, M_{f} is smooth. The isotropy group G_{f} acts on M_{f}, so we can form the (smooth) fiber product G ×_{G_{f}} M_{f}. For formal reasons there is a G-equivariant algebraic map

i: G ×_{G_{f}} M_{f} → Z,  i(g, m) = g · m.  (8.8)(e)

It is easy to check that i is a bijection on points. The tangent space to X at a point z ∈ M_{f} fits in a short exact sequence

0 → T_{z}(M_{f}) → T_{z}(Z) → T_{eG_{f}} (G/G_{f}) → 0.

Here the second map is dπ, and the first is the differential of the inclusion of M_{f} in Z. There is a similar exact sequence for T_{z}(G×_{G_{f}} M_{f}), and di provides a map from the first exact sequence to the second. The five lemma then guarantees that di is an isomorphism, and, it follows that i must be an isomorphism. Therefore π is a fibration. (It also follows that M_{f} is smooth.) The last requirement in Definition 1.9 (that π × ρ embeds Z in X × M) follows similarly by inspecting tangent spaces; we omit the details. Q.E.D.

9. Construction of representations. In this section we will fill in some details in the construction of representations outlined at the end of the introduction. We work with a complex reductive group G, and a coadjoint orbit X = G · f ≃ G/G_{f} (cf. (8.1)). Fix a metaplectic representation (τ_{f}, H_{f}) of the metaplectic cover Mp(𝔤/𝔤_{f}), and the corresponding cover G̃_{f} of G_{f} as in (6.1). We fix also an admissible orbit datum (π, H_{π}) at f (Definition 6.2). Because the group of connected components of G̃_{f} is finite, the representation π is necessarily finite-dimensional. As in Definition 6.6, this gives rise to a Hilbert bundle

S_{π} = G ×_{G_{f}} H_{π} ⊗ H_{f},  (9.1)(a)

and to various Fréchet subbundles like S_{π}^{even,∞} (cf. (6.6)(c)).

We recall from Definition 6.7 the bundle B(X) of infinitesimal Lagrangians in X, and from Definition 7.6 the finite-dimensional vector bundle

V_{π} → B(X)  (9.1)(b)

over B(X). We write C^{∞}(B(X), V_{π}) for its space of smooth sections. Using Proposition 7.4, we find a natural inclusion

γ_{X}: C^{∞}(X, S_{π}^{even,∞}) ↪ C^{∞}(B(X), V_{π}).  (9.1)(c)

The image of γ_{X} consists of those smooth sections of V_{π} whose restriction to each fiber of B(X) belongs to the image of the corresponding map γ in Proposition 7.4. (Recall that the fiber over f of B(X) is the Lagrangian Grassmannian of the symplectic vector space 𝔤/𝔤_{f}.)

As in (8.8), we fix a component X_{𝔟}^{i}, and write Q for its stabilizer in G (a parabolic subgroup) and L for its smooth locus (a locally closed smooth Lagrangian subvariety of X). As in (8.8), we write M ≃ G/Q for the family of translates of L, and Z = G ×_{Q} L. The construction of (1.11)(c) provides a map of bundles over X

τ: Z → B(X).  (9.2)(a)

Using τ, we can pull the bundle V_{π} back to a G-equivariant vector bundle

τ^{∗}(V_{π}) → Z.  (9.2)(b)

(The rank of this vector bundle is the dimension of the admissible orbit datum π.) Smooth sections of V_{π} pull back to smooth sections of τ^{∗}(V_{π}):

τ^{∗} : C^{∞}(B(X), V_{π})) → C^{∞}(Z, τ^{∗}(V_{π}).  (9.2)(c)

Composing the maps of (9.1)(c) and (9.2)(c) gives a map

τ^{∗} ∘ γ_{X}: C^{∞}(X, S_{π}^{even,∞}) → C^{∞}(Z, τ^{∗}(V_{π})).  (9.2)(d)

The next ingredient we need is a finite-dimensional smooth representation (γ, W_{γ}) of the parabolic subgroup Q. Such a representation gives a vector bundle

W_{γ} → G/Q ≃ M,  (9.3)(a)

which pulls back by the fibration ρ to a vector bundle

ρ^{∗}(W_{γ}) → Z.  (9.3)(b)

In this way the space of smooth sections of W_{γ} may be identified with a space of sections of ρ^{∗}(W_{γ}):

ρ^{∗}: C^{∞}(M, W_{γ}) ↪ C^{∞}(Z, ρ^{∗}(W_{γ})).  (9.3)(c)

Last but not least, we need a G-equivariant isomorphism of vector bundles

j_{γ,π}: τ^{∗}(V_{π}) ˜→ ρ^{∗}(W_{γ}).  (9.3)(d)

The existence of this isomorphism is of course not automatic; it imposes a strong constraint on γ, which may be impossible to satisfy. Now (9.3)(c) and (9.2)(d) define G-invariant spaces of sections of the same vector bundle over Z; so it makes sense to consider their intersection. This intersection is the representation we want. Here is a precise statement.

Definition 9.4. Suppose we are in the setting of (9.1)–(9.3). That is, we fix a coadjoint orbit X = G · f for a complex reductive group G, an admissible orbit datum (π, H_{π}) at f (Definition 6.2), a smooth Lagrangian L (constructed as in (8.8)) and stabilized by a parabolic Q. Fix also a finite-dimensional smooth representation (γ, W_{γ}) of Q, and a G-equivariant isomorphism of vector bundles j_{γ,π} as in (9.3)(d). (Recall that γ and j_{γ,π} need not exist.) Then the smooth representation of G attached to (f, π, L, γ, j_{γ,π}) is by definition

V (f, π, L, γ, j_{γ,π}) = ρ^{∗}(C^{∞}(M, W_{γ})) ∩ j_{γ,π}(τ^{∗} ∘ γ_{X}(C^{∞}(X, S_{π}^{even,∞})),

a space of sections of ρ^{∗}(W_{γ}) over Z. Thus V (f, π, L, γ, j_{γ,π}) may be identified with a G-invariant subspace of C^{∞}(M, W_{γ}), which in turn is the space of smooth vectors in the degenerate principal series representation induced from γ on Q (non-normalized induction). In terms of the normalized induction of Example 5.8, this is

V (f, π, L, γ, j_{γ,π}) ⊂ Ind_{Q}^{G}(γ ⊗ ρ_{G/Q}^{−1}).

Example 9.5. Suppose G = GL(4, ℂ). We identify 𝔤^{∗} with the Lie algebra M(4, ℂ) (consisting of all four by four complex matrices), sending a matrix T to the linear functional f_{T} defined by f_{T} (S) = tr T S. We consider the coadjoint orbit X consisting of all rank two matrices f with f^{2} = 0. These are the nilpotent matrices corresponding to the partition 2+2 of 4; the orbit has dimension 8. We can take for a representative the matrix (written with two by two blocks)

f = ( \begin{matrix}0 & I \\ 0 & 0 \end{matrix} ) .

The isotropy group is the centralizer in G of the matrix f, namely

G_{f} = \{( \begin{matrix}A & B \\ 0 & A \end{matrix} ) | A ∈ GL(2, ℂ), B ∈ M(2, ℂ)\} .  (9.5)(a)

Because G is complex, the metaplectic cover G̃_{f} is trivial (isomorphic to G_{f} × ℤ/2ℤ). There is only one admissible orbit datum π: it is trivial on G_{f}, and acts by the non-trivial character on ℤ/2ℤ.

The variety X_{𝔟} is easily calculated by writing down the condition for an upper triangular matrix to have square zero; we find

X_{𝔟} = \{T = ( \begin{matrix}0 & a & b & c \\ 0 & 0 & d & e \\ 0 & 0 & 0 & f \\ 0 & 0 & 0 & 0 \end{matrix} ) | ad = df = ae + bf = 0, rank T = 2\}  (9.5)(b)

The rank condition picks out an open subset of the four-dimensional variety determined by the three equations. There are exactly two irreducible components:

X_{𝔟}^{1} = \{T = ( \begin{matrix}0 & a & b & c \\ 0 & 0 & 0 & e \\ 0 & 0 & 0 & f \\ 0 & 0 & 0 & 0 \end{matrix} ) | ae + bf = 0, (a, b) ≠ 0, (e, f) ≠ 0\}  (9.5)(c)

and

X_{𝔟}^{2} = \{T = ( \begin{matrix}0 & 0 & b & c \\ 0 & 0 & d & e \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{matrix} ) | be − dc ≠ 0\} .  (9.5)(d)

The second is preserved by G_{f}, and therefore leads to a polarization of X. We therefore concentrate on the first. It is smooth, and so equal to L; the stabilizer of L is the standard parabolic subgroup Q with Levi factor GL(1) × GL(2) × GL(1). Notice that L contains the base element f. Calculations are simplified by the fact that Q acts transitively on L; so

L ≃ Q/Q_{f},  Q_{f} = Q ∩ G_{f} = \{( \begin{matrix}A & B \\ 0 & A \end{matrix} ) | A ∈ GL(2) upper triangular,B ∈ M(2, ℂ)\} .  (9.6)(a)

It follows that Z ≃ G/Q_{f}. The equivariant line bundle τ^{∗}V_{π} is necessarily induced by a character α of Q_{f}. Proposition 7.4 implies that α is given by the square root of the absolute value of the (real) determinant of Q_{f} acting on the tangent space 𝔮/𝔮_{f} of L at f. This is

α ( \begin{matrix}A & B \\ 0 & A \end{matrix} ) = |xz^{−1}|^{2},  A = ( \begin{matrix}x & y \\ 0 & z \end{matrix} ) .  (9.6)(b)

The condition (9.3)(d) on the character γ of Q is simply γ|_{Q_{f}} = α. Such a character γ is determined by an arbitrary complex character β of ℂ^{×}, by the formula

γ ( \begin{matrix}r & ∗ & ∗ \\ 0 & S & ∗ \\ 0 & 0 & t \end{matrix} ) = β((det S)(rt)^{−1})|rt^{−1}|^{2}  (r, t ∈ ℂ^{×}, S ∈ GL(2)).  (9.6)(c)

The half-density bundle on G/Q is given by the character

ρ_{G/Q} ( \begin{matrix}r & ∗ & ∗ \\ 0 & S & ∗ \\ 0 & 0 & t \end{matrix} ) = |rt^{−1}|^{3}.  (9.7)(a)

Define γ′ = γ ⊗ ρ_{G/Q}^{−1}; then

γ′ ( \begin{matrix}r & ∗ & ∗ \\ 0 & S & ∗ \\ 0 & 0 & t \end{matrix} ) = β((det S)(rt)^{−1})|rt^{−1}|^{−1},  (9.7)(b)

and

V (f, π, L, γ, j_{γ,π}) ⊂ Ind_{Q}^{G}(γ′).  (9.7)(c)

We wish to replace Q by the associate standard parabolic subgroup Q_{1} with Levi subgroup GL(2) × GL(1) × GL(1). Define a character γ_{1} of Q_{1} by

γ_{1} ( \begin{matrix}S & ∗ & ∗ \\ 0 & r & ∗ \\ 0 & 0 & t \end{matrix} ) = β((det S)(rt)^{−1})|rt^{−1}|^{−1}.  (9.8)(a)

By standard results about parabolic induction, Ind_{Q}^{G}(γ′) and Ind_{Q_{1}}^{G}(γ_{1}) have exactly the same composition factors; and in fact they are isomorphic if β is a unitary character. Roughly speaking, therefore

V (f, π, L, γ, j_{γ,π}) ⊂ Ind_{Q_{1}}^{G}(γ_{1});  (9.8)(b)

the containment must be interpreted in terms of composition series if β is non-unitary. Now define Q_{2} to be the standard parabolic subgroup with Levi factor GL(2) × GL(2). Define a character γ_{2} of Q_{2} by

γ_{2} ( \begin{matrix}A & ∗ \\ 0 & B \end{matrix} ) = β((det A)(det B)^{−1}).  (9.8)(c)

By calculation in GL(2) and induction by stages, we have a containment

Ind_{Q_{2}}^{G}(γ_{2}) ⊂ Ind_{Q_{1}}^{G}(γ_{1}).  (9.8)(d)

The containments (9.8)(b) and (9.8)(d) suggest (but do not prove) that (at least if β is unitary)

V (f, π, L, γ, j_{γ,π}) = Ind_{Q_{2}}^{G}(γ_{2}).  (9.8)(e)

The representations on the right are obviously unitary whenever β is; so we may hope that Definition 9.4 is actually producing unitary representations in that case.

Suppose now that we repeat the entire calculation using the other Lagrangian L^{2} (from (9.5)(d)). This time the stabilizer is Q_{2}, and it turns out that the characters of Q_{2} allowed by the condition (9.3)(d) are precisely those given by (9.8)(c).

We want to draw two conclusions from the example. First, the geometric considerations of this paper (involving symplectic spinors and so on) led to certain non-unitary degenerate series representations; yet these non-unitary representations very often had interesting unitary components. Second, the non-canonical choice of Lagrangian L in Definition 9.4 may not affect the representations finally constructed as much as one might first guess.

We conclude with a few general remarks about Definition 9.4. Each element f ∈ X defines a subvariety M_{f} of M, the collection of all Lagrangians in M containing f. This is just the fiber of π over f (see (8.8)(d)). The map τ of (1.11)(c) carries M_{f} into B(𝔤/𝔤_{f}), the Lagrangian Grassmannian for the tangent space to X at f. Over B(𝔤/𝔤_{f}) we have the vector bundle V_{π}; Proposition 7.4 embeds H_{π} ⊗ H_{f}^{even,∞} as a space of smooth sections of V_{π}. Let us call these sections metaplectic.

Suppose ϕ is a section of W_{γ} on M belonging to our representation space V (f, π, L, γ, j_{γ,π}). Then the restriction of ϕ to M_{f} must be equal to the pullback (via τ and the isomorphism j_{γ,π}) of a metaplectic section of V_{π}. Said more loosely, ϕ must be metaplectic on each subvariety M_{f} of M. This condition is probably not sufficient for belonging to V (f, π, L, γ, j_{γ,π}), but it is certainly necessary; and in some sense it seems to be the main requirement.

We have concentrated almost exclusively on complex groups. For nilpotent orbits in real reductive groups, Corollary 8.7 is almost certainly still true; so most of the formalism of sections 8 and 9 can be set up. This leads to subrepresentations of degenerate principal series again. It is not entirely clear that this is the best or only way to proceed, however. Lemma 8.2 fails for elliptic semisimple orbits in the real case, and one is forced to introduce complex polarizations (and the machinery of cohomological parabolic induction) to construct unitary representations. It may be that nilpotent orbits in the real case should be treated using ideas from cohomological induction, and that at least some of the associated representations should appear inside cohomologically induced representations. We hope to return to these questions in a future paper.

References.

[1] R. Abraham and J. Marsden, Foundations of Mechanics. Benjamin/Cummings, Reading, Massachusetts, 1978.

[2] V. Arnold, Mathematical Methods of Classical Mechanics. Springer-Verlag, New York-Heidelberg-Berlin, 1978.

[3] L. Auslander and B. Kostant, “Polarization and unitary representations of solvable Lie groups,” Invent. Math. 14 (1971), 255–354.

[4] A. Borel, Linear algebraic groups. Springer-Verlag, New York, 1991.

[5] W. Casselman, “Canonical extensions of Harish-Chandra modules to representations of G,” Can. J. Math. 41 (1989), 385–438.

[6] M. Duflo, “Théorie de Mackey pour les groupes de Lie algébriques,” Acta Math. 149 (1982), 153–213.

[7] V. Ginsburg, “𝔊-modules, Springer’s representations and bivariant Chern classes,” Adv. in Math. 61 (1986), 1–48.

[8] V. Guillemin and S. Sternberg, “A generalization of the notion of polarization,” Ann. Glob. Analysis and Geometry 4 (1986), 327–347.

[9] V. Guillemin and S. Sternberg, Geometric Asymptotics, revised edition. Mathematical Surveys and Monographs 14. American Mathematical Society, Providence, Rhode Island (1990).

[10] A. Kirillov, “Unitary representations of nilpotent Lie groups,” Uspehi Mat. Nauk. 17 (1962), 57–110.

[11] B. Kostant, “Quantization and unitary representations,” in Lectures in Modern Analysis and Applications, (C. Taam, ed.), Lecture Notes in Mathematics 170. Springer-Verlag, Berlin-Heidelberg-New York, 1970.

[12] B. Kostant, “Symplectic spinors,” 139–152 in Geometria Simplettica e Fisica Matematica. Symposia Mathematica XIV, Istituto Nazionale di Alta Matematica. Academic Press, London and New York, 1974.

[13] B. Kostant, “Coadjoint orbits and a new symbol calculus for line bundles,” in Conference on Differential Geometric Methods in Theoretical Physics (G. Denardo and H.D. Doebner, eds.). World Scientific, Singapore, 1983.

[14] G. Lion and M. Vergne, The Weil representation, Maslov index, and theta series. Birkhäuser, Boston-Basel-Berlin, 1980.

[15] C. Moeglin, M.-F. Vigneras, and J.-L. Waldspurger, Correspondances de Howe sur un corps padique, Lecture Notes in Mathematics 1291. Springer-Verlag, Berlin-Heidelberg-New York, 1987.

[16] H. Ozeki and M. Wakimoto, “On polarizations of certain homogeneous spaces,” Hiroshima Math. J. 2 (1972), 445–482.

[17] L. Rothschild and J. Wolf, “Representations of semisimple Lie groups associated to nilpotent coadjoint orbits,” Ann. Sci. École Norm. Sup. (4) 7 (1974), 155–173.

[18] D. Shale, “Linear symmetries of free boson fields,” Trans. Amer. Math. Soc. 103 (1962), 149–167.

[19] N. Spaltenstein, “On the fixed point set of a unipotent element on the variety of Borel subgroups,” Topology 16 (1977), 203–204.

[20] P. Torasso, “Quantification geometrique, operateurs d’entrelacement et représentations unitaires de SL_{3}(ℝ),” Acta Math. 150 (1983), 153–242.

[21] D. Vogan, “Noncommutative algebras and unitary representations,” in The Mathematical Heritage of Hermann Weyl, R. O. Wells, Jr., ed., Proceedings of Symposia in Pure Mathematics, 48. American Mathematical Society, Providence, Rhode Island, 1988.

[22] D. Vogan, “Associated varieties and unipotent representations,” in Harmonic Analysis on Reductive Groups, W. Barker and P. Sally, eds. Birkhäuser, Boston-Basel-Berlin, 1991.

[23] A. Weil, “Sur certaines groupes d’opérateurs unitaires,” Acta Math. 111 (1964), 143–211.