text:
# Spinc-quantization and the 𝐾-multiplicities of the discrete series

## 1. Introduction and statement of the results

The purpose of this paper is to show that the ‘quantization commutes with reduction’ principle of Guillemin-Sternberg  holds for the coadjoint orbits that parametrize the discrete series of a real connected semi-simple Lie group.

### 1.1. Discrete series and $`K`$-multiplicities

Let $`G`$ be a connected, real, semisimple Lie group with finite center. By definition, the discrete series of $`G`$ is the set of isomorphism classes of irreducible, square integrable, unitary representations of $`G`$. Let $`K`$ be a maximal compact subgroup of $`G`$, and $`T`$ be a maximal torus in $`K`$. Harish-Chandra has shown that $`G`$ has a discrete series if and only if $`T`$ is a Cartan subgroup of $`G`$ . For the remainder of this paper, we may therefore assume that $`T`$ is a Cartan subgroup of $`G`$.

Let us fix some notation. We denote by $`𝔤,𝔨,𝔱`$ the Lie algebras of $`G,K,T`$, and by $`𝔤^{},𝔨^{},𝔱^{}`$ their duals. Let $`\mathrm{\Lambda }^{}𝔱^{}`$ be the set of real weights: $`\alpha \mathrm{\Lambda }^{}`$ if $`i\alpha `$ is the differential of a character of $`T`$. Let $`_c\mathrm{\Lambda }^{}`$ be respectively the set of roots for the action of $`T`$ on $`𝔨`$ and $`𝔤`$. We choose a system of positive roots $`_c^+`$ for $`_c`$. We denote by $`𝔱_+^{}`$ the corresponding Weyl chamber, and we let $`\rho _c`$ be half the sum of the elements of $`_c^+`$. The set $`\mathrm{\Lambda }_+^{}:=\mathrm{\Lambda }^{}𝔱_+^{}`$ parametrizes the unitary dual of $`K`$. For $`\mu \mathrm{\Lambda }_+^{}`$, let $`\chi __\mu ^_K`$ be the character of the irreducible $`K`$-representation with highest weight $`\mu `$.

Harish-Chandra parametrizes the discrete series by a discrete subset $`\widehat{G}_d`$ of regular elements of the Weyl chamber $`𝔱_+^{}`$ . He associates to any $`\lambda \widehat{G}_d`$ an invariant eigendistribution on $`G`$, denoted by $`\mathrm{\Theta }_\lambda `$, which is shown to be the global trace of an irreducible, square integrable, unitary representation $`_\lambda `$ of $`G`$. It is a generalized function on $`G`$, invariant by conjugation, which admits a restriction to $`K`$ denoted by $`\mathrm{\Theta }_\lambda |_K`$. The distribution $`\mathrm{\Theta }_\lambda |_K`$ corresponds to the global trace of the induced representation of $`K`$ on $`_\lambda `$. It admits a decomposition

$$\mathrm{\Theta }_\lambda |_K=\underset{\mu \mathrm{\Lambda }_+^{}}{}\text{m}_\mu (\lambda )\chi __\mu ^_K,$$

where the integers $`\text{m}_\mu (\lambda )`$ satisfy certain combinatorial identities called the Blattner formulas .

The main goal of the paper is to relate the multiplicities $`\text{m}_\mu (\lambda )`$ to the geometry of the coadjoint orbit $`G\lambda 𝔤^{}`$ as predicted by the Guillemin-Sternberg principle evoked above.

Before stating our result we recall how a representation belonging to the discrete series can be realized as the quantization of a coadjoint orbit.

### 1.2. Realisation of the discrete series

In the 60’s, Kostant and Langlands conjectured realisations of the discrete series in terms of $`\mathrm{L}^2`$ cohomology that fit into the general framework of quantization. The proof of this conjecture was given by Schmid somes years later . Let us recall the procedure for a fixed $`\lambda \widehat{G}_d`$.

The manifold $`G\lambda `$ carries several $`G`$-invariant complex structures. For convenience we work with the complex structure $`J`$ defined by the following condition: each weight $`\alpha `$ for the $`T`$-action on the tangent space $`(\text{T}_\lambda (G\lambda ),J)`$ satisfies $`(\alpha ,\lambda )>0`$.

Let $`^+`$ be the set of positive roots defined by $`\lambda `$: $`\alpha ^+(\alpha ,\lambda )>0`$. Let $`\rho `$ be half the sum of the elements of $`^+`$. The condition $`\lambda \widehat{G}_d`$ imposes that $`\lambda \rho `$ is a weight for $`T`$, so we can consider the line bundle

$$\stackrel{~}{L}:=G\times _T_{\lambda \rho }$$

over $`G\lambda G/T`$: this line bundle carries a canonical holomorphic structure. Let $`\mathrm{\Omega }^k(\stackrel{~}{L})`$ be the space of $`\stackrel{~}{L}`$-valued $`(0,k)`$ forms on $`G\lambda `$, and $`\overline{}_{\stackrel{~}{L}}:\mathrm{\Omega }^k(\stackrel{~}{L})\mathrm{\Omega }^{k+1}(\stackrel{~}{L})`$ be the Dolbeault operator. The choice of $`G`$-invariant hermitian metrics on $`G\lambda `$ and on $`\stackrel{~}{L}`$ give meaning to the formal adjoint $`\overline{}_{\stackrel{~}{L}}^{}`$ of the $`\overline{}_{\stackrel{~}{L}}`$ operator, and to the Dolbeault-Dirac operator $`\overline{}_{\stackrel{~}{L}}+\overline{}_{\stackrel{~}{L}}^{}`$.

The $`\mathrm{L}^2`$ cohomology of $`\stackrel{~}{L}`$, which we denote by $`\text{H}_{(2)}^{}(G\lambda ,\stackrel{~}{L})`$, is equal to the kernel of the differential operator $`\overline{}_{\stackrel{~}{L}}+\overline{}_{\stackrel{~}{L}}^{}`$ acting on the subspace of $`\mathrm{\Omega }^{}(\stackrel{~}{L})`$ formed by the square integrable elements.

###### Theorem 1.1.

(Schmid).

Let $`\lambda \widehat{G}_d`$.

(i) $`\text{H}_{(2)}^k(G\lambda ,\stackrel{~}{L})=0`$ if $`k\frac{dim(G/K)}{2}`$.

(ii) If $`k=\frac{dim(G/K)}{2}`$, then $`\text{H}_{(2)}^k(G\lambda ,\stackrel{~}{L})`$ is the irreducible representation $`_\lambda `$.

So, the representation $`_\lambda `$ is the quantization of the coadjoint orbit $`G\lambda `$ beeing the index of the Dolbeault-Dirac operator $`\overline{}_{\stackrel{~}{L}}+\overline{}_{\stackrel{~}{L}}^{}`$ (in the $`\mathrm{L}^2`$ sense and modulo $`(1)^{\frac{dim(G/K)}{2}}`$). In the next subsection, we briefly recall the ‘quantization commutes with reduction’ principle of Guillemin-Sternberg, and in subsection 1.4 we state our main result.

### 1.3. Quantization commutes with reduction

Let $`M`$ be a Hamiltonian $`K`$-manifold with symplectic form $`\omega `$ and moment map $`\mathrm{\Phi }:M𝔨^{}`$. The coadjoint orbits $`G\lambda `$ introduced earlier are the key examples here. Each is equipped with its Kirillov-Kostant-Souriau symplectic form $`\omega `$, and the action of $`G`$ is Hamiltonian with moment map $`G\lambda 𝔤^{}`$ equal to the inclusion. Let $`K`$ be the maximal compact Lie subgroup of $`G`$ introduced in subsection 1.2. The induced action of $`K`$ on $`G\lambda `$ is Hamiltonian, and the corresponding moment map $`\mathrm{\Phi }:G\lambda 𝔨^{}`$ is equal to the composition of the inclusion $`G\lambda 𝔤^{}`$ with the projection $`𝔤^{}𝔨^{}`$.

In the process of quantization one tries to associate a unitary representation of $`K`$ to the data $`(M,\omega ,\mathrm{\Phi })`$. In this general framework, when $`M`$ is compact and under certain integrability conditions, we associate to these data a virtual representation of $`K`$ defined as the equivariant index of a $`\mathrm{Spin}^c`$ Dirac operator: it’s the $`\mathrm{Spin}^c`$ quantization. We need two auxilliary data :

(i) A prequantum line bundle $`LM`$: it is a $`K`$-equivariant Hermitian line bundle equipped with $`K`$-invariant connection whose curvature form is $`i\omega `$.

(ii) A $`K`$-invariant almost complex structure $`J`$ on $`M`$, compatible with the symplectic structure: $`(v,w)\omega (v,Jw)`$ defines a metric.

One considers then the $`K`$-equivariant $`\mathrm{Spin}^c`$ Dirac operator $`D_L`$ corresponding to the $`\mathrm{Spin}^c`$ structure on $`M`$ defined by $`J`$, and twisted by the line bundle $`L`$ . The $`\mathrm{Spin}^c`$-quantization of $`(M,\omega ,\mathrm{\Phi })`$ is the equivariant index of the differential operator $`𝒟_L`$

$$RR^^K(M,L):=\text{Index}_M^K(𝒟_L)R(K),$$

where $`R(K)`$ is the representation ring of $`K`$. When $`K`$ is reduced to $`\{e\}`$, the $`\mathrm{Spin}^c`$-quantization of $`(M,\omega )`$ is just an integer: $`RR(M,L)`$.

A fundamental result of Marsden-Weinstein asserts that if $`\xi 𝔨^{}`$ is a regular value of the moment map $`\mathrm{\Phi }`$, the reduced space

$$M_\xi :=\mathrm{\Phi }^1(\xi )/K_\xi \mathrm{\Phi }^1(K\xi )/K$$

is an orbifold equipped with a symplectic structure $`\omega _\xi `$ (which one calls also symplectic quotient). For any dominant weight $`\mu \mathrm{\Lambda }_+^{}`$ which is a regular value of $`\mathrm{\Phi }`$,

$$L_\mu :=(L|_{\mathrm{\Phi }^1(\mu )}_\mu )/K_\mu $$

is a prequantum orbifold-line bundle over $`(M_\mu ,\omega _\mu )`$. The definition of $`\mathrm{Spin}^c`$-index carries over to the orbifold case, hence $`RR(M_\mu ,L_\mu )`$ is defined. In , this is extended further to the case of singular symplectic quotients, using partial (or shift) desingularization. So the integer $`RR(M_\mu ,L_\mu )`$ is well defined for every $`\mu \mathrm{\Lambda }_+^{}`$.

The following Theorem was conjectured by Guillemin-Sternberg  and is known as “quantization commutes with reduction” .

###### Theorem 1.2.

(Meinrenken, Meinrenken-Sjamaar). Let $`(M,\omega ,\mathrm{\Phi })`$ be a compact Hamiltonian $`K`$-manifold prequantized by $`L`$. Let $`RR^^K(M,)`$ be the equivariant Riemann-Roch character defined by means of a compatible almost complex structure on $`M`$. We have the following equality in $`R(K)`$

$$RR^^K(M,L)=\underset{\mu \mathrm{\Lambda }_+^{}}{}RR(M_\mu ,L_\mu )\chi __\mu ^_K.$$

###### Remark 1.3.

For a compact Hamiltonian $`K`$-manifold $`(M,\omega ,\mathrm{\Phi })`$, the Convexity Theorem  asserts that $`\mathrm{\Delta }:=\mathrm{\Phi }(M)𝔱_+^{}`$ is a convex rational polytope. In Theorem 1.2, we have $`RR(M_\mu ,L_\mu )=0`$ if $`\mu \mathrm{\Delta }`$.

Other proofs can be found in . For an introduction and further references see .

A natural question is to extend Theorem 1.2 to the non-compact Hamiltonian $`K`$-manifolds which admit a proper moment map. In this situation, the reduced space $`M_\xi :=\mathrm{\Phi }^1(\xi )/K_\xi `$ is compact for every $`\xi 𝔨^{}`$, so the integer $`RR(M_\mu ,L_\mu ),\mu \mathrm{\Lambda }_+^{},`$ can be defined like before.

###### Conjecture 1.4.

Let $`(M,\omega ,\mathrm{\Phi })`$ be a Hamiltonian $`K`$-manifold with proper moment map, and prequantized by $`L`$. Let $`\overline{}_L+\overline{}_L^{}`$ be the Dolbeault-Dirac operator defined by means of a $`K`$-invariant compatible almost complex structure, and $`K`$-invariant metric on $`M`$ and $`L`$. Then

$$\mathrm{L}^2\mathrm{Index}^_K\left(\overline{}_L+\overline{}_L^{}\right)=\underset{\mu \mathrm{\Lambda }_+^{}}{}RR(M_\mu ,L_\mu )\chi __\mu ^_K.$$

We present in the next subsection the central result of this paper that shows that Conjecture 1.4 is true, apart from a $`\rho _c`$-shift, for the coadjoint orbits that parametrize the discrete series.

### 1.4. The results

Consider the Hamiltonian action of $`K`$ on the coadjoint orbit $`G\lambda `$. Since $`G\lambda `$ is closed in $`𝔤^{}`$, the moment map $`\mathrm{\Phi }:G\lambda 𝔨^{}`$ is proper . Our main Theorem can be stated roughly as follows.

###### Theorem 1.5.

Let $`\text{m}_\mu (\lambda ),\mu \mathrm{\Lambda }_+^{}`$, be the $`K`$-multiplicities of the representation $`_\lambda |_K`$. For $`\mu \mathrm{\Lambda }_+^{}`$ we have:

(i) If $`\mu +\rho _c`$ is a regular value of $`\mathrm{\Phi }`$, the orbifold $`(G\lambda )_{\mu +\rho _c}:=\mathrm{\Phi }^1(\mu +\rho _c)/T`$, oriented by its symplectic form $`\omega _{\mu +\rho _c}`$, carries a $`\mathrm{Spin}^c`$ structure such that

$$\text{m}_\mu (\lambda )=𝒬\left((G\lambda )_{\mu +\rho _c}\right),$$

where the RHS is the index of the corresponding $`\mathrm{Spin}^c`$ Dirac operator on the reduced space $`(G\lambda )_{\mu +\rho _c}`$.

(ii) In general, one can define an integer $`𝒬\left((G\lambda )_{\mu +\rho _c}\right)`$, as the index of a $`\mathrm{Spin}^c`$ Dirac operator on a reduced space $`(G\lambda )_\xi `$ where $`\xi `$ is a regular value of $`\mathrm{\Phi }`$, close enough to $`\mu +\rho _c`$. We still have $`\text{m}_\mu (\lambda )=𝒬\left(M_{\mu +\rho _c}\right)`$.

Our Theorem states that the decomposition of $`\mathrm{\Theta }_\lambda |_K`$ into $`K`$-irreducible components follows the philosophy of Guillemin-Sternberg:

$$(1)\mathrm{\Theta }_\lambda |_K=\underset{\mu \mathrm{\Lambda }_+^{}}{}𝒬\left((G\lambda )_{\mu +\rho _c}\right)\chi __\mu ^_K.$$

We know from Theorem 1.1 that $`\mathrm{\Theta }_\lambda |_K=(1)^{\frac{dim(G/K)}{2}}\mathrm{L}{}_{}{}^{2}\mathrm{Index}^_K(\overline{}_{\stackrel{~}{L}}+\overline{}_{\stackrel{~}{L}}^{})`$, hence Theorem 1.5 states also that

$$(2)\mathrm{L}{}_{}{}^{2}\mathrm{Index}^_K(\overline{}_{\stackrel{~}{L}}+\overline{}_{\stackrel{~}{L}}^{})=(1)^{\frac{dim(G/K)}{2}}\underset{\mu \mathrm{\Lambda }_+^{}}{}𝒬\left((G\lambda )_{\mu +\rho _c}\right)\chi __\mu ^_K.$$

The main difference between Conjecture 1.4 and Theorem 1.5 is the $`\rho _c`$-shift and the choices of $`\mathrm{Spin}^c`$ structure on the symplectic quotients $`M_\mu `$ and $`(G\lambda )_{\mu +\rho _c}`$.

The $`\rho _c`$-shift is due to the fact that the line bundle $`\stackrel{~}{L}`$ is not a prequantum line bundle over $`(G\lambda ,\omega )`$. The difference on the choice of $`\mathrm{Spin}^c`$ structure comes from the fact that the complex structure $`J`$ on $`G\lambda `$ is not compatible with the symplectic structure (unless $`G=K`$ is compact). Hence $`J`$ does not descend to the symplectic reductions $`(G\lambda )_{\mu +\rho _c}`$ in general: the choice of the $`\mathrm{Spin}^c`$ structure on them need some care (see Propositions 4.10 and 4.11).

###### Remark 1.6.

For a Hamiltonian $`K`$-manifold $`M`$ with proper moment map $`\mathrm{\Phi }`$, the Convexity Theorem  asserts that $`\mathrm{\Delta }:=\mathrm{\Phi }(M)𝔱_+^{}`$ is a convex rational polyhedron. In Theorem 1.5, we have $`𝒬((G\lambda )_{\mu +\rho _c})=0`$ if $`\mu +\rho _c`$ does not belong to the relative interior of $`\mathrm{\Delta }`$ (see Prop. 2.4).

### 1.5. Outline of the Proof

We have to face the following difficulties:

$`[1]`$ The symplectic manifold $`G\lambda `$ is not compact.

$`[2]`$ The complex structure on $`G\lambda `$ is not compatible with the symplectic form $`\omega `$. In other words, the Kirillov-Kostant-Souriau symplectic form does not define a Kähler structure on $`G\lambda `$ unless $`G=K`$ is compact.

$`[3]`$ The line bundle $`\stackrel{~}{L}`$ is not a prequantum line bundle over $`(G\lambda ,\omega )`$. It’s what we call in the rest of this paper a $`\kappa `$-prequantum111Formally, $`\stackrel{~}{L}`$ is the tensor product of a prequantum line bundle over $`(G\lambda ,\omega )`$ with a square root of $`\kappa `$. line bundle over $`(G\lambda ,\omega ,J)`$: if $`\kappa `$ denotes the canonical line bundle of $`(G\lambda ,J)`$, the tensor product $`\stackrel{~}{L}^2\kappa ^1`$ is a prequantum line bundle over $`(G\lambda ,2\omega )`$.

The first step of the proof is to solve the difficulties $`[2]`$ and $`[3]`$ in the compact situation. In Section 2, we give a modified version of Theorem 1.2 when $`(M,\omega ,\mathrm{\Phi })`$ is a compact Hamiltonian $`K`$-manifold which is equipped with an almost complex structure $`J`$ \- not necessarily compatible with $`\omega `$ \- and a $`\kappa `$-prequantum line bundle $`\stackrel{~}{L}`$.

###### Theorem 1.7.

Let $`RR^^K(M,)`$ be the Riemann-Roch character defined by $`J`$. If the infinitesimal stabilizers for the action of $`K`$ on $`M`$ are Abelian, we have

$$(3)RR^^K(M,\stackrel{~}{L})=\epsilon \underset{\mu \mathrm{\Lambda }_+^{}}{}Q(M_{\mu +\rho _c})\chi __\mu ^_K,$$

where $`\epsilon =\pm 1`$ is the ‘quotient’ of the orientations induced by the almost complex structure, and the symplectic form.

In $`(3)`$, the integer $`Q(M_{\mu +\rho _c})`$ are computed like in Theorem 1.5 (see Def. 2.4 for a more precise definition).

In the second step of the proof, we extend $`(3)`$ to a non-compact setting. Instead of working with the $`\mathrm{L}^2`$-Index, we define in Section 3 a generalized Riemann-Roch character $`RR_\mathrm{\Phi }^^K(M,)`$ when $`(M,\omega ,\mathrm{\Phi })`$ is a Hamiltonian manifold such that the function $`\mathrm{\Phi }^2:M`$ has a compact set of critical points. For every $`K`$-vector bundle $`EM`$, the distribution $`RR_\mathrm{\Phi }^^K(M,E)`$ is defined as the index of a transversally elliptic operator on $`M`$ . When the manifold is compact, the maps $`RR_\mathrm{\Phi }^^K(M,)`$ and $`RR^^K(M,)`$ coincide.

We prove in Section 4 that Theorem 1.7 generalizes to

###### Theorem 1.8.

Let $`(M,\omega ,\mathrm{\Phi })`$ be a Hamiltonian $`K`$-manifold with proper moment map and such that the function $`\mathrm{\Phi }^2:M`$ has a compact set of critical points. If the infinitesimal stabilizers are Abelian, and under Assumption 3.6, we have

$$(4)RR_\mathrm{\Phi }^^K(M,\stackrel{~}{L})=\epsilon \underset{\mu \mathrm{\Lambda }_+^{}}{}𝒬(M_{\mu +\rho _c})\chi __\mu ^_K,$$

for every $`\kappa `$-prequantum line bundle.

In contrast to $`(3)`$, the RHS of $`(4)`$ is in general an infinite sum. Assumption 3.6 is needed to control the data on the non-compact manifold $`M`$: it asserts in particular that for any coadjoint orbit $`𝒪`$ of $`K`$, the square of the moment map $`\mathrm{\Phi }_𝒪:M\times 𝒪𝔨^{},(m,\xi )\mathrm{\Phi }(m)\xi ,`$ has a compact set of critical points.

In the final section we consider, for $`\lambda \widehat{G}_d`$, the case of the coadjoint orbit $`G\lambda `$ with the Hamiltonian $`K`$-action. The moment map $`\mathrm{\Phi }`$ is proper and the critical set of $`\mathrm{\Phi }^2`$ coincides with $`K\lambda `$, hence is compact. Thus the generalized Riemann-Roch character $`RR_\mathrm{\Phi }^^K(G\lambda ,)`$ is well defined, and we want to investigate the index $`RR_\mathrm{\Phi }^^K(M,\stackrel{~}{L})`$ for the $`\kappa `$-prequantum line bundle $`\stackrel{~}{L}:=G\times _T_{\lambda \rho }`$.

On one hand we are able to compute $`RR_\mathrm{\Phi }^^K(G\lambda ,\stackrel{~}{L})`$ explicitly in term of the holomorphic induction map $`\mathrm{Hol}__T^^K`$. Let $`𝔭`$ be the orthogonal complement of $`𝔨`$ in $`𝔤`$. It inherits a complex structure and an action of the torus $`T`$. The element $`_{}^{}𝔭R(T)`$ admits a polarized inverse $`[_{}^{}𝔭]_\lambda ^1R^{\mathrm{}}(T)`$ (see \[Section 5\]). In Subsection 5.2 we prove that

$$(5)RR_\mathrm{\Phi }^^K(G\lambda ,\stackrel{~}{L})=(1)^{\frac{dim(G/K)}{2}}\mathrm{Hol}__T^^K\left(t^{\lambda \rho _c+\rho _n}\left[_{}^{}𝔭\right]_\lambda ^1\right),$$

where $`\rho _n=\rho \rho _c`$ is half the sum of the non-compact roots. On the other hand, we show (Lemma 5.4) that the Blattner formulas can be reinterpreted through $`\mathrm{Hol}__T^^K`$ as follows:

$$(6)\mathrm{\Theta }_\lambda |_K=\mathrm{Hol}__T^^K\left(t^{\lambda \rho _c+\rho _n}\left[_{}^{}𝔭\right]_\lambda ^1\right).$$

From $`(5)`$ and $`(6)`$ we obtain

$$(7)RR_\mathrm{\Phi }^^K(G\lambda ,\stackrel{~}{L})=(1)^{\frac{dim(G/K)}{2}}\mathrm{\Theta }_\lambda |_K=\mathrm{L}{}_{}{}^{2}\mathrm{Index}^_K(\overline{}_{\stackrel{~}{L}}+\overline{}_{\stackrel{~}{L}}^{}).$$

Since in this context $`\epsilon =(1)^{\frac{dim(G/K)}{2}}`$, the Theorem follows from $`(4)`$ and $`(7)`$, provided one verifies that Assumption 3.6 holds for $`G\lambda `$. This is done in the final subsection of this paper.

Acknowledgments. I would like to thank Michèle Vergne for suggesting this problem, and helpful discussions.

Notation

Throughout the paper, $`K`$ will denote a compact, connected Lie group, and $`𝔨`$ its Lie algebra. In Sections 2, 3, and 4, we consider a $`K`$-Hamiltonian action on a manifold $`M`$. And we use there the following notation.

* $`T`$ : maximal torus of $`K`$ with Lie algebra $`𝔱`$
* $`W`$ : Weyl group of $`(K,T)`$
* $`\mathrm{\Lambda }=\mathrm{ker}(\mathrm{exp}:𝔱T)`$ : integral lattice of $`𝔱`$
* $`\mathrm{\Lambda }^{}=\mathrm{hom}(\mathrm{\Lambda },2\pi )`$ : real weight lattice
* $`𝔱_+^{},\rho _c`$ : Weyl chamber and corresponding half sum of the positive roots
* $`\mathrm{\Lambda }_+^{}=\mathrm{\Lambda }^{}𝔱_+^{}`$ : set of positive weights
* $`\chi __\mu ^_K`$ : character of the irreducible $`K`$-representation with highest weight $`\mu \mathrm{\Lambda }_+^{}`$
* $`𝕋_\beta `$ : subtorus of $`T`$ generated by $`\beta 𝔱`$
* $`M^\gamma `$ : submanifold of points fixed by $`\gamma 𝔨`$
* $`\text{T}M`$ : tangent bundle of $`M`$
* $`\text{T}_KM`$ : set of tangent vectors orthogonal to the $`K`$-orbits in $`M`$
* $`\mathrm{\Phi }`$ : moment map
* $`\stackrel{~}{L}`$ : $`\kappa `$-prequantum line bundle
* $`\stackrel{~}{}_{[\mu ]}=K\times _T_\mu `$ : $`\kappa `$-prequantum line bundle over the coadjoint orbit $`K(\mu +\rho _c)`$
* $`\text{Cr}(\mathrm{\Phi }^2)`$ : critical set of the function $`\mathrm{\Phi }^2`$
* $`\mathrm{\Delta }=\mathrm{\Phi }(M)𝔱_+^{}`$ : moment polytope
* $``$ : vector field generated by $`\mathrm{\Phi }`$
* $`\text{m}_\mu (E)`$ : multiplicity of $`RR_\mathrm{\Phi }^^K(M,E)`$ relatively to $`\mu \mathrm{\Lambda }_+^{}`$.

In the final section, we consider the particular case of the $`K`$-action on $`M:=G\lambda `$. Here $`G`$ is a connected real semi-simple Lie group with finite center admitting $`K`$ as a maximal compact subgroup, and $`T`$ as a compact Cartan subgroup.

Let us recall the definition of the holomorphic induction map $`\mathrm{Hol}__T^^K`$. Every $`\mu \mathrm{\Lambda }^{}`$ defines a 1-dimensional $`T`$-representation, denoted $`_\mu `$, where $`t=\mathrm{exp}X`$ acts by $`t^\mu :=e^{i\mu ,X}`$. We denote by $`R(K)`$ (resp. $`R(T)`$) the ring of characters of finite-dimensional $`K`$-representations (resp. $`T`$-representations). We denote $`R^{\mathrm{}}(K)`$ (resp. $`R^{\mathrm{}}(T)`$) the set of generalized characters of $`K`$ (resp. $`T`$). An element $`\chi R^{\mathrm{}}(K)`$ is of the form $`\chi =_{\mu \mathrm{\Lambda }_+^{}}\text{m}_\mu \chi __\mu ^_K`$, where $`\mu \text{m}_\mu ,\mathrm{\Lambda }_+^{}`$ has at most polynomial growth. Likewise, an element $`\chi R^{\mathrm{}}(T)`$ is of the form $`\chi =_{\mu \mathrm{\Lambda }^{}}\text{m}_\mu t^\mu `$, where $`\mu \text{m}_\mu ,\mathrm{\Lambda }^{}`$ has at most polynomial growth. We denote $`w\mu =w(\mu +\rho _c)\rho _c`$ the affine action of the Weyl group on $`\mathrm{\Lambda }^{}`$. The holomorphic induction map

$$\mathrm{Hol}__T^^K:R^{\mathrm{}}(T)R^{\mathrm{}}(K)$$

is characterized by the following properties:

i) $`\mathrm{Hol}__T^^K(t^\mu )=\chi __\mu ^_K`$ for every $`\mu \mathrm{\Lambda }_+^{}`$,

ii) $`\mathrm{Hol}__T^^K(t^{w\mu })=(1)^w\mathrm{Hol}__T^^K(t^\mu )`$ for every $`wW`$ and $`\mu \mathrm{\Lambda }^{}`$,

iii) $`\mathrm{Hol}__T^^K(t^\mu )=0`$ if $`W\mu \mathrm{\Lambda }_+^{}=\mathrm{}`$.

## 2. $`\mathrm{Spin}^c`$-quantization of compact Hamiltonian $`K`$-manifolds

In this Section we give a modified version of the ‘quantization commutes with reduction’ principle.

Let $`M`$ be a compact Hamiltonian $`K`$-manifold with symplectic form $`\omega `$ and moment map $`\mathrm{\Phi }:M𝔨^{}`$ characterized by the relation $`d\mathrm{\Phi },X=\omega (X_M,)`$, where $`X_M`$ is the vector field on $`M`$ generated by $`X𝔨`$ : $`X_M(m):=\frac{d}{dt}\mathrm{exp}(tX).m|_{t=0}`$, for $`mM`$.

Let $`J`$ be a $`K`$-invariant almost complex structure on $`M`$ which is not assumed to be compatible with the symplectic form. We denote $`RR^^K(M,)`$ the Riemann-Roch character defined by $`J`$. Let us recall the definition of this map.

Let $`EM`$ be a complex $`K`$-vector bundle. The almost complex structure on $`M`$ gives the decomposition $`\text{T}^{}M=_{i,j}^{i,j}\text{T}^{}M`$ of the bundle of differential forms. Using Hermitian structure in the tangent bundle $`\text{T}M`$ of $`M`$, and in the fibers of $`E`$, we define a Dolbeault-Dirac operator $`\overline{}_E+\overline{}_E^{}:𝒜^{0,even}(M,E)𝒜^{0,odd}(M,E)`$, where $`𝒜^{i,j}(M,E):=\mathrm{\Gamma }(M,^{i,j}\text{T}^{}M_{}E)`$ is the space of $`E`$-valued forms of type $`(i,j)`$. The Riemann-Roch character $`RR^^K(M,E)`$ is defined as the index of the elliptic operator $`\overline{}_E+\overline{}_E^{}`$:

$$RR^^K(M,E)=\text{Index}_M^K(\overline{}_E+\overline{}_E^{})R(K)$$

viewed as an element of $`R(K)`$, the character ring of $`K`$. An alternative definition goes as follows. The almost complex structure defines a canonical invariant $`\mathrm{Spin}^c`$ structure222See subsection 4.1 for a short review on the notion of $`\mathrm{Spin}^c`$ structure.. The $`\mathrm{Spin}^c`$ Dirac operator of $`M`$ with coefficient in $`E`$ has the same principal symbol as $`\sqrt{2}(\overline{}_E+\overline{}_E^{})`$ (see e.g. ), and therefore has the same equivariant index.

In the Kostant-Souriau framework, $`M`$ is prequantized if there is a $`K`$-equivariant Hermitian line bundle $`L`$ with a $`K`$-invariant Hermitian connection $`^L`$ of curvature $`i\omega `$. The line bundle $`L`$ is called a prequantum line bundle for the Hamiltonian $`K`$-manifold $`(M,\omega ,\mathrm{\Phi })`$. Recall that the data $`(^L,\mathrm{\Phi })`$ are related by the Kostant formula

(2.1)
$$^L(X)_{X_M}^L=i\mathrm{\Phi },X,X𝔨.$$

Here $`^L(X)`$ is the infinitesimal action of $`X`$ on the section of $`LM`$.

The tangent bundle $`\text{T}M`$ endowed with $`J`$ is a complex vector bundle over $`M`$, and we consider its complex dual $`\text{T}_{}^{}M:=\mathrm{hom}_{}(\text{T}M,)`$. We suppose first that the canonical line bundle $`\kappa :=det\text{T}_{}^{}M`$ admits a $`K`$-equivariant square root $`\kappa ^{1/2}`$. If $`M`$ is prequantized by $`L`$, a standard procedure in the geometric quantization literature is to tensor $`L`$ by the bundle of half-forms $`\kappa ^{1/2}`$ . We consider the index $`RR^^K(M,L\kappa ^{1/2})`$ instead of $`RR^^K(M,L)`$. In many contexts, the tensor product $`\stackrel{~}{L}=L\kappa ^{1/2}`$ has a meaning even if $`L`$ nor $`\kappa ^{1/2}`$ exist.

###### Definition 2.1.

An Hamiltonian K-manifold $`(M,\omega ,\mathrm{\Phi })`$, equipped with an almost complex structure, is $`\kappa `$-prequantized by an equivariant line bundle $`\stackrel{~}{L}`$ if $`L_{2\omega }:=\stackrel{~}{L}^2\kappa ^1`$ is a prequantum line bundle for $`(M,2\omega ,2\mathrm{\Phi })`$.

The basic examples are the regular coadjoint orbits of $`K`$. For any $`\mu \mathrm{\Lambda }_+^{}`$, consider the regular coadjoint orbit $`𝒪^{\mu +\rho _c}:=K(\mu +\rho _c)`$ with the compatible complex structure. The line bundle $`\stackrel{~}{}_{[\mu ]}=K\times _T_\mu `$ is a $`\kappa `$-prequantum line bundle over $`𝒪^{\mu +\rho _c}`$, and we have

(2.2)
$$RR^^K(𝒪^{\mu +\rho _c},\stackrel{~}{}_{[\mu ]})=\chi __\mu ^_K$$

for any $`\mu \mathrm{\Lambda }_+^{}`$.

Definition 2.1 can be rewritten in the $`\mathrm{Spin}^c`$ setting (see subsection 4.1 for a brief review on $`\mathrm{Spin}^c`$-structures). The almost complex structure induces a $`\mathrm{Spin}^c`$ structure $`P`$ with canonical line bundle $`det_{}\text{T}M=\kappa ^1`$. If $`(M,\omega ,J)`$ is $`\kappa `$-prequantized by $`\stackrel{~}{L}`$ one can twist $`P`$ by $`\stackrel{~}{L}`$, and then define a new $`\mathrm{Spin}^c`$ structure with canonical line bundle $`\kappa ^1\stackrel{~}{L}^2=L_{2\omega }`$ (see Lemma 4.2).

###### Definition 2.2.

A symplectic manifold $`(M,\omega )`$ is $`\mathrm{Spin}^c`$-prequantized if there exists a $`\mathrm{Spin}^c`$ structure with canonical line bundle $`L_{2\omega }`$ which is a prequantum line bundle on $`(M,2\omega )`$. If a compact Lie group acts on $`M`$, the $`\mathrm{Spin}^c`$-structure is required to be equivariant. Here we take the symplectic orientation on $`M`$.

When $`(M,\omega ,J)`$ is $`\kappa `$-prequantized by $`\stackrel{~}{L}`$, one wants to compute the $`K`$-multiplicities of $`RR^^K(M,\stackrel{~}{L})`$ in geometrical terms, like in Theorem 1.2.

###### Definition 2.3.

An element $`\xi 𝔨^{}`$ is a quasi-regular value of $`\mathrm{\Phi }`$ if all the $`K_\xi `$-orbits in $`\mathrm{\Phi }^1(\xi )`$ have the same dimension. A quasi-regular value is generic if the submanifold $`\mathrm{\Phi }^1(\xi )`$ is of maximal dimension.

For any quasi-regular value $`\xi 𝔨^{}`$, the reduced space $`M_\xi :=\mathrm{\Phi }^1(\xi )/K_\xi `$ is an orbifold equipped with a symplectic structure $`\omega _\xi `$. Let $`\stackrel{~}{L}`$ be a $`\kappa `$-prequantum line bundle over $`M`$, and let $`L_{2\omega }:=\stackrel{~}{L}^2\kappa ^1`$ be the corresponding prequantum line bundle for $`(M,2\omega )`$. For any dominant weight $`\mu \mathrm{\Lambda }_+^{}`$ such that $`\mu +\rho _c`$ is a quasi-regular value of $`\mathrm{\Phi }`$,

$$(L_{2\omega }|_{\mathrm{\Phi }^1(\mu +\rho _c)}_{2(\mu +\rho _c)})/T$$

is a prequantum orbifold-line bundle over $`(M_{\mu +\rho _c},2\omega _{\mu +\rho _c})`$.

The following Proposition is the main point for computing the $`K`$-multiplicities of $`RR^^K(M,\stackrel{~}{L})`$ in terms of the reduced spaces $`M_{\mu +\rho _c}:=\mathrm{\Phi }^1(\mu +\rho _c)/T`$, $`\mu \mathrm{\Lambda }_+^{}`$. It deals with the coherence of the definition of an integer valued map $`\mu \mathrm{\Lambda }_+^{}𝒬(M_{\mu +\rho _c})`$. In the next proposition we suppose that $`(M,\omega ,\mathrm{\Phi })`$ is a Hamiltonian $`K`$-manifold with proper moment map. The set $`\mathrm{\Phi }(M)𝔱_+^{}`$ is denoted by $`\mathrm{\Delta }`$. By the Convexity Theorem  it is a convex rational polyhedron, referred to as the moment polyhedron.

###### Definition-Proposition 2.4.

Let $`(M,\omega ,\mathrm{\Phi })`$ be a Hamiltonian $`K`$-manifold, with proper moment map. We denote $`\mathrm{\Delta }^o`$ the relative interior of the moment polyhedron $`\mathrm{\Delta }:=\mathrm{\Phi }(M)𝔱_+^{}`$. Let $`\stackrel{~}{L}`$ be a $`\kappa `$-prequantum line bundle relative to an almost complex structure $`J`$. Let $`\mu \mathrm{\Lambda }_+^{}`$.

* If $`\mu +\rho _c\mathrm{\Delta }^o`$, we set $`𝒬(M_{\mu +\rho _c})=0`$.
* If $`\mu +\rho _c`$ is a generic quasi-regular value of $`\mathrm{\Phi }`$, then the $`\mathrm{Spin}^c`$ prequantization defined by the data $`(J,\stackrel{~}{L})`$ induces a $`\mathrm{Spin}^c`$ prequantization on the symplectic quotient $`(M_{\mu +\rho _c},\omega _{\mu +\rho _c})`$. We denote $`𝒬(M_{\mu +\rho _c})`$ the index of the corresponding $`\mathrm{Spin}^c`$ Dirac operator.
* If $`\mu +\rho _c\mathrm{\Delta }^o`$, we take $`\xi `$ generic and quasi-regular sufficiently close to $`\mu +\rho _c`$. The reduced space $`M_\xi :=\mathrm{\Phi }^1(\xi )/T`$ inherits a $`\mathrm{Spin}^c`$-structure with canonical line bundle $`(L_{2\omega }|_{\mathrm{\Phi }^1(\xi )}_{2(\mu +\rho _c)})/T`$. The index $`𝒬(M_\xi )`$ of the corresponding $`\mathrm{Spin}^c`$ Dirac operator on $`M_\xi `$ does not depend of $`\xi `$, when $`\xi `$ is sufficiently close to $`\mu +\rho _c`$ : it is denoted $`𝒬(M_{\mu +\rho _c})`$.

When $`\xi =\mu +\rho _c`$ is a generic quasi-regular of $`\mathrm{\Phi }`$, the line bundle $`(L_{2\omega }|_{\mathrm{\Phi }^1(\xi )}_{2(\mu +\rho _c)})/T`$ is a prequantum line bundle over $`(M_{\mu +\rho _c},\mathrm{\hspace{0.17em}2}\omega _{\mu +\rho _c})`$: so the second point of this ‘definition’ is in fact a particular case of the third point. But we prefer to keep it since it outlines the main point: $`\mathrm{Spin}^c`$ prequantization is preserved under symplectic reductions.

The existence of $`\mathrm{Spin}^c`$-structures on symplectic quotient is proved in Subsection 4.2. The hard part is to show that the index $`𝒬(M_\xi )`$ does not depend of $`\xi `$, for $`\xi `$ sufficiently close to $`\mu +\rho _c`$: it is done in Subsection 4.3.

Note that Definition 2.4 becomes trivial when $`\mathrm{\Delta }^o`$ is not included in the interior of the Weyl chamber: $`𝒬(M_{\mu +\rho _c})=0`$ for all $`\mu \mathrm{\Lambda }_+^{}`$. However, in this paper we work under the assumption that the infinitesimal stabilizers for the $`K`$-action are Abelian. And that imposes $`\mathrm{\Delta }^o\mathrm{Interior}\{\mathrm{Weyl}\mathrm{chamber}\}`$ (see Lemma 4.9).

The following ‘quantization commutes with reduction’ Theorem holds for the $`\kappa `$-prequantum line bundles.

###### Theorem 2.5.

Let $`(M,\omega ,\mathrm{\Phi })`$ be a compact Hamiltonian $`K`$-manifold equipped with an almost complex structure $`J`$. Let $`\stackrel{~}{L}`$ be a $`\kappa `$-prequantum line bundle over $`M`$, and let $`RR^^K(M,)`$ be the Riemann-Roch character defined by $`J`$. If the infinitesimal stabilizers for the action of $`K`$ on $`M`$ are Abelian, we have the following equality in $`R(K)`$

(2.3)
$$RR^^K(M,\stackrel{~}{L})=\epsilon \underset{\mu \mathrm{\Lambda }_+^{}}{}Q(M_{\mu +\rho _c})\chi __\mu ^_K,$$

where $`\epsilon =\pm 1`$ is the ‘quotient’ of the orientations defined by the almost complex structure, and by the symplectic form.

Theorem 2.5 will be proved in a stronger form in Section 4.

Let us now give an example where the stabilizers for the action of $`K`$ on $`M`$ are not Abelian, and where (2.3) does not hold. Suppose that the group $`K`$ is not Abelian, so we can consider a face $`\sigma \{0\}`$ of the Weyl chamber. Let $`\rho _{c,\sigma }`$ be half the sum of the positive roots which vanish on $`\sigma `$, and consider the coadjoint orbit $`M:=K(\rho _c\rho _{c,\sigma })`$ equipped with its compatible complex structure. Since $`\rho _c\rho _{c,\sigma }`$ belongs to $`\sigma `$, the trivial line bundle $`M\times M`$ is $`\kappa `$-prequantum, and the image of the moment map $`\mathrm{\Phi }:M𝔨^{}`$ does not intersect the interior of the Weyl chamber. So $`M_{\mu +\rho _c}=\mathrm{}`$ for every $`\mu `$, thus the RHS of (2.3) is equal to zero. But the LHS of (2.3) is $`RR^^K(M,)`$ which is equal to $`1`$, the character of the trivial representation. $`\mathrm{}`$

Theorem 2.5 can be extended in two directions. First one can bypass the condition on the stabilizers by the following trick. Starting from a $`\kappa `$-prequantum line bundle $`\stackrel{~}{L}M`$, one can form the product $`M\times (K\rho _c)`$ with the coadjoint orbit through $`\rho _c`$. The Kunneth formula gives

$$RR^^K(M\times (K\rho _c),\stackrel{~}{L})=RR^^K(M,\stackrel{~}{L})RR^^K(K\rho _c,)=RR^^K(M,\stackrel{~}{L})$$

since $`RR^^K(K\rho _c,)=1`$. Now we can apply Theorem 2.5 to compute the multiplicities of $`RR^^K(M\times (K\rho _c),\stackrel{~}{L})`$ since $`\stackrel{~}{L}`$ is a $`\kappa `$-prequantum line bundle over $`M\times (K\rho _c)`$, and the stabilizers for the $`K`$-action on $`M\times (K\rho _c)`$ are Abelian. Finally we see that the multiplicity of the irreducible representation with highest weight $`\mu `$ in $`RR^^K(M,\stackrel{~}{L})`$ is equal to $`\epsilon 𝒬((M\times (K\rho _c))_{\mu +\rho _c})`$.

On the other hand, we can extend Theorem 2.5 to the $`\mathrm{Spin}^c`$ setting. It will be treated in a forthcoming paper.

## 3. Quantization of non-compact Hamiltonian $`K`$-manifolds

In this section $`(M,\omega ,\mathrm{\Phi })`$ denotes a Hamiltonian $`K`$-manifold, not necessarily compact, but with proper moment map $`\mathrm{\Phi }`$. Let $`J`$ be an almost complex structure on $`M`$, and let $`\stackrel{~}{L}`$ be a $`\kappa `$-prequantum line bundle over $`(M,\omega ,J)`$ (see Def. 2.1). From Proposition 2.4 the infinite sum

(3.4)
$$\underset{\mu \mathrm{\Lambda }_+^{}}{}𝒬(M_{\mu +\rho _c})\chi __\mu ^_K$$

is a well defined element of $`\widehat{R}(K):=\mathrm{hom}_{}(R(K),)`$.

The aim of this section is to realize this sum as the index of a transversally elliptic symbol naturally associated to the data $`(M,\mathrm{\Phi },J,\stackrel{~}{L})`$.

### 3.1. Transversally elliptic symbols

Here we give the basic definitions of the theory of transversally elliptic symbols (or operators) defined by Atiyah in . For an axiomatic treatment of the index morphism see Berline-Vergne  and for a short introduction see .

Let $`M`$ be a compact $`K`$-manifold. Let $`p:\text{T}MM`$ be the projection, and let $`(,)_M`$ be a $`K`$-invariant Riemannian metric. If $`E^0,E^1`$ are $`K`$-equivariant vector bundles over $`M`$, a $`K`$-equivariant morphism $`\sigma \mathrm{\Gamma }(\text{T}M,\mathrm{hom}(p^{}E^0,p^{}E^1))`$ is called a symbol. The subset of all $`(m,v)\text{T}M`$ where $`\sigma (m,v):E_m^0E_m^1`$ is not invertible is called the characteristic set of $`\sigma `$, and is denoted by $`\text{Char}(\sigma )`$.

Let $`\text{T}_KM`$ be the following subset of $`\text{T}M`$ :

$$\text{T}_KM=\left\{(m,v)\text{T}M,(v,X_M(m))__M=0\mathrm{for}\mathrm{all}X𝔨\right\}.$$

A symbol $`\sigma `$ is elliptic if $`\sigma `$ is invertible outside a compact subset of $`\text{T}M`$ ($`\text{Char}(\sigma )`$ is compact), and is transversally elliptic if the restriction of $`\sigma `$ to $`\text{T}_KM`$ is invertible outside a compact subset of $`\text{T}_KM`$ ($`\text{Char}(\sigma )\text{T}_KM`$ is compact). An elliptic symbol $`\sigma `$ defines an element in the equivariant $`K`$-theory of $`\text{T}M`$ with compact support, which is denoted by $`\text{K}_K(\text{T}M)`$, and the index of $`\sigma `$ is a virtual finite dimensional representation of $`K`$ .

A transversally elliptic symbol $`\sigma `$ defines an element of $`\text{K}_K(\text{T}_KM)`$, and the index of $`\sigma `$ is defined as a trace class virtual representation of $`K`$ (see  for the analytic index and  for the cohomological one). Remark that any elliptic symbol of $`\text{T}M`$ is transversally elliptic, hence we have a restriction map $`\text{K}_K(\text{T}M)\text{K}_K(\text{T}_KM)`$, and a commutative diagram

(3.5)
Using the excision property, one can easily show that the index map $`\text{Index}_𝒰^K:\text{K}_K(\text{T}_K𝒰)R^{\mathrm{}}(K)`$ is still defined when $`𝒰`$ is a $`K`$-invariant relatively compact open subset of a $`K`$-manifold (see \[section 3.1\]).

### 3.2. Thom symbol deformed by the moment map

To a $`K`$-invariant almost complex structure $`J`$ one associates the Thom symbol $`\text{Thom}__K(M,J)`$, and the corresponding Riemann-Roch character $`RR^^K`$ when $`M`$ is compact . Let us recall the definitions.

Consider a $`K`$-invariant Riemannian metric $`q`$ on $`M`$ such that $`J`$ is orthogonal relatively to $`q`$, and let $`h`$ be the Hermitian structure on $`\text{T}M`$ defined by : $`h(v,w)=q(v,w)iq(Jv,w)`$ for $`v,w\text{T}M`$. The symbol

$$\text{Thom}__K(M,J)\mathrm{\Gamma }(M,\mathrm{hom}(p^{}(_{}^{even}\text{T}M),p^{}(_{}^{odd}\text{T}M)))$$

at $`(m,v)\text{T}M`$ is equal to the Clifford map

(3.6)
$$\mathrm{Cl}_m(v):_{}^{even}\text{T}_mM_{}^{odd}\text{T}_mM,$$

where $`\mathrm{Cl}_m(v).w=vwc_h(v).w`$ for $`w_{}^{}\text{T}_xM`$. Here $`c_h(v):_{}^{}\text{T}_mM^1\text{T}_mM`$ denotes the contraction map relative to $`h`$. Since the map $`\mathrm{Cl}_m(v)`$ is invertible for all $`v0`$, the symbol $`\text{Thom}__K(M,J)`$ is elliptic when $`M`$ is compact.

The important point is that for any $`K`$-vector bundle $`E`$, $`\text{Thom}__K(M,J)p^{}E`$ corresponds to the principal symbol of the twisted $`\mathrm{Spin}^c`$ Dirac operator $`𝒟_E`$ . So, when $`M`$ is a compact manifold, the Riemann-Roch character $`RR^^K(M,):\text{K}_K(M)R(K)`$ is defined by the following relation

(3.7)
$$RR^^K(M,E)=\text{Index}_M^K\left(\text{Thom}__K(M,J)p^{}E\right).$$

Since the class of $`\text{Thom}__K(M,J)`$ in $`\text{K}_K(\text{T}M)`$ is independent of the choice of the Riemannian structure, the Riemann-Roch character $`RR^^K(M,)`$ also does not depend on this choice.

Consider now the case of a non-compact Hamiltonian $`K`$-manifold $`(M,\omega ,\mathrm{\Phi })`$. We choose a $`K`$-invariant scalar product on $`𝔨^{}`$, and we consider the function $`\mathrm{\Phi }^2:M`$. Let $``$ be the Hamiltonian vector field for $`\frac{1}{2}\mathrm{\Phi }^2`$, i.e. the contraction of the symplectic form by $``$ is equal to the $`1`$-form $`\frac{1}{2}d\mathrm{\Phi }^2`$. In fact the vector field $``$ only depends on $`\mathrm{\Phi }`$. The scalar product on $`𝔨^{}`$ gives an identification $`𝔨^{}𝔨`$, hence $`\mathrm{\Phi }`$ can be consider as a map from $`M`$ to $`𝔨`$. We have then

(3.8)
$$_m=(\mathrm{\Phi }(m))_M|_m,mM,$$

where $`(\mathrm{\Phi }(m))_M`$ is the vector field on $`M`$ generated by $`\mathrm{\Phi }(m)𝔨`$.

###### Definition 3.1.

The Thom symbol deformed by the moment map, which is denoted by $`\text{Thom}__K^\mathrm{\Phi }(M,J)`$, is defined by the relation

$$\text{Thom}__K^\mathrm{\Phi }(M,J)(m,v):=\text{Thom}__K(M,J)(m,v_m)$$

for any $`(m,v)\text{T}M`$. Likewise, any equivariant map $`S:M𝔨`$ defines a Thom symbol $`\text{Thom}__K^S(M,J)`$ deformed by the vector field $`S_M:mS(m)_M|_m`$ : $`\text{Thom}__K^S(M,J)(m,v):=\text{Thom}__K(M,J)(m,vS_M(m))`$.

Atiyah first proposed to ‘deform’ the symbol of an elliptic operator by the vector field induced by an $`S^1`$-action in order to localize its index on the fixed point submanifold, giving then another proof of the Lefschetz fixed-point theorem \[Lecture 6\]. Afterwards the idea was exploited by Vergne to give a proof of the ‘quantization commutes with reduction’ theorem in the case of an $`S^1`$-action . In , we extended this procedure for an action of a compact Lie group. Here, we use this idea to produce a transversally elliptic symbol on a non-compact manifold.

The characteristic set of $`\text{Thom}__K^\mathrm{\Phi }(M,J)`$ corresponds to $`\{(m,v)\text{T}M,v=_m\}`$, the graph of the vector field $``$. Since $``$ belongs to the set of tangent vectors to the $`K`$-orbits, we have

$`\text{Char}\left(\text{Thom}__K^\mathrm{\Phi }(M,J)\right)\text{T}_KM`$ $`=`$ $`\{(m,0)\text{T}M,_m=0\}`$
$``$ $`\{mM,d\mathrm{\Phi }_m^2=0\}.`$
Therefore the symbol $`\text{Thom}__K^\mathrm{\Phi }(M,J)`$ is transversally elliptic if and only if the set $`\text{Cr}(\mathrm{\Phi }^2)`$ of critical points of the function $`\mathrm{\Phi }^2`$ is compact.

###### Definition 3.2.

Let $`(M,\omega ,\mathrm{\Phi })`$ be a Hamiltonian $`K`$-manifold with $`\text{Cr}(\mathrm{\Phi }^2)`$ compact. For any invariant almost complex structure $`J`$, the symbol $`\text{Thom}__K^\mathrm{\Phi }(M,J)`$ is transversally elliptic. For any $`K`$-vector bundle $`EM`$, the tensor product $`\text{Thom}__K^\mathrm{\Phi }(M,J)p^{}E`$ is transversally elliptic and we denote by

$$RR_\mathrm{\Phi }^^K(M,E)R^{\mathrm{}}(K)$$

its index333Here we take a $`K`$-invariant relatively compact open subset $`𝒰`$ of $`M`$ such that $`\text{Cr}(\mathrm{\Phi }^2)𝒰`$. Then the restriction of $`\text{Thom}__K^\mathrm{\Phi }(M,J)`$ to $`𝒰`$ defines a class $`\text{Thom}__K^\mathrm{\Phi }(M,J)|_𝒰\text{K}_K(\text{T}_K𝒰)`$. Since the index map is well defined on $`𝒰`$, one sets $`RR_\mathrm{\Phi }^^K(M,E):=\text{Index}_𝒰^K(\text{Thom}__K^\mathrm{\Phi }(M,J)|_𝒰p^{}E|_𝒰)`$. A simple application of the excision property shows us that the definition does not depend on the choice of $`𝒰`$. In order to simplify our notation (when the almost complex structure is understood), we write $`RR_\mathrm{\Phi }^^K(M,E):=\text{Index}_M^K(\text{Thom}__K^\mathrm{\Phi }(M)p^{}E)`$.. In the same way, an equivariant map $`S:M𝔨`$ defines a transversally elliptic symbol $`\text{Thom}__K^S(M,J)`$ if and only if $`\{mM,S_M(m)=0\}`$ is compact. If this holds one defines the localized Riemann-Roch character $`RR_S^^K(M,E):=\text{Index}_M^K(\text{Thom}__K^S(M)p^{}E)`$.

###### Remark 3.3.

If $`M`$ is compact the symbols $`\text{Thom}__K(M,J)`$ and $`\text{Thom}__K^\mathrm{\Phi }(M,J)`$ are homotopic as elliptic symbols, thus the maps $`RR^^K(M,)`$ and $`RR_\mathrm{\Phi }^^K(M,)`$ coincide (see section 4 of ).

We end up this subsection with some technical remarks about the symbols $`\text{Thom}__K^S(M,J)`$ associated to an equivariant map $`S:M𝔨`$, and an almost complex structure.

Let $`𝒰`$ be a $`K`$-invariant open subspace of $`M`$. The restriction $`\text{Thom}__K^S(M,J)|_𝒰=\text{Thom}__K^S(𝒰,J)`$ is transversally elliptic if and only if $`\{mM,S_M(m)=0\}𝒰`$ is compact. Let $`j^{^{𝒰,𝒱}}:𝒰𝒱`$ be two $`K`$-invariant open subspaces of $`M`$, where $`j^{^{𝒰,𝒱}}`$ denotes the inclusion. If $`\{mM,S_M(m)=0\}𝒰=\{mM,S_M(m)=0\}𝒱`$ is compact, the excision property tells us that

$$j_{}^{^{𝒰,𝒱}}\left(\text{Thom}__K^S(𝒰,J)\right)=\text{Thom}__K^S(𝒱,J),$$

where $`j_{}^{^{𝒰,𝒱}}:\text{K}_K(\text{T}_K𝒰)\text{K}_K(\text{T}_K𝒱)`$ is the pushforward map (see \[Section 3\]).

###### Lemma 3.4.

1. If $`\{mM,S_M(m)=0\}𝒰`$ is compact, then the class defined by $`\text{Thom}__K^S(𝒰,J)`$ in $`\text{K}_K(\text{T}_K𝒰)`$ does not depend on the choice of a Riemannian metric.
2. Let $`S^0,S^1:M𝔨`$ be two equivariant maps. Suppose there exist an open subset $`𝒰M`$, and a vector field $`\theta `$ on $`𝒰`$ such that $`(S_M^0,\theta )__M`$ and $`(S_M^1,\theta )__M`$ are $`>0`$ outside a compact subset $`𝒦`$ of $`𝒰`$. Then, the equivariant symbols $`\text{Thom}__K^{S^1}(𝒰,J)`$ and $`\text{Thom}__K^{S^0}(𝒰,J)`$ are transversally elliptic and define the same class in $`\text{K}_K(\text{T}_K𝒰)`$.
3. Let $`J^0,J^1`$ be two almost complex structures on $`𝒰`$, and suppose that $`\{mM,S_M(m)=0\}𝒰`$ is compact. The transversally elliptic symbols $`\text{Thom}__K^S(𝒰,J^0)`$ and $`\text{Thom}__K^S(𝒰,J^1)`$ define the same class if there exists a homotopy $`J^t,t[0,1]`$ of $`K`$-equivariant almost complex structures between $`J^0`$ and $`J^1`$.

Proof. Two $`J`$-invariant Riemannian metrics $`q_0,q_1`$ are connected by $`q_t:=`$ $`(1t)q_0+tq_1`$. Hence the transversally elliptic symbols $`\text{Thom}__K^S(𝒰,J,q_0)`$ and $`\text{Thom}__K^S(𝒰,J,q_1)`$ are tied by the homotopy $`t\text{Thom}__K^S(𝒰,J,q_t)`$. The point 1. is then proved. The proof of 2. is similar to our deformation process in . Here we consider the maps $`S^t:=tS^1+(1t)S^0,t[0,1]`$, and the corresponding symbols $`\text{Thom}__K^{S^t}(𝒰,J)`$. The vector field $`\theta `$, ensures that $`\text{Char}(\text{Thom}__K^{S^t}(𝒰,J))\text{T}_K𝒰𝒦`$ is compact. Hence $`t\text{Thom}__K^{S^t}(𝒰,J)`$ defines a homotopy of transversally elliptic symbols. The proof of 3. is identical to the proof of Lemma 2.2 in . $`\mathrm{}`$

###### Corollary 3.5.

When $`\{mM,S_M(m)=0\}`$ is compact, the generalized Riemann-Roch character $`RR_S^^K(M,)`$ does not depend on the choice of a Riemannian metric. $`RR_S^^K(M,)`$ does not change either if the almost complex structure is deformed smoothly and equivariantly in a neighborhood of $`\{mM,S_M(m)=0\}`$.

In Subsections 3.3 and 3.4, we set up the technical preliminaries that are needed to compute the $`K`$-multiplicity of $`RR_\mathrm{\Phi }^^K(M,\stackrel{~}{L})`$.

In Section 4, we compute the $`K`$-multiplicity of $`RR_\mathrm{\Phi }^^K(M,\stackrel{~}{L})`$, when the moment map is is proper, in terms of the symplectic quotients $`M_{\mu +\rho _c},\mu \mathrm{\Lambda }_+^{}`$.

### 3.3. Counting the $`K`$-multiplicities

Let $`E`$ be a $`K`$-vector bundle over a Hamiltonian manifold $`(M,\omega ,\mathrm{\Phi })`$ and suppose that $`\text{Cr}(\mathrm{\Phi }^2)`$ is compact. One wants to compute the $`K`$-multiplicities of $`RR_\mathrm{\Phi }^^K(M,E)R^{\mathrm{}}(K)`$, i.e. the integers $`\text{m}_\mu (E),\mu \mathrm{\Lambda }_+^{}`$ such that

(3.9)
$$RR_\mathrm{\Phi }^^K(M,E)=\underset{\mu \mathrm{\Lambda }_+^{}}{}\text{m}_\mu (E)\chi __\mu ^_K.$$

For this purpose one use the classical ‘shifting trick’. By definition, one has $`\text{m}_\mu (E)=[RR_\mathrm{\Phi }^^K(M,E)V_\mu ^{}]^K`$, where $`V_\mu `$ is the irreducible $`K`$-representation with highest weight $`\mu `$, and $`V_\mu ^{}`$ is its dual. We know from (2.2) that the $`K`$-trace of $`V_\mu `$ is $`\chi __\mu ^_K=RR^^K(𝒪^{\stackrel{~}{\mu }},\stackrel{~}{}_{[\mu ]})`$, where

(3.10)
$$\stackrel{~}{\mu }=\mu +\rho _c.$$

Hence the $`K`$-trace of the dual $`V_\mu ^{}`$ is equal to $`RR^^K(\overline{𝒪^{\stackrel{~}{\mu }}},\stackrel{~}{}_{[\mu ]})`$, where $`\overline{𝒪^{\stackrel{~}{\mu }}}`$ is the coadjoint orbit $`𝒪^{\stackrel{~}{\mu }}`$ with opposite symplectic structure and opposite complex structure. Let $`\text{Thom}__K(\overline{𝒪^{\stackrel{~}{\mu }}})`$ be the equivariant Thom symbol on $`\overline{𝒪^{\stackrel{~}{\mu }}}`$. Then the trace of $`V_\mu ^{}`$ is equal to $`\text{Index}_{𝒪^{\stackrel{~}{\mu }}}^K(\text{Thom}__K(\overline{𝒪^{\stackrel{~}{\mu }}})\stackrel{~}{}_{[\mu ]})`$, and finally the multiplicative property of the index \[Theorem 3.5\] gives

$$\text{m}_\mu (E)=\left[\text{Index}_{M\times 𝒪^{\stackrel{~}{\mu }}}^K\left((\text{Thom}__K^\mathrm{\Phi }(M)p^{}E)(\text{Thom}__K(\overline{𝒪^{\stackrel{~}{\mu }}})\stackrel{~}{}_{[\mu ]})\right)\right]^K.$$

See , for the definition of the exterior product $`:\text{K}_K(\text{T}_KM)\times \text{K}_K(\text{T}𝒪^{\stackrel{~}{\mu }})\text{K}_K(\text{T}_K(M\times 𝒪^{\stackrel{~}{\mu }}))`$.

The moment map relative to the Hamiltonian $`K`$-action on $`M\times \overline{𝒪^{\stackrel{~}{\mu }}}`$ is

$`\mathrm{\Phi }_{\stackrel{~}{\mu }}:M\times \overline{𝒪^{\stackrel{~}{\mu }}}`$ $``$ $`𝔨^{}`$
(3.11) $`(m,\xi )`$ $``$ $`\mathrm{\Phi }(m)\xi `$
For any $`t`$, we consider the map $`\mathrm{\Phi }_{t\stackrel{~}{\mu }}:M\times \overline{𝒪^{\stackrel{~}{\mu }}}𝔨^{},\mathrm{\Phi }_{t\stackrel{~}{\mu }}(m,\xi ):=\mathrm{\Phi }(m)t\xi `$.

###### Assumption 3.6.

There exists a compact subset $`𝒦M`$, such that, for every $`t[0,1]`$, the critical set of the function $`\mathrm{\Phi }_{t\stackrel{~}{\mu }}^2:M\times 𝒪^{\stackrel{~}{\mu }}`$ is contained in $`𝒦\times 𝒪^{\stackrel{~}{\mu }}`$.

If $`M`$ satisfies Assumption 3.6 at $`\stackrel{~}{\mu }`$, one has a generalized Riemann-Roch character $`RR_{\mathrm{\Phi }_{\stackrel{~}{\mu }}}^^K(M\times \overline{𝒪^{\stackrel{~}{\mu }}},)`$ since $`\text{Cr}(\mathrm{\Phi }_{\stackrel{~}{\mu }}^2)`$ is compact.

###### Proposition 3.7.

Let $`\text{m}_\mu (E)`$ be the multiplicity of $`RR_\mathrm{\Phi }^^K(M,E)`$ relatively to the highest weight $`\mu \mathrm{\Lambda }_+^{}`$. If $`M`$ satisfies Assumption 3.6 at $`\stackrel{~}{\mu }`$, then

$$\text{m}_\mu (E)=\left[RR_{\mathrm{\Phi }_{\stackrel{~}{\mu }}}^^K(M\times \overline{𝒪^{\stackrel{~}{\mu }}},E\stackrel{~}{}_{[\mu ]})\right]^K.$$

Proof. One has to show that the transversally elliptic symbols $`\text{Thom}__K^\mathrm{\Phi }(M)\text{Thom}__K(\overline{𝒪^{\stackrel{~}{\mu }}})`$ and $`\text{Thom}__K^{\mathrm{\Phi }_{\stackrel{~}{\mu }}}(M\times \overline{𝒪^{\stackrel{~}{\mu }}})`$ define the same class in $`\text{K}_K(\text{T}_K(M\times 𝒪^{\stackrel{~}{\mu }}))`$ when $`M`$ satisfies Assumption 3.6 at $`\stackrel{~}{\mu }`$. Let $`\sigma _1,\sigma _2`$ be respectively the Thom symbols $`\text{Thom}__K(M)`$ and $`\text{Thom}__K(\overline{𝒪^{\stackrel{~}{\mu }}})`$. The symbol $`\sigma _I=\text{Thom}__K^\mathrm{\Phi }(M)\text{Thom}__K(\overline{𝒪^{\stackrel{~}{\mu }}})`$ is defined by

$$\sigma _I(m,\xi ,v,w)=\sigma _1(m,v_m)\sigma _2(\xi ,w),$$

where $`(m,v)\text{T}M`$, $`(\xi ,w)\text{T}𝒪^{\stackrel{~}{\mu }}`$, and $``$ is defined in (3.8). Let $`^t`$ be the vector field on $`M\times 𝒪^{\stackrel{~}{\mu }}`$ generated by the map $`\mathrm{\Phi }_{t\stackrel{~}{\mu }}:M\times 𝒪^{\stackrel{~}{\mu }}𝔨`$. For $`(m,\xi )M\times 𝒪^{\stackrel{~}{\mu }}`$, we have $`_{(m,\xi )}^t=(_{(m,\xi )}^{a,t},_{(m,\xi )}^{b,t})`$ where $`_{(m,\xi )}^{a,t}\text{T}_mM`$ and $`_{(m,\xi )}^{b,t}\text{T}_\xi 𝒪^{\stackrel{~}{\mu }}`$. The symbol $`\sigma _{II}=\text{Thom}__K^{\mathrm{\Phi }_{\stackrel{~}{\mu }}}(M\times \overline{𝒪^{\stackrel{~}{\mu }}})`$ is defined by

$$\sigma _{II}(m,\xi ,v,w)=\sigma _1(m,v_{m,\xi }^{a,1})\sigma _2(\xi ,w_{(m,\xi )}^{b,1}).$$

We connect $`\sigma _I`$ and $`\sigma _{II}`$ through two homotopies. First we consider the symbol $`A`$ on $`[0,1]\times \text{T}(M\times 𝒪^{\stackrel{~}{\mu }})`$ defined by

$$A(t;m,\xi ,v,w)=\sigma _1(m,v_{m,\xi }^{a,t})\sigma _2(\xi ,w_{(m,\xi )}^{b,t}),$$

for $`t[0,1]`$, and $`(m,\xi ,v,w)\text{T}(M\times 𝒪^{\stackrel{~}{\mu }})`$. We have $`\text{Char}(A)=\{(t;m,\xi ,v,w)|v=_{m,\xi }^{a,t},\mathrm{and}w=_{(m,\xi )}^{b,t}\}`$ and

$`\text{Char}(A){\displaystyle [0,1]\times \text{T}_K(M\times 𝒪^{\stackrel{~}{\mu }})}`$ $`=`$ $`\{(t;m,\xi ,0,0)|(m,\xi )\text{Cr}(\mathrm{\Phi }_{t\stackrel{~}{\mu }}^2)\}`$
$``$ $`[0,1]\times 𝒦\times 𝒪^{\stackrel{~}{\mu }},`$
where $`𝒦M`$ is the compact subset of Assumption 3.6. Thus $`A`$ defines a homotopy of transversally elliptic symbols. The restriction of $`A`$ to $`t=1`$ is equal to $`\sigma _{II}`$. The restriction of $`A`$ to $`t=0`$ defines the following transversally elliptic symbol

$$\sigma _{III}(m,\xi ,v,w)=\sigma _1(m,v_m)\sigma _2(\xi ,w_{(m,\xi )}^{b,0})$$

since $`_{m,\xi }^{a,0}=_m`$ for every $`(m,\xi )M\times 𝒪^{\stackrel{~}{\mu }}`$. Next, we consider the symbol $`B`$ on $`[0,1]\times \text{T}(M\times 𝒪^{\stackrel{~}{\mu }})`$ defined by

$$B(t;m,\xi ,v,w)=\sigma _1(m,v_m)\sigma _2(\xi ,wt_{(m,\xi )}^{b,0}).$$

We have $`\text{Char}(B)=\{(t;m,\xi ,v,w)|v=_m,\mathrm{and}w=t_{(m,\xi )}^{b,0}\}`$ and

$`\text{Char}(B){\displaystyle [0,1]\times \text{T}_K(M\times 𝒪^{\stackrel{~}{\mu }})}`$
$`\{(t;m,\xi ,v=_m,w=t_{(m,\xi )}^{b,0}),_m^2+t_{(m,\xi )}^{b,0}^2=0\}.`$
In particular $`\text{Char}(B)[0,1]\times \text{T}_K(M\times 𝒪^{\stackrel{~}{\mu }})`$ is contained in $`\{(t;m,\xi ,0,w=t_{(m,\xi )}^{b,0}),m\text{Cr}(\mathrm{\Phi }^2)\}`$ which is compact since $`\text{Cr}(\mathrm{\Phi }^2)`$ is compact. So, $`B`$ defines a homotopy of transversally elliptic symbols between $`\sigma _I=B|_{t=0}`$ and $`\sigma _{III}=B|_{t=1}`$. We have finally proved that $`\sigma _I,\sigma _{II},\sigma _{III}`$ define the same class in $`\text{K}_K(\text{T}_K(M\times 𝒪^{\stackrel{~}{\mu }}))`$. $`\mathrm{}`$

When $`E=\stackrel{~}{L}`$ is a $`\kappa `$-prequantum line bundle over $`M`$, the line bundle $`\stackrel{~}{L}\stackrel{~}{}_{[\mu ]}`$ is a $`\kappa `$-prequantum line bundle over $`M\times \overline{𝒪^{\stackrel{~}{\mu }}}`$. Therefore Proposition 3.7 shows that under Assumption 3.6 the $`K`$-multiplicities of $`RR_\mathrm{\Phi }^^K(M,\stackrel{~}{L})`$ have the form

(3.12)
$$\left[RR_\mathrm{\Phi }^^K(𝒳,\stackrel{~}{L}_𝒳)\right]^K,$$

where $`(𝒳,\omega _𝒳,\mathrm{\Phi })`$ is a Hamiltonian K-manifold with $`\text{Cr}(\mathrm{\Phi }^2)`$ compact, and $`\stackrel{~}{L}_𝒳`$ is a $`\kappa `$-prequantum line bundle over $`𝒳`$ relative to a $`K`$-invariant almost complex structure. In order to compute the quantity (3.12), we exploit in the next subsection the localization techniques developed in .

### 3.4. Localization of the map $`RR_\mathrm{\Phi }^^K`$

For a detailed account on the procedure of localization that we use here, see Sections 4 and 6 of . In this section $`(𝒳,\omega _𝒳,\mathrm{\Phi })`$ is a Hamiltonian $`K`$-manifold which is equipped with a $`K`$-invariant almost complex structure, and a $`\kappa `$-prequantum line bundle $`\stackrel{~}{L}`$. We suppose furthermore that $`\text{Cr}(\mathrm{\Phi }^2)`$ is compact. We give here a condition under which $`[RR_\mathrm{\Phi }^^K(𝒳,\stackrel{~}{L})]^K`$ depends only on the data in the neighborhood of $`\mathrm{\Phi }^1(0)`$.

For any $`\beta 𝔨`$, let $`𝒳^\beta `$ be the symplectic submanifold of points of $`𝒳`$ fixed by the torus $`𝕋_\beta `$ generated by $`\beta `$. Following Kirwan , the critical set $`\text{Cr}(\mathrm{\Phi }^2)`$ decomposes as

(3.13)
$$\text{Cr}(\mathrm{\Phi }^2)=\underset{\beta }{}C_\beta ^^K,\mathrm{with}C_\beta ^^K=K.(𝒳^\beta \mathrm{\Phi }^1(\beta )),$$

where $``$ is the subset of $`𝔱_+^{}`$ defined by $`:=\{\beta 𝔱_+^{},𝒳^\beta \mathrm{\Phi }^1(\beta )\mathrm{}\}`$. Since $`\text{Cr}(\mathrm{\Phi }^2)`$ is supposed to be compact, $``$ is finite.

For each $`\beta `$, let $`𝒰^^\beta 𝒳`$ be a $`K`$-invariant relatively compact open neighborhood of $`C_\beta ^^K`$ such that $`\overline{𝒰^^\beta }\text{Cr}(\mathrm{\Phi }^2)=C_\beta ^^K`$. The restriction of the transversally elliptic symbol $`\text{Thom}__K^\mathrm{\Phi }(𝒳)`$ to the subset $`𝒰^^\beta `$ defines $`\text{Thom}__K^\mathrm{\Phi }(𝒰^^\beta )\text{K}_K(\text{T}_K𝒰^^\beta )`$.

###### Definition 3.8.

For every $`\beta `$, we denote by $`RR_\beta ^^K(𝒳,)`$ the Riemann-Roch character localized near $`C_\beta ^^K`$, which is defined by

$$RR_\beta ^^K(𝒳,E)=\text{Index}_{𝒰^^\beta }^K\left(\text{Thom}__K^\mathrm{\Phi }(𝒰^^\beta )p^{}E|_{𝒰^^\beta }\right),$$

for every $`K`$-vector bundle $`E𝒳`$.

The excision property tells us that

(3.14)
$$RR_\mathrm{\Phi }^^K(𝒳,E)=\underset{\beta }{}RR_\beta ^^K(𝒳,E)$$

for every $`K`$-vector bundle $`E𝒳`$ (see \[Section 4\]). In particular, $`[RR_\mathrm{\Phi }^^K(𝒳,\stackrel{~}{L})]^K=_\beta [RR_\beta ^^K(𝒳,\stackrel{~}{L})]^K`$, and our main point here is to find suitable conditions under which $`[RR_\beta ^^K(𝒳,\stackrel{~}{L})]^K=0`$ for $`\beta 0`$.

Let $`\beta `$ be a non-zero element in $`𝔨`$. For every connected component $`𝒵`$ of $`𝒳^\beta `$, let $`𝒩_𝒵`$ be the normal bundle of $`𝒵`$ in $`𝒳`$. Let $`\alpha _1^𝒵,\mathrm{},\alpha _l^𝒵`$ be the real infinitesimal weights for the action of $`𝕋_\beta `$ on the fibers of $`𝒩_𝒵`$. The infinitesimal action of $`\beta `$ on $`𝒩_𝒵`$ is a linear map with trace equal to $`\sqrt{1}_i\alpha _i^𝒵,\beta `$.

###### Definition 3.9.

Let us denote by $`\text{Tr}_\beta |𝒩_𝒵|`$ the following positive number

$$\text{Tr}_\beta |𝒩_𝒵|:=\underset{i=1}{\overset{l}{}}|\alpha _i^𝒵,\beta |,$$

where $`\alpha _1^𝒵,\mathrm{},\alpha _l^𝒵`$ are the the real infinitesimal weights for the action of $`𝕋_\beta `$ on the fibers of $`𝒩_𝒵`$. For any $`𝕋_\beta `$-equivariant real vector bundle $`𝒱𝒵`$ (resp. real $`𝕋_\beta `$-equivariant real vector space $`E`$), we define in the same way $`\text{Tr}_\beta |𝒱|0`$ (resp. $`\text{Tr}_\beta |E|0`$).

###### Remark 3.10.

If $`𝒱=𝒱^1𝒱^2`$, we have $`\text{Tr}_\beta |𝒱|=\text{Tr}_\beta |𝒱^1|+\text{Tr}_\beta |𝒱^2|`$, and if $`𝒱^{}`$ is an equivariant real subbundle of $`𝒱`$, we get $`\text{Tr}_\beta |𝒱|\text{Tr}_\beta |𝒱^{}|`$. In particular one see that $`\text{Tr}_\beta |𝒩_𝒵|=\text{Tr}_\beta |\text{T}𝒳|_𝒵|`$, and then, if $`E_m\text{T}_m𝒳`$ is a $`𝕋_\beta `$-invariant real vector subspace for some $`m𝒵`$, we have $`\text{Tr}_\beta |𝒩_𝒵|\text{Tr}_\beta |E_m|`$.

The following Proposition and Corollary give us an essential condition under which the number $`[RR_\mathrm{\Phi }^^K(𝒳,\stackrel{~}{L})]^K`$ only depends on data localized in a neighborhhood of $`\mathrm{\Phi }^1(0)`$.

###### Proposition 3.11.

Let $`\stackrel{~}{L}`$ be a $`\kappa `$-prequantum line bundle over $`𝒳`$. The multiplicity of the trivial representation in $`RR_\beta ^^K(𝒳,\stackrel{~}{L})`$ is equal to zero if

(3.15)
$$\beta ^2+\frac{1}{2}\text{Tr}_\beta |𝒩_𝒵|2(\rho _c,\beta )>0$$

for every connected component $`𝒵`$ of $`𝒳^\beta `$ which intersects $`\mathrm{\Phi }^1(\beta )`$. Condition (3.15) always holds if $`\beta 𝔨\{0\}`$ is $`K`$-invariant or if $`\beta >\rho _c`$.

Since every $`\beta `$ belongs to the Weyl chamber, we have $`2(\rho _c,\beta )=\text{Tr}_\beta |𝔨/𝔱|`$, and then (3.15) can be rewritten as $`\beta ^2+\frac{1}{2}\text{Tr}_\beta |𝒩_𝒵|\text{Tr}_\beta |𝔨/𝔱|>0`$. From (3.14), we get

###### Corollary 3.12.

If condition (3.15) holds for all non-zero $`\beta `$, we have

$$\left[RR_\mathrm{\Phi }^^K(𝒳,\stackrel{~}{L})\right]^K=\left[RR_0^^K(𝒳,\stackrel{~}{L})\right]^K$$

where $`RR_0^^K(𝒳,)`$ is the Riemann-Roch character localized near $`\mathrm{\Phi }^1(0)`$ (see Definition 3.8). In particular, $`[RR_\mathrm{\Phi }^^K(𝒳,\stackrel{~}{L})]^K=0`$ if (3.15) holds for all non-zero $`\beta `$, and $`0\mathrm{Image}(\mathrm{\Phi })`$.

### 3.5. Proof of Proposition 3.11

When $`\beta 𝔨`$ is $`K`$-invariant, the scalar product $`(\rho _c,\beta )`$ vanishes and then (3.15) trivially holds. Let us show that (3.15) holds when $`\beta >\rho _c`$. Let $`𝒵`$ a connected component of $`𝒳^\beta `$ which intersects $`\mathrm{\Phi }^1(\beta )`$. Let $`m\mathrm{\Phi }^1(\beta )𝒵`$, and let $`E_m\text{T}_m𝒳`$ be the subspace spanned by $`X_𝒳(m),X𝔨`$. We have $`E_m𝔨/𝔨_m`$, where $`𝔨_m:=\{X𝔨,X_𝒳(m)=0\}`$. Since $`\mathrm{\Phi }(m)=\beta `$, and $`\mathrm{\Phi }`$ is equivariant $`𝔨_m𝔨_\beta :=\{X𝔨,[X,\beta ]=0\}`$, so $`\text{T}_m𝒳`$ contains a $`𝕋_\beta `$-equivariant subspace isomorphic to $`𝔨/𝔨_\beta `$. So we have $`\text{Tr}_\beta |𝒩_𝒵|\text{Tr}_\beta |𝔨/𝔨_\beta |=2(\rho _c,\beta )`$, and then

$`\beta ^2+{\displaystyle \frac{1}{2}}\text{Tr}_\beta |𝒩_𝒵|2(\rho _c,\beta )`$ $``$ $`\beta ^2(\rho _c,\beta )`$
$`>`$ $`0`$
since $`\beta >\rho _c`$. $`\mathrm{}`$

We prove now that condition (3.15) forces $`[RR_\beta ^^K(𝒳,\stackrel{~}{L})]^K`$ to be equal to $`0`$. Let $`\text{m}_{\beta ,\mu }(E)`$ be the $`K`$-multiplicities of the localized Riemann-Roch character $`RR_\beta ^^K(𝒳,E)`$ introduced in Definition 3.8 : $`RR_\beta ^^K(𝒳,E)=_{\mu \mathrm{\Lambda }_+^{}}\text{m}_{\beta ,\mu }(E)\chi __\mu ^_K`$. We show now that $`\text{m}_{\beta ,0}(\stackrel{~}{L})=0`$, by using the formulas of localization that we proved in  for the maps $`RR_\beta ^^K(𝒳,)`$.

First case : $`\beta `$ is a non-zero $`K`$-invariant element of $`𝔨^{}`$.

We show here the following relation for the multiplicities $`\text{m}_{\beta ,\mu }(\stackrel{~}{L})`$ :

(3.16)
$$\text{m}_{\beta ,\mu }(\stackrel{~}{L})0(\mu ,\beta )\beta ^2+\frac{1}{2}\text{Tr}_\beta |𝒩_𝒵|\mathrm{for}\mathrm{some}𝒵𝒳^\beta ,$$

in particular $`\text{m}_{\beta ,0}(\stackrel{~}{L})=0`$.

Since $`𝕋_\beta `$ belongs to the center of $`K`$, $`𝒳^\beta `$ is a symplectic $`K`$-invariant submanifold of $`𝒳`$. Let $`𝒩`$ be the normal bundle of $`𝒳^\beta `$ in $`𝒳`$. The $`K`$-invariant almost complex structure of $`𝒳`$ induces a $`K`$-invariant almost complex structure on $`𝒳^\beta `$, and a complex structure on the fibers of $`𝒩𝒳^\beta `$. Then we have a Riemann-Roch character $`RR_\beta ^^K(𝒳^\beta ,)`$ localized along $`𝒳^\beta \mathrm{\Phi }^1(\beta )`$ with the decomposition $`RR_\beta ^^K(𝒳^\beta ,F)=_𝒵RR_\beta ^^K(𝒵,F|_𝒵)`$, where the sum is taken over the connected components $`𝒵𝒳^\beta `$ which intersect $`\mathrm{\Phi }^1(\beta )`$. The torus $`𝕋_\beta `$ acts linearly on the fibers of the complex vector bundle $`𝒩`$, thus we can associate the polarized complex $`K`$-vector bundle $`𝒩^{+,\beta }`$ and $`(𝒩)^{+,\beta }`$ (see Definition 5.5 in ): for any real $`𝕋_\beta `$-weight $`\alpha `$ on $`𝒩^{+,\beta }`$, or on $`(𝒩)^{+,\beta }`$, we have

(3.17)
$$(\alpha ,\beta )>0.$$

We proved the following localization formula in Section 6.2 of  which holds in $`\widehat{R}(K)`$ for any $`K`$-vector bundle $`E`$ over $`𝒳`$ :

(3.18)
$$RR_\beta ^^K(𝒳,E)=(1)^{r_𝒩}\underset{k}{}RR_\beta ^^K(𝒳^\beta ,E|_{𝒳^\beta }det𝒩^{+,\beta }S^k((𝒩)^{+,\beta })).$$

Here $`r_𝒩`$ is the locally constant function on $`𝒳^\beta `$ equal to the complex rank of $`𝒩^{+,\beta }`$, and $`S^k()`$ is the $`k`$-th symmetric product over $``$.

Let $`i:𝔱_\beta 𝔱`$ be the inclusion of the Lie algebra of $`𝕋_\beta `$, and let $`i^{}:𝔱^{}𝔱_\beta ^{}`$ be the canonical dual map. Let us recall the basic relationship between the $`𝕋_\beta `$-weight on the fibers of a $`K`$-vector bundle $`F𝒳^\beta `$ and the $`K`$-multiplicities of $`RR_\beta ^^K(𝒳^\beta ,F)\widehat{R}(K)`$: if the irreducible representation $`V_\mu `$ occurs in $`RR_\beta ^^K(𝒳^\beta ,F)`$, then $`i^{}(\mu )`$ is a $`𝕋_\beta `$-weight on the fibers of $`F`$ (see Appendix B in ).

If one now uses (3.18), one sees that $`\text{m}_{\mu ,\beta }(\stackrel{~}{L})0`$ only if $`i^{}(\mu )`$ is a $`𝕋_\beta `$-weight on the fibers of some $`\stackrel{~}{L}|_𝒵det𝒩_𝒵^{+,\beta }S^k((𝒩_𝒵)^{+,\beta })`$. Since $`(i^{}(\mu ),\beta )=(\mu ,\beta )`$, (3.16) will be proved if one shows that each $`𝕋_\beta `$-weight $`\gamma __𝒵`$ on $`\stackrel{~}{L}|_𝒵det𝒩_𝒵^{+,\beta }S^k((𝒩_𝒵)^{+,\beta })`$ satisfies

(3.19)
$$(\gamma __𝒵,\beta )\beta ^2+\frac{1}{2}\text{Tr}_\beta |𝒩_𝒵|.$$

Let $`\alpha __𝒵`$ be the $`𝕋_\beta `$-weight on the fiber of the line bundle $`\stackrel{~}{L}|_𝒵det𝒩_𝒵^{+,\beta }`$. Since any $`𝕋_\beta `$-weight on $`S^k((𝒩)^{+,\beta })`$ satisfies (3.17), (3.19) holds if

(3.20)
$$(\alpha __𝒵,\beta )\beta ^2+\frac{1}{2}\text{Tr}_\beta |𝒩_𝒵|$$

for every $`𝒵𝒳^\beta `$ which intersects $`\mathrm{\Phi }^1(\beta )`$. Let $`L_{2\omega }`$ be the prequantum line bundle on $`(M,2\omega ,2\mathrm{\Phi })`$ such that $`\stackrel{~}{L}^2=L_{2\omega }\kappa `$ (where $`\kappa `$ is by definition equal to $`det(\text{T}_{}^{}𝒳)det(\text{T}𝒳)^1`$). We have

$$(\stackrel{~}{L}|_𝒵det(𝒩^{+,\beta }))^2=L_{_{2\omega }}|_𝒵det(\text{T}𝒳)^1|_𝒵det(𝒩^{+,\beta })^2.$$

So $`2\alpha __𝒵=\alpha _1+\alpha _2`$ where $`\alpha _1`$, $`\alpha _2`$ are respectively $`𝕋_\beta `$-weights on $`L_{_{2\omega }}|_𝒵`$ and $`det(\text{T}𝒳)^1|_𝒵det(𝒩^{+,\beta })^2`$. The Kostant formula (2.1) on $`L_{_{2\omega }}|_𝒵`$ gives $`(\alpha _1,X)=2(\beta ,X)`$ for every $`X𝔱_\beta `$, in particular

(3.21)
$$(\alpha _1,\beta )=2\beta ^2.$$

On $`𝒵`$, the complex vector bundle $`\text{T}𝒳`$ has the following decomposition, $`\text{T}𝒳|_𝒵=\text{T}𝒵𝒩^{,\beta }𝒩^{+,\beta }`$, where $`𝒩^{,\beta }`$ is the orthogonal complement of $`𝒩^{+,\beta }`$ in $`𝒩`$: every $`𝕋_\beta `$-weight $`\delta `$ on $`𝒩^{,\beta }`$ verifies $`(\delta ,\beta )<0`$. So we get the decomposition $`det(\text{T}𝒳)^1|_𝒵det(𝒩^{+,\beta })^2=det(\text{T}𝒵)det(𝒩^{,\beta })^1det(𝒩^{+,\beta }),`$ which gives

(3.22)
$$(\alpha _2,\beta )=\text{Tr}_\beta |𝒩_𝒵|$$

since $`𝕋_\beta `$ acts trivially on $`\text{T}𝒵`$. Finally (3.20) follows trivially from (3.21) and (3.22).

Second case : $`\beta `$ such that $`K_\beta K`$.

Consider the induced Hamiltonian action of $`K_\beta `$ on $`𝒳`$, with moment map $`\mathrm{\Phi }_{K_\beta }:𝒳𝔨_\beta ^{}`$. Let $`^{}`$ be the indexing set for the critical point of $`\mathrm{\Phi }_{K_\beta }^2`$ (see (3.13)). Following Definition 3.8, for each $`\beta ^{}^{}`$ we consider the $`K_\beta `$-Riemann-Roch character $`RR_\beta ^{}^{^{K_\beta }}(𝒳,)`$ localised along $`C_\beta ^{}^{^{K_\beta }}=K_\beta .(𝒳^\beta ^{}\mathrm{\Phi }_{K_\beta }^1(\beta ^{}))`$. Here $`\beta `$ is a $`K_\beta `$-invariant element of $`^{}`$ with $`C_\beta ^{^{K_\beta }}=𝒳^\beta \mathrm{\Phi }^1(\beta )`$.

Let $`\mathrm{Hol}__T^^K:R^{\mathrm{}}(T)R^{\mathrm{}}(K)`$, $`\mathrm{Hol}__T^{^{K_\beta }}:R^{\mathrm{}}(T)R^{\mathrm{}}(K_\beta )`$, and $`\mathrm{Hol}_{_{K_\beta }}^^K:R^{\mathrm{}}(K_\beta )R^{\mathrm{}}(K)`$ be the holomorphic induction maps (see Appendix B in ). Recall that $`\mathrm{Hol}__T^^K=\mathrm{Hol}_{_{K_\beta }}^^K\mathrm{Hol}__T^{^{K_\beta }}`$. The choice of a Weyl chamber determines a complex structure on the real vector space $`𝔨/𝔨_\beta `$. We denote by $`\overline{𝔨/𝔨_\beta }`$ the vector space endowed with the opposite complex structure.

The induction formula that we proved in \[Section 6\] states that

(3.23)
$$RR_\beta ^^K(𝒳,E)=\mathrm{Hol}_{_{K_\beta }}^^K\left(RR_\beta ^{^{K_\beta }}(𝒳,E)_{}^{}\overline{𝔨/𝔨_\beta }\right)$$

for every equivariant vector bundle $`E`$. Let us first write the decomposition $`RR_\beta ^{^{K_\beta }}(𝒳,\stackrel{~}{L})=_{\mu \mathrm{\Lambda }_\beta ^+}\text{m}_{\beta ,\mu }(\stackrel{~}{L})\chi __\mu ^{_{K_\beta }}`$ into irreducible characters of $`K_\beta `$. Since $`\beta `$ is $`K_\beta `$-invariant we can use the result of the First case. In particular (3.16) tells us that

(3.24)
$$\text{m}_{\beta ,\mu }(\stackrel{~}{L})0(\mu ,\beta )\beta ^2+\frac{1}{2}\text{Tr}_\beta |𝒩_𝒵|$$

for some connected component $`𝒵𝒳^\beta `$ which intersects $`\mathrm{\Phi }^1(\beta )`$.

Each irreducible character $`\chi __\mu ^{_{K_\beta }}`$ is equal to $`\mathrm{Hol}__T^{^{K_\beta }}(t^\mu )`$, so from (3.23) we get $`RR_\beta ^^K(M,\stackrel{~}{L})=\mathrm{Hol}__T^^K\left((_\mu m_{\beta ,\mu }(\stackrel{~}{L})t^\mu )\mathrm{\Pi }_{\alpha ^+(𝔨/𝔨_\beta )}(1t^\alpha )\right)`$ where $`^+(𝔨/𝔨_\beta )`$ is the set of positive $`T`$-weights on $`𝔨/𝔨_\beta `$ : so $`\alpha ,\beta >0`$ for all $`\alpha ^+(𝔨/𝔨_\beta )`$. Finally , we see that $`RR_\beta ^^K(M,\stackrel{~}{L})`$ is a sum of terms of the form $`m_{\beta ,\mu }(\stackrel{~}{L})\mathrm{Hol}__T^^K(t^{\mu \alpha _I})`$ where $`\alpha _I=_{\alpha I}\alpha `$ and $`I`$ is a subset of $`^+(𝔨/𝔨_\beta )`$. We know that $`\mathrm{Hol}__T^^K(t^\mu ^{})`$ is either $`0`$ or the character of an irreducible representation (times $`\pm 1`$) ; in particular $`\mathrm{Hol}__T^^K(t^\mu ^{})`$ is equal to $`\pm 1`$ only if $`(\mu ^{},X)0`$ for every $`X𝔱_+`$ (see Appendix B in ). So $`[RR_\beta ^^K(M,\stackrel{~}{L})]^K0`$ only if there exists a weight $`\mu `$ such that $`m_{\beta ,\mu }(\stackrel{~}{L})0`$ and that $`\mathrm{Hol}__T^^K(t^{\mu \alpha _I})=\pm 1`$. The first condition imposes $`(\mu ,\beta )\beta ^2+\frac{1}{2}\text{Tr}_\beta |𝒩_𝒵|`$ for some connected component $`𝒵𝒳^\beta `$, and the second one gives $`(\mu ,\beta )(\alpha _I,\beta )`$. Combining the two we end up with

$$\beta ^2+\frac{1}{2}\text{Tr}_\beta |𝒩_𝒵|(\alpha _I,\beta )\underset{\alpha ^+(𝔨/𝔨_\beta )}{}(\alpha ,\beta )=2(\rho _c,\beta ),$$

for some connected component $`𝒵𝒳^\beta `$ which intersects $`\mathrm{\Phi }^1(\beta )`$. This completes the proof that $`[RR_\beta ^^K(M,\stackrel{~}{L})]^K=0`$ if $`\beta ^2+\frac{1}{2}\text{Tr}_\beta |𝒩_𝒵|>2(\rho _c,\beta )`$ for every component $`𝒵𝒳^\beta `$ which intersects $`\mathrm{\Phi }^1(\beta )`$. $`\mathrm{}`$

## 4. Quantization commutes with reduction

Let $`(M,\omega ,\mathrm{\Phi })`$ be a Hamiltonian $`K`$-manifold equipped with an almost complex structure $`J`$. In this section, we assume that the moment map $`\mathrm{\Phi }`$ is proper and that the set $`\text{Cr}(\mathrm{\Phi }^2)`$ of critical points of $`\mathrm{\Phi }^2:M`$ is compact. We denote the corresponding Riemann-Roch character by $`RR_\mathrm{\Phi }^^K(M,)`$ (see Definition 3.2). Let $`\mathrm{\Delta }:=\mathrm{\Phi }(M)𝔱_+^{}`$ be the moment polyhedron.

The main result of this section is the following

###### Theorem 4.1.

Suppose that $`M`$ satisfies Assumption 3.6 at every $`\stackrel{~}{\mu }𝔱^{}`$, and that the infinitesimal stabilizers for the $`K`$-action on $`M`$ are Abelian. If $`\stackrel{~}{L}`$ is a $`\kappa `$-prequantum line bundle over $`(M,\omega ,\mathrm{\Phi },J)`$, we have

(4.25)
$$RR_\mathrm{\Phi }^^K(M,\stackrel{~}{L})=\epsilon \underset{\mu \mathrm{\Lambda }_+^{}}{}𝒬(M_{\mu +\rho _c})\chi __\mu ^_K,$$

where $`\epsilon =\pm 1`$ is the ‘quotient’ of the orientation $`o(J)`$ defined by the almost complex structure and the orientation $`o(\omega )`$ defined by the symplectic form. Here the integer $`𝒬(M_{\mu +\rho _c})`$ is computed by Proposition 2.4. In particular, $`𝒬(M_{\mu +\rho _c})=0`$ if $`\mu +\rho _c`$ does not belong to the relative interior of $`\mathrm{\Delta }`$.

The same result holds in the traditional ‘prequantum’ case. Suppose that $`M`$ satisfies Assumption 3.6 at every $`\mu 𝔱^{}`$, and that the almost complex structure $`J`$ is compatible with $`\omega `$. If $`L`$ is prequantum line bundle over $`(M,\omega ,\mathrm{\Phi })`$, we have $`RR_\mathrm{\Phi }^^K(M,L)=_{\mu \mathrm{\Lambda }_+^{}}RR(M_\mu ,L_\mu )\chi __\mu ^_K`$.

The next Lemma is the first step in computing the $`K`$-multiplicities $`\text{m}_\mu (\stackrel{~}{L})`$ of $`RR_\mathrm{\Phi }^^K(M,\stackrel{~}{L})`$. Since $`(M,\mathrm{\Phi })`$ satisfies Assumption 3.6 at every $`\stackrel{~}{\mu }`$, we know from Proposition 3.7 that $`\text{m}_\mu (\stackrel{~}{L})=[RR_{\mathrm{\Phi }_{\stackrel{~}{\mu }}}^^K(M\times \overline{𝒪^{\stackrel{~}{\mu }}},\stackrel{~}{L}\stackrel{~}{}_{[\mu ]})]^K`$ for every $`\mu \mathrm{\Lambda }_+^{}`$.

Let $`RR_0^^K(M\times \overline{𝒪^{\stackrel{~}{\mu }}},)`$ be the Riemann-Roch character localized near $`\mathrm{\Phi }_{\stackrel{~}{\mu }}^1(0)\mathrm{\Phi }^1(\mu +\rho _c)`$ (see Definition 3.8). This map is the zero map if $`\mathrm{\Phi }^1(\mu +\rho _c)=\mathrm{}`$.

###### Lemma 4.2.

Let $`\stackrel{~}{L}`$ be a $`\kappa `$-prequantum line bundle over $`M`$. Suppose that the infinitesimal stabilizers for the $`K`$-action are Abelian and that Assumption 3.6 is satisfied at $`\stackrel{~}{\mu }`$. We have then

(4.26)
$$\text{m}_\mu (\stackrel{~}{L})=\left[RR_0^^K(M\times \overline{𝒪^{\stackrel{~}{\mu }}},\stackrel{~}{L}\stackrel{~}{}_{[\mu ]})\right]^K.$$

In particular $`\text{m}_\mu (\stackrel{~}{L})=0`$ if $`\mu +\rho _c`$ does not belong to the moment polyhedron $`\mathrm{\Delta }`$.

Proof. The lemma follows from Corollary 3.12, applied to the Hamiltonian manifold $`𝒳:=M\times \overline{𝒪^{\stackrel{~}{\mu }}}`$, with moment map $`\mathrm{\Phi }_{\stackrel{~}{\mu }}`$ and $`\kappa `$-prequantum line bundle $`\stackrel{~}{L}\stackrel{~}{}_{[\mu ]}`$. Let $`\beta 0`$ such that $`𝒳^\beta \mathrm{\Phi }_{\stackrel{~}{\mu }}^1(\beta )\mathrm{}`$. Let $`𝒩`$ be the normal bundle of $`𝒳^\beta `$ in $`𝒳`$, and let $`x𝒳^\beta \mathrm{\Phi }_{\stackrel{~}{\mu }}^1(\beta )`$. From the criterion of Proposition 3.11, it is sufficient to show that

(4.27)
$$\beta ^2+\frac{1}{2}\text{Tr}_\beta |𝒩_x|2(\rho _c,\beta )>0.$$

Write $`x=(m,\xi )`$ with $`mM^\beta `$ and $`\xi (𝒪^{\stackrel{~}{\mu }})^\beta `$. We know that $`\text{Tr}_\beta |𝒩_x|=\text{Tr}_\beta |\text{T}_x𝒳|=\text{Tr}_\beta |\text{T}_mM|+\text{Tr}_\beta |\text{T}_\xi 𝒪^{\stackrel{~}{\mu }}|`$.

Since the stabilizer $`𝔨_\xi 𝔱`$ is Abelian and $`\beta 𝔨_\xi `$ we have $`𝔨_\xi 𝔨_\beta `$. Then the tangent space $`\text{T}_\xi 𝒪^{\stackrel{~}{\mu }}𝔨/𝔨_\xi `$ contains a copy of $`𝔨/𝔨_\beta `$, so $`\text{Tr}_\beta |\text{T}_\xi 𝒪^{\stackrel{~}{\mu }}|\text{Tr}_\beta |𝔨/𝔨_\beta |=2(\rho _c,\beta )`$. On the other hand, $`\text{T}_mM`$ contains the vector space $`E_m𝔨/𝔨_m`$ spanned by $`X_M(m),X𝔨`$. We have assume that the stabilizer subalgebra $`𝔨_m`$ is Abelian, and since $`\beta 𝔨_m`$, we get $`𝔨_m𝔨_\beta `$. Thus $`𝔨/𝔨_\beta E_m\text{T}_mM`$ and $`\text{Tr}_\beta |\text{T}_mM|2(\rho _c,\beta )`$. Finally (4.27) is proved since $`\frac{1}{2}(\text{Tr}_\beta |\text{T}_mM|+\text{Tr}_\beta |\text{T}_\xi 𝒪^{\stackrel{~}{\mu }}|)2(\rho _c,\beta )`$. $`\mathrm{}`$

The remaining part of this section is devoted to the proof of Theorem 4.1. Following the preceding Lemma we have to show that

(4.28)
$$\left[RR_0^^K(M\times \overline{𝒪^{\stackrel{~}{\mu }}},\stackrel{~}{L}\stackrel{~}{}_{[\mu ]})\right]^K=\epsilon 𝒬(M_{\mu +\rho _c}),$$

where $`𝒬(M_{\mu +\rho _c})`$ is defined in Proposition 2.4.

In Subsection 4.1, we recall the basic notions about $`\mathrm{Spin}^c`$-structures. The existence of induced $`\mathrm{Spin}^c`$-structures on symplectic quotient is proved in Subsection 4.2. The proof of (4.28) is settled in Subsection 4.3. We give in the same time the proof of the ‘hard part’ of Proposition 2.4: the fact that the index $`𝒬(M_\xi )`$ does not depend on $`\xi `$, for $`\xi `$ sufficiently close to $`\mu +\rho _c`$.

### 4.1. $`\mathrm{Spin}^c`$ structures and symbols

We refer to Lawson-Michelson  for background on $`\mathrm{Spin}^c`$-structures, and to Duistermaat  for a discussion of the symplectic case.

The group $`\mathrm{Spin}_n`$ is the connected double cover of the group $`\mathrm{SO}_n`$. Let $`\eta :\mathrm{Spin}_n\mathrm{SO}_n`$ be the covering map, and let $`\epsilon `$ be the element who generates the kernel. The group $`\mathrm{Spin}_n^c`$ is the quotient $`\mathrm{Spin}_n\times __2\mathrm{U}_1`$, where $`_2`$ acts by $`(\epsilon ,1)`$. There are two canonical group homomorphisms

$$\eta :\mathrm{Spin}_n^c\mathrm{SO}_n,\mathrm{Det}:\mathrm{Spin}_n^c\mathrm{U}_1.$$

Note that $`\eta ^\mathrm{c}=(\eta ,\mathrm{Det}):\mathrm{Spin}_n^c\mathrm{SO}_n\times \mathrm{U}_1`$ is a double covering map.

Let $`p:EM`$ be a oriented Euclidean vector bundle of rank $`n`$, and let $`\mathrm{P}_{\mathrm{SO}}(E)`$ be its bundle of oriented orthonormal frames. A $`\mathrm{Spin}^c`$-structure on $`E`$ is a $`\mathrm{Spin}_n^c`$ principal bundle $`\mathrm{P}_{\mathrm{Spin}^\mathrm{c}}(E)M`$, together with a $`\mathrm{Spin}^c`$-equivariant map $`\mathrm{P}_{\mathrm{Spin}^\mathrm{c}}(E)\mathrm{P}_{\mathrm{SO}}(E)`$. The line bundle

(4.29)
$$𝕃:=\mathrm{P}_{\mathrm{Spin}^\mathrm{c}}(E)\times _{\mathrm{Det}}$$

is called the canonical line bundle associated to $`\mathrm{P}_{\mathrm{Spin}^\mathrm{c}}(E)`$. We have then a double covering map444If $`P`$, $`Q`$ are principal bundle over $`M`$ respectively for the groups $`G`$ and $`H`$, we denote simply by $`P\times Q`$ their fibering product over $`M`$ which is a $`G\times H`$ principal bundle over $`M`$.

(4.30)
$$\eta _E^\mathrm{c}:\mathrm{P}_{\mathrm{Spin}^\mathrm{c}}(E)\mathrm{P}_{\mathrm{SO}}(E)\times \mathrm{P}_\mathrm{U}(𝕃),$$

where $`\mathrm{P}_\mathrm{U}(𝕃):=\mathrm{P}_{\mathrm{Spin}^\mathrm{c}}(E)\times _{\mathrm{Det}}\mathrm{U}_1`$ is the associated $`\mathrm{U}_1`$-principal bundle over $`M`$.

A $`\mathrm{Spin}^c`$-structure on a oriented Riemannian manifold is a $`\mathrm{Spin}^c`$-structure on its tangent bundle. If a group $`K`$ acts on the bundle $`E`$, preserving the orientation and the Euclidean structure, we define a $`K`$-equivariant $`\mathrm{Spin}^c`$-structure by requiring $`\mathrm{P}_{\mathrm{Spin}^\mathrm{c}}(E)`$ to be a $`K`$-equivariant principal bundle, and (4.30) to be $`(K\times \mathrm{Spin}_n^c)`$-equivariant.

Let $`\mathrm{\Delta }_{2m}`$ be the complex Spin representation of $`\mathrm{Spin}_{2m}^c`$. Recall that $`\mathrm{\Delta }_{2m}=\mathrm{\Delta }_{2m}^+\mathrm{\Delta }_{2m}^{}`$ inherits a canonical Clifford action $`𝐜:^{2m}\text{End}_{}(\mathrm{\Delta }_{2m})`$ which is $`\mathrm{Spin}_{2m}^c`$-equivariant, and which interchanges the grading : $`𝐜(v):\mathrm{\Delta }_{2m}^\pm \mathrm{\Delta }_{2m}^{}`$, for every $`v^{2m}`$. Let

(4.31)
$$𝒮(E):=\mathrm{P}_{\mathrm{Spin}^\mathrm{c}}(E)\times _{\mathrm{Spin}_{2m}^c}\mathrm{\Delta }_{2m}$$

be the spinor bundle over $`M`$, with the grading $`𝒮(E):=𝒮(E)^+𝒮(E)^{}`$. Since $`E=\mathrm{P}_{\mathrm{Spin}^\mathrm{c}}(E)\times _{\mathrm{Spin}_{2m}^c}^{2m}`$, the bundle $`p^{}𝒮(E)`$ is isomorphic to $`\mathrm{P}_{\mathrm{Spin}^\mathrm{c}}(E)\times _{\mathrm{Spin}_{2m}^c}(^{2m}\mathrm{\Delta }_{2m})`$.

Let $`\overline{E}`$ be the bundle $`E`$ with opposite orientation. A $`\mathrm{Spin}^c`$ structure on $`E`$ induces a $`\mathrm{Spin}^c`$ on $`\overline{E}`$, with the same canonical line bundle, and such that $`𝒮(\overline{E})^\pm =𝒮(E)^{}`$.

###### Definition 4.3.

Let $`\text{S-Thom}(E):p^{}𝒮(E)^+p^{}𝒮(E)^{}`$ be the symbol defined by

$`\mathrm{P}_{\mathrm{Spin}^\mathrm{c}}(E)\times _{\mathrm{Spin}_{2m}^c}(^{2m}\mathrm{\Delta }_{2m}^+)`$ $``$ $`\mathrm{P}_{\mathrm{Spin}^\mathrm{c}}(E)\times _{\mathrm{Spin}_{2m}^c}(^{2m}\mathrm{\Delta }_{2m}^{})`$
$`[p;v,w]`$ $``$ $`[p,v,𝐜(v)w].`$
When $`E`$ is the tangent bundle of a manifold $`M`$, the symbol $`\text{S-Thom}(E)`$ is denoted by $`\text{S-Thom}(M)`$. If a group $`K`$ acts equivariantly on the $`\mathrm{Spin}^c`$-structure, we denote by $`\text{S-Thom}_K(E)`$ the equivariant symbol.

The characteristic set of $`\text{S-Thom}(E)`$ is $`M\{\mathrm{zero}\mathrm{section}\mathrm{of}E\}`$, hence it defines a class in $`\text{K}(E)`$ if $`M`$ is compact (this class is a free generator of the $`\text{K}(M)`$-module $`\text{K}(E)`$ ). When $`E=\text{T}M`$, the symbol $`\text{S-Thom}(M)`$ corresponds to the principal symbol of the $`\mathrm{Spin}^c`$ Dirac operator associated to the $`\mathrm{Spin}^c`$-structure . If moreover $`M`$ is compact, the number $`𝒬(M)`$ is defined as the index of $`\text{S-Thom}(M)`$. If we change the orientation, note that $`𝒬(\overline{M})=𝒬(M)`$.

###### Remark 4.4.

It should be noted that the choice of the metric on the fibers of $`E`$ is not essential in the construction. Let $`g_0,g_1`$ be two metric on the fibers of $`E`$, and suppose that $`(E,g_0)`$ admits a $`\mathrm{Spin}^c`$-structure denoted by $`\mathrm{P}_{\mathrm{Spin}^\mathrm{c}}(E,g_0)`$. The trivial homotopy $`g_t=(1t).g_0+t.g_1`$ between the metrics, induces a homotopy between the principal bundles $`\mathrm{P}_{\mathrm{SO}}(E,g_0)`$, $`\mathrm{P}_{\mathrm{SO}}(E,g_1)`$ which can be lifted to a homotopy between $`\mathrm{P}_{\mathrm{Spin}^\mathrm{c}}(E,g_0)`$ and a $`\mathrm{Spin}^c`$-bundle over $`(E,g_1)`$. When the base $`M`$ is compact, the corresponding symbols $`\text{S-Thom}(E,g_0)`$ and $`\text{S-Thom}(E,g_1)`$ define the same class in $`\text{K}(E)`$.

These notions extend to the orbifold case. Let $`M`$ be a manifold with a locally free action of a compact Lie group $`H`$. The quotient $`𝒳:=M/H`$ is an orbifold, a space with finite quotient singularities. A $`\mathrm{Spin}^c`$ structure on $`𝒳`$ is by definition a $`H`$-equivariant $`\mathrm{Spin}^c`$ structure on the bundle $`\text{T}_HMM`$, where $`\text{T}_HM`$ is identified with the pullback of $`\text{T}𝒳`$ via the quotient map $`\pi :M𝒳`$. We define in the same way $`\text{S-Thom}(𝒳)\text{K}_{orb}(\text{T}𝒳)`$, such that $`\pi ^{}\text{S-Thom}(𝒳)=\text{S-Thom}_H(\text{T}_HM)`$. Here $`\text{K}_{orb}`$ denotes the $`K`$-theory of proper vector bundles . The pullback by $`\pi `$ induces an isomorphism $`\pi ^{}:\text{K}_{orb}(\text{T}𝒳)\text{K}_H(\text{T}_HM)`$. The number $`𝒬(𝒳)`$ is defined as the index of $`\text{S-Thom}(𝒳)`$, or equivalently as the multiplicity of the trivial representation in $`\text{Index}_M^H(\text{S-Thom}_H(\text{T}_HM))`$.

Consider now the case of a Hermitian vector bundle $`EM`$, of complex rank $`m`$. The orientation on the fibers of $`E`$ is given by the complex structure $`J`$. Let $`\mathrm{P}_\mathrm{U}(E)`$ be the bundle of unitary frames on $`E`$. We denote by $`\mathrm{i}:\mathrm{U}_m\mathrm{SO}_{2m}`$ the canonical inclusion map. We have a morphism $`\mathrm{j}:\mathrm{U}_m\mathrm{Spin}_{2m}^c`$ which makes the diagram

(4.32)
commutative . Then

(4.33)
$$\mathrm{P}_{\mathrm{Spin}^\mathrm{c}}(E):=\mathrm{Spin}_{2m}^c\times _\mathrm{j}\mathrm{P}_\mathrm{U}(E)$$

defines a $`\mathrm{Spin}^c`$-structure over $`E`$, with canonical line bundle equal to $`det_{}E`$.

###### Lemma 4.5.

Let $`M`$ be a manifold equipped with an almost complex structure $`J`$. The symbol $`\text{S-Thom}(M)`$ defined by the $`\mathrm{Spin}^c`$-structure (4.33), and the Thom symbol $`\text{Thom}(M,J)`$ defined in Section 3.2 coincide.

Proof. The Spinor bundle $`𝒮`$ is of the form $`\mathrm{P}_{\mathrm{Spin}^\mathrm{c}}(\text{T}M)\times _{\mathrm{Spin}_{2m}^c}\mathrm{\Delta }_{2m}=`$ $`\mathrm{P}_\mathrm{U}(\text{T}M)\times _{\mathrm{U}_m}\mathrm{\Delta }_{2m}`$. The map $`𝐜:^{2m}\text{End}_{}(\mathrm{\Delta }_{2m})`$, when restricted to the $`\mathrm{U}_m`$-equivariant action through $`\mathrm{j}`$, is equivalent to the Clifford map $`\mathrm{Cl}:^{2m}\text{End}_{}(^m)`$ (with the canonical action of $`\mathrm{U}_m`$ on $`^{2m}`$ and $`^m`$). Then $`𝒮=_{}\text{T}M`$ endowed with the Clifford action. $`\mathrm{}`$

###### Lemma 4.6.

Let $`P`$ be a $`\mathrm{Spin}^c`$-structure over $`M`$, with bundle of spinors $`𝒮`$, and canonical line bundle $`𝕃`$. For every Hermitian line bundle $`LM`$, there exists a unique $`\mathrm{Spin}^c`$-structure $`P_L`$ with bundle of spinors $`𝒮L`$, and canonical line bundle $`𝕃L^2`$ ($`P_L`$ is called the $`\mathrm{Spin}^c`$-structure $`P`$ twisted by $`L`$).

Proof. Take $`P_L=P\times _{\mathrm{U}_1}\mathrm{P}_\mathrm{U}(L)`$.

We finish this subsection with the following definitions. Let $`(M,o)`$ be an oriented manifold. Suppose that

* a connected compact Lie $`K`$ acts on $`M`$
* $`(M,o,K)`$ carries a $`K`$-equivariant $`\mathrm{Spin}^c`$-structure
* one has an equivariant map $`\mathrm{\Psi }:M𝔨`$.

Suppose first that $`M`$ is compact. The symbol $`\text{S-Thom}__K(M)`$ is then elliptic and defines a map

$$𝒬^^K(M,):\text{K}_K(M)R(K)$$

by the relation $`𝒬^^K(M,V):=\text{Index}_M^K(\text{S-Thom}__K(M)V)`$. Thus $`𝒬^^K(M,V)`$ is the equivariant index of the $`\mathrm{Spin}^c`$ Dirac operator on $`M`$ twisted by $`V`$.

Let $`\mathrm{\Psi }_M`$ be the equivariant vector field on $`M`$ defined by $`\mathrm{\Psi }_M(m):=\mathrm{\Psi }(m)_M|_m`$.

###### Definition 4.7.

The symbol $`\text{S-Thom}__K(M)`$ deformed by the map $`\mathrm{\Psi }`$, which is denoted by $`\text{S-Thom}__K^\mathrm{\Psi }(M)`$, is defined by the relation

$$\text{S-Thom}__K^\mathrm{\Psi }(M)(m,v):=\text{S-Thom}__K(M)(m,v\mathrm{\Psi }_M(m))$$

for any $`(m,v)\text{T}M`$. The symbol $`\text{S-Thom}__K^\mathrm{\Psi }(M)`$ is transversally elliptic if and only if $`\{mM,\mathrm{\Psi }_M(m)=0\}`$ is compact. When this holds one defines the localized map $`𝒬_\mathrm{\Psi }^^K(M,V):=\text{Index}_M^K(\text{S-Thom}__K^\mathrm{\Psi }(M)V)`$.

We finish this section with an adaptation of Lemma 9.4 and Corollary 9.5 of \[Appendix B\] to the localized map $`𝒬_\mathrm{\Psi }^^K(M,)`$. Let $`\beta 𝔱_+^{}`$ be a non-zero element in the center of the Lie algebra $`𝔨𝔨^{}`$ of $`K`$. We suppose here that the subtorus $`i:𝕋_\beta K`$, which is equal to the closure of $`\{\mathrm{exp}(t.\beta ),t\}`$, acts trivially on $`M`$. Let $`\text{m}_\mu (V),\mu \mathrm{\Lambda }_+^{}`$ be the $`K`$-multiplicities of $`𝒬_\mathrm{\Psi }^^K(M,V)`$.

###### Lemma 4.8.

If $`\text{m}_\mu (V)0`$, $`i^{}(\mu )`$ is a weight for the action of $`𝕋_\beta `$ on $`V𝕃^{\frac{1}{2}}`$. If each weight $`\alpha `$ for the action of $`𝕋_\beta `$ on $`V𝕃^{\frac{1}{2}}`$ satisfies $`(\alpha ,\beta )>0`$, then $`[𝒬_\mathrm{\Psi }^^K(M,V)]^K=0`$.

### 4.2. $`\mathrm{Spin}^c`$ structures on symplectic reductions

Let $`(M,\omega ,\mathrm{\Phi })`$ be a Hamiltonian $`K`$-manifold, such that $`\mathrm{\Phi }`$ is proper. Let $`J`$ be a $`K`$-invariant almost complex structure on $`M`$. And let $`\stackrel{~}{L}`$ be a $`\kappa `$-prequantum line bundle over $`(M,\omega ,J)`$. Since we do not impose a compatibility condition between $`J`$ and $`\omega `$, the almost complex structure does not descend to the symplectic quotients in general. Nevertheless we prove in this section that the $`\mathrm{Spin}^c`$ prequantization defined by the data $`(\stackrel{~}{L},J)`$ induces a $`\mathrm{Spin}^c`$ prequantization on the symplectic quotients $`M_{\mu +\rho _c}`$.

Let $`𝒴`$ be the subset $`\mathrm{\Phi }^1(\mathrm{interior}(𝔱_+^{}))`$. When $`𝒴\mathrm{}`$, the Principal-cross-section Theorem tells us that $`𝒴`$ is a Hamiltonian $`T`$-submanifold of $`M`$, with moment map the restriction of $`\mathrm{\Phi }`$ to $`𝒴`$ .

###### Lemma 4.9.

If the infinitesimal stabilizers for the $`K`$-action on $`M`$ are Abelian, the symplectic slice $`𝒴`$ is not empty.

Proof. There exists a unique relatively open face $`\tau `$ of the Weyl chamber $`𝔱_+^{}`$ such that $`\mathrm{\Phi }(M)\tau `$ is dense in $`\mathrm{\Phi }(M)𝔱_+^{}`$. The face $`\tau `$ is called the principal face of $`(M,\mathrm{\Phi })`$ . All points in the open face $`\tau `$ have the same connected centralizer $`K_\tau `$. The Principal-cross-section Theorem tells us that $`𝒴_\tau :=\mathrm{\Phi }^1(\tau )`$ is a Hamiltonian $`K_\tau `$-manifold, where $`[K_\tau ,K_\tau ]`$ acts trivially . Here we have assumed that the subalgebras $`𝔨_m:=\{X𝔨,X_M(m)=0\}`$, $`mM`$, are Abelian. Hence $`[𝔨_\tau ,𝔨_\tau ]𝔨_m`$ for every $`m𝒴_\tau `$, and this imposes $`[𝔨_\tau ,𝔨_\tau ]=0`$. Therefore the subgroup $`K_\tau `$ is Abelian, and this is the case only if $`\tau `$ is the interior of the Weyl chamber. $`\mathrm{}`$

For the remaining of this section, we assume that $`𝒴\mathrm{}`$, so that the relative interior $`\mathrm{\Delta }^o`$ of the moment polyhedron is a dense subset of $`\mathrm{\Phi }(𝒴)`$. On $`M`$, we have the orientation $`o(J)`$ defined by the almost complex structure and the orientation $`o(\omega )`$ defined by the symplectic form. We denote their ‘quotient’ by $`\epsilon =\pm 1`$. On the symplectic quotients we will have also two orientations, one induces by $`\omega `$, and the other induces by $`J`$, with the same ‘quotient’ $`\epsilon `$.

###### Proposition 4.10.

The almost complex structure $`J`$ induces

i) an orientation $`o(𝒴)`$ on $`𝒴`$, and

ii) a $`T`$-equivariant $`\mathrm{Spin}^c`$ structure on $`(𝒴,o(𝒴))`$ with canonical line bundle $`det_{}(\text{T}M|_𝒴)_{2\rho _c}`$.

Proof. On $`𝒴`$, we have the decomposition $`\text{T}M|_𝒴=\text{T}𝒴[𝔨/𝔱]`$, where $`[𝔨/𝔱]`$ denotes the trivial bundle $`𝒴\times 𝔨/𝔱`$ corresponding of the subspace of $`\text{T}M|_𝒴`$ formed by the vector fields generated by the infinitesimal action of $`𝔨/𝔱`$. The choice of the Weyl chamber induces a complex structure on $`𝔨/𝔱`$, and hence an orientation $`o([𝔨/𝔱])`$. This orientation can be also defined by a symplectic form of the type $`\omega _{𝔨/𝔱}(X,Y)=\xi ,[X,Y]`$, where $`\xi `$ belongs to the interior $`𝔱_+^{}`$. Let $`o(𝒴)`$ be the orientation on $`𝒴`$ defined by $`o(J)|_𝒴=o(𝒴)o([𝔨/𝔱])`$. On $`𝒴`$, we have also the orientation $`o(\omega _𝒴)`$ defined by the symplectic form $`\omega _𝒴`$. Note that if $`o(J)=\epsilon o(\omega )`$, we have also $`o(𝒴)=\epsilon o(\omega _𝒴)`$.

Let $`\mathrm{P}:=\mathrm{Spin}_{2n}^c\times _{\mathrm{U}_n}\mathrm{P}_\mathrm{U}(\text{T}M)`$ be the $`\mathrm{Spin}^c`$ structure on $`M`$ induced by $`J`$ (see (4.33)). When restricted to $`𝒴`$, $`\mathrm{P}|_𝒴`$ defines a $`\mathrm{Spin}^c`$ structure on the bundle $`\text{T}𝒴[𝔨/𝔱]`$. Let $`q`$ be a $`T`$-invariant Riemannian structure on $`\text{T}𝒴[𝔨/𝔱]`$ such that $`\text{T}𝒴`$ is orthogonal with $`[𝔨/𝔱]`$, and $`q`$ equals the Killing form on $`[𝔨/𝔱]`$. Following Remark 4.4, $`\mathrm{P}|_𝒴`$ induces a $`\mathrm{Spin}^c`$ structure $`\mathrm{P}^{}`$ on $`(\text{T}𝒴[𝔨/𝔱],q)`$, with the same canonical line bundle $`𝕃=det_{}(\text{T}M|_𝒴)`$. Since the $`\mathrm{SO}_{2k}\times \mathrm{U}_l`$-principal bundle $`\mathrm{P}_{\mathrm{SO}}(\text{T}𝒴)\times \mathrm{U}(𝔨/𝔱)`$ is a reduction555Here $`2n=dimM`$, $`2k=dim𝒴`$ and $`2l=dim(𝔨/𝔱)`$, so $`n=k+l`$. of the $`\mathrm{SO}_{2n}`$ principal bundle $`\mathrm{P}_{\mathrm{SO}}(\text{T}𝒴[𝔨/𝔱])`$, we have the commutative diagram

(4.34)
where $`\mathrm{Q}`$ is a $`(\eta ^\mathrm{c})^1(\mathrm{SO}_{2k}\times \mathrm{U}_l)\mathrm{Spin}_{2k}^c\times \mathrm{U}_l`$-principal bundle. Finally we see that $`\mathrm{Q}^{}=\mathrm{Q}/\mathrm{U}_l`$ is a $`\mathrm{Spin}^c`$ structure on $`\text{T}𝒴`$. Since $`(\mathrm{U}(𝔨/𝔱)\times \mathrm{P}_\mathrm{U}(𝕃))/\mathrm{U}_l\mathrm{P}_\mathrm{U}(𝕃_{2\rho _c})`$, the corresponding canonical line bundle is $`𝕃^{}=𝕃_{2\rho _c}`$. $`\mathrm{}`$

Let $`\mathrm{Aff}(\mathrm{\Delta })`$ be the affine subspace generated by moment polyhedron $`\mathrm{\Delta }`$, and let $`\stackrel{}{\mathrm{\Delta }}`$ be the subspace of $`𝔱^{}`$ generated by $`\{mnm,n\mathrm{\Delta }\}`$. Let $`T_\mathrm{\Delta }`$ the subtorus of $`T`$ with Lie algebra $`𝔱_\mathrm{\Delta }`$ equal to the orthogonal (for the duality) of $`\stackrel{}{\mathrm{\Delta }}`$. It is not difficult to see that $`T_\mathrm{\Delta }`$ corresponds to the connected component of the principal stabilizer for the $`T`$-action on $`𝒴`$.

Here we consider the symplectic quotient $`M_\xi :=\mathrm{\Phi }^1(\xi )/T`$ for generic quasi-regular values $`\xi \mathrm{\Delta }^o`$ (see Definition 2.3). For such $`\xi `$, the fiber $`\mathrm{\Phi }^1(\xi )`$ is a smooth submanifold of $`M`$, with a locally free action of $`T/T_\mathrm{\Delta }`$, and with a tubular neighborhood equivariantly diffeomorphic to $`\mathrm{\Phi }^1(\xi )\times \stackrel{}{\mathrm{\Delta }}`$. Recall that $`M_\xi `$ inherits a canonical symplectic form $`\omega _\xi `$.

###### Proposition 4.11.

Let $`\mu \mathrm{\Lambda }_+^{}`$ such that $`\stackrel{~}{\mu }=\mu +\rho _c`$ belongs to $`\mathrm{\Delta }`$. Let $`\stackrel{~}{L}`$ be a $`\kappa `$-prequantum line bundle. For every generic quasi-regular value $`\xi \mathrm{\Delta }^o`$, the $`\mathrm{Spin}^c`$ structures on $`𝒴`$, when twisted by $`\stackrel{~}{L}|_𝒴_\mu `$, induces a $`\mathrm{Spin}^c`$ structure on the reduced space $`M_\xi :=\mathrm{\Phi }^1(\xi )/T`$ with canonical line bundle $`(L_{2\omega }|_{\mathrm{\Phi }^1(\xi )}_{2\stackrel{~}{\mu }})/T`$. Here we have two choices for the orientations : $`o(M_\xi )`$ induced by $`o(𝒴)`$, and $`o(\omega _\xi )`$ defined by the symplectic form $`\omega _\xi `$. They are related by $`o(M_\xi )=\epsilon o(\omega _\xi )`$.

###### Remark 4.12.

The preceding Proposition will be used

i) when $`\xi =\mu +\rho _c`$ is a generic quasi-regular value of $`\mathrm{\Phi }`$: the symplectic quotient $`(M_{\mu +\rho _c},\omega _{\mu +\rho _c})`$ is then $`\mathrm{Spin}^c`$ prequantized. Or

ii) for general $`\mu +\rho _c\mathrm{\Delta }`$. One takes then $`\xi `$ generic quasi-regular close enough to $`\mu +\rho _c`$.

Proof of the Proposition. Let $`\xi \mathrm{\Delta }^o`$ be a generic quasi-regular value of $`\mathrm{\Phi }`$, and $`𝒵:=\mathrm{\Phi }^1(\xi )`$. This is a submanifold of $`𝒴`$ with a trivial action of $`T_\mathrm{\Delta }`$ and a locally free action of $`T/T_\mathrm{\Delta }`$. We denote the quotient map by $`\pi :𝒵M_\xi `$. We identify $`\pi ^{}(\text{T}M_\xi )`$ with the orthogonal complement (with respect to a Riemannian metric) of the trivial bundle $`[𝔱/𝔱_\mathrm{\Delta }]`$ formed by the vector fields generated by the infinitesimal action of $`𝔱/𝔱_\mathrm{\Delta }`$. On the other hand the tangent bundle $`\text{T}𝒴`$, when restricted to $`𝒵`$, decomposes as $`\text{T}𝒴|_𝒵=\text{T}𝒵[\stackrel{}{\mathrm{\Delta }}]`$, so we have

(4.35) $`\text{T}𝒴|_𝒵`$ $`=`$ $`\pi ^{}(\text{T}M_\xi )[𝔱/𝔱_\mathrm{\Delta }][\stackrel{}{\mathrm{\Delta }}]`$
$`=`$ $`\pi ^{}(\text{T}M_\xi )[𝔱/𝔱_\mathrm{\Delta }],`$
with the convention $`𝔱/𝔱_\mathrm{\Delta }=𝔱/𝔱_\mathrm{\Delta }i`$ and $`\stackrel{}{\mathrm{\Delta }}=𝔱/𝔱_\mathrm{\Delta }`$. Since $`𝔱/𝔱_\mathrm{\Delta }`$ is canonically oriented by the complex multiplication by $`i`$, the orientation $`o(𝒴)`$ determines an orientation $`o(M_\xi )`$ on $`\text{T}M_\xi `$ through (4.35).

Now we proceed like the proof of Proposition 4.11. Let $`\mathrm{Q}^{}`$ be the $`\mathrm{Spin}^c`$ structure on $`𝒴`$ introduced in Proposition 4.11, and let $`\mathrm{Q}^\mu `$ be $`\mathrm{Q}^{}`$ twisted by the line bundle $`\stackrel{~}{L}|_𝒴_\mu `$ : its canonical line bundle is $`det_{}(\text{T}M)|_𝒴_{2\rho _c}(\stackrel{~}{L}|_𝒴_\mu )^2=L_{2\omega }|_𝒴_{2\stackrel{~}{\mu }}`$. The $`\mathrm{SO}_{2k^{}}\times \mathrm{U}_l^{}`$-principal bundle $`\mathrm{P}_{\mathrm{SO}}(\pi ^{}(\text{T}M_\xi ))\times \mathrm{U}(𝔱/𝔱_\mathrm{\Delta })`$ is a reduction666Here $`2k=dim𝒴`$, $`2k^{}=dimM_\xi `$ and $`l^{}=dim(𝔱/𝔱_\mathrm{\Delta })`$, so $`k=k^{}+l^{}`$. of the $`\mathrm{SO}_{2k}`$ principal bundle $`\mathrm{P}_{\mathrm{SO}}(\pi ^{}(\text{T}M_\xi )[𝔱/𝔱_\mathrm{\Delta }])`$; we have the commutative diagram

(4.36)
where $`𝕃=L_{2\omega }|_𝒴_{2\stackrel{~}{\mu }}`$. Here $`\mathrm{Q}^{\prime \prime }`$ is a $`(\eta ^\mathrm{c})^1(\mathrm{SO}_{2k^{}}\times \mathrm{U}_l^{})\mathrm{Spin}_{2k^{}}^c\times \mathrm{U}_l^{}`$-principal bundle. The Kostant formula (2.1) tells us that the action of $`T_\mathrm{\Delta }`$ is trivial on $`𝕃|_𝒵`$, since $`\xi \stackrel{~}{\mu }\stackrel{}{\mathrm{\Delta }}`$. Thus the action of $`T_\mathrm{\Delta }`$ is trivial on $`\mathrm{Q}^{\prime \prime }`$. Finally we see that $`\mathrm{Q}_\xi =\mathrm{Q}^{\prime \prime }/(\mathrm{U}_l^{}\times T)`$ is a $`\mathrm{Spin}^c`$ structure on $`M_\xi `$ with canonical line bundle $`𝕃_\xi =(L_{2\omega }|_𝒵_{2\stackrel{~}{\mu }})/T`$. $`\mathrm{}`$

### 4.3. Definition of $`𝒬(M_{\mu +\rho _c})`$

First we give three different ways to define the quantity $`𝒬(M_{\mu +\rho _c})`$ for any $`\mu \mathrm{\Lambda }_+^{}`$. The compatibility of these different definitions proves Theorem 4.1 and the ‘hard part’ of Proposition 2.4 simultaneously. First of all $`𝒬(M_{\mu +\rho _c})=0`$ if $`\mu +\rho _c\mathrm{\Delta }`$.

First definition.

If $`\mu +\rho _c\mathrm{\Delta }^o`$ is a generic quasi regular value of $`\mathrm{\Phi }`$, $`M_{\mu +\rho _c}:=\mathrm{\Phi }^1(\mu +\rho _c)/T`$ is a symplectic orbifold. We know from Proposition 4.11 that $`M_{\mu +\rho _c}`$ inherits $`\mathrm{Spin}^c`$-structures, with the same canonical line bundle $`(L_{2\omega }|_{\mathrm{\Phi }^1(\stackrel{~}{\mu })}_{2\stackrel{~}{\mu }})/T`$, for the two choices of orientation $`o(M_{\mu +\rho _c})`$ and $`o(\omega _{\mu +\rho _c})`$. We denote the index of the $`\mathrm{Spin}^c`$ Dirac operator associated to the $`\mathrm{Spin}^c`$ structure on $`(M_{\mu +\rho _c},o(\omega _{\mu +\rho _c}))`$ by $`𝒬(M_{\mu +\rho _c})`$ and the index of the $`\mathrm{Spin}^c`$ Dirac operator associated to the $`\mathrm{Spin}^c`$ structure on $`(M_{\mu +\rho _c},o(M_{\mu +\rho _c}))`$ by $`𝒬(M_{\mu +\rho _c},o(M_{\mu +\rho _c}))`$. Since $`o(M_{\mu +\rho _c})=\epsilon o(\omega _{\mu +\rho _c})`$, we have $`𝒬(M_{\mu +\rho _c})=\epsilon 𝒬(M_{\mu +\rho _c},o(M_{\mu +\rho _c}))`$.

Second definition.

We can also define $`Q(M_{\mu +\rho _c})`$ by shift ‘desingularization’ as follows. If $`\mu +\rho _c\mathrm{\Delta }`$, one considers generic quasi regular values $`\xi \mathrm{\Delta }^o`$, close enough to $`\mu +\rho _c`$. Following Proposition 4.11, $`M_\xi =\mathrm{\Phi }^1(\xi )/T`$ inherits a $`\mathrm{Spin}^c`$ structure, with canonical line bundle $`(L_{2\omega }|_{\mathrm{\Phi }^1(\xi )}_{2\stackrel{~}{\mu }})/T`$. Then we set $`𝒬(M_{\mu +\rho _c}):=𝒬(M_\xi )`$, where the RHS is the index of the $`\mathrm{Spin}^c`$ Dirac operator associated to the $`\mathrm{Spin}^c`$ structure on $`(M_\xi ,o(\omega _\xi ))`$. If we take the orientation $`o(M_\xi )`$ induced by $`o(𝒴)`$ we have another index $`𝒬(M_\xi ,o(M_\xi ))=\epsilon 𝒬(M_\xi )`$. Here one has to show that these quantities does not depend on the choice of $`\xi `$ when $`\xi `$ is close enough to $`\mu +\rho _c`$. We will see that $`𝒬(M_{\mu +\rho _c})=0`$ when $`\mu +\rho _c\mathrm{\Delta }^o`$.

Third definition.

We can use the characterization of the multiplicity $`\text{m}_\mu (\stackrel{~}{L})`$ given in Lemma 4.2. The number $`𝒬(M_{\mu +\rho _c})`$ is the multiplicity of the trivial representation in $`\epsilon RR_0^^K(M\times \overline{𝒪^{\stackrel{~}{\mu }}},\stackrel{~}{L}\stackrel{~}{}_{[\mu ]})`$.

We have to show the compatibility of these definitions, that is

(4.37)
$$\mathrm{if}\mu +\rho _c\mathrm{\Delta }^o:\left[RR_0^^K(M\times \overline{𝒪^{\stackrel{~}{\mu }}},\stackrel{~}{L}\stackrel{~}{}_{[\mu ]})\right]^K=𝒬(M_\xi ,o(M_\xi ))$$

for any generic quasi regular value $`\xi \mathrm{\Delta }^o`$ close enough to $`\mu +\rho _c`$. And

(4.38)
$$\mathrm{if}\mu +\rho _c\mathrm{\Delta }^o:\left[RR_0^^K(M\times \overline{𝒪^{\stackrel{~}{\mu }}},\stackrel{~}{L}\stackrel{~}{}_{[\mu ]})\right]^K=0.$$

We have proved already (Lemma 4.2) that $`[RR_0^^K(M\times \overline{𝒪^{\stackrel{~}{\mu }}},\stackrel{~}{L}\stackrel{~}{}_{[\mu ]})]^K=0`$ if $`\mu +\rho _c\mathrm{\Delta }`$.

We work now with a fixed element $`\mu \mathrm{\Lambda }_+^{}`$ such that $`\stackrel{~}{\mu }=\mu +\rho _c`$ belongs to $`\mathrm{\Delta }`$. During the remaining part of this section, $`𝒴`$ will denote a small $`T`$-invariant open neighborhood of $`\mathrm{\Phi }^1(\mu +\rho _c)`$ in the symplectic slice $`\mathrm{\Phi }^1(\mathrm{interior}(𝔱_+^{}))`$.

We will check in Subsection 4.5, that the functions $`\mathrm{\Phi }\stackrel{~}{\mu }^2`$ and $`\mathrm{\Phi }\xi ^2`$ have compact critical set on $`𝒴`$ when $`\xi \mathrm{Aff}(\mathrm{\Delta })`$ is close enough to $`\stackrel{~}{\mu }`$. Since the manifold $`(𝒴,o(𝒴))`$ carries a $`T`$-invariant $`\mathrm{Spin}^c`$-structure, we consider the localized maps $`𝒬_{\mathrm{\Phi }\stackrel{~}{\mu }}^^T(𝒴,)`$ and $`𝒬_{\mathrm{\Phi }\xi }^^T(𝒴,)`$ (see Definition 4.7). The proof of (4.37) and (4.38) is divided into two steps. We first relate the maps $`RR_0^^K(M\times \overline{𝒪^{\stackrel{~}{\mu }}},)`$ and $`𝒬_{\mathrm{\Phi }\stackrel{~}{\mu }}^^T(𝒴,)`$ through the induction map

(4.39)
$$\mathrm{Ind}__T^^K:𝒞^{\mathrm{}}(T)𝒞^{\mathrm{}}(K)^K.$$

Here $`𝒞^{\mathrm{}}(T)`$, $`𝒞^{\mathrm{}}(K)`$ denote respectively the set of generalized functions on $`T`$ and $`K`$, and the $`K`$ invariants are taken with the conjugation action. The map $`\mathrm{Ind}__T^^K`$ is defined as follows : for $`\varphi 𝒞^{\mathrm{}}(T)`$, we have $`_K\mathrm{Ind}__T^^K(\varphi )(k)f(k)𝑑k=\frac{\text{vol}(K,dk)}{\text{vol}(T,dt)}_T\varphi (t)f|_T(t)dt`$, for every $`f𝒞^{\mathrm{}}(K)^K`$.

###### Proposition 4.13.

Let $`E`$ and $`F`$ be respectively $`K`$-equivariant complex vector bundles over $`M`$ and $`𝒪^{\stackrel{~}{\mu }}`$. We have the following equality

$$RR_0^^K(M\times \overline{𝒪^{\stackrel{~}{\mu }}},EF)=\mathrm{Ind}__T^^K\left(𝒬_{\mathrm{\Phi }\stackrel{~}{\mu }}^^T(𝒴,E|_𝒴F|_{\overline{e}})\right)$$

in $`R^{\mathrm{}}(K)`$. It gives in particular that

$$\left[RR_0^^K(M\times \overline{𝒪^{\stackrel{~}{\mu }}},EF)\right]^K=\left[𝒬_{\mathrm{\Phi }\stackrel{~}{\mu }}^^T(𝒴,E|_𝒴F|_{\overline{e}})\right]^T.$$

After we compute the map $`𝒬_{\mathrm{\Phi }\stackrel{~}{\mu }}^^T(𝒴,)`$ by making the shift $`\stackrel{~}{\mu }\xi `$.

###### Proposition 4.14.

Suppose $`\xi \mathrm{Aff}(\mathrm{\Delta })`$ is close enough to $`\stackrel{~}{\mu }`$. Then

i) the maps $`𝒬_{\mathrm{\Phi }\stackrel{~}{\mu }}^^T(𝒴,)`$ and $`𝒬_{\mathrm{\Phi }\xi }^^T(𝒴,)`$ are equal,

ii) if furthermore $`\xi \mathrm{\Delta }^o`$ is a generic quasi-regular value of $`\mathrm{\Phi }`$ we get

$$\left[𝒬_{\mathrm{\Phi }\xi }^^T(𝒴,\stackrel{~}{L}|_𝒴_\mu )\right]^T=𝒬(M_\xi ,o(M_\xi )),$$

iii) and if $`\xi \mathrm{\Delta }`$, $`[𝒬_{\mathrm{\Phi }\xi }^^T(𝒴,\stackrel{~}{L}|_𝒴_\mu )]^T=0`$.

Finally, if $`\xi \mathrm{Aff}(\mathrm{\Delta })`$ is close enough to $`\stackrel{~}{\mu }`$, Propositions 4.13 and 4.14 give

$`\left[RR_0^^K(M\times \overline{𝒪^{\stackrel{~}{\mu }}},\stackrel{~}{L}\stackrel{~}{}_{[\mu ]})\right]^K`$ $`=`$ $`\left[𝒬_{\mathrm{\Phi }\stackrel{~}{\mu }}^^T(𝒴,\stackrel{~}{L}|_𝒴_\mu )\right]^T`$
$`=`$ $`\left[𝒬_{\mathrm{\Phi }\xi }^^T(𝒴,\stackrel{~}{L}|_𝒴_\mu )\right]^T.`$
If $`\mu +\rho _c\mathrm{\Delta }^o`$, we choose $`\xi \mathrm{\Delta }^o`$ close to $`\mu +\rho _c`$: in this case (4.37) follows from (4.3) and the point $`ii)`$ of Proposition 4.14. If $`\mu +\rho _c\mathrm{\Delta }^o`$, we choose $`\xi `$ close to $`\mu +\rho _c`$ and not in $`\mathrm{\Delta }`$: in this case (4.38) follows from (4.3) and the point $`iii)`$ of Proposition 4.14.

Propositions 4.13 and 4.14 are proved in the next subsections.

### 4.4. Proof of Proposition 4.13

The induction formula of Proposition 4.13 is essentially identical to the one we proved in . The main difference is that the almost complex structure is not assumed to be compatible with the symplectic structure.

We identify the coadjoint orbit $`𝒪^{\stackrel{~}{\mu }}`$ with $`K/T`$. Let $`^{\stackrel{~}{\mu }}`$ be the Hamiltonian vector field of the function $`\frac{1}{2}\mathrm{\Phi }_{\stackrel{~}{\mu }}^2:M\times \overline{K/T}`$. Here $`𝒴`$ denotes a small neighborhood of $`\mathrm{\Phi }^1(\stackrel{~}{\mu })`$ in the symplectic slice $`\mathrm{\Phi }^1(\mathrm{interior}(𝔱_+^{}))`$ such that the open subset $`𝒰:=(K\times _T𝒴)\times \overline{K/T}`$ is a neighborhood of $`\mathrm{\Phi }_{\stackrel{~}{\mu }}^1(0)=K(\mathrm{\Phi }^1(\stackrel{~}{\mu })\times \{\overline{e}\})`$ which verifies $`\overline{𝒰}\{^{\stackrel{~}{\mu }}=0\}=\mathrm{\Phi }_{\stackrel{~}{\mu }}^1(0)`$.

From Definition 3.8, the localized Riemann-Roch character $`RR_0^^K(M\times \overline{K/T},)`$ is computed by means of the Thom class $`\text{Thom}_K^{\mathrm{\Phi }_{\stackrel{~}{\mu }}}(𝒰)\text{K}_K(\text{T}_K𝒰)`$. On the other hand, the localized map $`𝒬_{\mathrm{\Phi }\stackrel{~}{\mu }}^^T(𝒴,)`$ is computed by means of the class $`\text{S-Thom}_T^{\mathrm{\Phi }\stackrel{~}{\mu }}(𝒴)\text{K}_T(\text{T}_T𝒴)`$ (see Definition 4.7). Proposition 4.13 will follow from a simple relation between these two transversally elliptic symbols.

First, we consider the isomorphism $`\varphi :𝒰𝒰^{}`$, $`\varphi ([k;y],[h])=[k;[k^1h],y]`$, where $`𝒰^{}:=K\times _T(\overline{K/T}\times 𝒴)`$. Let $`\varphi ^{}:\text{K}_K(\text{T}_K𝒰^{})\text{K}_K(\text{T}_K𝒰)`$ be the induced isomorphism. Then one considers the inclusion $`i:TK`$ which induces an isomorphism $`i_{}:\text{K}_T(\text{T}_T(\overline{K/T}\times 𝒴))\text{K}_K(\text{T}_K𝒰^{})`$ (see ). Let $`j:𝒴\overline{K/T}\times 𝒴`$ be the $`T`$-invariant inclusion map defined by $`j(y):=(\overline{e},y)`$. We have then a pushforward map $`j_!:\text{K}_T(\text{T}_T𝒴)\text{K}_T(\text{T}_T(\overline{K/T}\times 𝒴))`$. Finally we get a map

$$\mathrm{\Theta }:=\varphi ^{}i_{}j_!:\text{K}_T(\text{T}_T𝒴)\text{K}_K(\text{T}_K𝒰),$$

such that $`\text{Index}_𝒰^K(\mathrm{\Theta }(\sigma ))=\mathrm{Ind}__T^^K(\text{Index}_𝒴^T(\sigma ))`$ for every $`\sigma \text{K}_T(\text{T}_T𝒴)`$ (see Section 3 in ).

Proposition 4.13 is an immediate consequence of the following

###### Lemma 4.15.

We have the equality

$$\mathrm{\Theta }\left(\text{S-Thom}_T^{\mathrm{\Phi }\stackrel{~}{\mu }}(𝒴)\right)=\text{Thom}_K^{\mathrm{\Phi }_{\stackrel{~}{\mu }}}(𝒰).$$

Proof. Let $`𝒮`$ be the bundle of spinors on $`K\times _T𝒴`$: $`𝒮=\mathrm{P}\times _{\mathrm{Spin}_{2k}^c}\mathrm{\Delta }_{2k}`$, where $`\mathrm{P}\mathrm{P}_{\mathrm{SO}}(\text{T}(K\times _T𝒴))\times \mathrm{P}_\mathrm{U}(𝕃)`$ is the $`\mathrm{Spin}^c`$ structure induced by the complex structure. From Proposition 4.10, we have the reductions

(4.41)
Here $`Q/\mathrm{U}_l`$ is the induced $`\mathrm{Spin}^c`$-structure on $`𝒴`$. Let us denote by $`p:\text{T}(K\times _T𝒴)K\times _T𝒴`$, $`p__𝒴:\text{T}𝒴𝒴`$ and $`p_{_{K/T}}:\text{T}(K/T)K/T`$ the canonical projections. Using (4.41), we see that

$$p^{}𝒮=\left(K\times _Tp__𝒴^{}𝒮(𝒴)\right)p_{_{K/T}}^{}_{}\text{T}(K/T),$$

where $`𝒮(𝒴)`$ is the spinor bundle on $`𝒴`$. Hence we get the decomposition

$$\text{S-Thom}_K(K\times _T𝒴)=\text{Thom}_K(K/T)K\times _T\text{S-Thom}_T(𝒴).$$

The transversally elliptic symbol $`\text{Thom}_K^{\mathrm{\Phi }_{\stackrel{~}{\mu }}}(𝒰)`$ is equal to

$$\left[\text{Thom}_K(K/T)\text{Thom}_K(\overline{K/T})K\times _T\text{S-Thom}_T(𝒴)\right]_{\mathrm{deformed}\mathrm{by}^{\stackrel{~}{\mu }}},$$

hence $`\sigma _1:=(\varphi ^1)^{}\text{Thom}_K^{\mathrm{\Phi }_{\stackrel{~}{\mu }}}(𝒰)`$ is equal to

$$\left[\text{Thom}_K(K/T)K\times _T\left(\text{Thom}_T(\overline{K/T})\text{S-Thom}_T(𝒴)\right)\right]_{\mathrm{deformed}\mathrm{by}^{}},$$

where $`^{^{}}=\varphi _{}(^{\stackrel{~}{\mu }})`$.

Using the decomposition $`\text{T}𝒰^{}K\times _T(𝔨/𝔱K\times _T(\overline{𝔨/𝔱})\text{T}𝒴)`$, a small computation gives $`^{}(m)=pr_{𝔨/𝔱}(h\stackrel{~}{\mu })+R(m)+_{\stackrel{~}{\mu }}(y)+S(m)`$ for $`m=[k;[h],y]𝒰^{}`$, where777A small computation shows that $`R(m)=pr_{𝔨/𝔱}(h^1(pr_𝔱(h\stackrel{~}{\mu })\mathrm{\Phi }(y)))`$, and $`S(m)=[\stackrel{~}{\mu }pr_𝔱(h\stackrel{~}{\mu })]_𝒴(y)`$. $`R(m)\overline{𝔨/𝔱}`$ and $`S(m)\text{T}_y𝒴`$ vanishes when $`mK\times _T(\{\overline{e}\}\times 𝒴)`$, i.e. when $`[h]=\overline{e}`$. Here $`_{\stackrel{~}{\mu }}`$ is the Hamiltonian vector field of the function $`\frac{1}{2}\mathrm{\Phi }\stackrel{~}{\mu }^2:𝒴`$.

The transversally elliptic symbol $`\sigma _1`$ is equal to the exterior product

$$\sigma _1(m,\xi _1+\xi _2+v)=𝐜(\xi _1pr_{𝔨/𝔱}(h\stackrel{~}{\mu }))𝐜(\xi _2R(m))𝐜(v_{\stackrel{~}{\mu }}S(m)),$$

with $`\xi _1𝔨/𝔱`$, $`\xi _2\overline{𝔨/𝔱}`$, and $`v\text{T}𝒴`$.

Now, we simplify the symbol $`\sigma _1`$ without changing its $`K`$-theoretic class. Since $`\text{Char}(\sigma _1)\text{T}_K𝒰^{}=K\times _T(\{\overline{e}\}\times 𝒴)`$, we transform $`\sigma _1`$ through the $`K`$-invariant diffeomorphism $`h=e^X`$ from a neighborhood of $`0`$ in $`\overline{𝔨/𝔱}`$ to a neighborhood of $`\overline{e}`$ in $`\overline{K/T}`$. That gives $`\sigma _2K_K(\text{T}_K(K\times _T(𝔨/𝔱\times 𝒴)))`$ defined by

$`\sigma _2([k;X,y],\xi _1+\xi _2+v)=`$
$`𝐜(\xi _1pr_{𝔨/𝔱}(e^X\stackrel{~}{\mu }))𝐜(\xi _2R(m))𝐜(v_{\stackrel{~}{\mu }}S(m)).`$
Now trivial homotopies link $`\sigma _2`$ with the symbol

$$\sigma _3([k,X,y],\xi _1+\xi _2+v)=𝐜(\xi _1[X,\stackrel{~}{\mu }])𝐜(\xi _2)𝐜(v_{\stackrel{~}{\mu }}),$$

where we have removed the terms $`R(m)`$ and $`S(m)`$, and where we have replaced $`pr_{𝔨/𝔱}(e^X\stackrel{~}{\mu })=[X,\stackrel{~}{\mu }]+\mathrm{o}([X,\stackrel{~}{\mu }])`$ by the term $`[X,\stackrel{~}{\mu }]`$. Now we see that $`\sigma _3=i_{}(\sigma _4)`$ where the symbol $`\sigma _4K_T(\text{T}_T(𝔨/𝔱\times 𝒴))`$ is defined by

$$\sigma _4(X,y;\xi _2+v)=𝐜([X,\stackrel{~}{\mu }])𝐜(\xi _2)𝐜(v_{\stackrel{~}{\mu }}).$$

So $`\sigma _4`$ is equal to the exterior product of $`(y,v)𝐜(v_{\stackrel{~}{\mu }})`$, which is $`\text{S-Thom}_T^{\mathrm{\Phi }\stackrel{~}{\mu }}(𝒴)`$, with the transversally elliptic symbol on $`𝔨/𝔱`$: $`(X,\xi _2)𝐜([X,\stackrel{~}{\mu }])𝐜(\xi _2)`$. But the $`K`$-theoretic class of this former symbol is equal to $`k_!()`$, where $`k:\{0\}𝔨/𝔱`$ (see subsection 5.1 in ). This shows that

$$\sigma _4=k_!()\text{S-Thom}_T^{\mathrm{\Phi }\stackrel{~}{\mu }}(𝒴)=j_!(\text{S-Thom}_T^{\mathrm{\Phi }\stackrel{~}{\mu }}(𝒴)).$$

$`\mathrm{}`$

### 4.5. Proof of Proposition 4.14

In this subsection, $`\stackrel{~}{\mu }=\mu +\rho _c`$ is fixed, and is assumed to belong to $`\mathrm{\Delta }`$. The induced moment map on the symplectic slice $`\mathrm{\Phi }^1(\mathrm{Interior}(𝔱_+^{}))`$ is still denoted by $`\mathrm{\Phi }`$. Let $`r>0`$ be the smallest non zero critical value of $`\mathrm{\Phi }\stackrel{~}{\mu }`$, and let $`𝒴=\mathrm{\Phi }^1\{\xi \mathrm{Aff}(\mathrm{\Delta }),\xi \stackrel{~}{\mu }<\frac{r}{2}\}`$.

For $`\xi \mathrm{Aff}(\mathrm{\Delta })`$, we consider $`\xi _t=t\xi +(1t)\stackrel{~}{\mu }`$, $`0t1`$. If one shows that there exists a compact subset $`𝒦𝒴`$ such that $`\text{Cr}(\mathrm{\Phi }\xi _t^2)𝒴𝒦`$, the family $`\text{S-Thom}_T^{\mathrm{\Phi }\xi _t}(𝒴)`$, $`0t1`$, defines then a homotopy of transversally elliptic symbols between $`\text{S-Thom}_T^{\mathrm{\Phi }\stackrel{~}{\mu }}(𝒴)`$ and $`\text{S-Thom}_T^{\mathrm{\Phi }\xi }(𝒴)`$. It shows that $`𝒬_{\mathrm{\Phi }\stackrel{~}{\mu }}^^T(𝒴,)`$ and $`𝒬_{\mathrm{\Phi }\xi }^^T(𝒴,)`$ are equal.

We describe now $`\text{Cr}(\mathrm{\Phi }\xi _t^2)𝒴`$ using a parametrization introduced in \[Section 6\]. Let $``$ be the collection of affine subspaces of $`𝔱^{}`$ generated by the image under $`\mathrm{\Phi }`$ of submanifolds $`𝒵`$ of the following type: $`𝒵`$ is a connected component of $`𝒴^H`$ which intersects $`\mathrm{\Phi }^1(\stackrel{~}{\mu })`$, $`H`$ being a subgroup of $`T`$. The set $``$ is finite since $`\mathrm{\Phi }^1(\stackrel{~}{\mu })`$ is compact and thus has a finite number of stabilizers for the $`T`$ action. Note that $``$ is reduced to $`\mathrm{Aff}(\mathrm{\Delta })`$ if $`\stackrel{~}{\mu }`$ is a generic quasi regular value of $`\mathrm{\Phi }`$. For $`A`$, we denote by $`\beta (,A)`$ the orthogonal projection on $`A`$. Let $`_\xi =\{\beta (\xi ,A)\xi A\}`$. Like in \[Section 4.3\], we see that

(4.42)
$$\text{Cr}(\mathrm{\Phi }\xi ^2)𝒴=\underset{\beta _\xi }{}\left(𝒴^\beta \mathrm{\Phi }^1(\beta +\xi )\right)$$

if $`\xi \stackrel{~}{\mu }<\frac{r}{2}`$. If we take $`𝒦:=\mathrm{\Phi }^1\{\xi \mathrm{Aff}\mathrm{\Delta },\xi \stackrel{~}{\mu }\frac{r}{3}\}`$, we have $`\text{Cr}(\mathrm{\Phi }\xi ^2)𝒴𝒦`$ for $`\xi \stackrel{~}{\mu }\frac{r}{3}`$. Thus point $`i)`$ of Proposition 4.14 is proved.

Now we fix $`\xi \mathrm{Aff}(\mathrm{\Delta })`$ close enough to $`\stackrel{~}{\mu }`$. And for each $`\beta _\xi `$, we denote by $`𝒬_\beta ^^T(𝒴,)`$ the map localized near $`𝒴^\beta \mathrm{\Phi }^1(\beta +\xi )`$. The excision property tells us, like in (3.14), that

$$𝒬_{\mathrm{\Phi }\xi }^^T(𝒴,)=\underset{\beta _\xi }{}𝒬_\beta ^^T(𝒴,).$$

Note that $`0_\xi `$ if and only if $`\mathrm{\Phi }^1(\xi )\mathrm{}`$. Point iii) of Proposition 4.14 will follow from the following

###### Lemma 4.16.

Let $`\xi \mathrm{Aff}(\mathrm{\Delta })`$ close enough to $`\stackrel{~}{\mu }`$, and let $`\beta `$ be a non-zero element of $`_\xi `$. Then $`[𝒬_\beta ^^T(𝒴,\stackrel{~}{L}|_𝒴)]^T=0`$. Hence $`[𝒬_{\mathrm{\Phi }\xi }^^T(𝒴,\stackrel{~}{L}|_𝒴_\mu )]^T=0`$, if $`\xi \mathrm{\Delta }`$.

For the point $`ii)`$ of Proposition 4.14, we also need the

###### Lemma 4.17.

If $`\xi \mathrm{\Delta }^o`$ is a generic quasi regular value of $`\mathrm{\Phi }`$, we have $`[𝒬_0^^T(𝒴,\stackrel{~}{L}|_𝒴_\mu )]^T=𝒬(M_\xi ,o(M_\xi ))`$.

Other versions of Lemmas 4.17 and 4.16 are already known : in the $`\mathrm{Spin}`$-case for an $`S^1`$-action by Vergne , and by the author  when the $`\mathrm{Spin}^c`$-structure comes from an almost complex structure.

We review briefly the arguments, since they work in the same way. We consider the $`\mathrm{Spin}^c`$ structure on $`𝒴`$ defined in Proposition 4.10, that we twist by the line bundle $`\stackrel{~}{L}|_𝒴_\mu `$ : it defines a $`\mathrm{Spin}^c`$ structure $`\mathrm{Q}^\mu `$ on $`𝒴`$ with canonical line bundle $`𝕃^\mu :=L_{2\omega }_{2\stackrel{~}{\mu }}`$. We consider then the symbol $`\text{S-Thom}_{T,\mu }^{\mathrm{\Phi }\xi }(𝒴)`$ constructed with $`\mathrm{Q}^\mu `$ (see Definition 4.7). For $`\beta _\xi `$, the term $`𝒬_\beta ^^T(𝒴,\stackrel{~}{L}|_𝒴_\mu )`$ is by definition the $`T`$-index of $`\text{S-Thom}_{T,\mu }^{\mathrm{\Phi }\xi }(𝒴)|_{𝒰^\beta }`$, where $`𝒰^\beta `$ is a sufficiently small open neighborhood of $`𝒴^\beta \mathrm{\Phi }^1(\beta +\xi )`$ in $`𝒴`$.

Proof of Lemma 4.17. A neighborhood $`𝒰^0`$ of $`𝒵:=\mathrm{\Phi }^1(\xi )`$ is diffeomorphic to a neighborhood of $`𝒵`$ in $`𝒵\times \stackrel{}{\mathrm{\Delta }}`$, where $`\mathrm{\Phi }\xi :𝒵\times \stackrel{}{\mathrm{\Delta }}\stackrel{}{\mathrm{\Delta }}`$ is the projection to the second factor. Let $`pr:𝒵\times \stackrel{}{\mathrm{\Delta }}𝒵`$ be the projection to the first factor. We still denote by $`𝒬^\mu `$ the $`\mathrm{Spin}^c`$-structure on $`𝒵\times \stackrel{}{\mathrm{\Delta }}`$ equal to $`pr^{}(𝒬^\mu |_𝒵)`$. We easily show that $`𝒬_\beta ^^T(𝒴,\stackrel{~}{L}|_𝒴_\mu )`$ is equal to the $`T`$-index of $`\sigma _𝒵=\text{S-Thom}_{T,\mu }^{\mathrm{\Phi }\xi }(𝒵\times \stackrel{}{\mathrm{\Delta }})`$. Let $`\mathrm{Q}^{\prime \prime }`$ be the reduction of $`𝒬^\mu |_𝒵`$ introduced in (4.36). Since $`𝒬^\mu |_𝒵=\mathrm{Spin}_{2k}^c\times _{(\mathrm{Spin}_{2k^{}}^c\times \mathrm{U}_{2l^{}})}\mathrm{Q}^{\prime \prime }`$, the bundle of spinors $`𝒮`$ over $`𝒵\times \stackrel{}{\mathrm{\Delta }}`$ decomposes as

$$𝒮=pr^{}(\pi ^{}𝒮(M_\xi )𝒵\times (𝔱/𝔱_\mathrm{\Delta })).$$

Here $`𝒮(M_\xi )`$ is the bundle of spinors on $`M_\xi `$ induces by the $`\mathrm{Spin}^c`$-structure $`\mathrm{Q}^{\prime \prime }/\mathrm{U}_{2l^{}}`$, and $`\pi :𝒵M_\xi `$ is the quotient map. Inside the trivial bundle $`𝒵\times (𝔱/𝔱_\mathrm{\Delta })`$, we have identified $`𝒵\times (𝔱/𝔱_\mathrm{\Delta }i)`$ with the subspace of $`\text{T}𝒵`$ formed by the vector fields generated by the infinitesimal action of $`𝔱/𝔱_\mathrm{\Delta }`$, and $`𝒵\times (𝔱/𝔱_\mathrm{\Delta })`$ with $`𝒵\times \stackrel{}{\mathrm{\Delta }}\text{T}(𝒵\times \stackrel{}{\mathrm{\Delta }})|_𝒵`$. For $`(z,f)𝒵\times \stackrel{}{\mathrm{\Delta }}`$, let us decompose $`v\text{T}_{(z,f)}(𝒵\times \stackrel{}{\mathrm{\Delta }})`$ as $`v=v_1+X+iY`$, where $`v_1\pi ^{}(\text{T}M_\xi )`$, and $`X+iY𝔱/𝔱_\mathrm{\Delta }`$. The map $`\sigma _𝒵(z,f;v_1+X+iY)`$ acts on $`𝒮(M_\xi )_z(𝔱/𝔱_\mathrm{\Delta })`$ as the product

$$𝐜_z(v_1)𝐜(X+i(Yf)),$$

which is homotopic888See \[Section 6.1\]. to the transversally elliptic symbol

$$𝐜_z(v_1)𝐜(f+iX).$$

So we have proved that $`\sigma _𝒵=j_!\pi ^{}(\text{S-Thom}(M_\xi ))`$, where $`j_!:K_T(\text{T}_T𝒵)K_T(\text{T}_T(𝒵\times \stackrel{}{\mathrm{\Delta }}))`$ is induced by the inclusion $`j:𝒵𝒵\times \stackrel{}{\mathrm{\Delta }}`$. The last equality finishes the proof (see \[Section 6.1\]).

Proof of Lemma 4.16. The equality $`[𝒬_\beta ^^T(𝒴,\stackrel{~}{L}|_𝒴_\mu )]^T=0`$ comes from a localization formula on the submanifold $`𝒴^\beta `$ for the map $`𝒬_\beta ^^T(𝒴,)`$ (see ). The normal bundle $`𝒩`$ of $`𝒴^\beta `$ in $`𝒴`$ carries a complex structure $`J_𝒩`$ on the fibers such that each $`𝕋_\beta `$-weight $`\alpha `$ on $`(𝒩,J_𝒩)`$ satisfies $`(\alpha ,\beta )>0`$. The principal bundle $`\mathrm{Q}^\mu `$, when restricted to $`𝒴^\beta `$ admits the reduction

(4.43)
Hence $`\mathrm{Q}^\beta :=\mathrm{Q}^{}/\mathrm{U}(l)`$ is a $`\mathrm{Spin}^c`$-structure on $`𝒴^\beta `$ with canonical line bundle equal to $`𝕃^\beta :=𝕃^\mu |_{𝒴^\beta }(det𝒩)^1=L_{2\omega }|_{𝒴^\beta }_{2\stackrel{~}{\mu }}(det𝒩)^1`$. Let $`𝒬_\beta ^T(𝒴^\beta ,)`$ be the map defined by $`\mathrm{Q}^\beta `$ and localized near $`\mathrm{\Phi }^1(\beta +\xi )𝒴^\beta `$ by $`\mathrm{\Phi }\xi `$. Following the argument of \[Section 6\] one obtains

$$𝒬_\beta ^^T(𝒴,\stackrel{~}{L}|_𝒴_\mu )=(1)^l\underset{k}{}𝒬_\beta ^^T(𝒴^\beta ,det𝒩S^k(𝒩)),$$

where $`S^k(𝒩)`$ is the k-th symmetric product of $`𝒩`$, and $`l=\mathrm{rank}_{}𝒩`$. Thus, it is sufficient to prove that $`[𝒬_\beta ^^T(𝒴^\beta ,det𝒩S^k(𝒩))]^T=0`$ for every $`k`$. For this purpose, we use Lemma 4.8. Let $`\alpha `$ be the $`𝕋_\beta `$-weight on $`det𝒩`$. From the Kostant formula (2.1), the $`𝕋_\beta `$-weight on $`L_{2\omega }|_{𝒴^\beta }`$ is equal to $`2(\beta +\xi )`$. Hence any $`𝕋_\beta `$-weight $`\gamma `$ on $`det𝒩S^k(𝒩)(𝕃^\beta )^{1/2}`$ is of the form

$$\gamma =\beta +\xi \stackrel{~}{\mu }+\frac{1}{2}\alpha +\delta $$

where $`\delta `$ is a $`𝕋_\beta `$-weight on $`S^k(𝒩)`$. So $`(\gamma ,\beta )=(\beta +\xi \stackrel{~}{\mu },\beta )+\frac{1}{2}(\alpha ,\beta )+(\delta ,\beta )`$. But the $`𝕋_\beta `$-weights on $`𝒩`$ are ‘positive’ for $`\beta `$, so $`(\alpha ,\beta )>0`$ and $`(\delta ,\beta )0`$. On the other hand, $`\beta +\xi =\beta (\xi ,A)`$ is the orthogonal projection of $`\xi `$ on some affine subspace $`A𝔱_+^{}`$ which contains $`\stackrel{~}{\mu }`$: hence $`(\beta +\xi \stackrel{~}{\mu },\beta )=0`$. This proves that $`(\gamma ,\beta )>0`$. $`\mathrm{}`$

## 5. Quantization and the discrete series

In this section we apply Theorem 4.1 to the coadjoint orbits that parametrize the discrete series of a real, connected, semi-simple Lie group $`G`$, with finite center. Nice references on the subject of ‘the discrete series’ are .

Let $`K`$ be a maximal compact subgroup of $`G`$, and $`T`$ be a maximal torus in $`K`$. For the remainder of this section, we assume that $`T`$ is a Cartan subgroup of $`G`$. The discrete series of $`G`$ is then non-empty and is parametrized by a subset $`\widehat{G}_d`$ in the dual $`𝔱^{}`$ of the Lie algebra of $`T`$ .

Let us fix some notation. Let $`_c\mathrm{\Lambda }^{}`$ be respectively the set of (real) roots for the action of $`T`$ on $`𝔨`$ and $`𝔤`$. We choose a system of positive roots $`_c^+`$ for $`_c`$, we denote by $`𝔱_+^{}`$ the corresponding Weyl chamber, and we let $`\rho _c`$ be half the sum of the elements of $`_c^+`$. We denote by $`B`$ the Killing form on $`𝔤`$. It induces a scalar product (denoted by $`(,)`$) on $`𝔱`$, and then on $`𝔱^{}`$. An element $`\lambda 𝔱^{}`$ is called regular if $`(\lambda ,\alpha )0`$ for every $`\alpha `$, or equivalently, if the stabilizer subgroup of $`\lambda `$ in $`G`$ is $`T`$. Given a system of positive roots $`^+`$ for $``$, consider the subset $`\mathrm{\Lambda }^{}+\frac{1}{2}_{\alpha ^+}\alpha `$ of $`𝔱^{}`$. This does not depend on the choice of $`^+`$, and we denote it by $`\mathrm{\Lambda }_\rho ^{}`$ .

The discrete series of $`G`$ are parametrized by

(5.44)
$$\widehat{G}_d:=\{\lambda 𝔱^{},\lambda \mathrm{regular}\}\mathrm{\Lambda }_\rho ^{}𝔱_+^{}.$$

When $`G=K`$ is compact, the set $`\widehat{G}_d`$ equals $`\mathrm{\Lambda }_+^{}+\rho _c`$, and it parametrizes the set of irreducible representations of $`K`$. Harish-Chandra has associated to any $`\lambda \widehat{G}_d`$ an invariant eigendistribution on $`G`$, denoted by $`\mathrm{\Theta }_\lambda `$, which is shown to be the global trace of an irreducible, square integrable, unitary representation of $`G`$.

On the other hand we associate to $`\lambda \widehat{G}_d`$, the regular coadjoint orbit $`M:=G\lambda `$. It is a symplectic manifold with a Hamiltonian action of $`K`$. Since the vectors $`X_M,X𝔤`$, span the tangent space at every $`\xi M`$, the symplectic $`2`$-form is determined by

$$\omega (X_M,Y_M)_\xi =\xi ,[X,Y].$$

The corresponding moment map $`\mathrm{\Phi }:M𝔨^{}`$ for the $`K`$-action is the composition of the inclusion $`i:M𝔤^{}`$ with the projection $`𝔤^{}𝔨^{}`$. The vector $`\lambda `$ determines a choice $`^{+,\lambda }`$ of positive roots for the $`T`$-action on $`𝔤`$ : $`\alpha ^{+,\lambda }(\alpha ,\lambda )>0`$. We recall now how the choice of $`^{+,\lambda }`$ determines a complex structure on $`M`$. First take the decomposition $`𝔤=𝔱_\alpha 𝔤_\alpha `$ where $`𝔤_\alpha :=\{v𝔤\mathrm{exp}(X).v=e^{i\alpha ,X}v\mathrm{for}\mathrm{any}X𝔱`$}. It gives the following $`T`$-equivariant decomposition of the complexified tangent space of $`M`$ at $`\lambda `$ :

$$\text{T}_\lambda M=\underset{\alpha }{}𝔤_\alpha =𝔫\overline{𝔫},$$

with $`𝔫=_{\alpha ^{+,\lambda }}𝔤_\alpha `$. We have then a $`T`$-equivariant isomorphism $`:\text{T}_\lambda M𝔫`$ equal to the composition of the inclusion $`\text{T}_\lambda M\text{T}_\lambda M`$ with the projection $`𝔫\overline{𝔫}𝔫`$. The $`T`$-equivariant complex structure $`J_\lambda `$ on $`\text{T}_\lambda M`$ is determined by the relation $`(J_\lambda v)=i(v)`$. Hence, the set of real infinitesimal weights for the $`T`$-action on $`(\text{T}_\lambda M,J_\lambda )`$ is $`^{+,\lambda }`$. Since $`M`$ is a homogeneous space, $`J_\lambda `$ defines an invariant almost complex structure $`J`$ on $`M`$, which is in fact integrable. Through the isomorphism $`MG/T`$, the canonical line bundle $`\kappa =det_{}(\text{T}M)^1`$ is equal to $`\kappa =G\times _T_{2\rho }`$ with $`\rho =\frac{1}{2}_{\alpha ^{+,\lambda }}\alpha `$.

If $`\lambda \widehat{G}_d`$, then $`\lambda \rho `$ is a weight, and

(5.45)
$$\stackrel{~}{L}:=G\times _T_{\lambda \rho }G/T$$

is a $`\kappa `$-prequantum line bundle over $`(M,\omega ,J)`$. We have shown in , that $`\text{Cr}(\mathrm{\Phi }^2)`$ is compact, equal to the $`K`$-orbit $`K\lambda `$. Then the generalized Riemann-Roch character $`RR_\mathrm{\Phi }^^K(M,)`$ is well defined (see Definition 3.2). The main result of this section is the following

###### Theorem 5.1.

We have the following equality of tempered distributions on $`K`$

$$\mathrm{\Theta }_\lambda |_K=(1)^{\frac{dim(G/K)}{2}}RR_\mathrm{\Phi }^^K(G\lambda ,\stackrel{~}{L}),$$

where $`\mathrm{\Theta }_\lambda |_K`$ is the restriction of the eigendistribution $`\mathrm{\Theta }_\lambda `$ to the subgroup $`K`$.

The proof of Theorem 5.1 is given in Subsection 5.2. It uses the Blattner formulas in an essential way (see Subsection 5.1).

With Theorem 5.1 at our disposal we can exploit the result of Theorem 4.1 to compute the $`K`$-multiplicities, $`\text{m}_\mu (\lambda )`$, of $`\mathrm{\Theta }_\lambda |_K`$ in term of the reduced spaces. By definition we have

(5.46)
$$\mathrm{\Theta }_\lambda |_K=\underset{\mu \mathrm{\Lambda }_+^{}}{}\text{m}_\mu (\lambda )\chi __\mu ^_K\mathrm{in}R^{\mathrm{}}(K).$$

The moment map $`\mathrm{\Phi }:G\lambda 𝔨^{}`$ is proper since the $`G\lambda `$ is closed in $`𝔤^{}`$ . We show (Lemma 5.5) that the moment polyhedron $`\mathrm{\Delta }=\mathrm{\Phi }(G\lambda )𝔱_+^{}`$ is of dimension $`dimT`$. Thus on the relative interior $`\mathrm{\Delta }^o`$ of the moment polyhedron, the notions of generic quasi-regular values and regular values coincide : they concern the elements $`\xi \mathrm{\Delta }^o`$ such that $`\mathrm{\Phi }^1(\xi )`$ is a smooth submanifold with a locally free action of $`T`$. We have shown (Subsection 4.3) how to define the quantity $`Q((G\lambda )_{\mu +\rho _c})`$ as the index of a suitable $`\mathrm{Spin}^c`$ Dirac operator on $`\mathrm{\Phi }^1(\xi )/T`$, where $`\xi \mathrm{\Delta }^o`$ is a regular value of $`\mathrm{\Phi }`$ close enough to $`\mu +\rho _c`$.

###### Proposition 5.2.

For every $`\mu \mathrm{\Lambda }_+^{}`$, we have

$$\text{m}_\mu (\lambda )=𝒬\left((G\lambda )_{\mu +\rho _c}\right).$$

In particular $`\text{m}_\mu (\lambda )=0`$ if $`\mu +\rho _c`$ does not belong to the relative interior of the moment polyhedron $`\mathrm{\Delta }`$.

Proof. A small check of orientations shows that $`\epsilon =(1)^{\frac{dim(G/K)}{2}}`$, thus this proposition follows from Theorems 4.1 and 5.1 if one checks that the following holds: $`(G\lambda ,\mathrm{\Phi })`$ satisfies Assumption 3.6, and the infinitesimal $`K`$-stabilizers are Abelian. The first point will be handled in Subsection 5.3. The second point is obvious since $`MG/T`$: all the $`G`$-stabilizers are conjugate to $`T`$, so all the $`K`$-stabilizers are Abelian. $`\mathrm{}`$

### 5.1. Blattner formulas

In this section, we fix $`\lambda \widehat{G}_d`$. Let $`^{+,\lambda }`$ be the system of positive roots defined $`\lambda `$: $`\alpha ^{+,\lambda }(\alpha ,\lambda )>0`$. Then $`_c^+^{+,\lambda }`$, and $`\rho =\frac{1}{2}_{\alpha ^{+,\lambda }}\alpha `$ decomposes in $`\rho =\rho _c+\rho _n`$ where $`\rho _n=\frac{1}{2}_{\alpha _n^+}\alpha `$ and $`_n^+=^{+,\lambda }_c^+`$.

Let $`𝒫:\mathrm{\Lambda }^{}`$ be the partition function associated to the set $`_n^+`$ : for $`\mu \mathrm{\Lambda }^{}`$, $`𝒫(\mu )`$ is the number of distinct ways we can write $`\mu =_{\alpha _n^+}k_\alpha \alpha `$ with $`k_\alpha `$ for all $`\alpha `$. The following Theorem is known as the Blattner formulas and was first proved by Hecht and Schmid .

###### Theorem 5.3.

For $`\mu \mathrm{\Lambda }_+^{}`$, we have

$$\text{m}_\mu (\lambda )=\underset{wW}{}(1)^w𝒫\left(w(\mu +\rho _c)(\mu _\lambda +\rho _c)\right),$$

where999We shall note that $`\mu _\lambda \mathrm{\Lambda }_+^{}`$ (see , section 5). $`\mu _\lambda :=\lambda \rho _c+\rho _n`$. Here $`W`$ is the Weyl group of $`(K,T)`$.

Using Theorem 5.3, we can describe $`\mathrm{\Theta }_\lambda |_K`$ through the holomorphic induction map $`\mathrm{Hol}__T^^K:R^{\mathrm{}}(T)R^{\mathrm{}}(K)`$. Recall that $`\mathrm{Hol}__T^^K`$ is characterized by the following properties: i) $`\mathrm{Hol}__T^^K(t^\mu )=\chi __\mu ^_K`$ for every dominant weight $`\mu \mathrm{\Lambda }_+^{}`$; ii) $`\mathrm{Hol}__T^^K(t^{w\mu })=(1)^w\mathrm{Hol}__T^^K(t^\mu )`$ for every $`wW`$ and $`\mu \mathrm{\Lambda }^{}`$; iii) $`\mathrm{Hol}__T^^K(t^\mu )=0`$ if $`W\mu \mathrm{\Lambda }_+^{}=\mathrm{}`$. Using these properties we have

(5.47)
$$\underset{\mu \mathrm{\Lambda }^{}}{}(\mu )\mathrm{Hol}__T^^K(t^\mu )=\underset{\mu \mathrm{\Lambda }_+^{}}{}\left[\underset{wW}{}(1)^w(w\mu )\right]\chi __\mu ^_K,$$

for every map $`:\mathrm{\Lambda }^{}`$.

For a weight $`\alpha \mathrm{\Lambda }^{}`$, with $`(\lambda ,\alpha )0`$, let us denote the oriented inverse of $`(1t^\alpha )`$ in the following way

$$[1t^\alpha ]_\lambda ^1=\{\begin{array}{cc}_kt^{k\alpha },\hfill & \mathrm{if}(\lambda ,\alpha )>0\hfill \\ t^\alpha _kt^{k\alpha },\hfill & \mathrm{if}(\lambda ,\alpha )<0.\hfill \end{array}$$

Let $`A=\{\alpha _1,\mathrm{},\alpha _l\}`$ be a set of weights with $`(\lambda ,\alpha _i)0,i`$. We denote by $`A^+=\{\epsilon _1\alpha _1,\mathrm{},\epsilon _l\alpha _l\}`$ the corresponding set of polarized weights: $`\epsilon _i=\pm 1`$ and $`(\lambda ,\epsilon _i\alpha _i)>0`$ for all $`i`$. The product $`\mathrm{\Pi }_{\alpha A}[1t^\alpha ]_\lambda ^1`$ is well defined in $`R^{\mathrm{}}(T)`$, and is denoted by $`[\mathrm{\Pi }_{\alpha A}(1t^\alpha )]_\lambda ^1`$. A small computation shows that

(5.48) $`\left[\mathrm{\Pi }_{\alpha A}(1t^\alpha )\right]_\lambda ^1`$ $`=`$ $`(1)^rt^\gamma \left[\mathrm{\Pi }_{\alpha A^+}(1t^\alpha )\right]_\lambda ^1`$
$`=`$ $`(1)^rt^\gamma {\displaystyle \underset{\mu \mathrm{\Lambda }^{}}{}}𝒫_{A^+}(\mu )t^\mu .`$
Here $`𝒫_{A^+}:\mathrm{\Lambda }^{}`$ is the partition function associated to $`A^+`$, $`\gamma =_{(\lambda ,\alpha )<0}\alpha `$, and $`r=\mathrm{}\{\alpha A,(\lambda ,\alpha )<0\}`$. This notation is compatible with the one we used in \[Section 5\]. If $`V`$ is a complex $`T`$-vector space where the subspace fixed by $`\lambda `$ is reduced to $`\{0\}`$, then $`_{}^{}VR(T)`$ admits a polarized inverse $`[_{}^{}V]_\lambda ^1=[\mathrm{\Pi }_{\alpha (V)}(1t^\alpha )]_\lambda ^1`$, where $`(V)`$ is the set of real infinitesimal $`T`$-weights on $`V`$.

###### Lemma 5.4.

We have the following equality in $`R^{\mathrm{}}(K)`$

$$\mathrm{\Theta }_\lambda |_K=\mathrm{Hol}__T^^K\left(t^{\mu _\lambda }\left[\mathrm{\Pi }_{\alpha _n^+}(1t^\alpha )\right]_\lambda ^1\right).$$

Proof. Let $`\mathrm{\Theta }R^{\mathrm{}}(K)`$ be the RHS in the equality of the Lemma. From (5.48), we have $`\mathrm{\Theta }=_{\mu \mathrm{\Lambda }^{}}𝒫(\mu )\mathrm{Hol}__T^^K(t^{\mu +\mu _\lambda })=_{\mu \mathrm{\Lambda }^{}}𝒫(\mu \mu _\lambda )\mathrm{Hol}__T^^K(t^\mu )`$. If we use now (5.47), we see that multiplicity of $`\mathrm{\Theta }`$ relative to the highest weight $`\mu \mathrm{\Lambda }_+^{}`$ is $`_{wW}(1)^w𝒫(w(\mu +\rho _c)(\mu _\lambda +\rho _c))`$. From Theorem 5.3 we conclude that $`\mathrm{\Theta }_\lambda |_K=\mathrm{\Theta }`$. $`\mathrm{}`$

### 5.2.  Proof of Theorem 5.1

In Lemma 5.4 we have used the Blattner formulas to write $`\mathrm{\Theta }_\lambda |_K`$ in term of the holomorphic induction map $`\mathrm{Hol}__T^^K`$. Theorem 5.1 is then proved if one shows that $`RR_\mathrm{\Phi }^^K(G\lambda ,\stackrel{~}{L})=(1)^r\mathrm{Hol}__T^^K(t^{\mu _\lambda }[\mathrm{\Pi }_{\alpha _n^+}(1t^\alpha )]_\lambda ^1)`$, with $`\mu _\lambda =\lambda \rho _c+\rho _n`$, and $`r=\frac{1}{2}dim(G/K)`$. More generally, we show in this section that for any $`K`$-equivariant vector bundle $`VG\lambda `$

(5.49)
$$RR_\mathrm{\Phi }^^K(G\lambda ,V)=(1)^r\mathrm{Hol}__T^^K(V_\lambda .t^{2\rho _n}.\left[\mathrm{\Pi }_{\alpha _n^+}(1t^\alpha )\right]_\lambda ^1),$$

where $`V_\lambda R(T)`$ is the fiber of $`V`$ at $`\lambda `$.

First we recall why $`\text{Cr}(\mathrm{\Phi }^2)=K\lambda `$ in $`M:=G\lambda `$ (see  for the general case of closed coadjoint orbits). One can work with an adjoint orbit $`M:=G\stackrel{~}{\lambda }`$ through the $`G`$-identification $`𝔤^{}𝔤`$ given by the Killing form; then $`\mathrm{\Phi }:M𝔨`$ is just the restriction on $`M`$ of the (orthogonal) projection $`𝔤𝔨`$. Let $`𝔭`$ be the orthogonal complement of $`𝔨`$ in $`𝔤`$. Every $`mM`$ decomposes as $`m=x_m+y_m`$, with $`x_m=\mathrm{\Phi }(m)`$ and $`y_m𝔭`$. The Hamiltonian vector field of $`\frac{1}{2}\mathrm{\Phi }^2`$ is $`_m=[x_m,m]=[x_m,y_m]`$ (see 3.8). Thus

$$\text{Cr}(\mathrm{\Phi }^2)=\{=0\}=\{mM,[x_m,y_m]=0\}.$$

Now, since $`\stackrel{~}{\lambda }`$ is elliptic, every $`mM`$ is also elliptic. If $`m\text{Cr}(\mathrm{\Phi }^2)`$, $`[m,x_m]=0`$ and $`m,x_m`$ are elliptic, hence $`y_m=mx_m`$ is elliptic and so is equal to $`0`$. Finally $`\text{Cr}(\mathrm{\Phi }^2)=G\stackrel{~}{\lambda }𝔨=K\stackrel{~}{\lambda }`$.

According to Definition 3.2, the computation of $`RR_\mathrm{\Phi }^^K(G\lambda ,\stackrel{~}{L})`$ holds on a small $`K`$-invariant neighborhood of $`K\lambda `$ of $`G\lambda `$. Our model for the computation will be

$$\stackrel{~}{M}:=K\times _T𝔭$$

endowed with the canonical $`K`$-action. The tangent bundle $`\text{T}\stackrel{~}{M}`$ is isomorphic to $`K\times _T(𝔯\text{T}𝔭)`$ where $`𝔯`$ is the $`T`$-invariant complement of $`𝔱`$ in $`𝔨`$. One has a symplectic form $`\stackrel{~}{\mathrm{\Omega }}`$ on $`\stackrel{~}{M}`$ defined by $`\stackrel{~}{\mathrm{\Omega }}_m(V,V^{})=\lambda ,[X,X^{}]+[v,v^{}]`$. Here $`m=[k,x]K\times _T𝔭`$, and $`V=[k,x;X+v]`$, $`X^{}=[k,x;X^{}+v^{}]`$ are two tangent vectors, with $`X,X^{}𝔯`$ and $`v,v^{}𝔭`$. A small computation shows that the $`K`$-action on $`(K\times _T𝔭,\stackrel{~}{\mathrm{\Omega }})`$ is Hamiltonian with moment map $`\stackrel{~}{\mathrm{\Phi }}:\stackrel{~}{M}𝔨^{}`$ defined by

$$\stackrel{~}{\mathrm{\Phi }}([k,x])=k\left(\lambda \frac{1}{2}pr_𝔱^{}(\lambda \mathrm{ad}(x)\mathrm{ad}(x))\right).$$

Here $`\mathrm{ad}(x)`$ is the adjoint action of $`x`$, and $`pr_𝔱^{}:𝔤^{}𝔱^{}`$ is the projection. Note first that the tangent space $`\text{T}_\lambda M`$ and $`\text{T}_{[1,0]}\stackrel{~}{M}`$ are canonically isomorphic to $`𝔯𝔭`$.

###### Lemma 5.5.

There exists a $`K`$-Hamiltonian isomorphism $`\mathrm{{\rm Y}}:𝒰\stackrel{~}{𝒰}`$, where $`𝒰`$ is a $`K`$-invariant neighborhood of $`K\lambda `$ in $`M`$, $`\stackrel{~}{𝒰}`$ is a $`K`$-invariant neighborhood of $`K/T`$ in $`\stackrel{~}{M}`$, and $`\mathrm{{\rm Y}}(\lambda )=[1,0]`$. We impose furthermore that the differential of $`\mathrm{{\rm Y}}`$ at $`\lambda `$ is the identity.

###### Corollary 5.6.

The cone $`\lambda +_{\alpha _n^+}^+\alpha `$ coincides with $`\mathrm{\Delta }=(G\lambda )𝔱_+^{}`$ in a neighborhood of $`\lambda `$. The polyhedral set $`\mathrm{\Delta }`$ is of dimension $`dimT`$.

Proof of the Corollary. The first assertion is an immediate consequence of Lemma 5.5 and of the convexity Theorem . Let $`X_o𝔱`$ such that $`\xi (X_o)=0`$ for all $`\xi \stackrel{}{\mathrm{\Delta }}`$, that is $`\alpha (X_o)=0`$ for all $`\alpha _n^+`$ : $`X_o`$ commutes with all elements in $`𝔭`$. Let $`𝔞`$ be a maximal Abelian subalgebra of $`𝔭`$, and let $`\mathrm{\Sigma }`$ be the set of weights for the adjoint action of $`𝔞`$ on $`𝔤`$ : $`𝔤=_{\alpha \mathrm{\Sigma }}𝔤_\alpha `$, where $`𝔤_\alpha =\{Z𝔤,[X,Z]=\alpha (X)Z\mathrm{for}\mathrm{all}X𝔞\}`$. Since $`[X_o,𝔞]=0`$, we have $`[X_o,𝔤_\alpha ]𝔤_\alpha `$ for all $`\alpha \mathrm{\Sigma }`$. But since $`[X_o,𝔭]=0`$ and $`𝔤_\alpha 𝔨=0`$ for all $`\alpha 0`$, we see that $`[X_o,𝔤_\alpha ]=0`$ for all $`\alpha 0`$. But $`X_o`$ belongs to the Abelian subalgebra $`𝔤_0`$, so $`[X_o,𝔤_\alpha ]=0`$. We see finally that $`X_o`$ belongs to the center of $`𝔤`$, and that implies $`X_o=0`$ since $`G`$ has a finite center. We have proved that $`\stackrel{}{\mathrm{\Delta }}^{}=0`$, or equivalently $`\stackrel{}{\mathrm{\Delta }}=𝔱`$. $`\mathrm{}`$

Proof of Lemma 5.5. The symplectic cross-section Theorem  asserts that the pre-image $`𝒴:=\mathrm{\Phi }^1(\mathrm{interior}(𝔱_+^{}))`$ is a symplectic submanifold provided with a Hamiltonian action of $`T`$. The restriction $`\mathrm{\Phi }|_𝒴`$ is the moment map for the $`T`$-action on $`𝒴`$. Moreover, the set $`K.𝒴`$ is a $`K`$-invariant neighborhood of $`K\lambda `$ in $`M`$ diffeomorphic to $`K\times _T𝒴`$. Since $`\lambda `$ is a fixed $`T`$-point of $`𝒴`$, a Hamiltonian model for $`(𝒴,\omega |_𝒴,\mathrm{\Phi }|_𝒴)`$ in a neighborhood of $`\lambda `$ is $`(\text{T}_\lambda 𝒴,\omega _\lambda ,\mathrm{\Phi }_\lambda )`$ where $`\omega _\lambda `$ is the linear symplectic form of the tangent space $`\text{T}_\lambda M`$ restricted to $`\text{T}_\lambda 𝒴`$, and $`\mathrm{\Phi }_\lambda :\text{T}_\lambda 𝒴𝔱^{}`$ is the unique moment map with $`\mathrm{\Phi }_\lambda (0)=\lambda `$. A small computation shows that $`x\lambda \mathrm{ad}(x)`$ is an isomorphism from $`𝔭`$ to $`\text{T}_\lambda 𝒴`$, and $`\mathrm{\Phi }_\lambda (x)=\lambda \frac{1}{2}pr_𝔱^{}(\lambda \mathrm{ad}(x)\mathrm{ad}(x))`$. $`\mathrm{}`$

We still denote the almost complex structure transported on $`\stackrel{~}{𝒰}K\times _T𝔭`$ through $`\mathrm{{\rm Y}}`$ by $`J`$. Since $`d\mathrm{{\rm Y}}(\lambda )`$ is the identity, $`J_{[1,0]}:𝔯𝔭𝔯𝔭`$ is equal to $`J_\lambda `$. Let $`\pi :K\times _T𝔭K/T`$, and $`\pi _{\stackrel{~}{𝒰}}:\stackrel{~}{𝒰}K/T`$ be the fibering maps. Remark that for any equivariant vector bundle $`V`$ over $`M`$ the vector bundle $`(\mathrm{{\rm Y}}^1)^{}(V|_𝒰)\stackrel{~}{𝒰}`$ is isomorphic to $`\pi _{\stackrel{~}{𝒰}}^{}(K\times _TV_\lambda )`$. At this stage, we have according to Definition 3.2

(5.50)
$$RR_\mathrm{\Phi }^^K(G\lambda ,V)=\text{Index}_{\stackrel{~}{𝒰}}^K\left(\text{Thom}__K^{\stackrel{~}{\mathrm{\Phi }}}(\stackrel{~}{𝒰},J)\pi _{\stackrel{~}{𝒰}}^{}(K\times _TV_\lambda )\right).$$

With the help of Lemma 3.4, we define now a simpler representative of the class defined by $`\text{Thom}__K^{\stackrel{~}{\mathrm{\Phi }}}(\stackrel{~}{𝒰},J)`$ in $`\text{K}_K(\text{T}_K\stackrel{~}{𝒰})`$. Consider the map

$`\underset{¯}{\lambda }:K\times _T𝔭`$ $``$ $`𝔨^{}`$
$`(k,x)`$ $``$ $`k\lambda ,`$
and let $`\underset{¯}{\lambda }_{\stackrel{~}{M}}`$ be the vector field on $`\stackrel{~}{M}`$ generated $`\underset{¯}{\lambda }`$ (see (3.8)). Note that $`\underset{¯}{\lambda }_{\stackrel{~}{M}}`$ never vanishes outside the zero section of $`K\times _T𝔭`$. Let $`(,)_{_{\stackrel{~}{M}}}`$ be the Riemannian metric on $`\stackrel{~}{M}`$ defined by $`(V,V^{})_{_{\stackrel{~}{M}}}=(X,X^{})+(v,v^{})`$ for $`V=[k,x;X+v]`$, $`V=[k,x;X^{}+v^{}]`$. A small computation shows that

$$(\stackrel{~}{},\underset{¯}{\lambda }_{\stackrel{~}{M}})_{_{\stackrel{~}{M}}}=\underset{¯}{\lambda }_{\stackrel{~}{M}}^2+o(\underset{¯}{\lambda }_{\stackrel{~}{M}}^2)$$

in the neighborhood of the zero section in $`K\times _T𝔭`$. Hence, if we take $`\stackrel{~}{𝒰}`$ small enough, $`(\stackrel{~}{},\underset{¯}{\lambda }_{\stackrel{~}{M}})_{_{\stackrel{~}{M}}}>0`$ on $`\stackrel{~}{𝒰}\{\mathrm{zero}\mathrm{section}\}`$, hence $`\text{Thom}__K^{\stackrel{~}{\mathrm{\Phi }}}(\stackrel{~}{𝒰},J)=\text{Thom}__K^{\underset{¯}{\lambda }}(\stackrel{~}{𝒰},J)`$ in $`\text{K}_K(\text{T}_K\stackrel{~}{𝒰})`$ (see Lemma 3.4).

Let $`\stackrel{~}{J}`$ be the $`K`$-invariant almost complex structure on $`\stackrel{~}{M}`$, constant on the fibers of $`\stackrel{~}{M}K/T`$, and equal to $`J_\lambda `$ at $`[1,0]`$ so that for $`[k,x]K\times _T𝔭`$, $`\stackrel{~}{J}_{[k,x]}(V)=`$ $`[k,x,J_\lambda (X+v)]`$ for $`V=[k,x,X+v]`$. Since the set $`\{\underset{¯}{\lambda }_{\stackrel{~}{M}}=0\}=K/T`$ is compact, using $`\stackrel{~}{J}`$ and the map $`\underset{¯}{\lambda }`$, one defines the localized Thom symbol

$$\text{Thom}__K^{\underset{¯}{\lambda }}(\stackrel{~}{M},\stackrel{~}{J})\text{K}_K(\text{T}_K\stackrel{~}{M}).$$

Through the canonical identification of the tangent spaces at $`[k,x]`$ and $`[k,0]`$, one can write $`\stackrel{~}{J}_{[k,x]}=\stackrel{~}{J}_{[k,0]}=J_{[k,0]}`$ for any $`[k,x]\stackrel{~}{𝒰}`$. We note that $`J`$ and $`\stackrel{~}{J}`$ are related on $`\stackrel{~}{𝒰}`$ by the homotopy $`J^t`$ of almost complex structures: $`J_{[k,x]}^t:=J_{[k,tx]}`$ for $`[k,x]\stackrel{~}{𝒰}`$. From Lemma 3.4, we conclude that the localized Thom symbols $`\text{Thom}__K^{\underset{¯}{\lambda }}(\stackrel{~}{𝒰},J)`$ and $`\text{Thom}__K^{\underset{¯}{\lambda }}(\stackrel{~}{M},\stackrel{~}{J})|_{\stackrel{~}{𝒰}}`$ define the same class in $`\text{K}_K(\text{T}_K\stackrel{~}{𝒰})`$, thus (5.50) becomes

(5.51)
$$RR_\mathrm{\Phi }^^K(G\lambda ,V)=\text{Index}_{\stackrel{~}{M}}^K\left(\text{Thom}__K^{\underset{¯}{\lambda }}(\stackrel{~}{M},\stackrel{~}{J})\pi ^{}(K\times _TV_\lambda )\right).$$

In order to compute (5.51), we use the induction morphism

$$i_{}:\text{K}_T(\text{T}_T𝔭)\text{K}_K(\text{T}_K(K\times _T𝔭))$$

defined by Atiyah in  (see \[Section 3\]). The map $`i_{}`$ enjoys two properties: first, $`i_{}`$ is an isomorphism and the $`K`$-index of $`\sigma \text{K}_K(\text{T}_K(K\times _T𝔭))`$ can be computed with the $`T`$-index of $`(i_{})^1(\sigma )`$.

Let $`\sigma :p^{}(E^+)p^{}(E^{})`$ be a $`K`$-transversally elliptic symbol on $`K\times _T𝔭`$, where $`p:\text{T}(K\times _T𝔭)K\times _T𝔭`$ is the projection, and $`E^+,E^{}`$ are equivariant vector bundles over $`K\times _T𝔭`$. So for any $`[k,x]K\times _T𝔭`$, we have a collection of linear maps $`\sigma ([k,x,X+v]):E_{[k,x]}^+E_{[k,x]}^{}`$ depending on the tangent vectors $`X+v`$. The symbol $`(i_{})^1(\sigma )`$ is defined by

(5.52)
$$(i_{})^1(\sigma )(x,v)=\sigma ([1,x,0+v]):E_{[1,x]}^+E_{[1,x]}^{}\mathrm{for}\mathrm{any}(x,v)\text{T}𝔭.$$

For $`\sigma =\text{Thom}__K^{\stackrel{~}{\mathrm{\Phi }}}(\stackrel{~}{M},\stackrel{~}{J})`$, the vector bundle $`E^+`$ (resp. $`E^{}`$) is $`_{}^{odd}\text{T}\stackrel{~}{M}`$ (resp. $`_{}^{even}\text{T}\stackrel{~}{M}`$). Since the complex structure leaves $`𝔯𝔨/𝔱`$ and $`𝔭`$ invariant one gets

$$(i_{})^1(\text{Thom}__K^{\underset{¯}{\lambda }}(\stackrel{~}{M},\stackrel{~}{J}))=\text{Thom}__T^\lambda (𝔭,J_\lambda )_{}^{}𝔨/𝔱,$$

and

(5.53)
$$(i_{})^1\left(\text{Thom}__K^{\underset{¯}{\lambda }}(\stackrel{~}{M},\stackrel{~}{J})\pi ^{}(K\times _TV_\lambda )\right)=\text{Thom}__T^\lambda (𝔭,J_\lambda )V_\lambda _{}^{}𝔨/𝔱,$$

where $`\text{Thom}__T^\lambda (𝔭,J_\lambda )`$ is the $`T`$-equivariant Thom symbol on the complex vector space $`(𝔭,J_\lambda )`$ deformed by the constant map $`𝔭𝔱,x\lambda `$. In (5.53), our notation uses the structure of $`R(T)`$-module for $`\text{K}_T(\text{T}_T𝔭)`$, hence we can multiply $`\text{Thom}__T^\lambda (𝔭,J_\lambda )`$ by $`V_\lambda _{}^{}𝔨/𝔱`$.

Theorem 4.1 of Atiyah in  tells us that

(5.54)
is a commutative diagram, with $`\stackrel{~}{M}=K\times _T𝔭`$, and where $`\mathrm{Ind}__T^^K`$ is the induction map (see (4.39)). In other words, $`\text{Index}_{\stackrel{~}{M}}^K(\sigma )=\mathrm{Ind}__T^^K(\text{Index}_𝔭^T((i_{})^1(\sigma )))`$. With (5.51), (5.53), and (5.54), we find

$`RR_\mathrm{\Phi }^^K(G\lambda ,V)`$ $`=`$ $`\mathrm{Ind}__T^^K\left(\text{Index}_𝔭^T(\text{Thom}__T^\lambda (𝔭,J_\lambda ))V_\lambda _{}^{}𝔨/𝔱\right)`$
$`=`$ $`\mathrm{Hol}__T^^K\left(\text{Index}_𝔭^T(\text{Thom}__T^\lambda (𝔭,J_\lambda ))V_\lambda \right).`$
(See the Appendix in  for the relation $`\mathrm{Hol}__T^^K()=\mathrm{Ind}__T^^K(_{}^{}𝔨/𝔱)`$.) But the index $`\text{Index}_𝔭^T(\text{Thom}__T^\lambda (𝔭,J_\lambda ))`$ is computed in Section 5 of :

$`\text{Index}_𝔭^T(\text{Thom}__T^\lambda (𝔭,J_\lambda ))`$ $`=`$ $`\left[\mathrm{\Pi }_{\alpha _n^+}(1t^\alpha )\right]_\lambda ^1`$
$`=`$ $`(1)^rt^{2\rho _n}\left[\mathrm{\Pi }_{\alpha _n^+}(1t^\alpha )\right]_\lambda ^1,`$
with $`r=\frac{1}{2}dim(G/K)`$. Equality (5.49) in then proved.

### 5.3. $`(G\lambda ,\mathrm{\Phi })`$ satisfies Assumption 3.6

Let $`M`$ be a regular elliptic coadjoint orbit for $`G`$, with the canonical Hamiltonian $`K`$-action. The goal of this section is to show that $`M`$ satisfies Assumption 3.6 at every $`\mu `$.

Let $`𝔤=𝔨𝔭`$ be the Cartan decomposition of $`𝔤`$. The Killing form $`B`$ provides a $`G`$-equivariant identification $`𝔤𝔤^{}`$ and $`K`$-equivariant identifications $`𝔨𝔨^{},𝔭𝔭^{}`$. The Killing form $`B`$ provides also a $`K`$-invariant Euclidean structure on $`𝔤`$ such that $`B(X,X)=X_1^2+X_2^2`$ and $`X^2=X_1^2+X_2^2`$, for $`X=X_1+X_2`$, with $`X_1𝔨,X_2𝔭`$.

Hence we can and we shall consider $`M`$ as a adjoint orbit of $`G`$: $`M=G\lambda `$ where $`\lambda 𝔨`$ is a regular element, i.e. $`G_\lambda =K_\lambda `$ is a maximal torus in $`K`$ (in this section $``$ means the adjoint action). The moment map $`\mathrm{\Phi }:M𝔨`$ is then the restriction to $`M`$ of the orthogonal projection $`𝔨𝔭𝔨`$. For $`\mu 𝔨`$, we consider the map $`\mathrm{\Phi }_\mu :M\times K\mu 𝔨,(m,n)\mathrm{\Phi }(m)n`$.

This section is devoted to the proof of the following

###### Proposition 5.7.

The set $`\text{Cr}(\mathrm{\Phi }_\mu ^2)`$ of critical points of $`\mathrm{\Phi }_\mu ^2`$ is a compact subset of $`M\times K\mu `$. More precisely, for any $`r0`$, there exists $`c(r)>0`$ such that

$$\text{Cr}(\mathrm{\Phi }_\mu ^2)\left(M\left\{\xi 𝔤,\xi c(r)\right\}\right)\times K\mu ,$$

whenever $`\mu r`$.

Note that $`M\{\xi 𝔤,\xi c(r)\}`$ is compact, thus Proposition 5.7 shows that $`M`$ satisfies Assumption 3.6 at every $`\mu `$. Let $`𝔞`$ be a maximal Abelian subalgebra of $`𝔭`$, and consider the map

$$F^\mu :(K\lambda )\times (K\mu )\times 𝔞$$

defined by $`F^\mu (m,n,X)=\frac{1}{2}e^Xm^22<e^Xm,n>`$.

###### Proposition 5.8.

For any $`r0`$, there exists $`c(r)>0`$ such that

$$(m,n,X)\text{Cr}(F^\mu )e^Xmc(r),$$

whenever $`\mu r`$.

We first show that Proposition 5.8 implies Proposition 5.7, and then we concentrate on the proof of Proposition 5.8.

Proposition 5.8$``$ Proposition 5.7. Consider the map $`\mathrm{\Phi }\mu :M𝔨`$. One easily sees that $`\text{Cr}(\mathrm{\Phi }_\mu ^2)=K(\text{Cr}(\mathrm{\Phi }_\mu ^2)(M\times \{\mu \}))`$, and $`\text{Cr}(\mathrm{\Phi }_\mu ^2)(M\times \{\mu \})\text{Cr}(\mathrm{\Phi }\mu ^2)\times \{\mu \}`$. Thus Proposition 5.7 is proved if one shows that for any $`r0`$, there exists $`c(r)>0`$ such that

$$\text{Cr}(\mathrm{\Phi }\mu ^2)M\left\{\xi 𝔤,\xi c(r)\right\},$$

whenever $`\mu r`$. Since the bilinear form $`B`$ is $`G`$-invariant, the map $`mB(m,m)`$ is constant on $`M`$, equal to $`\lambda ^2`$, and thus $`\mathrm{\Phi }(m)^2=\frac{1}{2}m^2+\frac{1}{2}\lambda ^2`$ for any $`mM`$. Finally we have on $`M`$ the equality

$$\mathrm{\Phi }(m)\mu ^2=\frac{1}{2}m^22<m,\mu >+\mathrm{cst}$$

where $`\mathrm{cst}=\frac{1}{2}\lambda ^2+\mu ^2`$. If we use now the Cartan decomposition $`G=K\mathrm{exp}(𝔭)`$, and the fact that $`𝔭=_{kK}k𝔞`$, we see that every element $`M`$ is of the form $`m=(k_1^1e^Xk_2)\lambda `$ with $`k_1,k_2K`$ and $`X𝔞`$. It follows that $`\mathrm{\Phi }((k_1^1e^Xk_2)\lambda )\mu ^2=F^\mu (m^{},n,X)+\mathrm{cst}`$ with $`m^{}=k_2\lambda `$, $`n=k_1\mu `$. It is now obvious that if $`m=(k_1^1e^Xk_2)\lambda \text{Cr}(\mathrm{\Phi }\mu ^2)`$ then $`(k_2\lambda ,k_1\mu ,X)\text{Cr}(F^\mu )`$. Finally, if Proposition 5.8 holds we get $`m=e^Xm^{}c(r)`$. $`\mathrm{}`$

Proof of Proposition 5.8. Let $`(m,n,X)\text{Cr}(F^\mu )`$. Then, the identity $`\frac{d}{dt}F^\mu (m,n,X+tX)|_{t=0}=0`$ gives

(5.55)
$$<e^Xm,e^X[X,m]>=2<e^X[X,m],n>.$$

The proof of Proposition 5.8 is then reduced to the

###### Lemma 5.9.

* For any $`r0`$, there exists $`d(r)>0`$ such that $`e^X[X,m]d(r)X`$ holds for every $`(m,n,X)K\lambda \times 𝔨\times 𝔞`$ satisfying (5.55) and $`nr`$.
* For any $`d>0`$ there exists $`c>0`$, such that for every $`(m,X)K\lambda \times 𝔞`$, we have $`e^X[X,m]dXe^Xmc`$.

Proof of i). Let $`\mathrm{\Sigma }`$ be the set of weights for the adjoint action of $`𝔞`$ on $`𝔤`$: $`𝔤=_{\alpha \mathrm{\Sigma }}𝔤_\alpha `$, where $`𝔤_\alpha =\{Z𝔤,[X,Z]=\alpha (X)Z\mathrm{for}\mathrm{all}X𝔞\}`$. Each $`m𝔤`$ amits a decomposition $`m=_\alpha m_\alpha `$, with $`m_\alpha 𝔤_\alpha `$, which is stable relatively to the Cartan involution:

(5.56)
$$\theta (m_\alpha )=m_\alpha ,\mathrm{for}\mathrm{every}m𝔤.$$

Suppose now that $`v:=(m,n,X)K\lambda \times 𝔨\times 𝔞`$ satisfies (5.55). We decompose $`mK\lambda `$ into $`m=_\alpha m_\alpha `$ with $`m_\alpha 𝔤_\alpha `$. Let $`\mathrm{\Sigma }_v^\pm :=\{\alpha \mathrm{\Sigma },m_\alpha 0\mathrm{and}\pm \alpha (X)>0\}`$. The LHS of (5.55) decomposes in $`\mathrm{LHS}=_\alpha e^{2\alpha (X)}\alpha (X)m_\alpha ^2`$, and

$`\mathrm{LHS}`$ $`=`$ $`{\displaystyle \underset{\alpha \mathrm{\Sigma }_v^+}{}}e^{2\alpha (X)}\alpha (X)m_\alpha ^2+{\displaystyle \underset{\alpha \mathrm{\Sigma }_v^{}}{}}e^{2\alpha (X)}\alpha (X)m_\alpha ^2`$
$``$ $`{\displaystyle \underset{\alpha \mathrm{\Sigma }_v^+}{}}e^{2\alpha (X)}{\displaystyle \frac{\alpha (X)^2}{RX}}m_\alpha ^2RX{\displaystyle \underset{\alpha \mathrm{\Sigma }_v^{}}{}}m_\alpha ^2[1]`$
with $`R:=sup_{\alpha ,X1}|\alpha (X)|`$. But

$`{\displaystyle \underset{\alpha \mathrm{\Sigma }_v^+}{}}e^{2\alpha (X)}\alpha (X)^2m_\alpha ^2`$ $`=`$ $`e^X[X,m]^2{\displaystyle \underset{\alpha \mathrm{\Sigma }_v^{}}{}}e^{2\alpha (X)}\alpha (X)^2m_\alpha ^2`$
$``$ $`e^X[X,m]^2R^2X^2{\displaystyle \underset{\alpha \mathrm{\Sigma }_v^{}}{}}m_\alpha ^2[2]`$
Since $`\alpha \mathrm{\Sigma }_v^+\alpha \mathrm{\Sigma }_v^{}`$, we have $`2_{\alpha \mathrm{\Sigma }_v^{}}m_\alpha ^2_{\alpha \mathrm{\Sigma }}m_\alpha ^2=m^2`$$`=\lambda ^2`$. So, the inequalities $`[1]`$ and $`[2]`$ give

(5.57)
$$\mathrm{LHS}\frac{e^X[X,m]^2}{RX}RX.\lambda ^2.$$

Since the RHS of (5.55) satisfies obviously $`\mathrm{RHS}2e^X[X,m].n`$, (5.55) and (5.57) yield

$$2e^X[X,m].n\frac{e^X[X,m]^2}{RX}RX.\lambda ^2.$$

In other words $`E:=e^X[X,m]`$ satisfies the polynomial inequality $`E^22aEb^20`$, with $`b=RX.\lambda `$ and $`a=RX.n`$. A direct computation gives

$$e^X[X,m]dX,$$

with $`d=R(n+\sqrt{n^2+\lambda ^2})`$. $`\mathrm{}`$

Proof of ii). Suppose that $`ii)`$ does not hold. So there is a sequence $`(m_i,X_i)_i`$ in $`K\lambda \times 𝔞`$ such that $`e^{X_i}[X_i,m_i]dX_i`$ but $`lim_i\mathrm{}e^{X_i}m_i=\mathrm{}`$. We write $`X_i=t_iv_i`$ with $`t_i0`$ and $`v_i=1`$. We can assume moreover that $`v_iv_{\mathrm{}}𝔞`$ with $`v_{\mathrm{}}=1`$, and $`m_im_{\mathrm{}}K\lambda `$ when $`i\mathrm{}`$.

But $`m_{\mathrm{}}K\lambda `$ is a regular element of $`G`$, and rank$`(G)=`$rank$`(K)`$, thus $`[v_{\mathrm{}},m_{\mathrm{}}]=_{\alpha \mathrm{\Sigma }}\alpha (v_{\mathrm{}})m_{\mathrm{},\alpha }0`$: there exists $`\alpha _o\mathrm{\Sigma }`$ such that $`\alpha _o(v_{\mathrm{}})m_{\mathrm{},\alpha _o}0`$, and then also $`\alpha _o(v_{\mathrm{}})m_{\mathrm{},\alpha _o}0`$ (see (5.56)).

On one hand the sequence $`e^{t_iv_i}.m_i=_\alpha e^{t_i\alpha (v_i)}m_{i,\alpha }`$ diverges. Hence $`(t_i)_i`$ is not bounded and so can be assumed to be divergent. On the other hand $`e^{t_iv_i}[v_i,m_i]=_\alpha e^{t_i\alpha (v_i)}\alpha (v_i)m_{i,\alpha }`$ is bounded, so the sequences $`e^{t_i\pm \alpha _o(v_i)}\alpha (v_i)m_{i,\pm \alpha _o}`$ are also bounded. But $`lim_i\mathrm{}\alpha (v_i)m_{i,\pm \alpha _o}=\alpha _o(v_{\mathrm{}})m_{\mathrm{},\pm \alpha _o}`$ $`0`$, hence the sequences $`e^{t_i\pm \alpha _o(v_i)}`$ are bounded. This contradicts the fact that $`lim_i\mathrm{}t_i=+\mathrm{}`$ and $`lim_i\mathrm{}\alpha _o(v_i)0`$. $`\mathrm{}`$