text:
# Pairing Instability in a Nematic Fermi Liquid

## Introduction

The occurrence of the novel anisotropic metallic states in strongly correlated electron systems has been demonstrated by the discoveries of the stripe phases in Cuprates tranquada ; kivelson99  and the anisotropic metallic states in high Landau levels of two dimensional electron systems in the quantum Hall regime eisenstein99 ; stormer99 ; cdw . The anisotropic states in these systems are close to the unidirectional charge-density-wave states and arise due to strong electron-electron interactions. In analogy to classical liquid crystals, nematic Fermi liquid state has been proposed as a primary candidate for the explanation of the quantum liquid states mentioned above fradkin99 ; macdonald00 ; wexler01 ; leo01 . In nematic Fermi liquids, the rotational symmetry is spontaneously broken while the translational symmetry is still preserved. There are two approaches to get nematic Fermi liquid states. One may consider first a smectic Fermi liquid state where both the rotational and translational symmetries are spontaneously broken fradkin99 ; macdonald00 . The melting of the smectic state via the proliferation of appropriate topological defects will lead to a nematic Fermi liquid state. Another approach is to introduce an interaction in ordinary Fermi liquid state such that the rotational symmetry will be spontaneously broken upon the tunning of the interaction.

Due to the broken translational symmetry, the smectic state is often visualized as a liquid version of a uni-directional crystalline state fradkin99 ; macdonald00 ; wexler01 ; leo01 . The low energy excitations come from the one-dimensional electronic degrees of freedom that reside along the stripes in Cuprates or along the edges of two different quantum Hall states in high Landau levels of two dimensional electron systems. The existence of the fermionic degrees of freedom adds a major difficulty in describing the melting of the smectic state to a nematic Fermi liquid state. So far, no well-defined procedure has been discovered in this direction. On the other hand, the second approach has been proved to be useful. Oganesyan, Kivelson and Fradkin vadim01  introduced a two-body interaction between quadrupolar densities of spinless fermions. Upon the tunning of the interaction, the rotational symmetry is spontaneously broken. The Goldstone mode of the nematic state is the director mode that corresponds to the angular rotation of the principal axis of the distorted Fermi surface. Its dispersion relation is highly anisotropic in space in contrast to the case of the broken spin symmetries. The interaction between the fermions and the anisotropic Goldstone mode leads to anisotropic self-energy of the quasiparticles that show non-Landau-Fermi liquid properties in certain directions. That is, the life time of the quasiparticles strongly depend on the spatial directions.

In this paper, we follow the second approach and investigate a pairing instability of a nematic Fermi liquid of two species of fermions. Two species represent either the spin degrees of freedom or the layer degrees of freedom of the bilayer system. Introducing an additional interaction between quadrupolar densities of two degrees of freedom, we find a nematic Fermi liquid state where there are two director modes that corresponds to the in-phase and out-of-phase modes of the angular rotations of two Fermi surfaces. In general, the out-of-phase mode is gapped while the in-phase mode remains gapless. It is found that the Goldstone mode of the nematic Fermi liquid state of the spinless fermions always mediates a repulsive interaction. On the other hand, in the case of the two species of fermions, the out-of-phase mode mediates an attractive interaction between fermions that belong to different species while the in-phase mode leads to a repulsive interaction. Since the out-of-phase mode is gapped, one expects that the in-phase mode will dominate in the low energy limit and the interaction between the fermions will be repulsive. In the presence of an external potential like a lattice potential, however, the in-phase mode will be also gapped. We find in this case that the combined total interaction due to the in-phase and out-of-phase modes can be attractive so that there will be a pairing instability between the fermions of different species.

## Choice of the Hamiltonian

We consider the following Hamiltonian for a two dimensional fermion system with two quantum numbers, $``$ and $``$.

$`H`$ $`=`$ $`{\displaystyle d^2r\underset{\alpha }{}\widehat{\mathrm{\Psi }}_\alpha ^{}(𝐫)ϵ(i)\widehat{\mathrm{\Psi }}_\alpha (𝐫)}`$ (1)
$`+`$ $`{\displaystyle d^2rd^2r^{}F_2(𝐫𝐫^{})\underset{\alpha }{}\mathrm{Tr}[\widehat{Q}_\alpha (𝐫)\widehat{Q}_\alpha (𝐫^{})]}`$ (2)
$`+`$ $`{\displaystyle d^2rd^2r^{}G_2(𝐫𝐫^{})\underset{\alpha \beta }{}\mathrm{Tr}[\widehat{Q}_\alpha (𝐫)\widehat{Q}_\beta (𝐫^{})]},`$ (3)
where $`\alpha ,\beta =,`$, $`\widehat{\mathrm{\Psi }}_\alpha `$ is an annihilation operator of the fermion with “spin” $`\alpha `$, and $`ϵ(i)`$ is the kinetic energy operator. Here the matrix quadrupolar density of the fermions is defined as

$$\widehat{Q}_\alpha (𝐫)=\frac{1}{k_F^2}\widehat{\mathrm{\Psi }}_\alpha ^{}(𝐫)\left(\begin{array}{cc}_x^2_y^2& 2_x_y\\ 2_x_y& _y^2_x^2\end{array}\right)\widehat{\mathrm{\Psi }}_\alpha (𝐫)$$
(4)
The interactions, $`F_2(𝐪)`$ between the fermions with the same “spin” and $`G_2(𝐪)`$ between the fermions with different “spin”, are given by

$$F_2(𝐪)=\frac{1}{F_2^1+\kappa _1q^2},G_2(𝐪)=\frac{1}{G_2^1+\kappa _2q^2}.$$
(5)
The details of the form of the interaction do not matter as far as the short range interaction is concerned.

Using a Hubbard-Stratonovich transformation, we can write the effective Lagrangian as

$``$ $`=`$ $`{\displaystyle d^2r\underset{\alpha }{}\mathrm{\Psi }_\alpha ^{}(𝐫,\tau )(_\tau +ϵ(i)\mu )\mathrm{\Psi }_\alpha (𝐫,\tau )}`$ (6)
$`+`$ $`{\displaystyle \frac{d^2q}{(2\pi )^2}F_2(𝐪)[1+g(𝐪)]\mathrm{Tr}[Q_+(𝐪,\tau )N_+(𝐪,\tau )]}`$ (7)
$`+`$ $`{\displaystyle \frac{d^2q}{(2\pi )^2}F_2(𝐪)[1g(𝐪)]\mathrm{Tr}[Q_{}(𝐪,\tau )N_{}(𝐪,\tau )]}`$ (8)
$`+`$ $`{\displaystyle \frac{d^2q}{(2\pi )^2}F_2(𝐪)[1+g(𝐪)]\mathrm{Tr}[N_+(𝐪,\tau )N_+(𝐪,\tau )]}`$ (9)
$`+`$ $`{\displaystyle \frac{d^2q}{(2\pi )^2}F_2(𝐪)[1g(𝐪)]\mathrm{Tr}[N_{}(𝐪,\tau )N_{}(𝐪,\tau )]},`$ (10)
where $`Q_\pm (𝐪,\tau )=Q_{}(𝐪,\tau )\pm Q_{}(𝐪,\tau )`$ with

$`Q_\alpha (𝐪,\tau )={\displaystyle \underset{𝐤}{}}\mathrm{\Psi }_{\alpha ,𝐤𝐪}^{}(\tau )\left(\begin{array}{cc}\mathrm{cos}(2\theta _𝐤)& \mathrm{sin}(2\theta _𝐤)\\ \mathrm{sin}(2\theta _𝐤)& \mathrm{cos}(2\theta _𝐤)\end{array}\right)\mathrm{\Psi }_{\alpha ,𝐤}(\tau )`$ (11)
and $`\theta _𝐤`$ is the angle between $`𝐤`$ and the $`x`$-axis. Here $`g(𝐪)=G_2(𝐪)/F_2(𝐪)`$ and $`N_\pm (𝐪,\tau )`$ are the Hubbard-Stratonovich matrix fields given by

$$N_\pm (𝐪,\tau )=\left(\begin{array}{cc}N_{\pm ,1}(𝐪,\tau )& N_{\pm ,2}(𝐪,\tau )\\ N_{\pm ,2}(𝐪,\tau )& N_{\pm ,1}(𝐪,\tau )\end{array}\right).$$
(12)
Notice that, at the saddle point of the action, $`N_\pm ^{sp}(𝐪)`$ corresponds to $`\widehat{Q}_\pm (𝐪)`$.

## Saddle Point and Gaussian Fluctuations

Now one can integrate out the fermionic degrees of freedom and get an effective action in terms of $`N_+(𝐪)`$ and $`N_{}(𝐪)`$. In the putative nematic liquid state, there should be a uniform saddle point solution $`N_\pm ^{sp}0`$. Notice that non-zero $`N_{+,1}^{sp}`$ ($`N_{,1}^{sp}`$) will lead to the symmetric (antisymmetric) distortions of the $``$ and $``$ Fermi surfaces. The finite value of $`N_{+,2}^{sp}`$ ($`N_{,2}^{sp}`$) determines the amount of the symmetric (antisymmetric) rotations of the principal axis of the $``$ and $``$ Fermi surfaces. The saddle point solution depends on the coefficient of the quadratic term in the long wavelength ($`𝐪0`$) expansion of the Landau-Ginzburg free energy.

$`(N_+,N_{})=M_+\mathrm{Tr}[N_+^2]+M_{}\mathrm{Tr}[N_{}^2]+\mathrm{},`$ (13)
where $`\mathrm{}`$ represents the quartic and higher order terms. One can show that

$`M_+`$ $`=`$ $`2(1+g)F_2+(1+g)^2N(0)F_2^2`$ (14)
$`M_{}`$ $`=`$ $`2(1g)F_2+(1g)^2N(0)F_2^2,`$ (15)
where $`g=g(𝐪=0)`$ and $`N(0)`$ is the density of states at the Fermi level. We will choose $`F_2>0`$ and $`G_2<0`$ such that $`M_+<0`$. One can see that $`M_{}>0`$ in this case. Then the saddle point solution, $`N_+^{sp}0`$ and $`N_{}^{sp}=0`$, represents the symmetric distortions and rotations of the $``$ and $``$ Fermi surfaces.

The Gaussian fluctuations of $`N_\pm (𝐪)=N_\pm ^{sp}+\delta N_\pm (𝐪)`$ about the nematic liquid ground state and their interaction with the fermions can be written as ($`a=1,2`$)

$`_{\mathrm{eff}}`$ $`=`$ $`_{\mathrm{sp}}+_{\mathrm{int}}+_g,`$ (16)
$`_{\mathrm{int}}`$ $`=`$ $`{\displaystyle }{\displaystyle \frac{d^2q}{(2\pi )^2}}\{F_2(1+g)\mathrm{Tr}[Q_+(𝐪,\tau )\delta N_+(𝐪,\tau )]`$ (18)
$`+F_2(1g)\mathrm{Tr}[Q_{}(𝐪,\tau )\delta N_{}(𝐪,\tau )]\}+c.c.`$
$`_g`$ $`=`$ $`{\displaystyle \frac{d^2q}{(2\pi )^2}\delta N_{+,a}(𝐪,\tau )D_{+,ab}^1(𝐪,\tau )\delta N_{+,b}(𝐪,\tau )}`$ (19)
$`+`$ $`{\displaystyle \frac{d^2q}{(2\pi )^2}\delta N_{,a}(𝐪,\tau )D_{,ab}^1(𝐪,\tau )\delta N_{,b}(𝐪,\tau )},`$ (20)
where $`_{sp}`$ describes the mean field nematic liquid ground state.

The propagators of the Gaussian fluctuations, $`D_{\pm ,ab}^1`$ can be obtained at the Gaussian level (Random Phase Approximation) as follows.

$$D_{\pm ,ab}^1(𝐪,i\nu _n)=\mathrm{\Delta }_{\pm ,ab}+(1\pm g)^2F_2^2\mathrm{\Pi }_{ab}(𝐪,i\nu _n),$$
(21)
with

$$\mathrm{\Delta }_{\pm ,ab}=\left(\begin{array}{cc}\mathrm{\Delta }_{\pm ,1}& 0\\ 0& \mathrm{\Delta }_{\pm ,2}\end{array}\right),$$
(22)
and

$`\mathrm{\Pi }_{ab}(𝐪,i\nu _n)`$ $`=`$ $`N(0){\displaystyle \frac{d\theta }{2\pi }\frac{s}{s\mathrm{cos}(\theta \varphi )}}`$ (23)
$`\times `$ $`\left(\begin{array}{cc}2\mathrm{cos}^2(2\theta )& \mathrm{sin}(4\theta )\\ \mathrm{sin}(4\theta )& 2\mathrm{cos}^2(2\theta )\end{array}\right),`$ (24)
where $`\varphi `$ is the angle between $`𝐪`$ and the $`x`$-axis. Here $`\nu _n=2n\pi T`$ is the Matsubara frequency and $`s=i\nu _n/v_Fq`$. $`\mathrm{\Delta }_{\pm ,1}`$ and $`\mathrm{\Delta }_{\pm ,2}`$ represent the gap in each collective mode, that depend on the interaction strength, but their precise forms are not important in the following analysis. In the nematic liquid state, for generic interactions without an external perturbation, we expect that there should be one gapless Goldstone mode, the in-phase director mode, associated with $`\delta N_{+,2}`$. That is, $`\mathrm{\Delta }_{+,2}=0`$. If the system experiences a weak square lattice potential or a weak quadrupolar external perturbation, even the in-phase director mode will be gapped. As will be shown later, both the in-phase ($`\delta N_{+,2}`$) and out-of-phase ($`\delta N_{,2}`$) director modes will be important in the analysis of the pairing instability. The distortion modes, $`\delta N_{+,1}`$ and $`\delta N_{,1}`$, are gapped even in the absence of the interaction between $``$ and $``$ fermions, thus one expects that the gap of these modes will be even bigger. We will drop the distortion modes in the analysis from now on.

The propagators of two director modes in the long wavelength and low energy limits can be obtained as

$$D_{\pm ,22}^1=\mathrm{\Delta }_{\pm ,2}+\kappa _\pm q^2+2(1\pm g)^2N(0)F_2^2\frac{|\nu _n|}{v_Fq}\mathrm{sin}^2(2\varphi )$$
(25)
where, in the absence of the external potential, $`\mathrm{\Delta }_{+,2}=0`$ and $`\mathrm{\Delta }_{,2}>0`$. When there is a weak quadrupolar external potential, we expect $`\mathrm{\Delta }_{,2}>\mathrm{\Delta }_{+,2}0`$. Here $`\kappa _+`$ and $`\kappa _{}`$ are the stiffness coefficients and their detailed dependence on the interaction strength will not be important.

## Effective Two-Body Interaction

The fermions with $``$ and $``$ quantum numbers interact each other by exchanging two director modes just like the case of the phonon mediated electron-electron interaction. The effective two-body interaction between fermions can be obtained by integrating out the director modes in the effective action given by Eq.20 and using the following form of the coupling between the director modes and the fermions,

$$(1+g)\delta N_{+,2}(𝐪)\underset{𝐤}{}\mathrm{sin}(2\theta _𝐤)\mathrm{\Psi }_{\alpha ,𝐤𝐪}^{}\mathrm{\Psi }_{\alpha ,𝐤}+(1g)\delta N_{,2}(𝐪)\underset{𝐤}{}\mathrm{sin}(2\theta _𝐤)\mathrm{\Psi }_{\alpha ,𝐤𝐪}^{}\sigma _{\alpha \beta }^z\mathrm{\Psi }_{\beta ,𝐤}.$$
(26)
After integrating out the director modes, the effective interaction between fermions is obtained as

$`S_{\mathrm{eff}}^{}`$ $`=`$ $`{\displaystyle \underset{𝐪,\nu _n}{}}{\displaystyle \underset{𝐤,\omega _m}{}}{\displaystyle \underset{𝐤^{},\omega _m^{}}{}}{\displaystyle \underset{\alpha }{}}V_1(𝐤,𝐤^{},𝐪,i\nu _n)\mathrm{\Psi }_{\alpha ,𝐤𝐪}^{}(\omega _m\nu _n)\mathrm{\Psi }_{\alpha ,𝐤^{}+𝐪}^{}(\omega _m^{}+\nu _n)\mathrm{\Psi }_{\alpha ,𝐤^{}}(\omega _m^{})\mathrm{\Psi }_{\alpha ,𝐤}(\omega _m)`$ (27)
$`+`$ $`{\displaystyle \underset{𝐪,\nu _n}{}}{\displaystyle \underset{𝐤,\omega _m}{}}{\displaystyle \underset{𝐤^{},\omega _m^{}}{}}{\displaystyle \underset{\alpha \beta }{}}V_2(𝐤,𝐤^{},𝐪,i\nu _n)\mathrm{\Psi }_{\alpha ,𝐤𝐪}^{}(\omega _m\nu _n)\mathrm{\Psi }_{\beta ,𝐤^{}+𝐪}^{}(\omega _m^{}+\nu _n)\mathrm{\Psi }_{\beta ,𝐤^{}}(\omega _m^{})\mathrm{\Psi }_{\alpha ,𝐤}(\omega _m),`$ (28)
where

$`V_1(𝐤,𝐤^{},𝐪,i\nu _n)`$ $`=`$ $`F_2^2[(1+g)^2D_{+,22}(𝐪,i\nu _n)+(1g)^2D_{,22}(𝐪,i\nu _n)]\mathrm{sin}(2\theta _𝐤)\mathrm{sin}(2\theta _𝐤^{})`$ (29)
$`V_2(𝐤,𝐤^{},𝐪,i\nu _n)`$ $`=`$ $`F_2^2[(1+g)^2D_{+,22}(𝐪,i\nu _n)(1g)^2D_{,22}(𝐪,i\nu _n)]\mathrm{sin}(2\theta _𝐤)\mathrm{sin}(2\theta _𝐤^{}).`$ (30)
## Cooper Channel and Pairing Instability

Notice that $`V_1`$ and $`V_2`$ correspond to the interactions between the fermions with the same spin and with different spins respectively. In the Cooper channel, $`𝐤^{}=𝐤`$ and $`\theta _𝐤^{}=\theta _𝐤+\pi `$, $`V_1`$ is always repulsive while $`V_2`$ can be attractive if $`|g|=|G_2|/F_2>1`$. In the long wavelength and low energy limit, $`V_2`$ in the Cooper channel can be written as (here $`\mathrm{\Delta }_+\mathrm{\Delta }_{+,2}`$ and $`\mathrm{\Delta }_{}\mathrm{\Delta }_{,2}`$)

$$V_2(𝐤,𝐤,𝐪,i\nu _n)=\frac{A\mathrm{sin}^2(2\theta _𝐤)}{C+B\frac{|\nu _n|}{v_Fq}\mathrm{sin}^2(2\varphi )},$$
(31)
where

$`A`$ $`=`$ $`F_2^2[2|g|(\mathrm{\Delta }_++\mathrm{\Delta }_{})(|g|^2+1)(\mathrm{\Delta }_{}\mathrm{\Delta }_+)]`$ (32)
$`B`$ $`=`$ $`2N(0)F_2^2[(|g|^2+1)(\mathrm{\Delta }_++\mathrm{\Delta }_{})2|g|(\mathrm{\Delta }_{}\mathrm{\Delta }_+)]`$ (33)
$`C`$ $`=`$ $`\mathrm{\Delta }_+\mathrm{\Delta }_{}.`$ (34)
One can show that $`V_2<0`$ if the following condition is satisfied.

$$\frac{(\sqrt{\mathrm{\Delta }_{}}\sqrt{\mathrm{\Delta }_+})^2}{\mathrm{\Delta }_{}\mathrm{\Delta }_+}<|g|<\frac{(\sqrt{\mathrm{\Delta }_{}}+\sqrt{\mathrm{\Delta }_+})^2}{\mathrm{\Delta }_{}\mathrm{\Delta }_+}$$
(35)
## Gap Equation and Nematic Superconductor

Using the attractive interaction $`V_2`$, the gap equation for $`\mathrm{\Delta }(𝐤)=\mathrm{\Psi }_{,𝐤}\mathrm{\Psi }_{,𝐤}`$ can be written as

$$\mathrm{\Delta }(𝐤,i\nu _n)=T\underset{𝐤^{},\nu _n^{}}{}\frac{V_2(𝐤,𝐤,𝐤𝐤^{},i\nu _ni\nu _n^{})\mathrm{\Delta }(𝐤^{},i\nu _n^{})}{(\nu _n^{})^2+\xi _𝐤^{}^2+\mathrm{\Delta }^2(𝐤^{},i\nu _n^{})}$$
(36)
Assuming that $`\mathrm{\Delta }(𝐤,i\nu _n)`$ depends mostly on the angle $`\theta _𝐤`$, we can perform the integral over $`\xi _𝐤^{}=ϵ_𝐤^{}\mu `$ and get the following results at T=0.

$$\mathrm{\Delta }(\theta _𝐤,i\nu )=N(0)\frac{d\theta }{2\pi }\frac{d\nu ^{}}{2\pi }\frac{\mathrm{\Delta }(\theta _𝐤+\theta ,i\nu ^{})}{\sqrt{(\nu ^{})^2+\mathrm{\Delta }^2(\theta _𝐤+\theta ,i\nu ^{})}}\frac{A\mathrm{sin}^2(2\theta _𝐤)}{C+B\frac{|\nu \nu ^{}|}{v_Fk_F|\theta |}\mathrm{sin}^2(2\theta _𝐤)},$$
(37)
where $`\theta =\theta _𝐤\theta _𝐤^{}`$. We also use $`qk_F|\theta |`$ and $`\mathrm{sin}^2(2\varphi )\mathrm{sin}^2(2\theta _𝐤)`$ in the long wavelength and low frequency limit. One can easily see that $`\mathrm{\Delta }(\theta _𝐤,i\nu _n)`$ vanishes at $`\theta _𝐤=0,\pm \frac{\pi }{2},\pi `$. It can be shown that the frequency dependence in $`\mathrm{\Delta }(\theta _𝐤,i\nu _n)`$ is weak near $`\theta _𝐤=0,\pm \frac{\pi }{2},\pi `$. When $`\mathrm{\Delta }(\theta _𝐤)\mathrm{sin}^2(2\theta _𝐤)E_{\mathrm{eff}}`$, we get

$$\mathrm{\Delta }(\theta _𝐤)\frac{E_{\mathrm{eff}}}{\mathrm{sin}^2(2\theta _𝐤)}\mathrm{exp}\left(\frac{1}{N(0)V_{\mathrm{eff}}\mathrm{sin}^2(2\theta _𝐤)}\right),$$
(38)
where

$`E_{\mathrm{eff}}`$ $`=`$ $`{\displaystyle \frac{2\theta _c}{e}}{\displaystyle \frac{C}{B}}v_Fk_F`$ (39)
$`V_{\mathrm{eff}}`$ $`=`$ $`{\displaystyle \frac{\theta _c}{\pi }}{\displaystyle \frac{A}{C}}.`$ (40)
Here $`\theta _c`$ is the cut-off in the integral over the angle $`\theta `$ in Eq.37. This is the asymptotically correct solution near $`\theta _𝐤=0,\pm \frac{\pi }{2},\pi `$. Thus the gap is exponentially suppressed in the four directions. We have to mention, however, that the whole angular dependence of the gap will depend on the underlying lattice structure.

## Summary and Conclusion

We study the pairing instability in nematic Fermi liquid states where the orientational order is spontaneously broken due to an electron-electron interaction. In the case of the fermions with no internal quantum number (e.g. spin polarized electrons), the director mode – the goldstone mode of the nematic order – always mediates a repulsive interaction in the Cooper channel. On the other hand, when the fermions carry two different quantum numbers (e.g. spin or the layer index of the bilayer system), two independent director modes exist. The out-of-phase (in-phase) mode mediates an attractive (a repulsive) interaction in the Cooper channel. When an external potential is applied to the system so that the in-phase mode becomes gapped, the total effective interaction can be attractive under certain conditions. We showed that this leads to a pairing instability. The resulting pairing gap has a remarkable structure. There are four nodes where the gap vanishes and the gap is exponentially suppressed near the nodes. The angular dependence near the nodes cannot be described by an analytic expansion in the angle around the nodes.

Our results may have the following implications to the physics of Cuprates and quantum Hall effect. 1) The occurrence of the superconductivity in the stripe phase of Cuprates has been a subject of great interest. In particular, the question of whether the stripes (or some dynamic generalization) are in favor of or against superconductivity has not been clearly answered kivelson99 . Our work on the pairing instability of a nematic Fermi liquid state will shed some light on this question especially because the existence of the nematic order and its (pseudo-)Goldstone mode is crucial for the effective attractive interaction and pairing instability. 2) In the lowest Landau level, the Fermi-liquid-like state HLR  of the filling factor $`\nu =1/2`$ can be described as a Fermi liquid state of the quasiparticles called composite fermions jain . There have been a number of examples of the paired quantum Hall states in the bilayer generalization, where the ground state can be described as a paired state of the composite fermions read00 ; bonesteel ; ybkim . In the half-filled high Landau levels, the ground state is a highly anisotropic metallic state very close to a unidirectional charge density wave cdw . If one can consider the nematic Fermi liquid state of the composite fermions as a candidate for the anisotropic state mentioned above vadim01 , one would expect that an unusual paired quantum Hall state may occur in high Landau levels in the bilayer system, arising from the pairing instability of the underlying nematic Fermi liquid states of the composite fermions.

Acknowledgments We thank Steve Kivelson, Leo Radzihovsky, and Vadim Oganesyan for useful discussions. This work was supported in part by Canadian Institute for Advanced Research (H.Y.K. and Y.B.K.) and Alfred P. Sloan Foundation (Y.B.K.).


format: text

text:
# Two RHIC puzzles: Early thermalization and the HBT problem

## 1 Elliptic flow as an early QGP signature

The quark-gluon plasma (QGP) is a thermalized system and, as such, has thermal pressure. If the QGP is created in a heavy-ion collision, this pressure acts against the surrounding vacuum and causes a rapid collective expansion (“flow”) of the reaction zone, the “Little Bang”. In non-central collisions the initial overlap region of the two nuclei is elliptically deformed in the transverse plane, resulting in anisotropic pressure gradients. These cause a more rapid expansion into the reaction plane than perpendicular to it, resulting in an anisotropy of the final transverse momentum distribution called elliptic flow . Elliptic flow is quantified by the second harmonic coefficient $`\mathrm{v}_2(y,p_{};b)`$ of a Fourier expansion in $`\varphi _p`$ of the measured hadron spectrum $`dN/(dyp_{}dp_{}d\varphi _p)`$ . At midrapidity in collisions between equal-mass nuclei it is the lowest-order non-zero Fourier coefficient.

Since individual nucleon-nucleon collisions produce azimuthally symmetric spectra, any final state momentum anisotropies must be generated dynamically during the nuclear reaction. A non-zero value for the elliptic flow is thus an unambiguous sign for reinteractions among the produced particles, and its magnitude characterizes their strength. Microscopic transport calculations  show a monotonic dependence of $`\mathrm{v}_2`$ on the opacity (density times scattering cross section) of the produced matter which is inversely related to its thermalization time. Elliptic flow also requires the existence of an initial spatial anisotropy of the reaction zone, by colliding either deformed nuclei at $`b=\mathrm{\hspace{0.17em}0}`$ or spherical nuclei at $`b\mathrm{\hspace{0.17em}0}`$. The transport calculations strongly suggest that, for a given initial spatial anisotropy $`ϵ_x`$, the maximum momentum-space response $`\mathrm{v}_2`$ is obtained in the hydrodynamic limit which assumes a thermalization time which is much shorter than any macroscopic time scale in the system. Any significant delay of thermalization allows the initial spatial deformation to decay without concurrent build-up of momentum anisotropies, thereby reducing the finally observed elliptic flow signal .

This specific sensitivity of the elliptic flow to rescattering and pressure build-up in the early collision stages  (before the spatial deformation and the resulting anisotropies of the pressure gradients have disappeared ) puts $`\mathrm{v}_2`$ on the list of “early signatures” of the collision dynamics. In contrast to other early probes (which use rare signals such as hard photons and dileptons, heavy quarkonia and jets), $`\mathrm{v}_2`$ can be extracted from the bulk of the measured hadrons which are very abundant and thus easily accessible. In fact, the elliptic flow measurement in Au+Au collisions at $`\sqrt{s}=\mathrm{\hspace{0.17em}130}A`$ GeV  became the second publication of RHIC data and appeared within days of the end of the first RHIC run.

We present hydrodynamic results for hadronic spectra and elliptic flow at RHIC energies. We show that the hydrodynamic approach provides an excellent quantitative description of the bulk of the data and fails only for very peripheral Au+Au collisions and/or at high $`p_{}>1.52`$ GeV/$`c`$. That the hydrodynamic approach fails for small initial overlap regions or large hadron transverse momenta is not unexpected. What is surprising is that the hydrodynamic approach does work for such a wide range of impact parameters and $`p_{}`$ and quantitatively reproduces the momenta of more than 99% of the particles: below $`p_{}=\mathrm{\hspace{0.17em}1.5}`$ GeV/$`c`$ the elliptic flow data  actually exhaust the hydrodynamically predicted  upper limit. The significance of this agreement can hardly be overstressed, and it poses significant challenges for microscopic descriptions of the early collision dynamics. How the system is able to thermalize so fast is the first RHIC puzzle which we point out.

The second puzzle arises from two-particle correlation measurements (HBT interferometry)  which constrain the freeze-out distribution in space-time. The same hydrodynamic model which provides an almost perfect description of the momentum-space structure of the emitting source (spectra an elliptic flow) predicts a space-time distribution at freeze-out which, when translated into HBT radii, does not agree very well with the data. We argue in Sec. 3 that this points to a problem with our understanding of the late freeze-out stage of the collision.

## 2 Radial and elliptic flow from hydrodynamics

The natural language for describing collective flow phenomena is hydrodynamics. In the ideal fluid (non-viscous) limit used by us, this approach assumes that the microscopic momentum distribution is thermal at every point in space and time. This does not require chemical equilibrium – chemically non-equilibrated situations can be treated by solving separate and coupled conservation equations for the particle currents of individual particle species. The assumption of local thermal equilibrium is an external input, and hydrodynamics offers no insights about the equilibration mechanisms. It is clearly invalid during the initial particle production and early recattering stage, and it again breaks down towards the end when the matter has become so dilute that rescattering ceases and the hadrons “freeze out”. The hydrodynamic approach thus requires a set of initial conditions for the hydrodynamic variables at the earliest time at which the assumption of local thermal equilibrium is applicable, and a “freeze-out prescription” at the end. For the latter we use the Cooper-Frye algorithm  which implements an idealized sudden transition from perfect local thermal equilibrium to free-streaming. A better algorithm  switches from a hydrodynamic description to a microscopic hadron cascade at or shortly after the quark-hadron transition, before the matter becomes too dilute, and lets the cascade handle the freeze-out kinetics. The resulting flow patterns  from such an improved freeze-out algorithm don’t differ much from our simpler Cooper-Frye based approach.

The main advantage of the microscopic freeze-out algorithm  is that it also correctly reproduces the final chemical composition of the fireball, since the particle abundances already freeze out at hadronization, due to a lack of particle-number changing inelastic rescattering processes in the hadronic phase . Our version of the hydrodynamic approach uses an equation of state which assumes local chemical equilibrium all the way down to kinetic freeze-out at $`T_\mathrm{f}\mathrm{\hspace{0.17em}125}`$ MeV and thus is unable to reproduce the correct hadron yield ratios. We therefore adjust the normalization of the momentum spectra for the rarer particle species (kaons, protons, antiprotons) in central collisions by hand to reproduce the chemical equilibrium ratios at a chemical freeze-out temperature $`T_{\mathrm{chem}}=\mathrm{\hspace{0.17em}165}`$ MeV. The absolute normalization of the pion spectra is adjusted through the initial energy density in central collisions; for non-central collisions no new parameters enter since the centrality dependence of the initial conditions is completely controlled by the collision geometry.

To simplify the numerical task of solving the hydrodynamic equations we analytically impose boost invariant longitudinal expansion . This doesn’t give up any essential physics as long as we focus on the transverse expansion dynamics near midrapidity (the region which most RHIC experiments cover best). The hydrodynamic expansion starts at time $`\tau _{\mathrm{eq}}`$ which we fixed by a fit to hadron spectra from the SPS and then extrapolated to RHIC initial conditions (for details see ). For each impact parameter the initial energy density profile in the transverse plane is calculated from a Glauber parametrization using realistic nuclear thickness functions . The measured centrality dependence of the charged particle rapidity density at midrapidity, $`(dN_{\mathrm{ch}}/dy)(y=\mathrm{\hspace{0.17em}0})`$, selects certain allowed combinations of “hard” and “soft” mechanisms for the initial particle production . We here present results for initial conditions at $`\tau _{\mathrm{eq}}=\mathrm{\hspace{0.17em}0.6}`$ fm/$`c`$ calculated from a mixture of 25% “hard” (binary collision) and 75% “soft” (wounded nucleon) contributions  to the initial entropy density (or parton density), with a maximal entropy density $`s_{\mathrm{max}}=\mathrm{\hspace{0.17em}85}/`$fm3 at the fireball center in central collisions (corresponding to a maximal energy density $`e_{\mathrm{max}}=\mathrm{\hspace{0.17em}21.4}`$ GeV/fm3 and a maximal temperature $`T_{\mathrm{max}}=\mathrm{\hspace{0.17em}328}`$ MeV). At the standard time $`\tau =\mathrm{\hspace{0.17em}1}`$ fm/$`c`$ used in Bjorken’s formula  for estimating the energy density from the measured multiplicity density, this corresponds to an average energy density $`e(1\mathrm{fm}/c)=\mathrm{\hspace{0.17em}5.4}`$ GeV/fm3 which is about 70% higher than the value reported from 158 $`A`$ GeV Pb+Pb collisions at the SPS. (Note that $`e`$ at $`\tau _{\mathrm{eq}}=\mathrm{\hspace{0.17em}0.6}`$ fm/$`c`$ is nearly twice as large!) The corresponding profiles for peripheral collisions are then given by the Glauber model . Kinetic freeze-out was forced at $`T_\mathrm{f}=\mathrm{\hspace{0.17em}130}`$ MeV, independent of centrality.

Fig. 1 shows the (absolutely normalized) single particle $`p_{}`$-spectra for charged pions, antiprotons and kaons measured in Au+Au collisions at RHIC together with the hydrodynamical results. The latter were normalized in central collisions as described above, but their centrality dependence and shapes are then completely fixed by the model. The agreement with the data is impressive; for antiprotons the data go out to $`p_{}\mathrm{\hspace{0.17em}3}`$ GeV/$`c`$, and the hydrodynamic model still works within errors! Only for very peripheral collisions (impact parameter $`b>\mathrm{\hspace{0.17em}10}`$ fm) the data show a significant excess of high-$`p_{}`$ particles at $`p_{}>\mathrm{\hspace{0.17em}1.5}`$ GeV/$`c`$. Teaney et al.  showed that this excellent agreement requires a phase transition (soft region) in the equation of state; without the transition, the agreement is lost, especially when the constraints from SPS data and from the elliptic flow measurements below are taken into account.

In the hydrodynamic picture the fact that antiprotons become more abundant than pions for $`p_{}>2`$ GeV/$`c`$ (upper left panel of Fig. 1) is not surprising at all: it is a simple consequence of the strong radial flow at RHIC. For a hydrodynamically expanding thermalized fireball, at relativistic transverse momenta $`p_{}m_0`$ all hadron spectra have the same shape , and at fixed $`m_{}m_0`$ their relative normalization is given by $`(g_i\lambda _i)/(g_j\lambda _j)`$ (where $`g_{i,j}`$ is the spin-isospin degeneracy factor and $`\lambda _{i,j}=e^{\mu _{i,j}/T}`$ is the fugacity of hadron species $`i,j`$). At RHIC the baryon chemical potential at chemical freeze-out is small, $`\mu _B/T_{\mathrm{chem}}\mathrm{\hspace{0.17em}0.26}`$ , and $`\mu _\pi =\mathrm{\hspace{0.17em}0}`$; the $`\overline{p}/\pi ^{}`$ ratio at fixed and sufficiently large $`m_{}`$ is thus predicted to be larger than 1: $`(\overline{p}/\pi ^{})_m_{}=\mathrm{\hspace{0.17em}2}\mathrm{exp}[(\mu _B+\mu _\pi )/T_{\mathrm{chem}}]\mathrm{\hspace{0.17em}1.5}`$ (where the factor 2 arises from the spin degeneracy of the $`\overline{p}`$).

Figure 2 compares the differential elliptic flow $`\mathrm{v}_2(p_{})`$ from minimum bias Au+Au collisions at RHIC  with hydrodynamic calculations. For transverse momenta $`p_{}<\mathrm{\hspace{0.17em}2}`$ GeV/$`c`$ the data are seen to exhaust the upper limit for $`\mathrm{v}_2`$ obtained from the hydrodynamic calculations. Only for hadrons with $`p_{}>2`$ GeV/$`c`$ (i.e. fewer than 1% of all) $`\mathrm{v}_2`$ stays below the hydrodynamic upper limit, indicating incomplete thermalization of high-$`p_{}`$ particles. The data also show the hydrodynamically predicted mass-dependence of $`\mathrm{v}_2`$  (right panel). Similar conclusions can be drawn from a plot of the $`p_{}`$-integrated elliptic flow vs. centrality : only for large impact parameters $`b>\mathrm{\hspace{0.17em}7}`$ fm the measured $`\mathrm{v}_2`$ remains significantly below the hydrodynamic prediction, presumably indicating a lack of early thermalization when the initial overlap region becomes too small.

The excellent agreement with hydrodynamics becomes even more impressive after you begin to realize how easily it is destroyed: As stressed in Sec. 1, it requires the build-up of momentum anisotropies during the very early collision stages when the spatial anisotropy of the reaction zone is still appreciable, causing significant anisotropies of the pressure gradients. A delay in thermalization by more than about 1 fm/$`c`$ (2 fm/$`c`$) dilutes the spatial anisotropy and the hydrodynamically predicted elliptic flow coefficient by 10% (25%)  which is more than is allowed by the data. Parton cascade simulations with standard HIJING input generate almost no elliptic flow and require an artificial increase of the opacity of the partonic matter by a factor 80 to reproduce the RHIC data . Hadronic cascades of the RQMD and URQMD type (in which the high-density initial state is parametrized by non-interacting, pressureless QCD strings) predict  too little elliptic flow and a decrease of $`\mathrm{v}_2`$ from SPS to RHIC, contrary to the data. One can get close to the data by forcing the strings to “melt” into partons which then rescatter with large cross sections . What causes these large cross sections is still a mystery.

The elliptic flow is self-quenching : it makes the reaction zone grow faster along its initially short direction and thus eventually eliminates its own cause. As the spatial deformation of the fireball goes to zero, the elliptic flow saturates . The saturation time scale times $`c`$ is of the order of the transverse size of the initial overlap region (at lower energies it is a bit longer, see Figs. 7, 9 in ). At RHIC energies and above, the time it takes the collision zone to dilute from the high initial energy density to the critical value for hadronization is equal to or longer than this saturation time: most or all of the elliptic flow is generated before any hadrons even appear! It thus seems that the only possible conclusion from the successful hydrodynamic description of the observed radial and elliptic flow patterns is that the thermal pressure driving the elliptic flow is partonic pressure, and that the early stage of the collision must have been a thermalized quark-gluon plasma.

## 3 The RHIC HBT puzzle

Hydrodynamics not only predicts the momenta of the emitted hadrons, but also the spatial structure of the hadron emitting source at freeze-out. Bose-Einstein (a.k.a. Hanbury Brown-Twiss (HBT)) two-particle intensity interferometry allows to access the r.m.s. widths of the space-time distribution of hadrons with a given momentum $`p`$ . One of the interesting questions one can try to address with this tool is whether at RHIC the reaction zone flips the sign of its spatial deformation between initial impact and final freeze-out, as expected at very high collision energy . The answer to this question turns out to be non-trivial, on two different levels: first, hydrodynamics, at least with the presently implemented initial conditions and freeze-out algorithm, fails to reproduce even for central Au+Au collisions the measured HBT radii extracted from two-pion correlations . We’ll show how and explain why. Second, for expanding systems the HBT radii don’t measure the entire freeze-out region, but only the effective emission regions (“regions of homogeneity”) for particles of given momentum . For non-central collisions, due to the anisotropic transverse flow these can, at least in principle, have a different spatial deformation than the entire (momentum-integrated) freeze-out region, giving rise to a different behaviour of the HBT radii observed at different angles relative to the reaction plane than perhaps naively expected .

Fig. 3 shows the HBT radii $`R_{\mathrm{side}}`$, $`R_{\mathrm{out}}`$, and $`R_{\mathrm{long}}`$ for central Au+Au collisions at $`\sqrt{s}=\mathrm{\hspace{0.17em}130}A`$ GeV  compared with hydrodynamic results. The theoretical results only account for directly emitted pions; resonance decay pions would add an exponential tail to the emission time distribution and could cause a slight increase in the emission duration, without affecting much the spatial width of the emission region . The solid lines correspond to initial conditions and freeze-out parameters as used for the successful description of the momentum distributions in the previous section; clearly, $`R_{\mathrm{side}}`$ comes out too small whereas $`R_{\mathrm{out}}`$ and $`R_{\mathrm{long}}`$ are both too large in the model. The problem of $`R_{\mathrm{side}}`$ from hydrodynamics being too small is well-known from the SPS ; presumably it is mostly due to the sharp Cooper-Frye freeze-out and seems to be at least partially resolved if the freeze-out kinetics is handled microscopically within a hadronic cascade . The latter gives a more “fuzzy” spatial freeze-out distribution with larger r.m.s. width in the sideward direction; it is not so clear that it also resolves the problem with the $`K_{}`$-dependence of $`R_{\mathrm{side}}`$ which is much stronger in the data than in the model.

On the other hand, “fuzzy” freeze-out  and the inclusion of resonance decay contributions  only exacerbate the problems with $`R_{\mathrm{long}}`$ and $`R_{\mathrm{out}}`$. In high energy heavy-ion collisions the longitudinal HBT radius is controlled by the expansion dynamics via the longitudinal velocity gradient at freeze-out . For a boost invariant longitudinal flow profile this gradient decreases with time as $`1/\tau `$, leading to rather weak gradients (and correspondingly large values for $`R_{\mathrm{long}}`$) at the typical hydrodynamic freeze-out time of $`\mathrm{\hspace{0.17em}15}`$ fm/$`c`$. To reduce $`R_{\mathrm{long}}`$, either freeze-out must happen significantly earlier, or the expansion of the source must strongly violate longitudinal boost-invariance even close to mid-rapidity, somehow allowing for significantly stronger longitudinal flow velocity gradients at late times than predicted by the Bjorken profile $`v_L=z/t`$.

Noting that decoupling is driven by the strong radial flow developing during the expansion stage , we have tried to force the system to decouple earlier by initiating the transverse flow even before $`\tau _{\mathrm{eq}}`$. Even if the system is not yet locally thermalized, it will start to develop some transverse collective dynamics, albeit perhaps not as quickly as in a hydrodynamic approach. We have tested two extreme assumptions about the transverse expansion prior to thermalization : in one simulation, shown as the dot-dashed curve in Fig. 3, we started the hydrodynamic evolution directly at the parton formation time (for which we took the somewhat arbitrary value $`\tau _{\mathrm{form}}=\mathrm{\hspace{0.17em}0.2}`$ fm/$`c`$). In another limit (dashed lines in Fig. 3), we allowed the partons to stream freely from time $`\tau _{\mathrm{form}}`$ to $`\tau _{\mathrm{eq}}`$ and matched at $`\tau _{\mathrm{eq}}`$ the first row of the energy momentum tensor to an ideal fluid form, thereby extracting an initial transverse flow profile at $`\tau _{\mathrm{eq}}`$. In both cases the resulting transverse flow “seed” at $`\tau _{\mathrm{eq}}=\mathrm{\hspace{0.17em}0.6}`$ fm/$`c`$ caused the system to expand more rapidly and farther out into the transverse direction, freezing out 10-20% earlier. Fig. 3 shows that this helps with both $`R_{\mathrm{long}}`$ and $`R_{\mathrm{out}}`$, but not as much as required by the data.

To get close to the data, we would need to postulate freeze-out directly at the hadronization point (dotted lines in Fig. 3). This radical and entirely unmotivated assumption almost removes the discrepancy with $`R_{\mathrm{long}}`$ and with the average magnitude of $`R_{\mathrm{out}}`$, but it is still unable to reproduce the measured strong $`K_{}`$-depenence of $`R_{\mathrm{out}}`$ and $`R_{\mathrm{side}}`$, and it still yields much too small values for $`R_{\mathrm{side}}`$ at low $`K_{}`$. Correspondingly, the ratio $`R_{\mathrm{out}}/R_{\mathrm{side}}`$ (which is $`1`$ in the data) is still significantly overpredicted by the model (although much less so than by earlier hydrodynamic predictions  when constraints from RHIC momentum spectra were not yet available).

The problem with the small $`R_{\mathrm{side}}`$ cannot be fixed by pre-equilibrium transverse flow, either: even though the system then expands to larger values of $`r`$, $`R_{\mathrm{side}}`$ remains essentially unchanged (dashed and dash-dotted lines in Fig. 3). The homogeneity region simply moves farther out without increasing in size. We don’t see a way to move closer to the data by further modifying the initial conditions, and even the alterations we made to obtain Fig. 3 may turn out to be excluded by the singles spectra and the elliptic flow data (which we haven’t tested yet). We therefore believe that a resolution to the HBT puzzle must lie in the handling of the freeze-out process (although we don’t yet know how).

Let us further analyze the relationship between $`R_{\mathrm{out}}`$ and $`R_{\mathrm{side}}`$. The formula

$`R_{\mathrm{out}}^2R_{\mathrm{side}}^2=(x_{\mathrm{out}}\overline{x}_{\mathrm{out}})^2(x_{\mathrm{side}}\overline{x}_{\mathrm{side}})^2+\beta _{}^2(t\overline{t})^22\beta _{}(x_{\mathrm{out}}\overline{x}_{\mathrm{out}})(t\overline{t})`$ (1)
shows that their difference is controlled by three contributions which can partially compensate each other. There is always a positive contribution from the emission duration $`(\delta t)^2(t\overline{t})^2`$; for systems with longitudinal boost-invariance freeze-out happens at constant longitudinal proper time $`\tau =\tau _\mathrm{f}`$, and late freeze-out, which leads to a large longitudinal homogeneity region $`R_{\mathrm{long}}`$, thus implies contributions from a wide region along this $`\tau =\tau _\mathrm{f}`$ hyperbola, leading to a large variance $`\delta t`$. To reduce $`\delta t`$ we should therefore again freeze out earlier. In hydrodynamics with Cooper-Frye freeze-out , the term $`2\beta _{}(x_{\mathrm{out}}\overline{x}_{\mathrm{out}})(t\overline{t})`$ is also positive, since freeze-out happens from the outside inward, causing a negative $`x_{\mathrm{out}}t`$ correlation. These two positive contributions to $`R_{\mathrm{out}}^2R_{\mathrm{side}}^2`$ are partially cancelled by a negative geometric contribution $`(\delta x_{\mathrm{out}})^2(\delta x_{\mathrm{side}})^2`$: according to Figures 4 and 5 hydrodynamics predicts that pion emission is strongly surface dominated, especially for pions with non-zero transverse momentum $`K_{}`$, whose emission regions are tightly squeezed towards the edge of the fireball where the radial flow is strongest. This results in an “opaque source”, characterized by a smaller outward than sideward variance $`(\delta x_{\mathrm{out}})^2<(\delta x_{\mathrm{side}})^2`$ . The opacity of the hydrodynamic source is stronger at RHIC than SPS energies, but not strong enough to compensate for the two positive contributions in Eq. (1); this is the reason why we fail to reproduce the measured relation $`R_{\mathrm{out}}R_{\mathrm{side}}`$. Of course, things would be easier if the $`x_{\mathrm{out}}t`$ correlation were positive (i.e. pions at larger radial distances were emitted later), but we don’t see a good reason why this should be so.

Let us close with a quick preview of results for the HBT radii for non-central collisions, in particular their dependence on the angle $`\mathrm{\Phi }`$ of the transverse pair momentum $`\stackrel{}{K}_{}`$ relative to the reaction plane. Fig. 6 shows our hydrodynamic results for $`R_{\mathrm{side}}^2`$, $`R_{\mathrm{out}}^2`$, $`R_{\mathrm{os}}^2`$, and $`R_{\mathrm{long}}^2`$ for $`b=\mathrm{\hspace{0.17em}7}`$ fm Au+Au collisions at $`\sqrt{s}=\mathrm{\hspace{0.17em}130}A`$ GeV, plotted as functions of $`\mathrm{\Phi }`$ for a number of different values of $`K_{}=|\stackrel{}{K}_{}|`$. (Note that “out” and “side” denote the directions parallel and perpendicular to $`\stackrel{}{K}_{}`$ in the transverse plane .) While $`R_{\mathrm{long}}^2`$ is almost independent of $`\mathrm{\Phi }`$, the three other radius parameters show marked azimuthal dependences of the generic form (with all coefficients being positive)

$`R_{\mathrm{side}}^2(\mathrm{\Phi })=R_{\mathrm{s},0}^2+R_{\mathrm{s},2}^2\mathrm{cos}(2\mathrm{\Phi }),R_{\mathrm{out}}^2(\mathrm{\Phi })=R_{\mathrm{o},0}^2R_{\mathrm{o},2}^2\mathrm{cos}(2\mathrm{\Phi }),`$
$`R_{\mathrm{os}}^2(\mathrm{\Phi })=R_{\mathrm{os},2}^2\mathrm{sin}(2\mathrm{\Phi }).`$ (2)
Although the magnitudes of the coefficients $`R_\alpha ^2`$ in Fig. 6 are quite different (and presumably not too trustworthy, given the disagreement with the data for central collisions in Fig. 3), it is surprising that the signs and phases of the oscillations are identical to those calculated and measured at the AGS ! At the AGS radial flow effects are thought to be sufficiently weak that the oscillations can be interpreted purely geometrically , reflecting a spatially deformed source which is elongated perpendicular to the reaction plane (as is the case for the initial overlap region). Fig. 5 shows that at RHIC the reason for these oscillations is more subtle: except for small $`K_{}`$, where pions are emitted from the entire fireball, the strong radial flow squeezes the emission regions towards the edge of the fireball so that they no longer reflect directly the spatial deformation of the momentum-integrated emission function shown in Fig. 4. However, even though the measured HBT radii have little to do with the total widths of the fireball in $`x`$ and $`y`$ directions, the out-of-plane deformation of the latter imprints itself on the shape of the effective emission regions through the “opacity effect”, i.e. the fact that the emission is strongly surface-peaked, leading to azimuthal oscillations which are in phase with those expected from naive geometrical considerations of the entire source.

Even though the absolute magnitudes of the radii shown in Fig. 6 are affected by similar problems as those in Fig. 3, the phases of the azimuthal oscillations agree with preliminary STAR data , and also their amplitudes are at least in the right ballpark. This gives rise to some optimism that the HBT puzzle is limited in scope and can be resolved by evolutionary rather than revolutionary methods.

## Acknowledgement

This work was supported by the U.S. Department of Energy under contract DE-FG02-01ER41190.


text:
# Asymptotic faithfulness of the quantum 𝑆𝑈(𝑛) representations of the mapping class groups

## 1. Introduction

In this paper we shall study the finite dimensional quantum $`SU(n)`$ representations of the mapping class group of a genus $`g`$ surface. These form the only rigorously constructed part of the gauge-theoretic approach to Topological Quantum Field Theories in dimension $`3`$, which Witten proposed in his seminal paper \[W1\]. We discovered the asymptotic faithfulness property for these representations by studying this approach, which we will now briefly describe, leaving further details to section 2, 3 and the references given there.

Let $`\mathrm{\Sigma }`$ be a closed oriented surface of genus $`g2`$ and $`p`$ a point on $`\mathrm{\Sigma }`$. Fix $`d/nZ_{SU(n)}`$ in the center of $`SU(n)`$. Let $`M`$ be the moduli space of flat $`SU(n)`$-connections on $`\mathrm{\Sigma }p`$ with holonomy $`d`$ around $`p`$.

By applying geometric quantization to the moduli space $`M`$ one gets a certain finite rank vector bundle over Teichmüller space $`𝒯`$, which we will call the Verlinde bundle $`𝒱_k`$ at level $`k`$, where $`k`$ is any positive integer. The fiber of this bundle over a point $`\sigma 𝒯`$ is $`𝒱_{k,\sigma }=H^0(M_\sigma ,_\sigma ^k)`$, where $`M_\sigma `$ is $`M`$ equipped with a complex structure induced from $`\sigma `$ and $`_\sigma `$ is an ample generator of the Picard group of $`M_\sigma `$.

The main result pertaining to this bundle $`𝒱_k`$ is that its projectivization $`(𝒱_k)`$ supports a natural flat connection. This is a result proved independently by Axelrod, Della Pietra and Witten \[ADW\] and by Hitchin \[H\]. Now, since there is an action of the mapping class group $`\mathrm{\Gamma }`$ of $`\mathrm{\Sigma }`$ on $`𝒱_k`$ covering its action on $`𝒯`$, which preserves the flat connection in $`(𝒱_k)`$, we get for each $`k`$ a finite dimensional projective representation, say $`\rho _k^{n,d}`$, of $`\mathrm{\Gamma }`$, namely on the covariant constant sections of $`(𝒱_k)`$ over $`𝒯`$. This sequence of projective representations $`\rho _k^{n,d}`$, $`k_+`$ is the quantum $`SU(n)`$ representations of the mapping class group $`\mathrm{\Gamma }`$.

For each given $`(n,d,k)`$, we cannot expect $`\rho _k^{n,d}`$ to be faithful. However, V. Turaev conjectured a decade ago (see e.g. \[T\]) that there should be no non-trivial element of the mapping class group representing trivially under $`\rho _k^{n,d}`$ for all $`k`$, keeping $`(n,d)`$ fixed. We call this property asymptotic faithfulness of the quantum $`SU(n)`$ representations $`\rho _k^{n,d}`$. In this paper we prove Turaev’s conjecture:

###### Theorem 1.

Assume that $`n`$ and $`d`$ are coprime or that $`(n,d)=(2,0)`$ when $`g=2`$. Then we have that

$$\underset{k=1}{\overset{\mathrm{}}{}}\mathrm{ker}(\rho _k^{n,d})=\{\begin{array}{cc}\{1,H\}\hfill & g=2\text{}n=2\text{ and }d=0\hfill \\ \{1\}\hfill & \text{otherwise}.\hfill \end{array}$$

where $`H`$ is the hyperelliptic involution.

This Theorem states that for any element $`\varphi `$ of the mapping class group $`\mathrm{\Gamma }`$, which is not the identity element (and not the hyperelliptic involution in genus 2), there is an integer $`k`$ such that $`\rho _k^{n,d}(\varphi )`$ is not a multiple of the identity. We will suppress the superscript on the quantum representations and simply write $`\rho _k=\rho _k^{n,d}`$ throughout the rest of the paper.

Our key idea in the proof of Theorem 1 is the use of the endomorphism bundle $`End(𝒱_k)`$ and the construction of sections of this bundle via Toeplitz operators associated to smooth functions on the moduli space $`M`$. By showing that these sections are asymptotically flat sections of $`End(𝒱_k)`$ (see Theorem 6 for the precise statement), we prove that any element in the above intersection of kernels acts trivially on the smooth functions on $`M`$, hence acts by the identity on $`M`$ (see the proof of Theorem 7). Theorem 1 now follows directly from knowing which elements of the mapping class group act trivially on the moduli space $`M`$.

The assumptions on the pair $`(n,d)`$ in Theorem 1 are exactly picking out the cases where the moduli space $`M`$ is smooth. This means we can apply the work of Bordemann, Meinrenken and Schlichenmaier on Toeplitz operators on smooth Kähler manifolds, in particular their formula for the asymptotics in $`k`$ of the operator norm of Toeplitz operators and the asymptotic expansion of the product of two Toeplitz operators. - It is with the use of these results we establish that the Toeplitz operator sections are asymptotically flat with respect to Hitchin’s connection.

In the remaining cases, where $`M`$ is singular, we also have a proof of asymptotic faithfulness, where we use the desingularization of the moduli space, but this argument is technically quite a bit more involved. However, together with Michael Christ we have in \[AC\] extended some of the results of Bordemann, Meinrenken and Schlichenmaier and Karabegov and Schlichenmaier to the case of singular varieties. In \[A3\] the argument of the present paper will be repeated in the non-coprime case, making use of the results of \[AC\] to show that Theorem 1 holds in general without the coprime assumption.

The abelian case, i.e. the case where $`SU(n)`$ is replaced by $`U(1)`$, was considered in \[A2\], before we considered the case discussed in this paper. In this case, with the use of Theta-functions, explicit expressions for the Toeplitz operators associated to holonomy functions can be obtained. From these expressions it follows that the Toeplitz operators are not covariant constant even in this much simpler case (although the relevant connection is the one induced from the $`L_2`$-projection as shown by Ramadas in \[R1\]). However, they are asymptotically covariant constant, in fact we find explicit perturbations to all orders in $`k`$, which in this case, we argue, can be summed and used to create actual covariant constant sections of the endomorphism bundle. The result as far as the mapping class group goes, is that the intersection of the kernels over all $`k`$ in that case is the Torelli group.

Returning to the non-abelian case at hand, we know by the work of Laszlo \[La1\], that $`(𝒱_k)`$ with its flat connection is isomorphic to the projectivization of the bundle of conformal blocks for $`sl(n,)`$ with its flat connection over $`𝒯`$ as constructed by Tsuchiya, Ueno and Yamada \[TUY\]. This means that the quantum $`SU(n)`$ representations $`\rho _k`$ is the same sequence of representations as the one arising from the space of conformal blocks for $`sl(n,)`$.

By the work of Reshetikhin-Turaev, Topological Quantum Field Theories have been constructed in dimension $`3`$ from the quantum group $`U_qsl(n,)`$ (see \[RT1\], \[RT2\] and \[T\]) or alternatively from the Kauffman bracket and the Homfly-polynomial by Blanchet, Habegger, Masbaum and Vogel (see \[BHMV1\], \[BHMV2\] and \[B1\]).

In ongoing work of Ueno joint with this author (see \[AU1\], \[AU2\] and \[AU3\]), we are in the process of establishing that the TUY construction of the bundle of conformal blocks over Teichmüller space for $`sl(n,)`$ gives a modular functor, which in turn gives a TQFT, which is isomorphic to the $`U_qsl(n,)`$-Reshetikhin-Turaev TQFT. A corollary of this will be that the quantum $`SU(n)`$ representations are isomorphic to the ones that are part of the $`U_qsl(n,)`$-Reshetikhin-Turaev TQFT. Since it is well known that the Reshetikhin-Turaev TQFT is unitary one will get unitarity of the quantum $`SU(n)`$ representations from this. We note that unitarity is not clear from the geometric construction of the quantum $`SU(n)`$ representions. If the quantum $`SU(n)`$ representations $`\rho _k`$ are unitary, then we have for all $`\varphi \mathrm{\Gamma }`$ that

(1)
$$|Tr(\rho _k(\varphi ))|dim\rho _k.$$

From the above Theorem 1 follows directly, assuming unitarity, that one has the following

###### Corollary 1.

Assume that $`n`$ and $`d`$ are coprime or that $`(n,d)=(2,0)`$ when $`g=2`$. Then equality holds in (1) for all $`k`$, if and only if

$$\varphi \{\begin{array}{cc}\{1,H\}\hfill & g=2\text{}n=2\text{ and }d=0\hfill \\ \{1\}\hfill & \text{otherwise}.\hfill \end{array}$$

Furthermore, one will get that the norm of the Reshetikhin-Turaev quantum invariant for all $`k`$ and $`n=2`$ ($`n=3`$ in the genus $`2`$ case) can separate the mapping torus of the identity from the rest of the mapping tori as a purely TQFT consequence of Corollary 1.

In this paper we have initiated the program of using of the theory of Toeplitz operators on the moduli spaces in the study of TQFT’s. The main insight behind the program is the relation between these Toeplitz operators and Hitchin’s connection asymptotically in the quantum level $`k`$. Here we have presented the initial application of this program, namely the establishment of the asymptotic faithfulness property for the quantum representations of the mapping class groups. However this program can also be used to study other asymptotic properties of these TQFT’s. In particular we have used them to establish that the quantum invariants for closed 3-manifolds have asymptotic expansions in $`k`$. Topological consequences of this are that certain classical topological properties are determined by the quantum invariants, resulting in interesting topological conclusions, including very strong knot theoretical corollaries. Writeup of these further developments of this program is in progress.

We remark that it is also an interesting problem to understand how the Toeplitz operator constructions used in this paper are related to the deformation quantization of the moduli spaces described in \[AMR1\] and \[AMR2\]. In the abelian case, the resulting Berezin-Toeplitz deformation quantization was explicitly described in \[A2\] and it turns out to be equivalent to the one constructed in \[AMR2\] in this case.

This paper is organized as follows. In section 2 we give the basic setup of applying geometric quantization to the moduli space to construct the Verlinde bundle over Teichmüller space. In section 3 we review the construction of the connection in the Verlinde bundle. We end that section by stating the properties of the moduli space and the Verlinde bundle we will use. We will see that it is only a few elementary properties about the moduli space, Teichmüller space and the general form of the connection in the Verlinde bundle we really need. In section 4 we review the general results about Toeplitz operators on smooth compact Kähler manifolds we use in the following section 5, where we prove that the Toeplitz operators for smooth functions on the moduli space give asymptotically flat sections of the endomorphism bundle of the Verlinde bundle. Finally, in section 6 we prove the asymptotic faithfulness Theorem 1 above.

After the completion of this work Freedman and Walker, together with Wang, have found a proof of the Asymptotic faithfulness property for the SU(2)-BHMV-representations which uses BHMV-technology. Their paper has already appeared \[FWW\] (see also \[M2\] for a discussion). As alluded to before, we are working jointly with K. Ueno to establish that these representations are equivalent to our sequence $`\rho _k^{2,0}`$. However, as long as this has not been established, our result is logically independent of theirs.

For the SU(2)-BHMV-representations it is already known by the work of Roberts \[Ro\], that they are irreducible for $`k+2`$ prime and that they have infinite image by the work of Masbaum \[M1\], except for a few low values of $`k`$.

We would like to thank Nigel Hitchin, Bill Goldman and Gregor Masbaum for valuable discussion. Further thanks is due to the Clay Mathematical Institute for their financial support and to the Department of Mathematics, University of California, Berkeley for their hospitality, during the period, when this work was completed.

## 2. The gauge theory construction of the Verlinde bundle

Let us now very briefly recall the construction of the Verlinde bundle. Only the details needed in this paper will be given. We refer e.g. to \[H\] for further details. As in the introduction we let $`\mathrm{\Sigma }`$ be a closed oriented surface of genus $`g2`$ and $`p\mathrm{\Sigma }`$. Let $`P`$ be a principal $`SU(n)`$-bundle over $`\mathrm{\Sigma }`$. Clearly, all such $`P`$ are trivializable. As above let $`d/nZ_{SU(n)}`$. Throughout the rest of this paper we will assume that $`n`$ and $`d`$ are coprime, although in the case $`g=2`$ we also allow $`(n,d)=(2,0)`$. Let $`M`$ be the moduli space of flat $`SU(n)`$-connections in $`P|_{\mathrm{\Sigma }p}`$ with holonomy $`d`$ around $`p`$. We can identify

$$M=Hom_d(\stackrel{~}{\pi }_1(\mathrm{\Sigma }),SU(n))/SU(n).$$

Here $`\stackrel{~}{\pi }_1(\mathrm{\Sigma })`$ is the universal central extension

$$0\stackrel{~}{\pi }_1(\mathrm{\Sigma })\pi _1(\mathrm{\Sigma })1$$

as discussed in \[H\] and in \[AB\] and $`Hom_d`$ means the space of homomorphisms from $`\stackrel{~}{\pi }_1(\mathrm{\Sigma })`$ to $`SU(n)`$ which send the image of $`1`$ in $`\stackrel{~}{\pi }_1(\mathrm{\Sigma })`$ to $`d`$ (see \[H\]). When $`n`$ and $`d`$ are coprime, $`M`$ is a compact smooth manifold of dimension $`m=(n^21)(g1)`$. In general, when $`n`$ and $`d`$ are not coprime $`M`$ is not smooth, except in the case where $`g=2`$, $`n=2`$ and $`d=0`$. In this case $`M`$ is in fact diffeomorphic to $`^3`$. There is a natural homomorphism from the mapping class group to the outer automorphisms of $`\stackrel{~}{\pi }_1(\mathrm{\Sigma })`$, hence $`\mathrm{\Gamma }`$ acts on $`M`$.

We choose an invariant bilinear form $`\{,\}`$ on $`𝔤=\text{Lie}(SU(n))`$, normalized such that $`\frac{1}{6}\{\vartheta [\vartheta \vartheta ]\}`$ is a generator of the image of the integer cohomology in the real cohomology in degree $`3`$ of $`SU(n)`$, where $`\vartheta `$ is the $`𝔤`$-valued Maurer-Cartan $`1`$-form on $`SU(n)`$.

This bilinear form induces a symplectic form on $`M`$. In fact

$$T_{[A]}MH^1(\mathrm{\Sigma },d_A),$$

where $`A`$ is any flat connection in $`P`$ representing a point in $`M`$ and $`d_A`$ is the induced covariant derivative in the associated adjoint bundle. Using this identification, the symplectic form on $`M`$ is:

$$\omega (\phi _1,\phi _2)=_\mathrm{\Sigma }\{\phi _1\phi _2\},$$

where $`\phi _i`$ are $`d_A`$-closed $`1`$-forms on $`\mathrm{\Sigma }`$ with values in $`adP`$. See e.g. \[H\] for further details on this. The natural action of $`\mathrm{\Gamma }`$ on $`M`$ is symplectic.

Let $``$ be the Hermitian line bundle over $`M`$ and $``$ the compatible connection in $``$ constructed by Freed in \[Fr\]. This is the content of Corollary 5.22, Proposition 5.24 and equation (5.26) in \[Fr\] (see also the work of Ramadas, Singer and Weitsman \[RSW\]). By Proposition 5.27 in \[Fr\] we have that the curvature of $``$ is $`\frac{\sqrt{1}}{2\pi }\omega `$. We will also use the notation $``$ for the induced connection in $`^k`$, where $`k`$ is any integer.

By an almost identical construction, we can lift the action of $`\mathrm{\Gamma }`$ on $`M`$ to act on $``$ such that the Hermitian connection is preserved (See e.g. \[A1\]). In fact, since $`H^2(M,)`$ and $`H^1(M,)=0`$, it is clear that the action of $`\mathrm{\Gamma }`$ leaves the isomorphism class of $`(,)`$ invariant, thus alone from this one can conclude that a central extension of $`\mathrm{\Gamma }`$ acts on $`(,)`$ covering the $`\mathrm{\Gamma }`$ action on $`M`$. This is actually all we need in this paper, since we are only interested in the projectivized action.

Let now $`\sigma 𝒯`$ be a complex structure on $`\mathrm{\Sigma }`$. Let us review how $`\sigma `$ induces a complex structure on $`M`$ which is compatible with the symplectic structure on this moduli space. The complex structure $`\sigma `$ induces a $``$-operator on $`1`$-forms on $`\mathrm{\Sigma }`$ and via Hodge theory we get that

$$H^1(\mathrm{\Sigma },d_A)\mathrm{ker}(d_A+d_A).$$

On this kernel, consisting of the harmonic $`1`$-forms with values in $`adP`$, the $``$-operator acts and its square is $`1`$, hence we get an almost complex structure on $`M`$ by letting $`I=I_\sigma =`$. It is a classical result by Narasimhan and Seshadri (see \[NS1\]), that this actually makes $`M`$ a smooth Kähler manifold, which as such, we denote $`M_\sigma `$. By using the $`(0,1)`$ part of $``$ in $``$, we get an induced holomorphic structure in the bundle $``$. The resulting holomorphic line bundle will be denoted $`_\sigma `$. See also \[H\] for further details on this.

From a more algebraic geometric point of view, we consider the moduli space of S-equivalence classes of semi-stable bundles of rank $`n`$ and determinant isomorphic to the line bundle $`𝒪(d[p])`$. By using Mumford’s Geometric Invariant Theory, Narasimhan and Seshadri (see \[NS2\]) showed that this moduli space is a smooth complex algebraic projective variety which is isomorphic as a Kähler manifold to $`M_\sigma `$. Referring to \[DN\] we recall that

###### Theorem 2 (Drezet & Narasimhan).

The Picard group of $`M_\sigma `$ is generated by the holomorphic line bundle $`_\sigma `$ over $`M_\sigma `$ constructed above:

$$Pic(M_\sigma )=_\sigma .$$

###### Definition 1.

The Verlinde bundle $`𝒱_k`$ over Teichmüller space is by definition the bundle whose fiber over $`\sigma 𝒯`$ is $`H^0(M_\sigma ,_\sigma ^k)`$, where $`k`$ is a positive integer.

## 3. The projectively flat connection

In this section we will review Axelrod, Della Pietra and Witten’s and Hitchin’s construction of the projective flat connection over Teichmüller space in the Verlinde bundle. We refer to \[H\] and \[ADW\] for further details.

Let $`_k`$ be the trivial $`C^{\mathrm{}}(M,^k)`$-bundle over $`𝒯`$ which contains $`𝒱_k`$, the Verlinde sub-bundle. If we have a smooth one-parameter family of complex structures $`\sigma _t`$ on $`\mathrm{\Sigma }`$, then that induces a smooth one-parameter family of complex structures on $`M`$, say $`I_t`$. In particular we get $`\sigma _t^{}T_{\sigma _t}(𝒯)`$, which gives an $`I_t^{}H^1(M_{\sigma _t},T)`$ (here $`T`$ refers to the holomorphic tangent bundle of $`M_{\sigma _t}`$).

Suppose $`s_t`$ is a corresponding smooth one-parameter family in $`C^{\mathrm{}}(M,^k)`$ such that $`s_tH^0(M_{\sigma _t},_{\sigma _t}^k).`$ By differentiating the equation

$$(1+\sqrt{1}I_t)s_t=0,$$

we see that

$$\sqrt{1}I_t^{}s+(1+\sqrt{1}I_t)s_t^{}=0.$$

Hence, if we have an operator

$$u(v):C^{\mathrm{}}(M,^k)C^{\mathrm{}}(M,^k)$$

for all real tangent vectors to Teichmüller space $`vT(𝒯)`$, varying smoothly with respect to $`v`$, and satisfying

$$\sqrt{1}I_t^{}^{1,0}s_t+^{0,1}u(\sigma _t^{})(s_t)=0,$$

for all smooth curves $`\sigma _t`$ in $`𝒯`$, then we get a connection induced in $`𝒱_k`$ by letting

(2)
$$\widehat{}_v=\widehat{}_v^tu(v),$$

for all $`vT(𝒯)`$, where $`\widehat{}^t`$ is the trivial connection in $`_k`$.

In order to specify the particular $`u`$ we are interested in, we use the symplectic structure on $`\omega \mathrm{\Omega }^{1,1}(M_\sigma )`$ to define the tensor $`G=G(v)\mathrm{\Omega }^0(M_\sigma ,S^2(T))`$ by

$$v[I_\sigma ]=G(v)\omega ,$$

where $`v[I_\sigma ]`$ means the derivative in the direction of $`vT_\sigma (𝒯)`$ of the complex structure $`I_\sigma `$ on $`M`$. Following Hitchin, we give an explicit formula for $`G`$ in terms of $`vT_\sigma (𝒯)`$:

The holomorphic tangent space to Teichmüller space at $`\sigma 𝒯`$ is given by

$$T_\sigma ^{1,0}(𝒯)H^1(\mathrm{\Sigma }_\sigma ,K^1).$$

Furthermore, the holomorphic co-tangent space to the moduli space of semi-stable bundles at the equivalence class of a stable bundle $`E`$ is given by

$$T_{[E]}^{}M_\sigma H^0(\mathrm{\Sigma }_\sigma ,End_0(E)K).$$

Thinking of $`G(v)\mathrm{\Omega }^0(M_\sigma ,S^2(T))`$ as a quadratic function on $`T^{}=T^{}M_\sigma `$, we have that

$$G(v)(\alpha ,\alpha )=_\mathrm{\Sigma }Tr(\alpha ^2)v^{(1,0)}$$

where $`v^{(1,0)}`$ is the image of $`v`$ under the projection onto $`T^{1,0}(𝒯)`$. From this formula it is clear that $`G(v)H^0(M_\sigma ,S^2(T))`$ and that $`\widehat{}`$ agrees with $`\widehat{}^t`$ along the anti-holomorphic directions $`T^{0,1}(𝒯)`$. From Proposition (4.4) in \[H\] we have that this map $`vG(v)`$ from $`T_\sigma (𝒯)`$ to $`H^0(M_\sigma ,S^2(T))`$ is an isomorphism.

The particular $`u(v)`$ we are interested in is $`u_{G(v)}`$, where

(3)
$$u_G(s)=\frac{1}{2(k+n)}(\mathrm{\Delta }_G2_{GF}+\sqrt{1}kf_G)s.$$

The leading order term $`\mathrm{\Delta }_G`$ is the 2’nd order operator given by

$$\begin{array}{ccccc}\mathrm{\Delta }_G:C^{\mathrm{}}(M,^k)& \stackrel{^{1,0}}{}& C^{\mathrm{}}(M,T^{}^k)& \stackrel{G}{}& C^{\mathrm{}}(M,T^k)\\ & \stackrel{^{1,0}1+1^{1,0}}{}& C^{\mathrm{}}(M,T^{}T^k)& \stackrel{Tr}{}& C^{\mathrm{}}(M,^k),\end{array}$$

where we have used the Chern connection in $`T`$ on the Kähler manifold $`(M_\sigma ,\omega )`$.

The function $`F=F_\sigma `$ is the Ricci potential uniquely determined as the real function with zero average over $`M`$, which satisfies the following equation

(4)
$$Ric_\sigma =2n\omega +2\sqrt{1}\overline{}F_\sigma .$$

We usually drop the subscript $`\sigma `$ and think of $`F`$ as a smooth map from $`𝒯`$ to $`C^{\mathrm{}}(M)`$.

The complex vector field $`GFC^{\mathrm{}}(M_\sigma ,T)`$ is simply just the contraction of $`G`$ with $`FC^{\mathrm{}}(M_\sigma ,T^{})`$.

The function $`f_GC^{\mathrm{}}(M)`$ is defined by

$$f_G=\sqrt{1}v[F],$$

where $`v`$ is determined by $`G=G(v)`$ and $`v[F]`$ means the derivative of $`F`$ in the direction of $`v`$. We refer to \[ADW\] for this formula for $`f_G`$.

###### Theorem 3 (Axelrod, Della Pietra & Witten; Hitchin).

The expression (2) above defines a connection in the bundle $`𝒱_k`$, which induces a flat connection in $`(𝒱_k)`$.

Faltings has established this Theorem in the case where one replaces $`SU(n)`$ with a general semi-simple Lie group (see \[Fal\]).

We remark about genus $`2`$, that \[ADW\] covers this case, but \[H\] excludes this case, however, the work of Van Geemen and De Jong \[vGdJ\] extends Hitchin’s approach to the genus $`2`$ case.

As discussed in the introduction, we see by Laszlo’s Theorem that this particular connection is the relevant one to study.

It will be essential for us to consider the induced flat connection $`\widehat{}^e`$ in the endomorphism bundle $`End(𝒱_k)`$. Suppose $`\mathrm{\Phi }`$ is a section of $`End(𝒱_k)`$. Then for all sections $`s`$ of $`𝒱_k`$ and all $`vT(𝒯)`$ we have that

$$(\widehat{}_v^e\mathrm{\Phi })(s)=\widehat{}_v\mathrm{\Phi }(s)\mathrm{\Phi }(\widehat{}_v(s)).$$

Assume now that we have extended $`\mathrm{\Phi }`$ to a section of $`Hom(_k,𝒱_k)`$ over $`𝒯`$. Then

(5)
$$\widehat{}_v^e\mathrm{\Phi }=\widehat{}_v^{e,t}\mathrm{\Phi }[\mathrm{\Phi },u(v)]$$

where $`\widehat{}^{e,t}`$ is the trivial connection in the trivial bundle $`End(_k)`$ over $`𝒯`$.

Let us end this section by summerazing the properties we use about the moduli space in section 5 to prove Theorem 6, which intern implies Theorem 1:

The moduli space $`M`$ is a finite dimensional smooth compact manifold with a symplectic structure $`\omega `$, a Hermitian line bundle $``$ and a compatible connection $``$, whose curvature is $`\frac{\sqrt{1}}{2\pi }\omega `$. Teichmüller space $`𝒯`$ is a smooth connected finite dimensional manifold, which smoothly parametrizes Kähler structures $`I_\sigma `$, $`\sigma 𝒯`$ on $`(M,\omega )`$. For any positive integer $`k`$, we have inside the trivial bundle $`_k=𝒯\times C^{\mathrm{}}(M,^k)`$ the finite dimensional subbundle $`𝒱_k`$, given by

$$𝒱_k(\sigma )=H^0(M_\sigma ,_\sigma ^k)$$

for $`\sigma 𝒯`$. We have a connection in $`𝒱_k`$ given by

$$\widehat{}_v=\widehat{}_v^tu(v)$$

where $`\widehat{}_v^t`$ is the trivial connection in $`_k`$ and $`u(v)`$ is the second order differential operator $`u_{G(v)}`$ given in (3). - All we will need about the operator $`\mathrm{\Delta }_G2_{GF}`$, is that there is some finite set of vector fields $`X_r(v),Y_r(v),Z(v)C^{\mathrm{}}(M_\sigma ,T)`$, $`r=1,\mathrm{},R`$ (where $`vT_\sigma (𝒯)`$) all varying smoothly<sup>1</sup><sup>1</sup>1This makes sense when we consider the holomorphic tangent bundle $`T`$ of $`M_\sigma `$ inside the complexified real tangent bundle $`TM`$ of $`M`$. with $`vT(𝒯)`$, such that

(6)
$$\mathrm{\Delta }_{G(v)}2_{G(v)F}=\underset{r=1}{\overset{R}{}}_{X_r(v)}_{Y_r(v)}+_{Z(v)}.$$

This follows immediately from the definition of $`\mathrm{\Delta }_{G(v)}`$. From this we have the expression

(7)
$$u(v)=\frac{1}{2(k+n)}\left(\underset{r=1}{\overset{R}{}}_{X_r(v)}_{Y_r(v)}+_{Z(v)}+nv[F]\right)\frac{1}{2}v[F].$$

All we need to use about $`F:𝒯C^{\mathrm{}}(M)`$ is that it is a smooth function, such that $`F_\sigma `$ is real valued on $`M`$ for all $`\sigma 𝒯`$.

## 4. Toeplitz operators on compact Kähler manifolds

In this section $`(N^{2m},\omega )`$ will denote a compact Kähler manifold on which we have a holomorphic line bundle $`L`$ admitting a compatible Hermitian connection whose curvature is $`\frac{\sqrt{1}}{2\pi }\omega `$. On $`C^{\mathrm{}}(N,L^k)`$ we have the $`L_2`$-inner product:

$$s_1,s_2=\frac{1}{m!}_N(s_1,s_2)\omega ^m$$

where $`s_1,s_2C^{\mathrm{}}(N,L^k)`$ and $`(,)`$ is the fiberwise Hermitian structure in $`L^k`$. This $`L_2`$-inner product gives the orthogonal projection $`\pi :C^{\mathrm{}}(N,L^k)H^0(N,L^k)`$. For each $`fC^{\mathrm{}}(N)`$ consider the associated Toeplitz operator $`T_f^{(k)}`$ given as the composition of the multiplication operator $`M_f:H^0(N,L^k)C^{\mathrm{}}(N,L^k)`$ with the orthogonal projection $`\pi :C^{\mathrm{}}(N,L^k)H^0(N,L^k)`$, so that

$$T_f^{(k)}(s)=\pi (fs).$$

Since the multiplication operator is a zero order differential operator, $`T_f^{(k)}`$ is a zero-order Toeplitz operator. Sometimes we will suppress the superscript $`(k)`$ and just write $`T_f=T_f^{(k)}`$.

Let us here give an explicit formula for $`\pi `$: Let $`h_{ij}=s_i,s_j`$, where $`s_i`$ is a basis of $`H^0(N,L^k)`$. Let $`h_{ij}^1`$ be the inverse matrix of $`h_{ij}`$. Then

(8)
$$\pi (s)=\underset{i,j}{}s,s_ih_{ij}^1s_j.$$

This formula will be useful when we have to compute the derivative of $`\pi `$ along a one-parameter curve of complex structures on the moduli space.

Suppose we have a smooth section $`XC^{\mathrm{}}(N,TN)`$ of the holomorphic tangent bundle of $`N`$. We then claim that the operator $`\pi _X`$ is a zero-order Toeplitz operator. Suppose $`s_1C^{\mathrm{}}(N,L^k)`$ and $`s_2H^0(N,L^k)`$, then we have that

$$X(s_1,s_2)=(_Xs_1,s_2).$$

Now, calculating the Lie derivative along $`X`$ of $`(s_1,s_2)\omega ^m`$ and using the above, one obtains after integration that

$$_Xs_1,s_2=\mathrm{\Lambda }d(i_X\omega )s_1,s_2,$$

where $`\mathrm{\Lambda }`$ denotes contraction with $`\omega `$. Thus

(9)
$$\pi _X=T_{f_X}^{(k)},$$

as operators from $`C^{\mathrm{}}(N,L^k)`$ to $`H^0(N,L^k)`$, where $`f_X=\mathrm{\Lambda }d(i_X\omega )`$. Iterating this, we find for all $`X_1,X_2C^{\mathrm{}}(TN)`$ that

(10)
$$\pi _{X_1}_{X_2}=T_{f_{X_2}f_{X_1}X_2(f_{X_1})}^{(k)}$$

again as operators from $`C^{\mathrm{}}(N,L^k)`$ to $`H^0(N,L^k)`$.

For $`XC^{\mathrm{}}(N,TN)`$, the complex conjugate vector field $`\overline{X}C^{\mathrm{}}(N,\overline{T}N)`$ is a section of the antiholomorphic tangent bundle, and for $`s_1,s_2C^{\mathrm{}}(N,L^k)`$, we have that

$$\overline{X}(s_1,s_2)=(_{\overline{X}}s_1,s_2)+(s_1,_Xs_2).$$

Computing the Lie derivative along $`\overline{X}`$ of $`(s_1,s_2)\omega ^m`$ and integrating, we get that

$$_{\overline{X}}s_1,s_2+(_X)^{}s_1,s_2=\mathrm{\Lambda }d(i_{\overline{X}}\omega )s_1,s_2.$$

Hence we see that

$$(_X)^{}=\left(_{\overline{X}}+f_{\overline{X}}\right)$$

as operators on $`C^{\mathrm{}}(N,L^k)`$. In particular, we see that

(11)
$$\pi (_X)^{}\pi =T_{f_{\overline{X}}}|_{H^0(N,L^k)}:H^0(N,L^k)H^0(N,L^k).$$

For two smooth sections $`X_1,X_2`$ of the holomorphic tangent bundle $`TN`$ and a smooth function $`hC^{\mathrm{}}(N)`$, we deduce from the formula for $`(_X)^{}`$ that

$`\pi (_{X_1})^{}(_{X_2})^{}h\pi `$ $`=`$ $`\pi \overline{X}_1\overline{X}_2(h)\pi +`$
$`\pi f_{\overline{X}_1}\overline{X}_2(h)\pi +\pi f_{\overline{X}_2}\overline{X}_1(h)\pi +`$
$`\pi \overline{X}_1(f_{\overline{X}_2})h\pi +\pi f_{\overline{X}_1}f_{\overline{X}_2}h\pi `$
as operators on $`H^0(N,L^k)`$.

We need the following Theorems on Toeplitz operators. The first is due to Bordemann, Meinrenken and Schlichenmaier (see \[BMS\]). The $`L_2`$-inner product on $`C^{\mathrm{}}(N,L^k)`$ induces an inner product on $`H^0(N,L^k)`$, which in turn induces the operator norm $``$ on $`End(H^0(N,L^k))`$.

###### Theorem 4 (Bordemann, Meinrenken and Schlichenmaier).

For any $`fC^{\mathrm{}}(N)`$ we have that

$$\underset{k\mathrm{}}{lim}T_f^{(k)}=\underset{xN}{sup}|f(x)|.$$

Since the association of the sequence of Toeplitz operators $`T_f^k`$, $`k_+`$ is linear in $`f`$, we see from this Theorem, that this association is faithful.

###### Theorem 5 (Schlichenmaier).

For any pair of smooth functions $`f_1,f_2C^{\mathrm{}}(N)`$, we have an asymptotic expansion

$$T_{f_1}^{(k)}T_{f_2}^{(k)}\underset{l=0}{\overset{\mathrm{}}{}}T_{c_l(f_1,f_2)}^{(k)}k^l,$$

where $`c_l(f_1,f_2)C^{\mathrm{}}(N)`$ are uniquely determined since $``$ means the following: For all $`L_+`$ we have that

(13)
$$T_{f_1}^{(k)}T_{f_2}^{(k)}\underset{l=0}{\overset{L}{}}T_{c_l(f_1,f_2)}^{(k)}k^l=O(k^{(L+1)}).$$

Moreover, $`c_0(f_1,f_2)=f_1f_2`$.

This Theorem was proved in \[Sch\] and is published in \[Sch1\] and \[Sch2\], where it is also proved that the formal generating series for the $`c_l(f_1,f_2)`$’s gives a formal deformation quantization of the Poisson structure on $`N`$ induced from $`\omega `$. By examining the proof in \[Sch\] (or in \[Sch1\] and \[Sch2\]) of this Theorem, one observes that for continuous families of functions, the estimates in Theorem 5 are uniform over compact parameter spaces.

## 5. Toeplitz operators on moduli space and the projective flat connection

Let $`fC^{\mathrm{}}(M)`$ be a smooth function on the moduli space. We consider $`T_f^{(k)}`$ as a section of the endomorphism bundle $`End(𝒱_k)`$. The flat connection $`\widehat{}`$ in the projective bundle $`(𝒱_k)`$ induces the flat connection $`\widehat{}^e`$ in the endomorphism bundle $`End(𝒱_k)`$ as described in section 3. We shall now establish that the sections $`T_f^{(k)}`$ are in a certain sense asymptotically flat by proving the following Theorem.

###### Theorem 6.

Let $`\sigma _0`$ and $`\sigma _1`$ be two points in Teichmüller space and $`P_{\sigma _0,\sigma _1}`$ be the parallel transport in the flat bundle $`End(𝒱_k)`$ from $`\sigma _0`$ to $`\sigma _1`$. Then

$$P_{\sigma _0,\sigma _1}T_{f,\sigma _0}^{(k)}T_{f,\sigma _1}^{(k)}=O(k^1),$$

where $``$ is the operator norm on $`H^0(M_{\sigma _1},_{\sigma _1}^k)`$.

In the proof of this Theorem we will make use of the following Hermitian structure on $`_k`$

(14)
$$s_1,s_2_F=\frac{1}{m!}_M(s_1,s_2)e^F\omega ^m,$$

where we recall that $`F=F_\sigma `$ is the Ricci potential, which is a real smooth function on $`M_\sigma `$ for each $`\sigma 𝒯`$ determined by equation (4). In Lemma 1 below we will see that $`,_F`$ is uniform equivalent to the constant $`L_2`$-Hermitian structure on $`_k`$, when both are restricted to $`𝒱_k`$ over any compact subset of $`𝒯`$. The constant $`L_2`$-Hermitian structure on $`_k`$ is not asymptotically flat with respect to $`\widehat{}`$, but Proposition 2 below shows that the Hermitian structure $`,_F`$ restricted to $`𝒱_k`$ is asymptotically flat with respect to $`\widehat{}`$. - It therefore induces a Hermitian structure on $`𝒱_k^{}𝒱_k`$ (which we also denote $`,_F`$), which is asymptotic flat with respect to $`\widehat{}^e`$.

This suggests one considers the smooth function $`t|P_{\sigma _0,\sigma _t}T_{f,\sigma _0}^{(k)}T_{f,\sigma _t}^{(k)}|_F`$ and establishes an $`O(k^1)`$ estimate for its derivative uniformly over $`J`$, since by Lemma 1 and the first inequality in (17) below, $`O(k^1)`$ control on $`|P_{\sigma _0,\sigma _1}T_{f,\sigma _0}^{(k)}T_{f,\sigma _1}^{(k)}|_F`$ implies Theorem 6. An $`O(k^1)`$ estimate on $`|\widehat{}_{\sigma _t^{}}^eT_{f,\sigma _t}^{(k)}|_F`$ uniformly over $`J`$ would imply this estimate, however we are only able to establish that $`\widehat{}_{\sigma _t^{}}^eT_{f,\sigma _t}^{(k)}`$ is $`O(k^1)`$ uniformly over the interval, which is considerably weaker because of the $`\frac{m}{2}`$’th power of $`k`$ in the second inequality in (17) below.

We shall therefore perturb the function $`f`$ by adding on sufficiently many terms of the form $`(h_l)_tk^l`$ ($`l=1,\mathrm{},r>m/2`$), where $`(h_l)_tC^{\mathrm{}}(M)`$, $`tJ`$, so as to obtain a smooth one-parameter family of functions $`(f_r)_t=f+_{l=1}^r(h_l)_tk^l`$. The $`(h_l)_t`$’s are determined inductively in $`l`$ such that $`\widehat{}_{\sigma _t^{}}^eT_{(f_r)_t,\sigma _t}^{(k)}`$ is $`O(k^{r1})`$ uniformly over $`J`$. This is the content of Proposition 1 below. That estimate on the covariant derivative of the Toeplitz operator $`T_{(f_r)_t,\sigma _t}^{(k)}`$ will allow us to prove that $`P_{\sigma _0,\sigma _1}T_{(f_r)_0,\sigma _0}^{(k)}T_{(f_r)_1,\sigma _1}^{(k)}`$ is $`O(k^1)`$ by analyzing the derivative of

$$t|P_{\sigma _0,\sigma _t}T_{(f_r)_0,\sigma _0}^{(k)}T_{(f_r)_t,\sigma _t}^{(k)}|_F^2.$$

Since we can arrange that $`(f_r)_0=f`$ and since $`T_{(f_r)_1,\sigma _1}^{(k)}T_{f,\sigma _1}^{(k)}`$ is $`O(k^1)`$, this will allow us to prove Theorem 6.

First however, we need to establish a useful formula for the derivative of the orthogonal projection $`\pi `$ along the curve $`\sigma _t`$. To this end, consider a basis of covariant constant sections<sup>2</sup><sup>2</sup>2We will from now on mostly suppress the subscript $`t`$ on various quantities defined along the curve $`\sigma _t`$. $`s_i=(s_i)_t`$, $`i=1,\mathrm{},Rank𝒱_k`$, of $`𝒱_k`$ over the curve $`\sigma _t`$:

$$s_i^{}=u_G(s_i),i=1,\mathrm{},Rank𝒱_k.$$

Recall formula (8) for the projection $`\pi :C^{\mathrm{}}(M,^k)H^0(M_{\sigma _t},_{\sigma _t}^k)`$ and compute the derivative along $`\sigma _t`$: For any fixed $`sC^{\mathrm{}}(M,^k)`$, we have that

$`\pi ^{}(s)`$ $`=`$ $`{\displaystyle \underset{i,j}{}}s,s_i^{}h_{ij}^1s_j`$
$`+{\displaystyle \underset{i,j}{}}s,s_i(h_{ij}^1)^{}s_j`$
$`+{\displaystyle \underset{i,j}{}}s,s_ih_{ij}^1s_j^{}.`$
An easy computation gives that

$$(h_{ij}^1)^{}=\underset{l,r}{}h_{il}^1(s_l^{},s_r+s_l,s_r^{})h_{rj}^1,$$

so

$`\pi \pi ^{}(s)`$ $`=`$ $`{\displaystyle \underset{i,j}{}}u_G^{}s,s_ih_{ij}^1s_j`$
$`{\displaystyle \underset{i,l,m,j}{}}s,s_ih_{il}^1s_l,s_m^{}h_{mj}^1s_j`$
$`=`$ $`\pi u_G^{}(s){\displaystyle \underset{m,j}{}}\pi s,s_m^{}h_{mj}^1s_j`$
$`=`$ $`\pi u_G^{}(s)\pi u_G^{}\pi (s).`$
Hence we conclude that

(15)
$$\pi \pi ^{}=\pi u_G^{}\pi u_G^{}\pi .$$

Having derived the formula for the derivative of $`\pi `$, we now proceed to construct the needed perturbation of $`f`$. Let $`r`$ be a non-negative integer and let

$$(f_r)_t=\underset{i=0}{\overset{r}{}}(h_i)_tk^i,$$

where the $`(h_i)_t`$, $`tJ`$, for now are arbitrary smooth one-parameter families of smooth functions on $`M`$, however we will fix $`(h_0)_t=f`$ for all $`tJ`$. We have that $`T_{f_r}^{(k)}`$ is a section of $`End(𝒱_k)`$ over the curve $`\sigma _t`$. According to formula (5), we have that

$$\widehat{}_\sigma ^{}^e(T_{f_r})=(T_{f_r})^{}[u_G,T_{f_r}]=\pi f_r^{}+\pi ^{}f_r[u_G,\pi f_r].$$

Since $`\widehat{}_\sigma ^{}^e(T_{f_r})`$ is a section of $`End(𝒱_k)`$, we have of course that

$$\pi \widehat{}_\sigma ^{}^e(T_{f_r})\pi =\widehat{}_\sigma ^{}^e(T_{f_r})\pi :H^0(M_{\sigma _t},_{\sigma _t}^k)H^0(M_{\sigma _t},_{\sigma _t}^k).$$

###### Proposition 1.

Given $`f`$ and a non-negative integer $`r`$, there exists unique smooth one-parameter families of functions $`(h_i)_tC^{\mathrm{}}(M)`$, $`i=1,\mathrm{},r`$ and $`tJ`$ such that

(16)
$$\underset{tJ}{sup}\widehat{}_\sigma ^{}^e(T_{f_r})=O(k^{r1}),$$

and $`(h_i)_0=0`$, $`i=1,\mathrm{},r`$.

###### Proof.

For any choice of $`h_i`$’s, $`i=1,\mathrm{},r`$, we compute using (15) that

$$\begin{array}{cc}\hfill \pi \widehat{}_\sigma ^{}^e(T_{f_r})\pi & =\underset{i=1}{\overset{r}{}}\pi h_i^{}\pi k^i\hfill \\ & +\underset{i=0}{\overset{r}{}}(\pi u_G^{}h_i\pi \pi u_G^{}\pi h_i\pi )k^i\hfill \\ & \underset{i=0}{\overset{r}{}}(\pi u_G\pi h_i\pi \pi h_iu_G\pi )k^i.\hfill \end{array}$$

Using (6) together with (9) and (10) we define the function $`H_GC^{\mathrm{}}(M)`$ independent of $`k`$, as follows

$$\pi (\mathrm{\Delta }_G2_{GF})=\pi H_G.$$

We also define $`H_iC^{\mathrm{}}(M)`$ which depend on $`h_i`$ and $`G`$ but are independent of $`k`$, by

$$\pi h_i(\mathrm{\Delta }_G2_{GF})=\pi H_i.$$

Using (11) and (4) we further define the function $`H_G^{}C^{\mathrm{}}(M)`$ independent of $`k`$, as follows

$$\pi (\mathrm{\Delta }_G^{}2(_{GF})^{})\pi =\pi H_G^{}\pi .$$

And we define $`H_i^{}C^{\mathrm{}}(M)`$ which depend on $`h_i`$ and $`G`$ but are independent of $`k`$, by

$$\pi (\mathrm{\Delta }_G^{}2(_{GF})^{})h_i\pi =\pi H_i^{}\pi .$$

Then we have that

$$\begin{array}{cc}\hfill \pi \widehat{}_\sigma ^{}^e(T_{f_r})\pi & =\underset{i=1}{\overset{r}{}}\pi h_i^{}\pi k^i\hfill \\ & \underset{i=0}{\overset{r}{}}\frac{1}{2(k+n)}(\pi H_G^{}\pi h_i\pi \pi H_i^{}\pi )k^i\hfill \\ & \underset{i=0}{\overset{r}{}}\frac{1}{2(k+n)}(\pi H_G\pi h_i\pi \pi H_i\pi )k^i\hfill \\ & +\underset{i=0}{\overset{r}{}}\frac{k}{2(k+n)}\sqrt{1}(\pi \overline{f}_G\pi h_i\pi \pi \overline{f}_Gh_i\pi )k^i\hfill \\ & \underset{i=0}{\overset{r}{}}\frac{k}{2(k+n)}\sqrt{1}(\pi f_G\pi h_i\pi \pi h_if_G\pi )k^i.\hfill \end{array}$$

Because of (13) and Theorem 4, we see for any $`r`$, that condition (16) is equivalent to

$$\begin{array}{cc}\hfill \underset{tJ}{sup}& \underset{i=1}{\overset{r}{}}\pi h_i^{}\pi k^i\hfill \\ & \frac{1}{2k}\underset{i=0}{\overset{m}{}}\underset{j=0}{\overset{r}{}}(1)^jn^j(\underset{l=0}{\overset{r}{}}\pi c_l(H_G^{},h_i)\pi k^l\pi H_i^{}\pi )k^{ij}\hfill \\ & \frac{1}{2k}\underset{i=0}{\overset{r}{}}\underset{j=0}{\overset{r}{}}(1)^jn^j(\underset{l=0}{\overset{r}{}}\pi c_l(H_G,h_i)\pi k^l\pi H_i\pi )k^{ij}\hfill \\ & +\frac{\sqrt{1}}{2k}\underset{i=0}{\overset{r}{}}\underset{j=0}{\overset{r}{}}\underset{l=1}{\overset{r}{}}(1)^jn^j\pi c_l(\overline{f}_G,h_i)\pi k^{lij+1}\hfill \\ & \frac{\sqrt{1}}{2k}\underset{i=0}{\overset{r}{}}\underset{j=0}{\overset{r}{}}\underset{l=1}{\overset{r}{}}(1)^jn^j\pi c_l(f_G,h_i)\pi k^{lij+1}=O(k^{r1}).\hfill \end{array}$$

Note that the expression inside the norm is a polynomial in $`k^1`$, say $`_{i=1}^{N_r}\pi (C_i)_t\pi k^i`$, where each coefficient $`\pi (C_i)_t\pi `$ is the Toeplitz operator associated to a smooth one-parameter family $`(C_i)_tC^{\mathrm{}}(M)`$, $`tJ`$. We also note that the functions $`C_i`$, $`i=1,\mathrm{},r`$ do not depend on $`r`$ and in the same range for $`i`$ they are all of the following form

$$C_i=h_i^{}D_i$$

where $`(D_i)_tC^{\mathrm{}}(M)`$, $`tJ`$, is a linear combination of

$$H_j^{},H_j,c_l(H_G^{},h_j),c_l(H_G,h_j),c_l(\overline{f}_G,h_j),c_l(f_G,h_j)C^{\mathrm{}}(J,C^{\mathrm{}}(M)),$$

for $`j=0,\mathrm{},i1`$ and $`l=0,\mathrm{},i`$, e.g. for $`i=1`$ we get

$$D_1=\frac{1}{2}(H_G^{}h_0H_0^{}H_Gh_0H_0\sqrt{1}c_1(\overline{f}_G,h_0)+\sqrt{1}c_1(f_G,h_0)).$$

Now observe by Theorem 4 that condition (16) is equivalent to $`C_i=0`$ for $`i=1,\mathrm{}r`$. From this it follows that we can inductively uniquely determine the functions $`h_i`$. First of all, since the coefficient of $`k^0`$ is zero, we see that (16) holds for $`r=0`$. Now assume that we have determined $`h_1,\mathrm{}h_{r1}`$ uniquely such that $`(h_i)_0=0`$ and

$$h_i^{}=D_i,$$

for $`i=1,\mathrm{},r1`$. By the above we then observe that condition (16) holds if and only if $`C_r=0`$, which means if and only if

$$h_r^{}=D_r.$$

But since we require $`(h_r)_0=0`$, we must have that

$$(h_r)_t=_0^t(D_r)_s𝑑s.$$

$`\mathrm{}`$

###### Lemma 1.

The Hermitian structure on $`_k`$

$$s_1,s_2_F=\frac{1}{m!}_M(s_1,s_2)e^F\omega ^m$$

and the constant $`L_2`$-Hermitian structure on $`_k`$

$$s_1,s_2=\frac{1}{m!}_M(s_1,s_2)\omega ^m$$

are equivalent uniformly in $`k`$ when restricted to $`𝒱_k`$ over any compact subset $`K`$ of $`𝒯`$.

###### Proof.

We clearly have that

$$|s|_F^2T_{e^F}^{(k)}|s|^2,$$

so by Theorem 4 we see that there exists a constant $`C`$ (depending on K) such that

$$|s|_FC|s|,$$

for all $`k`$. Conversely, we have that

$$\begin{array}{cc}\hfill |s|^2& =\pi e^{\frac{1}{2}F}e^{\frac{1}{2}F}\pi s,s\hfill \\ & |(\pi e^{\frac{1}{2}F}e^{\frac{1}{2}F}\pi \pi e^{\frac{1}{2}F}\pi e^{\frac{1}{2}F}\pi )s,s|\hfill \\ & +|\pi e^{\frac{1}{2}F}\pi e^{\frac{1}{2}F}\pi s,s|\hfill \\ & \pi e^{\frac{1}{2}F}e^{\frac{1}{2}F}\pi \pi e^{\frac{1}{2}F}\pi e^{\frac{1}{2}F}\pi |s|^2\hfill \\ & +\pi e^{\frac{1}{2}F}\pi |\pi e^{\frac{1}{2}F}\pi s||s|.\hfill \end{array}$$

By Theorem 4 and 5 we see there exists constants $`C^{}`$ and $`C^{\prime \prime }`$ (again depending on K) such that

$$|s|\frac{C^{}}{k}|s|+C^{\prime \prime }|s|_F.$$

But then we have for all sufficiently large $`k`$ that

$$|s|2C^{\prime \prime }|s|_F.$$

Hence we have established the claimed equivalence.

$`\mathrm{}`$

Along any smooth one-parameter family of complex structures $`\sigma _t`$, we have that

$$\frac{d}{dt}s_1,s_2_F=\widehat{}_{\sigma _t^{}}^ts_1,s_2_F+s_1,\widehat{}_{\sigma _t^{}}^ts_2_F\frac{F}{t}s_1,s_2_F.$$

So, if we let

$$E(s)=\frac{d}{dt}|s|_F^2\widehat{}_{\sigma _t^{}}s,s_Fs,\widehat{}_{\sigma _t^{}}s_F$$

recalling that $`\sqrt{1}f_G=\frac{F}{t}`$, $`\widehat{}_v=\widehat{}_v^tu(v)`$ and formula (7), we have for all sections $`s`$ of $`𝒱_k`$ that

$`E(s)`$ $`=`$ $`{\displaystyle \frac{1}{2(k+n)}}(\pi e^F(\mathrm{\Delta }_G2_{GF}\sqrt{1}nf_G)s,s`$
$`+s,\pi e^F(\mathrm{\Delta }_G2_{GF}\sqrt{1}nf_G)s).`$
Hence by combining Theorem 4, (6), (9) and (10) we have proved that

###### Proposition 2.

The Hermitian structure (14) is asymptotically flat with respect to the connections $`\widehat{}`$, i.e. for any compact subset $`K`$ of $`𝒯`$, there exists a constant $`C`$ such that for all sections $`s`$ of $`𝒱_k`$ over $`K`$, we have that

$$|E(s)|\frac{C}{k+n}|s|_F^2$$

over $`K`$.

We note that this proposition implies the same proposition for sections of $`End(𝒱_k)`$ with respect to the induced Hermitian structure on $`End(𝒱_k)=𝒱_k^{}𝒱_k`$, which we also denote $`,_F`$. We denote the analogous quantity of $`E`$ for the endomorphism bundle by $`E_e`$.

###### Proof of Theorem 6.

Let $`\sigma _t`$, $`tJ`$ be a smooth one-parameter family of complex structures such that $`\sigma _t`$ is a curve in $`𝒯`$ between the two points in question. By Lemma 1 the Hermitian structures $`,`$ and $`,_F`$ on $`𝒱_k`$ are equivalent uniformly in $`k`$ over compact subsets of $`𝒯`$. It follows there exists constants $`C_1`$ and $`C_2`$, such that we have the following inequalities over the image of $`\sigma _t`$ in $`𝒯`$ for the operator norm $``$ and the norm $`||_F`$ on $`End(𝒱_k)`$:

(17)
$$C_1||_FC_2\sqrt{P_{g,n}(k)},$$

where $`P_{g,n}(k)`$ is the rank of $`𝒱_k`$ given by the Verlinde formula. By the Riemann-Roch Theorem this is a polynomial in $`k`$ of degree $`m`$.

Because of these inequalities, we choose an integer $`r`$ bigger than $`m/2`$ and let $`f_r`$ be as provided by Proposition 1 and we define $`n_k:J[0,\mathrm{})`$ by

$$n_k(t)=|\mathrm{\Theta }_k(t)|_F^2$$

where

$$\mathrm{\Theta }_k(t):𝒱_{k,\sigma _t}𝒱_{k,\sigma _t}$$

is given by

$$\mathrm{\Theta }_k(t)=P_{\sigma _0,\sigma _t}T_{(f_r)_0,\sigma _0}^{(k)}T_{(f_r)_t,\sigma _t}^{(k)}.$$

The functions $`n_k`$ are differentiable in $`t`$ and we compute that

$$\begin{array}{cc}\hfill \frac{dn_k}{dt}& =\widehat{}_{\sigma _t^{}}^e(\mathrm{\Theta }_k(t)),\mathrm{\Theta }_k(t)_F+\mathrm{\Theta }_k(t),\widehat{}_{\sigma _t^{}}^e(\mathrm{\Theta }_k(t))_F+E_e(\mathrm{\Theta }_k(t))\hfill \\ & =\widehat{}_{\sigma _t^{}}^eT_{(f_r)_t,\sigma _t}^{(k)},\mathrm{\Theta }_k(t)_F\mathrm{\Theta }_k(t),\widehat{}_{\sigma _t^{}}^eT_{(f_r)_t,\sigma _t}^{(k)}_F+E_e(\mathrm{\Theta }_k(t)).\hfill \end{array}$$

Using the above, we get the following estimate

$$\begin{array}{cc}\hfill |\frac{dn_k}{dt}|& 2|\widehat{}_{\sigma _t^{}}^eT_{(f_r)_t,\sigma _t}^{(k)}|_F|\mathrm{\Theta }_k(t)|_F+|E_e(\mathrm{\Theta }_k(t))|\hfill \\ & 2C_2\sqrt{P_{g,n}(k)}\widehat{}_{\sigma _t^{}}^eT_{(f_r)_t,\sigma _t}^{(k)}n_k^{1/2}+|E_e(\mathrm{\Theta }_k(t))|.\hfill \end{array}$$

Consequently we can apply Proposition 1 and 2 to obtain that there exists a constant $`C`$ such that

$$|\frac{dn_k}{dt}|\frac{C}{k}(n_k^{1/2}+n_k).$$

This estimate implies that

$$n_k(t)(\mathrm{exp}(\frac{Ct}{2k})1)^2.$$

But by (17) we get that

$$P_{\sigma _0,\sigma _1}T_{(f_r)_0,\sigma _0}^{(k)}T_{(f_r)_1,\sigma _1}^{(k)}=\mathrm{\Theta }_k(1)C_1n_k(1)^{1/2}.$$

The Theorem then follows from these two estimates, since $`(f_r)_0=f`$ and

$$T_{(f_r)_1,\sigma _1}^{(k)}T_{f,\sigma _1}^{(k)}=O(k^1).$$

$`\mathrm{}`$

## 6. Asymptotic faithfulness

Recall that the flat connection in the bundle $`(𝒱_k)`$ gives the projective representation of the mapping class group

$$\rho _k:\mathrm{\Gamma }Aut((V_k)),$$

where $`(V_k)=`$ covariant constant sections of $`(𝒱_k)`$ over Teichmüller space.

###### Theorem 7.

For any $`\varphi \mathrm{\Gamma }`$, we have that

$$\varphi \underset{k=1}{\overset{\mathrm{}}{}}\mathrm{ker}\rho _k$$

if and only if $`\varphi `$ induces the identity on $`M`$.

###### Proof.

Suppose we have a $`\varphi \mathrm{\Gamma }`$. Then $`\varphi `$ induces a symplectomorphism of $`M`$ which we also just denote $`\varphi `$ and we get the following commutative diagram for any $`fC^{\mathrm{}}(M)`$

$$\begin{array}{ccccc}H^0(M_\sigma ,_\sigma ^k)& \stackrel{\varphi ^{}}{}& H^0(M_{\varphi (\sigma )},_{\varphi (\sigma )}^k)& \stackrel{P_{\varphi (\sigma ),\sigma }}{}& H^0(M_\sigma ,_\sigma ^k)\\ T_{f,\sigma }^{\left(k\right)}& & T_{f\varphi ,\varphi \left(\sigma \right)}^{\left(k\right)}& & P_{\varphi \left(\sigma \right),\sigma }T_{f\varphi ,\varphi \left(\sigma \right)}^{\left(k\right)}& & \\ H^0(M_\sigma ,_\sigma ^k)& \stackrel{\varphi ^{}}{}& H^0(M_{\varphi (\sigma )},_{\varphi (\sigma )}^k)& \stackrel{P_{\varphi (\sigma ),\sigma }}{}& H^0(M_\sigma ,_\sigma ^k),\end{array}$$

where $`P_{\varphi (\sigma ),\sigma }:H^0(M_{\varphi (\sigma )},_{\varphi (\sigma )}^k)H^0(M_\sigma ,_\sigma ^k)`$ on the horizontal arrows refer to parallel transport in the Verlinde bundle itself, whereas $`P_{\varphi (\sigma ),\sigma }`$ refers to the parallel transport in the endomorphism bundle $`End(𝒱_k)`$ in the last vertical arrow. Suppose now $`\varphi _{k=1}^{\mathrm{}}\mathrm{ker}\rho _k`$, then $`P_{\varphi (\sigma ),\sigma }\varphi ^{}=\rho _k(\varphi )Id`$ and we get that $`T_{f,\sigma }^{(k)}=P_{\varphi (\sigma ),\sigma }T_{f\varphi ,\varphi (\sigma )}^{(k)}`$. By Theorem 6 we get that

$$\begin{array}{cc}\hfill \underset{k\mathrm{}}{lim}T_{ff\varphi ,\sigma }^{(k)}& =\underset{k\mathrm{}}{lim}T_{f,\sigma }^{(k)}T_{f\varphi ,\sigma }^{(k)}\hfill \\ & =\underset{k\mathrm{}}{lim}P_{\varphi (\sigma ),\sigma }T_{f\varphi ,\varphi (\sigma )}^{(k)}T_{f\varphi ,\sigma }^{(k)}=0.\hfill \end{array}$$

By Bordemann, Meinrenken and Schlichenmaier’s Theorem 4, we must have that $`f=f\varphi `$. Since this holds for any $`fC^{\mathrm{}}(M)`$, we must have that $`\varphi `$ acts by the identity on $`M`$.

$`\mathrm{}`$

###### Proof of Theorem 1.

Our main Theorem 1 now follows directly from Theorem 7, since it is known that the only element of $`\mathrm{\Gamma }`$, which acts by the identity on the moduli space $`M`$ is the identity, if $`g>2`$. If $`g=2`$, $`n=2`$ and $`d`$ is even, it is contained in the sub-group generated by the hyper-elliptic involution, else it is also the identity.

A way to see this using the moduli space of flat $`SL(n,)`$-connections goes as follows: We have that $`M`$ is a component of the real slice in $``$, the moduli space of flat $`SL(n,C)`$-connections on $`\mathrm{\Sigma }p`$, whose holonomy around $`p`$ has trace $`n\mathrm{exp}(2\pi \sqrt{1}\frac{d}{n})`$. Hence if $`\varphi `$ acts by the identity on $`M`$, it will also act by the identity on an open neighbourhood of $`M`$ in $``$, since it acts holomorphically on $``$. But since $``$ is connected, $`\varphi `$ must act by the identity on the entire $`SL(n,)`$-moduli space $``$. Now the generalized Teichmüller space $`\stackrel{~}{𝒯}_p`$ of $`\mathrm{\Sigma }p`$ is also included in $``$, hence we get that $`\varphi `$ acts by the identity on $`\stackrel{~}{𝒯}_p`$. But then the statement about $`\varphi `$ follows by classical theory of how $`\mathrm{\Gamma }`$ acts on $`\stackrel{~}{𝒯}_p`$.

$`\mathrm{}`$