text:
# Higher-order ℏ corrections in the semiclassical quantization of chaotic billiards

## 1 Introduction

The relation between the eigenvalue spectrum of a quantum system and the periodic orbits of the corresponding classical system is a question of fundamental importance for both integrable and chaotic dynamical systems. The well-established Gutzwiller trace formula Gut67 ; Gut71 ; Gut90  for classically chaotic systems and its analogue for integrable systems, the Berry-Tabor formula Ber76 ; Ber77 , provide the semiclassical density of states in terms of a sum over all periodic orbits of the system. However, each trace formula is only the leading-order term of an expansion of the exact density of states in powers of $`\mathrm{}`$, and therefore in general the resulting semiclassical eigenvalues are only approximations to the exact quantum ones. In recent years, two basic methods have been developed for determining higher-order $`\mathrm{}`$ corrections to the Gutzwiller trace formula in terms of periodic orbit contributions Alo93 ; Gas93 ; Vat94 ; Vat96 ; Ros94 ; Gre01 . Unfortunately, even with $`\mathrm{}`$ corrections included, the trace formulae usually suffer from being divergent in the region where the physical eigenvalues or resonances are located. For specific systems, higher-order $`\mathrm{}`$ corrections to the semiclassical eigenvalues have explicitly been calculated by cycle expansion techniques Alo93 ; Vat96 ; Ros94 . However, this method is applicable only to systems with special features, namely to hyperbolic systems with a known complete symbolic dynamics.

Recently, it has been demonstrated how the convergence problems of the semiclassical trace formulae can be circumvented by the application of harmonic inversion techniques Mai97b ; Mai98 ; Mai99a . The harmonic inversion method is capable of extracting semiclassical eigenvalues from a finite set of periodic orbits with very high precision and resolution. In contrast to other semiclassical methods, harmonic inversion does not require any special properties of the system, and can therefore be applied to a wide range of physical systems. In Refs. Mai99a ; Mai98c ; Wei00  a general procedure has been developed for including higher-order $`\mathrm{}`$ corrections to the trace formulae in the harmonic inversion scheme. So far, this method has only been tested for an integrable system, viz. the circle billiard. The general procedure, however, does not depend on the type of the underlying classical dynamics, and is applicable also to chaotic systems. In this paper, we demonstrate how the method works for chaotic systems, and apply it to the open three-disk scatterer, which has become a standard example for the semiclassical quantization of chaotic systems Mai98 ; Mai99a ; Gas89 ; Cvi89 ; Eck93 ; Eck95 ; Wir99 .

The harmonic inversion method is used in two directions: First, we carry out a harmonic analysis of the spectrum of the differences between the exact (complex) quantum eigenvalues and the semiclassical resonances of the three-disk system. We show how this enables one to determine, for each orbit, the first-order $`\mathrm{}`$ correction term (and, in principle, all higher-order correction terms) to the Gutzwiller formula. We confirm our results by comparing with the values calculated by a specialization of an analytical approach developed by Vattay and Rosenqvist Vat94 ; Vat96  for two-dimensional billiards Ros94 . Second, we take the analytical correction terms to the Gutzwiller formula and compute, using the classical periodic orbit data and harmonic inversion, the first-order $`\mathrm{}`$ corrections to the semiclassical resonances of the three-disk system. Thus we illustrate that corrected semiclassical eigenvalues can be calculated by harmonic inversion despite the convergence problems of the trace formulae. We compare the zeroth and first-order approximations to the resonances with the exact quantum eigenvalues, and can quantitatively assess the increase in accuracy produced by including the next-order corrections.

## 2 Higher-order $`\mathrm{}`$ corrections to Gutzwiller’s trace formula

Gutzwiller’s trace formula for chaotic systems gives a semiclassical approximation to the response function (i.e., the trace of the Green’s function) of a quantum system in terms of the periodic orbits of the corresponding classical system. The semiclassical response function consists of a smooth background, and an oscillating part $`g(E)=\overline{g}(E)+g^{\mathrm{osc}}(E)`$, where the oscillating part is given by Gut90 ; Bal74

$$g^{\mathrm{osc}}(E)=\mathrm{i}\underset{\mathrm{po}}{}\frac{T_{\mathrm{po}}}{r|det(M_{\mathrm{po}}\mathrm{𝟏})|^{1/2}}\mathrm{exp}\left[\mathrm{i}\left(\frac{S_{\mathrm{po}}}{\mathrm{}}\mu _{\mathrm{po}}\frac{\pi }{2}\right)\right].$$
(1)
The sum in (1) runs over all periodic orbits (po) of the system, including multiple traversals. Here, $`T_{\mathrm{po}}`$ and $`S_{\mathrm{po}}`$ are the period and the action of the orbit, $`M_{\mathrm{po}}`$ and $`\mu _{\mathrm{po}}`$ denote the monodromy matrix and the Maslov index, and the repetition number $`r`$ counts the traversals of the underlying primitive orbit (“primitive” means that there is no sub-period). The semiclassical density of states $`\rho (E)`$ is related to the response function via

$$\rho (E)=\frac{1}{\pi }\mathrm{Im}g(E).$$
(2)
In general, the semiclassical eigenvalues or resonances obtained from the Gutzwiller formula agree with the exact quantum ones only in leading order of $`\mathrm{}`$. To improve the accuracy of the semiclassical eigenvalues, higher-order $`\mathrm{}`$ correction terms to the Gutzwiller formula have to be included. Two different methods for the calculation of such higher-order $`\mathrm{}`$ terms have been derived for chaotic systems, one by Gaspard and Alonso Alo93 ; Gas93 , and the other by Vattay and Rosenqvist Vat94 ; Vat96 ; Ros94 . The latter method has been specialized to two-dimensional chaotic billiards in Ref. Ros94 . An extension of the method of Gaspard and Alonso has recently been published in Ref. Gre01 . We will adopt the method of Vattay and Rosenqvist to compute the first-order $`\mathrm{}`$ corrections to the semiclassical resonances of the open three-disk scatterer.

Vattay and Rosenqvist give a quantum generalization of the Gutzwiller formula, which is of the form

$$g(E)=\overline{g}(E)+\frac{1}{\mathrm{i}\mathrm{}}\underset{p}{}\underset{l}{}\left(T_p(E)\mathrm{i}\mathrm{}\frac{\mathrm{d}\mathrm{ln}R_p^l(E)}{\mathrm{d}E}\right)\underset{r=1}{\overset{\mathrm{}}{}}\left(R_p^l(E)\right)^r\mathrm{exp}\left(\frac{\mathrm{i}}{\mathrm{}}rS_p(E)\right).$$
(3)
The first sum runs over all primitive periodic orbits; $`T_p`$ and $`S_p`$ are the traversal time and the action of the periodic orbit, respectively. The sum over $`r`$ corresponds to multiple traversals of the primitive orbit. The quantities $`R_p^l`$ are associated with the local eigenspectra determined by the local Schrödinger equation in the neighbourhood of the periodic orbits. An expansion of the quantities $`R_p^l`$ in powers of $`\mathrm{}`$,

$`R^l(E)`$ $`=`$ $`\mathrm{exp}\left\{{\displaystyle \underset{m=0}{\overset{\mathrm{}}{}}}\left({\displaystyle \frac{\mathrm{i}\mathrm{}}{2}}\right)^mC_l^{(m)}\right\}`$ (4)
$``$ $`\mathrm{exp}\left(C_l^{(0)}\right)\left(1+{\displaystyle \frac{\mathrm{i}\mathrm{}}{2}}C_l^{(1)}+\mathrm{}\right),`$
yields the $`\mathrm{}`$ expansion of the generalized trace formula (3). For two-dimensional hyperbolic systems, the zeroth-order terms are given by

$$\mathrm{exp}\left(C_l^{(0)}\right)=\frac{\mathrm{e}^{\mathrm{i}\mu _p\pi /2}}{|\lambda _p|^{1/2}\lambda _p^l},$$
(5)
where $`\mu _p`$ and $`\lambda _p`$ are the Maslov index and the expanding stability eigenvalue (i.e., the stability eigenvalue with an absolute value larger than one) of the orbit, respectively. By summation over $`l`$, the Gutzwiller trace formula is regained as zeroth-order approximation to Eq. (3). If the zeroth-order terms do not depend on the energy, as is the case for billiard systems, the first-order correction to the Gutzwiller formula can be written as

$$g_1(E)=\frac{1}{\mathrm{i}\mathrm{}}\underset{\mathrm{po}}{}\underset{l}{}\frac{T_{\mathrm{po}}(E)}{r}\mathrm{exp}\left(C_l^{(0)}\right)\frac{\mathrm{i}\mathrm{}}{2}C_l^{(1)}\mathrm{exp}\left(\frac{\mathrm{i}}{\mathrm{}}S_{\mathrm{po}}(E)\right),$$
(6)
where the first sum in Eq. (6) now runs over all periodic orbits, including multiple traversals, and $`r`$ is the repetition number with respect to the underlying primitive orbit.

An explicit recipe for the calculation of the correction terms $`C_l^{(1)}`$ for two-dimensional chaotic billiards was given in Ref. Ros94 . The correction terms must in general be calculated numerically from the periodic orbit data. A numerical code which determines the first-order corrections for two-dimensional chaotic billiards can also be found in Ref. Ros94 . We have used that code to compute the correction terms $`C_l^{(1)}`$ for the three-disk system for a comparison with the correction terms calculated by harmonic inversion.

## 3 The open three-disk scatterer

As a model system for the calculation of higher-order $`\mathrm{}`$ corrections to the Gutzwiller formula by harmonic inversion, we consider the open three-disk system, which consists of three equally spaced hard disks of unit radius. This system, in particular the case of the relatively large disk separation $`d=6`$, has served as an archetype for the application of semiclassical quantization techniques in many investigations in recent years Mai98 ; Mai99a ; Cvi89 ; Eck93 ; Eck95 ; Wir99 . We will consider the case $`d=6`$, as well as the small separation $`d=2.5`$. In our calculations, we make use of the symmetry reduction of the three-disk system introduced in Refs. Cvi89 ; Cvi93  and concentrate on states of the $`A_1`$ subspace.

As for all billiard systems, the shape of the periodic orbits in the three-disk system is independent of the wave number $`k=\sqrt{2mE}/\mathrm{}`$, and the action scales as

$$S/\mathrm{}=ks,$$
(7)
where the scaled action $`s`$ is equal to the physical length of the orbit. We consider the density of states as a function of the wave number

$$\rho (k)=\frac{1}{\pi }\mathrm{Im}g(k),$$
(8)
with a scaled response function $`g(k)`$. Since the wave number $`k`$ is proportional to $`\mathrm{}^1`$, it can be considered as an effective Planck constant,

$$k=\mathrm{}_{\mathrm{eff}}^1.$$
(9)
The $`\mathrm{}`$ expansion of the exact quantum response function can therefore be written as a power series in $`k^1`$:

$$g(k)=\overline{g}(k)+g^{\mathrm{osc}}(k)$$
(10)
with

$$g^{\mathrm{osc}}(k)=\underset{n=0}{\overset{\mathrm{}}{}}g_n(k)=\underset{n=0}{\overset{\mathrm{}}{}}\frac{1}{k^n}\underset{\mathrm{po}}{}𝒜_{\mathrm{po}}^{(n)}\mathrm{e}^{\mathrm{i}s_{\mathrm{po}}k}.$$
(11)
The second sum runs over all periodic orbits including multiple traversals. The zeroth-order amplitudes $`𝒜_{\mathrm{po}}^{(0)}`$ correspond to the Gutzwiller formula, whereas for $`n>0`$, the amplitudes $`𝒜_{\mathrm{po}}^{(n)}`$ give the $`n^{\mathrm{th}}`$-order corrections $`g_n(k)`$ to the response function.

Applying the Gutzwiller trace formula to the (symmetry reduced) three-disk system yields for the zeroth-order amplitudes in Eq. (11) ($`A_1`$ subspace)

$`𝒜_{\mathrm{po}}^{(0)}`$ $`=`$ $`\mathrm{i}{\displaystyle \underset{\mathrm{po}}{}}{\displaystyle \frac{s_{\mathrm{po}}\mathrm{e}^{\mathrm{i}\frac{\pi }{2}\mu _{\mathrm{po}}}}{r|det(M_{\mathrm{po}}1)|^{1/2}}}`$ (12)
$`=`$ $`\mathrm{i}{\displaystyle \underset{\mathrm{po}}{}}(1)^{l_s}{\displaystyle \frac{s_{\mathrm{po}}}{r|(\lambda _{\mathrm{po}}1)(\frac{1}{\lambda _{\mathrm{po}}}1)|^{1/2}}},`$
where $`M_{\mathrm{po}}`$ is the monodromy matrix of the orbit, $`l_s`$ is the symbol length, $`s_{\mathrm{po}}`$ the scaled action, and $`\lambda _{\mathrm{po}}`$ the expanding stability eigenvalue of the orbit. The Maslov index $`\mu _{\mathrm{po}}`$ for this system is given by $`2l_s`$. The quantity $`r`$ designates the repetition number with respect to the corresponding primitive orbit. The first-order amplitudes of the $`\mathrm{}`$ expansion (11) following from Eq. (6) read

$$𝒜_{\mathrm{po}}^{(1)}=\frac{s_{\mathrm{po}}}{r}\underset{l}{}\frac{(1)^{l_s}}{|\lambda _{\mathrm{po}}|^{1/2}\lambda _{\mathrm{po}}^l}\frac{C_l^{(1)}}{2\mathrm{}k}.$$
(13)
Since the terms $`C_l^{(1)}`$ are proportional to the momentum $`\mathrm{}k`$, as was shown in Ref. Ros94 , the amplitudes are independent of the scaling parameter $`k`$. The correction terms $`C_l^{(1)}`$ have to be determined numerically. We use the code developed by Rosenqvist and Vattay Ros94 ; Vat . The code requires the flight times between the bounces and the reflection angles as an input. These parameters have to be calculated numerically for each periodic orbit. As the contributions to the amplitude (13) for different $`l`$ are proportional to $`|\lambda _{\mathrm{po}}|^{l\frac{1}{2}}`$, the sum over $`l`$ converges fast if the absolute value of the stability eigenvalue $`\lambda _{\mathrm{po}}`$ is large. For most orbits, the leading term $`l=0`$ turns out to be already sufficient. It is only for the very shortest orbits that terms of higher order in $`l`$ have to be included to ensure convergence of the sum to within, say, 3 significant digits.

## 4 Harmonic analysis of the quantum spectrum

### 4.1 Theory

In Refs. Mai99a ; Wei00  it was demonstrated that the amplitudes $`𝒜_{\mathrm{po}}^{(n)}`$ of the $`\mathrm{}`$ expansion (11) can be obtained by a harmonic inversion analysis of the exact quantum spectrum. The general procedures do not depend on any special properties of the system and can be applied to both integrable and chaotic systems. In Ref. Wei00 , they were tested for the circle billiard, as an example of an integrable system. We will now use the same procedures for the open three-disk system, as a representative of a chaotic system.

We start by briefly recapitulating the main ideas of the procedures developed in Refs. Mai99a ; Wei00 . The exact quantum mechanical response function, in terms of the wave number $`k`$, can be written as

$$g^{\mathrm{qm}}(k)=\underset{j}{}\frac{m_j}{kk_j+\mathrm{i0}},$$
(14)
where the $`k_j`$ are the exact eigenvalues or resonances of $`k`$, and $`m_j`$ are their multiplicities. Eq. (11) gives the $`\mathrm{}`$ expansion of (14) in terms of periodic orbit contributions. The first-order amplitudes $`𝒜_{\mathrm{po}}^{(0)}`$ of the expansion (11) can be determined by adjusting the exact response function (14) to the form of the semiclassical approximation

$$g^{\mathrm{osc}}(k)\underset{\mathrm{po}}{}𝒜_{\mathrm{po}}^{(0)}\mathrm{e}^{\mathrm{i}ks_{\mathrm{po}}}$$
(15)
by harmonic inversion Mai97a . Note that for chaotic billiards the leading-order amplitudes $`𝒜_{\mathrm{po}}^{(0)}`$ as well as the higher-order amplitudes $`𝒜_{\mathrm{po}}^{(n)}`$ are independent of the wave number $`k`$. In a direct harmonic analysis of the quantum spectrum only the zeroth-order term of the expansion (11) fulfills the ansatz for the harmonic inversion procedure. The higher-order terms act as a kind of weak noise which is separated from the zeroth-order “signal” by the harmonic inversion procedure. The direct harmonic analysis of the quantum signal will therefore yield exactly the lowest-order amplitudes $`𝒜_{\mathrm{po}}^{(0)}`$.

The $`n^{\mathrm{th}}`$-order amplitudes $`𝒜_{\mathrm{po}}^{(n)}`$ can be determined if the exact eigenvalues $`k_j`$ and their $`(n1)^{\mathrm{st}}`$-order approximations $`k_{j,n1}`$ are given. The $`(n1)^{\mathrm{st}}`$-order approximation to the response function can be written in the form (14), with $`k_j`$ replaced with the approximation $`k_{j,n1}`$. One can then calculate the difference between the exact quantum mechanical and the $`(n1)^{\mathrm{st}}`$-order response function and compare it with the expression resulting from the expansion (11),

$$g^{\mathrm{qm}}(k)\underset{m=0}{\overset{n1}{}}g_m(k)=\underset{m=n}{\overset{\mathrm{}}{}}g_m(k)=\underset{m=n}{\overset{\mathrm{}}{}}\frac{1}{k^m}\underset{\mathrm{po}}{}𝒜_{\mathrm{po}}^{(m)}\mathrm{e}^{\mathrm{i}s_{\mathrm{po}}k}.$$
(16)
The leading-order terms in (16) are $`k^n`$, i.e., multiplication by $`k^n`$ yields

$$k^n\left[g^{\mathrm{qm}}(k)\underset{m=0}{\overset{n1}{}}g_m(k)\right]=\underset{\mathrm{po}}{}𝒜_{\mathrm{po}}^{(n)}\mathrm{e}^{\mathrm{i}s_{\mathrm{po}}k}+𝒪\left(\frac{1}{k}\right).$$
(17)
The right-hand side of (17) now has assumed a form which is again suited to the harmonic inversion procedure. More precisely, the harmonic inversion of the weighted difference spectrum (17) will yield the periods $`s_{\mathrm{po}}`$ and the $`n^{\mathrm{th}}`$-order amplitudes $`𝒜_{\mathrm{po}}^{(n)}`$ of the $`\mathrm{}`$ expansion (11).

### 4.2 Application to the three-disk scattering system

We now apply the procedure to the three-disk system with disk separation $`d=6`$. As a first step we perform a harmonic analysis of the exact quantum spectrum to obtain the leading-order ($`n=0`$) periodic orbit contributions to the density of states. The exact quantum values for $`d=6`$ were taken from Wirzba Wir99 ; Cvi97 ; Wir . Figure 1 shows the quantum mechanical density of states $`\rho (k)=(1/\pi )\mathrm{Im}g(k)`$ for real values of the wave number $`k`$ resulting from the four leading bands of the $`A_1`$ subspace. Note that this set of resonances is of course not complete as the subleading bands with large negative imaginary part are not included.

The spectrum in Figure 1 served as the signal for the harmonic inversion procedure. The results of the analysis turned out to be more accurate if the lowest part of the signal, determined by the “most quantum” resonances with very small real part, is cut off. We analyzed the spectrum in the range $`\mathrm{Re}k[50,250]`$ to obtain the periodic orbit contributions in two different intervals of the scaled action. The results are presented in Figure 2. The solid lines give the sizes of the imaginary parts of the semiclassical amplitudes $`𝒜_{\mathrm{po}}^{(0)}`$ calculated directly from Eq. (12) using the classical periodic orbit data, as a function of the scaled action of the orbits. The crosses show the amplitudes resulting from the harmonic inversion of the quantum spectrum. Note the different scales of the two plots. The results of the harmonic inversion are seen to be in excellent agreement with those from the classical calculations of the amplitudes $`𝒜_{\mathrm{po}}^{(0)}`$ entering into Gutzwiller’s trace formula, clearly confirming the validity of the latter.

In a second step, we now determine the next-to-leading-order $`\mathrm{}`$ corrections to the Gutzwiller trace formula for the three-disk system by the harmonic analysis of the difference spectrum between the exact quantum resonances and the semiclassical resonances of the $`A_1`$ subspace. The semiclassical resonances for disk separation $`d=6`$ had been calculated by Wirzba Wir99 ; Cvi97 ; Wir  from a cycle expansion of the Gutzwiller-Voros zeta function. \[Since the Gutzwiller-Voros zeta function is directly related to the Gutzwiller trace formula without further approximations, the semiclassical resonances resulting from both expressions will be the same.\] The weighted difference spectrum is shown in Figure 3. Note that due to the limited radius of convergence of the cycle expansion only resonances with $`\mathrm{Im}k0.8`$ were available and could be included in the signal.

In Figure 4, the crosses designate the results for the first-order amplitudes $`𝒜_{\mathrm{po}}^{(1)}`$ obtained from the harmonic inversion of the difference spectrum shown in Fig. 3, which was analyzed in the region $`\mathrm{Re}k[100,250]`$. For comparison, we also determined the first-order amplitudes $`𝒜_{\mathrm{po}}^{(1)}`$ for each orbit following the method of Vattay and Rosenqvist described above (see Eq. (13)). These results are represented in Figure 4 by solid lines.

For almost all orbits, the harmonic inversion results for $`𝒜_{\mathrm{po}}^{(1)}`$ are seen to be in perfect agreement with the amplitudes calculated by the method of Refs. Vat94 ; Vat96 ; Ros94 . There is, however, one exception, namely the distinct discrepancy for the orbit with symbolic code ‘1’ (scaled action $`s4.267949`$). The deviation is systematic and appears in the same way if the parameters of the harmonic inversion procedure (such as signal length etc.) are varied. This point still needs further clarification. A possible explanation for the discrepancy may lie in the fact that the set of resonances from which the signal was constructed was not complete, since only resonances near the real axis could be included. However, this does not explain why only one orbit is strongly affected. On the other hand, the error might also be due to the theory of Refs. Vat94 ; Vat96 ; Ros94 , or its application to the three-disk system. In fact, the ‘1’ orbit is the orbit with the largest contributions from terms of higher order in $`l`$ to the sum in (13). The contributions from the different $`l`$ terms and the converged sum over $`l`$ of the five shortest orbits are given in Table 1. For comparison, the last column of Table 1 shows the corresponding values following from the amplitudes of the harmonic inversion results. The ‘1’ orbit exhibits the largest deviation between the $`l=0`$ contribution and the converged sum over $`l`$, followed by the ‘0’ orbit. For orbits with a symbol length of 2 or longer, the contributions of higher $`l`$ terms to the amplitude $`𝒜_{\mathrm{po}}^{(1)}`$ are already so small (due to the large absolute value of the stability eigenvalue $`\lambda `$) that it is impossible to decide whether or not there is a discrepancy between these terms and the harmonic inversion results. (Note that this is also true for the period doubling of the orbits ‘0’ and ‘1’ in Table 1.) However, the harmonic inversion results for the ‘0’ orbit, which also shows a relatively large contribution from the $`l=1`$ term, are in agreement with the theory. Again, it cannot be explained why only the ‘1’ orbit is affected (although in this case the reasons might lie in the special geometrical properties of the ‘0’ orbit).

Concluding our discussion of Figure 4 and Table 1, we notice that the harmonic inversion results indeed confirm the validity of the $`l=0`$ approximation to the formula (6) for orbits with large stability eigenvalues. On the other hand, the results demonstrate that the theory of higher-order $`\mathrm{}`$ corrections to the Gutzwiller formula still contains unanswered questions, and further investigations are necessary.

## 5 Corrections to the semiclassical eigenvalues

We now turn to the problem of obtaining corrections to the semiclassical eigenvalues of chaotic systems from the $`\mathrm{}`$ expansion (11) of the periodic orbit sum. A general procedure for including higher-order $`\mathrm{}`$ corrections in the harmonic inversion scheme was developed in Refs. Mai99a ; Mai98c ; Wei00 , where it was applied to the circle billiard as an example of an integrable system. In the following, we briefly recapitulate the main ideas of the procedure and then apply the technique to the open three-disk system.

### 5.1 Theory

For periodic orbit quantization, usually only the zeroth-order contributions $`𝒜_{\mathrm{po}}^{(0)}`$ to the expanded response function (11), corresponding to the Gutzwiller formula (or, for integrable systems, the Berry-Tabor formula), are considered. In the harmonic inversion scheme for semiclassical quantization Mai97b ; Mai98 ; Mai99a , semiclassical approximations to the eigenvalues or resonances are determined by adjusting the Fourier transform of the principal periodic orbit sum

$$C_0(s)=\underset{\mathrm{po}}{}𝒜_{\mathrm{po}}^{(0)}\delta (ss_{\mathrm{po}})$$
(18)
to the functional form of the corresponding exact quantum expression (i.e., the Fourier transform of the exact response function (14))

$$C_{\mathrm{qm}}(s)=\mathrm{i}\underset{j}{}m_j\mathrm{e}^{\mathrm{i}k_js},$$
(19)
with $`k_j`$ the eigenvalues or resonances and $`m_j`$ their multiplicities.

Since for $`n1`$ the asymptotic expansion (11) of the semiclassical response function suffers from the singularities at $`k=0`$, higher-order $`\mathrm{}`$ terms cannot directly be included in the harmonic inversion scheme. Instead, the correction terms to the semiclassical eigenvalues can be calculated separately, order by order. We assume that the $`(n1)^{\mathrm{st}}`$-order $`\mathrm{}`$ approximations $`k_{j,n1}`$ to the exact eigenvalues have already been obtained and the $`n^{\mathrm{th}}`$-order approximations $`k_{j,n}`$ are to be calculated. In terms of these approximations to the eigenvalues, the difference between the two subsequent approximations to the quantum mechanical response function reads

$`g_n(k)`$ $`=`$ $`{\displaystyle \underset{j}{}}\left({\displaystyle \frac{m_j}{kk_{j,n}+\mathrm{i0}}}{\displaystyle \frac{m_j}{kk_{j,n1}+\mathrm{i0}}}\right)`$ (20)
$``$ $`{\displaystyle \underset{j}{}}{\displaystyle \frac{m_j\mathrm{\Delta }k_{j,n}}{(k\overline{k}_{j,n}+\mathrm{i0})^2}},`$
with $`\overline{k}_{j,n}=\frac{1}{2}(k_{j,n}+k_{j,n1})`$ and $`\mathrm{\Delta }k_{j,n}=k_{j,n}k_{j,n1}`$. Integration of (20) and multiplication by $`k^n`$ yields

$$𝒢_n(k)=k^ng_n(k)dk=\underset{j}{}\frac{m_jk^n\mathrm{\Delta }k_{j,n}}{k\overline{k}_{j,n}+\mathrm{i0}}.$$
(21)
The periodic orbit approximation to (21) is obtained from the term $`g_n(k)`$ in the periodic orbit sum (11) by integration and multiplication by $`k^n`$, yielding

$$𝒢_n(k)=\mathrm{i}\underset{\mathrm{po}}{}\frac{1}{s_{\mathrm{po}}}𝒜_{\mathrm{po}}^{(n)}\mathrm{e}^{\mathrm{i}ks_{\mathrm{po}}}+𝒪\left(\frac{1}{k}\right).$$
(22)
One can now Fourier transform both (21) and (22), and obtains ($`n1`$)

$`C_n(s)`$ $``$ $`{\displaystyle \frac{1}{2\pi }}{\displaystyle _{\mathrm{}}^+\mathrm{}}𝒢_n(k)\mathrm{e}^{\mathrm{i}ks}dk`$ (23)
$`=`$ $`\mathrm{i}{\displaystyle \underset{j}{}}m_j(\overline{k}_j)^n\mathrm{\Delta }k_{j,n}\mathrm{e}^{\mathrm{i}\overline{k}_js}`$
$`\stackrel{\mathrm{h}.\mathrm{i}.}{=}`$ $`\mathrm{i}{\displaystyle \underset{\mathrm{po}}{}}{\displaystyle \frac{1}{s_{\mathrm{po}}}}𝒜_{\mathrm{po}}^{(n)}\delta (ss_{\mathrm{po}}).`$ (24)
Equations (23) and (24) imply that the $`\mathrm{}`$ expansion of the semiclassical eigenvalues can be obtained, order by order, by adjusting the periodic orbit signal (24) to the functional form of (23) by harmonic inversion (h.i.). The frequencies $`\overline{k}_j`$ of the periodic orbit signal (24) are the semiclassical eigenvalues or resonances, averaged over different orders of $`\mathrm{}`$. Note that the accuracy of these values does not necessarily increase with increasing order $`n`$. The corrections $`\mathrm{\Delta }k_{j,n}`$ to the eigenvalues are not obtained from the frequencies, but from the amplitudes, $`m_j(\overline{k}_j)^n\mathrm{\Delta }k_{j,n}`$, of the periodic orbit signal.

### 5.2 Application to the three-disk scattering system

We have applied the above procedure to the open three-disk scatterer with disk separations $`d=6`$ and $`d=2.5`$. For both cases, we first calculated the zeroth-order $`\mathrm{}`$ approximations to the resonances and then determined the first-order $`\mathrm{}`$ corrections to the semiclassical results following the scheme outlined above.

For disk separation $`d=6`$, we used the periodic orbits up to length $`s_{\mathrm{max}}=56`$ to calculate the resonances in the region $`0\mathrm{Re}k250`$. In the first-order amplitudes (13), only the leading-order term $`l=0`$ of the sum was included. (The terms with $`l1`$ contribute significantly only to the ‘1’ orbit and thus basically do not effect the semiclassical resonances.) The results for the first-order corrections $`\mathrm{\Delta }k_1`$ were added to the semiclassical results to obtain the first-order approximations $`k_1`$ to the resonances. Table 2 shows part of the results in the regions $`\mathrm{Re}k[0,12]`$ and $`\mathrm{Re}k[150,155]`$ with $`\mathrm{Im}k0.5`$. For comparison, the exact quantum resonances $`k_{\mathrm{ex}}`$ from Refs. Wir99 ; Cvi97 ; Wir  are also given. We note that the semiclassical resonances $`k_0`$ obtained by harmonic inversion agree with the semiclassical cycle expansion values from Refs. Wir99 ; Cvi97 ; Wir  to all digits given. Figure 5 compares the semiclassical errors of the zeroth-order (crosses) and first-order (squares) approximations as a function of the real part of the resonances. The deviations of the real and imaginary parts from the exact quantum values are shown separately. Only resonances with an imaginary part $`\mathrm{Im}k0.5`$ were included in the plot.

The results presented in Figure 5 show that by including the first-order corrections a significant improvement in the accuracy of the real parts of the resonances is achieved. This is evident from Figure 5 in spite of the fact that a one-to-one correspondence between the zeroth and first-order values plotted is difficult to establish with the naked eye. For most resonances, the real part of the first-order approximation lies between two and five orders of magnitude closer to the exact quantum values than the zeroth-order approximation. Only for the “most quantum” resonances, with very low real parts, the improvement is rather small. The reason for this lies in the nature of the semiclassical approximation as an approximation itself: in order to improve these values, second or higher-order terms of the $`\mathrm{}`$ expansion must be considered.

The accuracy of the imaginary parts of the semiclassical resonances is less significantly increased by the first-order corrections than that of the real parts. For some resonances, the zeroth-order approximation lies even closer to the exact quantum values than the first-order approximation. This was also observed in Refs. Alo93 ; Ros94 , where the first-order $`\mathrm{}`$ corrections to the resonances was calculated using the cycle expansion technique. As discussed in Alo93 ; Ros94 , the first-order corrections to the periodic orbit sum mainly improve the real part of the resonances, while the imaginary part can be expected to be improved by second-order $`\mathrm{}`$ corrections.

A similar behaviour can be found for disk separation $`d=2.5`$. Here, we calculated the semiclassical resonances and their first-order $`\mathrm{}`$ corrections in the range $`0\mathrm{Re}k90`$ and $`0.82\mathrm{Im}k0`$ from the periodic orbits up to length $`s_{\mathrm{max}}=12`$. In the first-order amplitudes (13), again only the $`l=0`$ term was included. Table 3 compares part of the results for the first-order approximations to the resonances with the zeroth-order approximations and the exact quantum values. The zeroth order resonances $`k_0`$ obtained by harmonic inversion agree with the cycle expansion values calculated by Wirzba Wir99 ; Cvi97 ; Wir  (not shown) to at least four significant digits.

Again, we determined the semiclassical error of the first-order approximations to the resonances in comparison to that of the zeroth-order approximation. The results are presented in Figure 6.

The general behaviour of the values is similar to that in the case $`d=6`$ discussed above, although the improvement of the accuracy achieved by the first-order corrections is not as spectacular as for $`d=6`$. The reason for this may partly lie in the error induced by the harmonic inversion method, which for $`d=2.5`$ is larger already in the zeroth-order approximation than for $`d=6`$. The results could in principle be improved by extending the signal to longer orbits. On the other hand, in the part of the spectrum considered, second and higher-order $`\mathrm{}`$ corrections may be more important than in the case $`d=6`$. However, it is evident from Figure 6 that, apart from the resonances with very small real parts, the semiclassical error of the real parts of the resonances could still be reduced, even for the small disk separation of $`d=2.5`$, by the first-order $`\mathrm{}`$ corrections by one or two orders of magnitude.

## 6 Conclusions

In this paper we have demonstrated the power of the harmonic inversion technique in explicitly determining higher-order $`\mathrm{}`$ corrections to the Gutzwiller trace formula and to the semiclassical eigenvalues of a completely chaotic system, namely the three-disk scattering system. The method has been used in two directions: (1) for the harmonic analysis of the exact quantum spectrum, (2) for the direct calculation of higher-order corrections to the semiclassical eigenvalues from classical periodic orbit data.

The harmonic analysis of the exact quantum spectrum of the three-disk system with the “standard” literature disk separation of $`d=6`$ first yielded the zeroth-order semiclassical amplitudes of the periodic orbit sum (i.e., the amplitudes entering the Gutzwiller formula), which were found to be in perfect agreement with the Gutzwiller amplitudes calculated directly from classical periodic orbit data. Next, from the exact quantum resonances and their zeroth order approximations, we were able to compute the first-order amplitudes applying harmonic inversion to Eq. (17). We could verify the correctness of the values obtained in this way by comparing with the results of an alternative theoretical approach Vat94 ; Vat96 ; Ros94  for calculating first-order corrections to the Gutzwiller formula in chaotic billiards, which we implemented for the three-disk system. The results turned out to be in very good agreement (on the order of 1.5 per cent, or better), with one notable exception, namely the ‘1’ orbit, for which a distinct discrepancy (on the order of 20 per cent) persisted. We have discussed possible origins of the discrepancy although an ultimate reason could not be identified. Therefore, in spite of the very good agreement of the results in all other cases, we have to conclude that the theory of $`\mathrm{}`$ corrections to the Gutzwiller formula still contains unanswered questions. \[We note that in fact for integrable systems there does not yet exist a general theory for higher-order $`\mathrm{}`$ corrections to the Berry-Tabor formula at all.\]

In the direct calculation of higher-order corrections to semiclassical eigenvalues from classical periodic orbit data, we first evaluated the first-order correction amplitudes to the Gutzwiller formula as given by the theory of Vattay and Rosenqvist, and then, by harmonic inversion of Eq. (24), determined the first-order $`\mathrm{}`$ corrections to the semiclassical (complex) eigenvalues of resonances of the three-disk scattering system with disk separations $`d=6`$ and $`d=2.5`$. For both distances, the semiclassical error, as compared to the exact quantum values, of the zeroth-order results (obtained from the Gutzwiller formula by harmonic inversion) for the real parts of the resonances could be significantly reduced by including the first-order $`\mathrm{}`$ corrections: the accuracy was increased by two to five orders of magnitude for $`d=6`$, and still by one to two orders of magnitude for $`d=2.5`$. Only for the “most quantum” resonances, with very small real parts, the increase in accuracy was found to be rather modest. It turned out that the accuracy of the imaginary parts of the semiclassical eigenvalues of resonances was less significantly increased by the first-order corrections; here, second-order corrections would have to be considered.

Although in our calculations we have used literature values for the semiclassical resonances and for the zeroth and first-order amplitudes, we could have performed, in principle, all calculations knowing the exact quantum resonances only. The analysis of the exact quantum spectrum yields the semiclassical amplitudes, which in turn can be used to calculate the semiclassical resonances. Then, by an analysis of the difference spectrum between semiclassical and exact resonances, the first order amplitudes can be determined, which again can be used to obtain the first-order corrections to the resonances. Although we have concentrated in our examples on obtaining first-order $`\mathrm{}`$ corrections, it is evident from our discussion that the next-order corrections could be obtained iteratively in an analogous manner by repeated application of Eqs. (17), and (23), (24), respectively.

In summary, we have demonstrated that harmonic inversion – as a means for circumventing the convergence problems of semiclassical trace formulae – is indeed a very efficient and universal tool, not only for semiclassical quantization, but also for the explicit calculation of higher-order $`\mathrm{}`$ corrections to the semiclassical eigenvalues or resonances even in chaotic systems. Moreover, harmonic inversion does not rely on specific assumptions for the systems under consideration, and therefore an application of the methods presented in this paper to other chaotic systems will be promising and worthwhile.

###### Acknowledgements.

We are grateful to G. Vattay for sending us the code for the calculation of the first-order corrections to the Gutzwiller formula for chaotic billiards. We thank A. Wirzba for communicating to us quantum mechanical and semiclassical resonances of the three-disk system. This work was supported by Deutsche Forschungsgemeinschaft and Deutscher Akademischer Austauschdienst.


format: text

text:
# The Frobenius map, rank 2 vector bundles and Kummer’s quartic surface in characteristic 2 and 3

## 1 Introduction

Let $`X`$ be a smooth projective curve of genus $`2`$ defined over an algebraically closed field $`k`$ of characteristic $`p>0`$. The moduli space $`\mathrm{M}_X`$ of semi-stable rank $`2`$ vector bundles with fixed trivial determinant is isomorphic to the linear system $`|2\mathrm{\Theta }|^3`$ over $`\mathrm{Pic}^1(X)`$ and the $`k`$-linear relative Frobenius map $`F:XX_1`$ induces by pull-back a rational map (the Verschiebung)

$$\begin{array}{ccc}\mathrm{M}_{X_1}& \stackrel{V}{}& \mathrm{M}_X\\ D& & D& & \\ |2\mathrm{\Theta }_1|& \stackrel{\stackrel{~}{V}}{}& |2\mathrm{\Theta }|\end{array}$$
(1.1)
The vertical maps $`D`$ are isomorphisms and the Verschiebung $`V:EF^{}E`$ coincides via $`D`$ with a rational map $`\stackrel{~}{V}`$ given by polynomial equations of degree $`p`$ (Proposition 7.2). The Kummer surfaces $`\mathrm{Kum}_X`$ and $`\mathrm{Kum}_{X_1}`$ are canonically contained in the linear systems $`|2\mathrm{\Theta }|`$ and $`|2\mathrm{\Theta }_1|`$ and coincide with the semi-stable boundary of the moduli spaces $`\mathrm{M}_X`$ and $`\mathrm{M}_{X_1}`$. Moreover $`\stackrel{~}{V}`$ maps $`\mathrm{Kum}_{X_1}`$ onto $`\mathrm{Kum}_X`$.

Our interest in the diagram (1.1) comes from questions related to the action of the Frobenius map on vector bundles like e.g. surjectivity of $`V`$, density of Frobenius-stable bundles, loci of Frobenius-destabilized bundles (see \[LP\]). These questions are largely open when the rank of the bundles, the genus of the curve or the characteristic of the field are arbitrary. In \[LP\] we made use of the exceptional isomorphism $`D:\mathrm{M}_X|2\mathrm{\Theta }|`$ in the genus $`2`$, rank $`2`$ case and determined the equations of $`\stackrel{~}{V}`$ when $`X`$ is an ordinary curve and $`p=2`$, which allowed us to answer the above mentioned questions. In this paper we obtain the equations of $`\stackrel{~}{V}`$ in two more cases:

* $`p=2`$ and $`X`$ non-ordinary with Hasse-Witt invariant equal to 1,
* $`p=3`$ and any $`X`$.

In case (1) we consider a family $`𝒳`$ of genus $`2`$ curves parametrized by a discrete valuation ring with ordinary generic fibre $`𝒳_\eta `$ and special fibre isomorphic to $`X`$. We obtain the equations of $`\stackrel{~}{V}`$ for $`X`$ (Theorem 5.1) by specializing the quadrics $`P_{ij}`$ defining the Verschiebung $`V_\eta :\mathrm{M}_{𝒳_{1\eta }}\mathrm{M}_{𝒳_\eta }`$ associated to $`𝒳_\eta `$ (section 5). In order to determine the limit of the $`P_{ij}`$’s we use the explicit formulae (Proposition 3.1) of the coefficients of the $`P_{ij}`$’s, which coincide with the coefficients of Kummer’s quartic surface $`\mathrm{Kum}_{𝒳_\eta }`$, in terms of the coefficients of an affine equation of the ordinary curve $`𝒳_\eta `$. As in the ordinary case we easily deduce from the equations of $`\stackrel{~}{V}`$ a full description of the Verschiebung $`V`$ (Proposition 5.4).

In case (2) we show that the cubic equations of $`\stackrel{~}{V}`$ are given by the polar equations of a Kummer surface $`S|2\mathrm{\Theta }_1|`$ (Theorem 6.1). Moreover $`S`$ is isomorphic to $`\mathrm{Kum}_X`$ and the $`16`$ nodes of $`S`$ correspond to the $`16`$ base points of $`\stackrel{~}{V}`$. We deduce that $`V`$ is surjective and of degree $`11`$ (Corollary 6.4).

In the appendix we show that over any smooth curve $`X`$ of genus $`g2`$ defined over an algebraically closed field of characteristic $`p>0`$ and for any integer $`r2`$, there exist Frobenius-destabilized bundles of rank $`r`$, i.e., semi-stable bundles $`E`$ such that $`F^{}E`$ is not semi-stable (Theorem 7.4).

We thank M. Raynaud for helpful discussions.

## 2 Preliminaries on genus $`2`$ curves in characteristic $`2`$

We consider a smooth ordinary curve $`X`$ of genus $`2`$ defined over a field $`K`$ of characteristic $`2`$. We denote by $`\overline{K}`$ its algebraic closure. Let $`k(X)`$ be the function field of $`X`$ and $`\omega _X`$ the canonical bundle over $`X`$.

### 2.1 Weierstrass points

After taking a finite extension of $`K`$ and applying an automorphism of $`_K^1`$ we can assume that the three Weierstrass points of $`X`$ are $`0,1`$ and $`\mathrm{}`$. We consider the birational Abel-Jacobi map

$$AJ:S^2XJX,P_1+P_2𝒪_X(P_1+P_2)\omega _X^1.$$
(2.1)
We observe that the three nonzero elements $`[0],[1],[\mathrm{}]`$ of $`JX[2]`$ are $`K`$-rational and are given by

$$[0]:=AJ(1+\mathrm{}),[1]:=AJ(0+\mathrm{}),[\mathrm{}]:=AJ(0+1).$$
(2.2)
For later use we mention that the sheaf of locally exact differential forms (see \[R\] section 4) equals

$$B=𝒪_X(0+1+\mathrm{})\omega _X^1.$$

We recall that $`B`$ is a theta-characteristic of $`X`$, i.e., $`B^2\omega _X`$.

### 2.2 Level $`2`$ structure

A level $`2`$ structure is an isomorphism $`\psi :JX[2]\stackrel{}{}𝔽_2^2`$. Note that two level $`2`$ structures differ by an automorphism of $`𝔽_2^2`$, i.e., an element of $`\mathrm{GL}(2,𝔽_2)=𝔖_3`$, where $`𝔖_3`$ is the symmetric group. It is well-known that a level $`2`$ structure $`\psi `$ is equivalent to an ordering of the Weierstrass points of $`X`$. We refer e.g. to \[DO\] page 141 for the characteristic zero case, which can easily be adapted to the characteristic two case. Because of the choices involved in the degeneration of $`X`$ to a non-ordinary curve (section 5.1), we consider the ordering $`1,\mathrm{},0`$ of the Weierstrass points. With the notation of (2.2) the corresponding level $`2`$ structure $`\psi `$ is given by

$$\psi ([0])=(1,0),\psi ([1])=(0,1),\psi ([\mathrm{}])=(1,1).$$
(2.3)
A level $`2`$ structure $`\psi `$ allows us to construct (\[LP\] section 2) the theta basis $`\{X_g\}_{g𝔽_2^2}`$ of the space $`H^0(JX,2\mathrm{\Theta })`$. We denote the four sections by $`X_B,X_0,X_1,X_{\mathrm{}}`$ and introduce the rational functions $`Z_{}k(JX)`$ defined by

$$Z_0=\frac{X_0}{X_B},Z_1=\frac{X_1}{X_B},Z_{\mathrm{}}=\frac{X_{\mathrm{}}}{X_B}.$$
(2.4)
We recall that $`X_B`$ is the theta function with associated zero divisor $`2\mathrm{\Theta }_B`$, where

$$\mathrm{\Theta }_B:=\{LJX|h^0(X,LB)1\}.$$

### 2.3 Birational models

Let $`X`$ be an ordinary smooth curve of genus $`2`$ defined over $`\overline{K}`$ and $`\psi `$ a level $`2`$ sturcture. It follows from \[L\] page 28 and \[B\] Proposition 1.4 that the pair $`(X,\psi )`$ is uniquely represented by an affine equation of the form

$$y^2+x(x+1)y=x(x+1)(ax^3+(a+b)x^2+cx+c),$$
(2.5)
with $`a,b,c\overline{K}`$. Moreover if $`X`$ is defined over $`K`$, the coefficients $`a,b,c`$ lie in a finite extension of $`K`$. The next lemma is an immediate consequence of \[B\] Proposition 1.5.

###### 2.1 Lemma.

The curve $`X`$ defined by the equation (2.5) is smooth if and only if $`abc0`$.

Let $`\stackrel{~}{}_3`$ denote the moduli space parametrizing pairs $`(X,\psi )`$ of smooth ordinary genus $`2`$ curves $`X`$ defined over $`\overline{K}`$ equipped with a level $`2`$ structure. It follows from the previous remark that $`\stackrel{~}{}_3`$ is an affine variety,

$$\stackrel{~}{}_3=\mathrm{Spec}\overline{K}[a,b,c,\frac{1}{abc}].$$

Fixing the curve $`X`$ the symmetric group $`𝔖_3`$ acts naturally on the level $`2`$ structures $`\psi `$. It can be shown that this $`𝔖_3`$-action on $`\stackrel{~}{}_3`$ coincides with the permutation action of $`𝔖_3`$ on the coefficients $`a,b,c`$.

### 2.4 Normal form

We introduce the rational function $`Yk(X)`$ defined by $`Y=\frac{y}{x(x+1)}`$. Then (2.5) becomes

$$Y^2+Y=R(x),\text{with}R(x)=\frac{ax^3+(a+b)x^2+cx+c}{x(x+1)}.$$
(2.6)
We also observe that, given a polynomial $`S(x)`$, the involution $`i_S:𝔸_k^2𝔸_k^2`$, $`(x,y)(x,y+x(x+1)S(x))`$ transforms the equation (2.6) of $`X`$ into $`Y^2+Y=R+S^2+S`$.

### 2.5 Kummer’s quartic equation

For a pair $`(X,\psi )`$ it has been shown in \[LP\] Proposition 4.1 that there exist constants $`\lambda _0,\lambda _1,\lambda _{\mathrm{}}\overline{K}`$ such that the following equality holds in $`k(JX)`$,

$$\lambda _0^2(Z_0^2+Z_1^2Z_{\mathrm{}}^2)+\lambda _1^2(Z_1^2+Z_0^2Z_{\mathrm{}}^2)+\lambda _{\mathrm{}}^2(Z_{\mathrm{}}^2+Z_0^2Z_1^2)+\lambda _0\lambda _1\lambda _{\mathrm{}}Z_0Z_1Z_{\mathrm{}}=0.$$
(2.7)
The constants $`\lambda _0,\lambda _1,\lambda _{\mathrm{}}`$ are related via $`\psi `$ (2.3) to the $`\{\lambda _g\}_{g𝔽_2^2}`$ used in \[LP\] Proposition 4.1 as follows: $`\lambda _0=\frac{\lambda _{10}}{\lambda _{00}}`$, $`\lambda _1=\frac{\lambda _{01}}{\lambda _{00}}`$, $`\lambda _{\mathrm{}}=\frac{\lambda _{11}}{\lambda _{00}}`$.

## 3 The coefficients $`\lambda _{}`$ of Kummer’s quartic surface $`\mathrm{Kum}_X`$ for ordinary $`X`$

###### 3.1 Proposition.

Given a curve $`X`$ with a level $`2`$ structure $`\psi `$ represented by an affine equation (2.5) with $`a,b,cK`$. Then the coefficients of the equation (2.7) of its Kummer surface $`\mathrm{Kum}_X`$ are

$$\lambda _0^2=\frac{1}{ab},\lambda _1^2=\frac{1}{ac},\lambda _{\mathrm{}}^2=\frac{1}{bc}.$$

Let $`\{x_g\}`$ be the dual basis of the theta basis $`\{X_g\}_{g𝔽_2^2}`$ of $`|2\mathrm{\Theta }|=^3`$. Then the homogeneous equation of $`\mathrm{Kum}_X`$ is

$$c(x_{00}^2x_{10}^2+x_{01}^2x_{11}^2)+b(x_{00}^2x_{01}^2+x_{10}^2x_{11}^2)+a(x_{00}^2x_{11}^2+x_{01}^2x_{10}^2)+x_{00}x_{01}x_{10}x_{11}=0.$$

The idea of the proof is to consider the pull-back of the rational functions $`Z_{}`$ (2.4) by the Abel-Jacobi map (2.1) to the symmetric product $`S^2X`$ and to do the computations in the function field $`k(S^2X)k(X\times X)`$. Since $`X`$ is given by the equation (2.5), we have natural coordinates $`x_1,y_1`$ and $`x_2,y_2`$ on $`X\times X`$. For notational convenience, we also use $`Y_i=\frac{y_i}{x_i(x_i+1)}`$ for $`i=1,2`$.

We use two lemmas.

###### 3.2 Lemma.

Suppose that there exist polynomials $`A,BK[x_1,x_2]`$, which satisfy

$$(Y_1+Y_2)A(x_1,x_2)=B(x_1,x_2).$$
(3.1)
Then $`A=B=0`$.

###### Proof.

Squaring the relation (3.1) and using (2.6) leads to the equation

$$(Y_1+Y_2)A^2+(R(x_1)+R(x_2))A^2+B^2=0.$$

Applying again (3.2), the first term transforms into $`AB`$. Clearing denominators, we arrive at a polynomial equation, which only holds if $`A=B=0`$; e.g. take the total degree of $`A`$ and $`B`$. ∎

###### 3.3 Lemma.

The pull-back by the Abel-Jacobi map $`AJ`$ of the rational function $`Z_{\mathrm{}}k(JX)`$ equals

$$AJ^{}(Z_{\mathrm{}})=\alpha _{\mathrm{}}\frac{P(x_1,x_2)}{(Y_1+Y_2)^2},\text{with}P(x_1,x_2)=\frac{(x_1+x_2)^2}{x_1x_2(x_1+1)(x_2+1)}$$

Similary we have

$$AJ^{}(Z_0)=\alpha _0\frac{P(x_1,x_2)}{(Y_1+Y_2)^2}x_1x_2,AJ^{}(Z_1)=\alpha _1\frac{P(x_1,x_2)}{(Y_1+Y_2)^2}(x_1+1)(x_2+1),$$

for some nonzero constants $`\alpha _0,\alpha _1,\alpha _{\mathrm{}}K`$.

###### Proof.

The first equality follows immediately from Theorem 2 \[AG\] and the other two from Proposition 5 \[AG\]. ∎

###### Proof of Proposition 3.1.

We write $`Q=\frac{P(x_1,x_2)}{(Y_1+Y_2)^2}`$. Using Lemma 3.3 the pull-back to $`S^2X`$ of the equation (2.7) equals

$`\lambda _0^2\left[\alpha _0^2x_1^2x_2^2Q^2+\alpha _1^2\alpha _{\mathrm{}}^2(x_1+1)^2(x_2+1)^2Q^4\right]+\lambda _1^2\left[\alpha _1^2(x_1+1)^2(x_2+1)^2Q^2+\alpha _0^2\alpha _{\mathrm{}}^2x_1^2x_2^2Q^4\right]+`$
$`\lambda _{\mathrm{}}^2\left[\alpha _{\mathrm{}}^2Q^2+\alpha _0^2\alpha _1^2x_1^2x_2^2(x_1+1)^2(x_2+1)^2Q^4\right]+\lambda _0\lambda _1\lambda _{\mathrm{}}\left[\alpha _0\alpha _1\alpha _{\mathrm{}}x_1x_2(x_1+1)(x_2+1)Q^3\right]=0`$
We divide by $`Q^2`$ and multiply by $`(Y_1+Y_2)^4`$,

$`(Y_1+Y_2)^4\left[\lambda _0^2\alpha _0^2x_1^2x_2^2+\lambda _1^2\alpha _1^2(x_1+1)^2(x_2+1)^2+\lambda _{\mathrm{}}^2\alpha _{\mathrm{}}^2\right]+`$
$`(Y_1+Y_2)^2\left[\lambda _0\lambda _1\lambda _{\mathrm{}}\alpha _0\alpha _1\alpha _{\mathrm{}}Px_1x_2(x_1+1)(x_2+1)\right]+S=0.`$ (3.2)
where $`S`$ is the sum of the remaining terms (not containing $`Y_1,Y_2`$). After taking the square root (note that the entire expression is a square of a polynomial in the $`x_i`$’s and $`Y_i`$’s), applying (2.6) and Lemma 3.2, we obtain that the coefficients of $`(Y_1+Y_2)^2`$ and $`(Y_1+Y_2)^4`$ are the same, i.e.,

$$\lambda _0^2\alpha _0^2x_1^2x_2^2+\lambda _1^2\alpha _1^2(x_1+1)^2(x_2+1)^2+\lambda _{\mathrm{}}^2\alpha _{\mathrm{}}^2=\lambda _0\lambda _1\lambda _{\mathrm{}}\alpha _0\alpha _1\alpha _{\mathrm{}}Px_1x_2(x_1+1)(x_2+1).$$

An easy computation shows that this equality only holds if

$$\lambda _0\alpha _0=\lambda _1\alpha _1=\lambda _{\mathrm{}}\alpha _{\mathrm{}}=1.$$

Now we replace the $`\alpha `$’s and the sum $`S`$ by their expressions in the square root of the equation (3)

$`(Y_1+Y_2)^2(x_1+x_2)+(Y_1+Y_2)(x_1+x_2)+`$
$`P\left[{\displaystyle \frac{\lambda _0}{\lambda _1\lambda _{\mathrm{}}}}(x_1+1)(x_2+1)+{\displaystyle \frac{\lambda _1}{\lambda _0\lambda _{\mathrm{}}}}x_1x_2+{\displaystyle \frac{\lambda _{\mathrm{}}}{\lambda _0\lambda _1}}x_1x_2(x_1+1)(x_2+1)\right]=0.`$
We introduce the constants $`\mu _0,\mu _1,\mu _{\mathrm{}}K`$ defined by $`\mu _{\mathrm{}}=\frac{\lambda _{\mathrm{}}}{\lambda _0\lambda _1},\mu _0=\frac{\lambda _0}{\lambda _1\lambda _{\mathrm{}}},\mu _1=\frac{\lambda _1}{\lambda _0\lambda _{\mathrm{}}}`$. The previous equality becomes after replacing $`P`$ by its expression and dividing by $`(x_1+x_2)`$

$$\left[Y_1^2+Y_1+\frac{\mu _0}{x_1}+\frac{\mu _1}{x_1+1}+\mu _{\mathrm{}}x_1\right]+\left[Y_2^2+Y_2+\frac{\mu _0}{x_2}+\frac{\mu _1}{x_2+1}+\mu _{\mathrm{}}x_2\right]=0.$$

This equation holds in $`k(X\times X)`$ and since the variables $`(x_1,Y_1)`$ and $`(x_2,Y_2)`$ are separated, each of the two terms equals zero. So we can drop the indices and we obtain an equation

$$Y^2+Y=\mu _{\mathrm{}}x+\frac{\mu _0}{x}+\frac{\mu _1}{x+1}=\frac{\mu _{\mathrm{}}x^3+\mu _{\mathrm{}}x^2+(\mu _0+\mu _1)x+\mu _0}{x(x+1)},$$
(3.3)
which has to be equivalent (after applying an automorphism of $`𝔸_{\overline{K}}^2`$) to the normal form (2.6) of the equation of $`X`$. The automorphism is given by $`i_S`$ (see section 2.4) with $`S(x)=s\overline{K}`$ satisfying $`s^2+s=\mu _1`$. Hence (3.3) is equivalent to $`Y^2+Y=R(x)`$ with $`R(x)=\frac{\mu _{\mathrm{}}x^3+(\mu _{\mathrm{}}+\mu _1)x^2+\mu _0x+\mu _0}{x(x+1)}`$. Hence by uniqueness of the normal form, we obtain $`a=\mu _{\mathrm{}},b=\mu _1,c=\mu _0`$ and therefore also the relations claimed in the proposition. ∎

## 4 Deformation of genus $`2`$ curves

Let $`X/k`$ be a smooth curve with Hasse-Witt invariant equal to $`1`$. We introduce a family $`𝒳`$ over $`R=k[[t]]`$ such that the special fibre $`𝒳_0`$ is isomorphic to $`X`$ and the generic fibre $`𝒳_\eta `$ is an ordinary genus $`2`$ curve. Here $`\eta `$ (resp. $`0`$) is the generic (resp. closed) point of $`\mathrm{Spec}(R)`$. In section 5.1 we give an example of a family $`𝒳`$ with given special fibre $`X`$. Let $`𝒥𝒳`$ be its associated Jacobian scheme over $`\mathrm{Spec}(R)`$.

### 4.1 $`2`$-divisible groups

Let $`𝒥𝒳(2)`$ be the $`2`$-divisible group of $`𝒥𝒳`$, which is finite and flat over $`\mathrm{Spec}(R)`$. We consider the canonical exact sequence

$$0𝒥𝒳(2)^0𝒥𝒳(2)𝒥𝒳(2)^{et}0,$$
(4.1)
where $`𝒥𝒳(2)^0`$ (resp. $`𝒥𝒳(2)^{et})`$ is a connected (resp. étale) $`2`$-divisible group. Taking again the connected component of the Cartier dual of $`𝒥𝒳(2)^0`$ we obtain a filtration

$$𝒥𝒳(2)^{00}𝒥𝒳(2)^0𝒥𝒳(2),$$

with quotients given by the $`2`$-divisible groups

$$𝒥𝒳(2)/𝒥𝒳(2)^0=𝒥𝒳(2)^{et}_2/_2,𝒥𝒳(2)^0/𝒥𝒳(2)^{00}𝔾_m(2).$$

The $`2`$-divisible group $`𝒥𝒳(2)^{00}`$ is self-dual, of dimension $`1`$ and of height $`2`$. Because of the uniqueness of $`2`$-divisible groups over $`k`$ with these properties (see e.g. \[D\] Examples page 93), the special fibre $`𝒥𝒳(2)_0^{00}`$ $`(=𝒥𝒳(2)^{00}_Rk)`$ is isomorphic to the $`2`$-divisible group associated to the supersingular elliptic curve $`E^{ss}/k`$. We recall that there exists a unique (up to isomorphism) supersingular curve $`E^{ss}`$, which is defined by $`j=0`$. Therefore by a theorem of Serre-Tate \[K\], there exists an elliptic curve $`_𝒳`$ over $`\mathrm{Spec}(R)`$ such that $`(_𝒳)_0E^{ss}`$ and the associated $`2`$-divisible group $`_𝒳(2)`$ is isomorphic to $`𝒥𝒳(2)^{00}`$ over $`\mathrm{Spec}(R)`$.

### 4.2 Deformation of elliptic curves

In this section we compute the linear action of the $`2`$-torsion point of $``$ on the space of second order theta functions $`H^0(,2\mathrm{\Theta })`$ for a family of elliptic curves $`/\mathrm{Spec}(R)`$ with supersingular special fibre $`_0E^{ss}`$ and ordinary generic fibre $`_\eta `$.

#### 4.2.1 Addition on an ordinary elliptic curve

Let $`E`$ be an ordinary elliptic curve defined over a field $`K`$ by the homogeneous equation

$$Y^2Z+a_1XYZ=X^3+a_2X^2Z+a_4XZ^2$$
(4.2)
with $`a_1,a_2,a_4K`$ and $`a_10`$. We take as origin the inflection point $`\mathrm{}`$ with projective coordinates $`(0:1:0)`$. The projection with center $`\mathrm{}`$ gives a $`2:1`$ morphism $`E\stackrel{\pi }{}_K^1`$, with $`X,Z`$ projective coordinates on $`_K^1`$. The Abel-Jacobi map $`EJE`$, $`e𝒪_E(e\mathrm{})`$ identifies $`E`$ with $`JE`$. Under this identification the theta divisor $`\mathrm{\Theta }_B`$, associated to the canonical theta-characteristic $`B=𝒪_E(P\mathrm{})JE[2]`$, becomes the $`2`$-torsion point $`P`$ with projective coordinates $`(0:0:1)`$. Moreover, using this identification, we have $`𝒪_{JE}(2\mathrm{\Theta })=𝒪_E(2P)=\pi ^{}𝒪_^1(1)`$.

The point $`BJE[2]`$ induces a linear involution, denoted by $`g`$, on the space

$$W=H^0(JE,𝒪(2\mathrm{\Theta }))=H^0(E,\pi ^{}𝒪_^1(1)),$$
(4.3)
such that for all nonzero $`sW`$ we have $`T_B\mathrm{div}(s)=\mathrm{div}(g.s)`$. Here $`T_B`$ denotes translation in $`JE`$ by the point $`B`$. The space $`W`$ has two distinguished bases: first the coordinate functions $`\{X,Z\}`$ and secondly the Theta basis $`\{X_0,X_1\}`$ (see \[LP\] section 2). Since the canonical section $`X_0W`$ (associated to the divisor $`\mathrm{\Theta }_B`$) is proportional to $`X`$, there exists a nonzero $`aK`$ such that

$$g.X=aZ,g.Z=a^1X.$$

In order to determine $`a`$ in terms of the coefficients $`a_iK`$, we choose one of the two points on $`E`$ with projective coordinates of the form $`(1:Y:1)`$ and call it $`A`$. By construction we have $`A\mathrm{div}(X+Z)`$ and, after applying $`T_B`$, we obtain $`A+PT_B\mathrm{div}(X+Z)`$. Since

$$T_B\mathrm{div}(X+Z)=\mathrm{div}(g.X+g.Z)=\mathrm{div}(aZ+a^1X),$$

we deduce that

$$\left(\frac{X}{Z}\right)(A+P)=a^2.$$

Now the addition formula for elliptic curves (see e.g. \[S\] page 59) implies that $`\left(\frac{X}{Z}\right)(A+P)=a_4`$. Hence $`a=(a_4)^{1/2}`$ and the Theta basis of $`W`$ is given by

$$X_0=X,X_1=g.X_0=(a_4)^{1/2}Z.$$
(4.4)
#### 4.2.2 An example

We consider the family of elliptic curves $``$ over $`\mathrm{Spec}(R)`$ defined by the homogeneous equation

$$V^2Z+t^4UVZ+VZ^2=U^3+UZ^2$$
(4.5)
with origin $`\mathrm{}=(0:1:0)`$. The generic fiber $`_\eta `$ is an ordinary elliptic curve over $`\mathrm{Spec}(K)`$, with $`K=k((t))`$, and the special fibre is supersingular,i.e., $`_0E^{ss}`$. The $`2`$-torsion point $`P_\eta `$ of $`_\eta `$ has projective coordinates

$$P_\eta =t^6(u_0:v_0:1),$$

with $`u_0=t^4`$, $`v_0=t^2+t^6`$, which specializes to $`\mathrm{}_0_0`$.

The $`R`$-module $`H^0(,2\mathrm{\Theta })`$ is free, of rank $`2`$, and an $`R`$-basis is given by $`\{U,Z\}`$. In order to compute the linear action $`g`$ of the $`2`$-torsion point $`P`$ on $`H^0(,2\mathrm{\Theta })`$, we consider the generic fibre $`_\eta `$. The change of variables $`X=U+u_0Z`$ and $`Y=V+v_0Z`$ transforms equation (4.5) into (4.2), with $`a_1=t^4`$, $`a_2=u_0`$, and $`a_4=1+u_0^2+t^4v_0=1+t^8+t^2+t^2`$. With the notation of section 4.2.1 we find $`a=t^4(1+t^3+t^4+t^5)=(a_4)^{1/2}`$. Therefore the action of $`g`$ on $`H^0(,2\mathrm{\Theta })`$ is given by the formulae

$`g.U={\displaystyle \frac{1}{1+t^3+t^4+t^5}}U+{\displaystyle \frac{t^2+t^4+t^6}{1+t^3+t^4+t^5}}Z,`$
$`g.Z={\displaystyle \frac{t^4}{1+t^3+t^4+t^5}}U+{\displaystyle \frac{1}{1+t^3+t^4+t^5}}Z,`$
and, using (4.4), the theta functions $`X_0`$ and $`X_1`$ can be expressed in the $`R`$-basis $`\{U,Z\}`$ as follows

$$\begin{array}{ccc}X_0& =& t^4U+Z,\hfill \\ X_1& =& g.X_0=(1+t^3+t^4+t^5)Z.\hfill \end{array}$$

Note that the two sections $`X_0_Rk`$ and $`X_1_Rk`$ coincide at the special fibre $`H^0(_0,2\mathrm{\Theta }_{|_0})H^0(,2\mathrm{\Theta })_Rk`$.

## 5 Equations of $`\stackrel{~}{V}`$ for non-ordinary $`X`$

### 5.1 Specializing an ordinary curve

Let $`X/k`$ be a smooth genus $`2`$ curve with Hasse-Witt invariant equal to $`1`$. Following e.g. \[L\] $`X`$ is birational to an affine curve given by an equation of the form

$$y^2+xy=\lambda x^5+\mu x^3+x,$$

with $`\lambda ,\mu k`$ and $`\lambda 0`$. The projection $`(x,y)x`$ defines a separable double cover $`X_k^1`$ ramified at $`0`$ and $`\mathrm{}`$. Let $`_R^1`$ be the projective line over $`R=k[[s]]`$ with affine coordinate $`x`$. We introduce the family $`𝒳_R^1`$ defined by the projective closure of the affine curve with equation

$$y^2+(sx^2+x)y=\lambda x^5+\mu x^3+x.$$

The special fibre $`𝒳_0/k`$ equals $`X`$ and the generic fibre $`𝒳_\eta /K`$ of the family $`𝒳`$ is a smooth ordinary curve of genus $`2`$, which is birational to the curve (defined over a finite extension of $`K`$) given by the standard equation (2.5) with coefficients

$$a=\lambda /s^3,b=\alpha ^2+\alpha ,c=s,\text{and}\alpha ^2=\lambda /s^3+\mu /s+s.$$
(5.1)
Let $`𝒥𝒳`$ be the associated Jacobian scheme and $`𝒥𝒳[2]/R`$ be the group scheme of $`2`$-torsion points. Then we have the following isomorphisms

$$𝒥𝒳[2]_\eta \left(/2\right)^2\times \mu _2^2/K,𝒥𝒳[2]_0JX[2]\left(/2\right)\times \mu _2\times G_{l,l}/k,$$

where $`G_{l,l}`$ is the unique self-dual local-local group scheme of dimension 1 and length 4. Note that $`G_{l,l}`$ is isomorphic to the group scheme of $`2`$-torsion points $`E^{ss}[2]`$ (see section 4.2.2). The étale parts of both fibres can be described in terms of Weierstrass points as follows.

The $`3`$ Weierstrass points of $`𝒳_\eta _K^1`$ are $`0_\eta `$, $`\mathrm{}_\eta `$, and $`1_\eta `$ with affine coordinate $`1/s`$, which specialize to $`0`$, $`\mathrm{}`$ and $`\mathrm{}`$ respectively. We obtain by (2.2) the three nonzero elements of $`𝒥𝒳[2]_\eta ^{et}`$, which we denote by $`[0]_\eta `$, $`[1]_\eta `$ and $`[\mathrm{}]_\eta `$. At the special fibre the nonzero 2-torsion point in $`JX[2]^{et}/2`$ equals $`AJ(0+\mathrm{})`$, which we denote by $`[1]_0`$. We see that $`[1]_\eta `$ and $`[\mathrm{}]_\eta `$ specialize to $`[1]_0JX[2]^{et}`$, and $`[0]_\eta `$ specializes to $`0`$.

### 5.2 Decomposing Heisenberg groups

We are interested in the linear action of the Heisenberg group scheme $`/R`$, which is a central extension (see \[M\] page 221)

$$0\mu _2𝒥𝒳[2]0,$$
(5.2)
on the free $`R`$-module $`𝒲=H^0(𝒥𝒳[2],2\mathrm{\Theta })`$ of rank $`4`$. We choose the splitting over $`R`$ of the connected-étale exact sequence

$$0𝒥𝒳[2]^0𝒥𝒳[2]𝒥𝒳[2]^{et}0$$

determined by the nonzero $`2`$-torsion point $`[1]=AJ(0+\mathrm{})𝒥𝒳[2]`$. Note that $`𝒥𝒳[2]^{et}/2`$ and that $`[\mathrm{}]𝒥𝒳[2]`$ determines a different splitting. Passing to the Cartier dual we obtain a decomposition over $`R`$,

$$𝒥𝒳[2]=/2\times \mu _2\times 𝒥𝒳[2]^{00}.$$

Pulling-back the central extension (5.2) by the canonical inclusions of $`/2\times \mu _2`$ and $`𝒥𝒳[2]^{00}`$ into $`𝒥𝒳[2]`$ we obtain the two Heisenberg groups $`^{et}`$ and $`^0`$

$$0\mu _2^{et}/2\times \mu _20,0\mu _2^0𝒥𝒳[2]^{00}0.$$

It is clear that the Heisenberg group scheme $``$ (5.2) is isomorphic to the quotient $`^0\times ^{et}/\mu _2`$, where $`\mu _2`$ acts diagonally on $`^0\times ^{et}`$. Let $`𝒲^{et}`$ and $`𝒲^0`$ be the sub-$`R`$-modules of $`𝒲`$ fixed by the subgroups $`^0`$ and $`^{et}`$ of $``$. By general theory of Heisenberg groups, $`𝒲`$ is the unique irreducible $``$-module of weight $`1`$, which implies an $``$-isomorphism

$$𝒲𝒲^0𝒲^{et}.$$

Moreover $`𝒲^0`$ (resp. $`𝒲^{et}`$) is the unique irreducible $`^0`$ (resp. $`^{et}`$)-module of weight $`1`$.

Let $`H/k`$ be the Heisenberg group scheme associated to $`/2\times \mu _2`$ ($`E[2]`$ for any ordinary elliptic curve $`E/k`$) and let $`W`$ be the unique irreducible $`H`$-module of weight $`1`$. Note that $`W`$ is isomorphic (as $`H`$-module) to the space (4.3). It is clear that we have the following isomorphisms

$$^{et}H_kR,𝒲^{et}W_kR.$$

We will denote by $`\{Z_0,Z_1\}`$ the “constant” $`R`$-basis of $`𝒲^{et}`$ induced by the Theta basis $`\{X_0,X_1\}`$ of $`W`$,i.e., $`Z_i:=X_i_k1`$.

The structure of the $`^0`$-module $`𝒲^0`$ is determined by analyzing the group scheme $`𝒥𝒳[2]^{00}`$. In section 4.1 we considered the $`2`$-divisible group $`𝒥𝒳(2)^{00}`$ and we showed the existence of an elliptic curve $`_𝒳/R`$ such that $`_𝒳(2)𝒥𝒳(2)^{00}`$. In particular $`_𝒳[2]𝒥𝒳[2]^{00}`$. We observe that the $`j`$-invariants of the elliptic curves $`_𝒳/R`$ and $`/R`$ (section 4.2.2) lie in the maximal ideal of $`R`$, because $`(_𝒳)_0_0E^{ss}`$. Therefore there exists a relation of the form

$$s^n=ut^m$$
(5.3)
between the two uniformizing parameters $`s`$ in (5.1) and $`t`$ in (4.5), with $`u`$ invertible in $`k[[t]]`$ and $`n,m^{}`$, and we can assume, after passing to the ramified cover given by (5.3), that the $`^0`$-module $`𝒲^0`$ equals $`H^0(,2\mathrm{\Theta })`$.

In order to have a consistent notation we denote the $`R`$-basis $`\{U,Z\}`$ of $`𝒲^0=H^0(,2\mathrm{\Theta })`$ by $`\{Z_0,Z_1\}`$ and recall from section 4.2.2 the transition formulae

$$X_0=t^4Z_0+Z_1,X_1=(1+t^3+t^4+t^5)Z_1.$$
(5.4)
Let $`\{x_i\}`$ and $`\{z_i\}`$ denote the dual $`K`$-bases of $`\{X_i\}`$ and $`\{Z_i\}`$ in both spaces $`𝒲^0`$ and $`𝒲^{et}`$. Then the $`4`$ tensors $`z_{ij}:=z_iz_j𝒲^{}`$ form an $`R`$-basis and the dual Theta functions $`x_{ij}:=x_ix_j`$ can be expressed as follows (after chasing denominators)

$$\begin{array}{cc}x_{00}=(1+t^3+t^4+t^5)z_{00},& x_{10}=z_{00}+t^4z_{10},\\ x_{01}=(1+t^3+t^4+t^5)z_{01},& x_{11}=z_{01}+t^4z_{11}.\end{array}$$
(5.5)
Note that the coordinate $`x_{10}`$ specializes to $`x_{00}`$ and $`x_{11}`$ specializes to $`x_{01}`$. Via the level 2 structure (2.3) this parallels the specialization of the $`2`$-torsion points $`[0]_\eta `$, $`[1]_\eta `$, and $`[\mathrm{}]_\eta `$ (section 5.1).

### 5.3 Specializing the quadrics

It can be shown as in \[LP\] section 5 that the identification $`\mathrm{M}_X|2\mathrm{\Theta }|`$ extends to the relative case $`𝒳\mathrm{Spec}(R)`$,i.e., we have an isomorphism $`\mathrm{M}_𝒳(𝒲)`$ over $`\mathrm{Spec}(R)`$. Therefore the relative Frobenius morphism $`𝒳𝒳_1`$ (over $`\mathrm{Spec}(R)`$) induces by pull-back a rational map

$$\begin{array}{ccccc}(𝒲_1)\hfill & & \stackrel{\stackrel{~}{𝒱}}{}& & \hfill (𝒲)\\ & & & & \\ & & \mathrm{Spec}(R)& & \end{array}$$

with $`𝒲_1=H^0(𝒥𝒳_1,2\mathrm{\Theta }_1)`$. We recall that the map $`\stackrel{~}{𝒱}`$ is given by a linear system of $`4`$ quadrics. Over the generic point $`\eta \mathrm{Spec}(R)`$ the equations of the map $`\stackrel{~}{𝒱}_\eta :(𝒲_1)_\eta (𝒲)_\eta `$ are of the form (\[LP\] Proposition 3.1 (3))

$$\stackrel{~}{𝒱}:x=(x_{ij})(\lambda _{00}P_{00}(x):\lambda _{01}P_{01}(x):\lambda _{10}P_{10}(x):\lambda _{11}P_{11}(x))$$
(5.6)
with

$$\begin{array}{c}P_{00}(x)=x_{00}^2+x_{01}^2+x_{10}^2+x_{11}^2,\hfill \\ P_{01}(x)=x_{00}x_{01}+x_{10}x_{11},\hfill \end{array}\begin{array}{c}P_{10}(x)=x_{00}x_{10}+x_{01}x_{11},\hfill \\ P_{11}(x)=x_{00}x_{11}+x_{10}x_{01}.\hfill \end{array}$$

Here we use the $`K`$-basis of Theta coordinates $`x_{ij}`$ on $`(𝒲_1)_\eta `$ and $`(𝒲)_\eta `$. Moreover Proposition 3.1 relates the coefficients $`\lambda _{ij}`$ (defined up to a scalar) to the coefficients $`a,b,cK`$ (5.1),

$$(\lambda _{00}:\lambda _{01}:\lambda _{10}:\lambda _{11})=(\sqrt{abc}:\sqrt{b}:\sqrt{c}:\sqrt{a}).$$

Since the Theta coordinates $`x_{ij}`$ are no longer independent after specialization, we express the equations (5.6) of $`\stackrel{~}{𝒱}`$ in the $`R`$-basis $`\{z_{ij}\}`$ using the transition formulae (5.5). A straight-forward computations shows that the map

$$\stackrel{~}{𝒱}:z=(z_{ij})(R_{00}(z):R_{01}(z):R_{10}(z):R_{11}(z))$$
(5.7)
is given by the quadrics

$$\begin{array}{ccc}\hfill R_{00}(z)& =& \frac{\sqrt{abc}}{1+t^3+t^4+t^5}\left[(t^{12}+t^{16}+t^{20})(z_{00}^2+z_{01}^2)+t^{16}(z_{10}^2+z_{11}^2)\right],\hfill \\ \hfill R_{01}(z)& =& \frac{\sqrt{b}}{1+t^3+t^4+t^5}\left[(t^{12}+t^{16}+t^{20})z_{00}z_{01}+t^8(z_{00}z_{11}+z_{10}z_{01})+t^{16}z_{10}z_{11}\right],\hfill \\ \hfill R_{10}(z)& =& \frac{1}{t^4}\left[R_{00}+\sqrt{c}(1+t^6+t^8+t^{10})\left(z_{00}^2+z_{01}^2+t^8(z_{00}z_{10}+z_{11}z_{01})\right)\right],\hfill \\ \hfill R_{11}(z)& =& \frac{1}{t^4}\left[R_{01}+\sqrt{a}(1+t^6+t^8+t^{10})t^8(z_{00}z_{11}+z_{01}z_{10})\right].\hfill \end{array}$$
(5.8)
Note that we square the coefficients in (5.5) when considering coordinates on $`(𝒲_1)`$. At the special fibre the map (5.7) specializes to $`\stackrel{~}{𝒱}_0`$ obtained by putting $`t=0`$ after having divided the $`R_{ij}`$’s by $`t^\alpha `$, where $`\alpha `$ is the lowest valuation appearing in the expressions (5.8). The map $`\stackrel{~}{𝒱}_0`$ coincides with the Verschiebung $`\stackrel{~}{V}`$ of the curve $`X`$, because both maps extend to rational maps over $`R`$ and coincide over $`K`$. Since the image of $`\stackrel{~}{V}`$ is non-degenerate (it contains the Kummer surface $`\mathrm{Kum}_X|2\mathrm{\Theta }|`$), the lowest valuations for each of the $`4`$ quadrics $`R_{ij}`$ coincide (otherwise the image of $`\stackrel{~}{𝒱}_0`$ is contained in a hyperplane).

We work out the specialization of the quadrics as follows. We write $`\nu =\frac{m}{n}`$ and replace $`s`$ by $`vt^\nu `$, with $`vk[[t]]`$ invertible, in the expression of the coefficients $`a,b,c`$ of (5.1). Note that the (rational) valuations of $`\sqrt{a},\sqrt{b},\sqrt{c}`$ are $`\frac{3}{2}\nu ,\frac{3}{2}\nu ,\frac{1}{2}\nu `$ respectively. First we observe that the valuations of $`R_{01}`$ and $`R_{00}`$ equal $`8\frac{3}{2}\nu `$ and $`12\frac{5}{2}\nu `$ respectively. Since they coincide, we obtain $`\nu =4`$,i.e.,

$$R_{00}=t^2(z_{00}^2+z_{01}^2)+\text{h.o.t.},R_{01}=t^2(z_{00}z_{11}+z_{10}z_{01})+\text{h.o.t.},$$

up to some multiplicative nonzero constants. Next we see that the expansions of $`R_{10}`$ and $`R_{11}`$ are given by

$$R_{10}=t^2(z_{00}^2+z_{01}^2+z_{10}^2+z_{11}^2)+\text{h.o.t.},R_{11}=t^2(z_{00}z_{01})+\text{h.o.t.},$$

up to some multiplicative nonzero constants and some multiple of $`R_{00}`$ and $`R_{01}`$ respectively. Thus we have shown

###### 5.1 Theorem.

Let $`X`$ be a smooth curve with Hasse-Witt invariant equal to $`1`$. There exist coordinates $`\{z_{ij}\}`$ on $`|2\mathrm{\Theta }_1|`$ and $`\{y_{ij}\}`$ on $`|2\mathrm{\Theta }|`$ such that the equations of $`\stackrel{~}{V}`$ are given by

$$|2\mathrm{\Theta }_1|\stackrel{\stackrel{~}{V}}{}|2\mathrm{\Theta }|,z=(z_{ij})y=(y_{ij})=(\lambda _{00}Q_{00}(z):\lambda _{01}Q_{01}(z):\lambda _{10}Q_{10}(z):\lambda _{11}Q_{11}(z))$$

with

$$\begin{array}{c}Q_{00}(z)=z_{00}^2+z_{01}^2,\hfill \\ Q_{01}(z)=z_{00}z_{11}+z_{10}z_{01},\hfill \end{array}\begin{array}{c}Q_{10}(z)=z_{00}^2+z_{01}^2+z_{10}^2+z_{11}^2,\hfill \\ Q_{11}(z)=z_{00}z_{01},\hfill \end{array}$$

and the $`\lambda _{ij}`$’s are nonzero constants depending on the curve $`X`$.

###### 5.2 Remark.

We note that the equations of $`\stackrel{~}{V}`$ given in Theorem 5.1 are written in two sets of coordinates on $`|2\mathrm{\Theta }|`$ and $`|2\mathrm{\Theta }_1|`$ which do not necessarily correspond under the $`k`$-semi-linear isomorphism $`JX_1JX`$.

###### 5.3 Remark.

In case $`X`$ is a non-ordinary curve with Hasse-Witt invariant equal to $`0`$, i.e., $`X`$ is supersingular, we observe that the $`2`$-divisible group $`JX(2)=JX(2)^{00}`$ (see section 4.1) is self-dual, of dimension $`2`$ and height $`4`$. There exists a finite number of isomorphism classes of such $`2`$-divisible groups over $`k`$ (see \[D\] page 93). Moreover one can show that $`JX(2)`$ cannot be isomorphic to the product $`E^{ss}(2)\times E^{ss}(2)`$.

### 5.4 Applications

As in \[LP\] section 6 we can easily deduce from Theorem 5.1 a full description of the Verschiebung $`\stackrel{~}{V}`$. Since the computations are analoguous to those of \[LP\] Proposition 6.1, we leave them to the reader.

###### 5.4 Proposition.

Let $`X`$ be a smooth genus $`2`$ curve with Hasse-Witt invariant equal to $`1`$.

1. There exists a unique stable bundle $`E_{BAD}\mathrm{M}_{X_1}`$, which is destabilized by the Frobenius map, i.e., $`F^{}E_{BAD}`$ is not semi-stable. We have $`E_{BAD}=F_{}B^1`$ and its projective coordinates are $`(0:0:1:1)`$.
2. Let $`H_1`$ be the hyperplane in $`|2\mathrm{\Theta }_1|`$ defined by $`z_{00}+z_{01}=0`$. The map $`\stackrel{~}{V}`$ contracts $`H_1`$ to the conic $`\mathrm{Kum}_XH`$, where $`H`$ is the hyperplane in $`|2\mathrm{\Theta }|`$ defined by $`y_{00}=0`$.
3. The fiber of $`\stackrel{~}{V}`$ over a point $`[E]\mathrm{M}_X`$ is

* a non-degenerate $`/2`$-orbit of a point $`[E_1]\mathrm{M}_{X_1}`$, if $`[E]H`$
    * empty, if $`[E]H(H\mathrm{Kum}_X)`$
    * a projective line passing through $`E_{BAD}`$, if $`[E]H\mathrm{Kum}_X`$

In particular, $`\stackrel{~}{V}`$ is dominant and non-surjective. The separable degree of $`\stackrel{~}{V}`$ is $`2`$.

## 6 Equations of $`\stackrel{~}{V}`$ in characteristic $`3`$

Let $`X`$ be a smooth curve of genus $`2`$ defined over a field of characteristic $`p=3`$. The main result of this section is

###### 6.1 Theorem.

1. There exists an embedding $`\alpha :\mathrm{Kum}_X|2\mathrm{\Theta }_1|`$ such that the equality of hypersurfaces in $`|2\mathrm{\Theta }_1|`$

$$\stackrel{~}{V}^1(\mathrm{Kum}_X)=\mathrm{Kum}_{X_1}\alpha (\mathrm{Kum}_X)$$

holds set-theoretically.
2. The cubic equations of $`\stackrel{~}{V}`$ are given by the $`4`$ partials of the quartic equation of the Kummer surface $`\alpha (\mathrm{Kum}_X)|2\mathrm{\Theta }_1|`$. In other words, $`\stackrel{~}{V}`$ is the polar map of the surface $`\alpha (\mathrm{Kum}_X)`$.

###### Proof.

Given $`LJX`$ and a section $`\phi H^0(X,\omega L^2)`$, we can consider, using adjunction and relative duality for the map $`F`$, the vector bundle map

$$F_{}L\stackrel{\phi }{}L\omega ,$$

where we also write $`L`$ for the pull-back $`\iota ^{}L`$ under the $`k`$-semi-linear isomorphism $`\iota :X_1X`$. We define $`E_L:=\mathrm{ker}(\phi )`$. Hence there is an exact sequence of vector bundles over $`X_1`$

$$0E_LF_{}L\stackrel{\phi }{}L\omega 0.$$
(6.1)
It is staight-forward to show that $`E_L`$ has determinant $`𝒪_{X_1}`$, is semi-stable, that $`E_L=E_{L^1}`$, (if $`L^1L`$) and that $`F^{}E_L`$ is $`S`$-equivalent to $`LL^1`$. Indeed, by adjunction, the inclusion $`E_LF_{}L`$ induces a nonzero map $`F^{}E_LL`$.

We observe that for $`L`$ such that $`L^2𝒪`$, $`\mathrm{dim}H^0(\omega L^2)=1`$, hence $`\phi `$ is uniquely defined (up to a scalar). For $`L`$ such that $`L^2=𝒪`$, we obtain a projective line $`_L^1`$ of rank $`2`$ vector bundles $`E_{L,\phi }`$ with $`\phi |\omega |`$. The variety of pairs $`(L,\phi )`$ is isomorphic to the blowing-up $`\mathrm{Bl}_2(JX)`$ of $`JX`$ at the $`16`$ $`2`$-torsion points $`LJX[2]`$. Hence we obtain a morphism

$$e:\mathrm{Bl}_2(JX)\mathrm{M}_{X_1}|2\mathrm{\Theta }_1|,(L,\phi )E_L.$$

We will determine the image of the morphism $`e`$. First we consider the image of the $`16`$ exceptional divisors $`_L^1`$ for $`LJX[2]`$. We identify $`H^0(X_1,\omega _1)=H^0(X,\omega )`$.

###### 6.2 Lemma.

Given $`LJX[2]`$ and $`w`$ a Weierstrass point,i.e., $`2w|\omega |`$. Then we have $`E_{L,\phi }BL^1(w)`$ with $`Div(\phi )=2w`$.

###### Proof.

We tensorize the exact sequence (6.1) with $`L`$, using $`F_{}LL=F_{}(LF^{}L)=F_{}𝒪_X`$ (projection formula)

$$\begin{array}{ccccccccc}& & & & 𝒪_{X_1}& & & & \\ & & & & & ^\phi & & & \\ 0& & E_{L,\phi }L& & F_{}𝒪_X& & \omega _{X_1}& & 0\\ & & & ^{\widehat{\phi }}& & & & & \\ & & & & B& & & & \end{array}$$

The vertical arrows form an exact sequence (see \[R\] section 4.1). The upper diagonal map defined as composite map $`𝒪_{X_1}F_{}𝒪_X\omega _{X_1}`$ equals $`\phi `$, which implies that the lower diagonal map $`\widehat{\phi }:E_{L,\phi }LB`$ vanishes at $`w`$. Hence, using stability of $`B`$ and $`\mathrm{det}B=\omega `$, we obtain $`E_{L,\phi }LB(w)`$. ∎

We observe that $`L(w)`$ is a theta-characteristic $`\kappa `$. Moreover the image of $`_L^1`$ in $`|2\mathrm{\Theta }_1|=^3`$ is a conic and it is straight-forward to establish that the $`16`$ points $`B\kappa ^1`$ (with $`\kappa `$ varying over the set of theta-characteristics) and the $`16`$ hyperplanes spanned by the $`16`$ conics $`e(_L^1)`$ form a classical $`16_6`$-configuration of nodes and tropes, whose associated Kummer surface equals $`\mathrm{Kum}_X`$ (see \[GD\]). We recover the curve $`X`$ e.g. by taking the $`2:1`$ cover over a $`_L^1`$ branched at the $`6`$ nodes on $`_L^1`$. Since $`\mathrm{deg}e(\mathrm{Bl}_2(JX))=4`$ and $`e(\mathrm{Bl}_2(JX))`$ is singular at the $`16`$ points $`B\kappa ^1`$, the image $`e(\mathrm{Bl}_2(JX))`$ coincides with $`\alpha (\mathrm{Kum}_X)`$ for an embedding $`\alpha :\mathrm{Kum}_X\mathrm{M}_{X_1}`$

By Proposition 7.2 the map $`\stackrel{~}{V}`$ is given by a linear system $`||`$ of $`4`$ cubics on $`|2\mathrm{\Theta }_1|`$. The key fact underlying Theorem 6.1 is a striking relationship between cubics and quartics on $`|2\mathrm{\Theta }_1|`$ (\[vG\] Proposition 2): the $`4`$ cubics in $`||`$ are the $`4`$ partial derivatives of a Heisenberg invariant quartic, whose zero divisor we denote by $`Q|2\mathrm{\Theta }_1|`$.

By \[R\] Remarque 4.1.2 (2) the $`16`$ points $`B\kappa ^1`$ (a $`JX_1[2]`$-orbit) are contained in the base locus of $`||`$. Hence $`Q`$ is singular at the $`16`$ points $`B\kappa ^1`$. Since a Kummer surface is determined by its singular points, we conclude that $`Q\alpha (\mathrm{Kum}_X)`$ and that $`||`$ is given by the $`4`$ partials of $`\alpha (\mathrm{Kum}_X)`$. This completes the proof of Theorem 6.1. ∎

###### 6.3 Remark.

1. The rational map $`e:\mathrm{Kum}_X\mathrm{M}_{X_1}`$ (defined away from the $`16`$ nodes of $`\mathrm{Kum}_X`$) is the birational inverse of $`\stackrel{~}{V}`$.
2. One has the following scheme-theoretical equality (among divisors in $`|2\mathrm{\Theta }_1|`$)

$$\stackrel{~}{V}^1(\mathrm{Kum}_X)=\mathrm{Kum}_{X_1}+2\alpha (\mathrm{Kum}_X).$$

###### 6.4 Corollary.

1. The map $`\stackrel{~}{V}`$ has exactly $`16`$ base points, which correspond bijectively to the

* $`16`$ nodes of the surface $`\alpha (\mathrm{Kum}_X)|2\mathrm{\Theta }_1|`$
    * $`16`$ stable rank $`2`$ vector bundles $`B\kappa ^1\mathrm{M}_{X_1}|2\mathrm{\Theta }_1|`$, where $`B`$ is the bundle of locally exact differentials and $`\kappa `$ a theta-characteristic of $`X_1`$.
2. The map $`\stackrel{~}{V}`$ is surjective, separable and of degree $`11`$.

###### Proof.

We only have to show part 2, since part 1 is clear from the proof of Theorem 6.1. We recall that the rational map $`\stackrel{~}{V}`$, which is defined away from the $`16`$ points $`B\kappa ^1`$, coincides with the polar map of the Kummer surface $`\mathrm{Kum}_X`$. It is well-known that $`\stackrel{~}{V}`$ can be resolved into a morphism $`𝒱:\mathrm{Bl}(|2\mathrm{\Theta }_1|)|2\mathrm{\Theta }|`$ by blowing-up these $`16`$ points in $`|2\mathrm{\Theta }_1|`$. We denote by $`E_\kappa `$ the exceptional divisor over $`B\kappa ^1`$ and by $`H_\kappa |2\mathrm{\Theta }|`$ the hyperplane $`𝒱(E_\kappa )`$. Note that the $`H_\kappa `$ are the $`16`$ tropes of $`\mathrm{Kum}_X|2\mathrm{\Theta }|`$ and that $`𝒱_{|E_\kappa }`$ is a linear isomorphism. It is clear that the image of $`\stackrel{~}{V}`$ contains the complement of the $`16`$ hyperplanes $`H_\kappa `$.

Let us check that the $`H_\kappa `$ are also contained in the image of $`\stackrel{~}{V}`$: a simple computation shows that the cubic $`C_\kappa :=\stackrel{~}{V}^1(H_\kappa )|2\mathrm{\Theta }_1|`$ is singular at the point $`B\kappa ^1`$ and that the restriction of $`\stackrel{~}{V}`$ to the cubic $`C_\kappa `$ concides with the (birational) projection with center $`B\kappa ^1`$. Moreover the projectivized tangent cone at $`B\kappa ^1`$ to $`C_\kappa `$ is the conic $`Q_\kappa H_\kappa `$ through the $`6`$ nodes (recall that $`2Q_\kappa =H_\kappa \mathrm{Kum}_X`$). Therefore any point in $`H_\kappa Q_\kappa `$ lies in the image of $`\stackrel{~}{V}`$. To finish the argument we observe that $`Q_\kappa \mathrm{Kum}_X`$ and that $`\stackrel{~}{V}:\mathrm{Kum}_{X_1}\mathrm{Kum}_X`$ is surjective. ∎

###### 6.5 Remark.

1. We recall (\[LP\] Remark 6.2) that surjectivity only holds for $`S`$-equivalence classes (not isomorphism classes!). In fact, there always exist semi-stable bundles $`E`$ which do not descend by Frobenius.
2. The number of base points and the degree of $`\stackrel{~}{V}`$ was also obtained in \[O\] by computing the number of connections (on certain unstable bundles) with zero $`p`$-curvature.
3. It would be interesting to have an explicit description of the $`11`$ vector bundles in a general fiber $`\stackrel{~}{V}^1(E)`$ of the polar map $`\stackrel{~}{V}`$.

## 7 Appendix: base points of $`\stackrel{~}{V}`$

In this section we consider a smooth curve $`X`$ of genus $`g2`$ defined over an algebraically closed field $`k`$ of caracteristic $`p>0`$. We denote by $`\mathrm{M}_X(r)`$ (resp. $`\mathrm{M}_{X_1}(r)`$) the moduli space of semi-stable rank $`r`$ vector bundles over $`X`$ (resp. $`X_1`$) and by $``$ (resp. $`_1`$) the determinant line bundle over $`\mathrm{M}_X(r)`$ (resp. $`\mathrm{M}_{X_1}(r)`$). The relative Frobenius map $`F:XX_1`$ induces by pull-back a rational map

$$V:\mathrm{M}_{X_1}(r)\mathrm{M}_X(r),$$

called the Verschiebung. Let $``$ be the indeterminacy locus of $`V`$, i.e., the closed subscheme of $`\mathrm{M}_{X_1}(r)`$ consisting of semi-stable bundles $`E`$ such that $`F^{}E`$ is not semi-stable. Let $`U=\mathrm{M}_{X_1}(r)`$ be the open subset where $`V`$ is a morphism.

### 7.1 General facts

###### 7.1 Proposition.

We have an isomorphism $`V^{}()\left(_1^p\right)_{|U}`$.

###### Proof.

Let $`_X(r)`$ and $`_{X_1}(r)`$ be the moduli stacks of rank $`r`$ vector bundles over $`X`$ and $`X_1`$ and let $``$ and $`_1`$ be the universal bundles with trivialized determinant on $`X\times _X(r)`$ and $`X_1\times _{X_1}(r)`$. It is well-known that the inverses of the determinant of cohomology, which we denote by $`\mathrm{det}Rp_{}`$ and $`\mathrm{det}Rp_1_1`$ descend (after restriction to the semi-stable loci) to the line bundles $``$ and $`_1`$ on the moduli spaces $`\mathrm{M}_X(r)`$ and $`\mathrm{M}_{X_1}(r)`$. Now, since $`\mathrm{det}Rp_{}`$ commutes with base change, we have an isomorphism over the moduli stack $`_{X_1}(r)`$

$$V^{}(\mathrm{det}Rp_{})\mathrm{det}Rp_{}\left((F\times \mathrm{Id})^{}_1\right).$$

Moreover we have a commutative diagram

$`X\times _{X_1}(r)`$ $`\stackrel{F\times \mathrm{Id}}{}`$ $`X_1\times _{X_1}(r)`$
$`^p`$ $`^{p_1}`$
$`_{X_1}(r)`$
where $`p`$ and $`p_1`$ denote the projections on the second factor. Since $`F\times \mathrm{Id}`$ is an affine morphism, we have $`R^1(F\times \mathrm{Id})_{}=0`$. Hence

$$\mathrm{det}Rp_{}\left((F\times \mathrm{Id})^{}_1\right)\mathrm{det}Rp_1\left((F\times \mathrm{Id})_{}(F\times \mathrm{Id})^{}_1\right)\mathrm{det}Rp_1(_1F_{}𝒪_X).$$

The last equality follows from the projection formula. Using a filtration by line bundles of the rank $`p`$ bundle $`F_{}𝒪_X`$ and by showing that $`\mathrm{det}Rp_1(_1𝒪_{X_1}(D))=\mathrm{det}Rp_1(_1)`$ for an effective divisor $`D`$, we show that $`\mathrm{det}Rp_1(_1F_{}𝒪_X)(\mathrm{det}Rp_1_1)^p`$. We obtain the isomorphism of the lemma by descent on $`U`$. ∎

###### 7.2 Proposition.

If $`g=2`$ and $`r=2`$, then $`\mathrm{dim}=0`$ and the rational map $`\stackrel{~}{V}`$ is given by polynomials of degree $`p`$.

###### Proof.

The fact that $`\mathrm{dim}=0`$ is proved in Theorem 3.2 \[JX\]. This implies that $`V^{}()`$ extends uniquely to $`_1^p`$ over $`\mathrm{M}_{X_1}`$ and the lemma follows from the isomorphism $`_1𝒪_^3(1)`$. ∎

###### 7.3 Remark.

For general $`g,r,p`$ we do not know an estimate of the dimension of $``$.

### 7.2 Existence of base points

###### 7.4 Theorem.

The indeterminacy locus $``$ is non-empty.

###### Proof.

First it will be enough to show non-emptiness of $``$ in the case $`r=2`$, since taking direct sums with the trivial bundle implies non-emptiness for arbitrary $`r`$. Secondly it suffices to show non-emptiness of $``$ after a field extension $`k^{}/k`$, with $`k^{}`$ algebraically closed.

Let $`\overline{_g}`$ be the coarse moduli space of stable genus $`g`$ curves defined over $`k`$, which is an irreducible projective variety \[DM\]. Let $`\eta `$ be the generic point of $`\overline{_g}`$. The choice of a geometric point $`\overline{\eta }`$ over $`\eta `$ defines a smooth curve $`𝒳_{\overline{\eta }}`$ over $`\overline{k(\eta )}`$, the algebraic closure of the function field $`k(\eta )`$ of $`\overline{_g}`$. The curve $`𝒳_{\overline{\eta }}`$ is defined over a finite extension $`K`$ of $`k(\eta )`$ and we denote by $`𝒳_K`$ some model of $`𝒳_{\overline{\eta }}`$, i.e., $`𝒳_K\times _K\overline{k(\eta )}𝒳_{\overline{\eta }}`$.

The curve $`X/k`$ defines a $`k`$-rational point $`x`$ of $`\overline{_g}`$, which lies in the closure of $`\eta `$. The local ring $`A_x`$ at the generic point of the exceptional divisor of the blowing-up of $`\overline{_g}`$ at the point $`x`$ is a discrete valuation ring with fraction field $`k(\eta )`$ and residue field containing $`k`$. By the stable reduction theorem (Corollary 2.7 \[DM\]) there exists a finite extension $`L`$ of $`K`$, and therefore also of $`k(\eta )`$, such that $`𝒳_L`$ is the generic fibre of a stable curve $`𝒳`$ over the integral closure $`A`$ of $`A_x`$ in $`L`$. Note that $`A`$ is a discrete valuation ring with fraction field $`L`$ and with residue field, denoted by $`k(s)`$, containing $`k`$. Moreover the diagram

$`\mathrm{Spec}(A)`$
$``$ $`^𝒳`$
$`\mathrm{Spec}(A_x)`$ $``$ $`\overline{_g}`$
commutes when restricted to $`\mathrm{Spec}(L)\mathrm{Spec}(A)`$ and therefore commutes because $`\overline{_g}`$ is separated. It follows that the special point $`s\mathrm{Spec}(A)`$ maps to $`x`$, i.e., there exists an isomorphism $`X\times _k\overline{k(s)}𝒳_s\times _{k(s)}\overline{k(s)}`$.

In summary, we have constructed a stable curve $`𝒳`$ over a discrete valuation ring $`A`$ with generic fibre $`𝒳_L`$ and geometric special fibre isomorphic to $`X\times _k\overline{k(s)}`$. The fraction field of $`A`$ is $`L`$ and its residue field $`k(s)`$.

We now choose a tree of $`_k^1`$’s, denoted by $`X^{}`$, defining a closed point $`x^{}`$ in the boundary of $`\overline{_g}`$. Repeating the above contruction with $`x^{}`$ instead of $`x`$, we obtain a stable curve $`𝒳^{}`$ over a discrete valuation ring $`A^{}`$ with generic fibre $`𝒳_L^{}`$ and geometric special fibre isomorpic to $`X^{}\times _k\overline{k(s^{})}`$. The fraction field of $`A^{}`$ is $`L^{}`$, a finite extension of $`k(\eta )`$, and its residue field is $`k(s^{})`$. Moreover the isomorphism $`X^{}\times _k\overline{k(s^{})}𝒳_s^{}\times _{k(s^{})}\overline{k(s^{})}`$ is defined over a finite extension of $`k(s^{})`$.

We choose a finite extension of $`k(\eta )`$ containing both $`L`$ and $`L^{}`$, which we call again $`L`$, and take the integral closures in $`L`$ of $`A`$ and $`A^{}`$, which we call again $`A`$ and $`A^{}`$. Thus we have constructed two stable curves $`𝒳`$ and $`𝒳^{}`$ over $`A`$ and $`A^{}`$ such that $`𝒳_L𝒳_L^{}`$ and which specialize to $`X`$ and $`X^{}`$ respectively.

Let $`\widehat{L}`$ be the fraction field of the completion $`\widehat{A^{}}`$ of $`A^{}`$. By construction the curve $`𝒳_{\widehat{L}}𝒳_{\widehat{L}}^{}`$ is a Mumford-Tate curve and, by the main result of \[G\], there exists a stable rank $`2`$ vector bundle $`\widehat{}`$ over $`𝒳_{\widehat{L}}`$ such that $`F^{}\widehat{}`$ is not semi-stable.

###### 7.5 Lemma.

There exists a finite extension $`L_1`$ of $`L`$ contained in the field $`\widehat{L}`$ and a stable bundle $`_1`$ over $`𝒳_{L_1}`$ such that $`_1_{L_1}\widehat{L}\widehat{}`$ and $`F^{}_1`$ is not semi-stable.

###### Proof.

Let $`\widehat{\pi }:F^{}\widehat{}\widehat{}`$ be a maximal destabilizing quotient of $`\widehat{}`$. There exist a models $`_{k(S)},_{k(S)}`$ and $`\pi _{k(S)}`$ of $`\widehat{\pi }`$ over $`𝒳_{k(S)}`$, where $`k(S)`$ is an extension of finite type of $`L`$. The field $`k(S)`$ is the function field for some algebraic variety $`S`$ over $`L`$. Shrinking $`S`$ if necessary, one can assume that $`\pi _{k(S)}`$ comes from

$$\pi _S:F^{}_S_S,$$

where $`_S`$ is a family of stable bundles over $`𝒳_L`$ parametrized by $`S`$ (stability is an open condition). We now choose a closed point $`sS`$ and pull-back the family $`_S`$ under the inclusion $`sS`$. We thus obtain a stable bundle $`_{L_1}`$ over $`𝒳_{L_1}`$, where $`L_1`$ is the residue field at the point $`s`$, which is a finite extension of $`L`$. ∎

Again we take the integral closures $`A_1`$ and $`A_1^{}`$ of the discrete valuation rings $`A`$ and $`A^{}`$ in $`L_1`$. By the previous lemma we have a stable bundle $`_1`$ and a destabilizing quotient $`_1`$ over $`𝒳_{L_1}=𝒳_{L_1}^{}`$

$$\pi _{L_1}:F^{}_1_1.$$

After possibly taking a finite extension of $`L_1`$, we can assume \[La\] that $`_1`$ and $`_1`$ have models over $`𝒳\mathrm{Spec}(A)`$ with $`\left(_1\right)_{\overline{s}}`$ semi-stable over $`X\times _k\overline{k(s)}`$. By semi-continuity, we have

$$\mathrm{Hom}(F^{}_{1\overline{s}},_{\overline{s}})0,$$

which shows that $`F^{}_{1\overline{s}}`$ is not semi-stable.

Yves Laszlo

Université Pierre et Marie Curie, Case 82

Analyse Algébrique, UMR 7586

4, place Jussieu

75252 Paris Cedex 05 France

e-mail: laszlo@math.jussieu.fr