text:
# Supernova Neutrinos and the LSND Evidence for Neutrino Oscillations

## I INTRODUCTION

In recent years, the treatment of neutrino transport in the environment of a core-collapse supernova (SN) explosion has improved to the point of making realistic predictions on the observables for neutrinos reaching the Earth . Of particular interest for this paper are the average energies at the neutrinospheres, i.e. the surfaces of last scattering for the neutrinos, estimated to be $`1013`$ MeV for $`\nu _e`$, $`1417`$ MeV for $`\overline{\nu }_e`$, $`2327`$ MeV for $`\nu _{\mu ,\tau },\overline{\nu }_{\mu ,\tau }`$ .
The differences in temperatures between the various neutrino flavors can be qualitatively understood. Heavy-lepton neutrinos can interact only via neutral-current (NC) processes, the main contribution to their transport opacity coming from neutrino-nucleon scattering, which dominates over neutrino-electron scattering. In addition to this same NC contribution, the transport opacity for $`\nu _e`$’s and $`\overline{\nu }_e`$’s depends also on the charged-current (CC) absorptions $`\nu _e+np+e^{}`$ and $`\overline{\nu }_e+pn+e^+`$, respectively. Therefore, the $`\nu _e`$\- and $`\overline{\nu }_e`$-spheres are located at larger radii with respect to the other neutrinospheres, that is at lower densities and lower temperatures. Moreover, in a neutron-rich environment, $`\nu _e+np+e^{}`$ dominates over $`\overline{\nu }_e+pn+e^+`$: the emergent $`\nu _e`$’s originate from layers further outside the center of the star compared to $`\overline{\nu }_e`$’s, therefore at lower temperatures. The total energy released in a SN explosion is approximately equipartitioned between the different neutrino and antineutrino flavors .
The above predictions can be confronted with the observation of the supernova $`\overline{\nu }_e`$ energy spectrum detected on Earth. Neutrino oscillations are expected to modify the spectrum since $`E_{\overline{\nu }_e}<E_{\overline{\nu }_\mu ,\overline{\nu }_\tau }`$. The energy dependence of the neutrino cross-section in the detector material, approximately $`\sigma _{\overline{\nu }_ep}(E_{\overline{\nu }_e}1.29\text{MeV})^2`$ , helps in making the $`\overline{\nu }_e`$ energy spectrum distortion a sensitive experimental probe to neutrino oscillations. This is because higher energy neutrinos interact significantly more than lower energy ones.
We show that the extent of the spectrum modification depends crucially on the specifics of the neutrino mixing scheme and on the neutrino mass hierarchy under consideration, and we derive the relevant formulas assuming an adiabatic propagation for the antineutrinos in the supernova environment. Antineutrinos propagate adiabatically if the varying matter density they encounter changes slowly enough so that transitions between local (instantaneous) Hamiltonian eigenstates can be neglected throughout the entire antineutrino propagation.
So far, neutrinos from one supernova have been detected and their energy measured: SN1987A was observed by the Kamiokande-2 and IMB-3 detectors. The overall 20 events seen by those two detectors have all been interpreted as $`\overline{\nu }_e`$ interactions . We examine the constraint of such observations on the LSND allowed region of $`\overline{\nu }_\mu \overline{\nu }_e`$ oscillations , for various neutrino mass and mixing models. If the LSND evidence is confirmed by the MiniBooNE experiment , several models can be excluded or constrained on the basis of the observations of the supernova SN1987A and possibly future supernovæ.

## II ADIABATIC OSCILLATIONS AND NEUTRINO MIXING SCHEMES

### II.1 $`\overline{\nu }_e`$ energy spectrum and the permutation factor

In the presence of neutrino oscillations, the $`\overline{\nu }_e`$ flux reaching the Earth, $`F_{\overline{\nu }_e}`$, can be different from the primary flux at the neutrinosphere, $`F_{\overline{\nu }_e}^0`$. We will assume that, at production, the energy of active antineutrinos is equally divided into the three active flavors, i.e. that $`_0^{\mathrm{}}𝑑E_{\overline{\nu }_\alpha }E_{\overline{\nu }_\alpha }F_{\overline{\nu }_\alpha }^0`$ has the same numerical value for $`\alpha =e,\mu ,\tau `$. Moreover, we will also consider neutrino mixing models where the three active neutrino species are augmented by a fourth sterile neutrino with no standard weak couplings: in those cases, we will assume that the sterile component is negligible at production.
The neutrino flux reaching the Earth is:

$`F_{\overline{\nu }_e}=(p_{\mu e}+p_{\tau e})F_{\overline{\nu }_\mu }^0+p_{ee}F_{\overline{\nu }_e}^0`$
$`(pF_{\overline{\nu }_\mu }^0+(1p)F_{\overline{\nu }_e}^0)`$ (1)
where we have defined the permutation factor $`p`$ as:

$$p=\frac{p_{\mu e}+p_{\tau e}}{p_{\mu e}+p_{\tau e}+p_{ee}}$$
(2)
and $`p_{\mu ,\tau ,ee}`$ are the probabilities for a $`\overline{\nu }_\mu `$, $`\overline{\nu }_\tau `$, $`\overline{\nu }_e`$ respectively at the neutrinosphere to oscillate into a $`\overline{\nu }_e`$. In Eqs.1,2, we have assumed that $`p`$ is energy-independent (as will be justified later), and that $`E_{\overline{\nu }_\mu }=E_{\overline{\nu }_\tau }`$. In Eq. 1, we neglect the (energy-independent) proportionality factor since we will not deal with event rates, but only with neutrino energy distributions.

### II.2 Neutrino propagation in the adiabatic approximation

In vacuum, the Hamiltonian that governs neutrino propagation is diagonal in the mass eigenstate basis $`|\nu _i`$:

$$(H_0)_{ij}\nu _i|H_0|\nu _j=E_i\delta _{ij}$$
(3)
If the neutrinos all have the same relativistic momentum $`p`$, their energies $`E_i`$ differ only by a term proportional to their squared-mass differences, since $`E_ip+m_i^2/2p`$. If $`U`$ is the unitary mixing matrix that relates the flavor eigenstates $`|\nu _\alpha `$ to the mass eigenstates via $`|\nu _\alpha =U_{\alpha i}|\nu _i`$, the elements of the vacuum Hamiltonian in the flavor basis are given by :

$$(H_0)_{\alpha \beta }=U_{\alpha i}^{}U_{\beta i}\frac{m_i^2}{2p}$$
(4)
where we have neglected the contribution $`p\delta _{\alpha \beta }`$ in $`(H_0)_{\alpha \beta }`$, which is irrelevant for neutrino oscillations.
In matter, $`\overline{\nu }_e`$’s undergo coherent CC forward-scattering from electrons, and all active flavor antineutrinos coherent NC forward-scattering from electrons, protons, and neutrons in the medium. These processes give rise to an interaction potential $`V=V_W+V_Z`$, which is diagonal in the flavor basis and proportional to the matter density $`\rho `$:

$$(V)_{\alpha \beta }=A_\alpha \frac{G_F\rho }{m_N}\delta _{\alpha \beta }$$
(5)
where $`A_\alpha `$ is a proportionality constant, in general different for $`\alpha =e,\mu ,\tau ,\text{or }s`$, $`G_F`$ the Fermi constant, and $`m_N`$ the nucleon mass. The relevant Hamiltonian for neutrino propagation in matter is therefore $`HH_0+V`$.
At the neutrinosphere, the density $`\rho `$ is so high ($`10^{12}g/cm^3`$ ) that the interaction potential dominates over the vacuum Hamiltonian, so that the propagation eigenstates coincide with the flavor eigenstates. As the propagation eigenstates free-stream outwards, toward regions of lower density, their flavor composition changes, ultimately reaching the flavor composition of the mass eigenstates in the vacuum. Given that the neutrinos escape the SN as mass eigenstates, no further flavor oscillations occur on their path to the Earth.
More specifically, making use of the adiabatic approximation and of the fact that no energy-level crossing is permitted, the flavor eigenstate at the neutrinosphere with the maximum interaction potential reaches Earth as the mass eigenstate with the biggest neutrino mass. In general, the energy level order is maintained throughout the neutrino propagation in the SN ejecta.
This is illustrated in Tab. 1 for three neutrinos in the row labelled “Normal $`(1+1+1)`$”, where we have taken $`A_\gamma >A_\beta >A_\alpha `$ and $`m_3>m_2>m_1`$.

For example, the probability for a $`\overline{\nu }_\alpha `$ to emerge from the SN environment as a $`\overline{\nu }_\beta `$ is given by:

$`p_{\alpha \beta }=|\overline{\nu }_\beta |U^{evol}|\overline{\nu }_\alpha |^2=|U_{\beta i}\overline{\nu }_i|U^{evol}|\overline{\nu }_\alpha |^2=`$
$`|U_{\beta i}^{}\delta _{i,1}|^2=|U_{\beta 1}|^2`$ (6)
where $`U^{evol}`$ is the adiabatic evolution operator. In Eq.6, we have used Tab. 1 to get:

$$\overline{\nu }_i|U^{evol}|\overline{\nu }_\alpha =\delta _{i,1}$$
(7)
This result can be immediately generalized to any number of antineutrino generations. Also, as long as the adiabatic approximation is satisfied, the formula does not depend on the specific dynamics for the neutrino propagation, for example on the number and position in the SN environment of MSW-resonances. We will comment more on the validity of the adiabatic approximation in the next section.
In this paper, we consider three or four flavor components, including a sterile one. At tree-level, the proportionality factors $`A_\alpha `$ in the interaction potential for neutral matter are :

$$A=\{\begin{array}{cc}(13Y_e)/\sqrt{2},\hfill & \text{for }\overline{\nu }_e\hfill \\ (1Y_e)/\sqrt{2},\hfill & \text{for }\overline{\nu }_\mu ,\overline{\nu }_\tau \hfill \\ 0,\hfill & \text{for }\overline{\nu }_s\hfill \end{array}$$
(8)
where $`Y_e`$ is the electron fraction per nucleon. Following the assumptions of , we use $`Y_e(1+E_{\overline{\nu }_e}/E_{\nu _e})^1>1/3`$ at the neutrinosphere. Considering also one-loop electroweak radiative corrections, a difference in the $`\overline{\nu }_\mu `$ and $`\overline{\nu }_\tau `$ interaction potentials of magnitude $`(A_\mu A_\tau )/A_\mu 10^4`$ appears due to the difference in the charged lepton masses . At the neutrinosphere, this second-order effect in the interaction potential dominates over the vacuum Hamiltonian terms (as long as $`|m_i^2m_j^2|<10eV^2`$ for all $`i,j`$), and removes the $`\overline{\nu }_\mu \overline{\nu }_\tau `$ degeneracy. Therefore, for the antineutrino channel considered here, we take:

$$A_\mu >A_\tau >A_s>A_e$$
(9)
For the neutrino channel, one should substitute $`AA`$ in Eq.8, and the order in Eq.9 would be inverted.
Therefore, given a specific neutrino mass and mixing model, the permutation factor can be easily evaluated in the adiabatic approximation, and its numerical value does not depend on the neutrino energy. We will comment on possible energy-dependent Earth matter effects in the next section. In practice, one proceeds backwards: given a certain measured value of $`p`$, it is possible to constrain possible models for neutrino oscillations. This approach is used for example in  to constrain models explaining the solar and atmospheric neutrino data; in this paper, we focus on 3 and 4-neutrino models explaining the LSND data.

### II.3 Possible mixing schemes

The results for the $`\overline{\nu }_\mu `$,$`\overline{\nu }_\tau `$,$`\overline{\nu }_e\overline{\nu }_e`$ adiabatic oscillation probabilities, the permutation factor $`p`$, and the LSND oscillation amplitude $`\mathrm{sin}^22\vartheta `$ as a function of the mixing parameters and $`p`$ for the eight possible mass and mixing schemes considered below are given in Tab. 2. The mass hierarchy and the adiabatic propagation of the neutrino eigenstates for these mixing schemes are depicted in Tab. 1.

The simplest possible mixing scheme is a $`(1+1)`$ model explaining only $`\overline{\nu }_\mu \overline{\nu }_e`$ LSND oscillations in vacuum, and not the atmospheric or solar oscillations:

$$\left(\begin{array}{c}\overline{\nu }_e\hfill \\ \overline{\nu }_\mu \hfill \end{array}\right)=\left(\begin{array}{cc}\hfill \mathrm{cos}\vartheta & \hfill \mathrm{sin}\vartheta \\ \hfill \mathrm{sin}\vartheta & \hfill \mathrm{cos}\vartheta \end{array}\right)\left(\begin{array}{c}\overline{\nu }_1\hfill \\ \overline{\nu }_2\hfill \end{array}\right)$$
(10)
where the mixing angle $`\vartheta `$ can assume any value in the range $`0<\vartheta <\pi /4`$.
We consider a $`(2+1)`$ model motivated, for example, by CPT-violating scenarios (see, e.g. ), in which atmospheric and LSND oscillations in the antineutrino channel are obtained via the mixing :

$$\left(\begin{array}{c}\overline{\nu }_e\hfill \\ \overline{\nu }_\mu \hfill \\ \overline{\nu }_\tau \hfill \end{array}\right)=\left(\begin{array}{ccc}1& \frac{1}{2}\alpha & \frac{\sqrt{3}}{2}\alpha \\ \alpha & \frac{1}{2}& \frac{\sqrt{3}}{2}\\ 0& \frac{\sqrt{3}}{2}& \frac{1}{2}\end{array}\right)\left(\begin{array}{c}\overline{\nu }_1\hfill \\ \overline{\nu }_2\hfill \\ \overline{\nu }_3\hfill \end{array}\right)$$
(11)
The matrix in Eq.11 is chosen to ensure large $`\overline{\nu }_\mu \overline{\nu }_\tau `$ mixing for atmospheric neutrinos ($`\mathrm{sin}^22\vartheta _{atm}=3/4`$), while the LSND $`\overline{\nu }_\mu \overline{\nu }_e`$ mixing is fixed by the parameter $`\alpha `$ ($`\mathrm{sin}^22\vartheta _{LSND}=4\alpha ^2`$).
The most popular models which explain the solar, atmospheric and LSND signatures (and the null results obtained by other experiments) via neutrino oscillations invoke the existence of a sterile neutrino $`\overline{\nu }_s`$. One example of a $`(2+2)`$ model is the following, which is taken from :

$$\left(\begin{array}{c}\overline{\nu }_s\hfill \\ \overline{\nu }_e\hfill \\ \overline{\nu }_\mu \hfill \\ \overline{\nu }_\tau \hfill \end{array}\right)=\left(\begin{array}{cccc}\frac{1}{\sqrt{2}}& \frac{1}{\sqrt{2}}& 0& 0\\ \frac{1}{\sqrt{2}}& \frac{1}{\sqrt{2}}& \beta & \beta \\ \beta & \beta & \frac{1}{\sqrt{2}}& \frac{1}{\sqrt{2}}\\ 0& 0& \frac{1}{\sqrt{2}}& \frac{1}{\sqrt{2}}\end{array}\right)\left(\begin{array}{c}\overline{\nu }_0\hfill \\ \overline{\nu }_1\hfill \\ \overline{\nu }_2\hfill \\ \overline{\nu }_3\hfill \end{array}\right)$$
(12)
where one pair of nearly degenerate mass eigenstates has maximal $`\nu _e\nu _s`$ mixing for solar neutrinos and the other pair has maximal $`\nu _\mu \nu _\tau `$ mixing for atmospheric neutrinos. Small inter-doublet mixings through the $`\beta `$ parameter accomodates the LSND result ($`\mathrm{sin}^2\vartheta _{LSND}=8\beta ^2`$).
Recent experimental results  show that pure $`\nu _e\nu _s`$ solar oscillations are excluded at high significance. We therefore consider a more general $`(2+2)`$ scenario, in which solar neutrinos can undergo any combination of $`\nu _e\nu _s`$ and $`\nu _e\nu _\tau `$ oscillations, while atmospheric neutrinos can undergo any combination of $`\nu _\mu \nu _\tau `$ and $`\nu _\mu \nu _s`$ oscillations. We follow the procedure in  to obtain this more general mixing starting from Eq.12, by substituting the $`(\overline{\nu }_s,\overline{\nu }_\tau )`$ states with the rotated states $`(\overline{\nu }_s^{},\overline{\nu }_\tau ^{})`$:

$$\left(\begin{array}{c}\overline{\nu }_s^{}\hfill \\ \overline{\nu }_\tau ^{}\hfill \end{array}\right)=\left(\begin{array}{cc}\hfill \mathrm{cos}\phi _s& \hfill \mathrm{sin}\phi _s\\ \hfill \mathrm{sin}\phi _s& \hfill \mathrm{cos}\phi _s\end{array}\right)\left(\begin{array}{c}\overline{\nu }_s\hfill \\ \overline{\nu }_\tau \hfill \end{array}\right)$$
(13)
where the rotation angle $`\phi _s`$ fixes the sterile component in the atmospheric doublet ($`0<\phi _s<\pi /2`$). Eq.12 then becomes:

$$\left(\begin{array}{c}\overline{\nu }_s\hfill \\ \overline{\nu }_e\hfill \\ \overline{\nu }_\mu \hfill \\ \overline{\nu }_\tau \hfill \end{array}\right)=\left(\begin{array}{cccc}\frac{\mathrm{cos}\phi _s}{\sqrt{2}}& \frac{\mathrm{cos}\phi _s}{\sqrt{2}}& \frac{\mathrm{sin}\phi _s}{\sqrt{2}}& \frac{\mathrm{sin}\phi _s}{\sqrt{2}}\\ \frac{1}{\sqrt{2}}& \frac{1}{\sqrt{2}}& \beta & \beta \\ \beta & \beta & \frac{1}{\sqrt{2}}& \frac{1}{\sqrt{2}}\\ \frac{\mathrm{sin}\phi _s}{\sqrt{2}}& \frac{\mathrm{sin}\phi _s}{\sqrt{2}}& \frac{\mathrm{cos}\phi _s}{\sqrt{2}}& \frac{\mathrm{cos}\phi _s}{\sqrt{2}}\end{array}\right)\left(\begin{array}{c}\overline{\nu }_0\hfill \\ \overline{\nu }_1\hfill \\ \overline{\nu }_2\hfill \\ \overline{\nu }_3\hfill \end{array}\right)$$
(14)
which contains Eq.12 in the specific case $`\phi _s=0`$. We note that the LSND oscillation amplitude formula $`\mathrm{sin}^22\vartheta _{LSND}=8\beta ^2`$ holds also for the more general case of Eq.14, and that our results are independent of the value of $`\phi _s`$ (see Tab. 2).
Another possible 4-neutrino model has a $`(3+1)`$ hierarchy; as an example for this model, here we consider the following mixing, which is also taken from :

$$\left(\begin{array}{c}\overline{\nu }_e\hfill \\ \overline{\nu }_\mu \hfill \\ \overline{\nu }_\tau \hfill \\ \overline{\nu }_s\hfill \end{array}\right)=\left(\begin{array}{cccc}\frac{1}{\sqrt{2}}& \frac{1}{\sqrt{2}}& 0& \gamma \\ \frac{1}{2}& \frac{1}{2}& \frac{1}{\sqrt{2}}& \delta \\ \frac{1}{2}& \frac{1}{2}& \frac{1}{\sqrt{2}}& 0\\ \frac{1}{2}\delta \frac{1}{\sqrt{2}}\gamma & \frac{1}{2}\delta \frac{1}{\sqrt{2}}\gamma & \frac{1}{\sqrt{2}}\delta & 1\end{array}\right)\left(\begin{array}{c}\overline{\nu }_1\hfill \\ \overline{\nu }_2\hfill \\ \overline{\nu }_3\hfill \\ \overline{\nu }_4\hfill \end{array}\right)$$
(15)
where the solar and atmospheric oscillations are approximately described by oscillations of three active neutrinos, and the LSND result by a coupling of $`\overline{\nu }_\mu `$ and $`\overline{\nu }_e`$ through small mixings with $`\overline{\nu }_s`$ that has a mass eigenvalue widely separated from the others ($`\mathrm{sin}^22\vartheta _{LSND}=4\gamma ^2\delta ^2`$). For the $`(3+1)`$ scenario, the constraint given by the permutation probability $`p`$ is not sufficient to determine the LSND oscillation amplitude $`\mathrm{sin}^22\vartheta _{LSND}`$. Therefore, the constraint on $`|U_{\mu 4}|^2=\delta ^2`$ given by the CDHS and Super-K experiments will also be used, as explained later.
We should note that the mixing matrices defined in Eqs. 10\- 15 are approximations in the sense that the matrices are unitary only up to order $`𝒪(\alpha ,\beta ,\gamma ,\delta )`$. These are the parameters in the mixings responsible for LSND-type oscillations, which we let float for our analysis, but we know they are small.

In order to determine the permutation factor for the mixing models, we also need to specify the neutrino mass hierarchy. In this paper, we consider for each mixing model both the cases of a “normal” and a “LSND-inverted” mass hierarchies. By “normal” hierarchy, here we mean that $`m_i>m_j`$ for $`i>j`$, where $`m_i`$ is the mass eigenvalue for the $`|\overline{\nu }_i`$ state. We define the “LSND-inverted” hierarchies as the ones obtained substituting $`\mathrm{\Delta }m_{LSND}\mathrm{\Delta }m_{LSND}`$ in the normal hierarchies, without changing the hierarchy of the eventual solar and atmospheric splittings (see Tab. 1); $`\mathrm{\Delta }m_{LSND}`$ is the neutrino mass difference responsible for LSND oscillations.
A common feature to all the mixing schemes is apparent in Tab. 2. In the adiabatic approximation, normal mass hierarchies predict small permutation factors, while an almost complete permutation would be present for LSND-inverted hierarchies.
Given the specific neutrino mixing models considered here, we can now partially address the question whether the adiabatic approximation is applicable in this context. At a resonance, where the non-adiabaticity is maximal, this is a good approximation if the width of the resonance region is large compared with the local neutrino oscillation length. The width of the resonance is, in turn, determined by the characteristic length scale of the radial matter density variations at the resonance. While there are reliable models for the matter density profile of the progenitor star, there are still uncertainties on the profile seen by neutrinos in their free-streaming propagation.
It is now thought that neutrino heating of the proto-neutron star mantle drives the supernova explosion, which would happen with a $`1s`$ delay after the creation of the shock-wave, ultimately responsible for the explosion; during this delay, the shock-wave would be stalled at a radius of $`200km`$ from the neutron star, corresponding to a density $`\rho 10^910^{10}g/cm^3`$ . Therefore, the density profile in the proximity of the stalled shock-wave, which is difficult to model reliably, is a potential site for non-adiabatic oscillations.
In Fig. 1 we show the energy splittings between the local neutrino energy eigenvalues $`E_i`$, as a function of matter density, for all eight neutrino models considered here. For an $`n`$-neutrino model, we plot $`E_{i,i+1}E_iE_{i+1}`$, where $`i=1,\mathrm{},n1`$; the eigenvalues are ordered such that $`E_1>E_2>\mathrm{}>E_n`$. Clearly, a resonance corresponds to a local minimum in one of the curves. As can be seen from Fig. 1, all the resonances (except the inconsequential one in Fig.1f between $`\overline{\nu }_\mu `$ and $`\overline{\nu }_\tau `$ ) lie at densities well below the stalled shock-wave density of $`\rho 10^910^{10}g/cm^3`$. Therefore, the impact of level crossing between propagation eigenstates is likely to be small even where the neutrinos encounter the shock-wave.
If the SN neutrinos cross the Earth on their way to the detector, as for example happened for the SN1987A $`\overline{\nu }_e`$’s detected by the Kamiokande-2 and IMB-3 detectors, it is also necessary to evaluate the importance of Earth matter effects in the neutrino propagation. Clearly, for neutrino oscillation models where no solar splitting is involved (for example the $`(1+1)`$ and $`(2+1)`$ models in this paper), this effect is negligible. In the models where such a splitting is allowed (i.e. the $`(2+2)`$ and $`(3+1)`$ models considered here), the situation is more complicated. However, the Earth matter effects have been shown to be small in this case as well for a large fraction of the SN $`\overline{\nu }_e`$ energy spectrum (below $`40`$ MeV) , and for the sake of simplicity will not be considered further.

## III CONSTRAINTS ON LSND FROM SN1987A OBSERVATIONS

Twenty $`\overline{\nu }_e`$ events from the supernova SN1987A were observed by the Kamiokande-2 (Kam-2) and IMB-3 detectors. Kam-2 saw 12 events with an average energy of $`E_{det}=14.7`$ MeV, IMB-3 (which had a higher energy threshold than Kam-2) detected 8 events with $`E_{det}=31.9`$ MeV .
From a comparison of the measured energy spectra ($`F_{\overline{\nu }_e}`$) with theoretical models of neutrino emission ($`F_{\overline{\nu }_e}^0`$ and $`F_{\overline{\nu }_\mu }^0`$), it is possible to infer the permutation factor $`p`$ in Eq. 1. SN1987A observations are consistent with no-oscillations (i.e. $`p=0`$). In Appendix A, we derive a conservative upper bound on $`p`$ of $`p<0.22`$ at $`99\%`$ CL, by applying a Kholmogorov-Smirnov test on the joint Kam-IMB dataset and a range of supernova neutrino emission models.
One important result of our analysis is immediately apparent from the values of the permutation factors $`p`$ as a function of the mixing parameters in Tab. 2, and from the fact that the value of $`p`$ inferred from SN1987A data has to be less than $`0.22`$ at $`99\%`$ CL. The four mixing schemes considered, explaining the LSND effect via a LSND-inverted neutrino mass hierarchy, are all incompatible with SN1987A data.
We now consider the normal hierarchy cases. For the $`(1+1)`$, $`(2+1)`$ and $`(2+2)`$ models with the mixings of Eqs. 10-12, the bound on the permutation factor $`p`$ unambigously determines the constraint on the LSND oscillation amplitude $`\mathrm{sin}^22\vartheta _{LSND}`$ (see Tab. 2). At $`99\%`$ CL, SN1987A data provide no constraints on the $`(2+1)`$ model, and a constraint which is weaker than existing bounds from the accelerator experiment Karmen  and the reactor experiment Bugey  for the $`(1+1)`$ and $`(2+2)`$ models (see Fig. 2a). Therefore, these models are compatible with the SN1987A data.
As already mentioned, for the $`(3+1)`$ model, the permutation factor does not fully determine the LSND oscillation amplitude: $`\mathrm{sin}^22\vartheta _{LSND}`$ depends not only on $`p`$, but also on $`|U_{\mu 4}|^2=\delta ^2`$. Here we use the $`\mathrm{\Delta }m_{LSND}^2`$-dependent constraints on $`\delta ^2`$ from the $`\nu _\mu `$-disappearance experiments CDHS (for $`\mathrm{\Delta }m_{LSND}^2>0.3eV^2`$) and Super-K (for $`\mathrm{\Delta }m_{LSND}^2<0.3eV^2`$) . Moreover, another complication arises in evaluating exclusion regions for (3+1) models: given the $`99\%`$ CL upper bounds on $`\gamma ^2=|U_{e4}|^2`$ from SN1987A and $`\delta ^2=|U_{\mu 4}^2|`$ from CDHS and Super-K, what is the $`99\%`$ CL upper bound on $`\mathrm{sin}^22\vartheta _{LSND}=4\gamma ^2\delta ^2`$? We follow the method described in  to estimate this bound. The same method is applied to estimate the $`99\%`$ CL upper limit on $`\mathrm{sin}^22\vartheta _{LSND}`$ coming from Bugey (for $`\gamma ^2`$) and CDHS and Super-K (for $`\delta ^2`$), that is without using the SN1987A data. The results for the $`(3+1)`$ model with normal neutrino mass hierarchy and mixing given by Eq. 15 is shown in Fig. 2b. Also for this model, we find that existing constraints (the Bugey constraint on $`\delta ^2`$, in this case) are stronger than the SN1987A one.
Tab. 3 summarizes the SN1987A constraints obtained in this paper on the LSND allowed region, for the various neutrino mass and mixing models considered.

## IV CONCLUSIONS

We have investigated the effect that 3- and 4-neutrino oscillation schemes would have in modifying the energy spectrum of supernova $`\overline{\nu }_e`$’s. Throughout the paper, we apply the adiabatic approximation for the antineutrino propagation in the supernova environment and neglect Earth matter effects. Moreover, we have used our results to test the compatibility between the SN1987A data and the LSND evidence for $`\overline{\nu }_\mu \overline{\nu }_e`$ oscillations.
We have provided specific relations for the permutation factor, which gives the admixture of a higher energy flux to the original $`\overline{\nu }_e`$ flux at production from $`\overline{\nu }_\mu ,\overline{\nu }_\tau \overline{\nu }_e`$ oscillations, for various neutrino mass and mixing models. The permutation factor may be measurable with good accuracy in future supernova experiments.
Based on SN1987A data only, which seem to indicate a small (if non-zero) value for the permutation factor, we are able to exclude all of the four models considered which would explain the LSND signal via a “LSND-inverted” neutrino mass hierarchy, as defined in the text. For the normal mass hierarchy schemes considered, SN1987A data do not provide any stronger constraints on the LSND allowed region for oscillations than those already obtained with reactor, accelerator and atmospheric neutrinos; additional experimental input is necessary to unambiguously discern the neutrino mass and mixing properties. Undoubtedly, the detection of supernova neutrinos by present or near-term experiments would prove very useful in this respect.

###### Acknowledgements.

We thank Kevork Abazajian, Vernon Barger, John Beacom, Nicole Bell, Steve Brice, Klaus Eitel, Lam Hui, Hitoshi Murayama and Michael Shaevitz for valuable discussions and useful suggestions. This work was supported by NSF and by the Sloan Foundation.

## Appendix A UPPER BOUNDS ON THE PERMUTATION FACTOR FROM SN1987A DATA

In this appendix, we discuss the statistical methodology and the physics assumptions used to estimate the upper bound on the permutation factor $`p`$ quoted in the text, $`p<0.22`$ at $`99\%`$ CL. We use the same statistical methodology as in , that is we use the Kholmogorov-Smirnov test on the joint Kam-IMB dataset to derive the upper bound. Most of the physics assumptions are identical to those in .
The expected energy spectrum for the positrons, observed in the Kamiokande and IMB detectors via the reaction $`\overline{\nu }_epe^+n`$, is:

$$n_i(E_{det})=\frac{N_{p,i}}{4\pi D^2}_0^{\mathrm{}}𝑑E_+P_i(E_{det},E_+)\eta _{0,i}(E_+)\sigma _{\overline{\nu }_ep}(E_++Q)F_{\overline{\nu }_e}(E_++Q)$$
(16)
where $`i`$ refers to either Kam or IMB, $`N_{p,i}`$ is the number of target protons in the detectors, $`D`$ the distance between the Large Magellanic Cloud and the Earth, $`E_{det}`$ ($`E_+`$) is the detected (true) positron energy, $`Qm_nm_p=1.29\text{MeV}E_{\overline{\nu }_e}E_+`$, $`P_i(E_{det},E_+)`$ and $`\eta _{0,i}(E_+)`$ the energy resolution functions and efficiency curves taken from , $`\sigma _{\overline{\nu }_ep}(E_++Q)E_+^2`$ the neutrino interaction cross-section taken from  (neglecting nuclear recoil), and finally $`F_{\overline{\nu }_e}(E_++Q)`$ the neutrino flux at the detector taken from Eq.1. We assume “unpinched” Fermi-Dirac distributions for the fluxes $`F_{\overline{\nu }_\alpha }^0`$, $`\alpha =e,\mu `$, appearing in Eq.1:

$$F_{\overline{\nu }_\alpha }^0(E)\frac{E^2}{E_{\overline{\nu }_\alpha }T_\alpha ^3(e^{E/T_\alpha }+1)}$$
(17)
where $`E_{\overline{\nu }_\alpha }3.15T_\alpha `$ at the denominator ensures energy equipartition.
The cumulative distribution function used for the Kholmogorov-Smirnov test is:

$$(E_{det})=_0^{E_{det}}𝑑E(n_{Kam}(E)+n_{IMB}(E))$$
(18)
Fig.3 shows the upper bound on the permutation factor $`p`$ obtained from SN1987A data, at $`99\%`$ CL, as a function of the average energies $`E_{\overline{\nu }_e}`$, $`E_{\overline{\nu }_\mu }`$. As expected, the bound becomes more stringent for supernova models in which the neutrino average energies are higher. SN1987A data are incompatible at $`99\%`$ CL with all supernova neutrino models predicting $`E_{\overline{\nu }_e}>16.6`$ MeV, for all values of $`p`$ and $`E_{\overline{\nu }_\mu }`$. We adopt a conservative approach, and quote as the upper bound on $`p`$ the largest value for supernova neutrino models in the range $`14<E_{\overline{\nu }_e}<17`$ MeV, $`23<E_{\overline{\nu }_\mu }<27`$ MeV, that is the one corresponding to $`E_{\overline{\nu }_e}=14`$ MeV, $`E_{\overline{\nu }_\mu }=23`$ MeV (cross in Fig.3).


format: text

text:
# Strong enhancement of current, efficiency and mass separation in Brownian motors driven by non Gaussian noises

(December 14th, 2001)

## Abstract

We study a Brownian motor driven by a colored non Gaussian noise source with a $`q`$-dependent probability distribution, where $`q`$ is a parameter indicating the departure from Gaussianity. For $`q=1`$ the noise is Gaussian (Ornstein–Uhlenbeck), while, for $`q>1`$, the probability distribution falls like a power law. In the latter case, we find a marked enhancement of both the current and the efficiency of the Brownian motor in the overdamped regime. We also analyze the case with inertia and show that, again for $`q>1`$, a remarkable increase of the ratchet’s mass separation capability is obtained.

The study of noise induced transport by ”ratchets” has attracted an increasing number of researchers that have produced a vast literature . Among other aspects, the motivation of these studies has been prompted by both their possible biological interest  as well as their potential technological applications. Since the initial works, besides the built-in ratchet-like bias and correlated fluctuations (see for instance ), different aspects have been studied like tilting  and pulsating  potentials, velocity inversions , etc. There are some reviews where it is possible to grasp the state of the art .

So far, almost all studies have used Gaussian noises, with few exceptions that mainly exploited dichotomic processes . Here we analyze the effect of a particular class of colored non Gaussian noise on the transport properties of Brownian motors. Such a noise source is based on the so called Tsallis statistics  with a probability distribution that depends on $`q`$, a parameter indicating the departure from the Gaussian ($`q=1`$) behavior. Some of the motivations for studying the effect of non Gaussian noises are, in addition to the intrinsic interest within the realm of noise induced phenomena, different experimental data indicating that for some biologically motivated systems fluctuations can have a non Gaussian character. An example are current measurements through voltage-sensitive ion channels in a cell membrane or experiments on the sensory system of rat skin . It is also worth remarking here that recent detailed studies on the source of fluctuations in some biological systems  clearly indicate that noise sources in general could be non Gaussian and their distribution bounded. Here, again as a consequence of the non Gaussian character of the driven noise, we find a remarkable increase of the current together with an enhancement of the motor efficiency that, additionally, shows an optimum for a given degree of non Gaussianity. Moreover, when the inertia is taken into account we find that, when departing from the Gaussian case, there is a remarkable increment in the mass separation efficiency of these devices.

We start considering the general system

$$m\frac{d^2x}{dt^2}=\gamma \frac{dx}{dt}V^{}(x)F+\xi (t)+\eta (t),$$
(1)
where $`m`$ is the mass of the particle, $`\gamma `$ the friction constant, $`V(x)`$ the ratchet potential, $`F`$ is a constant “load” force, and $`\xi (t)`$ the thermal noise satisfying $`\xi (t)\xi (t^{})=2\gamma KT\delta (tt^{})`$. Finally, $`\eta (t)`$ is the time correlated forcing (with zero mean) that keeps the system out of thermal equilibrium allowing the rectification of the motion. For this type of ratchet model several different kinds of time correlated forcing have been considered in the literature . We may distinguish the cases of deterministic (such as the periodic forcing and the “synthetic noise” discussed in the pioneering work of Magnasco ) and of stochastic forcing. Among the stochastic forcings the more usual noise sources are dichotomic and Ornstein–Uhlenbeck (OU) noises . Here, as indicated above, we will study the effects of a different family of stochastic forcing having non Gaussian statistics. The main characteristic introduced by the non Gaussian form of the forcing we consider here is the appearance of arbitrary strong “kicks” with relatively high probability when compared, for example, with the OU Gaussian process. As we shall see, in a general situation (without fine tuning of the parameters), this leads to the above indicated remarkable effects.

We will consider the dynamics of $`\eta (t)`$ as described by the Langevin equation

$$\frac{d\eta }{dt}=\frac{1}{\tau }\frac{d}{d\eta }V_q(\eta )+\frac{1}{\tau }\zeta (t),$$
(2)
with $`\zeta (t)\zeta (t)^{}=2D\delta (tt^{})`$ and

$$V_q(\eta )=\frac{D}{\tau (q1)}\mathrm{ln}[1+\frac{\tau }{D}(q1)\frac{\eta ^2}{2}].$$
(3)
Previous studies of such processes in connection with stochastic resonance problems  and dynamical trapping  have shown that the non Gaussianity of the noise leads to interesting effects. For $`q=1`$, the process $`\eta `$ is the (Gaussian) Ornstein–Uhlenbeck one (with correlation time equal to $`\tau `$), while for $`q1`$ it is a non Gaussian process. For $`q<1`$ the probability distribution has a cut–off at $`\omega =[(1q)\tau /(2D)]^{\frac{1}{2}}`$, and is given by:

$$P_{q<1}(\eta )=\{\begin{array}{cc}\frac{1}{Z_q}[1(\frac{\eta }{\omega })^2]^{\frac{1}{1q}}& \mathrm{if}|\eta |<\omega .\\ 0& \mathrm{otherwise}\end{array}$$
(4)
where $`Z_q`$ is the normalization constant. For $`1<q<3`$, the probability distribution (for $`\mathrm{}<\eta <\mathrm{}`$) is

$$P_{q>1}(\eta )=\frac{1}{Z_q}\left[1+\frac{\tau }{2D}(q1)\eta ^2\right]^{\frac{1}{1q}}.$$
(5)
While keeping $`D`$ constant, the width or dispersion of the distribution increases with $`q`$. This means that, the higher the $`q`$, the stronger the “kicks” that the particle will receive. The second moment of the distribution (or intensity of the non Gaussian noise, that we call $`D_{ng}`$) diverges for $`q5/31.66`$ while, for $`q5/3`$, is given by

$$D_{ng}\eta =\frac{2D}{\tau (53q)}.$$
(6)
Here, and in what follows, we shall keep $`q<5/3`$.

For the ratchet potential we will first consider the same form as in  (with period $`2\pi `$)

$$V(x)=𝑑x(\mathrm{exp}[\alpha \mathrm{cos}(x)]/J_0(i\alpha )1),$$
(7)
with $`\alpha =16`$. The integrand is the ratchet force (-V’(x)) appearing in Eq.(1).

Firstly, we will analyze the overdamped regime setting $`m=0`$ and $`\gamma =1`$. We are interested on analyzing the dependence of the mean current $`J=\frac{dx}{dt}`$ and the efficiency $`\epsilon `$ on the different parameters. In particular, their dependence on $`q`$, the parameter indicating the degree of non Gaussianity of the noise distribution. The efficiency of the ratchet system is defined as the ratio of the work (per unit time) done by the particle “against” the load force $`F`$, into the mean power injected to the system through the external forcing $`\eta `$ .

$$\epsilon =\frac{\frac{1}{T_f}_{x=x(0)}^{x=x(T_f)}F𝑑x(t)}{\frac{1}{T_f}_{x=x(0)}^{x=x(T_f)}\eta (t)𝑑x(t)}$$
(8)
For the numerator we get $`F\frac{dx}{dt}=FJ`$, while, for the denominator

$$\frac{1}{T_f}_0^{T_f}\eta (t)\frac{dx}{dt}𝑑t=\frac{1}{\gamma T_f}_0^{T_f}\eta (t)^2𝑑t=\frac{2D}{\gamma T_f\tau (53q)}.$$
(9)
Interesting and complete discussions on the thermodynamics and energetics of ratchet systems can be found in .

It is worth mentioning here that the “effective Markovian approximation” introduced in  is not adequate for the present case as it only properly works in the low $`\tau `$ limit. Here, in the overdamped regime we are able to give an approximate analytical solution for the problem, which is expected to be valid in the large correlation time regime ($`\frac{\tau }{D}>>1`$): we perform the adiabatic approximation of solving the Fokker-Planck equation associated to Eq. (1) assuming a constant value of $`\eta `$ . This leads us to obtain a value for the current $`J(\eta )`$ which depends on $`\eta `$ and, in order to get the final result, we should perform an average of $`J(\eta )`$ over $`\eta `$ using the distribution $`P_q(\eta )`$ with the desired value of $`q`$.

In Fig. 1, we show typical analytical results for the current and the efficiency as functions of $`q`$ together with results coming from numerical simulations (for the complete system given by Eqs.(1) and (2)). Calculations have been done in a region of parameters similar to the one studied in  but considering (apart from the difference provided by the non Gaussian noise) a non–zero load force that leads to a non–vanishing efficiency. As can be seen, although there is not a quantitative agreement between theory and simulations, the adiabatic approximation predicts qualitatively very well the behavior of $`J`$ (and $`\epsilon `$) as $`q`$ is varied. As shown in the figure, the current grows monotonously with $`q`$ (at least for $`q<5/3`$) while there is an optimal value of $`q`$ ($`>1`$) which gives the maximum efficiency. This fact is interpreted as follows: when $`q`$ is increased, the width of the $`P_q(\eta )`$ distribution grows and high values of the non Gaussian noise become more frequent, this leads to an improvement of the current. Although the mean value of $`J`$ increases monotonously with $`q`$, the grow of the width of $`P_q(\eta )`$ leads to an enhancement of the fluctuations around this mean value. This is the origin of the efficiency’s decay that occurs for high values of $`q`$: in this region, in spite of having a large (positive) mean value of the current, for a given realization of the process, the transport of the particle towards the desired direction is far from being assured.

In Fig. 2 we show results from simulations for $`J`$ and $`\epsilon `$ as functions of $`q`$ for different values of $`D`$, the intensity of the white noise in Eq. (2). The results correspond to $`KT=0`$, hence, the only noise present in the system is the non Gaussian one. On the curve corresponding to the results for $`J`$ (Fig. 2.a.), we indicate with error bars the dispersion of the results. The huge growth of the dispersion occurring for $`q>1.3`$, as well as the decay of the efficiency for the same values of $`q`$, is apparent.

It is worth recalling here that $`q=1`$ corresponds to the Gaussian OU noise (analyzed, for example in  and ). Hence, our results show that the transport mechanism becomes more efficient when the stochastic forcing has a non Gaussian distribution with $`q>1`$.

In the previous calculations, we have analyzed the values of $`J`$ and $`\epsilon `$ as functions of $`q`$ for fixed values of $`D`$. However, there are situations where we should consider that the non Gaussian noise source is the “primary” source, for instance see  for biologically motivated problems. Besides this, there could also be situations of clear technological interest. For these reasons we wondered which would be the dependence of $`J`$ and $`\epsilon `$ on $`q`$ when we fix the intensity of the non Gaussian noise $`D_{ng}`$ defined in Eq.(6). In such case, the energy per unit time supplied to the ratchet is independent of $`q`$. Our results (not shown) indicate that, at variance to what occurs in calculations for fixed $`D`$, for constant $`D_{ng}`$, the optimum value of $`q`$ which maximizes the efficiency also maximizes the mean current $`J`$, which decays for larger values of $`q`$. This result is easy to understand by observing that the width of the $`P_q(\eta )`$ distribution decreases with $`q`$ when $`D_{ng}`$ is kept constant.

Now we turn to study the $`m0`$ case, that is, the situations in which the inertia effects are relevant. In Fig. 3 we show the dependence of the current $`J`$ on the mass $`m`$ for different values of $`q`$. The results are from simulations for zero temperature and without load force. It can be seen that, as $`m`$ is increased from $`0`$, the inertial effects initially contribute to increase the current, until an optimal value of $`m`$ is reached. As it is expected, for high values of $`m`$, the motion of the particle becomes difficult and, for $`m\mathrm{}`$, the current vanishes.

An interesting effect appears for $`q=1.3`$: for a well defined interval of the value of the mass (ranging approximately from $`m=100`$ to $`m=5000`$), a negative current is observed. This can be explained as a consequence of the high value of the mass, that makes the inertial effects much more important than those of the ratchet potential. Due to the high value of $`m`$, most of the time the particle will be almost at rest around the minimum of the ratchet potential. However, a very high value of $`\eta `$ may occur (that may last for an interval of time of the order of $`\tau `$) causing the particle to move a period of the potential to the right or to the left. The obvious question is, what is the value of the force $`\eta `$ needed in order to make the particle jump a period of the potential to the left (right), during a time $`\tau `$?. Assuming no friction (which is of no importance for high values of $`m`$), a constant (average) value of the ratchet force $`V^{}`$, and performing a classical calculation we get:

$$\eta =|V^{}|+\frac{2md}{\tau ^2},$$
(10)
where $`d`$ is the distance from one minimum of $`V(x)`$ to the next left (right) maximum of $`V(x)`$. In our system, for a jump to the left, we have $`|V^{}|4.5`$ and $`d1.1`$, while, for a jump to the right, we have $`|V^{}|.96`$ and $`d5.2`$. Considering $`\tau =100/(2\pi )`$, for $`m=800`$ (the value for which in Fig. 3 the minimum of the current is obtained for $`q=1.3`$) we find that the required value of $`\eta `$ for a jump to the left (right) is $`|\eta _l|6.9`$ ($`|\eta _r|32.4`$). Hence, for this value of the mass, the jumps to the left are more probable than the jumps to the right, and a negative current is to be expected. In contrast, for a value of the mass $`m=100`$ (approximately where the current change from being positive to being negative) we find $`|\eta _l|5.4`$ and $`|\eta _r|5.1`$, i.e. both jumps are almost equiprobable. Note that, for $`m=800`$, the large value of $`m`$ makes the second term in Eq. (10) more important than the contribution of $`|V^{}|`$ (the inertial effect dominates over the ratchet force), hence, the difference in the value of $`\eta `$ necessary for a jump to the left or to the right comes, essentially, from the difference on the distances ($`d`$) that the heavy particle should be displaced to each side.

Until this point, the analysis makes no mention of the value of $`q`$ of the noise distribution. The fact that the $`(J<0)`$–effect appears for $`q=1.3`$ and not for $`q=1`$ is understood as a consequence of the fact that, for $`q=1`$, the occurrence of a value of noise $`|\eta _L|`$ is highly improbable in the regime where the inertia dominates over the effect of the ratchet force. Hence, the particle remains essentially motionless.

These results imply that separation of masses (particles with different masses moving in opposite directions) occurs, and that this happens in the absence of load force, due to the “big help” given by the non Gaussian noise. To our knowledge, separation of masses by ratchets with zero load force have not being reported before. However, the separation found here occurs for particles with a ratio of masses of the order of $`10`$ or more (say $`800/80`$ in Fig. 3.), while it has been shown in  that, with a load force and considering simply OU noise, particles of much closer masses can be separated by ratchets.

Now we analyze in more detail the problem of mass separation, an aspect that has been studied in some works . In view of the results discussed above, it is reasonable to expect that non Gaussian noises may improve the capability of mass separation in ratchets in more general situations. Reference  was one of the primary works discussing mass separation by ratchets. There, the authors analyzed a ratchet system like the one described by Eq. (1) considering OU noise as external forcing (in our case it corresponds to $`q=1`$). They studied (both numerically and analytically) the dynamics for different values of the correlation time of the forcing $`\tau `$, finding that there is a region of parameters where mass separation occurs. This means that the direction of the current is found to be mass–dependent: the “heavy” species moves in the negative sense while the “light” one, do so in the positive sense.

Here, in order to compare results, we analyze the same system studied in  but considering the non Gaussian forcing described by Eq.(1). Hence, we study the system in Eq.1 with $`V(x)=[\mathrm{sin}(2\pi x)+.25\mathrm{sin}(4\pi x)]/(2\pi )`$ as the ratchet potential. We focus on the region of parameters where, in  (for $`q=1`$), separation of masses was found. We fix $`\gamma =2,KT=.1,\tau =.75`$, and $`D=.1875`$ and consider the values of the masses $`m=m_1=0.5`$ and $`m=m_2=1.5`$ as in . Our main result here is that the separation of masses is enhanced when a non–Gaussian noise with $`q>1`$ is considered. In Fig. 4.a. we show $`J`$ as function of $`q`$ for $`m_1=0.5`$ and $`m_2=1.5`$. It can be seen that there is an optimum value of $`q`$ that maximizes the difference of currents. In that figure, this value, which is close to $`q=1.25`$, is indicated with a vertical double arrow. Another double arrow indicates the separation of masses occurring for $`q=1`$ (Gaussian OU forcing). The calculations are for a load force $`F=0.25`$. We have observed that, when the value of the load force is varied, the difference between the curves remain approximately constant but both are shifted together to positive or negative values (depending on the sign of the variation of the loading). By controlling this parameter it is possible to achieve, for example, the situation shown in Fig. 4.b., where, for the value of $`q`$ at which the difference of currents is maximal, the heavy “species” remains static on average (has $`J=0`$), while the light one has $`J>0`$. Also it is possible to get the situation shown in Fig. 4.c, at which the two species moves in the opposite direction with equal absolute velocity.

Summarizing, we have systematically studied the effect of a colored non Gaussian noise source on the transport properties of a Brownian motor. What we have found is that a departure from Gaussian behavior, given by a value of $`q`$ larger than 1, induces a remarkable increase of the current together with an enhancement of the motor efficiency. The latter shows, in addition, an optimum value for a given degree of non Gaussianity. When inertia is taken into account we also find a considerable increment in the mass separation capability.

It is worth mentioning that in the studies of the influence of non Gaussian noises in stochastic resonance and other related phenomena, the system’s response enhancement occurs for $`q<1`$, as discussed in . The reason can be traced back to the dependence of the mean-first-passage-time with $`q`$, and the interplay between transition rates and modulation frequency. In contrast, in the present work, and due to the “favorable” influence of high values of the noise for increasing the transport effect, we observe that the relevant effect occurs in the opposite case, that is for $`q>1`$.

These studies could be of interest for possible relation with biologically motivated problems  as well as for potential technological applications, for instance in ”nanomechanics” . More specific studies of aspects in one or other area will be the subject of further work.

Acknowledgements: The authors thanks to V.Grunfeld for a critical reading of the manuscript. Partial support from CONICET, Argentina is acknowledged. HSW wants to thank to the IMEDEA and Universitat de les Illes Balears, Palma de Mallorca, Spain, for the kind hospitality extended to him.


text:
# X-ray Statistical Properties of the central cool component in clusters of galaxies

## 1. Sample and ASCA analysis

For the study we used brightest 85 clusters observed with ASCA whose redshifts distribute from 0.004 to 0.201 with a median of 0.046. From each GIS and SIS instrument, a spectrum over a central region of $`2^{}`$ radius and one outside $`2^{}`$ radius were accumulated. We fitted each set of ASCA spectra with the two-temperature model (2T model), in which a hot component is filling entire cluster region, while in the central region a cool component is allowed to coexist with the hot component forming two-phase plasma. We then obtained X-ray luminosity in 0.1-2.4 keV band and temperature of cool and hot component, $`L_\mathrm{c}`$, $`T_\mathrm{c}`$, $`L_\mathrm{h}`$, and $`T_\mathrm{h}`$. Full results are given in Ikebe et al. (2001). An example of the 2T model fit is shown in Fig. 1 for Abell 1795, which has one of the brightest central cool component.

## 2. Validity confirmation of the 2T model with XMM-Newton data

The validity of the 2T model was tested with the latest X-ray data of Abell 1795 taken with XMM-Newton. The 11 annular spectra of EPIC-PN were simultaneously fitted with the 2T model, where each annular spectrum is modeled with sum of two isothermal component, and each component is assumed to have common temperature throughout all radii. The fit is good and the best-fit two temperatures are derived to be $`T_\mathrm{c}`$=3.3 keV and $`T_\mathrm{h}`$=6.4 keV, which are compared with the temperature profile obtained with the single-phase model (Fig. 2). The $`T_\mathrm{c}`$ and $`T_\mathrm{h}`$ nicely agree with the central coolest temperature and that of the isothermal component in the outer region, respectively. Therefore, the 2T modeling is valid to the most advanced X-ray data and usable to characterize the property of the central cool component and the surrounding ICM.

## 3. Results and Discussions

Based on the results from the 2T model fitting to the ASCA data of the 85 selected clusters, we investigated various correlations. In the study, we classified the cluster sample into three groups, which are Strong Cool Component (SCC), X-ray Dominant (XD) and non-X-ray Dominant (nXD). An SCC cluster is defined as one showing a very strong (statistically very significant) cool component in the ASCA spectrum. Among the rest of the clusters, one in which the X-ray peak coincides with the Brightest Cluster Galaxy (BCG) is classified as an XD cluster, while nXD is defined as one without such a galaxy. All the SCC clusters turned out to be XDs, too.

For the SCC clusters, we determined the temperatures and emission measures of each cool and hot component. Figure 3(a) shows the cool component luminosity as a function of the cool component temperature for the SCC clusters. A positive correlation is clearly seen. Figure 4 shows the correlation between the two temperatures of the hot and cool component for the SCC clusters. Surprisingly, the ratios between the two temperatures, $`T_\mathrm{c}/T_\mathrm{h}`$, is virtually constant and the relation, $`T_\mathrm{c}`$=$`T_\mathrm{h}/2`$ well represents the correlation.

For clusters that do not show very strong cool component, i.e. non-SCC clusters, we fixed $`T_\mathrm{c}`$ at the half value of each mean temperature derived with a single-temperature model fitting, according to the $`T_\mathrm{c}`$=$`T_\mathrm{h}/2`$ relation. The luminosity of the cool component thus estimated for the non-SCC clusters is illustrated in Fig. 3(b). The results of the SCC clusters are also overlayed. SCC and XD clusters exhibit systematically more luminous cool component than nXD clusters. Actually, many of nXD clusters give only upper limit to $`L_\mathrm{c}`$.

We obtained the $`LT`$ relation of the hot component, which is illustrated in Fig. 5(a). Unlike for the cool component, the three classes do not show any systematic difference. It is then clearly shown that the X-ray characteristics of the XD and nXD clusters are segregated mainly by the central cool component. To compare with the $`LT`$ relation of the cool component would be very interesting. In the Fig. 5(b), the $`L_\mathrm{c}`$$`T_\mathrm{c}`$ relation for the SCC clusters shown in Fig. 3 is overlayed on the $`L_\mathrm{h}`$$`T_\mathrm{h}`$ relation. A surprising agreement is clearly seen. The best fit power-law function to the $`L_\mathrm{h}`$$`T_\mathrm{h}`$ relation is compared with the SCC $`L_\mathrm{c}`$$`T_\mathrm{c}`$ relation in Fig. 3. The similarity of the two $`L`$$`T`$ relation suggests that both the cool and hot components are related to individual gravitational bound objects. Therefore, the cool component can be naturally interpreted as ICM filling a self-gravitating system whose size is comparable to a giant elliptical galaxy or a group of galaxies, which is immersed in the host cluster.

## References

Ikebe et al. 2001, A&A, submitted

Tamura et al. 2001, A&A, 365, L87

Peterson et al. 2001, A&A, 365, L104


text:
# Detection of negative superhumps in a LMXRB – an end to the long debate on the nature of V1405 Aql (X1916-053)

## 1 Introduction

### 1.1 Permanent superhumps

Permanent superhumps have been observed so far in about 20 cataclysmic variables (CVs) (Patterson 1999). These show superhumps (quasi-periodicities shifted by a few percent from their orbital periods) in their optical light curves during normal brightness state. In contrast, SU UMa systems (see Warner 1995 for a review of SU UMa systems and CVs in general) have superhumps only during their bright dwarf nova outbursts (superoutbursts).

Permanent superhumps can either be a few percent longer than the orbital periods and they are called ‘$`\mathrm{𝐩𝐨𝐬𝐢𝐭𝐢𝐯𝐞}`$ $`\mathrm{𝐬𝐮𝐩𝐞𝐫𝐡𝐮𝐦𝐩𝐬}`$’, or shorter – ‘$`\mathrm{𝐧𝐞𝐠𝐚𝐭𝐢𝐯𝐞}`$ $`\mathrm{𝐬𝐮𝐩𝐞𝐫𝐡𝐮𝐦𝐩𝐬}`$’. The positive superhump is explained as the beat between the binary motion and the precession of an accretion disc in the apsidal plane. Similarly, the negative superhump is understood as the beat between the orbital period and the nodal precession of the disc (Patterson 1999).

According to theory, superhumps are developed when the accretion disc extends beyond the 3:1 resonance radius and becomes elliptical (Whitehurst & King 1991). Osaki (1996) suggested that permanent superhump systems differ from other subclasses of non-magnetic CVs in their relatively short orbital periods and high mass-transfer rates, resulting in accretion discs that are thermally stable but tidally unstable. Retter & Naylor (2000) gave observational support to this idea. Simulations suggest that superhumps can only occur in binary systems with small mass ratios – $`q=M_{donor}/M_{compact}0.33`$ (Whitehurst 1988; Whitehurst & King 1991; Murray 2000). Although this condition is easily met in short orbital period LMXRBs, whose primaries are compact massive objects, superhumps have been only seen in a few LMXRBs in outburst (e.g. O’Donoghue & Charles 1996). Here we report the detection of permanent superhumps in a persistent LMXRB.

### 1.2 V1405 Aql (X1916-053 / 4U 1915-05)

V1405 Aql was first discovered as an X-ray source showing type-I bursts (Becker et al. 1977; Doxsey et al. 1977) suggesting that its primary star is a neutron star. White & Swank (1982) and Walter et al. (1982) independently found $``$3000-s periodic dips in its X-ray light curve. The dips are believed to occur due to obscuration of the disc edge (probably where the accretion stream impacts the disc). Such a short orbital period implies a low mass hydrogen-deficient secondary star (Nelson, Rappaport & Joss 1986). The 21-mag optical counterpart of V1405 Aql was identified by Schmidtke (1988) and Grindlay et al. (1988). They also reported a detection of an optical periodicity, about one percent longer than the X-ray period. The difference between the X-ray and optical periods was confirmed by further extensive observations of V1405 Aql (Smale et al. 1988; 1989; 1992; Callanan 1993; Callanan, Grindlay & Cool 1995; Yoshida et al. 1995; Church et al. 1997; 1998; Ko et al. 1999; Morley et al. 1999; Chou, Grindlay & Bloser 2001; Homer et al. 2001).

The various models offered so far for the periodicities found in V1405 Aql can be grouped into two basic models – a triple system (Grindlay 1986; Grindlay et al. 1988) and superhumps (Schmidtke 1988; White 1989). According to the first model, which was motivated by the detection of a possible 199-d periodicity in the X-ray light curve of V1405 Aql (Priedhorsky & Terrell 1984), the longer 3028-s period is the binary inner orbital period, while the shorter 3001-s period is the beat between the binary period and the $``$4-d orbital period of a third companion. The 199-d period is explained by the eccentricity of the inner binary orbit (Chou et al. 2001). The second model suggests that the 3001-s period is the binary period and that the 3028-s period is a positive superhump. We note that both models predict the presence of a beat periodicity of $``$4 d. Indeed evidence for a period of $``$3.9 d has been found in the X-ray light curve of V1405 Aql (Chou et al. 2001; see also Homer et al. 2001).

The debate on the nature on V1405 Aql has continued (Chou et al. 2001; Homer et al. 2001; Haswell et al. 2001), although some preference was given to the superhump scenario. One of the arguments against the triple system model is that the 199-d period was not confirmed by further observations, although a second possible evidence for this periodicity was found by Smale & Lochner (1992). On the other hand, an argument against the superhump model is that the 3028-s period is quite stable while superhump periods are unstable (Callanan et al. 1993; 1995; Chou et al. 2001; Homer et al. 2001).

There is a strong observational link between positive and negative superhumps in CVs as light curves of several systems show both (Patterson 1999; Arenas et al. 2000; Retter et al. 2002b). In addition, Patterson (1999) found that period deficits in negative superhumps are about half period excesses in positive superhumps: $`ϵ_{}0.5ϵ_+`$, where $`ϵ=(P_{superhump}P_{orbital})/P_{orbital}`$. Thus, we decided to look in available photometric data on V1405 Aql for negative superhumps, which would be predicted to have a period near 2986 s. Indeed we have found a new periodicity in the X-ray light curve of V1405 Aql. These findings were reported by Retter, Chou & Bedding (2002a), and are presented below. We also present a new relation for systems that have both types of superhumps, and discuss the mechanisms for superhumps in the X-ray and optical regimes.

## 2 Observations and Analysis

We have re-analysed existing X-ray photometry that was presented by Chou et al. (2001). The dataset consists of about 150 ksec of observations by the RXTE satellite obtained during 17 occasions between 1996 February to October.

In Figs. 1a $`\&`$ 2a the power spectrum (Scargle 1982) of 10 successive runs in 1996 May is presented. No de-trending method was used. Bursts were not observed during these observations. In addition to the two known periods (3001 $`\&`$ 3028 s, marked as f1 $`\&`$ f2) and their 1-d-1 aliases, there is a third peak (labelled f3) together with its 1-d-1 alias pattern. After fitting and subtracting the first term of the two known frequencies (f1 $`\&`$ f2), the third, which corresponds to the periodicity 0.034483$`\pm `$0.000013 d (2979.3$`\pm `$1.1 s), becomes the strongest peak in the residual power spectrum (Figs. 1b $`\&`$ 2b). The error was calculated by 1000 simulations (see e.g. Retter et al. 2002b). The subtraction of nine more terms of the periods (to account for a non-sinusoidal shape) and the 3.9-d period (whose peak in the raw power spectrum is not significant) yields similar results. This point is valid for all tests listed below. The power spectrum of the whole 1996 dataset resembles the result of the 10 runs in May, however, it is noisier as the gaps between the observations are long.

In Fig. 3 we show the mean shape of the 2979-s period in 1996 May. The full amplitude of the sinusoidal fit to the light curve is 8.0$`\pm `$1.2 counts/sec, which translates to 25$`\pm `$4% of the mean count rate. This is smaller than the mean amplitude of the 3028-s period – 28$`\pm `$4% and the mean depth of the 3001-s dips – 35$`\pm `$5%. The superposition of these three signals can explain the deepest observed dips that sometimes reach almost 100% (i.e. total obscuration).

### 2.1 Tests

#### 2.1.1 The window function

To check whether the third peak in the power spectrum could be an artifact of the window function, a noiseless simulation of the 1996 May set was created. A synthetic light curve was built using two sinusoids at the 3001 and 3028-s periods. These sinusoids were given the same amplitudes they have in the data and sampled according to the window function. There was no evidence for significant power at the proposed period.

#### 2.1.2 Uncorrelated noise

In an attempt to check whether uncorrelated noise could be responsible for the presence of the new peak in the power spectrum, the two previously known periods (3001 and 3028 s) were subtracted from the data. The remaining points, assumed to be white noise, were shuffled randomly between the timings. A power spectrum was calculated for each random configuration. The highest peak at the range f=20-40 d-1 in 1000 simulations did not reach the height of f3.

In a different test, that checks the significance of a third peak in the presence of two others, we took the synthetic light curves (see previous section) and now added noise, defined as the root mean square of the original data minus the periods modelled. We then searched for the highest peak in a small interval (28.9-29.1 cycles/day) around the candidate period (f3). The lower value is dictated by the nearby presence of the 3001-s period (f1). In 1000 simulations, no peak reached the height of the candidate periodicity. Figs. 1c $`\&`$ 2c show an individual example of this simulation.

#### 2.1.3 Correlated noise

Another test we tried was to assess the probability that correlated noise could be responsible for the candidate periodicity. In the absence of a model for the correlated noise, the best test is to use the repeatability between different datasets. Thus the 1996 May data were divided into two subsets – the first and last five runs. Power spectra of both sets were calculated after the two known periods (3001 $`\&`$ 3028 s) were removed. Fig. 4 displays the results. In both sets the strongest peak (up to a 1-d-1 alias) is very near the peak at the power spectrum of the whole set (Figs. 1 $`\&`$ 2).

Given that we have found a period in the first subset, we can ask how likely it is that the strongest period in the second subset would be consistent with it. The probability of the highest peak in another dataset being, by chance, compatible with the candidate period in the first set is 0.04. This was calculated from (i) twice the frequency error (0.02 d-1) on the peak in the first subset and (ii) the range over which it could occur taken as the spacing of the 1-d-1 aliases. This test suggests that the candidate periodicity is 96% significant. We note, however, that the frequency error was calculated by 1000 simulations. A naive estimate of the error as the width at half maximum of the peak yields a somewhat lower significance level.

#### 2.1.4 The influence of the dips

To eliminate the possibility that the new periodicity in V1405 Aql arises from random variations in the shape, width and / or depth of the dips, all points connected with the dips were manually rejected. The highest peak in the power spectrum of the remaining data is consistent with f3 (Figs. 1d $`\&`$ 2d). This test shows that the ‘persistent’ light curve (i.e. without the dips) is dominated by the 2979-s periodicity.

## 3 Discussion

### 3.1 The new period

A period of 2979 s has been found in the X-ray light curve of V1405 Aql in addition to the other known periods. This feature was actually noticed by Chou et al. (2001) who mistakenly identified it with a 3.9-d sideband of the 3001-s period. It is, however, $`5\sigma `$ away from the expected sideband at 2974.2 s and we have shown above that it is not an alias. We also note that a period of 2984.6$`\pm `$6.8 s was detected in the Ginga data of this object (Smale et al. 1989). As it is inconsistent with either the 3001 and 3028-s periods, it probably represents the new periodicity as well.

The 2979-s period is shorter than the 3001-s period by about 0.7 percent. Assuming that the 3001-s period is the orbital period and that the 3028-s period is a positive superhump, the new period is naturally explained as a negative superhump. Our suggestion implies a nodal precession of $``$4.8 d. Indeed Chou et al. (2001) found that the phase jitter of the X-ray dips in the 1996 May data is modulated with a period of 4.86 d (as well as with the $``$3.9-d period). Homer et al. (2001) reached a similar conclusion from a different dataset and found a period of 4.74$`\pm `$0.05 d. Thus, the superhump model can explain this periodicity as well.

The 2979-s period in the X-ray light curve of V1405 Aql and the beat period at $``$4.8 d cannot be explained by the triple system model. This scenario was motivated by the possible detection of the 199-d period that has not been confirmed by further observations (Section 1.2). Homer et al. (2001) suggested a few other possible periodicities ($``$8.6, $``$12, $``$23, $``$83 d) in the X-ray light curve of V1405 Aql. It is very likely that V1405 Aql has random variations of the order of a few tens of day, but not a coherent long-term periodicity. We thus believe that the triple system model for V1405 Aql is finally ruled out.

Table 1 lists the periods observed in V1405 Aql and their interpretations according to the superhump model. The presence of an extensive number of periodicities can explain many peculiarities in the light curve of the system that have been previously discussed by many authors.

### 3.2 A new relation for superhumps

Patterson (1999) proposed that the negative superhump deficit is about half the positive superhump excess (Section 1.2). As the 3028-s period is about 0.9 percent longer than the 3001-s period, the corresponding ratio in V1405 Aql (0.79) is somewhat larger than this. We have checked the connection between the deficit and excess of negative and positive superhumps in CVs and found a new relation. Table 2 presents the data on the periods in systems showing both types of superhumps. In Fig. 5 we show this ratio for these systems, and we see a clear trend as a function of orbital period. Our result for V1405 Aql fits this trend very well.

### 3.3 Implications

Our results strengthen the observational link between positive and negative superhumps, and supports the idea that the two types of superhumps have a similar physical origin, namely a precessing accretion disc. Retter et al. (2002b) speculated that every permanent superhump system may have both kinds of superhumps. The data on V1405 Aql support this idea.

Wood, Montgomery & Simpson (2000) showed that a tilted accretion disc can explain the presence of negative superhumps, however, it is still unclear what physical force drives the disc to precess in the nodal plane (Murray & Armitage 1998). Wynn (personal communication) suggested that a strong magnetic field of the primary white dwarf can cause the disc to precess in the nodal plane. If this idea is confirmed, negative superhumps are expected in every permanent superhump LMXRB, as their primaries have magnetic fields typically much stronger than in CVs.

Montgomery (2001) and Montgomery et al. (in preparation) developed analytic expressions for the two types of superhumps. The theoretical values for the ratio $`\varphi `$=$`ϵ_{}/ϵ_+`$ mentioned above, were found to be consistent with the observations of CVs. Montgomery et al.’s results can also explain the relation shown in Fig. 5 for CVs (see their Fig. 6). Using their equation 9 & Fig. 4 (but baring in mind the difference in primary mass) and the observed $`\varphi `$ parameter in V1405 Aql (Table 2), we can derive a weak upper limit on its mass ratio of $`q0.08`$. For a primary neutron star at the Chandrasekhar limit, this implies an upper limit on the mass of the secondary – $`M_20.12M_{}`$, which is presumably a helium white dwarf.

The fact that binary systems with very different configurations (CVs – a hydrogen-rich white dwarf and a red companion; AM CVn – two degenerate helium white dwarfs and V1405 Aql – a neutron star and a helium white dwarf) obey the relation presented in Fig. 5 is quite surprising. It might be a coincidence, or it may suggest that $`\varphi `$ indeed depends on the orbital period and not on the mass ratio. Further determinations of this parameter in more permanent superhump systems are required to study this issue.

### 3.4 The mechanism for negative superhumps

The mechanism which places the negative superhump signal into the X-ray light curve is unclear, but changing vertical disc structure seems a promising candidate. It is clear that positive superhumps coincide with increased disc thickness (Billington et al. 1996), the evidence being consistent with the idea that the area of the disc involved in producing the superhump light thickens at the time of the increased dissipation. In the nodal precession model, Wood et al. (2000) showed how the passage of the secondary star past each of the two halves of the disc out of the midplane increases the dissipation in that half. They also required that the side of the disc closest to the orbital plane is the most disrupted. If, by analogy with positive superhumps, this disruption increases the vertical extent of the disc, then it will increase the obscuration of the X-ray source. As the disc precesses, the timing of the obscuration will change with phase, producing the required modulation at the negative superhump period. The lack of structure and large range in phase of the negative superhump modulation implies that both the disc structure and occulted region are large, the latter presumably being the inner accretion disc or a corona. This scenario can also explain why negative superhump signals have not been observed so far in the X-ray light curves of low inclination systems.

### 3.5 The mechanism for positive superhumps

Haswell et al. (2001) pointed out that the dissipation of energy in the disc, which is believed to be responsible for the positive superhump mechanism in CVs in the optical, cannot be applied to LMXRBs. They suggested that instead the disc area is changing with the superhump period, and thus predicted that superhumps would appear mainly in low inclination systems. Figs. 1 & 2 show that the X-ray data is modulated with the positive superhump (f2). However, when the dips are rejected from the light curve, this peak disappears. Thus in V1405 Aql only the dips show the positive superhump, presumably since their amplitude and / or phase vary with the apsidal precession period. Note that the phase jitter of the dips is modulated with the 3.9-d period (Chou et al. 2001). Therefore, a simple explanation of this behaviour is that thickening of the disc rim, which causes an increase obscuration of the X-ray source, forms the positive superhump. This is consistent with our suggestion for the mechanism for the negative superhump in the X-ray (previous section). The disc thickening would allow the disc to intercept more of the irradiating flux, thus producing optical superhump light seen at all inclinations in a similar way to the Haswell et al. area effect. In addition, though, it would introduce dips into the X-ray and perhaps optical orbital light curves, allowing the superhump to be visible in high inclination systems, in contrast to the prediction of Haswell et al.

## 4 Acknowledgments

We owe a great debt to the anonymous referee for very useful comments that significantly improved the paper. We thank Michele Montgomery for sending us an early version of her paper prior to publication. AR was supported by PPARC and is currently supported by the Australian Research Council.


text:
# Degree Bounds in Quantum Schubert Calculus

## 1. Introduction and Main Results

Let $`X=Gr(l,^n)`$ be the Grassmannian of $`l`$-dimensional subspaces in $`^n`$. The classical cohomology ring $`\mathrm{H}^{}(X,)`$ has an additive basis of Schubert classes $`\{\sigma _\lambda \}`$, indexed by the Young diagrams $`\lambda `$ (identified with the corresponding partitions) contained in the $`l\times k`$ rectangle, where $`k=nl`$ (we denote this by $`\lambda l\times k`$). The product of two Schubert classes in $`\mathrm{H}^{}(X,)`$ is given by

(1)
$$\sigma _\lambda \sigma _\mu =\underset{\nu l\times k}{}c_{\lambda ,\mu }^\nu \sigma _\nu ,$$

where $`c_{\lambda ,\mu }^\nu `$ is the Littlewood-Richardson coefficient (see, e.g., ).

The (small) quantum cohomology ring $`\mathrm{QH}^{}(X)`$ is a certain deformation of $`\mathrm{H}^{}(X,)`$ that has been extensively studied in recent years; see, e.g.,  and references therein. This ring is canonically isomorphic to the Verlinde algebra of $`𝔰𝔩_l`$ at level $`k`$ (see, e.g., ); consequently, the results presented here can be reformulated in representation theoretic language.

The additive structure of $`\mathrm{QH}^{}(X)`$ is essentially the same as that of $`\mathrm{H}^{}(X,)`$: the Schubert classes form a basis of $`\mathrm{QH}^{}(X)`$ as a free module over $`[q]`$, where $`q`$ is an indeterminate. The multiplicative structure of $`\mathrm{QH}^{}(X)`$ is defined by

(2)
$$\sigma _\lambda \sigma _\mu =\underset{\nu l\times k}{}\underset{d0}{}q^d\lambda ,\mu ,\nu ^{}_d\sigma _\nu ,$$

where $`\nu ^{}=(k\nu _l,\mathrm{},k\nu _1)`$ is the complement of $`\nu `$ in the $`l\times k`$ rectangle, and $`\lambda ,\mu ,\nu ^{}_d`$ is a three-point, genus-zero Gromov-Witten invariant of $`X`$. Note that we use “$``$” to distinguish the quantum multiplication from the product in the classical cohomology ring. Setting $`q=0`$ recovers (1) because $`c_{\lambda ,\mu }^\nu =\lambda ,\mu ,\nu ^{}_0`$.

Bertram, Ciocan-Fontanine and Fulton  have given a combinatorial rule to compute $`\sigma _\lambda \sigma _\mu `$ and thus the Gromov-Witten invariants $`\lambda ,\mu ,\nu ^{}_d`$. To describe this rule, we need some terminology and notation. An $`n`$-rim hook of a Young diagram $`\lambda `$ is a connected subset of $`n`$ boxes of $`\lambda `$ that does not contain a $`2\times 2`$ square. The width of an $`n`$-rim hook is the number of columns it occupies. An $`n`$-rim hook is legal if removing it from a Young diagram gives a valid Young diagram (see Figure 1). For a partition $`\rho `$, we define its $`n`$-core, denoted $`\mathrm{core}_n(\rho )`$, to be the partition corresponding to the Young diagram obtained by repeatedly removing legal $`n`$-rim hooks from $`\rho `$ until further removals are not possible. It is well known (see, e.g., ) that this procedure defines $`\mathrm{core}_n(\rho )`$ uniquely. Let $`r_n(\rho )=\frac{\rho \mathrm{core}_n(\rho )}{n}`$ be the number of $`n`$-rim hooks removed in this process, and set $`ϵ(\rho )=(1)^{{\scriptscriptstyle (kwidth(R_i))}}`$, where $`R_1,\mathrm{},R_{r_n(\rho )}`$ are these $`n`$-rim hooks. With this notation, the rule obtained in  is as follows:

(3)
$$\sigma _\lambda \sigma _\mu =\underset{\nu l\times k}{}\underset{\begin{array}{c}c_{\lambda ,\mu }^\rho 0\\ \rho _1k\\ \mathrm{core}_n(\rho )=\nu \end{array}}{}q^{r_n(\rho )}ϵ(\rho )c_{\lambda ,\mu }^\rho \sigma _\nu .$$

The Gromov-Witten invariants $`\lambda ,\mu ,\nu ^{}`$ are known to be nonnegative; thus the coefficient of each term $`q^d\sigma _\nu `$ in the right-hand side of (3) is nonnegative. It remains an open problem to find a direct combinatorial proof of this statement; such a proof would provide a generalization of the Littlewood-Richardson rule to the quantum Schubert calculus of the Grassmannian.

Let $`\mathrm{d}_{\mathrm{min}}`$ denote the smallest degree of $`q`$ such that $`q^{\mathrm{d}_{\mathrm{min}}}`$ appears in (2) with nonzero coefficient. The following theorem provides an affirmative answer to a question posed by A. Ram (private communication).

###### Theorem 1.1.

Let $`\lambda ,\mu l\times k`$. Among all $`\rho `$ with $`\rho _1k`$ and $`c_{\lambda ,\mu }^\rho 0`$, pick those with the largest n-core (equivalently, the smallest value of $`r_n(\rho )`$). Then their contributions to (3) do not cancel each other out. In other words, $`\mathrm{d}_{\mathrm{min}}`$ equals the smallest degree of $`q`$ that appears in the right-hand side of (3) (before cancellations).

Our proof of Theorem 1.1 will also prove a recent result of Fulton and Woodward (see Theorem 1.2 below). For partitions $`\lambda ,\mu l\times k`$, place $`\lambda `$ against the upper left corner of the rectangle. Then rotate $`\mu `$ 180 degrees and place it in the lower right corner (see Figure 2). We will refer to $`\mathrm{rotate}(\mu )`$ as the resulting subshape of $`l\times k`$. Let $`𝔡`$ be the side length of the largest square that fits inside $`\lambda \mathrm{rotate}(\mu )`$. The following theorem was conjectured by Fulton, and later proved by Fulton and Woodward , using moduli spaces.111Actually, Fulton and Woodward proved a more general result that applies to any homogeneous space $`X=G/P`$, where $`G`$ is a simply connected complex semisimple Lie group and $`P`$ its parabolic subgroup.

###### Theorem 1.2.

 $`\mathrm{d}_{\mathrm{min}}=𝔡`$.

An alternative proof of Theorem 1.2 was later given by A. Buch , using an elegant geometric argument combined with combinatorics. We will utilize the combinatorial part of Buch’s proof below.

Let us now describe our proof of Theorems 1.1 and 1.2. The proof is entirely combinatorial once the nonnegativity of the Gromov-Witten invariants and the fact that $`\mathrm{QH}^{}(X)`$ is an associative ring  are granted. Our argument is based on the following result.

###### Theorem 1.3.

Let $`a\times A`$, $`b\times B`$ and $`c\times C`$ be rectangular Young diagrams contained in $`l\times k`$. Then the following are equivalent:

* there exist Young diagrams $`\lambda ,\mu ,\nu l\times k`$ containing $`a\times A`$, $`b\times B`$ and $`c\times C`$, respectively, such that $`\sigma _\lambda \sigma _\mu \sigma _\nu 0`$;
* $`\sigma _{a\times A}\sigma _{b\times B}\sigma _{c\times C}0`$;
* all of the five conditions below hold:

+ $`a+bl`$ or $`A+Bk`$;
    + $`a+cl`$ or $`A+Ck`$;
    + $`b+cl`$ or $`B+Ck`$;
    + $`a+b+cl`$ or $`A+B+C2k`$;
    + $`a+b+c2l`$ or $`A+B+Ck`$.

A combinatorial proof of Theorem 1.3 is given in Section 2. S. Fomin (private communication) has observed that the necessity of the conditions (i)-(v) can be derived from the Horn inequalities describing the “Klyachko cone” $`\{(\lambda ,\mu ,\nu )c_{\lambda ,\mu }^\nu 0\}`$; specifically see \[5, (11)\].

###### Corollary 1.4.

Let $`\lambda ,\mu ,\rho `$ be partitions such that $`\lambda ,\mu l\times k`$ and $`\rho _1k`$. If $`c_{\lambda ,\mu }^\rho 0`$, then $`(l+𝔡)\times 𝔡\rho `$.

###### Proof.

Suppose $`c_{\lambda ,\mu }^\rho 0`$ but $`(l+𝔡)\times 𝔡\rho `$. Choose any positive integer $`Ll`$ such that $`\rho L\times k`$. Then we have $`\sigma _\lambda \sigma _\mu \sigma _\rho ^{}0`$. Notice that $`(L(l+𝔡)+1)\times (k𝔡+1)\rho ^{}`$. By the definition of $`𝔡`$, there exist rectangles $`a\times A\lambda `$ and $`b\times B\mu `$ such that $`a+bl+𝔡`$ and $`A+Bk+𝔡`$. Set $`\nu =\rho ^{},c=L(l+𝔡)+1`$ and $`C=k𝔡+1`$, then it is easy to check that both corresponding inequalities (iv) are violated, a contradiction of Theorem 1.3 (see Figure 3 below). ∎

*Proof of Theorems 1.1 and 1.2.* Let $`\rho `$ be a partition whose first part is at most $`k`$ and whose $`n`$-core is contained in $`l\times k`$. If $`c_{\lambda ,\mu }^\rho 0`$, then Corollary 1.4 gives $`(l+𝔡)\times 𝔡\rho `$, implying that at least $`𝔡`$ rim hook removals are needed to obtain $`\mathrm{core}_n(\rho )`$. Hence the smallest degree of $`q`$ that occurs in (3), and therefore $`\mathrm{d}_{\mathrm{min}}`$, is at least $`𝔡`$.

To complete the proofs, it remains to show that $`\mathrm{d}_{\mathrm{min}}`$ and thus the smallest degree of $`q`$ that appears in (3) is at most $`𝔡`$. To this end, we borrow an argument of A. Buch , used in his own proof of Theorem 1.2. We reproduce his argument below:

Let $`\stackrel{~}{\lambda }`$ and $`\overline{\lambda }`$ be the partitions obtained by removing the leftmost $`𝔡`$ columns and the top $`𝔡`$ rows of $`\lambda `$, respectively. Also, let $`\stackrel{~}{\overline{\lambda }}`$ be the partition obtained by removing both the leftmost $`𝔡`$ columns and the top $`𝔡`$ rows of $`\lambda `$. Set $`\alpha =(k+𝔡\lambda _𝔡,\mathrm{},k+𝔡\lambda _1)`$ and let $`\beta _i=\mathrm{max}(𝔡\lambda _{l+1i},0)`$ for $`1il`$. In other words, $`\alpha `$ is the complement of the bottom $`𝔡`$ rows of $`\mathrm{rotate}(\lambda )`$ in $`l\times (k+𝔡)`$, and $`\beta `$ is the complement of $`\mathrm{rotate}(\lambda )`$ in the rightmost $`𝔡`$ columns (see Figure 4 below).

It follows from the Littlewood-Richardson rule that the expansion of $`\sigma _\lambda \sigma _\beta `$ contains the class $`\sigma _{(𝔡^l)+\stackrel{~}{\lambda }}=\sigma _{(𝔡^l)}\sigma _{\stackrel{~}{\lambda }}`$. It also follows that $`\sigma _{\stackrel{~}{\lambda }}\sigma _\alpha `$ contains $`\sigma _{(k^𝔡),\stackrel{~}{\overline{\lambda }}}=\sigma _{(k^𝔡)}\sigma _{\stackrel{~}{\overline{\lambda }}}`$. It is not hard to check directly from (3) that $`\sigma _{(𝔡^l)}\sigma _{(k^𝔡)}=q^𝔡`$. By the nonnegativity of the Gromov-Witten invariants and the associativity of $`\mathrm{QH}^{}(X)`$, $`\sigma _\lambda \sigma _\beta \sigma _\alpha `$ contains the product $`\sigma _{(𝔡^l)}\sigma _{(k^𝔡)}\sigma _{\stackrel{~}{\overline{\lambda }}}=q^𝔡\sigma _{\stackrel{~}{\overline{\lambda }}}`$. Note that $`\stackrel{~}{\overline{\lambda }}\mathrm{rotate}(\mu )=\mathrm{}`$, which is well known to be equivalent to $`\sigma _{\stackrel{~}{\overline{\lambda }}}\sigma _\mu 0`$. Thus we conclude that $`\sigma _\lambda \sigma _\mu \sigma _\alpha \sigma _\beta `$ contains $`q^𝔡`$ times some Schubert class. Therefore, the product $`\sigma _\lambda \sigma _\mu `$ must have a term involving a degree of $`q`$ less than or equal to $`𝔡`$, and we are done.∎

## 2. Proof of Theorem 1.3

For $`\alpha ,\beta l\times k`$, define $`\varkappa (\alpha ,\beta )`$ to be the intersection of all Young diagrams $`\gamma l\times k`$ such that $`c_{\alpha ,\beta }^\gamma 0`$. In the event the set of such $`\gamma `$ is empty, we set $`\varkappa (\alpha ,\beta )=l\times k`$ by convention. We will make use of the following observation, which follows immediately from this definition:

###### Lemma 2.1.

Let $`\alpha ,\beta ,m\times Ml\times k`$. Then $`\sigma _\alpha \sigma _\beta \sigma _{m\times M}0`$ if and only if $`\varkappa (\alpha ,\beta )\mathrm{rotate}(m\times M)=\mathrm{}`$.

Our proof of Theorem 1.3 also uses the following two lemmas.222Although we will not need it here, this Lemma 2.2 generalizes to a statement about the cohomology of any homogeneous space $`G/P`$. The proof is analogous to the one given above.

###### Lemma 2.2.

Let $`\alpha ,\beta ,\lambda ,\mu l\times k`$ be such that $`\alpha \lambda `$ and $`\beta \mu `$. Then $`\varkappa (\alpha ,\beta )\varkappa (\lambda ,\mu )`$.

###### Proof.

It suffices to prove the case when $`\lambda =\alpha `$ and $`\mu \beta `$ is a single box. Since every term of $`(\sigma _\alpha \sigma _\beta )\sigma _1`$ is indexed by a partition containing $`\varkappa (\alpha ,\beta )`$, the same is true for $`\sigma _\alpha (\sigma _\beta \sigma _1)=\sigma _\lambda (\sigma _\mu +\text{nonnegative terms})`$. The claim follows. ∎

The next lemma is proved by a straightforward application of the Littlewood-Richardson Rule. Details are left to the reader.

###### Lemma 2.3.

Let $`m\times M,n\times Nl\times k`$. If $`\sigma _{m\times M}\sigma _{n\times N}=0`$, then $`\varkappa (m\times M,n\times N)=l\times k`$. Otherwise,

$`\varkappa (m\times M,n\times N)`$ $`=`$ $`(m\times M)(n\times N)`$
$`((m+n)\times (M+Nk))`$
$`((m+nl)\times (M+N)),`$
where each of the last two rectangles in the right-hand side is understood to be empty if one of its dimensions is negative, or if it does not fit inside $`l\times k`$.

*Proof of Theorem 1.3.* Suppose (I) holds. Then there exists $`\rho l\times k`$ with $`c_{\lambda ,\mu }^\rho 0`$ and $`\rho \mathrm{rotate}(\nu )=\mathrm{}`$. Hence $`\varkappa (\lambda ,\mu )\mathrm{rotate}(\nu )=\mathrm{}`$ which by Lemma 2.2 implies

(4)
$$\varkappa (a\times A,b\times B)\mathrm{rotate}(c\times C)=\mathrm{}.$$

By Lemma 2.1, (4) is equivalent to (II), which itself trivially implies (I). Lastly, it follows from Lemma 2.3 that (4) is equivalent to (III).∎

## 3. An Upper Bound, Open Problems and Conjectures

It is an open problem to determine $`\mathrm{d}_{\mathrm{max}}`$, the largest degree of $`q`$ such that $`q^{\mathrm{d}_{\mathrm{max}}}`$ appears in (2) with nonzero coefficient. In particular, $`\mathrm{d}_{\mathrm{max}}`$ is not equal to the largest degree of $`q`$ appearing in (3). Next, we present a simple upper bound for $`\mathrm{d}_{\mathrm{max}}`$. For a Young diagram $`\alpha `$, let $`\mathrm{diag}(\alpha )`$ be the size of the largest square contained in $`\alpha `$.

###### Proposition 3.1.

$`\mathrm{d}_{\mathrm{max}}\mathrm{min}(\mathrm{diag}(\lambda ),\mathrm{diag}(\mu )).`$

We note that for a large class of pairs $`(\lambda ,\mu )`$, this inequality is sharper than the obvious upper bound $`\mathrm{d}_{\mathrm{max}}\frac{\lambda +\mu }{n}`$.

*Proof of Proposition 3.1* Let $`\lambda `$ be a Young diagram and $`\lambda =(\alpha _1,\mathrm{}\alpha _t\beta _1,\mathrm{},\beta _t)`$ its Frobenius notation; the following is a well known identity for Schur functions (see, e.g. ):

(5)
$$s_\lambda =det(s_{(\alpha _i\beta _j)})_{1i,jt}.$$

From (5) and our description of $`\mathrm{QH}^{}(X)`$ it follows that

(6)
$$\sigma _\lambda =det(\sigma _{(\alpha _i\beta _j)})_{1i,jt}.$$

It is easy to check from the Littlewood-Richardson rule that for any $`(\alpha \beta ),\lambda l\times k`$, the largest degree of $`q`$ that appears in $`\sigma _{(\alpha \beta )}\sigma _\lambda `$ is 1. By (6) we have

$$\sigma _\lambda \sigma _\mu =det(\sigma _{(\alpha _i\beta _j)})_{1i,jt}\sigma _\mu =\underset{\tau 𝒮_n}{}\mathrm{sign}(\tau )(\underset{i=1}{\overset{t}{}}\sigma _{(\alpha _i\beta _{\tau (i)})})\sigma _\mu .$$

Thus $`M\mathrm{diag}(\lambda )`$. Switching the roles of $`\lambda `$ and $`\mu `$ gives $`M\mathrm{diag}(\mu )`$. ∎

It is an interesting problem to determine precisely which degrees of $`q`$ appear in the product $`\sigma _\lambda \sigma _\mu `$. Based on extensive computational evidence, we conjecture:

###### Conjecture 3.2.

The product $`\sigma _\lambda \sigma _\mu `$ involves $`q^d`$ for any $`d[\mathrm{d}_{\mathrm{min}},\mathrm{d}_{\mathrm{max}}]`$.

More seems to be true.

###### Conjecture 3.3.

Let $`d1`$ be an integer. If $`\lambda ,\mu ,\nu _d0`$, then either there exists $`\alpha \nu `$ such that $`\lambda ,\mu ,\alpha _{d1}0`$, or else $`\lambda ,\mu ,\alpha _j=0`$ for all $`\alpha l\times k`$ and all $`0jd1`$.

## Acknowledgments

This work was partially carried out while the author was visiting the Fields Institute in Toronto, and later at the Isaac Newton Institute in Cambridge. We thank both institutes for their hospitality. We are most deeply indebted to Sergey Fomin, whose guidance greatly improved the content and form of this paper. We benefited from conversations with Anders Buch, Bill Fulton, Arun Ram and Chris Woodward. Most of our computational investigations were done using Anders Buch’s package .