text:
# I Introduction

## I Introduction

The seesaw mechanism  is a simple framework to account for the small effective mass of the left-handed neutrino. It requires only a modest extension of the minimal standard model, namely the addition of the right-handed neutrino. As a consequence of this inclusion, both a Dirac mass term for the neutrino and a Majorana mass term for the right-handed neutrino are allowed. While the Dirac mass, $`m_\nu `$, is expected to be of the same order of magnitude as the quark or charged lepton mass, the Majorana mass of the right-handed neutrino, $`m_R`$, is not constrained and may be very large. If this is the case, a small effective Majorana mass for the left-handed neutrino, $`m_L(m_\nu /m_R)m_\nu `$, is generated.

At the same time, the out-of-equilibrium decay of the heavy neutrino can produce a baryon asymmetry through the so-called baryogenesis via leptogenesis mechanism . Hence, the existence of the heavy Majorana neutrino may explain both the smallness of the effective neutrino mass and the baryon asymmetry in the universe. Moreover, the Majorana nature of the light neutrino, generated by the Majorana nature of the heavy neutrino, allows the neutrinoless double beta decay, because of lepton number violation at high energy . Thus the mass scale of the heavy neutrino could be a new fundamental scale in physics.

For three generations of fermions, the light neutrino mass matrix $`M_L`$ as well as the baryon asymmetry $`Y_B`$ depend on the Dirac neutrino mass matrix $`M_\nu `$ and the heavy neutrino mass matrix $`M_R`$. In this paper we describe, in a simplified but instructive approach, the structure of $`M_\nu `$ and $`M_R`$ within the seesaw mechanism and the consequences for the baryon asymmetry generated in the baryogenesis via leptogenesis mechanism and for the neutrinoless double beta decay.

As a first approximation, we assume a diagonal form for the Dirac neutrino mass matrix, and then the effect of a non diagonal form is analyzed. The prediction for the neutrinoless double beta decay can remain unchanged because for a fixed $`M_L`$ and any choice of $`M_\nu `$ one can find a certain $`M_R`$ which reproduces $`M_L`$ through the seesaw formula, while the impact on the amount of baryon asymmetry is significant, because those $`M_R`$ and $`M_\nu `$ determine $`Y_B`$ through the leptogenesis formula.

The outline of the paper is the following. In section II we give an approximate description of the effective neutrino mass matrix. In sections III and IV we briefly discuss the seesaw mechanism and the baryogenesis via leptogenesis, respectively. In section V the forms of the heavy neutrino mass matrix, and their implications for the amount of baryon asymmetry and the rate for neutrinoless double beta decay are reviewed, according to different mass spectra of light neutrinos. In this section we assume a diagonal Dirac neutrino mass matrix. The important effect of a non diagonal Dirac neutrino mass matrix on the amount of baryon asymmetry is explored in section VI, where we also propose two different models of mass matrices, which produce an enhancement of the baryon asymmetry. Finally, in section VII we summarize the subject.

## II The effective neutrino mass matrix

The lepton mixing matrix $`U`$ (called the Maki-Nakagawa-Sakata (MNS) matrix ), which relates mass eigenstates to flavor eigenstates, by means of the unitary transformation $`\nu _\alpha =U_{\alpha i}\nu _i`$ ($`\alpha =e,\mu ,\tau `$; $`i=1,2,3`$), can be parametrized as the standard form of the Cabibbo-Kobayashi-Maskawa (CKM) quark mixing matrix  (including a phase $`\delta `$) times a diagonal phase matrix $`P=\text{diag}(\text{e}^{\text{i}\phi _1/2},\text{e}^{\text{i}\phi _2/2},1)`$. The two phases $`\phi _1`$ and $`\phi _2`$ are present only if the effective neutrino is a Majorana particle, and thus they are sometimes called the Majorana phases, in contrast with the phase $`\delta `$ which is called the Dirac phase. Moreover, contrary to quark mixings, which are small, lepton mixings can be large. In fact, the mixing of atmospheric neutrinos, related to $`U_{\mu 3}`$, is almost maximal, while the mixing of solar neutrinos, related to $`U_{e2}`$, may be large or small, although the large mixing is favoured . In the case of double large mixing, the lepton mixing matrix is given by

$$U\left(\begin{array}{ccc}\frac{1}{\sqrt{2}}& \frac{1}{\sqrt{2}}& ϵ\\ \frac{1}{2}(1+ϵ)& \frac{1}{2}(1ϵ)& \frac{1}{\sqrt{2}}\\ \frac{1}{2}(1ϵ)& \frac{1}{2}(1+ϵ)& \frac{1}{\sqrt{2}}\end{array}\right),$$
(1)
while for single large mixing it is given by

$$U\left(\begin{array}{ccc}1& 0& ϵ\\ \frac{ϵ}{\sqrt{2}}& \frac{1}{\sqrt{2}}& \frac{1}{\sqrt{2}}\\ \frac{ϵ}{\sqrt{2}}& \frac{1}{\sqrt{2}}& \frac{1}{\sqrt{2}}\end{array}\right),$$
(2)
see for example Ref.. As we said, the double large mixing is favoured. The mixing $`ϵ=U_{e3}`$ is very small, $`ϵ0.1`$, according to the result of the Chooz experiment . If we call $`D_L`$ the diagonal matrix of light neutrino masses,

$$D_L=\text{diag}(m_1,m_2,m_3),$$
(3)
then, in the basis where the charged lepton mass matrix is diagonal, $`M_e=D_e`$, we get

$$M_L=UD_LU^T.$$
(4)
In the flavor basis we have $`M_L=U_eUD_LU^TU_e^T`$, where $`U_e`$ diagonalizes $`M_e`$ by $`U_e^TM_eU_e`$, where $`M_e`$ is here supposed to be symmetric. We do not consider general phases in $`U`$. However, we allow the masses $`m_1`$ and $`m_2`$ to be both positive and negative, corresponding to phases $`\phi _{1,2}=0`$ and $`\phi _{1,2}=\pi `$, respectively, in the lepton mixing matrix. In such a way, we consider the two extreme cases for $`\phi _{1,2}`$ and the general case should be intermediate between them.

Let us call the elements of $`M_L`$ as $`M_{\alpha \beta }`$ with $`\alpha =e,\mu ,\tau `$ and $`\beta =e,\mu ,\tau `$. Then, for the double large mixing we have the approximate expressions

$$M_{ee}=\frac{m_1}{2}+\frac{m_2}{2}+ϵ^2m_3$$

$$M_{e\mu }=\frac{m_1}{2\sqrt{2}}(1+ϵ)+\frac{m_2}{2\sqrt{2}}(1ϵ)+\frac{ϵm_3}{\sqrt{2}}$$

$$M_{e\tau }=\frac{m_1}{2\sqrt{2}}(1ϵ)\frac{m_2}{2\sqrt{2}}(1+ϵ)+\frac{ϵm_3}{\sqrt{2}}$$

$$M_{\mu \mu }=\frac{m_1}{4}(1+ϵ)^2+\frac{m_2}{4}(1ϵ)^2+\frac{m_3}{2}$$

$$M_{\mu \tau }=\frac{m_1}{4}(1ϵ^2)\frac{m_2}{4}(1ϵ^2)+\frac{m_3}{2}$$

$$M_{\tau \tau }=\frac{m_1}{4}(1ϵ)^2+\frac{m_2}{4}(1+ϵ)^2+\frac{m_3}{2}$$

and for the single large mixing the corresponding expressions

$$M_{ee}=m_1+ϵ^2m_3$$

$$M_{e\mu }=\frac{ϵm_1}{\sqrt{2}}+\frac{ϵm_3}{\sqrt{2}}$$

$$M_{e\tau }=\frac{ϵm_1}{\sqrt{2}}+\frac{ϵm_3}{\sqrt{2}}$$

$$M_{\mu \mu }=\frac{ϵ^2m_1}{2}+\frac{m_2}{2}+\frac{m_3}{2}$$

$$M_{\mu \tau }=\frac{ϵ^2m_1}{2}\frac{m_2}{2}+\frac{m_3}{2}$$

$$M_{\tau \tau }=\frac{ϵ^2m_1}{2}+\frac{m_2}{2}+\frac{m_3}{2}.$$

Of course, both $`M_L`$ and $`M_R`$ are symmetric matrices. Note that for the single large mixing we have $`M_{e\mu }=M_{e\tau }`$ and $`M_{\mu \mu }=M_{\tau \tau }`$.

The element $`M_{ee}=U_{ei}^2m_i`$ is involved in the neutrinoless double beta decay. The experimental upper bound for $`|M_{ee}|`$, obtained from non observation of the process, is $`|M_{ee}|<0.38h`$ , where the factor $`h=0.62.8`$  is present because of the uncertainty in the calculation of the nuclear matrix element. The recent positive evidence  is controversial  and is not used here. The parameter $`M_{ee}`$ is the unique element in $`M_L`$ which can be tested in a direct way, because for the other elements the theoretical prediction is very much below the experimental data .

From the study of oscillations of atmospheric and solar neutrinos we know that $`|m_3^2m_2^2||m_2^2m_1^2|`$, so that there are three main mass spectra for the light neutrino, the normal hierarchy $`m_3^2m_2^2,m_1^2`$, the inverse hierarchy $`m_1^2m_2^2m_3^2`$, and the nearly degenerate spectrum $`m_1^2m_2^2m_3^2`$ (see Ref.). In particular, from atmospheric oscillations we get $`|m_3^2m_2^2|10^3`$ eV2, while from solar oscillations $`|m_2^2m_1^2|10^5`$ eV2 for the large mixing and $`|m_2^2m_1^2|10^6`$ eV2 for the small mixing. Then, with the normal hierarchy we get $`m_3^210^3`$ eV2, and with the inverse hierarchy $`m_{1,2}^210^3`$ eV2. For the nearly degenerate spectrum we expect $`m_{1,2,3}^2`$ around 1 eV2. In fact, the experimental upper bound on the parameter $`m_{\nu _e}=(U_{ei}^2m_i^2)^{1/2}`$, obtained from the endpoint energy of electrons in single beta decay, is $`m_{\nu _e}<2.5`$ eV . In contrast with $`M_{ee}`$, in $`m_{\nu _e}`$ cancellations cannot occur. If for the normal hierarchy also $`m_2^2m_1^2`$ is assumed, then we get $`m_2^210^5`$ eV2 for the large mixing and $`m_2^210^6`$ eV2 for the small mixing.

As a matter of fact, for solar neutrinos there are at least three oscillation solutions with a large mixing angle : the large mixing angle (LMA) matter oscillation with $`|m_2^2m_1^2|10^5`$ eV2, the low-mass (LOW) matter oscillation with $`|m_2^2m_1^2|10^7`$ eV2, and the vacuum oscillation (VO) with $`|m_2^2m_1^2|10^{10}`$ eV2. In our paper we refer mainly to the LMA solution which is the most favoured .

Zeroth order forms for $`M_L`$ can be obtained by setting $`ϵ=0`$ and $`(m_1,m_2,m_3)`$ equal to $`(0,0,1)`$, $`(1,1,0)`$, $`(1,1,0)`$, $`(1,1,1)`$, $`(1,1,1)`$, $`(1,1,1)`$, $`(1,1,1)`$. We call these mass spectra $`A`$, $`B_1`$, $`B_2`$, $`C_0`$, $`C_1`$, $`C_2`$, $`C_3`$, respectively. Of course, type $`A`$ is the normal hierarchy, type $`B`$ the inverse hierarchy, and type $`C`$ nearly degenerate. In the present paper we write only the zeroth order form of $`M_L`$, although we study the full element $`M_{ee}`$. For a more detailed description of $`M_L`$ see for example .

## III Seesaw mechanism

Since in the standard model with right-handed neutrinos the Dirac neutrino mass is generated on the same footing as the up quark masses, and the charged lepton masses on the same footing as the down quark masses, for the corresponding mass matrices we first assume the relations $`M_\nu M_u`$ and $`M_eM_d`$, that is

$$M_\nu \text{diag}(m_u,m_c,m_t),$$
(5)
and $`M_e\text{diag}(m_e,m_\mu ,m_\tau )`$, where we neglect the small Dirac lepton mixing analogous to the quark mixing. In other terms, we set $`U_e𝟙`$ and $`U_\nu 𝟙`$, where $`U_e`$ diagonalizes $`M_e`$ and $`U_\nu `$ diagonalizes $`M_\nu `$. As a matter of fact, the mass hierarchy of charged leptons is similar to that of down quarks. See Ref. for a summary on quark and lepton mass matrices. Because of the seesaw mechanism, we have

$$M_LM_\nu M_R^1M_\nu ,$$
(6)
where the heavy neutrino masses $`M_1`$, $`M_2`$, $`M_3`$ have to be much larger than the elements of the matrix $`M_\nu `$, which is here supposed to be symmetric. Inverting such a formula, the heavy neutrino mass matrix can be achieved,

$$M_RM_\nu M_L^1M_\nu .$$
(7)
Since $`M_L^1=UD_L^1U^T`$, we can get $`M_L^1`$ from $`M_L`$ by changing $`m_i`$ with $`1/m_i`$ in (4). We must keep in mind that for the inverse hierarchy the seesaw mechanism implies a cancellation of the Dirac hierarchy for the third and second generation, and for the nearly degenerate spectrum also with the first generation, which seems unnatural, especially for the VO solution. Note that for zero mixing we have $`m_1=m_u^2/M_1`$, $`m_2=m_c^2/M_2`$ and $`m_3=m_t^2/M_3`$.

In this paper we follow a kind of inverse or bottom-up approach, namely we will determine $`M_R`$ from $`M_L`$ (and $`M_\nu `$). In the alternative direct or top-down approach both $`M_R`$ and $`M_\nu `$ are obtained from a theoretical framework and the inferred $`M_L`$ is matched to neutrino phenomenology. We would like to stress that for any precise choice of $`M_\nu `$ we can reproduce any form of $`M_L`$ by adjusting $`M_R`$. However, as we will see in the following sections, the form of both $`M_\nu `$ and $`M_R`$ has a crucial impact on the amount of baryon asymmetry generated in the leptogenesis mechanism, so that we can use the constraint from baryogenesis to study $`M_R`$ and $`M_\nu `$. Unfortunately, we are not yet able to determine the mass matrices with precision. Nevertheless, some important considerations, involving also the neutrinoless double beta decay, can be done and this would be a central issue of our paper.

## IV Baryogenesis from leptogenesis

A baryon asymmetry can be generated from a lepton asymmetry . In fact, this lepton asymmetry is produced by the out-of-equilibrium CP-violating decay of heavy neutrinos. The electroweak sphalerons , which violates $`B+L`$ but conserve $`BL`$, transform part of this asymmetry into a baryon asymmetry. Then, the baryon asymmetry, defined as $`Y_B=(n_Bn_{\overline{B}})/7n_\gamma =\eta /7`$, where $`n_B`$, $`n_{\overline{B}}`$, $`n_\gamma `$ are number densities and $`\eta `$ is the baryon-to-photon ratio, can be written as (see Ref. and references therein)

$$Y_B\frac{1}{2}\frac{1}{g^{}}dϵ_1,$$
(8)
with the CP violating asymmetry in the decay of the lightest heavy neutrino with mass $`M_1M_2<M_3`$ given by

$$ϵ_1\frac{3}{16\pi v^2}\left[\frac{[(M_D^{}M_D)_{12}]^2}{(M_D^{}M_D)_{11}}\frac{M_1}{M_2}+\frac{[(M_D^{}M_D)_{13}]^2}{(M_D^{}M_D)_{11}}\frac{M_1}{M_3}\right],$$
(9)
where $`M_D=M_\nu U_R`$ and $`U_R^TM_RU_R=D_R`$, and $`vm_t`$ is the VEV of the Higgs doublet. The lightest heavy neutrino is in equilibrium during the decays of the two heavier ones, washing out the lepton asymmetry generated by them. The factor 1/2 represents the part of the lepton asymmetry converted into a baryon asymmetry . The parameter $`g^{}100`$ is the number of light degrees of freedom in the theory. Finally, the quantity $`d`$ is a dilution factor which mostly depends on the mass parameter

$$\stackrel{~}{m}_1=\frac{(M_D^{}M_D)_{11}}{M_1},$$
(10)
although for high values of $`\stackrel{~}{m}_1`$ some dependence on $`M_1`$ shows up. Minor dilution, $`d`$ of order $`10^1`$, is obtained for $`10^5\text{eV}\stackrel{~}{m}_110^2\text{eV}`$, while outside this range the dilution grows (that is $`d`$ diminishes) . In fact, if $`\stackrel{~}{m}_1`$ is too small, it is not possible to produce a sufficient number of heavy neutrinos at high temperature, while if $`\stackrel{~}{m}_1`$ is too large, the washout effect of lepton number violating scatterings is too strong and destroys the generated asymmetry. In order to be consistent with primordial nucleosynthesis, the baryon asymmetry $`Y_B`$ must be in the range $`10^{11}10^{10}`$ . At best, $`Y_B`$ is smaller than $`ϵ_1`$ by three orders of magnitude. It is clear that when we obtain $`M_R`$ from $`M_\nu `$ and $`M_L`$ through the inverse seesaw formula (7), a determination of the baryon asymmetry is also achieved. In the following two sections we try a partial selection of mass matrices using the bound on the amount of baryon asymmetry.

## V Symplified analysis of the seesaw mechanism

In this section we determine the structure of the heavy neutrino mass matrix by assuming the diagonal form (5) for the Dirac neutrino mass matrix. In the next section the effect of a non diagonal form is discussed. In any subsection we write the zeroth order form of $`M_L`$, according to the three possible hierarchies of light neutrino masses, and then we study the heavy neutrino mass matrix and the implications for the baryon asymmetry and for the neutrinoless double beta decay.

### A Normal hierarchy

For both kinds of mixings the zeroth order form for $`M_L`$ is given by

$$M_L\left(\begin{array}{ccc}0& 0& 0\\ 0& \frac{1}{2}& \frac{1}{2}\\ 0& \frac{1}{2}& \frac{1}{2}\end{array}\right)m_3,$$
(11)
where a dominant block in the $`\mu \tau `$ sector appears (see for example ). The overall scale is $`m_310^210^1`$ eV.

#### 1 Double large mixing

If there is full hierarchy of light neutrino masses, $`m_3m_2m_1`$, the matrix $`M_L^1`$ takes a nearly democratic form , so that

$$M_R\left(\begin{array}{ccc}m_u^2& m_um_c& m_um_t\\ m_um_c& m_c^2& m_cm_t\\ m_um_t& m_cm_t& m_t^2\end{array}\right)\frac{1}{m_1}.$$
(12)
This matrix is diagonalized by the rotation

$$U_R\left(\begin{array}{ccc}1& \frac{m_u}{m_c}& \frac{m_u}{m_t}\\ \frac{m_u}{m_c}& 1& \frac{m_c}{m_t}\\ \frac{m_u}{m_t}& \frac{m_c}{m_t}& 1\end{array}\right),$$
(13)
with eigenvalues $`M_1m_u^2/m_1`$, $`M_2m_c^2/m_1`$, $`M_3m_t^2/m_1`$. For the full hierarchy and large solar mixing we have $`m_2^210^5`$ eV2, hence $`m_110^3`$ eV and the overall scale is $`M_Rm_t^2/m_110^{16}`$ GeV. In this section we take the full hierarchy as reference model.

If $`m_3m_2m_1`$ (partially degenerate spectrum), we get

$$M_R\left(\begin{array}{ccc}m_u^2& ϵm_um_c& ϵm_um_t\\ ϵm_um_c& m_c^2& m_cm_t\\ ϵm_um_t& m_cm_t& m_t^2\end{array}\right)\frac{1}{m_1},$$
(14)
where very small elements $`M_{R12}`$ and $`M_{R13}`$ appear. The overall scale and the heavy masses do not change with respect to the full hierarchical case.

The further condition $`m_2<0`$ leads to the form

$$M_R\left(\begin{array}{ccc}ϵ^2\frac{m_1}{m_3}m_u^2& m_um_c& m_um_t\\ m_um_c& (\frac{m_1}{m_3}+ϵ)m_c^2& \frac{m_1}{m_3}m_cm_t\\ m_um_t& \frac{m_1}{m_3}m_cm_t& (\frac{m_1}{m_3}ϵ)m_t^2\end{array}\right)\frac{1}{m_1}.$$
(15)
If $`m_1ϵm_3`$ the overall scale is lowered to $`M_Rm_t^2/m_310^{14}`$ GeV. For $`m_1ϵm_3`$ we get the interesting form

$$M_R\left(\begin{array}{ccc}ϵ^3m_u^2& m_um_c& m_um_t\\ m_um_c& 2ϵm_c^2& ϵm_cm_t\\ m_um_t& ϵm_cm_t& 0\end{array}\right)\frac{1}{m_1}.$$
(16)
We stress the sharp difference between matrix (16) and matrices (12) or (14). While in (12) and (14) the largest element is $`M_{R33}`$, in (16) it is $`M_{R13}`$ or $`M_{R23}`$. In the first case the structure of $`M_R`$ is roughly similar to the Dirac neutrino mass matrix $`M_\nu `$ in (5), that is a nearly diagonal form. In the second case, $`M_R`$ is roughly offdiagonal. As a consequence, also the overall mass scale is different, $`m_t^2/m_110^{16}`$ GeV for the nearly diagonal form and $`m_um_t/m_110^{11}`$ GeV for the nearly offdiagonal form.

Let us discuss the implications for the baryogenesis via leptogenesis and for the neutrinoless double beta decay. The baryon asymmetry for the full normal hierarchy is $`Y_B10^{16}`$. Due to the suppression of $`M_{R12}`$ and $`M_{R13}`$, the baryon asymmetry is smaller in the partially degenerate spectrum than in the full normal hierarchy. In fact, the relation $`M_D^{}M_DM_Rm_1`$ holds in both cases, since $`M_D^{}M_D`$ and $`M_Rm_1`$ are diagonalized by the same $`U_R`$ with the same eigenvalues. For the nearly offdiagonal form, the baryon asymmetry is enhanced to a sufficient level, due to the moderate hierarchy within $`M_R`$. In the neutrinoless double beta decay we get $`M_{ee}m_210^310^2`$ eV if $`m_2>0`$, and $`M_{ee}(m_2^2m_1^2)/m_{1,2}10^410^3`$ eV if $`m_2<0`$. Therefore, the nearly offdiagonal form for $`M_R`$ tends to enhance $`Y_B`$ but to suppress $`M_{ee}`$. Thus we have found that if the diagonal $`M_\nu `$ in (5) is assumed then in order to get sufficient baryon asymmetry the matrix $`M_R`$ must be roughly offdiagonal, which leads to a negative $`m_2`$ and an approximate prediction for $`M_{ee}`$, smaller by one order with respect to the case of a roughly diagonal $`M_R`$. The negative value for $`m_2`$ could be an indication that phases $`\phi _1`$ and $`\phi _2`$ are very different from each other.

#### 2 Single large mixing

In this case we have the form

$$M_R\left(\begin{array}{ccc}m_u^2& ϵm_um_c& ϵm_um_t\\ ϵm_um_c& (\frac{m_1}{m_2}+ϵ^2)m_c^2& (\frac{m_1}{m_2}ϵ^2)m_cm_t\\ ϵm_um_t& (\frac{m_1}{m_2}ϵ^2)m_cm_t& (\frac{m_1}{m_2}+ϵ^2)m_t^2\end{array}\right)\frac{1}{m_1},$$
(17)
and for $`m_1ϵ^2m_2`$ the structure of $`M_R`$ is nearly diagonal with the overall scale given by $`m_t^2/m_210^{15}`$ GeV. The baryon asymmetry can get a moderate enhancement, because of the factor $`m_2/m_1`$, and $`M_{ee}m_110^4`$ eV. If $`m_1ϵ^2m_2`$ the element $`M_{R23}`$ vanishes. For $`m_2<0`$ and $`m_1ϵ^2m_2`$, we have a vanishing $`M_{R33}`$, leading to a nearly offdiagonal form at a lower scale.

### B Inverse hierarchy

The zeroth order form of $`M_L`$ for spectrum $`B_1`$ is given by

$$M_L\left(\begin{array}{ccc}0& \frac{1}{\sqrt{2}}& \frac{1}{\sqrt{2}}\\ \frac{1}{\sqrt{2}}& 0& 0\\ \frac{1}{\sqrt{2}}& 0& 0\end{array}\right)m_1,$$
(18)
$$M_L\left(\begin{array}{ccc}1& 0& 0\\ 0& \frac{1}{2}& \frac{1}{2}\\ 0& \frac{1}{2}& \frac{1}{2}\end{array}\right)m_1,$$
(19)
according to the double or single large mixing. For spectrum $`B_2`$ we have

$$M_L\left(\begin{array}{ccc}1& 0& 0\\ 0& \frac{1}{2}& \frac{1}{2}\\ 0& \frac{1}{2}& \frac{1}{2}\end{array}\right)m_1$$
(20)
for both cases. The overall scale is $`m_{1,2}10^210^1`$ eV. This is the same as the normal hierarchy, because both are determined by atmospheric oscillations.

#### 1 Double large mixing

The heavy neutrino mass matrix for spectrum $`B_2`$ is given by

$$M_R\left(\begin{array}{ccc}\frac{m_3}{m_1}m_u^2& ϵm_um_c& ϵm_um_t\\ ϵm_um_c& m_c^2& m_cm_t\\ ϵm_um_t& m_cm_t& m_t^2\end{array}\right)\frac{1}{m_3}.$$
(21)
If $`m_2<0`$, spectrum $`B_1`$, we get

$$M_R\left(\begin{array}{ccc}ϵ^2m_u^2& \frac{m_3}{m_1}m_um_c& \frac{m_3}{m_1}m_um_t\\ \frac{m_3}{m_1}m_um_c& m_c^2& m_cm_t\\ \frac{m_3}{m_1}m_um_t& m_cm_t& m_t^2\end{array}\right)\frac{1}{m_3}.$$
(22)
In the inverse pattern there is stronger hierarchy in $`M_R`$ with respect to the full normal pattern, see the first row and column in (21) and (22). The form of $`M_R`$ is always nearly diagonal and the offdiagonal form cannot be realized. The mass scale is $`M_Rm_t^2/m_310^{15}`$ GeV. Note also the difference between the inverse hierarchy and the partially degenerate spectrum in the element $`M_{R11}`$, which is responsible for the inversion of the light neutrino masses $`m_{1,2}`$ and $`m_3`$.

As a consequence, the baryon asymmetry is even smaller than in the partially degenerate spectrum with $`m_2>0`$, because also $`M_1`$ is suppressed. Instead, the rate for neutrinoless double beta decay, related to $`M_{ee}`$, can be enhanced. In particular, for spectrum $`B_2`$ we have $`M_{ee}m_{1,2}10^210^1`$ eV, which is by one order higher than for the full normal hierarchy. For spectrum $`B_1`$ we obtain $`M_{ee}(m_2^2m_1^2)/m_{1,2}10^410^3`$ eV.

#### 2 Single large mixing

In this case the heavy neutrino mass matrix is the same as for the previous case with $`m_2>0`$, and $`M_{ee}m_110^210^1`$ eV.

### C Nearly degenerate spectrum

Here the light masses are around 1 eV. The zeroth order form of $`M_L`$ for spectrum $`C_0`$ is diagonal,

$$M_L\left(\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right)m_1,$$
(23)
for both kinds of mixing. For spectrum $`C_1`$ we have

$$M_L\left(\begin{array}{ccc}0& \frac{1}{\sqrt{2}}& \frac{1}{\sqrt{2}}\\ \frac{1}{\sqrt{2}}& \frac{1}{2}& \frac{1}{2}\\ \frac{1}{\sqrt{2}}& \frac{1}{2}& \frac{1}{2}\end{array}\right)m_1,$$
(24)
$$M_L\left(\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right)m_1,$$
(25)
and for spectrum $`C_2`$

$$M_L\left(\begin{array}{ccc}0& \frac{1}{\sqrt{2}}& \frac{1}{\sqrt{2}}\\ \frac{1}{\sqrt{2}}& \frac{1}{2}& \frac{1}{2}\\ \frac{1}{\sqrt{2}}& \frac{1}{2}& \frac{1}{2}\end{array}\right)m_1,$$
(26)
$$M_L\left(\begin{array}{ccc}1& 0& 0\\ 0& 0& 1\\ 0& 1& 0\end{array}\right)m_1,$$
(27)
according to the double or single large mixing. Finally, for spectrum $`C_3`$

$$M_L\left(\begin{array}{ccc}1& 0& 0\\ 0& 0& 1\\ 0& 1& 0\end{array}\right)m_1$$
(28)
for both kinds of mixing. The overall scale is $`m_{1,2,3}0.11`$ eV, determined by single beta decay. In the nearly degenerate spectrum several delicate cancellations among the terms of $`M_{ee}`$ may occur and our study of the heavy neutrino mass matrix is only indicative.

#### 1 Double large mixing

The heavy neutrino mass matrix for the spectra $`C_0`$, $`C_1`$, $`C_2`$, $`C_3`$ is respectively given by the forms

$$M_R\left(\begin{array}{ccc}m_u^2& \rho m_um_c& \rho m_um_t\\ \rho m_um_c& m_c^2& ϵ^2m_cm_t\\ \rho m_um_t& ϵ^2m_cm_t& m_t^2\end{array}\right)\frac{1}{m_1},$$
(29)
$$M_R\left(\begin{array}{ccc}ϵ^2m_u^2& m_um_c& m_um_t\\ m_um_c& m_c^2& m_cm_t\\ m_um_t& m_cm_t& m_t^2\end{array}\right)\frac{1}{m_1},$$
(30)
$$M_R\left(\begin{array}{ccc}ϵ^2m_u^2& m_um_c& m_um_t\\ m_um_c& m_c^2& m_cm_t\\ m_um_t& m_cm_t& m_t^2\end{array}\right)\frac{1}{m_1},$$
(31)
$$M_R\left(\begin{array}{ccc}m_u^2& ϵm_um_c& ϵm_um_t\\ ϵm_um_c& ϵ^2m_c^2& m_cm_t\\ ϵm_um_t& m_cm_t& ϵ^2m_t^2\end{array}\right)\frac{1}{m_1}.$$
(32)
In (29), and (34) below, we write $`\rho `$ whenever a cancellation at the level of $`ϵ`$ occurs. We see that matrices (29), (30), (31) are roughly close to the diagonal form, while (32) is not. The overall mass scale is $`M_R10^{13}`$ GeV.

For spectra $`C_0`$ and $`C_3`$ the baryon asymmetry is generally suppressed or much suppressed with respect to the full normal hierarchy, while for spectra $`C_1`$ and $`C_2`$ it is comparable. Instead, the rate for neutrinoless double beta decay can be further enhanced with respect to the inverse hierarchy. In particular, for spectra $`C_0`$ and $`C_3`$ we have $`M_{ee}m_10.11`$ eV, by one order higher than for the spectrum $`B_2`$. For spectra $`C_1`$ and $`C_2`$ we have cancellations leading to $`M_{ee}10^510^4`$ eV.

#### 2 Single large mixing

For spectra $`C_0`$ and $`C_3`$ we have the same matrices as for the previous mixing. For spectra $`C_1`$ and $`C_2`$ we get

$$M_R\left(\begin{array}{ccc}m_u^2& ϵm_um_c& ϵm_um_t\\ ϵm_um_c& m_c^2& ϵ^2m_cm_t\\ ϵm_um_t& ϵ^2m_cm_t& m_t^2\end{array}\right)\frac{1}{m_1},$$
(33)
$$M_R\left(\begin{array}{ccc}m_u^2& \rho m_um_c& \rho m_um_t\\ \rho m_um_c& ϵ^2m_c^2& m_cm_t\\ \rho m_um_t& m_cm_t& ϵ^2m_t^2\end{array}\right)\frac{1}{m_1}.$$
(34)
In this two cases the baryon asymmetry is suppressed or much suppressed with respect to the full normal hierarchy, and $`M_{ee}0.11`$ eV.

## VI Seesaw mechanism and baryon asymmetry

In this section, instead of the diagonal form, we take the realistic mass matrices, expressed in terms of the Cabibbo parameter $`\lambda =0.22`$ and the overall mass scale,

$$M_e\left(\begin{array}{ccc}\lambda ^6& \lambda ^3& \lambda ^5\\ \lambda ^3& \lambda ^2& \lambda ^2\\ \lambda ^5& \lambda ^2& 1\end{array}\right)m_b,$$
(35)
$$M_\nu \left(\begin{array}{ccc}\lambda ^{12}& \lambda ^6& \lambda ^{10}\\ \lambda ^6& \lambda ^4& \lambda ^4\\ \lambda ^{10}& \lambda ^4& 1\end{array}\right)m_t.$$
(36)
These forms can be motivated by an $`U(2)`$ horizontal symmetry, see Ref. and references therein. Again we neglect $`U_e`$ with respect to $`U`$. However, the effect of $`U_\nu 𝟙`$ is crucial. Of course, $`M_L^1`$ can be obtained from the previous section by deleting the quark masses in $`M_R`$. In the following calculation we will assume that no cancellations occur between two quantities of the same order in $`\lambda `$.

For the full normal hierarchy and double large mixing we get

$$M_R\left(\begin{array}{ccc}\lambda ^{12}& \lambda ^{10}& \lambda ^6\\ \lambda ^{10}& \lambda ^8& \lambda ^4\\ \lambda ^6& \lambda ^4& 1\end{array}\right)\frac{m_t^2}{m_1},$$
(37)
diagonalized by

$$U_R\left(\begin{array}{ccc}1& \lambda ^2& \lambda ^6\\ \lambda ^2& 1& \lambda ^4\\ \lambda ^6& \lambda ^4& 1\end{array}\right),$$
(38)
with eigenvalues $`M_3m_t^2/m_1`$, $`M_2\lambda ^8M_3`$, $`M_1\lambda ^{12}M_3`$, and consistent with the $`U(2)`$ horizontal symmetry . The baryon asymmetry is enhanced with respect to the diagonal case but remains too small, $`Y_B10^{14}`$. In fact, the relation $`M_D^{}M_DM_Rm_1`$ holds, and one obtains

$$ϵ_1\frac{3}{16\pi }\left(\frac{\lambda ^{20}}{\lambda ^{12}}\lambda ^4+\frac{\lambda ^{12}}{\lambda ^{12}}\lambda ^{12}\right)10^{10},$$
(39)
and $`\stackrel{~}{m}_1m_1`$. The same $`M_R`$ and $`Y_B`$ come out for the partially degenerate spectrum with $`m_2>0`$, the inverse hierarchy, and the nearly degenerate spectra $`C_1`$ and $`C_2`$, although the scale of $`M_R`$ is changed accordingly.

For the partially degenerate spectrum with $`m_2<0`$, assuming for example both $`ϵ\lambda ^4`$ and $`M_{R33}\lambda ^{12}m_t^2/m_1`$, we obtain

$$M_R\left(\begin{array}{ccc}\lambda ^{16}& \lambda ^{12}& \lambda ^{10}\\ \lambda ^{12}& \lambda ^{10}& \lambda ^6\\ \lambda ^{10}& \lambda ^6& \lambda ^8\end{array}\right)\frac{m_t^2}{m_1}.$$
(40)
By considering $`M_R^{}M_R`$, one finds that $`M_R`$ is diagonalized by

$$U_R\left(\begin{array}{ccc}1& \lambda ^4& \lambda ^6\\ \lambda ^4& 1& \lambda ^2\\ \lambda ^6& \lambda ^2& 1\end{array}\right),$$
(41)
with eigenvalues $`M_3\lambda ^6m_t^2/m_1`$, $`M_2M_3`$, $`M_1\lambda ^4M_3`$. Then we obtain

$$ϵ_1\frac{3}{16\pi }\left(\frac{\lambda ^{16}}{\lambda ^{12}}\lambda ^4+\frac{\lambda ^{12}}{\lambda ^{12}}\lambda ^4\right)10^4,$$
(42)
and $`\stackrel{~}{m}_1\lambda ^2m_1`$, so that a sufficient amount of baryon asymmetry can be easily achieved. Note that here the dominant term in the leptogenesis formula is the second one. The mass scale in (37) is given by $`m_t^2/m_110^{16}`$ GeV, and in (40) by $`\lambda ^6m_t^2/m_110^{12}`$ GeV.

For spectrum $`C_0`$ we have the nearly diagonal form

$$M_R\left(\begin{array}{ccc}\lambda ^{12}& \lambda ^{10}& \lambda ^{10}\\ \lambda ^{10}& \lambda ^8& \lambda ^4\\ \lambda ^{10}& \lambda ^4& 1\end{array}\right)\frac{m_t^2}{m_1},$$
(43)
and for spectrum $`C_3`$

$$M_R\left(\begin{array}{ccc}\lambda ^{16}& \lambda ^{10}& \lambda ^6\\ \lambda ^{10}& \lambda ^8& \lambda ^4\\ \lambda ^6& \lambda ^4& \lambda ^4\end{array}\right)\frac{m_t^2}{m_1},$$
(44)
The baryon asymmetry is very small in the case $`C_0`$ and moderate in the case $`C_3`$.

For the single large mixing the results are roughly similar to the double large mixing. Few changes can be easily shown and are not discussed here. Thus we see that using non diagonal Dirac neutrino mass matrices and doing an order-of-magnitude analysis lead to few forms for the heavy neutrino mass matrix. Then the two questions of determining the symmetry generating mass matrices and discovering their coefficient should be addressed. For example, horizontal $`U(1)`$ or $`U(2)`$ symmetries can be used. This is a fundamental subject that we will try to discuss elsewhere. We need not start from matrices (35), (36) but other forms are possible as well .

A simple way to enhance the baryon asymmetry is by means of a quite moderate hierarchy in the Dirac neutrino mass matrix, that is for its eigenvalues and/or for its mixing angles . This is quite evident from the leptogenesis formula. For example, instead of (36), one can adopt

$$M_\nu \left(\begin{array}{ccc}\lambda ^6& \lambda ^3& \lambda ^5\\ \lambda ^3& \lambda ^2& \lambda ^2\\ \lambda ^5& \lambda ^2& 1\end{array}\right)m_t,$$
(45)
that is a matrix similar to charged lepton masses but with the same overall scale as up quark masses. Let us discuss this issue. In section III we have assumed $`M_eM_d`$ and $`M_\nu M_u`$. This is the simplest hypothesis within the standard model, and in the supersymmetric model can be motivated by the fact that the two pairs $`M_{e,d}`$ and $`M_{\nu ,u}`$ are generated by two distinct Higgs doublets. However, Yukawa couplings for the Dirac neutrino can be very different from the Yukawa couplings for the up quarks. If this case occurs, the hierarchy of masses and mixings in the Dirac neutrino sector can be very different from the mass hierarchy of up quarks and the CKM quark mixing, respectively. When we take matrices (35) and (45) we obtain

$$M_R\left(\begin{array}{ccc}\lambda ^6& \lambda ^5& \lambda ^3\\ \lambda ^5& \lambda ^4& \lambda ^2\\ \lambda ^3& \lambda ^2& 1\end{array}\right)\frac{m_t^2}{m_1},$$
(46)
$$ϵ_1\frac{3}{16\pi }\left(\frac{\lambda ^{10}}{\lambda ^6}\lambda ^2+\frac{\lambda ^6}{\lambda ^6}\lambda ^6\right)10^6,$$
(47)
and a sufficient amount of baryon asymmetry, $`Y_B10^{10}`$. Both the internal hierarchy and the overall scale of $`M_\nu `$ are important. In fact, if the overall scale in (45) is $`m_{b,\tau }`$ instead of $`m_t`$, the baryon asymmetry again suppressed . The relation $`M_\nu (\mathrm{tan}\beta )M_e`$ can be obtained within supersymmetric left-right models . For very large $`\mathrm{tan}\beta `$ we get a Dirac neutrino mass matrix similar to (45).

## VII Discussion

Assuming a nearly diagonal mass matrix for Dirac neutrinos, we have studied the structure of the Majorana mass matrix of right-handed neutrinos within the seesaw framework, and its implications for the baryogenesis via leptogenesis and for the neutrinoless double beta decay. Then we have explored the effect of a non diagonal mass matrix for Dirac neutrinos. In this context, we find few possibilities to obtain a sufficient level of baryon asymmetry. The only case when the asymmetry is large should be the nearly offdiagonal form of $`M_R`$. Usually, the behaviour of $`M_{ee}`$ is opposite to $`Y_B`$, namely when $`Y_B`$ is suppressed $`M_{ee}`$ is enhanced and viceversa. For the offdiagonal form we get $`M_{ee}10^410^3`$ eV. In the supersymmetric formula for the leptogenesis, the baryon asymmetry is only slightly enhanced . In fact, although there are new decay channels, also the washout process is stronger.

When a moderate hierarchy in $`M_\nu `$ is adopted, as in (45), then $`M_R`$ need not be close to the offdiagonal form. Therefore, instead of $`M_{ee}10^410^3`$ eV, we can yield the value $`M_{ee}10^310^2`$ eV for the normal hierarchy and $`M_{ee}10^210^1`$ eV for the inverse hierarchy. Note that these predictions cover three different ranges of values for $`M_{ee}`$, so that informations from the neutrinoless double beta decay could clarify the structure of fermion mass matrices, if the leptogenesis mechanism is valid.

As a conclusion, we find that, if $`Y_B`$ has to be within the allowed range, then retaining quark-lepton mass relations $`M_eM_d`$ and $`M_\nu M_u`$ leads to the roughly offdiagonal form for $`M_R`$ and the prediction $`M_{ee}10^410^3`$ eV. If $`M_\nu M_u`$ is not true, then $`M_R`$ can be roughly close to the diagonal form and $`M_{ee}`$ larger than $`10^3`$ eV, with $`Y_B`$ in the allowed range. This is the central result of our paper. We have proposed such two different kinds of model in the previous section.

It has been suggested  that the nearly offdiagonal form for $`M_R`$ is consistent with the nonsupersymmetric unified model $`SO(10)`$ with an intermediate symmetry breaking scale, where the heavy neutrino mass is generated, while the nearly diagonal form for $`M_R`$ is consistent with the supersymmetric version without such an intermediate scale, the heavy neutrino mass being generated at the unification scale. In both cases, quark-lepton symmetry is valid. Then, we ask if the framework described here could be embedded in unified models. On the one hand, the leptogenesis scenario could work within unified models . On the other hand, unified model have not yet received decisive support from the experimental detection of proton decay, so that we find attractive to keep the minimal scenario, as was the motivation of the original paper on the leptogenesis. Within a unified framework, the leptogenesis constraint favours the nonsupersymmetric model.

The present paper focuses on the general structure of mass matrices and does not exclude that possible fine tuning could produce a sufficient amount of baryon asymmetry , even with nearly diagonal mass matrices. Other interesting papers on the relation between leptogenesis and mass matrices are reported in Ref..

In our simple approach we have not considered the effect of running masses and mixings from the low scale to the high $`M_R`$ scale where the seesaw formula (6) applies. At our level of approximation, only for the supersymmetric version with large $`\mathrm{tan}\beta `$ they can be significant . In particular, for spectrum $`C_0`$ both the double and the single large mixing are converted to nearly zero mixing, and for spectra $`B_2`$ and $`C_3`$ the double large mixing is converted to single large mixing. Such spectra are characterized by degeneracy in mass and sign. However, it has been argued  that the running of vacuum expectation values could improve the stability of the lepton mixing matrix.


format: text

text:
# Solvable Quotients of Kähler Groups

## 1 . Introduction.

We first recall a definition from \[AN\].

###### Definition 1.1

A solvable group $`\mathrm{\Gamma }`$ has finite rank, if there is a decreasing sequence $`\mathrm{\Gamma }=\mathrm{\Gamma }_0\mathrm{\Gamma }_1\mathrm{}\mathrm{\Gamma }_{m+1}=\{1\}`$ of subgroups, each normal in its predecessor, such that $`\mathrm{\Gamma }_i/\mathrm{\Gamma }_{i+1}`$ is abelian and $`Q(\mathrm{\Gamma }_i/\mathrm{\Gamma }_{i+1})`$ is finite dimensional for all $`i`$.

In what follows $`_r`$ denotes a free group with the number of generators $`rZ_+\{\mathrm{}\}`$. Our main result is

###### Theorem 1.2

Let $`M`$ be a compact Kähler manifold. Assume that the fundamental group $`\pi _1(M)`$ is defined by the sequence

$$\{1\}F\pi _1(M)\stackrel{p}{}H\{1\}$$

where $`H`$ is a solvable group of finite rank of the form

$$\{0\}AHB\{0\}$$

with non-trivial abelian groups $`A,B`$ so that $`QAQ^m`$ and $`m1`$. Assume also that $`p^1(A)\pi _1(M)`$ does not admit a surjective homomorphism onto $`_{\mathrm{}}`$. Then all eigen-characters of the conjugate action of $`B`$ on the vector space $`QA`$ are torsion.

In Lemma 2.3 we will show that the condition for $`p^1(A)`$ holds if $`F`$ does not admit a surjective homomorphism onto $`_{\mathrm{}}`$.

Using Theorem 1.2 we prove a result on solvable quotients of Kähler groups.

###### Theorem 1.3

Assume that a Kähler group $`G`$ is defined by the sequence

$$\{1\}FG\stackrel{q}{}H\{1\}$$

where $`F`$ does not admit a surjective homomorphism onto $`_{\mathrm{}}`$ and $`H`$ is a solvable group of finite rank. Then there exist normal subgroups $`H_1H_2`$ of $`H`$ so that

$`H_1`$ has finite index in $`H`$;

$`H_1/H_2`$ is nilpotent, and

$`H_2`$ is torsion.

###### Remark 1.4

(1) Clearly the conclusion of Theorem 1.2 is valid for $`H`$ being an extension of $`Z^n`$ by $`Z^m`$, $`n,m1`$, and $`F`$ being a finitely generated group. Assume that not all eigen-characters of the action of $`Z^n`$ on $`QZ^m`$ are torsion. Then a semidirect product of such $`F`$ and $`H`$ (where $`F`$ is a normal subgroup of this product) is not a Kähler group.
(2) Let $`G`$ be a Kähler group. By $`DG=D^1G`$ we denote the derived subgroup of $`G`$, and set $`D^iG=DD^{i1}G`$. Assume that $`H:=G/D^nG`$, $`n1`$, is a solvable group of finite rank. Then it was proved in \[AN, Th. 4.9\] and \[Ca, Th. 2.2\] that $`H`$ satisfies conditions (a)-(c) of Theorem 1.3. It is a consequence of the fact that $`G`$ does not admit a surjective homomorphism onto $`_r`$ with $`2r<\mathrm{}`$.

## 2 . Proof of Theorem 1.2.

In what follows $`T_n(C)GL_n(C)`$ denotes the Lie group of upper triangular matrices. Let $`T_2T_2(C)`$ be the Lie group of matrices of the form

$$\left(\begin{array}{cc}a& b\\ 0& 1\end{array}\right)(aC^{},bC),$$

$`D_2T_2`$ and $`N_2T_2`$ be the groups of diagonal and unipotent matrices. Let $`M`$ be a compact Kähler manifold. For a homomorphism $`\rho Hom(\pi _1(M),T_2)`$ we let $`\rho _aHom(\pi _1(M),C^{})`$ denote the upper diagonal character of $`\rho `$. The main result used in our proofs is the following

###### Proposition 2.1

Assume that $`\pi _1(M)`$ is defined by the sequence

$$\{1\}F\pi _1(M)H\{1\}$$

where the normal subgroup $`F`$ does not admit a surjective homomorphism onto $`_{\mathrm{}}`$. Assume that $`\rho Hom(\pi _1(M),T_2)`$ satisfies $`FKer(\rho _a)`$ but $`FKer(\rho )`$. Then $`\rho _a`$ is a torsion character.

Proof. Given a character $`\xi Hom(\pi _1(M),C^{})`$, let $`C_\xi `$ denote the associated $`\pi _1(M)`$-module. We define $`\mathrm{\Sigma }^1(M)`$ to be the set of characters $`\xi `$ such that $`H^1(\pi _1(M),C_\xi )`$ is nonzero. The structure of $`\mathrm{\Sigma }^1(M)`$ was described in the consequent papers of Beauville \[Be\], Simpson \[S\], Campana \[Ca\].
BSC Theorem. There is a finite number of surjective holomorphic maps with connected fibres $`f_i:MC_i`$ onto smooth compact complex curves of genus $`1`$ and torsion characters $`\rho _i,\xi _jHom(\pi _1(M),C^{})`$ such that

$$\mathrm{\Sigma }^1(M)=\underset{i}{}\rho _if_i^{}Hom(\pi _1(C_i),C^{})\underset{j}{}\{\xi _j\}.$$

Further, the group $`N_2`$ acts on $`T_2`$ by conjugation. Any two homomorphisms from $`Hom(\pi _1(M),T_2)`$ belonging to the orbit of this action will be called equivalent.

Let $`\rho Hom(\pi _1(M),T_2)`$ satisfy the conditions of Proposition 2.1. Then it is well known that the class of equivalence of $`\rho `$ is uniquely defined by an element $`c_\rho H^1(\pi _1(M),C_{\rho _a})`$ (see e.g. \[A, Prop. 2\]). In particular, if $`c_\rho =0`$, $`\rho `$ is equivalent to a representation into $`D_2`$. In our case, $`c_\rho 0`$ because $`FKer(\rho )`$. Thus $`\rho _a`$ satisfies the conditions of BSC Theorem. If $`\rho _a`$ coincides with one of $`\xi _j`$ then it is torsion by the above theorem. So assume that $`\rho _a=\rho _if_i^{}\varphi `$ for some torsion character $`\rho _i`$ and $`\varphi Hom(\pi _1(C_i),C^{})`$. Let $`K:=Ker(\rho _i)\pi _1(M)`$ and $`p:M_1M`$ be the Galois covering of $`M`$ corresponding to the finite abelian Galois group $`\pi _1(M)/K`$. Let $`M_1\stackrel{g}{}C\stackrel{h}{}C_i`$ be the Stein factorization of $`f_ip`$. Here $`g`$ is a morphism with connected fibres onto a smooth curve $`C`$ and $`h`$ is a finite morphism. Assume, to the contrary, that $`\rho _a`$ is not torsion. Then we prove

###### Lemma 2.2

$`FKer(g_{})`$.

Proof. Set $`G:=(f_i)_{}(F)\pi _1(C_i)`$. According to the assumptions of Proposition 2.1 we have either (a) $`G`$ is a subgroup of finite index in $`\pi _1(C_i)`$, or (b) $`G`$ is isomorphic to $`_r`$ with $`r<\mathrm{}`$. Let us consider (a). Since by definition $`FKer(\rho _a)`$ and $`\rho _i`$ is torsion, $`\varphi (G)`$ is a finite abelian group. Then $`G_1:=Ker\varphi G`$ is a subgroup of finite index in $`G`$. In particular, $`G_1`$ is a subgroup of finite index in $`\pi _1(C_i)`$ and so it is not free. But by our assumption, $`\varphi `$ is not torsion and so $`Ker(\varphi )_{\mathrm{}}`$. Thus $`G_1`$ is also free as a subgroup of $`_{\mathrm{}}`$. This shows that (a) is never happen.

Consider now (b). Using the fact that $`(f_i)_{}`$ is a surjection, we conclude that $`G`$ is a normal subgroup of $`\pi _1(C_i)`$. Let $`SC_i`$ be a regular covering corresponding to $`Q:=\pi _1(C_i)/G`$. If $`r2`$ then the group $`Iso(S)`$ of isometries of $`S`$ (with respect to the hyperbolic metric) is finite and since $`Q`$ is infinite we have $`r1`$. If $`r=1`$, any discrete subgroup of $`Iso(S)`$ is virtually cyclic and in particular does not act cocompactly on $`S`$. Thus $`r=0`$ which means that $`G=\{e\}`$ and $`FKer(f_i)_{}`$. Then the assumption of the proposition implies that $`FK=\pi _1(M_1)`$.

Now consider $`\stackrel{~}{\rho }:=\rho |_{\pi _1(M_1)}`$ with the upper diagonal character $`\stackrel{~}{\rho }_a:=\rho _a|_{\pi _1(M_1)}`$. Since $`\pi _1(M_1)\pi _1(M)`$ is a subgroup of finite index, $`\stackrel{~}{\rho }_a`$ is also not torsion. Set $`\stackrel{~}{\varphi }:=h^{}\varphi `$. Then $`\stackrel{~}{\rho }_a=g^{}\stackrel{~}{\varphi }`$. Let $`G_1:=g_{}(F)\pi _1(C)`$. Then the same argument for $`G_1`$ as above for $`G`$ (with $`\stackrel{~}{\rho }_a`$ and $`\stackrel{~}{\varphi }`$ instead of $`\rho _a`$ and $`\varphi `$) yields $`FKer(g_{})`$. $`\mathrm{}`$

According to Lemma 2.2 and the assumptions of Proposition 2.1 we have that $`\stackrel{~}{\rho }|_{Ker(g_{})}`$ is non-trivial and $`\stackrel{~}{\rho }_a`$ is the pullback of a character from $`Hom(\pi _1(C),C^{})`$. Then from \[Br, Proposition 3.6\] it follows that $`\stackrel{~}{\rho }_a`$ is torsion. Therefore $`\rho _a`$ is torsion, as well. This contradiction proves the proposition. $`\mathrm{}`$

We are ready to prove Theorem 1.2. According to the assumptions of the theorem there is a homomorphism $`i`$ of $`H`$ into the Lie group $`R`$ of the form

$$\{0\}C^mRB\{0\}$$

whose kernel is $`Tor(A)`$. Here we idenitify $`C^m`$ with $`CA`$. Consider the action $`s:BGL_m(C)`$ by conjugation. Since, by the definition of $`\pi _1(M)`$, $`B`$ is a finitely generated abelian group, $`s=_{j=1}^ds_j`$ where $`s_j`$ is equivalent to a nilpotent representation $`BT_{m_j}(C)`$ with a diagonal character $`\rho _j`$. Here $`_{j=1}^dm_j=m`$. From this decomposition it follows that there is an invariant $`B`$-submodule $`V_jC^m`$ of $`dim_CV_j=m1`$ such that $`W_j=C^m/V_j`$ is a one-dimensional $`B`$-module and the action of $`B`$ on $`W_j`$ is defined as multiplication by the character $`\rho _j`$. By definition, $`V_j`$ is a normal subgroup of $`R`$ and the quotient group $`R_j=R/V_j`$ is defined by the sequence

$$\{0\}CR_jB\{0\}.$$

Here the action of $`B`$ on $`C`$ is multiplication by the character $`\rho _j`$. (As before the associated $`B`$-module is denoted by $`C_{\rho _j}`$.) Let us denote by $`t_j`$ the composite homomorphism $`\pi _1(M)H\stackrel{i}{}RR_j`$. Further, the equivalence class of extensions of $`B`$ by $`C`$ isomorphic to $`R_j`$ is defined by an element $`c_jH^2(B,C_{\rho _j})`$. We assume that the character $`\rho _j`$ is non-trivial (for otherwise, $`\rho _j`$ is clearly torsion). Then $`H^2(B,C_{\rho _j})=0`$ (for the proof see e.g. \[AN, Lemma 4.2\]). This shows that $`R_j`$ is isomorphic to the semidirect product of $`C`$ and $`B`$, i.e., $`R_j=C\times B`$ with multiplication

$$(v_1,g_1)(v_2,g_2)=(v_1+\rho _j(g_1)v_2,g_1g_2),v_1,v_2C,g_1,g_2B.$$

Let us determine a map $`\varphi _j`$ of $`R_j`$ to $`T_2`$ by the formula

$$\varphi _j(v,g)=\left(\begin{array}{cc}\rho _j(g)& v\\ 0& 1\end{array}\right)$$

Obviously, $`\varphi _j`$ is a correctly defined homomorphism with upper diagonal character $`\rho _j`$. Hence $`\varphi _jt_j:\pi _1(M)T_2(C)`$ is a homomorphism non-trivial on $`p^1(A)\pi _1(M)`$ by its definition. Also $`p^1(A)Ker(\rho _jt_j)`$. Since by our assumptions $`p^1(A)`$ does not admit a surjective homomorphism onto $`_{\mathrm{}}`$, Proposition 2.1 applied to $`\varphi _jt_j`$ implies that $`\rho _j`$ is torsion. This completes the proof of the theorem. $`\mathrm{}`$

We prove now the following result.

###### Lemma 2.3

Assume that a group $`G`$ is defined by the sequence

$$\{1\}G_1GG_2\{1\}$$

where $`G_1,G_2`$ do not admit surjective homomorphisms onto $`_{\mathrm{}}`$. Then $`G`$ satisfies the similar property.

Proof of Lemma 2.3. Assume, to the contrary, that there is a surjective homomorphism $`\varphi :G_{\mathrm{}}`$. Then $`\stackrel{~}{G}_1:=\varphi (G_1)`$ is a normal subgroup of $`_{\mathrm{}}`$ and $`\stackrel{~}{G}_2:=_{\mathrm{}}/\stackrel{~}{G}_1`$ is a quotient of $`G_2`$. Now the assumption of the lemma implies that $`\stackrel{~}{G}_1_r`$ with $`r<\mathrm{}`$. Let $`X`$ be a complex hyperbolic surface with $`\pi _1(X)=_{\mathrm{}}`$ and $`SX`$ be the regular covering corresponding to $`\stackrel{~}{G}_2`$. Assume first that $`r1`$. Since $`\pi _1(S)=\stackrel{~}{G}_1`$, any subgroup of the group $`Iso(S)`$ of isometries of $`S`$ (with respect to the hyperbolic metric) is finitely generated. In particular $`\stackrel{~}{G}_2`$ is finitely generated (as well as $`\stackrel{~}{G}_1`$). This implies that $`_{\mathrm{}}`$ should be finitely generated which is wrong. Thus $`r=0`$ and $`\stackrel{~}{G}_2=_{\mathrm{}}`$. This contradicts to our assumption and shows that there is no such $`\varphi `$. $`\mathrm{}`$

## 3 . Proof of Theorem 1.4.

For a group $`L`$ set $`L^{ab}:=L/DL`$. We say that an $`L`$-module $`V`$ is quasi-unipotent if there is a subgroup $`L^{}L`$ of finite index whose elements act unipotently on $`V`$. To prove the theorem we will check the following condition from \[AN, Lemma 4.8\].

###### Lemma 3.1

Let $`H^{}H`$ be a subgroup of finite index. Then $`H^{}`$ acts quasi-unipotently on the finite-dimensional vector space $`Q(H^{}DH)^{ab}`$.

Proof. We set $`K:=H^{}/(H^{}DH)`$, $`G^{}:=q^1(H^{})`$, $`S:=D(H^{}DH)`$, and $`S^{}:=q^1(S)`$. Here $`K`$ is a finitely generated abelian group. Indeed, $`H^{}`$ is finitely generated as a subgroup of finite index of the finitely generated group $`H`$ ($`=`$ the image of the finitely generated group $`G`$). Thus $`K`$ is finitely generated as the image of $`H^{}`$. The fact that $`K`$ is abelian follows directly from the definition. Further, since $`H^{}`$ is a subgroup of finite index in $`H`$, $`G^{}`$ is a subgroup of finite index in $`G`$. In particular, it is Kähler. Moreover, we have

$$\{1\}S^{}G^{}L\{1\}$$

where $`L`$ is a solvable group of finite rank defined by the sequence

$$\{0\}(H^{}DH)^{ab}LK\{0\}.$$

Now the statement of the lemma is equivalent to the fact that all eigen-characters of the conjugate action of $`K`$ on $`Q(H^{}DH)^{ab}`$ are torsion. To prove that it suffices to show that $`S^{}`$ does not admit a surjective homomorphism onto $`_{\mathrm{}}`$, and then to apply Lemma 2.3 and Theorem 1.2.

Note that $`S^{}`$ is defined by the sequence

$$\{1\}FS^{}\stackrel{q}{}S\{1\}$$

where $`S`$ is a solvable group of finite rank. By our assumption $`F`$ does not admit a surjective homomorphism onto $`_{\mathrm{}}`$. Thus by Lemma 2.3, $`S^{}`$ satisfies the same property. $`\mathrm{}`$

Now Theorem 1.3 is the consequence of Lemma 3.1 and \[AN, Lemma 4.8\]. $`\mathrm{}`$


text:
# Meson production on complex nuclei by 𝜋⁻ with the Crystal Ball detector 11footnote 1Contribution to the International Workshop on Chiral Fluctuations in Hadronic Matter, Orsay, France, September 2001.

## 1 Introduction

The modification of hadron properties in nuclear matter is a major topic in nuclear studies. Of special importance is chiral restoration because it is a significant milestone on the long path which goes from quark confinement in nuclei at ordinary density and temperature to the high density and temperature of the quark–gluon plasma. Major changes are expected to occur for the $`\mathrm{f}_0(4001200)`$, or “$`\sigma `$” state, when it is produced on a nucleus even for low atomic number $`A`$, rather than on a nucleon. The $`\sigma `$ is the symbol for the correlated $`\pi \pi `$ system in the $`J=I=0`$ state. It plays an important role in the intermediate range $`NN`$ interaction . The quantum numbers of the $`\sigma `$ make it the chiral partner of the pion. Under chiral restoration, the $`\sigma `$ and $`\pi `$ should have degenerate masses, which implies that the $`\sigma `$ should have a large reduction in mass when approaching chiral restoration.

The interest in $`\sigma `$ medium modifications was fanned in a major way by a 1996 CHAOS publication  in which they reported on a measurement of the invariant mass spectrum of the $`\pi ^+\pi ^{}`$ system ($`m_{\pi ^+\pi ^{}}`$) that was produced by $`\pi ^+`$ interactions in complex nuclei. The $`m_{\pi ^+\pi ^{}}`$ spectrum near $`2m_\pi `$ was reported as “close to zero for $`A=2`$, and increases dramatically with increasing $`A`$”. A marked $`A`$–dependent peak at about $`2m_\pi `$ was interpreted by the CHAOS collaboration as follows: “the experimental results indicate that nuclear matter strongly modifies the $`\pi \pi `$ interaction in the $`J=I=0`$ channel”. The complete $`m_{\pi ^+\pi ^{}}`$ spectra for complex nuclei reported by CHAOS were stunningly similar to theoretical predictions of medium modifications made by Schuck et al.  and Chanfray et al. . It started a cascade of papers on $`\sigma `$ medium modifications  and played a prominent role in two recent workshops . There were no new experimental data until the Crystal Ball Collaboration made a careful search for a peak near $`2m_\pi `$ in the more favorable $`2\pi ^0`$ spectra produced by $`\pi ^{}`$ interactions. The four photons in the $`2\pi ^0`$ final state were detected with the Crystal Ball detector, which has a near $`4\pi `$ acceptance and detects $`\pi ^0`$’s down to zero kinetic energy. There was no indication of the CHAOS peak . In the meantime, Bonutti et al. hinted that the CHAOS peak was at least in part an artifact caused by the limited acceptance of the CHAOS detector .

The CHAOS Collaboration has shifted its interest to a change in the shape of the $`m_{\pi ^+\pi ^{}}`$ spectra obtained on complex targets . They proclaim new, different evidence for nuclear medium modifications from the shape of the ratio of $`m_{\pi ^+\pi ^{}}`$ for different targets. However, the difference in the slope of the $`m_{\pi ^+\pi ^{}}`$ spectra for different $`A`$ can be due to two very different mechanisms. One is the difference in interaction of the outgoing particles inside the nucleus. The other one is a generic change in the shape to lower $`m_{\pi \pi }`$ because of $`\sigma `$ medium modifications.

We present new data on $`2\pi ^0`$, $`\pi ^0`$, and $`\eta `$ production by $`\pi ^{}`$ of 750 MeV/$`c`$ that may help in investigating these two possibilities. Specifically, the following reactions have been studied: $`A(\pi ^{},\pi ^0\pi ^0)X^0`$, $`A(\pi ^{},\eta )X^0`$ (via the $`\eta \gamma \gamma `$ decay mode), and $`A(\pi ^{},\pi ^0)X^0`$, where $`A`$ is H, C, Al, or Cu. We will show experimental similarities and differences that have been found for the reactions above. Our principal focus will be on the reaction $`A(\pi ^{},\pi ^0\pi ^0)X`$.

## 2 Experimental method

Photons from $`\pi ^0`$ and $`\eta `$ decays were detected with the Crystal Ball (CB) multiphoton spectrometer (see Fig. 1). It is constructed of 672 optically isolated NaI(Tl) crystals that cover 93% of $`4\pi `$ sterad. Electromagnetic showers in the spectrometer are measured with an energy resolution $`\sigma _{s.d.}^E/E1.7\%/(E(\mathrm{GeV}))^{0.4}`$; the angular resolution for photon showers at energies 50–500 MeV is $`\sigma _{s.d.}^\theta =2^{}`$$`3^{}`$ in the polar angle and $`\sigma _{s.d.}^\varphi =2^{}/\mathrm{sin}\theta `$ in the azimuthal angle. Here $`\sigma _{s.d.}`$ is the standard deviation for a Gaussian distribution. The neutral final states were distinguished from charged ones by a veto barrel made of four plastic scintillation counters that covered the active volume of the CB. Most of our results presented here were obtained for the neutral final states. The charged final states were used to calculate the experimental corrections due to photon conversion and nuclei breakup.

The experiment was performed in the C6 beam line of the Brookhaven National Laboratory AGS. The centroid of the beam momentum is known to better than 1%. We used three solid targets: C (3.44 g/cm2), Al (1.69 g/cm2), and Cu (1.51 g/cm2). A 10 cm–long liquid hydrogen target (LH2) was used for the hydrogen data presented here.

The reaction $`A(\pi ^{},\pi ^0\pi ^0)X^0`$ was identified from events with four clusters. Two-cluster events were used to select the reactions $`A(\pi ^{},\eta )X^0`$ and $`A(\pi ^{},\pi ^0)X^0`$. See Refs.  for more details on the experimental apparatus, data analysis, and the Monte Carlo simulation for acceptance calculations. In order to facilitate comparisons for different targets, we calculated an “effective missing mass” for each nuclear target, given by

$$M_{\mathrm{target}}(A)=M(A,Z)M(A1,Z1),$$
(1)
where $`M(A,Z)`$ is the mass of the target nucleus, $`M(A1,Z1)`$ is the mass of the residual nucleus, and $`Z`$ is the atomic number of the target nucleus. Missing mass spectra for hydrogen as well as the effective missing mass spectra for nuclear targets were calculated for all three reactions assuming “quasifree” production.

## 3 Study of $`\pi ^0\pi ^0`$ production

The missing mass distributions of $`2\pi ^0`$ events obtained with the LH2 target as well as the spectra of the effective missing mass for different nuclear targets are presented in Fig. 2. A sharp neutron peak with width $`\sigma _{s.d.}35`$ MeV is seen in case of the hydrogen target. For the solid targets this peak broadens substantially. Some of the broadening is explained by the Fermi momentum of the target nucleons. Our Monte Carlo calculations indicate that the broadening is mainly due to nuclear effects such as $`\pi ^0`$ rescattering inside the nucleus. Our calculations of the pion rescattering and absorption used the model developed in Ref. . To select $`2\pi ^0`$ events with minimum rescattering distortions, we applied the effective-missing-mass cuts indicated in Fig. 2 by the two vertical dashed lines. The resulting $`2\pi ^0`$ invariant mass distributions, both with and without the missing mass cut, are given in Fig. 3. All spectra are compatible with a smooth distribution; i.e., there is no sharp peak. The broad maximum at $`m_{\pi ^0\pi ^0}510`$ MeV obtained for the hydrogen target disappears for carbon and other nuclear targets. Our invariant-mass distributions for C, Al, and Cu targets obtained without the cuts on the effective missing mass show broad maxima for $`2\pi ^0`$ masses below 400 MeV. The distributions obtained for C and Al become almost flat between 300 MeV and 480 MeV after the application of the effective-missing-mass cuts.

The kinetic-energy spectra of the $`\pi ^0\pi ^0`$ system are shown in Fig. 4 for our targets. The spectra differ markedly for various targets. The angular distributions for the $`2\pi ^0`$ system in the laboratory for different targets are shown in Fig. 4. The shapes of the angular distributions have similar features such as the broad maximum at about $`25^{}`$ for all targets including hydrogen. The minor increase in the broadness of the maxima as a function of $`A`$ can be explained by the Fermi momentum of the target protons. Neither the distribution of the kinetic energy nor the angular distributions were corrected for the Crystal Ball acceptance. The acceptance is a rather flat and smooth function of the kinetic energy and the lab angle.

To probe the net effect of the nuclear medium, we have investigated the cross sections as a function of $`A`$. To trace the disappearance of the high mass peak near 500 MeV in the invariant mass spectra, we have divided all distributions in Fig. 3 as indicated into a low–mass region (between 270 MeV and 445 MeV) and a high–mass region (above 445 MeV). The angular $`\mathrm{\Theta }_{lab}`$ distributions were used to calculate integrated cross sections, $`\sigma _{int}^{low}`$ and $`\sigma _{int}^{high}`$, for both mass regions as well as the total integrated cross section $`\sigma _{int}`$. Assuming that the nuclear dependence of the cross section is proportional to the geometrical factor $`\lambda _{geom}`$, where

$$\lambda _{geom}=A^{2/3}\frac{Z}{A}=ZA^{1/3},$$
(2)
we normalized our cross sections to $`\lambda _{geom}`$ and to the hydrogen cross section, $`\sigma _{int}`$(H), combining them into the ratio $`R`$:

$$R=\frac{\sigma (\pi ^{}A\pi ^0\pi ^0X)}{\sigma (\pi ^{}p\pi ^0\pi ^0n)}\times \frac{A^{1/3}}{Z}.$$
(3)
The results for $`\sigma _{int}^{low}`$, $`\sigma _{int}^{high}`$, and $`\sigma _{int}^{total}`$ for C, Al, and Cu are shown in Fig. 5 as a function of $`A^{2/3}`$. It can be seen that $`\sigma _{int}^{low}`$ remains rather constant and close to the expectation based on Eq. 2, which is shown in Fig. 5 by a straight line. In contrast, the $`\sigma _{int}^{high}`$ cross section falls off very rapidly from hydrogen to carbon; i.e., the maximum at about 510 MeV seen in the $`\pi ^0\pi ^0`$ invariant mass spectra (see Fig. 3) is absorbed rather than shifted toward lower masses. For higher $`A`$’s the $`\sigma _{int}^{high}`$ remains nearly constant. In summary, Fig. 5 shows that mainly high–mass $`2\pi ^0`$ events are absorbed by nuclear medium effects, whereas low mass events are less effected. In the case of $`\pi ^0\mathrm{\Delta }^0`$ production, the events may be absorbed via the $`\mathrm{\Delta }NNN`$ mechanism.

## 4 Study of $`\eta `$ production

Though $`\pi ^0`$ and $`\eta `$ mesons are similar in some aspects e.g., they both are pseudoscalar mesons with $`J^{PC}=0^+`$, they also have relevant differences in isospin and in quark composition. It is therefore very attractive to compare the behavior of $`\pi ^0`$ and $`\eta `$ mesons in nuclear matter. The theoretical interest has been discussed by Y. S. Golubeva et al. .

All $`\eta `$ results in this section are preliminary. The data were obtained from the $`\eta 2\gamma `$ decay mode. Figure 6 shows the $`2\gamma `$ invariant mass spectra for the H and Cu targets. The first peak in the spectrum is due to $`\pi ^0`$ (width $`\sigma _{s.d.}11.5`$ MeV), and the second peak is due to $`\eta `$ ($`\sigma _{s.d.}18.0`$ MeV). The background under the $`\pi ^0`$ and $`\eta `$ peaks is associated mainly with photon/neutron misidentification. In the case of copper, some additional background arises from products of nuclei breakup and recoil interactions.

The missing-mass and the effective-missing-mass distributions for $`\eta `$ production are shown in Fig. 7. A nearly background-free neutron peak can be seen for hydrogen. The widening of the peaks towards lower values of the effective missing mass for carbon and other nuclear targets is less dramatic than for the $`\pi ^0\pi ^0`$ case. It can be explained mainly by the nucleon Fermi momentum. It indicates that $`\eta `$’s are less affected by rescattering than pions. A similar behavior is observed in the $`\eta `$ angular distribution in the laboratory system (see Fig. 7). The $`\eta `$ angular distribution in the lab for hydrogen at this momentum is limited to about $`40^{}`$ by kinematics. It peaks at about $`38^{}`$ due to the Jacobian. For the solid targets, the angular distributions slightly exceed the two–body kinematical limit because of the Fermi momentum. At 750 MeV/$`c`$, the CB acceptance for this reaction is smooth and uniform.

By integrating the angular distributions, we obtained integrated cross sections $`\sigma _{int}`$ for the H, C, Al, and Cu targets. $`\sigma _{int}`$ was normalized to $`\lambda _{geom}`$ and to the hydrogen data. The results shown in Fig. 5 indicate that the $`\eta `$ cross section falls from H to C, and it is lower than the expectation based on the geometrical factor (see Eq. 2). It confirms the large $`\eta N`$ absorption cross section observed in earlier $`\eta `$ photoproduction experiments . From a comparison of the ratios in Fig. 5, one may conclude that the behavior of $`\sigma _{int}(\eta )`$ is remarkably similar to that obtained for the $`\sigma _{int}^{high}(\pi ^0\pi ^0)`$ ratio.

We summarize the results of this section as follows:

1. The shape of the $`\eta `$ missing mass is little influenced by nuclear media (see Fig. 7).
2. A strong $`\eta `$ absorption in nuclear targets was observed (see Fig. 5).

## 5 Study of single $`\pi ^0`$ production

Single $`\pi ^0`$ production is the predominant reaction among the three we have studied. In the case of hydrogen, the charge-exchange reaction (CEX) is well understood in terms of conventional $`\pi N`$ phase-shift analyses. For nuclear targets, however, CEX is accompanied by strong secondary reactions that lead to a large fraction of single $`\pi ^0`$ produced in reactions other than direct CEX. One of the contributing reactions is $`\mathrm{\Delta }^0`$ production via $`\pi ^{}p\pi ^0\mathrm{\Delta }^0`$. The $`\mathrm{\Delta }^0`$ can be rescattered or absorbed on other nucleons, e.g., $`\mathrm{\Delta }NNN`$, and the reaction ends with a single $`\pi ^0`$. In that case, the missing mass would be equal to the mass of the $`\mathrm{\Delta }^0`$. Figure 2 shows the single-$`\pi ^0`$ missing mass and the effective-missing-mass spectra for neutral two-cluster events. Compared to hydrogen, a substantial broadening of the “quasi-elastic” peak can be seen in Fig. 2 for all nuclear targets. This broadening reflects a large probability for the $`\pi ^0`$ to rescatter inside the nuclei. In contrast to the hydrogen data, the C, Al, and Cu effective-missing-mass data show a second maximum at masses of about 1200 MeV. Such maxima can be partially attributed to $`\pi ^0\mathrm{\Delta }^0`$ production followed by $`\mathrm{\Delta }^0`$ absorption in nuclei. This observation is in agreement with the conclusions of Sec. 3 regarding the absorption of $`2\pi ^0`$ events in nuclei.

For some applications it may be desirable to separate partly the secondary processes from the “quasi-elastic” CEX events by applying the cuts on the effective missing mass. The influence of those to-some-extent arbitrary cuts in the angular distribution and the pions kinetic energy is shown in Fig. 8. The distributions are not corrected for the Crystal Ball acceptance. The CB acceptance is smooth and an uniform function between 30 and 150. The acceptance for $`\pi ^0`$ drops very rapidly below 30 and above 150, due to the entrance and exit tunnels. In the angular distribution obtained for hydrogen, a clear backward peak is seen. The peak disappears for the other targets. A similar behavior for single $`\pi ^0`$ production on carbon was found by R. J. Peterson et al. . For a two–body kinematic process, the backward peak in the $`\pi ^0`$ angular distribution corresponds to pions with kinetic energy of about 200 MeV. The peak is clearly seen in the single-$`\pi ^0`$ kinetic-energy distribution for hydrogen (see Fig. 8). The peak disappears for carbon and other nuclear targets, which indicates a large probability for the absorption of $`\pi ^0`$’s with kinetic energy of about 200 MeV. A possible mechanism for such absorption is the formation of a $`\mathrm{\Delta }`$ inside nuclei in $`\pi ^0N`$ interactions followed by $`\mathrm{\Delta }`$ absorption.

## 6 Upper limit for BR$`(\sigma 2\gamma )`$ decay in nuclear media

The diphoton decay $`\sigma 2\gamma `$ in hadronic matter has been evaluated in Ref.  from optimized perturbation theory. The $`\sigma 2\gamma `$ decay is interesting because the photons are not affected by final-state interactions. Despite the low branching ratio, it might be possible to see the decay experimentally if there is a prominent $`\sigma `$ enhancement at low invariant mass in the nuclear medium. The main background is the thermal annihilation of two pions, $`\pi ^+\pi ^{}2\gamma `$.

We obtained the upper limit for BR$`(\sigma 2\gamma )`$ in carbon by comparing the $`\gamma \gamma `$ invariant-mass spectra of the hydrogen and carbon targets. Some additional cuts were applied to the spectra shown in Fig. 6 to reduce the low invariant mass background. The final spectra obtained for the hydrogen and carbon targets are shown in Fig. 9. The hydrogen spectrum is normalized to carbon in the area next to the $`\pi ^0`$ peak (at around 180 MeV). The difference taken between those spectra is compared to our Monte Carlo results in Fig. 9. The Monte Carlo results generated according to the prediction by Chiku and Hatsuda  were used to calculated the CB acceptance. With $`148\pm 48`$ $`\sigma 2\gamma `$ event candidates obtained, $`1.6\times 10^4`$ $`2\pi ^0`$ events detected for the carbon target, and $`15`$ % acceptance, we calculate the upper limit for BR$`(\sigma 2\gamma )`$ in carbon to be $`3.6\times 10^3\times \varphi `$ at 90% C.L., where $`\varphi `$ is the fraction of $`2\pi ^0`$ that is produced via the $`\sigma `$ intermediate state. See the talk by one of us (B.M.K.N.) in these proceedings for more details regarding the $`\sigma `$ contribution.

## 7 Summary and Conclusions

New preliminary measurements of $`\pi ^0`$, $`2\pi ^0`$, and $`\eta `$ production on hydrogen and complex nuclear targets (C, Al, Cu) by $`\pi ^{}`$ at 750 MeV/$`c`$ are reported.

1. The data at 750 MeV/$`c`$ show that absorption rather than medium effects is responsible for the change in the shape of the $`2\pi ^0`$ invariant mass spectra.
2. The preliminary upper limit, BR$`(\sigma \gamma \gamma )<3.6\times 10^3\varphi `$ at 90% C.L., has been obtained for $`\sigma \gamma \gamma `$ on a carbon target at 750 MeV/$`c`$.


text:
# Chandra reveals a double-sided X-ray jet in the quasar 3C9 at 𝑧=2.012

## 1 Introduction

While the detection of X-ray emission coincident with radio jet structures is not unprecedented (e.g. Cen A, Schreier et al 1979; M87, Biretta, Stern & Harris, 1991; 3C273, Harris & Stern 1987; Röser et al. 2000), the $`Chandra`$ X-ray Observatory (CXO) is providing an increasing quantity of high spatial and spectral resolution data on jetted structures associated with Active Galactic Nuclei (AGN).

The first Chandra observation of a radio–loud quasar, PKS 0637–752, showed the presence of X-rays extending for $``$ 10 arcsec (Chartas et al. 2000; Schwartz et al. 2000), having close spatial overlap with the known radio structure. Since then several jets observed with Chandra have shown associated X-ray emission (e.g. Pictor A, see Sambruna et al. 2002; and Worrall, Birkenshaw & Hardcastle 2001; Pesce et al 2001; Siemiginowska et al 2002; Hardcastle et al 2002). It is important to stress that such features have been detected in both radio–galaxies and blazars (i.e. objects observed at a small angle with respect to the inner jet direction).

When multifrequency data are available (radio and optical images) interesting clues on the properties of the emitting plasma are obtained, such as different trends with distance from the core in different bands (an intensity decrease is often seen with X-rays whereas the radio often shows the opposite trend); displacements in the peak emission and different spatial extensions are sometimes visible in the images in different bands in the detailed comparison of individual knot emission. Broad band information on spatially resolved structures have also been studied: the local broad band spectral energy distributions are often not straightforward to interpret, since in several cases the level of the X-ray emission is higher than what simple models predict (e.g. Chartas et al. 2000; Schwartz et al. 2000, Röser et al. 2000, Harris & Krawczynski 2002).

Models involving synchrotron, inverse Compton scattering of the synchrotron, Cosmic Microwave Background (CMB) and nuclear hidden blazar radiation to produce X-rays on such scales have been proposed (e.g. Celotti, Ghisellini & Chiaberge 2001) as well as models invoking synchrotron radiation from ultrarelativistic protons (Aharonian 2000). The most satisfactory model, involving inverse Compton scattering on the CMB, remarkably requires the presence of relativistic bulk flows on scale of 100 kpc. Such an emission process also implies that the X-ray surface brightness of jets is constant with redshift, as pointed out by Schwartz (2001). Furthermore, structures in the jet velocity field, typically the presence of a relativistic core and a slower moving outer ‘layer’ – as already suggested on several other grounds – have been proposed to account for the varied phenomenology of both blazars and radio–galaxies (e.g. Celotti et al. 2001).

While much consensus of the above general scenario has accumulated, X-ray structures with looser morphological similarity to the radio jets have been found and proposed to be of thermal origin. In particular recent data from the radio galaxy PKS 1138-262 at $`z=2.2`$ appear to support the view that a significant fraction of the detected X-ray flux arises from thermal, shock-heated, gas identified with the means for confinement for the radio emitting plasma on large scales (Carilli et al. 2002).

The study of multifrequency emission in jets thus provides important information on the jet physics and jet environment. Within this framework, we present here the discovery of extended X-ray emission associated with the quasar 3C 9, at $`z=2.012`$. Most notably the X-ray structure extends on both sides of the core. In this paper we present the data (Section 2) and discuss the likely origin of the emission (Section 3).

## 2 Observations and data analysis

3C 9 is a powerful quasar at redshift $`2`$ with extended radio emission (e.g. Bridle et al. 1994), classified as FR II. We observed it with Chandra for 16,235 s (cleaned of background flares) on 2001 June 10. 3C9 was imaged on the back-illuminated ACIS-B3 chip. The data clearly show extensions to the NW and SE of the nucleus point source (Fig. 1). The X-ray structure corresponds well to the radio jets (Fig. 2). Some possible additional extended X-ray emission is also seen to the W of the nucleus and in a ’spur’ above the SE jet.

Excluding the central region the counts in the jet and counterjet correspond to 18 and 12, respectively. While this of course does not allow any detailed spectral study, a simple power-law fit to the spectrum (grouped to a minimum of 10 ct per bin) gives a formal spectral index $`\mathrm{\Gamma }_s=1.6\pm 0.6`$ (with a normalization of $`K=(2.8\pm 0.9)\times 10^6`$ ph cm-2 s-1 keV-1) assuming the Galactic value for $`N_H=4.2\times 10^{20}`$ cm-2. Uncertainities are quoted at the 90 per cent confidence level. The X-ray flux (0.4–5 keV) amounts to $`1.23\times 10^{14}`$ erg cm-2 s-1 and the corresponding jet $`+`$ counterjet luminosity is $`2.4\times 10^{44}`$ erg s-1 in the 2–10 keV band (rest frame) (assuming $`H_0=50`$ km s-1 Mpc-1 and $`q_0=0.5`$). A thermal (MEKAL) fit to the spectrum yields a minimum (quasar rest frame) temperature of 4.1 keV (at the 90 per cent confidence level) with no constraint on the upper temperature.

The nucleus suffers mild pile-up (at the 4 per cent level). Spectral fitting of a power-law continuum gives a photon index $`\mathrm{\Gamma }_s=1.58\pm 0.1`$ (similar to that of the extended emission) and a flux of $`1.45\times 10^{13}`$ erg cm-2 s-1 in the 0.2–3.5 keV band.

## 3 Discussion

High resolution VLA radio imaging of 3C 9 shows two jets (Bridle et al. 1994). As can be seen in Fig. 1 the comparison of the radio and X-ray images shows that the X-ray extensions roughly coincide with the jet features, although apparently only in its inner parts before the jet bends (especially in the S structure).

In order to determine the origin of the large scale X-rays in 3C 9, assuming it is non–thermal radiation from the radio emitting plasma, let us first compare its properties with those of PKS 0637-752, at $`z=0.654`$. The total low frequency radio flux (at 408 MHz) of the two systems are similar, while the extended X-rays are a factor $`10`$ lower in 3C 9 (Schwartz et al. 2000). Also the nuclear (blazar-like) X-ray emission of PKS 0637-752 also exceeds that of 3C 9 by a similar factor ($`1050`$, Wilkes et al. 1994).

This implies that both the X-ray extended jet emission and (especially) the blazar core in 3C 9 are less prominent, with respect to the presumably isotropic extended radio, than in PKS 0637-752. The differing prominence of the blazar component could be simply ascribed to a difference in the angle between the (core) jet and the line of sight, as envisaged by unification schemes.

This is indeed supported by the flat radio spectrum of PKS 0637-752 compared to a steep spectrum in 3C 9 ($`\alpha _r`$1). Furthermore for PKS 0637-752 superluminal motion indicates bulk Lorentz factors (relative to the core jet) $`\mathrm{\Gamma }_b>`$ 17 and an angle between the jet axis and line of sight $`\theta <`$ 6 deg (Lovell 2000). On the other hand for 3C 9 we can consider another crucial point, namely the jet/counterjet ratios, in the radio and X-ray bands. They are $``$ 3.3 in the radio (assuming only the straight part) and in the range 0.9-2.6 in X-rays. These are remarkably similar and imply that on the extended scales $`\mathrm{\Gamma }_b<`$ 1.5 even for an angle of 80 deg (note that unification models would suggest 45 deg as a limit for a quasar). We conclude from these elements that 3C 9 is presumably observed at a larger viewing angle.

What can we then infer on the dominant radiation mechanism for the extended X-ray emission? On the one hand the energy density in the CMB radiation field is about a factor 10 higher in 3C 9. On the other, however, the nuclear X-ray luminosities (considered indicative of the blazar emission) of the two objects is of the same order, differing only by a factor $`2.5`$. While these two facts appear to suggest that also in 3C 9 the most likely photon field seed of the inverse Compton emission is the CMB radiation, if one takes into account the significant difference in viewing angle and thus of beaming, the blazar in 3C 9 can consistently be an order of magnitude larger (as seen at small angles). Together with the limit on $`\mathrm{\Gamma }_b`$ on the extended scales, this suggests that the energy density associated with the nuclear emission might dominate over the energy density in the CMB field and thus in 3C 9 we probably observe emission from a slow (jet layer?) component (with respect to the case of PKS 0637-752) scattering the blazar radiation. Such a possibility is supported by the morphology of the emission, as in 3C 9 the X-ray extended structure is (within the limits of resolution) closer to the nucleus up to $``$ 100 kpc (for the whole jet), indeed disappearing from detection after the jet bending, while in PKS 0637-752 it peaks in the large distance knots, at several hundred kpc. Note that in this situation of low relativistic speed we do not expect a large effect, e.g. on the jet/counterjet ratio, due to the anisotropic scattering of the close and distant jets.

While the evidence discussed so far might be reasonably convincing, an alternative view could ascribe the majority of the X-ray flux to thermal emission, as suggested by Carilli et al. (2002), from gas heated via shocks produced by the jet propagation and able to confine the jet itself. Indeed the quality of the morphological similarity between the radio and X-ray spatial distribution – even the evidence of a (spatially coincident) knot (in the S jet) – cannot distinguish between these possibilities. Similarly the spectral analysis cannot constrain the nature of the emission. Note also that the jet/counterjet ratio, in principle a powerful diagnostic for discriminating between the thermal and non-thermal scenarios, is consistent (due to the paucity of counts) with a similar, isotropic emission from the two jet sides.

Additional information for X-ray emission models is the extended Lyman $`\alpha `$ nebula around 3C9 found by Heckman et al (1991), with a tentative detection in Lyman $`\alpha `$ of a structure coincident with brighter part of the SE jet. This means that there is extensive ionized gas around the quasar, possibly at high gas pressure, as inferred by the redshifted H$`\alpha `$ and \[OIII\] observations reported by Wilman, Johnstone & Crawford (2000). If the extended X-ray emission is thermal bremsstrahlung then, assuming an emission temperature of 10 keV for the shocked gas and a volume of $`10\times 10\times 50`$ kpc3, the electron density is about 0.5 cm-3 meaning a pre-shock density of $`0.1`$ cm-3. This is high if the pre-shock temperature is 1 keV or more, comparable only to that seen in the centres of strong cooling flows in nearby rich clusters. The Chandra data are inconsistent with the extensive intracluster medium of a rich cluster around the quasar. (Ignoring the weak temperature dependence, the emissivity of the pre-shock gas is $`1/16`$ of that post-shock, so a volume 16 times that of the jet should have a similar flux which for a cylinder corresponds to a radius of about 4 arcsec; the data at this radius are however entirely consistent with the background.) The shock temperature must exceed about 2000 km s-1 to agree with our lower limit on the gas temperature. The total mass of shocked gas (assuming unit filling factor $`f`$ of the volume assumed above) is about $`10^{11}`$ M. Presumably there is much more gas beyond the region now shocked which occupies only a few per cent of the volume within the radius of the jet, raising the total gas mass to above $`10^{12}`$M. The mass varies as $`f^{1/2},`$ requiring a very low filling factor (say $`f10^3`$ if the mass is to be significantly reduced.

Only if the gas is cooler can high pre-shock densities be present and consistent with observations. The difficulty is then accounting for such an extensive and massive atmosphere of relatively dense gas. It might have to be in the form of many small dense cold clouds in order to minimise the mass. Perhaps both the X-ray and Lyman $`\alpha `$ emission are due to shocks and photoionization of a substantial mass of cold gas clouds which are remnants of galaxy formation and mergers.

Clearly, it is of great relevance to distinguish between these scenarios with higher quality data, which will allow the nonthermal jet and thermal shocked gas possibilities to be distinguished, thereby constraining either the jet velocity field, dissipation and particle acceleration processes or the interaction of the jet with its confining mechanism.

## 4 Acknowledgments

We thank Alan Bridle for sending us the radio image of 3C9. The Royal Society (ACF) and the Italian MIUR (AC) are thanked for financial support.