text:
# Constraints on the Star Formation Rate from Supernova Relic Neutrino Observations

## 1 Introduction

Neutrinos emitted from stellar core collapse fill the universe as a diffuse background radiation. The feasibility for the detection of these neutrinos has been considered by many authors (Bisnovatyi-Kogan & Seidov 1982; Krauss, Glashow & Schramm 1984; Totani, Sato & Yoshii 1996; Malaney 1997; Hartmann & Woosley 1997; Kaplinghat, Steigman & Walker 2000; Ando, Sato & Totani 2002). While estimates of the expected neutrino flux depend much upon authors, the authors are generally negative regarding the feasibility of their detection. The problem is large backgrounds from solar and reactor (anti)neutrinos at low energies and atmospheric neutrinos at high energies; a possible window in between (say, the neutrino energy $`E_\nu =`$15-50 MeV) is masked by a large background from electrons produced by decay of low energy muons that escape detection in the water $`\stackrel{ˇ}{\mathrm{C}}`$erenkov detector (Zhang et al, 1998). The decayed electron spectrum from muons, however, is precisely known, and the Super-Kamiokande (SK) group has demonstrated that this background can be subtracted (Totsuka, 2001). The limit derived on the neutrino event rate in the 18-50 MeV range is close to the value indicated by some optimistic estimates of the supernova relic neutrino flux, which encourages us to scrutinize the problem.

A major uncertainty in the calculation of the supernova relic neutrino flux is in the star formation rate (SFR) and its evolution towards the past. The work with high redshift galaxies over the last five years, however, has provided us with significant insight on the evolution of the global SFR. The estimates include the use of UV emissivity (Madau et al 1996, Lilly et al 1996, Connolly et al. 1997, Steidel et al. 1999, Treyer et al. 1998, Sullivan et al. 2000, Wilson et al. 2002), H$`\alpha `$ fluorescent emission (Gallego et al. 1995, Tresse et al. 2002, Glazebrook et al. 1999, Sullivan et al. 2000), radio emission (Serjeant, Gruppioni & Oliver 2002, Haarsma et al. 2000) and far-infrared emission (Flores et al., 1999). Most of the estimates of the SFR are convergent to $`0.2`$ dex among different authors, if the same observational techniques are used. The large uncertainty, however, resides in which techniques are to be used; Current estimates show an uncertainty of a factor of $`6`$ (0.8 dex). In particular, the SFR estimated from UV depends largely on extinction corrections. Madau et al. (1998) took $`E_{BV}=0.1`$, Steidel et al. (1999) indicated $`0.15`$ and Sullivan et al. (2000) derived $`0.13`$. This corresponds to an uncertainty of 0.36 dex in the SFR. The prime purpose of this paper is to consider whether we can obtain any meaningful constraints on the SFR from the current observation of the supernova relic neutrinos.

Another focus of this paper is to derive a lower limit on the relic neutrino event rate expected in the SK detector under reasonable assumptions on the input to the calculation. The calculation of the event rate suffers from uncertainties besides the SFR. Among the most important uncertainties are those in the spectrum of neutrinos. In this paper we try to reduce the uncertainty in the neutrino spectrum using observation of neutrinos from SN1987A as a constraint. We may assume that it is typical of type II supernovae, since we expect that the physics of core collapse is similar even if the optical appearance may have a wide variety. The other uncertainty concerns neutrino oscillation, in which $`\nu _e`$ and $`\nu _\mu `$ partly interchange during propagation through vacuum and Earth. Recent neutrino oscillation experiment showed unambiguously the presence of neutrino oscillation, and determined the oscillation parameters (Fukuda et al. 1998, Ahmad et al. 2001, Fukuda et al. 2001). We can now calculate accurately the effect of oscillation both in vacuo and in Earth. This is no longer a source of uncertainties.

Throughout this paper we adopt the natural units, $`c=\mathrm{}=1`$, and the Boltzmann constant $`k_B=1`$.

## 2 The local supernova rate and the local star formation rate

A number of extragalactic supernova surveys (van den Bergh & McClure 1994, Cappellaro et al. 1997, Tammann et al. 1994) yield the local supernova rate in units of SNu, i.e., the number of supernovae per $`10^{10}L_B()`$ per 100 year for each morphological type of galaxies. We translate it into the rate per unit cube of spatial volume, by averaging over morphological fractions of nearby galaxies E/S0 : Sa-Sb : Sbc-Sd = 0.32 : 0.28 : (Fukugita, Hogan & Peebles 1998; hereafter FHP), and by multiplying the $`B`$ band local luminosity density of the universe $`_B=2.4\pm 0.4\times 10^8hL_{}`$ Mpc-3 (Blanton et al 2001; Yasuda et al 2001), where $`h`$ is the Hubble constant in units of 100 km s-1Mpc-1. Counting both type Ib and Ic in addition to type II as core-collapse supernovae, we obtain the supernova rate $`R_{\mathrm{SN}}`$ as $`1.98\times 10^4`$, $`2.11\times 10^4`$ and $`4.44\times 10^4`$ $`h^3`$yr-1Mpc-3 for the three surveys. We take the geometric mean of the three values and refer to the largest and smallest of the three as the allowed range: $`2.65\genfrac{}{}{0pt}{}{+1.79}{0.67}\times 10^4`$ $`h^3`$yr-1Mpc-3. This is compared with a similar estimate $`4.7\times 10^4`$$`h^3`$yr-1Mpc-3 by Madau et al. (1998).

For a given star formation rate $`\psi (t)`$ (in units of M yr-1Mpc-3) $`R_{\mathrm{SN}}`$ is calculated as

$$R_{\mathrm{SN}}=\psi (t)\frac{_{m_c}^{m_u}𝑑m\varphi (m)/m}{_0^{m_u}𝑑m\varphi (m)},$$
(1)
where $`\varphi (m)`$ is the initial mass function (IMF) of stars, for which we take the Salpeter form $`\varphi (m)m^{1.35}`$ for $`m>1M_{}`$ and continue to the IMF of Gould, Bahcall & Flynn (1996) for a low mass (see FHP) so that the integral in the denominator is extended to zero mass. We set $`m_u=100M_{}`$. Since the Salpeter IMF has been adopted in virtually all literature that derived SFRs, our calculation of SFR can be directly compared to those, only with a downward correction of a factor of 1.65 that arises from the difference between the genuine Salpeter IMF with a lower cut off of 0.1$`M_{}`$ and our prescription. We take the critical mass for type II supernovae $`m_c`$ to be between 8 and 10 $`M_{}`$ according to Nomoto (1984).

The local star formation rate derived from (1) is $`\mathrm{log}\psi (t_0)(M_{}`$yr-1Mpc-3)$`=2.09\genfrac{}{}{0pt}{}{+0.22}{0.13}`$, where we take $`h=0.72`$. This may be compared with the SFR from an H$`\alpha `$ survey $`2.11\pm 0.04`$ (Tresse et al. 2002, Glazebrook et al. 1999; we take their independent data points).

## 3 Star formation rate and the relic neutrino event rate in the detector

Star formation rates

In Figure 1 we present estimates for the SFR as a function of the lookback time ($`=t_0t`$ with $`t_0`$ the present time, in units of Gyr). The data are taken from Glazebrook et al. (1999), Sullivan et al. (2000), Tresse et al. (2002), Steidel et al. (1999) and Haarsma et al. (2000), which cover most of the SFR work to date We include in the Figure the UV estimates of Wilson et al. (2002) (filled triangles with thin drawings). These data, however, largely disagree with other estimates and the reasons are unknown: so we do not refer to them further in this paper. We take the cosmology of $`\mathrm{\Omega }_0=0.3`$ and $`\lambda =0.7`$ with $`h=0.72`$ to draw this figure. The solid points are the SFR from H$`\alpha `$ (and H$`\beta `$) with the extinction estimated using the Balmer decrement, and the open points refer to the values from UV emissivity assuming zero extinction corrections. Radio observations are denoted by crosses. The grey points are obtained from far-infrared (ISO) observations. The SFR estimate from the supernova rate is shown at the zero lookback time. We also show the constraint from the supernova relic neutrino observations obtained from the present calculation given below.

The figure shows that the SFR obtained from a single indicator exhibits nearly an exponential dependence as a function of the lookback time, at least for $`z1`$, as expected in the closed box model and also in CDM model calculations (e.g. Nagamine et al. 2001). For example the SFR for $`tt_0<7`$ Gyr obtained from H$`\alpha `$ is fitted well with

$$\mathrm{log}\psi (t)_{\mathrm{H}\alpha }=(1.96\pm 0.04)+\mathrm{log}h+0.216h(t_0t).$$
(2)
The SFR derived from far-infrared observations is consistent with this curve. The radio data give a consistent slope, but the normalisation is dex times higher. The data from UV emissivity also indicate a line parallel to (2), but located lower by 0.50 dex With the standard extinction law (Seaton 1979, Cardelli et al. 1989) $`E_{BV}=0.20\pm 0.05`$ brings the UV data consistent with the H$`\alpha `$ data. In the following argument we assume the exponential law for the SFR

$$\psi (t)=\psi (t_0)\mathrm{exp}[(t_0t)/\tau ]$$
(3)
with $`\tau =2.8`$ Gyr for $`z<1`$. This law may not hold for $`z>1`$, but star formation at such high redshift is insignificant in a estimate of the supernova relic neutrino flux for $`E_\nu >12`$ MeV, which is of our current concern, and we do not need to specify any accurate functional form, as we will see below. We take $`\psi (t_0)`$ as a parameter. Alternatively, we may use $`E_{BV}`$ as a parameter taking $`\mathrm{log}\psi _{\mathrm{UV}}(t_0)=2.45`$ from UV emissivity with zero extinction as a fiducial value.

Neutrino spectrum from type II supernovae

We must deal with the uncertainty of the neutrino flux emergent from type II supernovae. The dominant neutrino emission arises from pair creation in the optically thick object. Hence the luminosity of each species of neutrinos is approximately equal, as demonstrated by many neutrino transport calculations given in Table 1. The total neutrino luminosity is close to $`3\times 10^{53}`$ erg, since the mass of all observed neutron stars takes a universal value of 1.4$`M_{}`$. We allot a 20% error to this estimate. The mean energy of neutrinos depends on details of calculations, e.g. ranging from 12 to 20 MeV for $`\overline{\nu }_e`$. The mean neutrino energies satisfy $`E_{\nu _e}<E_{\overline{\nu }_e}<E_{\nu _{\mu \tau }}`$, where $`\nu _{\mu \tau }`$ includes $`\nu _\mu `$, $`\nu _\tau `$ and their antiparticles. Neutrino transport calculations give $`E_{\nu \mu }/E_{\overline{\nu }_e}=1.6`$; see Table 1. We take this ratio to be 4/3, but the result of our calculation is not sensitive to this ratio once we introduce the observational constraint.

The model neutrino spectrum from supernovae is somewhat deviated from the zero-chemical potential Fermi distribution. It is usually parametrised by introducing an effective chemical potential $`\eta `$, i.e., $`f=[\mathrm{exp}(E_\nu /T_\nu \eta )+1]^1`$ with $`\eta =13`$ (Janka & Hillebrandt 1989).

For water $`\stackrel{ˇ}{\mathrm{C}}`$erenkov neutrino detectors the only reaction we must consider is $`\overline{\nu }_epe^+n`$. The cross section of $`\nu _e^{16}`$O is $`>20`$ times smaller. There is, however, an important contribution from $`\overline{\nu }_\mu \overline{\nu }_e`$ due to neutrino oscillation. Recent solar neutrino experiments at Sudbury and SK show that the mixing between $`\nu _e`$ and $`\nu _\mu `$ is nearly maximal. With the matter effect the neutrinos emergent from a supernova are $`\nu _2=\mathrm{sin}\theta \nu _e+\mathrm{cos}\theta \nu _\mu `$ and $`\nu _1=\mathrm{cos}\theta \nu _e+\mathrm{sin}\theta \nu _\mu `$, which are the mass eigenstates. The $`\overline{\nu }_e`$ detected in detectors are therefore $`\mathrm{cos}^2\theta `$ times the $`\overline{\nu }_e`$ flux and $`\mathrm{sin}^2\theta `$ times the $`\overline{\nu }_\mu `$ flux where $`\mathrm{sin}^22\theta 0.96`$ (Ahmad et al. 2001, Fukuda et al. 2001). This in principle increases the neutrino detection rate due to higher energies of the $`\overline{\nu }_\mu `$ flux.

The matter effect of Earth somewhat modifies the mixing ratio for the neutrino flux that passes through Earth. This effect is calculated assuming Earth as a sphere of a constant matter density, $`\rho _e(\mathrm{Earth})3.2`$g/cm3.

Constraints from SN1987A

At the epoch of SN1987A Kamiokande (Hirata et al, 1987) and Irvine-Michigan-Brookhaven Collaboration (IMB) (Bionta et al, 1987) detected neutrinos from core collapse. The gross characteristics of these neutrino events agree with what are expected. Here we use the detection of neutrino events at IMB, which has a larger fiducial volume and is more sensitive to rare, higher-energy neutrino events, to constrain the higher energy spectrum of supernova neutrinos. We estimate the event number $`N_{\mathrm{IMB}}`$ at IMB (5 kton water) as

$$N_{\mathrm{IMB}}=3.3\times 10^{32}_{20\mathrm{M}\mathrm{e}\mathrm{V}}^{60\mathrm{M}\mathrm{e}\mathrm{V}}𝑑E_\nu \sigma _p(E_\nu )F_{\overline{\nu }_e}(E_\nu )x(E_\nu ),$$
(4)
where $`\sigma _p(E_\nu )`$ is the cross section for $`\overline{\nu }_epe^+n`$ with the neutrino energy $`E_\nu `$, $`x(E_\nu )`$ is the trigger efficiency, and $`F_{\overline{\nu }_e}`$ is the $`\overline{\nu }_e`$ flux at the IMB detector,

$$F_{\overline{\nu }_e}(E_\nu )=P_{\mathrm{IMB}}(E_\nu )_{\overline{\nu }_e}+(1P_{\mathrm{IMB}}(E_\nu ))_{\overline{\nu }_\mu },$$
(5)
where $`P_{\mathrm{IMB}}(E_\nu )`$ is the conversion probability for $`\overline{\nu }_1\overline{\nu }_e`$ due to neutrino oscillation including matter effects of Earth and $`_i`$ is the neutrino flux for species $`i`$ without oscillation, for which we obtain

$`_i`$ $`=`$ $`3.99\times 10^{10}\mathrm{cm}^2\mathrm{MeV}^1\left({\displaystyle \frac{E_\nu }{\mathrm{MeV}}}\right)`$ (6)
$`\times \left({\displaystyle \frac{T_{\nu _i}}{\mathrm{MeV}}}\right)\left({\displaystyle \frac{E_{\mathrm{tot},\nu _\mathrm{i}}}{10^{53}\mathrm{erg}}}\right)C(\eta )f(\eta ,E_\nu ),`$
where $`E_{\mathrm{tot},\nu }`$ is the total neutrino energy, $`T_{\nu _i}`$ is temperature of neutrino for $`i`$th species and $`C(\eta )=𝑑E_\nu E_\nu ^3f(\eta ,E_\nu )/𝑑E_\nu E_\nu ^3f(0,E_\nu )`$. The calculation of $`P_{\mathrm{IMB}}`$ is standard and is carried out in a way similar to that in Lunardini & Smirnov (2001), taking neutrino trajectory inside Earth for SN1987A. Applying Poisson statistics to $`8`$ events observed at IMB, we obtain $`90`$% confidence limits on $`E_{\overline{\nu }_e}`$ and $`T_{\overline{\nu }_e}`$ as shown in Table 2. This constraint removes much of the uncertainty of the model neutrino flux: hence the result of our calculation in what follows depends only weakly on the assumptions on parameters we assumed for the model neutrino flux emergent from supernovae.

Supernova relic neutrino flux

The neutrino flux $`𝒥_{\nu _i}`$ is calculated as

$$𝒥_{\nu _i}(E_\nu )=_0^{z_f}𝑑z\frac{dt}{dz}(1+z)_{\nu _i}((1+z)E_\nu )R_{\mathrm{SN}}(z),$$
(7)
where the neutrino luminosity $`_\nu `$ is given by

$$_\nu =E_{\mathrm{tot},\nu }\frac{120}{7\pi ^4}\frac{E_\nu ^2}{T_\nu ^4}C(\eta )f(\eta ,E_\nu ).$$
(8)
$`z_f`$ is the formation epoch of galaxies, but the integral is dominated by a low redshift region. The supernova rate is related with the SFR as,

$$R_{\mathrm{SN}}(z)=1.22\times 10^2\psi (z)/M_{}$$
(9)
for $`m_c=8M_{}`$. We remark that the flux (7) is independent of the cosmology, since its dependence in $`dt/dz`$ is compensated by the volume factor of $`\psi (z)`$. We use for $`\psi `$ the exponential law for $`z<1`$ and assume a constant for $`z>1`$. We take $`\psi (t_0)`$ as a free parameter. This parameter is also translated to $`E_{BV}`$, taking the SFR from UV emissivity with zero extinction as a fiducial. The extinction is written $`\mathrm{\Delta }\mathrm{log}\psi =2.42E_{BV}`$ for $`z<1`$ using the standard extinction law (Seaton 1979, Cardelli et al. 1989).

The calculation of supernova relic neutrino flux $`J_{\overline{\nu }_e}`$ (after neutrino oscillation effect) is carried out in a similar way, with the modification that the Earth effect is integrated over all directions. The event rate with a fiducial volume of the 22.5 kton SK detector is calculated as

$`R_{\mathrm{SK}}`$ $`=`$ $`4.2\times 10^3\mathrm{yr}^1`$ (10)
$`\times {\displaystyle _{E_{\nu _{\mathrm{min}}}}^{E_{\nu _{\mathrm{max}}}}}\left({\displaystyle \frac{E_\nu }{\mathrm{MeV}}}\right)^2\left({\displaystyle \frac{J_{\overline{\nu }_e}(E_\nu )}{\mathrm{MeV}^1\mathrm{cm}^2\mathrm{sec}^1}}\right),`$
where $`E_{\nu _{\mathrm{min}(\mathrm{max})}}`$ is the minimum (maximum) energy of the neutrino detection, which we take $`E_{\nu _{\mathrm{min}}}=18`$ MeV and $`E_{\nu _{\mathrm{max}}}=50`$ MeV. Fig. 2 shows an example result for the neutrino event rate for SK as a function of $`E_{BV}`$ for a typical parameter set indicated in the figure. It is seen that the event rate exceeds the current SK limit, $`2.0`$ yr-1 at 90% confidence for the $`1850`$ MeV window (Totsuka, 2001), if $`E_{BV}>0.4`$. From this figure we can read the range of $`E_{BV}`$ which is consistent with the SFR from H$`\alpha `$ at the zero redshift. The SFRs from H$`\alpha `$ and the local supernova rate are consistent.

Fig. 3 shows the relative importance of supernovae at different redshifts for neutrino events. The solid histogram corresponds to neutrino events with energy between 18 MeV and 50 MeV. It shows that half the events arise from low-$`z`$ supernovae ($`z<0.25`$). If we decrease the energy of the detector window to $`12<E<18`$ MeV, non-zero redshift supernovae become more important. This means that we can learn the evolution of the SFR from gross spectroscopy of supernova relic neutrinos. This histogram also shows that the contribution from supernovae at $`z>1`$ is insignificant in so far as our consideration is restricted to $`E>12`$ MeV. The neutrino spectrum is presented in Figure 4.

Constraint on the star formation rate

We derive a constraint on the SFR by requiring that the supernova relic neutrino event should not exceed the SK limit allowing for uncertainty of the model neutrino flux shown in Fig. 2; the result is shown in Figure 1 above. The most conservative limit is obtained by taking $`m_c=10M_{}`$ and minimising the number of events at IMB for SN1987A. At a 90% confidence the limit means

$$\mathrm{log}\psi (t_0)1.40,\mathrm{or}E_{BV}0.48.$$
(11)
The constraint derived here is about 5 times higher than the SFR obtained from H$`\alpha `$ and far-infrared, and twice higher than the SFR from radio observations.

If we would take the opposite case, i.e. the maximum allowed flux (90% confidence for the IMB events) and $`m_c=8M_{}`$, the limit becomes stronger by a factor of 4; it nearly coincides with the SFR from H$`\alpha `$, and already overshoots the SFR from radio.

We emphasize that large uncertainties in the model supernova neutrino flux calculations are significantly reduced by empirical constraints derived from SN1987A. For instance an increase of $`\eta `$ is compensated by an increase of effective neutrino energy or else by an increase of neutrino luminosity, so that the neutrino flux in the energy range that concerns us changes little. Neutrino oscillation generically enhances the high energy tail of the neutrino spectrum. Under the empirical constraint, however, this is absorbed into the change of other parameters. As a result the prediction of supernova relic neutrinos at SK is modified little.

The lower limit of neutrino reaction rates at SK

Assuming $`\mathrm{log}\psi =2.15`$, which is the lower value of SFR from H$`\alpha `$ allowed within the error range (or SFR from UV with $`E_{BV}0.19`$) and taking our minimum neutrino flux estimate and $`m_c=10M_{}`$, we obtain

$$R_{\mathrm{SK}}0.40\mathrm{yr}^1.$$
(12)
A similar limit is derived if we take the lowest value of the supernova rate:

$$R_{\mathrm{SK}}0.44\mathrm{yr}^1.$$
(13)
The SK should see the supernova relic neutrino events if they increase the sensitivity by a factor of 5.

## 4 Conclusions

The limit on the SFR derived from the supernova relic neutrino observation at SK is already marginally significant even if we include the uncertainty of the model supernova neutrino flux. With the current data we can conclude that $`\psi (t_0)<0.040M_{}`$yr-1Mpc-3, which is 5 times the estimate from H$`\alpha `$ (0.0078$`\pm 0.0008M_{}`$yr-1Mpc-3) and twice that from radio observations. The SFR from local supernova surveys (0.0081$`\genfrac{}{}{0pt}{}{+0.0054}{0.0021}M_{}`$yr-1Mpc-3) is consistent with the estimate from H$`\alpha `$. For the SFR from UV emissivity, our result means $`E_{BV}<0.48`$ with the standard extinction law. The increase of SK statistics by a factor of 5 should positively detect the supernova relic neutrino events.

ACKNOWLEDGMENTS

We would like to thank Yoji Totsuka and Ken Nomoto for valuable discussions.


format: text

text:
# Probing anomalous right-handed top quark couplings in rare B decays

## I Introduction

After the discovery of the top quark at Tevatron, several properties of the top quark have been examined such as top quark mass , production cross section , and the production kinematics  etc.. The production of $`10^710^8`$ top quark pairs per year is expected at Large Hadron Collider (LHC), which will allow us to study the detailed structure of top quark couplings . The top quark dominantly decays through the $`tbW`$ channel and other channels are highly suppressed by small mixing angles. Thus the $`\overline{t}bW`$ coupling will be measured at LHC with high precision. Effects of the anomalous $`\overline{t}bW`$ coupling have been studied in many literatures in direct and indirect ways . The subdominant channel in the Standard Model (SM) is the Cabbibo-Kobayashi-Maskawa (CKM) nondiagonal decay $`tsW`$ of which branching ratio is estimated as

$`Br(tsW)1.6\times 10^3,`$ (1)
when $`|V_{ts}|=0.04`$ is assumed. Though the branching ratio of this channel is rather small, the large number of expected top quark production at LHC will give us a chance to measure the $`tsW`$ process and enable us to probe the $`\overline{t}sW`$ coupling responsible for this channel directly. Therefore the anomalous $`\overline{t}sW`$ coupling, which has not been seriously examined yet, is worth examining at present.

Before the LHC, we can study the top quark couplings indirectly in rare $`B`$ decays. Rare $`B`$ decays involving loop induced flavour-changing neutral transitions are sensitive to the properties of internal heavy particles, so they can provide a good probe of new physics beyond the SM. The radiative $`bs\gamma `$ and semileptonic $`bsl^+l^{}`$ decay are the most promising channels to examine the new physics effects. The branching ratio of inclusive $`BX_s\gamma `$ decay has been measured by CLEO , ALEPH  groups and recently by Belle collaboration from a 5.8 fb-1 data sample . We have the weighted average of the branching ratio of this channel as

$`Br(BX_s\gamma )=(3.23\pm 0.41)\times 10^4,`$ (2)
from those measurements. This channel has intensively studied at next-leading order (NLO) in the SM  and has provided stringent constraints on various new physics models . On the other hand, the first observation of $`bsl^+l^{}`$ decay is reported by the Belle group through the exclusive $`BKl^+l^{}`$ channel

$`Br(BKl^{}l^+)=(0.75_{0.21}^{+0.25}\pm 0.09)\times 10^6,`$ (3)
from a 30 fb-1 data. BaBar collaboration also present a bound on this mode and the $`BK^{}\mu ^{}\mu ^+`$ mode . The inclusive $`BX_sl^+l^{}`$ decay rate is to be measured soon as more data of $`B`$ decay will be accumulated. The measurement of this mode provides a complementary study on the flavour-changing penguin decays.

In this work, we examine the effects of the anomalous right-handed $`\overline{t}bW`$ and $`\overline{t}sW`$ couplings on the inclusive $`BX_s\gamma `$ and $`BX_sl^+l^{}`$ decays. We concentrate on the anomalous couplings of charged current interactions and ignore effects of new particles andj the neutral current interactions. With the anomalous right-handed couplings, we write the effective lagrangian as

$`={\displaystyle \frac{g}{\sqrt{2}}}{\displaystyle \underset{q=s,b}{}}V_{tq}\overline{q}\gamma ^\mu (P_L+\xi _qP_R)tW_\mu ^{}+H.c.,`$ (4)
where $`\xi _q`$ measures the new physics effects. In section 2, we present the effective Hamiltonian approach with the effective lagrangian Eq. (4). The $`BX_s\gamma `$ and $`BX_sl^+l^{}`$ decays are described in terms of the effective Hamiltonian and the effects of the anomalous couplings are analyzed in section 3. Our conclusion is given in section 4.

## II The effective Hamiltonian

In order to study the rare decay processes of $`B`$ meson, the effective field theoretical approach is required to incorporate the consistent QCD correction, which is substantial in rare $`B`$ decays. We can write the $`\mathrm{\Delta }B=1`$ effective Hamiltonian to describe $`bs\gamma `$ and $`bsl^+l^{}`$ processes as

$`_{eff}={\displaystyle \frac{4G_F}{\sqrt{2}}}V_{ts}^{}V_{tb}{\displaystyle \underset{i=1}{\overset{10}{}}}\left(C_i(\mu )O_i(\mu )+C_i^{}(\mu )O_i^{}(\mu )\right),`$ (5)
where the dimension 6 operators $`O_i`$ constructed in the SM are given in the Ref. , and $`O_i^{}`$ are their chiral conjugate operators. Matching the effective theory (5) and the lagrangian (4) at $`\mu =m_W`$ scale, we have the Wilson coefficients $`C_i(\mu =m_W)`$ and $`C_i^{}(\mu =m_W)`$. When we let $`\xi _q=0`$, we have the Wilson coefficients in the SM

$`C_2(m_W)`$ $`=`$ $`1,`$ (6)
$`C_7(m_W)`$ $`=`$ $`F(x_t),`$ (7)
$`C_8(m_W)`$ $`=`$ $`G(x_t),`$ (8)
$`C_9(m_W)`$ $`=`$ $`C_9^\gamma +C_9^Z+C_9^{\mathrm{}}`$ (9)
$`=`$ $`D_0(x_t)4\left(1{\displaystyle \frac{1}{4\mathrm{sin}^2\theta _W}}\right)C_0(x_t){\displaystyle \frac{1}{\mathrm{sin}^2\theta _W}}B_0(x_t),`$ (10)
$`C_{10}(m_W)`$ $`=`$ $`C_{10}^Z+C_{10}^{\mathrm{}}`$ (11)
$`=`$ $`{\displaystyle \frac{1}{\mathrm{sin}^2\theta _W}}C_0(x_t)+{\displaystyle \frac{1}{\mathrm{sin}^2\theta _W}}B_0(x_t),`$ (12)
$`C_i(m_W)`$ $`=`$ $`C_i^{}(m_W)=0,\mathrm{otherwise},`$ (13)
where $`F(x),G(x),D_0(x),C_0(x),B_0(x)`$ are the well-known Inami-Lim loop functions :

$`F(x)`$ $`=`$ $`{\displaystyle \frac{x(75x8x^2)}{24(x1)^3}}{\displaystyle \frac{x^2(23x)}{4(x1)^4}}\mathrm{ln}x,`$ (14)
$`G(x)`$ $`=`$ $`{\displaystyle \frac{x(2+5xx^2)}{8(x1)^3}}{\displaystyle \frac{3x^2}{4(x1)^4}}\mathrm{ln}x,`$ (15)
$`D_0(x)`$ $`=`$ $`{\displaystyle \frac{4}{9}}\mathrm{ln}x+{\displaystyle \frac{x^2(2519x)}{36(x1)^3}}+{\displaystyle \frac{x^2(5x^22x6)}{18(x1)^4}}\mathrm{ln}x,`$ (16)
$`C_0(x)`$ $`=`$ $`{\displaystyle \frac{x}{8}}\left({\displaystyle \frac{x6}{x1}}+{\displaystyle \frac{3x+2}{(x1)^2}}\mathrm{ln}x\right),`$ (17)
$`B_0(x)`$ $`=`$ $`{\displaystyle \frac{1}{4}}\left({\displaystyle \frac{x}{1x}}+{\displaystyle \frac{x}{(x1)^2}}\mathrm{ln}x\right).`$ (18)
Let us switch on the right-handed $`\overline{t}bW`$ and $`\overline{t}sW`$ couplings. Keeping the effects of anomalous couplings in linear order, we obtain the modified Wilson coefficients

$`C_7`$ $``$ $`C_7^{\mathrm{SM}}+\xi _b{\displaystyle \frac{m_t}{m_b}}\stackrel{~}{F}(x_t),`$ (19)
$`C_8`$ $``$ $`C_8^{\mathrm{SM}}+\xi _b{\displaystyle \frac{m_t}{m_b}}\stackrel{~}{G}(x_t),`$ (20)
$`C_9`$ $``$ $`C_9^{\mathrm{SM}}\xi _b{\displaystyle \frac{m_b}{m_t}}\stackrel{~}{D}(x_t),`$ (21)
and the new Wilson coefficients

$`C_7^{}`$ $`=`$ $`\xi _s{\displaystyle \frac{m_t}{m_b}}\stackrel{~}{F}(x_t),`$ (22)
$`C_8^{}`$ $`=`$ $`\xi _s{\displaystyle \frac{m_t}{m_b}}\stackrel{~}{G}(x_t),`$ (23)
$`C_9^{}`$ $`=`$ $`\xi _s{\displaystyle \frac{m_b}{m_t}}\stackrel{~}{D}(x_t),`$ (24)
with the new loop functions

$`\stackrel{~}{F}(x)`$ $`=`$ $`{\displaystyle \frac{20+31x5x^2}{12(x1)^2}}+{\displaystyle \frac{x(23x)}{2(x1)^3}}\mathrm{ln}x,`$ (25)
$`\stackrel{~}{G}(x)`$ $`=`$ $`{\displaystyle \frac{4+x+x^2}{4(x1)^2}}+{\displaystyle \frac{3x}{2(x1)^3}}\mathrm{ln}x,`$ (26)
$`\stackrel{~}{D}(x)`$ $`=`$ $`{\displaystyle \frac{x(5938x+25x^2+2x^3)}{36(x1)^4}}{\displaystyle \frac{2(x+1)}{3(x1)^5}}\mathrm{ln}x{\displaystyle \frac{x^2}{2(x1)^4}}\mathrm{ln}x.`$ (27)
Our new loop functions $`\stackrel{~}{F}(x)`$ and $`\stackrel{~}{G}(x)`$ agree with those in Ref.  and $`\stackrel{~}{D}(x)`$ is the first calculation. Note that the $`𝒪(\xi )`$ terms of $`Z`$–penguin diagram are suppressed by the heavy mass of $`Z`$–boson as $`m_b^2/m_Z^2`$, or $`q^2/m_Z^2`$ and we neglect them here. For the box diagram, the $`𝒪(\xi )`$ terms vanish by the chirality relation and the leading contribution is of $`\xi ^2`$ order. As a consequence, the contribution of order $`𝒪(\xi )`$ comes only through the $`\gamma `$-penguin and gluon penguin diagrams. Thus there exists no new effect in $`C_{10}`$ and $`C_{10}^{}=0`$.

The renormalization group (RG) evolution of the Wilson coefficients $`𝐂=(C_i,C_i^{})^{}`$ given by

$`\mu {\displaystyle \frac{d}{d\mu }}𝐂(M_W)={\displaystyle \frac{g^2}{16\pi ^2}}\gamma ^T𝐂(M_W),`$ (28)
is governed by a $`20\times 20`$ anomalous dimension matrix $`\gamma `$. Since the strong interaction preserves chirality, the operators $`Q_i^{}`$ are evolved separately without mixing between those and the SM operators. Thus the $`20\times 20`$ anomalous dimension matrix $`\gamma `$ is decomposed into two identical $`10\times 10`$ matrices $`\gamma _0`$ given in the SM as

$`\gamma =\left(\begin{array}{c}\gamma _00\\ 0\gamma _0\end{array}\right),`$ (31)
The $`10\times 10`$ anomalous dimension matrix $`\gamma _0`$ has been calculated to leading logarithmic level in Ref. . Using the initial condition at $`\mu =m_W`$,

$`(C_i(M_W),C_i^{}(M_W))=(0,1,0,0,0,0,C_7,C_8,C_9,C_{10},0,0,0,0,0,0,C_7^{},C_8^{},C_9^{},0),`$ (32)
we can solve the RG equation to obtain the Wilson coefficients evolved from $`\mu =m_W`$ to $`\mu =m_b`$ scales.

## III Rare $`B`$ decays

### A $`BX_s\gamma `$

The branching ratio of $`BX_s\gamma `$ process with the right-handed interactions are obtained at NLO

$`\mathrm{Br}(BX_s\gamma )`$ $`=`$ $`{\displaystyle \frac{\mathrm{Br}(BX_ce\overline{\nu })}{10.5\%}}`$ (35)
$`\times [B_{22}(\delta )+B_{77}(\delta )(|r_7|^2+|r_7^{}|^2)+B_{88}(\delta )(|r_8|^2+|r_8^{}|^2)`$
$`+B_{27}(\delta )Re(r_7)+B_{28}(\delta )Re(r_8)+B_{78}(\delta )(Re(r_7r_8^{})+Re(r_7^{}r_8^{}))],`$
where the ratios $`r_i`$ and $`r_i^{}`$ are defined by

$`r_i={\displaystyle \frac{C_i(m_W)}{C_i^{SM}(m_W)}}=1+{\displaystyle \frac{C_i^{\mathrm{New}}(m_W)}{C_i^{SM}(m_W)}},r_i^{}={\displaystyle \frac{C_i^{}(m_W)}{C_i^{SM}(m_W)}}.`$ (36)
The components $`B_{ij}(\delta )`$ depends on the kinematic cut $`\delta `$, of which numerical values are given in the Ref. .

With the measured branching ratio, Eq. (2), we can set the conservative bounds on the parameter $`\xi _b`$ and $`\xi _s`$ as

$`0.0021<\xi _b<0.0031`$ $`(\mathrm{A}),0.0485<\xi _b<0.0433(\mathrm{B}),`$ (38)
$`|\xi _s|<0.012,`$
at 2-$`\sigma `$ level. We assume that $`\xi _b`$ and $`\xi _s`$ are real for simplicity. The anomalous coupling $`\xi _b`$ contributes in the linear and quadratic order while $`\xi _s`$ dominantly contributes in the quadratic order since the contribution of the linear order is strongly suppressed by the ratio $`m_s^2/m_b^2`$. Thus the parameter $`\xi _s`$ is less constrained by the $`BX_s\gamma `$ measurement than $`\xi _b`$ in general. Due to the cancellation by the interference term $`B_{27}Re(r_7)`$, however, the large $`|\xi _b|`$ solution in the region B is also allowed, which gives the positive Wilson coefficient $`C_7(m_W)>0`$.

### B $`BX_sl^+l^{}`$

The dilepton invariant mass distribution of $`BX_sl^+l^{}`$ decays consists of following contributions :

$`{\displaystyle \frac{d\mathrm{Br}(BX_sl^{}l^+)}{d\widehat{s}}}={\displaystyle \frac{dB_0}{d\widehat{s}}}+{\displaystyle \frac{dB_{1/m_b^2}}{d\widehat{s}}}+{\displaystyle \frac{dB_{1/q^2}}{d\widehat{s}}},`$ (39)
where the first term denotes the decay at the parton level, the second term the power correction in the heavy quark effective theory (HQET), and the last term is due to the nonperturbative virtual quark loop effects with soft gluon. We have the explicit expression including the HQET corrections of order $`𝒪(1/m_b^2)`$ given in Ref. ,

$`{\displaystyle \frac{d\mathrm{Br}(BX_sl^{}l^+)}{d\widehat{s}}}`$ $`=`$ $`2B_0\left[({\displaystyle \frac{1}{3}}(1\widehat{s})^2(1+2\widehat{s})(2+\widehat{\lambda }_1)+(115\widehat{s}^2+10\widehat{s}^3)\widehat{\lambda }_2)\right)`$ (43)
$`\times (|C_9|^2+|C_9^{}|^2+|C_{10}|^2)`$
$`+({\displaystyle \frac{4}{3}}(1\widehat{s})^2(2+\widehat{s})(2+\widehat{\lambda }_1)+4(63\widehat{s}+5\widehat{s}^3)\widehat{\lambda }_2)){\displaystyle \frac{(|C_7|^2+|C_7^{}|^2)}{\widehat{s}}}`$
$`+(4(1\widehat{s})^2(2+\widehat{\lambda }_1)+4(56\widehat{s}+7\widehat{s}^2)\widehat{\lambda }_2))\mathrm{Re}(C_7C_9^{}+C_7^{}C_{}^{}{}_{9}{}^{})],`$
where $`\widehat{s}=(p_++p_{})^2/m_b^2`$ is the normalized dilepton invariant mass, $`\widehat{\lambda }_{1,2}=\lambda _{1,2}/m_b^2`$ the normalized HQET parameters, and the normalization constant is given by

$`B_0\mathrm{Br}(BX_ce\overline{\nu }){\displaystyle \frac{3\alpha ^2}{16\pi ^2}}{\displaystyle \frac{|V_{ts}^{}V_{tb}|^2}{|V_{cb}|^2}}{\displaystyle \frac{1}{f(\widehat{m}_c)\kappa (\widehat{m}_c)}},`$ (44)
with the phase space function $`f(\widehat{m}_c)`$ and the perturbative QCD correction $`\kappa (\widehat{m}_c)`$ of $`BX_cl\nu `$ decay. The long-distance correction due to the virtual $`c\overline{c}`$ loop is considered in Ref. . Numerically this correction is at 1 - 2% level in the region of $`\widehat{s}`$ considered here away from the resonances and we ignore them. We plot the differential branching ratio with respect to $`\widehat{s}`$ in the Fig. 1. The values of $`\xi _{b,s}`$ given in Eq. (14) are used. We consider the region for $`\widehat{s}`$ as 1 GeV$`{}_{}{}^{2}<\widehat{s}m_b^2<`$ 7.5 GeV2 in order to avoid the large resonant contribution of $`J/\psi `$ and $`\psi ^{}`$. We show that the total decay rate over this region is enhanced by the anomalous couplings. The $`\xi _s=0.012`$ leads to 10% enhancement of the branching ratio. The enhancement by $`\xi _b`$ in the region A is less than 1% and the branching ratio with $`\xi _b`$ in the region B is more than twice the SM prediction since there is no interference term between $`C_2`$ and $`C_7`$ in the $`BX_sl^{}l^+`$ decay rate, which cancels the enhanced $`|C_7|^2`$ contribution. Hence such a enhancement should be observed in the near future, if $`\xi _b`$ coupling in the region B exists.

The forward-backward (FB) asymmetry $`A_{FB}`$ is defined as

$`{\displaystyle \frac{dA_{FB}}{d\widehat{s}}}={\displaystyle _0^1}{\displaystyle \frac{d^2\mathrm{Br}}{d\widehat{s}d\mathrm{cos}\theta }}d\mathrm{cos}\theta {\displaystyle _1^0}{\displaystyle \frac{d^2\mathrm{Br}}{d\widehat{s}d\mathrm{cos}\theta }}d\mathrm{cos}\theta ,`$ (45)
where the angle $`\theta `$ is measured between $`b`$-quark and the positively charged lepton $`l^+`$ in the dilepton center-of-mass (CM) frame. The $`A_{FB}`$ distribution with respect to $`\widehat{s}`$ is depicted in the Fig. 2. We find that $`\xi _b`$ in the region A can bring a substantial shift of the differential FB asymmetry although the branching ratio is not much affected. It is because the branching ratio is dominated by the SM contribution $`|C_9|^2+|C_{10}|^2`$ but $`A_{FB}`$ is proportional to the substantial Wilson coefficient $`C_{10}`$ and shifted by the term $`\xi _bC_{10}`$. As a consequence, $`A_{FB}`$ summed over the region considered here is enhanced by 4 times with $`\xi _b=0.0021`$ and even its sign is changed if $`\xi _b=0.0031`$. With $`\xi _b`$ in the region B, the shift of $`dA_{FB}/d\widehat{s}`$ is huge as is the case of decay rate, because of the large shift of $`C_7`$. Besides, the anomalous $`\overline{t}sW`$ coupling does not affect $`A_{FB}`$, since $`C_{10}^{}=0`$.

## IV Concluding Remarks

We studied rare $`B`$ decays with the anomalous right-handed $`\overline{t}bW`$ and $`\overline{t}sW`$ couplings in the model-independent way. This kind of anomalous couplings can be obtained in the general left-right (LR) model based on SU(2)$`{}_{L}{}^{}\times `$SU(2)$`{}_{R}{}^{}\times `$U(1) gauge group  or the dynamical electroweak symmetry breaking model . In the LR model, the right-handed quark mixing is also an observable. If we do not demand the symmetry between left- and right-handed sectors manifest, the right-handed quark mixing is not necessarily identical to the left-handed quark mixing described by the CKM matrix. Thus we have right-handed charged current interactions, which is suppressed by the heavy mass of extra $`W`$ boson. but still enhanced relatively by the right-handed quark mixing matrix. On the other hand, when the electroweak symmetry breakdown is dynamical, one may expect that some nonuniversal interactions exist which lead to anomalous couplings on charged current interactions.

The constrainyt on $`\xi _s`$ by the $`BX_s\gamma `$ data is weaker than that of $`\xi _b`$ in the region A by the chirality relation. As a result, the considerable enhancement of the branching ratios for $`BX_sl^+l^{}`$ decay is possible with the anomalous $`\overline{t}sW`$ coupling while the influence of the anomalous $`\overline{t}bW`$ coupling in the region A is rather small. Besides anomalous $`\overline{t}bW`$ coupling can change the forward-backward asymmetry considerably under the $`BX_s\gamma `$ constraints, and even the sign of $`A_{FB}`$ may be reversed while the anomalous $`\overline{t}sW`$ coupling does not affect $`A_{FB}`$. Therefore it is possible to discriminate the effects of the anomalous $`\overline{t}bW`$ and $`\overline{t}sW`$ couplings if we combine the analysis of the branching ratio and $`A_{FB}`$ for $`BX_sl^+l^{}`$ process. On the other hand, relatively large value of $`|\xi _b|`$ in the region B is also allowed due to the cancellation between the $`C_2C_7`$ and $`|C_7|^2`$ terms for $`BX_s\gamma `$ decay, which leads to much larger branching ratio and altered $`A_{FB}`$ for $`BX_sl^+l^{}`$ decay. The measured branching ratio of the exclusive $`BKl^+l^{}`$ decay given in Eq. (3), is rather higher than the SM prediction, though it is still consistent with the SM  due to large errors and theoretical uncertainties. Thus it may be the clue of the new physics signal and the anomalous coupling $`\xi _b`$ in the region B can be a candidate of the new physics. If we measure $`A_{FB}`$ in the future, it will be a clear probe of the nature of the top quark anmalous couplings.

###### Acknowledgements.

We thank T. Morozumi and T. Goto for valuable comments and K. Hagiwara for hospitality. This work is supported in part by BK21 Core program of the Ministry of Education (MOE).


text:
# HST NICMOS Observations of the Polarization of NGC 10681footnote 11footnote 1Based on observations made with the NASA/ESA Hubble Space Telescope, obtained at the Space Telescope Science Institute, which is operated by the Association of Universities for Research in Astronomy, Inc., under NASA contract NAS 5-26555. These observations are associated with GTO proposal #7212, PI: E. F. Erickson.

## 1 Introduction

NGC 1068 is the closest (14.4 Mpc for H$`{}_{0}{}^{}=75`$ km s-1 Mpc-1, 72 pc arcsec-1) Seyfert 2 galaxy; thus it has been studied extensively at all wavelengths (see the summary of the literature by Bland-Hawthorn et al. 1997a). The barred spiral galaxy is $`7^{}`$ in diameter with the northern rotation axis apparently tipped towards the earth at an angle of about $`40^{}`$ (Schinnerer et al. 2000; Bland-Hawthorn et al. 1997a and references therein). A cone of polarized (scattered) optical and ultraviolet (UV) light (e.g., Scarrott et al. 1991; Miller, Goodrich, & Mathews 1991; Capetti et al. 1995a, 1995b) and co-spatial \[O III\] (Pogge 1988; Bruhweiler et al. 2001) and \[N II\] (Cecil, Bland, & Tully 1990) emission fan out from the nucleus to the northeast at a position angle of $`31^{}`$). Radio jets emanate from a bright radio “core” along a similar position angle (Wilson & Ulvestad 1983), but only a faint \[O III\] counterpart is observed toward the southwest (Unger et al. 1992) and very little UV/optical polarized light (Scarrott et al. 1991). The faintness of the southwest cone suggests that it lies behind the plane of the galaxy and is extinguished by the dust in the plane (e.g., Gallimore et al. 1994; Bland-Hawthorn et al. 1997b), whereas it is thought that the brighter northeast cone emerges from the nucleus in front of the galaxy plane (e.g., Kishimoto 1999; Bruhweiler et al. 2001). Broad permitted lines including Fe II are observed in the spectrum measured in polarized light, confirming that this proto-type Seyfert 2 (narrow-lined) galaxy harbors a Seyfert 1 (broad-lined) nucleus (Antonucci & Miller 1985; Antonucci, Hurt, & Miller 1994; Miller, Goodrich, & Mathews 1991; Inglis et al. 1995; Alexander et al. 1999) that is hidden from our direct view by a geometrically and optically thick dusty region. Light from the nucleus ionizes the clouds of the narrow line region; moreover, ionizing and UV photons from the nucleus are scattered by electrons into the more extended, diffuse ionized gas of the galaxy disk and halo (Bland-Hawthorn, Sokolowski, & Cecil 1991; Sokolowski, Bland-Hawthorn, & Cecil 1991; Neff et al. 1994). The light is also scattered (thus polarized) by electrons and dust into our line of sight.

The nucleus of the galaxy is not well defined at UV and visible wavelengths; instead, high resolution UV and visible imaging with the Hubble Space Telescope (HST) only shows a number of bright clouds and filaments (Evans et al. 1991; Macchetto et al. 1994; Bruhweiler et al. 2001). There have been a number of attempts to estimate its location from the direction of the UV polarization vectors measured with HST (e.g., Capetti et al. 1995a, 1995b; Kishimoto 1999). On the other hand, near-infrared (NIR) and mid-infrared images (Braatz et al. 1993; Marco, Alloin, & Beuzit 1997; Thatte et al. 1997; Rouan et al. 1998; Weinberger, Neugebauer, & Matthews 1999; Bock et al. 2000) show an intense unresolved source. Thompson et al. (2001), by correlating the optical H$`\alpha `$ and NIR P$`\alpha `$ images from HST, determined that the NIR point source lies at the location of the nucleus (to within the errors of measurement) as determined by Kishimoto (1999). These locations can be seen in Figure 7 and Table 2 of Kishimoto (1999), where he plots the nucleus position determined from the UV polarization measurements on the HST F501N \[O III\] image. The position (with uncertainty of $`0.04^{\prime \prime }`$) of Thompson et al. is consistent with both Kishimoto’s nucleus position and with Cloud B of Evans et al. (1991), also plotted on the figure. The radio source S1 is thought to be the location of the AGN nucleus and accretion disk because of the high velocities of its H2O masers emanating from an apparently rotating, 4 pc diameter disk (Gallimore et al. 1996b, 2001; Greenhill & Gwinn 1997) with interior hot thermal emission (Muxlow et al. 1996; Roy et al. 1998). The coordinates of source S1 agree with the positions of Cloud B and the nuclear position plotted by Kishimoto (1999) within the absolute uncertainty of $`0.08^{\prime \prime }`$ of the optical position of the galaxy (Capetti, Macchetto, & Lattanzi 1997b).

If the intense NIR unresolved source is really the galaxy and AGN nucleus, its light should be scattered and polarized with a centrosymmetric polarization pattern. Moreover, since NIR wavelengths are less affected by extinction than visible or UV wavelengths, the morphology of the scattering region can be better determined through measurement of the NIR polarized light. Ground-based measurements of the NIR polarization in $`1^{\prime \prime }`$ to $`2^{\prime \prime }`$ resolution have been made by Young et al. (1996), Packham et al. (1997), and Lumsden et al. (1999). They show that the polarization is not consistent with scattering in the immediate vicinity of the nucleus; Young et al., Packham et al., and Lumsden et al. all suggest that there is additional polarization caused by the light from the nucleus being absorbed by aligned dust grains in the AGN torus. Young et al. and Lumsden et al. also detect a dip in the polarized emission $`1^{\prime \prime }`$ south of the nucleus and all three groups see a southwest lobe of polarized emission that is not observed at visible wavelengths.

In this paper we report on 2 µm polarization observations taken by the Near Infrared Camera and Multi-Object Spectrometer (NICMOS) on HST. With the $`0.2^{\prime \prime }`$ spatial resolution of NICMOS, we see the unresolved polarized source that must be the nucleus and numerous scattering clouds in the NE and SW lobes. Section 2 describes the observations, section 3 the results and discussion, and section 4 summarizes the results.

## 2 Observations and Data Reduction

NGC 1068 was observed in two visits on 11 October and 15 November, 1997, with the telescope rolled such that NICMOS Camera 2 was oriented at $`63.519^{}`$ and $`154.519^{}`$ East of North, respectively. Integrations were taken at 4 dither positions separated by 10.5 pixels ($`0.78^{\prime \prime }`$) in each of the three Camera 2 polarizing filters, POL0L, POL120L, and POL240L, which cover a 1.9 to 2.1 µm bandpass. The detector array was read out in MULTIACCUM mode with sample sequence STEP16 to accumulate total times of 64 or 80 seconds per integration for visit 1 and 128 seconds for visit 2. We were able to recover the full dynamic range in this observation even though the nucleus saturates in 8 or 16 s because the design of the STEP16 MULTIACCUM sequence incorporates very short dwell times for the first few read-outs.

The raw data were calibrated using CALNICA (v. 3.3) as implemented by STScI in STSDAS v2.2. We included the CALNICA task BIASEQ but not the tasks PEDSKY or PEDSUB because there is no blank sky in the $`19.4^{\prime \prime }`$ images. The reference files used were the most recent Institute flats and non-linearity files (of January, 2001), but the dark files were provided by M. Rieke (2000, private communication). Each reduced image required additional corrections for quadrant bias and half the images needed slight adjustments for residual shading. Since images requiring no shading adjustment occurred during both visits, using the STScI temperature sensitive darks would not have negated the need for adjustments. These corrections were very small, $`0.05`$ counts s-1 whereas the nucleus center is $`3600`$ counts s-1 and the image edges $`0.5`$ counts s-1. Although the need for such corrections is not noticeable in the total intensity images, it is obvious in quantities that incorporate a difference image, such as the polarized flux.

For each polarizing filter, we aligned and shifted the four dither positions by centroiding the unresolved nucleus and then median combined them to remove bad pixels using the IDL program, IDP3222http://nicmos.as.arizona.edu. We used the equations of Hines, Schmidt, & Schneider (2000) to calculate the total flux ($`F`$), percentage polarization ($`p`$), polarized flux ($`p\times F`$), and position angle for each visit; we calculated statistical errors using the equations of Sparks & Axon (1999) and the estimated errors on the flux computed by CALNICA. The original pixel scale was $`0.0760^{\prime \prime }`$ by $`0.0753^{\prime \prime }`$ per pixel; the pixels were rectified and smoothed with a $`3\times 3`$ smoothing function (i.e., to approximately the HST spatial resolution of $`0.2^{\prime \prime }`$ at 2.0 µm) in the data reduction process to achieve higher signal/noise.

The total intensity, fractional polarization, and polarized flux images from the two visits are plotted in Figure 1. Two ghosts at pixel positions (23,35) and (8,20) were discovered in the POL0L and POL120L filters, respectively, and are marked in the images. Although the total fluxes for the two visits are very similar, the percentage polarization and polarized flux differ by 2–3 percent with visit 2 generally showing less polarization. The cause of the difference in polarization is not known. Because the differences between the visits in the percentage polarization and polarized flux are generally larger than the calculated statistical errors, we estimated the actual uncertainties (which we quote in later sections) as equal to 0.5 times the differences between the visits.

## 3 Results and Discussion

### 3.1 Nucleus

In Figure 1 the nucleus of NGC 1068 appears as an intense, unresolved source convolved with the HST point spread function (PSF) in both the total flux and the polarized flux images. Since we do not have a real-time example of the PSF from a star, we calculated the PSF for each of the visits using the program TinyTIM333http://www.stsci.edu/software/tinytim (Krist & Hook 1997, 1999). The nuclear region with contours of the TinyTIM PSF is plotted in Figure 2. It is not possible to remove the effects of the PSF by subtracting the calculated PSF from the observed nucleus — numerous blobs at the locations of PSF diffraction rings and spikes remain no matter what PSF scaling was used. In fact, one can see in Figure 2 that although there is general agreement between the calculated PSF contours and the polarized flux, there are also many regions that do not have good enough correspondence between the calculated PSF and the observed blobs for complete PSF subtraction. As a result, we can only say that there is a hint of additional structure within the first Airy ring surrounding the nucleus. It is possible that the nucleus is slightly extended in the north-south direction as was suggested by Weinberger et al. (1999), Rouan et al. (1998), and Thatte et al. (1997). It is interesting that apparently the radio jet is initially emitted in approximately a north-south direction and is diverted to the $`30^o`$ position angle $`0.3^{\prime \prime }`$ north of the nuclear source S1 (Gallimore, Baum, & O’Dea 1996a; Muxlow et al. 1996).

The nuclear polarization, averaged over the central $`3\times 3`$ pixels, is $`6.0\pm 1.2`$% and the position angle is $`122^{}\pm 1.5^{}`$, perpendicular to the eventual direction of the radio jet but not its initial direction. This percentage polarization is larger than that measured by Packham et al. (1997) or Lumsden et al (1999), probably because their measurements were diluted by the non-polarized flux from the galaxy core seen in their larger beams. The measured position angles are very similar.

Lumsden et al. (1997) studied their polarization position angles to check for the centrosymmetric pattern that would indicate scattering of light originating at a well defined location (e.g., the nucleus). They found that the NIR polarization vectors in the vicinity of the nucleus are not consistent with scattering of nuclear light only; instead they suggested that the nuclear emission is polarized by dichroic absorption as a result of passing through aligned dust grains. However, their beams were typically $`2^{\prime \prime }`$ or larger and thus could include non-nuclear light as well as nuclear. Alternatively, the nuclear polarized light could include light backscattered from the cones above the nucleus and scattered a second time on the accretion disk (Fischer, Henning, & Yorke 1996; Schinnerer et al. 2000).

In order to estimate what fraction of the nuclear polarized light is coming from the point source, we measured the nuclear fluxes and the calculated PSF in various apertures and corrected the fluxes by dividing by the PSF fraction in the same apertures. These results are given in Table 1. We see that the polarized flux from the nucleus of NGC 1068 completely dominates the polarized flux that would be seen in any $`1^{\prime \prime }2^{\prime \prime }`$ aperture such as were used in the ground-based polarization measurements; contributions from off-nuclear scattered light would be barely detectable. Only a small fraction of the polarized flux measured within a $`2^{\prime \prime }`$ aperture is not flux from the essentially unresolved nucleus (subtracting the PSF-corrected polarized flux measured in the $`0.209^{\prime \prime }`$ radius aperture from that measured in the $`1.00^{\prime \prime }`$ radius aperture gives $`3.4\pm 0.2`$ mJy).

The polarization vectors are plotted on a contour map of the polarized flux in Figure 3. Since the vectors describing the vibration of the electric field in the sky plane are perpendicular to the projected line between the illuminating source and the (optically thin) scatterers, one can locate the illuminator by computing the intersection of the lines perpendicular to the polarization position angle vectors (e.g., Lumsden et al. 1999; Kishimoto 1999; Capetti et al. 1995a). The procedure is to pick a pixel, assume it is the center, calculate the vector angle to every other pixel, and subtract the perpendicular polarization angle at those pixels. The means of the differences as a function of position angle from the assumed center are a minimum when the assumed center pixel is located at the illuminating source of the scattered light. For the NICMOS polarization vectors for regions greater than $`1^{\prime \prime }`$ from the peak, the polarization vectors point to the intense peak within $`<1`$ pixel. This confirms that the illuminating source of the scattered light is the same at 2 µm as the illuminating source estimated from optical polarization measurements (e.g., Kishimoto 1999), given the alignment of the optical and Paschen-$`\alpha `$ images of Thompson et al. (2001).

### 3.2 Inner Two Arcseconds

Even though most of the brighter emission blobs in the central $`2^{\prime \prime }`$ are due to the complex diffraction pattern from the bright central point source (i.e., the PSF), it is clear that there is some polarized emission extending along a ridge $`1^{\prime \prime }`$ north and south of the nucleus (Figs. 2 and 4). This appears to be the structure imaged by Thompson et al. (2001) in P$`\alpha `$ (their Fig. 1c) and may include some of cloud C of Evans et al. (1991). The other clouds seen in the inner $`1^{\prime \prime }`$ in UV polarization (e.g., Capetti et al. 1995a, 1995b) are obscured by the PSF and there is no good correspondence with the other UV polarization structures in the central $`2^{\prime \prime }`$. The emission spot described by Thompson et al. (2001) is not bright in polarized light.

Young et al. (1996) and Lumsden et al. (1999) noticed that there is a dip in the polarized flux about $`1^{\prime \prime }`$ south of the nucleus in their J and H images but not in their K band image. This dip is clearly present in our 2 µm NICMOS image (see especially the fractional polarization images in Fig. 1), although it is not clear whether it is a completely empty gap or whether there are some clouds of scatterers present (the fact that the observed polarized flux blobs do not completely correspond to the TinyTIM-calculated PSF is to be expected and does not necessarily indicate additional scattering sources in the region). Young et al. (1996) suggested that the dip is caused by obscuration from the dusty torus surrounding the AGN; this torus would have to have a size $`200`$ pc, which is substantially larger than the few-parsec-sized canonical AGN torus (see Antonucci 1993 and references therein) that directs the radiation into the observed scattering cones. Another possible source of the obscuring dust is the dense cloud or nuclear ring (diameter $`5^{\prime \prime }`$) seen in CO by Schinnerer et al. (2000). Schinnerer et al. estimate this ring to have a mass of $`5\times 10^7`$ M$`\mathrm{}`$ centered on the nucleus. Using the interstellar ratio of $`A_V`$/hydrogen column density of Bohlin, Savage, & Drake (1978), we estimate values of $`A_V`$ for this cloud ranging from 6 to 15. Although this cloud could provide the small dip seen by Young et al. and Lumsden et al., it would not provide enough extinction to produce a gap as empty as the polarization gap in Fig. 1 appears to be. We suggest that there is a real dearth of scatterers in the band $`1^{\prime \prime }`$ south of the nucleus.

### 3.3 Southwest and Northeast Lobes

The southwest and northeast lobes (see Fig. 4 for nomenclature) are seen to be highly clumped, with locally very strong polarization of 17% when smoothed with a $`3\times 3`$ smoothing to $`0.23^{\prime \prime }`$ per smoothed pixel (the NICMOS resolution is $`0.2^{\prime \prime }`$) and 20% in a single pixel. The southwest lobe is quite faint at UV and visible wavelengths; the reason probably is that it lies behind the plane of the host galaxy and is extinguished by interstellar dust. Observations of the H$`\alpha `$/H$`\beta `$ ratio by Bruhweiler et al. (2001) show that the northeast lobe seen in the ionized lines suffers very little extinction but the southwest lobe has H$`\alpha `$/H$`\beta 6`$ (the Case B recombination line ratio is 2.87). This implies $`A_V2`$ mag for ionized hydrogen clouds to the southwest of the nucleus.

In addition to inferring the location of the main illuminating source, one can also use the polarization vectors to infer whether there is any additional illuminating source besides the strong intensity peak (which is very close to the illuminating source, if not the actual source), or whether there is additional polarization from dichroic absorption by aligned interstellar dust grains in the galaxy. To do this, we measured the deviations of the perpendiculars to the polarization vectors from the vectors to the bright nucleus for regions not containing PSF contributions from the nucleus. Most of the largest deviations from zero occur where the intensity is less than the local maximum, in other words, in the regions of larger uncertainty. The only locations with large deviations that are consistent between the visits are at the extreme west edges of the northern and southern lobes. The uncertainty, though, is of similar size to the deviation, about 2 degrees whereas the deviation is about 4 degrees. We conclude that there is no strong evidence of any additional polarization component in the regions $`1^{\prime \prime }4^{\prime \prime }`$ from the nucleus from dichroic absorption of grains aligned by a magnetic field oriented other than perpendicular to the line joining the nucleus and the grain location in the galaxy. However, since magnetic fields in galaxies, including NGC 1068 (Scarrott et al. 1991), tend to follow the spiral arms, it is likely that any dichroic absorption would produce polarization with the same orientation as that produced by scattering of light emitted by the nucleus. On the other hand, we find that there is no correlation of the bands of polarized flux in the southwest lobe and regions of high H$`\alpha `$/H$`\beta `$ (large extinction) in Figure 3 of Bruhweiler et al. (2001). Thus, dichroic absorption is probably not the main contributor to the high polarization that we see in the southwest lobe in spite of the high optical extinction with the possible exception of the extreme southwest clump, where Bruhweiler et al. (2001) saw no appreciable H$`\alpha `$ emission. Most likely the higher polarized intensity in the southwest cone compared to the northeast cone is that it contains a higher density of dust (larger filling factor).

Jets emitting synchrotron radiation have been observed emanating from the nucleus by Wilson & Ulvestad (1983, 1987). In Fig. 4, we plot the 4.9 GHz radio contours on top of our combined polarized flux image. The jets appear to pass through the regions containing the scattering material without spreading out; only after they are beyond the regions exhibiting polarized emission do they spread out into extended lobes. This is the first observed correlation of some aspect of the south radio lobe with any optical, IR, or X-ray feature. One possibility is that the scattering material in the NE and SW lobes seen in NIR polarization is affecting the radio jets and the formation of the radio lobes; another is that the jets have pushed material into the dense dust clouds that are visible in scattered light.

### 3.4 Northeast Knot

The northeast “knot” was first identified by Miller et al. (1991) as a region or knot of high visible polarization about $`4.7^{\prime \prime }`$ northeast of the nucleus. An enhancement in the polarized emission is also apparent in the NIR images of Young et al. (1996), Packham et al. (1997), and Lumsden et al. (1999). However, this region does not stand out in the optical ionized line images of Bruhweiler et al. (2001): ionized gas is present but extended over a much larger region than the northeast knot, the excitation from the \[O III\]/H$`\beta `$ ratio is much lower right at the position of the knot than the surrounding gas, and the extinction inferred from the H$`\alpha `$/H$`\beta `$ ratio is only moderate.

The northeast knot is intriguing because it is clearly on the outside edge of the NE radio lobe observed by Wilson & Ulvestad (1983, 1987). In fact, it looks like the jet is forced to go past the cloud/knot and cannot penetrate it, as though the cloud is a massive obstruction in the path of the radio lobe. There is a fainter cloud of polarized emission immediately north of the radio lobe as well. Wilson & Ulvestad (1987) suggested that the radio polarization morphology and limb brightening of the NE radio lobe are caused by the high energy particles of the lobe encountering the interstellar medium of the galaxy in a radiative bow shock. Schinnerer et al. (2000) measured 12CO (1–0) emission about $`5^{\prime \prime }`$ north of the nucleus. Their largest cloud is just east of the location of the northeast knot — we suggest that the northeast knot and the north cloud are the edges of this cloud that is facing the AGN nucleus. This, then, would be the cloud that is confining the NE radio lobe. Miller et al. (1991) also suggested that the northeast knot has a direct view of the nucleus and its ionizing radiation.

### 3.5 Electron Scattering vs. Dust Scattering

It has been thought for some time that the ultraviolet and visible polarized flux in the nucleus of NGC 1068 is due to scattering of light by electrons in the vicinity of the AGN central engine (Miller et al. 1991; Antonucci 1993; Young et al. 1995). Even though scattering by dust is more efficient than electron scattering, electron scattering is suggested for the nuclear region because of the constancy of percentage polarization between UV and visible wavelengths, because the X-rays appear to be scattered in the same way as the visible polarized flux (Antonucci 1993), and because the H$`\beta `$ line shows evidence of broadening from scattering by high temperature electrons (Miller et al. 1991). The Chandra X-ray images (Young, Wilson, & Shopbell 2001) show good correlation with \[O III\] and H$`\alpha `$ images (Capetti, Axon, & Macchetto 1997a; Macchetto et al. 1994; Bruhweiler et al. 2001), which are similar to the HST UV polarized intensity images (Capetti et al. 1995a, 1995b).

If the polarized flux from the unresolved nucleus is due to dichroic absorption, as seems likely, then we should compare the non-nuclear NIR polarized flux to the visible polarized flux to investigate whether both are due to electron scattering. Inglis et al. (1995) measured the polarized spectrum from 0.43 to 0.69 µm in a $`1.3^{\prime \prime }\times 2.7^{\prime \prime }`$ slit at a number of positions, including positions $`2.5^{\prime \prime }`$ and $`5.0^{\prime \prime }`$ NE of the nucleus. In the position $`2.5^{\prime \prime }`$ NE of the nucleus (approximately the NE lobe), their polarized flux decreased from about 0.4 mJy to about 0.2 mJy over this spectral range. Our measured polarized flux in the same aperture is $`0.54`$ mJy after we subtract 0.15 mJy for the contribution from the diffraction spikes from the nucleus, as computed using TinyTIM (the actual peak of the NE lobe is a little further to the West and has larger flux). The position $`5.0^{\prime \prime }`$ NE of the nucleus is that of the northeast knot. Here Inglis et al. (1995) found that the polarized flux decreased from about 0.35 mJy at 0.45 µm to about 0.25 mJy at 0.69 µm. In a similar aperture we measured the polarized flux to be 0.65 mJy in the northeast knot at 2.0 µm. For both positions the ratio of the polarized flux at 2.0 µm divided by the polarized flux at 0.6 µm is the same, $`2.2`$. We conclude that the scattering is produced by the same process at both positions.

The possibilities for the large increase in scattered flux from visible to NIR wavelengths at these two non-nuclear positions are: 1) The intrinsic NGC 1068 Seyfert 1 AGN spectrum increases rapidly between visible and NIR wavelengths, even more so than is typical (Ward et al. 1987 observed that the fluxes, $`f_\nu `$, of bare, unreddened Seyfert 1 galaxy nuclei increase from visible to NIR wavelengths even though they are flat or increase from visible to UV wavelengths). Such an increase in the nuclear flux could be due to an increasing contribution from the very hot dust immediately surrounding the central engine. 2) The polarized flux seen at visible wavelengths suffers substantial extinction such that the apparent flux ratios of NIR/visible are much larger than the intrinsic ratios. If this were the case, though, there should be almost no UV polarized flux and the slowly decreasing polarized flux as a function of visible wavelength (Miller et al. 1991; Inglis et al. 1995) would not be observed. 3) There are systematic effects such that NICMOS measures too much polarized flux. Overall, however, our nuclear photometry at 2.0 µm is consistent with other NIR measurements (Weinberger et al. 1999; Lumsden et al. 1999; Packham et al. 1997), given the steep rise in nuclear flux between the H and K bands, and thus we do not consider this possibility to be likely.

For the northeast knot, Miller et al. (1991) suggested that the polarization was caused by dust scattering because the percentage polarization decreases rapidly to long wavelengths over the observed wavelength range of 0.33 to 0.65 µm and because the polarized H$`\beta `$ line is narrower than the nuclear polarized H$`\beta `$ line. An additional reason for rejecting electron scattering as the cause of the polarization in the northeast knot is that the northeast knot is not particularly prominent in the Chandra image (Young et al. 2001), whereas electron scattering is suggested as dominating the nuclear polarized flux because of the similarity of appearance of the nucleus in UV and X-ray images. If the scattering is also caused by dust at 2 µm, the nuclear source flux would have to increase by much more than a factor of 2.2 since the scattering cross section for interstellar dust decreases by a factor of order 20 from 0.6 µm to 2.0 µm. Inglis et al. (1995) modeled the scattering and polarization in the central regions of NGC 1068 with a model that includes both electron and dust scattering (Young et al. 1995). They could not get a fit to their NE positions unless they included electron scatterers as well as dust in the dust cloud at the location of the northeast knot. Packham et al. (1997) also modeled the scattered emission from the northeast knot including both dust and electron scattering and found an acceptable fit. Electrons are certainly present, given the ionization implied by the HST ionized line images (e.g., Capetti et al. 1997a; Bruhweiler et al. 2001; Thompson et al. 2001) and optical spectropolarimetry (Miller et al. 1991; Inglis et al. 1995). This ionization is from photons from the AGN and not from shocks from the radio lobes (Bruhweiler et al. 2001). It is probable that both electrons and dust contribute to the NIR scattered light from these non-nuclear regions.

## 4 Summary

We have used HST NICMOS Camera 2 to observe the polarized light in the center of NGC 1068 at 2 µm with a diffraction-limited PSF of $`0.2^{\prime \prime }`$ FWHM. These results are one more example of the value of high-spatial-resolution observations in disentangling complex sources, here the nucleus of one of the most important AGNs. Since the clouds emitting the polarized light measured in this study see the AGN directly, future work with a similar high-spatial-resolution instrument might be to observe any nuclear activity through its echos in the polarized flux as a function of nuclear distance (light travel time) (e.g., Gallimore et al. 2001).

The nucleus is an intense unresolved source, polarized at a level of 6% with a position angle of $`122^{}`$. It is likely that the polarization is due to dichroic absorption in the overlying molecular cloud.

There are two polarized lobes extending up to $`8^{\prime \prime }`$ northeast and southwest of the nucleus. The polarized flux in both lobes is very clumpy, with the maximum polarization occurring in the southwest lobe at a level of 17% when smoothed to $`0.23^{\prime \prime }`$ resolution. The southwest lobe is probably located behind the plane of the galaxy because it cannot be detected at optical wavelengths in spite of the fact that it is much brighter than the northeast lobe at 2 µm in polarized flux. The perpendiculars to the polarization vectors in these two lobes all point back to the intense unresolved source within one $`0.076^{\prime \prime }`$ Camera 2 pixel, thereby confirming that this is the illuminating source of the scattered light and therefore the probable AGN central engine.

Features in the polarized lobes include a gap at about $`1^{\prime \prime }`$ between the nucleus and the southwest lobe and a “knot” of emission about $`4.5^{\prime \prime }`$ northeast of the nucleus. Both features had been discussed by previous authors, but they are much better defined with the high spatial resolution of NICMOS. The gap is sufficiently deep that we suggest that it is due to a lack of scatterers rather than absorption by the overlying molecular cloud seen in CO by Schinnerer et al. (2000). The polarized flux images were aligned with the radio map of Wilson & Ulvestad (1983) by assuming that the intense point source is located at the same coordinates as the radio source S1 (it is within $`0.1^{\prime \prime }`$ of S1 according to the measurements of Capetti et al. 1997b and Thompson et al. 2001). The northeast knot and the north cloud may be the side of a molecular cloud that is facing the nucleus. This cloud may be constraining the northeast radio lobe at the head of the radio synchrotron-emitting jet.

Polarized fluxes were measured for positions $`2.5^{\prime \prime }`$ and $`5.0^{\prime \prime }`$ NE of the nucleus in a $`1.3\times 2.7^{\prime \prime }`$ aperture to correspond to the positions measured by Inglis et al. (1995) from 0.43 to 0.69 µm. (The nucleus position was not considered because of the severe effects of extinction at the shorter wavelengths and excess polarized flux from dichroic absorption at 2 µm.) The ratio of 2 µm flux to 0.6 µm flux is $`2.2`$ at both positions. This could indicate that the same scattering processes obtain at both $`2.5^{\prime \prime }`$ and $`5.0^{\prime \prime }`$ from the nucleus. It also shows that the intrinsic spectrum of the NGC 1068 nucleus increases strongly from optical wavelengths to NIR wavelengths, making it difficult to determine the wavelength dependence of the cross sections of the materials doing the scattering, since the intrinsic spectrum of the source of the scattered light — the AGN central engine — is not known.

## 5 Acknowledgements

We thank A. Wilson for the 4.9 GHz radio map used in the Fig. 3 overlay, M. Rieke for the dark correction files, R. Thompson for providing a preprint, J. Dotson and M. Rabbette for reading the manuscript, and E. Stobie for producing such excellent image manipulation software at Steward Observatory. We also thank the referee for his comments and suggestions, which we have used to improve the presentation. JPS and ASBS acknowledge support from NASA/Ames Research Center Research Interchange grants NCC2-647 and NCC2-1134 to the SETI Institute.