text:
# 2MASS Studies of Differential Reddening Across Three Massive Globular Clusters

## 1 INTRODUCTION

The giant Galactic globular cluster $`\omega `$ Centauri (NGC 5139, hereafter $`\omega `$ Cen) is not only the most massive globular cluster in the Milky Way, but its stars have also been shown to have a large spread in both age and metallicity (see for instance, many contributions in van Leeuwen et al. 2002). In light of these discoveries, it has been proposed (Lee et al. 1999, Majewski et al. 2000) — and now dynamically modeled (Tsujimoto & Shigeyama 2003) — that $`\omega `$ Cen may be the nucleus of a disrupting dSph, similar to the Galactic globular cluster M54. The second largest globular cluster, M54 is currently believed to be the core of the Sgr dSph (Da Costa & Armandroff 1995, Bassino & Muzzio 1995, Sarajedini & Layden 1995, Layden & Sarajedini 2000) and lies at a nucleation point for the several different metallicity populations of Sgr (Sarajedini & Layden 1995, Majewski et al. 2003). The recent observation of apparent tidal tails around $`\omega `$ Cen by Leon, Meylan & Combes (2000, hereafter LMC00) bolsters the notion of this cluster being the remnant of a larger, now disrupted parent object, and, with $`\omega `$ Cen and M54 as models, Ree et al. (2002), Lee et al. (2001, 2002), and Yoon et al. (2000) have proposed that other clusters with evidence of multiple populations, such as clusters with bimodal horizontal branches, may also be the nuclei of disrupting dSph satellite galaxies. Two clusters proposed to be M54 analogues (Ree et al. 2002) are NGC 6388 and NGC 6441, the third and fifth largest globular clusters in the Milky Way respectively, since both have observed characteristics suggesting internal metallicity spreads. Moreover, high mean metallicity clusters like these (see Table 1) should have only a red stub of a horizontal branch (HB), but NGC 6388 and NGC 6441 also have extended blue HB tails (Piotto et al. 1997, hereafter P97). A bimodal (or extended) metallicity distribution may explain both this curious HB morphology (P97) as well as the unusually wide red giant branches (RGB) observed in these clusters. Because it is difficult to understand how multiple generations of stars might be created in a stellar system of initial mass of order $`10^{56}M_{}`$ (which is not massive enough to retain significant amounts of gas after an initial starburst), it is possible that the multi-population clusters are the remnants of formerly larger stellar systems, for example dSph galaxies, where multi-epoch star formation histories are commonly found. The verification of the existence of additional multiple-population clusters would increase the need to formulate a general process — e.g., the disruption of initially larger systems — that leads to a more substantial class of these objects.

These scientific questions are complicated by the presence of non-negligible differential reddening in the fields of some the most interesting examples of potential multi-modal clusters. In this paper, we concentrate on two effects of differential reddening relevant to the globular cluster evolution picture outlined above: the potential creation of false signatures of both tidal tails around, and of multiple metallicities within, globular clusters. These issues are explored in case studies of the three massive globular clusters $`\omega `$ Cen, NGC 6388 and NGC 6441.

Tidal tail searches around Galactic satellites can be particularly vulnerable to differential reddening effects because search techniques typically rely on the detection of star count overdensities. Lines of sight with a lower dust column density will naturally reveal more stars to a given magnitude limit. This may create an apparent overdensity of stars in regions with lower extinction that could be mistaken for, or overemphasize, density enhancements from tidal tails. In the case of the reported tidal tails around $`\omega `$ Centauri by LMC00, the contours of the detected tails curiously and closely correspond to contours of the prominent IRAS $`100\mathrm{\mu m}`$ dust emission (and therefore, presumably, optical extinction by dust) in this field, as these authors have observed. Because LMC00 did not correct for differential extinction in their optical starcounts (S. Leon, private communication), at minimum there must be some dust-shaping of the morphology of mapped $`\omega `$ Cen tidal tails. We re-evaluate the possible impact of differential reddening on large field studies of $`\omega `$ Cen by comparing a customized tidal tail search on raw 2MASS data to one performed on 2MASS data differentially dereddened according to the dust maps of Schlegel et al. (1998).

The cause for the observed width of the RGB in NGC 6388 and NGC 6441 has been variously explained as differential reddening and/or the presence of multiple populations of different metallicity, with no clear consensus emerging. P97 originally concluded that differential reddening on scales greater than eight arcseconds was unlikely for NGC 6388 but possible for NGC 6441, although only to a few hundredths of a magnitude. Later evidence indicating the presence of strong differential reddening on small scales across both clusters was found by Raimondo et al. (2002) using HST-WFPC2 small area surveys similar to those used by P97. Pritzl et al. (2001, 2002) also find differential reddening in NGC 6441 and NGC 6388 using CTIO $`B`$ and $`V`$ band photometry, although they note that in NGC 6388 the differential reddening probably covers a smaller magnitude range. If differential reddening were significant, the reddening vector could reproduce the sloping red HBs and spread the RGB stars perpendicular to the unreddened RGBs, inflating their apparent width. Numerical experiments performed by Raimondo et al. (2002) indicate that differential reddening of order $`\mathrm{\Delta }`$E$`(BV)`$ 0.1 would be substantial enough to cause an unreddened HB of similar metallicity (e.g., that of 47 Tucanae) to resemble a sloping red stub similar to that of NGC 6388 or NGC 6441. Nevertheless, Raimondo et al. conclude that reddening is not the reason for the strange HB’s since their sloping nature is present even in small area subsamples exhibiting a thin RGB, which suggests that a second effect is at work.

Pritzl et al. (2001, 2002) and Ree et al. (2002) argue that variable metallicity may adequately explain the observed features of the NGC 6388 and NGC 6441 CMDs, and recent stellar evolutionary models (e.g., Sweigart & Catelan 1998) confirm that a metallicity spread can naturally create a sloping HB. Specifically, Ree et al. (2002) find that two distinct populations separated by 1.2 Gyr and 0.15 dex in \[Fe/H\] can reproduce both the observed wide RGBs and tilted HBs in the clusters. However, Pritzl et al. (2001, 2002) argue that the simple two-population model fails to reproduce the observed HB slope and that a more general metallicity spread is necessary.

Because selective extinction at the 2MASS wavelengths $`J`$ (1.25 $`\mathrm{\mu m}`$), $`H`$ (1.65 $`\mathrm{\mu m}`$), & $`K_s`$ (2.17 $`\mathrm{\mu m}`$, hereafter simply referred to as $`K`$) is roughly one tenth that in visual bands, the magnitude of differential reddening at these wavelengths is commensurately reduced. Thus, we apply the 2MASS point source catalog to test whether: (1) differential reddening affects interpretations of an infrared search for tidal tails around $`\omega `$ Cen, and (2) the NGC 6388 and NGC 6441 RGBs are broadened at near infrared wavelengths where differential reddening will have less impact on the width of the cluster RGB’s.

## 2 DATA

$`J`$, $`H`$, and $`K_S`$ band data were drawn from the 2MASS all-sky point-source working survey database as of July, 2002. Initial values of the mean field extinction, $`E(BV)`$, and distance modulus, $`(mM)`$, for each cluster were obtained from the Harris compilation as posted on the World Wide Web (http://physun.physics.mcmaster.ca/Globular.html) on June 22, 1999 (see Harris 1996). These were converted to the required infrared extinction parameters using the standard extinction models of Cardelli, Clayton, and Mathis (1989). Table 2 gives a complete list of the adopted extinction parameters.

## 3 $`\omega `$ CENTAURI: A 2MASS TALE

### 3.1 Tidal tail search method

The LMC00 study of optical starcounts in the $`\omega `$ Cen field makes no corrections for differential reddening because magnitude calibrations were not available for the majority of clusters in their study (S. Leon, private communication). As Figures 1a and 1b illustrate, the proposed tidal tails of $`\omega `$ Cen are from enhanced star counts whose pattern appears to follow regions of lower dust absorption on the IRAS $`100\mathrm{\mu m}`$ map. This effect is particularly noticeable on the western edge of the cluster. LMC00 note and discuss the possibility that the star counts are affected by reddening, but determine that the tidal tails are of a sufficiently large S/N that differential reddening should not significantly affect their tidal tail search.

By comparing starcount patterns obtained using raw 2MASS data with those from 2MASS data differentially dereddened according to the dust maps of Schlegel et al. (1998) we intend to re-evaluate whether uncorrected differential extinction can produce a false tidal tail signal. The 2MASS data has a $`99\%`$ completeness limit at $`J=16`$ mag in the direction of $`\omega `$ Cen, yielding almost 500,000 stars within a $`6^{}\times 6^{}`$ box centered on the cluster. Because a photometric search for diffuse tidal tails presents considerable challenges, not least of which is substantial field star contamination in this low latitude field, a two stage method is employed to minimize field star contamination and probe for starcount excesses as deeply into the surrounding field as possible: First, a coarse filtering mask in color-color-magnitude (CCM) space is used to extract only those stars whose CCM location is similar to that of stars observed at radii less than the cluster core radius. Although this coarse mask does not take into account foreground/background contamination of the cluster core, it filters out a significant number of stars whose CCM locations are significantly different from those in the cluster. Second, we then apply a signal-to-noise filter similar to that used by Odenkirchen et al. (2001) to separate cluster stars from field stars and emphasize starcount overdensities in the potential tidal tails.

In the first stage of filtering, a filtering mask is applied in CCM space to pre-select those CCM locations containing stars within 100 arcseconds of the center of the cluster (approximately $`2/3`$ the core radius $`r_c`$, see Table 1). We divide this irregularly shaped region in ($`J`$, ($`JH`$), ($`JK`$)) space into a $`50\times 50\times 50`$ grid in CCM space bounded by the limits of the CCM region; this grid size is selected to achieve a balance between coarse binning and low number statistics. With the individual cells described by indices ($`i`$, $`j`$, $`k`$) respectively, we define $`n_C(i,j,k)`$ as the number of stars within 100 arcseconds of the cluster center that fall within each grid cell in CCM space. To obtain an estimate of the number of field stars included in $`n_C(i,j,k)`$, we represent the field star distribution, $`n_F(i,j,k)`$, by the stars within an annulus $`100\stackrel{}{\mathrm{.}}0r150\stackrel{}{\mathrm{.}}0`$ (roughly three times the tidal radius of the cluster). We assume that the distribution of field stars is relatively even across the cluster and scale the field mask by the ratio $`q`$ of the areas over which the field and core regions were summed. The selection mask $`P`$ is defined as the ratio of the core mask to the sum of the core and scaled field masks.

$$P(i,j,k)=\frac{n_C(i,j,k)}{n_C(i,j,k)+q^1n_F(i,j,k)}$$
(1)
$`P(i,j,k)`$ will have a value near one for regions of CCM space where the core sample $`n_C(i,j,k)`$ is not substantially contaminated with field stars, and will decrease as contamination becomes more severe. The complete data set is filtered using this CCM selection mask: A star is rejected from the data set if the corresponding $`P(i,j,k)`$ is less than some minimum value $`P_{min}`$. This algorithm is a slightly more sophisticated means to eliminate stars unrelated to $`\omega `$ Cen than simple preliminary cuts in the color-magnitude (CM) plane. We find that a value of $`P_{min}=0.9`$ allows us to select cluster stars over field stars most optimally and appears to preserve the CMD of the cluster center best. This preliminary filtering rejects roughly 90% of the stars in the original data set.

In the second stage of filtering, we apply a modified version of Grillmair et al.’s (1995) method in the CM plane as described by Odenkirchen et al. (2001). This second stage eliminates field stars more effectively than the first stage, but we find that the method does not perform well unless a coarser pre-selection of stars is performed first to reduce the data set to a size that the more computationally complex second-stage routine can handle. With some recycling of notation, we define a grid in $`(J,(JK))`$ space, with indices ($`j`$, $`k`$) respectively, and find the signal- to-noise ratio ($`S/N`$) of probable cluster stars to field stars in each CM grid cell. We count the number of stars $`n_C(j,k)`$ that fall within each grid cell and lie within $`r=20\stackrel{}{\mathrm{.}}0`$ of the cluster center (chosen to correspond to the region in which LMC00 observe circular cluster surface density contours, see Fig. 1a), and the number of stars $`n_F(j,k)`$ that fall within the CM grid cell and lie between $`r=100\stackrel{}{\mathrm{.}}0`$ and $`r=150\stackrel{}{\mathrm{.}}0`$ arcminutes from the cluster center. The field star population is assumed to be uniform, and the expected number of field stars of a given ($`j`$, $`k`$) that fall within $`r=20\stackrel{}{\mathrm{.}}0`$ of the cluster core is subtracted from $`n_C(j,k)`$. With $`q`$ representing the ratio of the area of sky on which $`n_F`$ and $`n_C`$ respectively have been defined, the local S/N ratio of cluster to field stars for each grid cell ($`j`$, $`k`$) is, by standard Poissonian statistics,

$$s(j,k)=\frac{n_C(j,k)q^1n_F(j,k)}{\sqrt{n_C(j,k)+q^2n_F(j,k)}}$$
(2)
It is possible to use $`s(j,k)`$ as a mask for selection in the CM plane by eliminating all $`(j,k)`$ grid cells with $`s(j,k)`$ less than some limiting value $`s_{lim}`$, which is chosen such that the number of stars in the tidal tail region (as reported by LMC00) is a maximum relative to the field star population. The new matrix $`n_T(j,k)`$ represents the total number of stars in each $`(j,k)`$ grid cell for the region111We describe this region as the two sections of an annulus (of inner radius $`50\stackrel{}{\mathrm{.}}0`$ and outer radius $`100\stackrel{}{\mathrm{.}}0`$) located directly north and south of the circle described by the inner radius. of the cluster in which tidal tails have been reported by LMC00 (see Fig. 1a), as shown by the dashed lines in Fig. 1a. Starting from $`s_{lim}=s_{max}`$ (where $`s_{max}`$ is the maximum value of all $`s(j,k)`$) and iterating downward to $`s_{lim}=0`$, we calculate $`N_{tail}=_{j,k}n_T(j,k)`$ and $`N_{field}=_{j,k}n_F(j,k)`$ for all ($`j`$, $`k`$) such that $`s(j,k)s_{lim}`$. Hence, $`N_{field}`$ and $`N_{tail}`$ respectively are the total number of field stars and cluster stars in LMC00’s tidal tail regions whose locations in $`(J,(JK))`$ space satisfy $`s(j,k)s_{lim}`$. The optimal value of $`s_{lim}`$ is that which maximizes $`N_{tail}/N_{field}`$; hence this final selection mask filters the data set for all stars which fall within cells in CM space such that $`s(J,(JK))s_{lim}`$.

The positions of stars from this filtered set are then converted to a smooth density distribution by application of a two-dimensional Gaussian kernel with an adaptive half maximum width (e.g., Silverman 1986) for each star set to the distance of the 30th nearest neighbor in the plane of the sky. Each of these Gaussians is evaluated at finely spaced grid points in the plane of the sky, and these values are summed to create an image of relative stellar densities at each location on the grid (Fig. 1c and 1d) 222The reduced field of view of these figures is chosen to correspond to that of LMC00..

We find that this $`S/N`$ optimization method worked efficiently to reduce the 2MASS data set by a further 80-90% and brought out starcount overdensities around the cluster that have the appearance of tidal tails. Focusing the definition of $`n_T`$ on the regions found by LMC00 to possess tidal tails is considerably more effective at bringing these apparent tails out of the background than defining $`n_T`$ more generally; however, this particular approach obviously is not useful for a general survey for previously unknown tidal tail searches because it requires a priori knowledge of where tails may be found. We verified that the overdensities found by this method are not merely artifacts of the definition of $`n_T`$ in the region of the LMC00 expected tails by redefining $`n_T`$ in locations rotated around the cluster core by 90 degrees (that is, protruding east-west as opposed to north-south). Using this definition of $`n_T`$, no starcount overdensities were detected around the cluster, which supports that the apparent tails found with the first $`n_T`$ definition had not been an artificial product of the processing method and that the method of sampling $`n_T`$ in the tail region is a crucial step in bringing those tails out of the background noise.

### 3.2 Results

In the analysis of the raw 2MASS data described above a starcount excess to the north and south of the cluster is found. As Fig. 1c shows, to within the limits imposed by the smoothing kernel, the location of the 2MASS tails corresponds to that of the tails found by LMC00.

The same analysis has been repeated with 2MASS data dereddened according to the extinction maps of Schlegel et al. (1998)333We have used the programs available on the World Wide Web at http://www.astro.princeton.edu/~schlegel/dust/index.html. The Schlegel et al. maps have a resolution of approximately 6.1 arcminutes, which corresponds to about 1500 cells of reddening data over a $`4^{}\times 4^{}`$ field of view, and show differential reddening of order $`\mathrm{\Delta }E(BV)=0.18`$ mag over that field of view. In contrast to what is found for the raw 2MASS data, the results of the analysis on the dereddened 2MASS data do not reveal any elongation of the circular cluster contour lines or any overdensities resembling tidal tails (Fig. 1d). Fig. 2 plots radial surface density profiles for both original and dereddened 2MASS data, and shows the surface density of the dereddened cluster to fall off smoothly with radius, like a King (1966) profile. In contrast, the surface density of stars in the non-dereddened cluster data appears to drop at $`\mathrm{log}(r)=1.8`$ ($`r`$ 60 arcmin) — corresponding to the low starcount regions at this radius to the east and west of the cluster — and rise again at $`\mathrm{log}(r)=2.0`$ ($`r=100`$ arcmin) — which corresponds to high starcount regions to the north and south of the cluster (Fig. 1c).

Given the fidelity with which the apparent infrared and optical tails follow the regions of lower reddening on the IRAS 100 micron map (Fig. 1b, particularly on the western and southern edges of the cluster), and the disappearance of the infrared tails in the dereddened cluster data, we speculate that by analogy the $`\omega `$ Cen tidal tails previously detected with the optical starcounts, which are more affected by dust extinction, could have been artifacts of differential reddening. We must also point out, however, that our test of the 2MASS database is not completely analagous to the LMC00 analysis: Due to the $`J=16`$ mag completeness limit of 2MASS we are unable to probe to the same effective depth as the LMC00 study. The differences in relative depth are demonstrated by the fact that while LMC00 observe reasonably circular contours out to a radius of about 45 arcminutes (roughly the tidal radius, see Table 1), we only find circular contours out to about 40 arcminutes before we begin to observe distinct elongation of the contours into potential tidal tails (Fig. 1). Thus, the Leon et al. counts reach fainter surface brightness limits than does 2MASS and are sensitive to more diffuse, lower density tails than are we. Nevertheless, we regard the 2MASS test presented here as a cautionary example demonstrating that a proper accounting of differential reddening is an important step in mapping any tidal tails in the $`\omega `$ Cen (or any other heavily obscured) field.

## 4 DIFFERENTIAL REDDENING IN THE FIELDS OF NGC 6388 AND NGC 6441

### 4.1 Processing

To investigate the effects of differential reddening on the width of the RGB in the clusters NGC 6388 and NGC 6441 we use the uniform photometry of 2MASS to look for spatial patterns in the distribution of red giant star colors within each cluster. 47 Tucanae (NGC 104, hereafter 47 Tuc) is adopted as a control case for a cluster of similar mean metallicity but with little differential reddening (average $`E(BV)`$ 0.02). We also compare our results to the $`\omega `$ Cen data — although this cluster has an overall \[Fe/H\] much lower than the other clusters, it nonetheless provides a useful example of the effects of variable metallicity within a cluster over spatial scales with relatively little differential reddening on the scale of arcminutes ($`\mathrm{\Delta }E(BV)=0.01`$ mag, note that the previous section explored extinction variations in $`\omega `$ Cen on the scale of degrees).

#### 4.1.1 ($`JK`$) Color Distributions

Analysis of RGB color spreads at different magnitudes in one cluster as well as differences between clusters is simplified if we remove the mean RGB color-magnitude relation for each cluster (i.e. make the center of the RGB locus for each cluster appear vertical in the color-magnitude diagram). Since the \[Fe/H\] of the unreddened cluster 47 Tuc is similar to that of NGC 6388 and NGC 6441 to within about 0.25 dex (see Table 1), the shape, height and position of the RGBs for the three clusters in a reddening corrected, absolute color-magnitude space should be similar. Hence, we use the well-defined 2MASS CMD of 47 Tuc to “straighten” the RGB of the clusters in our sample (Figure 3a). We convert apparent to absolute magnitudes using the distance moduli in Table 2 and consider only that part of the RGB with absolute magnitude $`M_K2`$ to minimize contamination from cluster horizontal branch (HB) as well as field stars. A Legendre polynomial is fit to the 47 Tuc RGB, with a rejection threshhold of 2$`\sigma `$ iterated until convergence. The final polynomial is converted to power series coefficients up to third order, which are used to straighten the RGBs of all four clusters. We define an adjusted $`(JK)`$ color given by:

$$(JK)_A=(JK)E(JK)(a+b\times M_K+c\times M_K^2)$$
(3)
where $`E(JK)`$ values are given in Table 2, and the coefficients $`a`$, $`b`$, and $`c`$ are power series coefficients given in Table 3. Similar equations are derived for the $`((JH),K)`$ and $`((HK),K)`$ CMDs.

Using the adjusted values, histograms of colors readily show that the RGBs of NGC 6388 and NGC 6441 (Figure 4) are wider than those of 47 Tuc and even that of $`\omega `$ Cen, which is well established to have a metal spread from \[Fe/H\] $``$ -2.2 to -0.7 (Vanture, Wallerstein, & Suntzeff 2002). The latter result is somewhat surprising, given the fact that the metal-rich RGB of 47 Tuc provides a poor approximation to the metal-poor RGB of $`\omega `$ Cen and produces a clear tilt in the “adjusted” $`\omega `$ Cen RGB (Fig. 4). The Fig. 4 histograms are divided into five $`(JK)_A`$ colors bins in order to study the spatial distribution of RGB stars by color. The five color bins from the lowest value of $`(JK)_A`$ (most blue) to the highest (most red) are referred to as bins A through E, respectively. Because globular clusters are dynamically mixed on timescales that are short compared to chemical enrichment, there should be no significant difference in the spatial distribution of the stars in these five bins. For example, we show below (Section 4.2.1) that neither 47 Tuc nor $`\omega `$ Cen exhibit any such spatial variation. On the other hand, if differential extinction were significant, the distributions of stars by color would be uneven across the face of the cluster as a reflection of reddening variations. This is a test we now apply to NGC 6388 and NGC 6441.

Various methods to divide the sample population into color bins were explored. The results are not very sensitive to which method is chosen. The following technique ensures similar treatment of each cluster: Bin ranges are assigned to each cluster using a $`(JK)_A`$ histogram of a subsample of the purest cluster stars, defined by an angular radius where the density of stars decreased to a limiting density given by:

$$LimitingDensity=(1/e^3)\times (CentralDensityOuterDensity)$$
(4)
(where the outer density is determined as the density of stars at the edge of the field of view). In all cases, this resulted in a sample of stars within a radius of between roughly one and three arcminutes, which is outside the core radius but well within the tidal radius of each cluster (see Table 1). Using a color histogram with binning width 0.02 mag, the central three bins are defined within the $`(JK)_A`$ range from the central peak to the colors where counts decrease to $`1/e`$ of the central peak. Bin C is centered at the peak of the histogram and has 1/3 of the width of B+C+D. The range of bin A is determined as the color range blue-ward of bin B up to where the counts first drop to one (to eliminate extreme colored stars from the sample). Likewise, bin E is defined as the range red-ward of bin D up to where the counts first drop to one (Fig. 5). Field star contamination (or contamination by non-RGB stars in the cluster), not differential reddening, is assumed to be the reason for stars that lie very far from the fiducial RGB, and the described culling of the data set promises to retain only those stars that make up the bulk of the RGB and that could potentially have been scattered to their current locations in the CMD by differential reddening. These color bin definitions are applied to the full 300 arcsecond data set. Similar binnings were created for $`(JH)_A`$ and $`(HK)_A`$ histograms, although only the $`(JK)_A`$ and $`(JH)_A`$ distributions were found to be useful in our analysis (see below).

#### 4.1.2 ($`JH`$) and ($`HK`$) Colors

Before proceeding further, we apply one additional filter designed to preserve reddening effects and reduce noise from astrophysical and photometric scatter. This filter takes advantage of the fact that the degree of scatter due to reddening in one color should be correlated to that in another color. A Legendre polynomial with an iterative $`2\sigma `$ rejection threshhold (similar to that used earlier in Section 4.1.1) was fit to the cluster $`(JK)_A`$ vs. $`(JH)_A`$ color-color diagram (Fig. 6), and color-color cuts were performed to eliminate stars at greater than $`2\sigma `$ from the Legendre fit. This independently derived fit for each cluster is very close to the fiducial reddening vector for standard extinction models with $`R_V=3.1`$, lending further evidence to support the presence of differential reddening across each cluster. Figs. 6 and 7 show color-color plots of $`(JK)_A`$ versus $`(JH)_A`$ and $`(HK)_A`$ color, respectively, with the adopted outlier cuts shown.

Note that the dynamic range in color of $`(HK)_A`$ is small compared to the reddening effect sought, and it is difficult to determine which stars appear redder than average as a result of differential reddening instead of photometric errors or astrophysical reasons. We therefore reject $`(HK)_A`$ color from our analysis, since it does not have the discriminating power required to pick out the required distribution from the noise of points in Figure 7.

### 4.2 Results and Discussion

#### 4.2.1 Five Bin Results

Spatial plots of the stars in each of the $`(JK)_A`$ bins for 47 Tuc, $`\omega `$ Cen, NGC 6388 and NGC 6441 are shown in Figures 8 - 11 respectively. As expected for a monometallic population with little reddening, 47 Tuc shows no spatial segregation of the stars in the five bins. Although giant stars in $`\omega `$ Cen have been found to exhibit a spatial variation in metallicity (Jurcsik 1998), this variation is on scales of roughly 20 arcminutes; in contrast, the mixed metallicity population in our smaller field of view shows no spatial segregation by RGB color. Both 47 Tuc and $`\omega `$ Cen have few stars in bins A and E relative to their total population. In contrast, NGC 6388 exhibits a pronounced abundance of redder stars (bins D and E) in the northern and northwestern regions of the cluster compared to southern regions (Fig. 10), and the extreme bins A and E each contain significantly more stars relative to the total population of the cluster than these bins contained for 47 Tuc and $`\omega `$ Cen. Comparison to the 47 Tuc and $`\omega `$ Cen templates for monometallic and multi-metallic clusters (respectively) having comparatively little differential reddening over small scales leads one to the conclusion that it is unlikely that the NGC 6388 color spread is due primarily to metallicity effects. Rather, Fig. 10 suggests that differential reddening on small scales is causing (1) a greater number of stars to appear in bins A and E, and (2) large spatial variations by observed stellar color.

Likewise, NGC 6441 (Fig. 11) shows an enhancement of redder stars (bins D and E) in the southeastern quadrant of the field of view, supporting the view that it too is subject to substantial differential reddening. Using isochrone fitting in subfields, Heitsch & Richtler (1999) similarly find that the southeast region of NGC 6441 has higher $`E(BV)`$ than the northwestern region by about 0.15 mag. Layden et al. (1999) concur with Heitsch & Richtler to within a few hundredths of a magnitude using RR Lyrae stars and CMDs in sectors of an annulus around the cluster center. However, in contrast to the distinct dichotomy in NGC 6388 (redder stars in the northwest, bluer stars in the southeast), the spatial distribution of stars in bins A through C in NGC 6441 appears relatively constant (with the possible exception of decreased counts in the southeastern quadrant of bin A), and the cluster presents the appearance of an inhomogeneously distributed redder population superposed on a centrally concentrated bluer population background. This effect may be an artifact of the severe contamination of the cluster by field stars since NGC 6441 is located in an extremely crowded field near the Galactic center (as vividly demonstrated by the washed out starcount map shown in Figure 12). Indeed, Heitsch & Richtler (1999) note the strong contamination of their own CMD of NGC 6441 by field stars that cover the lower part of the cluster RGB. In this case, strong contamination by blue field stars may contribute to the apparent width of the cluster red giant branch.

#### 4.2.2 Reddening Maps

Figure 12 shows 2MASS starcount density (Panel a) and mean stellar color (Panel b) maps for all four of our clusters. In this figure, higher starcounts and redder mean colors appear whiter (Table 5 provides the numerical values for each greyscale). The field of view of these maps is more than 100 times larger than that used for the analyses in the rest of this section since this method only provides coarse resolution maps which, however, are useful for understanding qualitatively the global character and degree of patchiness of the dust obscuration around each cluster. For example, assuming that the average intrinsic star color in a given direction is relatively uniform over degree scales, the mean color maps can be interpreted as showing the general character of the reddening in the direction of each cluster. However, these maps have too coarse a greyscale to show color variations of order $`E(BV)0.1`$.

Fig. 12 shows that the mean stellar color in the direction of NGC 6388 is not only patchier than that in the direction of 47 Tuc or $`\omega `$ Cen but that the number of redder stars appears to be greater to the north of the cluster than to the south 444Note that this remark applies to the central few pixels immediately around the cluster., in agreement with the trend found above. Distinct regions of higher reddening and lower reddening are demarcated by a boundary that passes close to the center of the cluster. Differential reddening across this apparent dust boundary may produce the observed overabundance of redder RGB stars in the north - northwestern regions of the cluster compared to the south - southeastern regions.

The NGC 6441 field mean color map also shows evidence of patchy extinction, although it is not obvious whether any of these patches could produce the observed reddening patterns near the cluster that are likely the case for NGC 6388. However, the presence of large scale patchiness is sufficient to suggest the possibility of patchiness on the angular scales explored in Section 4.2.1.

#### 4.2.3 Kernel-smoothed color distributions

Histograms of star colors within regions of greater reddening should be offset from those of subregions with lesser reddening. To study the color distributions in subsections of the cluster fields, which will have larger statistical noise, we apply a uniform width-normalized Gaussian kernel (e.g., Silverman 1986) to the $`(JK)_A`$ color distribution of stars in NGC 6388 and NGC 6441. Figure 13 shows these smoothed distributions for the northwest and southeast quadrants of each cluster. The half maximum width of the kernel is the minimum value that produces a clearly defined central peak in the distribution for a given cluster. NGC 6388 required this smoothing width to be only 0.02 mag, while NGC 6441 required a wider Gaussian of width 0.06 mag to smooth out secondary peaks adequately.

The central peak of the distribution of the northwestern quadrant of NGC 6388 is offset from that of the southeastern quadrant by roughly $`\mathrm{\Delta }(JK)_A=`$ 0.061 mag, and the peaks of the northwestern and southeastern quadrants of NGC 6441 are offset by roughly $`\mathrm{\Delta }(JK)=`$ 0.072 mag. Because the width of the kernel used for NGC 6441 is of order the offset between the peaks, the precision of the derived $`\mathrm{\Delta }(JK)_A`$ is less for NGC 6441 than for NGC 6388. Nevertheless, the $`\mathrm{\Delta }(JK)_A`$ results for both clusters are in agreement with the qualitative arguments for differential reddening presented earlier in Sections 4.2.1 and 4.2.2, and our value for the differential reddening across NGC 6441 is consistent with the more detailed optical extinction maps produced by Layden et al. (1999) and Heitsch & Richtler (1999) to within 0.02 mag in $`E(JK)`$. If such an offset exists between the color distribution of stars in different regions of each cluster, then the artificial width produced by the superposition of these offset distributions may contribute to the thicker RGB of the entire cluster. Fig. 14 simulates the effect of superimposing two offset color distributions by artificially reddening the northern half of the monometallic cluster 47 Tucanae by 0.061 magnitudes in $`(JK)_A`$; it is observed that while such a superposition can artificially inflate the width of the color distribution (Fig. 13, lower panels), it is only by about half the amount required to re-create the width of the distributions observed in NGC 6388 and NGC 6441. The inability of this simulation to entirely explain the RGB widths of these clusters may be due to the presence of a large reddening spread (as opposed to the simplistic two value model used here), or it may be indicative of a contributing effect of differential metallicity.

#### 4.2.4 Kolmogorov-Smirnov Statistics

To test the significance of the differences in $`(JK)_A`$ color distribution in different quadrants of each cluster, we applied the Kolmogorov-Smirnov (KS) test to compare spatial color distributions numerically. As a calibration of the method, we first compared the color distribution across the clusters 47 Tuc and $`\omega `$ Cen. As Tables 6 and 7 show, the $`(JK)_A`$ color distribution of stars in, for example, the northwestern quadrant of the clusters correlates with the distribution of stars in the southeastern quadrant with significance levels of about 0.5 and 0.25 for 47 Tuc and $`\omega `$ Cen respectively. These significance levels are consistent with a probability that stars within the two regions could have been randomly drawn from one parent distribution. Tables 6 and 7 demonstrate that this relation significance is roughly similar for comparisons between all quadrants for both 47 Tuc and $`\omega `$ Cen.

Correlations between the southeastern and southwestern, and northeastern and northwestern quadrants of NGC 6388 have significance levels of roughly 0.9 and 0.5 respectively. In contrast, the KS test yields a probability of only 0.002 that the northwestern and southeastern quadrants share a similar color distribution (Table 8). Compare this to the variable metallicity cluster $`\omega `$ Cen, which maintains a relatively high correlation significance of color distribution between all quadrants in spite of its metallicity spread. It appears unlikely that a metallicity spread is the dominant cause of the RGB width in NGC 6388 (we expect mixing in clusters in any case). Instead, the stars in these regions almost certainly come from statistically distinct color populations, pointing again to the presence of strong differential reddening.

Application of the KS test to the $`(JK)_A`$ color distribution in the southeastern and northwestern quadrants of NGC 6441 yields a near zero likelihood of correlation (Table 9). Qualitatively, this is not surprising because there are very few stars from bins D and E in the northwestern quadrant while there are a considerably higher number of redder stars that lie in the southeastern quadrant. However, while these two quadrants are by orders of magnitude the least correlated regions of this globular cluster, correlations between all other combinations of quadrants are only of order 0.01 to 0.1, which suggests (as we have concluded in Section 4.2.2) that reddening in NGC 6441 is patchy on smaller angular scales than in NGC 6388.

If differential reddening is producing the observed width in these clusters, the RGB histograms in sub-regions of each cluster should also be narrower than that of each cluster as a whole. Using HST-WFPC2 visual band data for NGC 6388 and NGC 6441, Raimondo et al. (2002) find this to be the case for both clusters. In summary, it appears very probable that differential reddening is artificially inflating the apparent thickness of the RGB in these two clusters, although the thickness may not be fully explained by this effect alone.

## 5 DISCUSSION AND CONCLUSIONS

Differential extinction complicates photometric studies of Galactic stellar populations, and even nearby globular clusters with $`E(BV)`$ as low as about 0.1 mag may be visibly affected. In our search for tidal tails around the giant globular cluster $`\omega `$ Cen we have found that differentially dereddening 2MASS photometry largely eliminates apparent tidal tail candidates present in data that have not been corrected for extinction. Given the similarity of the false 2MASS tidal tail signature to the optical tails reported by Leon et al, it would be beneficial to verify the diffuse optical starcount distribution in the $`\omega `$ Cen field with deep visual photometry that accounts for the $`\mathrm{\Delta }E(BV)`$ variations.

We have also investigated the effects of differential reddening on the CMDs of the clusters NGC 6388 and NGC 6441, which exhibit abnormally large spreads in their RGBs. The analyses presented make use of spatial maps by stars in color bins, maps of mean stellar color, kernel-smoothed histograms of stellar color in sub-regions, and KS statistical analyses of the distribution of ($`JK`$) and ($`JH`$) colors across the clusters. These tests indicate that the northwestern quadrant of NGC 6388 is subject to about $`\mathrm{\Delta }E(BV)`$ 0.12 mag more reddening than the southeastern quadrant, and that the southeastern quadrant of NGC 6441 is more heavily reddened by about $`\mathrm{\Delta }E(BV)`$ 0.14 mag than the northwestern quadrant. These findings are consistent with previous results from Heitsch & Richtler (1999) and Layden et al. (1999). As numerical experiments performed here and by Raimondo et al. (2002) have shown, this amount of differential reddening could produce anomalously wider red giant branches for these clusters (in addition to the tilt observed in their horizontal branches, as demonstrated by Raimondo et al.). Raimondo et al. have shown that the RGB in sub-regions of these clusters is thin (as expected if differential reddening were the cause of the width of the RGB across the entire cluster). Athough we find that possible width inflation due to the superposition of thin RGBs from two quadrants subject to different amounts of differential extinction may not be sufficient to completely explain the observed width of the cluster RGBs, it may be possible that the RGB width could be explained more fully by a more realistic superposition of RGBs from smaller subregions. Therefore, if not accounting for the entire RGB-widening effect, at minimum differential reddening may be exaggerating metallicity-induced RGB spreads and the actual spread in age or metalliticy within the clusters is likely not as large as has previously been suggested (e.g., Pritzl et al. 2001, 2002).

The number of known globular clusters showing clearly anomalous metallicity spreads remains small. With few bona fide members, this “class” of globular cluster can still comfortably be accomodated by origin pictures that are exceptional, e.g., along the lines of the M54 - Sgr paradigm. However, even in the obvious case of the multi-population $`\omega `$ Cen, the M54 analogy may be less compelling than previously thought, and the picture that some globular clusters are the “residue” of dSph disruption more ambiguous.

The authors would like to thank S. Leon and G. Meylan for helpful discussion regarding the effects of reddening on $`\omega `$ Centauri. This publication makes use of data products from the Two Micron All Sky Survey, which is a joint project of the University of Massachusetts and the Infrared Processing and Analysis Center, funded by the National Aeronautics and Space Administration and the National Science Foundation.


format: text

text:
# The radiative ℎ⁰(𝐴⁰)→𝛾𝛾 decays in the two Higgs doublet model.

## 1 Introduction

The radiative decays $`S\gamma \gamma `$ (S: scalar, pseudo-scalar particle) are interesting to analyze since the output is the two photon system and it is rich due to two different polarizations of photons, namely the parallel and the perpendicular ones. This leads to two different amplitudes, CP odd and CP even, and the measurement of the photon spin polarization provides comprehensive information about the model parameters existing in the amplitudes. Besides this, such decays are the strong candidates to study the direct or indirect CP violation.

There are various studies on the two photon system in the literature. The decays of mesons into two photons have been analyzed theoretically  in various models and experimentally  in different collaborations. The decay of the standard model (SM) Higgs boson $`H^0`$ into two photons is another candidate for the two photon system as an output and it can provide a valuable information to understand the electroweak symmetry breaking and to predict the model parameters . In the SM, the $`H^0\gamma \gamma `$ decay exists at the loop level with the help of all massive particles, which have non-zero electromagnetic charge, running in the loop. The most important contributions come from the top quark and W boson. The branching ratio (BR) has been predicted at the order of the magnitude of $`10^3`$ in the framework of SM. The addition of the dimension six and dimension eight fermionic and bosonic operators enhances the SM width of this decay up to one order of magnitude .

In the two Higgs doublet model (2HDM) which is the minimal extension of the SM, there appears the well known new Higgs scalars $`h^0`$ and $`A^0`$ in addition to the SM one, $`H^0`$. The two photon decays of these new particles can provide a considerable information about the model parameters since they occur at least at the one loop level where all the charged fermions run in the loop. On the other hand it is possible to study the CP violating effects since the output is the two photon system and it has two different states, CP odd and CP even.

In the present work we analyze the two photon decays of new Higgs bosons $`h^0`$ and $`A^0`$, $`h^0(A^0)\gamma \gamma `$, in the framework of the model III version of the 2HDM. Since the charged fermions, which are running in the loop, are the source of the processes under consideration there exist various Yukawa couplings which are diagonal in flavor indices and complex in general. In our analysis we take into account only the Yukawa couplings corresponding to the heaviest fermions in each group of particles, up-quarks, down-quarks, charged leptons. We also use the restrictions obtained in the literature for the existing Yukawa couplings and we estimate the decay width of the processes at the order of magnitude of $`10^2GeV`$ for the intermediate values of the Yukawa couplings. On the other hand, these couplings are complex in general and a new parameter carrying the imaginary part of the coupling arises. However, the analysis of the CP even $`A_{h^0(A^0)}^+`$ and CP odd $`A_{h^0(A^0)}^{}`$ amplitudes, which correspond to the parallel and perpendicular spin polarization of photon system, shows that the strength of the complexity of the couplings can be predicted by measuring the relative magnitudes of CP odd and CP even amplitudes. In addition to this the measurement these amplitudes and the $`BR`$ of the processes provide wide information about the masses $`m_{h^0}`$ and $`m_{A^0}`$.

The paper is organized as follows: In Section II, we present the theoretical expression for the decay widths of $`h^0\gamma \gamma `$ and $`A^0\gamma \gamma `$ decays, in the framework of the model III. Section 3 is devoted to discussion and our conclusions.

## 2 The radiative $`h^0(A^0)\gamma \gamma `$ decays in the two Higgs doublet model

The Higgs bosons $`h^0`$ and $`A^0`$ can decay into two photon system with the help of the fermion loop, which includes all possible quarks and down leptons. The interaction between the scalar bosons and the fermions are derived by the Yukawa lagrangian and the most general one in the 2HDM reads

$`_Y`$ $`=`$ $`\eta _{ij}^U\overline{Q}_{iL}\stackrel{~}{\varphi _1}U_{jR}+\eta _{ij}^D\overline{Q}_{iL}\varphi _1D_{jR}+\xi _{ij}^U\overline{Q}_{iL}\stackrel{~}{\varphi _2}U_{jR}+\xi _{ij}^D\overline{Q}_{iL}\varphi _2D_{jR}`$ (1)
$`+`$ $`\eta _{ij}^E\overline{l}_{iL}\varphi _1E_{jR}+\xi _{ij}^E\overline{l}_{iL}\varphi _2E_{jR}+h.c.,`$
where $`L`$ and $`R`$ denote chiral projections $`L(R)=1/2(1\gamma _5)`$, $`\varphi _i`$ for $`i=1,2`$, are the two scalar doublets, $`\overline{Q}_{iL}`$ are left handed quark doublets, $`U_{jR}(D_{jR})`$ are right handed up (down) quark singlets, $`l_{iL}`$ ($`E_{jR}`$) are lepton doublets (singlets), with family indices $`i,j`$. Here the Yukawa matrices $`\xi _{ij}^{U,D}`$ and $`\xi _{ij}^E`$ have in general complex entries. The possible mixing between neutral Higgs bosons $`H^0`$ and $`h^0`$ is switched off by choosing the parametrization for $`\varphi _1`$ and $`\varphi _2`$ as

$`\varphi _1={\displaystyle \frac{1}{\sqrt{2}}}\left[\left(\begin{array}{cc}0& \\ v+H^0& \end{array}\right)+\left(\begin{array}{cc}\sqrt{2}\chi ^+& \\ i\chi ^0& \end{array}\right)\right];\varphi _2={\displaystyle \frac{1}{\sqrt{2}}}\left(\begin{array}{cc}\sqrt{2}H^+& \\ H_1+iH_2& \end{array}\right),`$ (8)
where only $`\varphi _1`$ has a vacuum expectation value;

$`<\varphi _1>={\displaystyle \frac{1}{\sqrt{2}}}\left(\begin{array}{cc}0& \\ v& \end{array}\right);<\varphi _2>=0,`$ (11)
and by considering the gauge and $`CP`$ invariant Higgs potential which spontaneously breaks $`SU(2)\times U(1)`$ down to $`U(1)`$ as

$`V(\varphi _1,\varphi _2)`$ $`=`$ $`c_1(\varphi _1^+\varphi _1v^2/2)^2+c_2(\varphi _2^+\varphi _2)^2`$ (12)
$`+`$ $`c_3[(\varphi _1^+\varphi _1v^2/2)+\varphi _2^+\varphi _2]^2+c_4[(\varphi _1^+\varphi _1)(\varphi _2^+\varphi _2)(\varphi _1^+\varphi _2)(\varphi _2^+\varphi _1)]`$
$`+`$ $`c_5[Re(\varphi _1^+\varphi _2)]^2+c_6[Im(\varphi _1^+\varphi _2)]^2+c_7,`$
with constants $`c_i,i=1,\mathrm{},7`$. This is the case where the SM (new) particles are collected in the first (second) doublet. Notice that, in the following we will replace $`\xi _{ij}^{U,D,E}`$ by $`\xi _{N,ij}^{U,D,E}`$ to emphasize that the couplings are related to the neutral interactions.

Now, we consider the radiative decays of the Higgs bosons $`h^0`$ and $`A^0`$, $`h^0(A^0)\gamma \gamma `$. These processes occur with the internal fermion flow Fig. (1) and the decay amplitudes strongly depend on the Yukawa couplings of the Higgs-fermion interactions. The decay amplitude for these radiative decays can be written in terms of two Lorentz structures:

$`M(S\gamma \gamma )=A_S^+F_{\mu \nu }F^{\mu \nu }+iA_S^{}F_{\mu \nu }\stackrel{~}{F}^{\mu \nu },`$ (13)
with $`\stackrel{~}{F}^{\mu \nu }=\frac{1}{2}ϵ_{\mu \nu \alpha \beta }F^{\alpha \beta }`$ and $`A_S^+`$ ($`A_S^{}`$) are CP even (odd) amplitude:

$`A_S^+`$ $`=`$ $`{\displaystyle \frac{c_1^S}{4\pi m_S^2}}{\displaystyle \underset{i}{}}m_iN_iQ_i^2g_{ii}^{S+}\{2+(1+4x_i^S)(Li_2[f_+]+Li_2[f_{}])\},`$
$`A_S^{}`$ $`=`$ $`{\displaystyle \frac{c_1^S}{4\pi m_S^2}}{\displaystyle \underset{i}{}}m_iN_iQ_i^2g_{ii}^S(Li_2[f_+]+Li_2[f_{}]),`$ (14)
where $`S`$ is the scalar $`h^0`$ or $`A^0`$, $`c_1^{h^0(A^0)}=(I)\sqrt{2}\alpha _e`$, $`\alpha _e=\frac{1}{137}`$, $`g_{ii}^{h^0\pm }=(\xi _{N,ii}^F\pm \xi _{N,ii}^F)`$, $`g_{ii}^{A^0\pm }=g_{ii}^{h^0}`$, $`F=U(D;E)`$ for $`i=u,c,t(d,s,b;e,\mu ,\tau )`$ , $`N_i`$ is the color factor, $`N_i=3(1)`$ for $`i=u,c,t,d,s,b(e,\mu ,\tau )`$, and $`Q_i`$ is the charge $`Q_i=\frac{2}{3}(\frac{1}{3};1)`$ for $`i=u,c,t(d,s,b;e,\mu ,\tau )`$. Here the functions $`Li_2[z]`$ are the PolyLogarithmic function $`Li_2[z]=_z^0\frac{Log[1t]}{t}𝑑t`$ and $`f_\pm =\frac{2}{1\pm \sqrt{14x_i}}`$ with $`x_i=\frac{m_i^2}{m_S^2}`$. Furthermore, we parametrize the complex Yukawa coupling $`\xi _{N,ii}^{U(D,E)}`$ as

$$\xi _{N,ii}^{U,D,E}=|\xi _{N,ii}^{U,D,E}|e^{i\theta _{ii}}.$$
(15)
Finally the decay width of the processes can be obtained by using the expression

$`\mathrm{\Gamma }_S={\displaystyle \frac{m_S^3}{16\pi }}(|A_S^+|^2+|A_S^{}|^2)`$ (16)
In the decays $`h^0\gamma \gamma `$ and $`A^0\gamma \gamma `$, the outgoing photons are in one of the possible states given by $`F^{\mu \nu }F_{\mu \nu }`$ and $`F^{\mu \nu }\stackrel{~}{F}_{\mu \nu }`$ which are even and odd in CP, respectively. These states corresponds to the parallel ($`ϵ_1.ϵ_2`$) and perpendicular ($`ϵ_1\times ϵ_2`$) spin polarizations of photons. Eq. (14) shows that the decay amplitude is CP even (odd) for the $`h^0\gamma \gamma `$ ($`A^0\gamma \gamma `$) decay in the case of real couplings, when the one loop calculations are taken into account. The strength of the complex part in the Yukawa couplings can be estimated by detecting the CP odd (even) amplitude relative to CP even (odd) one, in other words by studying the ratio $`\frac{|A_S^+|^2}{|A_S^{}|^2}`$, for the $`h^0\gamma \gamma `$ ($`A^0\gamma \gamma `$) decay. Therefore the precise determination of the photon polarization in the experiments give considerable information about the free parameters of the theory under consideration.

## 3 Discussion

This section is devoted to the analysis of the decay width and the amplitudes for the CP even and CP odd photon states of the processes $`h^0\gamma \gamma `$ and $`A^0\gamma \gamma `$. These processes occur at least at the loop order with the help of the internal charged fermions Fig. 1. They are driven by the diagonal Yukawa couplings $`\xi _{N,ii}^U`$, ($`i`$=up quarks), $`\xi _{N,ii}^D`$, ($`i`$=down quarks) and $`\xi _{N,ii}^E`$, ($`i`$=down leptons), which are the free parameters of the model used, and they should be restricted by using the appropriate experimental measurements. At first, we assume that the Yukawa couplings $`\xi _{N,ii}^U,i=u,c`$, $`\xi _{N,ii}^D,i=d,s`$ and $`\xi _{N,ii}^E,i=e,\mu `$, are small compared to $`\xi _{N,tt}^U`$, $`\xi _{N,bb}^D`$ and $`\xi _{N,\tau \tau }^E`$ since the strength of them are related to the masses of fermions denoted by the indices, similar to the Cheng-Sher scenerio . Second, we use the predictions for the couplings $`\xi _{N,bb}^D`$, $`\xi _{N,tt}^U`$, obtained by using the constraint region due to the restriction of the Wilson coefficient $`C_7^{eff}`$, which is the effective coefficient of the operator $`O_7=\frac{e}{16\pi ^2}\overline{s}_\alpha \sigma _{\mu \nu }(m_bR+m_sL)b_\alpha ^{\mu \nu }`$ (see  and references therein), in the region $`0.257|C_7^{eff}|0.439`$. Here upper and lower limits were calculated using the CLEO measurement

$`BR(BX_s\gamma )=(3.15\pm 0.35\pm 0.32)\mathrm{\hspace{0.17em}10}^4,`$ (17)
and all possible uncertainties in the calculation of $`C_7^{eff}`$ . The above restriction ensures to get upper and lower limits for $`\xi _{N,bb}^D`$, $`\xi _{N,tt}^U`$ (see  for details). At this stage we will make the parametrization

$$\xi _{N,ij}^{U,D,E}=\sqrt{\frac{4G_F}{\sqrt{2}}}\overline{\xi }_{N,ij}^{U,D,E},$$
(18)
where $`G_F`$ is the fermi constant and use it in the following.

Our numerical calculations are based on the choice intermediate values of the couplings for $`C_7^{eff}>0`$, $`\overline{\xi }_{N,bb}^D=30m_b`$, $`\overline{\xi }_{N,tt}^U0.0025GeV`$, respecting the constraints mentioned above. The coupling $`\overline{\xi }_{N,\tau \tau }^E`$ is the other one which should be restricted and we use the prediction obtained in the analysis of the decay $`\tau \mu \overline{\nu _i}\nu _i`$, $`i=e,\mu ,\tau `$, in . In this work the $`BR`$ is estimated at the order of the magnitude of $`10^610^4`$ for the range of the couplings, $`\overline{\xi }_{N,\tau \tau }^E30100GeV`$ and $`\overline{\xi }_{N,\tau \mu }^E1025GeV`$. In the present work, for the coupling $`\overline{\xi }_{N,\tau \tau }^E`$, we take the optimum value $`\overline{\xi }_{N,\tau \tau }^E30GeV`$. In addition to this we consider the general case that the couplings are complex (see eq. 15)

In Fig. 2 (3), we present $`m_{h^0}(m_{A^0})`$ dependence of the decay width $`\mathrm{\Gamma }`$ of the $`h^0(A^0)\gamma \gamma `$ decay, for the intermediate value of the complexity parameter $`sin\theta _{ii}=0.5`$ ($`i=t`$-quark, $`b`$-quark, $`\tau `$-lepton) of three Yukawa couplings. The Figure 2 (3) shows that the $`\mathrm{\Gamma }`$ is at the order of the magnitude of $`10^310^2GeV`$ for the decay $`h^0\gamma \gamma `$ ($`A^0\gamma \gamma `$) and it is not so much sensitive to the mass of $`h^0`$ in the given mass interval (decreases with the increasing values of the mass of $`A^0`$). With the possible measurement of this radiative decay a considerable information can be obtained for the mass of the scalar bosons, especially for $`m_{A^0}`$.

Fig. 4 (5) is devoted to the $`sin\theta `$ dependence of the $`\mathrm{\Gamma }(h^0(A^0)\gamma \gamma )`$ for $`m_{h^0}=85GeV`$ ($`m_{A^0}=90GeV`$). The solid line represents the case where all the Yukawa couplings are complex and $`sin\theta _{bb}=sin\theta _{tt}=sin\theta _{\tau \tau }`$, the dashed (small dashed; dotted) line represents the case where $`sin\theta _{tt}=sin\theta _{\tau \tau }=0`$ ($`sin\theta _{bb}=sin\theta _{\tau \tau }=0`$; $`sin\theta _{bb}=sin\theta _{tt}=0`$). We observe that the increasing values of $`sin\theta _{bb}`$ ($`sin\theta _{tt}`$, $`sin\theta _{\tau \tau }`$) causes to enhance (weakly supress, supress) the $`\mathrm{\Gamma }`$ in the $`h^0\gamma \gamma `$ decay. For the $`A^0\gamma \gamma `$ decay, the increasing values of $`sin\theta _{bb}`$ ($`sin\theta _{tt}`$, $`sin\theta _{\tau \tau }`$) causes to supress (enhance, supress) the $`\mathrm{\Gamma }`$. In the case that all the Yukawa couplings are complex and have the same value, there is an enhancement (suppression) in the $`\mathrm{\Gamma }`$ for the increasing values of $`sin\theta `$, for the $`h^0\gamma \gamma `$ ($`A^0\gamma \gamma `$) decay. These results may be effective in the determination of the complexity of the Yukawa couplings.

The ratio of CP even and CP odd amplitudes $`R_S=\frac{|A_S^+|^2}{|A_S^{}|^2}`$ is interesting since it depends strongly on the complex nature of the Yukawa couplings. In Fig. 6 (7), we present $`m_{h^0}`$ ($`m_{A^0}`$) dependence of the ratio $`R_{h^0}`$ ($`R_{A^0}`$) for the decay $`h^0(A^0)\gamma \gamma `$, for the intermediate value of the parameter $`sin\theta _{ii}=0.5`$. Figure 6 (7) shows that the $`R_{h^0}`$ ($`R_{A^0}`$) is greater than one (less than one) for the given range of mass, for the decay $`h^0\gamma \gamma `$ ($`A^0\gamma \gamma `$) and increases with the increasing values of the mass $`h^0`$ ($`A^0`$).

Fig.8 (9) represents the $`sin\theta `$ dependence of the ratio $`R_{h^0}`$ ($`R_{A^0}`$) for $`m_{h^0}=85GeV`$ ($`m_{A^0}=90GeV`$). The solid line represents the case where all the Yukawa couplings are complex and $`sin\theta _{bb}=sin\theta _{tt}=sin\theta _{\tau \tau }`$, the dashed (small dashed; dotted) line represents the case where $`sin\theta _{tt}=sin\theta _{\tau \tau }=0`$ ($`sin\theta _{bb}=sin\theta _{\tau \tau }=0`$; $`sin\theta _{bb}=sin\theta _{tt}=0`$). We observe that the ratio $`R_{h^0}`$ ($`R_{A^0}`$) is strongly sensitive to the complexity of the Yukawa couplings. For the real couplings $`|A_S^{}|^2`$ vanishes and the ratio $`R_{h^0}`$ tends to infinity. In such a case the only possible state of two photon system is the parallel polarization. The measurement of the perpendicular polarization for the photon system is a sign for the complexity of the Yukawa couplings. The ratio $`R_{h^0}`$ decreases with the increasing values of $`sin\theta `$ for all Yukawa couplings. This ratio is larger for the case that only the Yukawa coupling $`\xi _{N,tt}^U`$ (or $`\overline{\xi }_{N,\tau \tau }^E`$) is complex. This observation can make it possible to determine the relative strengths of the complexity of different Yukawa couplings. On the other hand, the ratio $`R_{A^0}`$ vanishes for the real coupling and only the perpendicular polarization state exists for the two photon system. Therefore, for the decay $`A^0\gamma \gamma `$, the measurement of the parallel polarization for the photon system is a sign for the complexity of the Yukawa couplings. The ratio $`R_{A^0}`$ increases with the increasing values of $`sin\theta `$ for all Yukawa couplings and it is smaller for the case that only the Yukawa coupling $`\xi _{N,tt}^U`$ (or $`\overline{\xi }_{N,\tau \tau }^E`$) is complex. Similar to the $`h^0\gamma \gamma `$ decay, this is informative in the determination of the relative strengths of the complexity of different Yukawa couplings.

Now we would like to summarize our results:

* We predict the decay width $`\mathrm{\Gamma }`$ of the decay $`h^0(A^0)\gamma \gamma `$ at the order of the magnitude of $`10^310^2GeV`$ and it is not so much sensitive to the mass of $`h^0`$ in the given range (decreases with the increasing values of the mass of $`A^0`$) Therefore its measurement can provide valuable information about the $`h^0(A^0)`$ masses and the Yukawa couplings.
* We study the ratio $`R_{h^0(A^0)}`$ which is constructed using the CP even $`A_{h^0(A^0)}^+`$ and CP odd $`A_{h^0(A^0)}^{}`$ amplitudes. For the case of real couplings $`R_{h^0}`$ ($`R_{A^0}`$) tends to infinity (zero) and the only possible state of two photon system is the parallel (perpendicular) polarization. The amount of complexity of the Yukawa couplings can explain the relative magnitude of the perpendicular (parallel) polarization. Therefore the measurement of the photon polarization of the decay $`h^0(A^0)\gamma \gamma `$ gives a considerable information about the complex nature of the Yukawa couplings and strength of their relative complexity.

Therefore, the experimental analysis of these radiative decays would ensure strong information about the scalar boson masses and the Yukawa couplings existing in the new physics beyond the SM.

## 4 Acknowledgement

This work has been supported by the Turkish Academy of Sciences in the framework of the Young Scientist Award Program. (EOI-TUBA-GEBIP/2001-1-8)


text:
# Cohomological divisibility and point count divisibility

## 1. Introduction

Nearly seventy years have passed since the theorem of Chevalley-Warning , : over a finite field $`k=𝔽_q`$ of characteristic $`p`$, if $`fk[X_1,\mathrm{},X_n]`$ is a polynomial in $`n1`$ variables of degree $`d<n`$, then the number of solutions in $`𝔸^n(k)`$ of $`f=0`$ is divisible by $`p`$. For $`d1`$, Ax  improved this to divisibility by $`q^\mu `$, where $`\mu `$ is the least non-negative integer which is $`\frac{nd}{d}`$. Thus

(1.1)
$$\mu =\mathrm{max}(0,\text{Ceiling}(\frac{nd}{d})).$$

In this form, the result remains true, if trivially so, without the assumption that $`n>d`$. Ax also noted that, by an elementary inclusion-exclusion argument, his result implied that if one had $`r1`$ polynomials $`f_ik[X_1,\mathrm{},X_n]`$, with $`f_i`$ of degree $`d_i1`$, then the number of solutions in $`𝔸^n(k)`$ of the system of equations $`f_1=\mathrm{}=f_r=0`$ is divisible by $`q^\lambda `$, where $`\lambda `$ is the least non-negative integer which is $`\frac{n_id_i}{{\scriptscriptstyle d_i}}`$, i.e.,

(1.2)
$$\lambda =\mathrm{max}(0,\text{Ceiling}(\frac{n_id_i}{d_i})).$$

Katz  improved this to divisibility by $`q^\mu `$, where $`\mu `$ is the least non-negative integer which is $`\frac{n_id_i}{\mathrm{max}_id_i}`$, i.e.,

(1.3)
$$\mu :=\mu (n;d_1,\mathrm{},d_r):=\mathrm{max}(0,\text{Ceiling}(\frac{n_id_i}{\mathrm{max}_id_i})).$$

An elementary proof of this result was given by Wan . Adolphson and Sperber  gave a divisibility estimate for additive character sums which, they showed, included this result. Wan  gave a divisibility estimate for multiplicative character sums which, he showed, included this result as well.

del Angel noted in  that if one reverses the inclusion-exclusion argument which Ax used to pass from a result for one polynomial to a result for several polynomials, then the estimate (1.3) for several polynomial gives the following result for one polynomial which is reducible: if one has $`r1`$ polynomials $`f_ik[X_1,\mathrm{},X_n]`$, with $`f_i`$ of degree $`d_i1`$, then the number of solutions in $`𝔸^n(k)`$ of the single equation $`f_1\mathrm{}f_r=0`$ is divisible by $`q^\mu `$, for $`\mu `$ = $`q^{\mu (n;d_1,\mathrm{},d_r)}`$ as above. If one replaces $`f_1\mathrm{}f_r`$ by any product $`f_1^{a_1}\mathrm{}f_r^{a_r}`$ with all $`a_i1`$, one has the same solutions in $`𝔸^n(k)`$, so the same divisibility. Notice that if one omits some of the factors in $`f_1^{a_1}\mathrm{}f_r^{a_r}`$, i.e., if one allows some $`a_i`$ to be zero, then the number of solutions of $`f_1^{a_1}\mathrm{}f_r^{a_r}=0`$ in $`𝔸^n(k)`$ remains divisible by the same $`q^\mu `$ \[simply because if one removes some of the $`f_i`$’s from consideration, the numerator $`n_id_i`$ of $`\mu `$ increases, and its denominator $`\mathrm{max}_id_i`$ decreases\].

With this result in hand, del Angel did inclusion-exclusion a la Ax and got the following result. Suppose one has a list of $`r1`$ polynomials $`f_ik[X_1,\mathrm{},X_n]`$, with $`f_i`$ of degree $`d_i1`$. Take any $`R1`$, and any list of $`R1`$ polynomials $`g_jk[X_1,\mathrm{},X_n]`$, such that $`g_j`$ is a product $`g_j=_if_i^{a_{i,j}}`$, with all $`a_{i,j}0`$. Then the equations $`g_1=\mathrm{}=g_R=0`$ define a closed subscheme $`X𝔸^n`$, whose number of $`k`$-points is divisible by $`q^{\mu (n;d_1,\mathrm{},d_r)}`$.

There is an obvious projective version of these affine results. If the $`f_i`$ are all homogeneous, then all the $`g_j`$ are homogeneous, and the same equations $`g_1=\mathrm{}=g_R=0`$ also define a closed subscheme $`X^{\mathrm{proj}}^{n1}`$ whose affine cone is the $`X`$ above. The number of $`𝔽_q`$-rational points of $`U:=^{n1}X^{\mathrm{proj}}`$ is easily seen to be divisible by $`q^{\mu (n;d_1,\mathrm{},d_r)}`$.

The common feature of these results is that one has a separated $`k`$-scheme $`X/k`$ of finite type, and an integer $`\kappa 1`$, such that for each finite extension $`𝔽_{q^\nu }`$ of $`𝔽_q`$, $`|X(𝔽_{q^\nu })|`$ is divisible by $`q^{\nu \kappa }`$. Some years ago, Deligne posed the problem of showing that in any such situation $`(X/k,\kappa )`$, the divisibilities of numbers of points result from corresponding divisibilities of all the eigenvalues of Frobenius on all the compact $`\mathrm{}`$-adic cohomology groups, $`\mathrm{}`$ any prime other than $`p`$, of the variety $`X/k`$ in question. As explained by Ax , such divisibilities of numbers of points imply (and indeed are equivalent to) the corresponding divisibility of all reciprocal zeros and poles of the zeta function. But for any given $`\mathrm{}p`$, the $`\mathrm{}`$-adic cohomological expression of the zeta function may well have cancellation: the problem is that there might be pairs of Frobenius eigenvalues on different $`\mathrm{}`$-adic groups which cancel, but which both have insufficient divisibility.

In the case of a general $`(X/k,\kappa )`$, Deligne’s problem remains open. In this note, we show it has an affirmative solution when $`(X/k,\kappa )`$ arises as an instance of the divisibility theorems explained above.

Acknowledgements: We warmly thank Spencer Bloch for his interest in and for his incisive comments on the work leading up to this article.

## 2. Statement of results: the affine case

We fix a finite field $`k=𝔽_q`$ of characteristic $`p`$ and a prime number $`\mathrm{}p`$. For a separated scheme $`X/k`$ of finite type, we abbreviate

(2.1)
$$H^i(X):=H^i(X\times _k\overline{k},_{\mathrm{}}),H_c^i(X):=H^i(X\times _k\overline{k},_{\mathrm{}}).$$

We denote by $`F=F_k`$ the geometric Frobenius endomorphism of these cohomology groups. By the Grothendieck-Lefschetz trace formula

(2.2)
$$|X(k)|=\underset{i}{}(1)^i\mathrm{Trace}(F;H_c^i(X)),\text{and}$$

$$|X(k_\nu )|=\underset{i}{}(1)^i\mathrm{Trace}(F^\nu ;H_c^i(X))$$

for $`k_\nu =𝔽_{q^\nu }`$. Recall , Corollaire 5.5.3, that all the eigenvalues of $`F`$ acting on $`H_c^i(X)`$ are algebraic integers, i.e. elements of $`\overline{_{\mathrm{}}}`$ which are integral over $``$.

###### Theorem 2.1.

Suppose one has a list of $`r1`$ polynomials $`f_ik[X_1,\mathrm{},X_n]`$, with $`f_i`$ of degree $`d_i1`$. Take any integer $`R1`$, and any list of $`R1`$ polynomials $`g_jk[X_1,\mathrm{},X_n]`$, such that each $`g_j`$ is a product $`g_j=_if_i^{a_{i,j}}`$, with all $`a_{i,j}0`$. Denote by $`X/k`$ the closed subscheme of $`𝔸^n(k)`$ defined by $`g_1=\mathrm{}=g_R=0`$. Then every eigenvalue of $`F`$ on every compact cohomology group $`H_c^i(X)`$ is divisible as an algebraic integer by $`q^{\mu (n;d_1,\mathrm{},d_r)}`$.

From the excision sequence

(2.3)
$$\mathrm{}H_c^i(𝔸^nX)H_c^i(𝔸^n)H_c^i(X)\mathrm{},$$

and the well-known cohomological structure of $`𝔸^n`$, namely

(2.4)
$$H_c^i(𝔸^n)=0\text{for}i2n,H_c^{2n}(𝔸^n)=_{\mathrm{}}(n),$$

where $`_{\mathrm{}}(n)`$ is the 1 dimensional vector space over $`_{\mathrm{}}`$ on which $`F`$ acts as $`q^n`$, one sees that Theorem 2.1 is equivalent to

###### Theorem 2.2.

Under the hypotheses of Theorem 2.1, every eigenvalue of $`F`$ on every compact cohomology group $`H_c^i(𝔸^nX)`$ is divisible as an algebraic integer by $`q^{\mu (n;d_1,\mathrm{},d_r)}`$.

Theorem 2.2 iself results from the following slightly sharper result, suggested by , Corollaire 5.3.3, (ii), and by , . For each integer $`j0`$, let us define

(2.5)
$$\mu _j(n;d_1,\mathrm{},d_r):=j+\mu (nj;d_1,\mathrm{},d_r).$$

Thus

(2.6)
$$\mu _0(n;d_1,\mathrm{},d_r)=\mu (n;d_1,\mathrm{},d_r),$$

and we have the obvious inequalities

(2.7)
$$\mu _{j+1}(n;d_1,\mathrm{},d_r)\mu _j(n;d_1,\mathrm{},d_r).$$

###### Theorem 2.3.

Under the hypotheses of Theorem 2.1, we have the following results.

* Every eigenvalue of $`F`$ on every group $`H_c^i(𝔸^nX)`$ is divisible as an algebraic integer by $`q^{\mu (n;d_1,\mathrm{},d_r)}`$.
* For each $`j1`$, every eigenvalue of $`F`$ on $`H_c^{n+j}(𝔸^nX)`$ is divisible as an algebraic integer by $`q^{\mu _j(n;d_1,\mathrm{},d_r)}`$.

## 3. The proof of Theorem 2.3

We first reduce to the case $`R=1`$. For $`j`$, with $`1jR`$, we denote by $`G_j𝔸^n`$ the hypersurface in $`𝔸^n`$ defined by $`g_j=0`$. Then

(3.1)
$$X=_{j=1}^RG_j,\text{hence}$$

$$𝔸^nX=_{j=1}^RU_j,\text{with}U_j=𝔸^nG_j.$$

One considers the spectral sequence ,  6.2.10.3,

(3.2)
$$E_1^{a,b}=_{1j_1<j_2<\mathrm{}<j_{a+1}R}H_c^b(_{i=1}^{a+1}U_{j_i})H_c^{ba}(𝔸^nX),$$

which is the cohomological incarnation of inclusion-exclusion. Thus every eigenvalue of F on a given group $`H_c^b(𝔸^nX)`$ is an eigenvalue of $`F`$ on some group $`H_c^b(_{i=1}^{a+1}U_{j_i})`$ for some $`a0`$. So it suffices to show the following two statements:

* For every $`b0`$ and every $`a0`$, every eigenvalue of $`F`$ on every group $`H_c^b(_{i=1}^{a+1}U_{j_i})`$ is divisible as an algebraic integer by $`q^{\mu (n;d_1,\mathrm{},d_r)}`$
* For every $`j1`$ and every $`a0`$, every eigenvalue of $`F`$ on every group $`H_c^{n+j}(_{i=1}^{a+1}U_{j_i})`$ is divisible as an algebraic integer by $`q^{\mu _j(n;d_1,\mathrm{},d_r)}`$.

But an $`(a+1)`$-fold intersection $`_{i=1}^{a+1}U_{j_i}`$ is the complement of the hypersurface of equation $`_{i=1}^{a+1}g_{j_i}=0`$, which falls under the case $`R=1`$ of Theorem 2.3. So it suffices to prove Theorem 2.3 universally for $`R=1`$. We use the weak Lefschetz theorem and induction on $`n`$, the number of variables. For $`n=1`$, we have $`\mu (1;d_1,\mathrm{},d_r)=0`$ (because $`_id_i1`$), so assertion 1) of the theorem is an (easy) instance of Deligne’s integrality theorem , Corollaire 5.5.3. Assertion 2) of the theorem holds for $`n=1`$ because the only possibly nonzero group $`H_c^{1+j}(𝔸^1X)`$ with $`j1`$ is $`H_c^2(𝔸^1X)`$, which is $`_{\mathrm{}}(1)`$. So suppose the theorem universally true for $`n1`$, in the case $`R=1`$.

Whatever the nonempty hypersurface $`G𝔸^n`$, $`𝔸^nG`$ is smooth and geometrically connected of dimension $`n`$, so Poincaré duality tells us that the cup product

(3.3)
$$H_c^i(𝔸^nG)\times H^{2ni}(𝔸^nG)H_c^{2n}(𝔸^nG)_{\mathrm{}}(n)$$

is a perfect pairing of $`F`$-modules. Since $`𝔸^nG`$ is affine of dimension $`n`$, the affine Lefschetz theorem tells us that

(3.4)
$$H^i(𝔸^nG)=0\text{for}i>n.$$

So by duality we have

(3.5)
$$H_c^i(𝔸^nG)=0\text{for}i<n.$$

We now take $`G`$ defined by $`_if_i^{a_i}=0`$.

We first prove assertion 1) of the theorem. As noted by Wan and exploited in , Introduction and in , given the divisibility

(3.6)
$$|(𝔸^nG)(k_\nu )|=\underset{in}{}(1)^i\mathrm{Trace}(F^\nu ;H_c^i(𝔸^nG))$$

by $`q^{\nu \mu }`$ for every $`\nu 1`$, it suffices to prove that for all but a single value of $`i`$ we have the asserted divisibility of Frobenius eigenvalues on $`H_c^i(𝔸^nG)`$. We choose $`i=n`$. So we need only show the divisibility by $`q^{\mu (n;d_1,\mathrm{},d_r)}`$ of Frobenius eigenvalues on $`H_c^i(𝔸^nG)`$ for $`i>n`$.

This divisibility is invariant under finite extension of the ground field. For a sufficiently general affine linear hyperplane $`A𝔸^n`$ defined by an equation of the form

(3.7)
$$A:\underset{i=1}{\overset{n}{}}\alpha _iX_i+\beta _i=0$$

with coefficients in a finite extension of $`k`$, (which, by extending $`k`$, we may assume to be $`k`$ itself), the restriction maps on ordinary cohomology

(3.8)
$$H^i(𝔸^nG)H^i(AAG)$$

are injective for $`i<n`$. This is an instance of the weak Lefschetz theorem (see , 3.4.1, applied to $`V=𝔸^nG`$, with $`\pi `$ the inclusion and $`f`$ the function $`0`$.) The Poincaré dual of this injection is a surjection

(3.9)
$$H_c^{i2}(AAG)(1)H_c^i(𝔸^nG)$$

for $`i>n`$. For $`A`$ general, each $`f_i|_A`$ will be a polynomial of the same degree $`d_i`$. So by induction on $`n`$, we know that every eigenvalue of $`F`$ on every group $`H_c^{i2}(AAG)`$ is divisible by $`q^{\mu (n1;d_1,\mathrm{},d_r)}`$. So every eigenvalue of $`F`$ on every group $`H_c^{i2}(AAG)(1)`$ is divisible by $`q^{\mu (n1;d_1,\mathrm{},d_r)+1}`$. But we have the inequality

(3.10)
$$\mu (n1;d_1,\mathrm{},d_r)+1=\mu _1(n;d_1,\mathrm{},d_r)\mu (n;d_1,\mathrm{},d_r).$$

This concludes the proof of assertion 1) of Theorem 2.3.

This same induction proves assertion 2) of Theorem 2.3. Indeed, for any integer $`j1`$, the induction shows that for $`i=n+j`$, every eigenvalue of $`F`$ on $`H_c^{n+j}(𝔸^nG)`$ is also an eigenvalue of $`F`$ on

(3.11)
$$H^{(n1)+(j1)}(AGA)(1).$$

By induction, every eigenvalue of $`F`$ on (3.11) is divisible by $`q^{1+\mu _{j1}(n;d_1,\mathrm{},d_r)}`$. And by definition we have

(3.12)
$$1+\mu _{j1}(n;d_1,\mathrm{},d_r)=\mu _j(n;d_1,\mathrm{},d_r).$$

This concludes the proof of Theorem 2.3.

## 4. The projective case

Here is the projective version. Suppose now that the $`f_i`$ are all homogeneous forms. Then each $`g_j`$ is again homogeneous, and the closed $`X/k`$ in $`𝔸^n`$ defined by $`g_1=\mathrm{}=g_R=0`$ is the affine cone over a closed $`X^{\mathrm{proj}}`$ in $`^{n1}`$, defined by the same equations.

###### Theorem 4.1.

Under the hypotheses of Theorem 2.1, suppose in addition all the $`f_i`$ are homogeneous. Then we have the following results.

* Every eigenvalue of $`F`$ on every group $`H_c^i(^{n1}X^{\mathrm{proj}})`$ is divisible as an algebraic integer by $`q^{\mu (n;d_1,\mathrm{},d_r)}`$.
* For each $`j1`$, every eigenvalue of $`F`$ on $`H_c^{n+j}(^{n1}X^{\mathrm{proj}})`$ is divisible as an algebraic integer by $`q^{\mu _j(n;d_1,\mathrm{},d_r)}`$.

###### Proof.

To prove the theorem, repeat the proof of Theorem 2.3 essentially verbatim. First reduce to the case $`R=1`$ by the same spectral sequence argument. Then do induction on $`n`$. Again, the case $`n=1`$ holds trivially. The open variety $`^{n1}X^{\mathrm{proj}}`$ is affine, smooth, and geometrically connected of dimension $`n1`$. The appropriate reference for the weak Lefschetz theorem in this context is  Appendix (d’après Deligne), A5, applied with its $`X:=^{n1}X^{\mathrm{proj}}`$, and $`f`$ the inclusion into $`^{n1}`$. ∎

## 5. Concluding remarks

### 5.1.  Concordance

In other work on this subject, e.g. in , , , one sees a different but equivalent expression of the divisibility, in terms of the floor, or integral part \[x\] of a real number x. One renumbers so that $`d_1d_2\mathrm{}d_r`$, and then one defines

(5.1)
$$\kappa (n;d_1,\mathrm{},d_r):=\mathrm{max}(0,[\frac{nd_2\mathrm{}d_r}{d_1}]).$$

This $`\kappa `$ is related to $`\mu `$ by the identity

(5.2)
$$\kappa (n;d_1,\mathrm{},d_r)=\mu (n+1;d_1,\mathrm{},d_r),$$

whose verification we leave to the reader.

### 5.2.  Axiomatization

The arguments we give here could easily be axiomatized, and then applied to any pair of cohomology theories $`(H^{},H_c^{})`$, defined on smooth separated $`\overline{k}`$-schemes of finite type, with values in finite-dimensional vector spaces over a field $`K`$ of characteristic zero, with the following properties:

* zeta function formula for $`H_c^{}`$,
* Poincaré duality between $`H^{}`$ and $`H_c^{}`$,
* affine Lefschetz and weak Lefschetz theorems for $`H^{}`$,
* spectral sequence of an open covering for $`H_c^{}`$.

An initial run through our arguments with $`\mu =0`$ shows that all Frobenius eigenvalues on $`H_c^{}`$ in any such theory are algebraic integers. We expect that Berthelot’s rigid cohomology will soon be such a pair of theories. Indeed, (i) is proven in , (ii) in , and (iv) in . Only (iii) is not yet proven. Once (iii) has been proven, then Theorems 2.3 and 4.1 will hold for the eigenvalues of Frobenius on compact rigid cohomology, with divisibility by $`q^\mu `$ as algebraic integers and hence with slopes of Frobenius $`\mu `$.

### 5.3.  Application to Hodge type

Deligne has also formulated a Hodge theoretic version of the problem considered here. Given a nonempty separated $``$-scheme $`X/`$ of finite type, recall that its compact support Hodge type $`\alpha (X/)`$ is the largest integer $`\alpha `$ such that the Hodge filtration $`\mathrm{Fil}`$, for the mixed Hodge structure on $`H_c^{}(X)`$, satisfies

(5.3)
$$\mathrm{Fil}^\alpha H_c^{}(X)=H_c^{}(X).$$

We can also ”spread out” $`X/`$ to a separated scheme $`𝒳/A`$ of finite type, with $`A`$ a subring of $``$ which is finitely generated as a $``$-algebra. For each such spreading out $`𝒳/A`$, we define $`\beta (𝒳/A)`$ to be the largest integer $`\beta `$ with the following property: for every finite field $`𝔽_q`$, and for every ring homomorphism $`\varphi :A𝔽_q`$, the number of points on the scheme $`𝒳_\varphi /𝔽_q`$ obtained from $`𝒳/A`$ by the extension of scalars $`\varphi `$ is divisible by $`q^\beta `$. We then define $`\beta (X/)`$ to be the maximum of $`\beta (𝒳/A)`$ for all spreadings out of $`X/`$. One sees easily that $`\beta (X/)\text{dim}(X)`$. The problem is to prove that $`\alpha (X/)=\beta (X/)`$. \[Presumably this equality holds because $`\alpha (X/)`$ and $`\beta (X/)`$ are both equal to some intrinsic ”motivic” invariant $`\gamma (X/)`$, cf. , sections 9-10 for Grothendieck’s discussion of this sort of question.\] In general, not much is known. But in cases where the divisibility theorems of Ax et al give an estimate $`\beta (X/)\kappa `$, one knows that $`\alpha (X/)\kappa `$, cf. , , , .

Indeed, given the results on the Hodge type of hypersurfaces as input, the (axiomatizations of the) proofs we give here of Theorems 2.3 and 4.1 also prove their analogues over $``$, with eigenvalue divisibility replaced by Hodge type. Consider first the projective case. The first assertion of the Hodge analogue of Theorem 4.1 is the result , , , . To prove the second assertion, reduce to the $`R=1`$ case by the spectral sequence, and then use induction and the weak Lefschetz theorem exactly as in the finite field case. Consider now the affine case, i.e., the Hodge analogue of Theorem 2.3. The spectral sequence reduces us to the case $`R=1`$. Once we know the first assertion in the case $`R=1`$, we get the second assertion in that case by using induction and the affine weak Lefschetz theorem, exactly as in the finite field case. To prove the first assertion in the $`R=1`$ case, we reduce it to the projective case. For this reduction, for each $`f_i[X_1,\mathrm{},X_n]`$, with $`f_i`$ of degree $`d_i1`$, denote by $`F_i[X_0,\mathrm{},X_n]`$ the homogeneous form of the same degree $`d_i`$ such that

(5.4)
$$f_i(X_1,\mathrm{},X_n)=F_i(1,X_1,\mathrm{},X_n),$$

and denote by $`F_{i,0}[X_1,\mathrm{},X_n]`$ the leading form of $`f_i`$, i.e.,

(5.5)
$$F_{i,0}(X_1,\mathrm{},X_n)=F_i(0,X_1,\mathrm{},X_n).$$

Given $`G`$ the hypersurface in $`𝔸^n`$ defined by $`_if_i^{a_i}=0`$, denote by $`G^{\mathrm{proj}}`$ the hypersurface in $`^n`$ defined by $`_iF_i^{a_i}=0`$, and denote by $`G_0^{\mathrm{proj}}`$ the hypersurface in $`^{n1}`$ defined by $`_iF_{i,0}^{a_i}=0`$. Then we have an excision sequence

(5.6)
$$\mathrm{}H_c^{i1}(^{n1}G_0^{\mathrm{proj}})H_c^i(𝔸^nG)H_c^i(^nG^{\mathrm{proj}})\mathrm{}.$$

But $`H_c^{}(^{n1}G_0^{\mathrm{proj}})`$ has Hodge type at least $`\mu (n;d_1,\mathrm{},d_r)`$, and $`H_c^{}(^nG^{\mathrm{proj}})`$ has Hodge type at least

(5.7)
$$\mu (n+1;d_1,\mathrm{},d_r)\mu (n;d_1,\mathrm{},d_r).$$

Thus $`H_c^{}(𝔸^nG)`$ has Hodge type at least $`\mu (n;d_1,\mathrm{},d_r)`$, as required.


text:
# Instantaneous frequency and amplitude identification using wavelets: Application to glass structure

We regret that it has come to our notice that some typographical errors appeared in the paper Harrop et al. (2002) in the description of the new mother wavelet function and associated normalization expressions.

The correct expressions for the mother wavelet function are (the equation numbers refer to those in the original paper):

$$\psi (t;\sigma )=e^{t^2/2}\left\{p(\sigma )\left[\mathrm{cos}(\sigma t)\kappa (\sigma )\right]+iq(\sigma )\mathrm{sin}(\sigma t)\right\}$$

$`p(\sigma )`$ $`=\pi ^{1/4}(1+3e^{\sigma ^2}4e^{3\sigma ^2/4})^{1/2}`$ (6a)
$`q(\sigma )`$ $`=\pi ^{1/4}(1e^{\sigma ^2})^{1/2}.`$ (6b)
The variance of the Gaussian approximation to the envelope of the mother wavelet is:

$$\sigma _\psi ^2(\sigma )=\frac{1}{4}\sqrt{\pi }\left\{q^2\left[1+\left(2\sigma ^21\right)e^{\sigma ^2}\right]+p^2\left\{\left[1+\left(32\sigma ^2\right)e^{\sigma ^2}\right]2\left(2\sigma ^2\right)e^{3\sigma ^2/4}\right\}\right\}.$$

With this formulation, the remainder of the paper is correct and all results are unaffected.

###### Acknowledgements.

We are grateful to Victor P. Ostanin for pointing out the mistake.