text:
# Near infrared spectroscopy of starburst galaxies

## 1 Introduction

Although a wealth of information has been accumulated on starburst galaxies over the last 30 years, there are still fundamental questions which are left unanswered. In particular, we do not know if starburst galaxies form stars according to a universal Salpeter’s initial mass function (IMF), as it seems to be the case in normal galaxies (Elmegreen 1999), or if their IMF slope is flat or has a high lower mass cutoff, which would allow starburst galaxies to form preferentially massive stars (Rieke et al. 1993). Neither do we know if the typical duration of a burst is short, comparable to dynamical time–scales (Mas-Hesse & Kunth 1999), or if bursts are spread over a longer period of time (Meurer 2000).

In the past, various observations were carried out in order to find an answer to these two questions. Some of these studies have yield quite surprising results. By comparing different luminosity ratios in the optical and far infrared, Coziol (1996) found that most starburst galaxies may have kept forming stars at near constant rates over the last few billion years. The same conclusion was reached by Goldader et al. (1997b) for a sample of luminous infrared galaxies (L$`{}_{IR}{}^{}>10^{10}`$ L). Using synthetic starburst models, these authors found continuous star formation and burst ages varying between $`10^7`$ and $`10^9`$ yr. They also found that the best fitted IMF is Salpeter like, but have an upper mass cutoff M$`{}_{up}{}^{}=30`$ M. This kind of IMF in starburst galaxies was previously reported by Doyon, Puxley & Joseph (1992), which also suggest that the upper mass cutoff may vary from galaxy to galaxy.

Using different arguments based on various luminosity ratios, Deutsch & Willner 1986, Coziol & Demers 1995 and Coziol 1996 have found that B type stars may be predominant in the ionised regions of massive starburst galaxies. Taken at face value, this last phenomenon seems consistent with the ages of bursts deduced by Goldader et al. and the special form of the IMF found by these authors and Doyon, Puxley & Joseph. However, as pointed out by Coziol 1996, assuming starburst is a self–sustained phenomenon, this observation could also be interpreted in terms of a sequence of short bursts spread over a few Gyr period.

Evidence of extended or multiple bursts was recently found in many well known active star forming galaxies, like NGC 6764 (Schinnerer, Eckart & Boller 2000), Arp 220 (Anantharamaiah et al. 2000), NGC 1614 (Alonso–Herrero et al. 2000) and most recently M82 (de Grijs, O’Connell & Gallagher 2000). In another recent paper, Meurer (2000), using HST, distinguished in nearby starburst galaxies different star clusters embedded in a diffuse glow of recently formed stars. Examining their colors, he concluded that these clusters are consistent with instantaneous bursts with very young ages (from 0 up to 100 Myr), while the diffuse light seems to be produced by stars formed continuously over a period ten times longer than the crossing times. A recent study of clusters in the irregular starburst galaxy NGC 1569 reveals similar clusters with ages between 3 Myrs and 1 Gyr (Hunter et al. 2000). Old (3 Gyr) merger remnants were recently reported in the radio galaxy NGC 1316 (Goudfrooij et al. 2000). Multiple bursts and low upper mass cutoff for the IMF may also be necessary to explain the number of different Wolf–Rayet sub–types in some WR galaxies (Schaerer et al. 2000). Even the small mass and less evolved HII galaxies seem to be formed of age–composite stellar systems (Raimann et al. 2000).

In the present article, we explore these issues further by studying the spectral characteristics in the near infrared (NIR) of a new sample of 48 starburst galaxies with different physical characteristics. As it is well known, the starburst family is composed of a great variety of galaxy types, spanning several orders in size, mass, luminosity and gas metallicity (Salzer, MacAlpine, & Boroson 1989). The burst characteristics of these galaxies may not necessarily be the same. By selecting galaxies with different physical characteristics, we therefore expect to draw a more general picture of the starburst phenomenon.

Our observations complete the study of luminous infrared galaxies (LIRG) made by Goldader et al. (1995, 1997a,b). By adding these two samples together, one can compare starburst galaxies which span a factor 100 in B and almost 1000 in FIR luminosity.

All the galaxies in our sample have modest far infrared (FIR) luminosity (L$`{}_{IR}{}^{}<10^{10}`$ L). According to the IRAS luminosity function (Rieke & Lebofsky 1986; Vader & Simon 1987; Saunders et al. 1990) such galaxies may form the bulk of the star forming galaxies in the nearby Universe. Our study, therefore, should reveal the “normal” behaviour in the NIR of starburst galaxies in the nearby Universe.

The K–band window in the NIR is ideal for our study. The extinction by dust in this part of the spectrum is $`1/10`$ that in the optical, which allows to compare star formation in galaxies with very different dust content. The K–band window also offers several diagnostic spectral lines for studying young stellar populations: the narrow emission line Br$`\gamma `$ provides some constraints on the relative number of O and B stars, while the CO band absorption, longward of 2.3 $`\mu `$m, can be used to estimate the ratio of Red Giants (RGs) over Red Super–Giants (RSGs) and better constrain the duration and age of the bursts (Doyon, Joseph & Wright 1994). All these parameters can now be easily deduce from available synthetic starburst models (Leitherer et al. 1999).

## 2 Presentation of the sample

### 2.1 Selection criteria and biases

The 48 galaxies in our sample were selected using optical criteria: they present a UV–excess in spectral lines or in the continuum, as established using an objective prism or the multiple filters technique. The galaxies were taken from various sources: the Montreal Blue Galaxy (MBG) survey (Coziol et al. 1993; 1994), the sample of compact Kiso galaxies observed by Augarde et al. (1994), the University of Michigan survey (Salzer, MacAlpine, & Boroson 1989) and the spectrophotometric catalogue of HII galaxies (Terlevich et al. 1991).

Discussion of the biases affecting these various surveys can be found in Coziol et al. (1997). In principle, by adding galaxies from objective prism and multiple filters surveys one can span the complete spectrum of starburst galaxies, including HII galaxies and Starburst Nucleus Galaxies (SBNGs). In practice, however, observation of HII galaxies in the NIR is more difficult, being less luminous than the SBNGs, the HII galaxies need too long exposure times. This explain the bias towards SBNGs in our sample.

By choosing galaxies which were selected based on optical criteria, we also favour galaxies which have a moderate to low FIR luminosity (Coziol et al. 1997). As we already mentioned in our introduction, this bias is opportune, as it allows us to study the most common type of starburst galaxies found in the nearby Universe (Rieke & Lebofsky 1986; Vader & Simon 1987; Saunders et al. 1990).

### 2.2 Physical characteristics

The physical characteristics of the galaxies in our sample are listed in Table 1. The redshifts, B magnitudes, morphologies and projected dimensions in the optical were all taken from NED111The NASA/IPAC Extragalactic Database.. The distances, in column 3, were estimated assuming H$`{}_{0}{}^{}=75`$ km s-1 Mpc-1, after correcting for the motion of the sun. These distances were used to determine the linear scale in kpc subtended in each galaxy by an angle of 1 arc–min on the sky (column 7). In many cases the morphology was not given. An examination of their images, as available in NED, allows us to determine that they look compact in appearance. These cases are identified in column 5. Only one galaxy, MBG 23388-1514, was not morphologically classified. Although we did not judged it compact, we could not establish its morphology.

Various spectroscopic data were collected from the literature to complement our analysis. Optical spectra of these galaxies were obtained using comparable apertures (2–3 arc–second) and median–low spectral resolutions, allowing fair comparisons with our data. When the flux is available, the observed H$`\alpha `$ luminosity (column 8) was estimated. Metallicities (column 9) were determined using the ratio R$`{}_{3}{}^{}=1.35\times `$(\[O iii\]$`\lambda 5007/\mathrm{H}\beta )`$ and the empirical relation proposed by Vacca & Conti (1992). For the metallicities of I Zw 18 and II Zw 40, we adopted the values estimated by Izotov, Thuan & Lipovetsky (1998) and Masegosa, Moles & Campos-Aguilar (1994). The activity types (column 10) were established by comparing the ratios \[NII\]$`\lambda 6584`$/H$`\alpha `$ with \[OIII\]$`\lambda 5007`$/H$`\beta `$ (Baldwin, Phillips & Terlevich 1981; Veilleux & Osterbrock 1987). The numbers in the last column identify the references for the various spectroscopic data.

We give in Table 2 some information in the FIR for 31 galaxies (64%) in our sample. These data come from the IRAS Faint Source Catalogue, as found in NED. The FIR luminosity in column 2 was determined using the relation (Londsdale et al. 1985): $`\mathrm{log}(\mathrm{L}_{\mathrm{IR}})=\mathrm{log}(\mathrm{F}_{\mathrm{IR}})+2\mathrm{log}[z(z+1)]+57.28`$, where $`z`$ is the redshift, $`\mathrm{F}_{\mathrm{IR}}=1.26\times 10^{11}(2.58f_{60}+f_{100})`$ erg cm-2 s-1 and $`f_{60}`$ and $`f_{100}`$ are the fluxes in Jansky at 60 and 100 $`\mu `$m respectively. In column 3 and 4, we also give the FIR spectral indices (Sekiguchi 1987): $`\alpha (\lambda _1,\lambda _2)=\mathrm{log}(f_{\lambda _1}/f_{\lambda _2})/\mathrm{log}(\lambda _2/\lambda _1)`$. To facilitate our analysis, only galaxies which have IRAS fluxes with high or intermediate qualities had their FIR characteristics estimated. The last column gives the classification of activity type as deduced from FIR criteria (see section 2.2).

### 2.3 Nature of the starburst galaxies in our sample and physical variety of starburst galaxy hosts

We show in Figure 1a the spectroscopic diagnostic diagram used to classify 28 galaxies in our sample with spectroscopic data available in the optical. In this diagram the horizontal dash line distinguishes between HII galaxies and SBNGs (Coziol 1996). We count 19 SBNGs for 9 HII galaxies. This result confirms our observational bias towards SBNGs. The thin continuous curve in Figure 1a follows the locus traced by normal disc HII regions (Coziol et al. 1994). On this curve, the gas metallicity, given as $`12+\mathrm{log}(\mathrm{O}/\mathrm{H})`$, increases as the level of excitation (related to \[OIII\]/H$`\beta `$) decreases. The galaxies in our sample cover the complete sequence in starburst’s metallicities. Considering only the SBNGs, we found a mean metallicity $`12+\mathrm{log}(\mathrm{O}/\mathrm{H})=8.7\pm 0.2`$, which is typical of this class of galaxies (Coziol et al. 1999).

Half of the galaxies that could not be classified in the optical have FIR characteristics which can be used to determine their starburst type: SBNGs or HII galaxies. In Figure 1b we show a diagnostic diagram based on IRAS spectral indices $`\alpha `$(60,25) and $`\alpha `$(100,60). Starburst galaxies are described in this diagram by a model with two dust components (Sekiguchi 1987): a cold component with a dust temperature of 27 K and a hot component with dust temperatures varying between 70 and 90 K. According to this model, starburst galaxies differ from normal spiral galaxies by their hot dust contribution, which is usually higher than 60%. Although very few HII galaxies are detected in the FIR, they can usually be distinguished from SBNGs from their hotter dust temperatures. This is indicated by the gray area in Figure 1b. According to this last criterion, most of the galaxies with FIR data in our sample are SBNGs.

### 2.4 Probability to find an AGN in our sample

Using FIR criteria we can also ascertain what is the probability that one of the galaxies in our sample is dominated by an AGN. According to Coziol et al. (1998), 99% of IRAS starburst galaxies have $`\alpha `$(60,25) $`<1.9`$ (the horizontal dash line in Figure 1b), while only 25% of Seyfert galaxies have such FIR colours. Almost all the galaxies in our sample have $`\alpha `$(60,25) $`<1.9`$, which excludes the possibility that one of these galaxies is dominated by an AGN. Note that the three galaxies exceeding this limit in our sample (Mrk 538, Mrk 401 and MBG 02072-1022) have already been classified as SBNGs from their spectra. In fact, all the galaxies previously classified as starburst based on their optical spectra occupy this region of the diagram, which thus confirms their nature as starburst.

Only six of the galaxies detected in the FIR diagram do not have a spectral classification in the optical. Based on the reported statistics above, and the total number of galaxies which was necessary to established it (see Coziol et al. 1998), we estimate to 7% the probability to find an AGN (which is probably not predominant) in this region of the diagram. This reduce to none the probability that one of these six galaxies also host an AGN. This results is not surprising considering the moderate FIR luminosity of these galaxies and the fact that the probability to find an AGN increases with the FIR luminosity (Veilleux, Kim & Sanders 1999).

The above conclusion hold also for the ten galaxies in our sample without spectral classification or FIR detection. Height of these galaxies were classified as starburst based on an objective prism spectrum, which rules out the presence of a broad line or luminous AGN. The last two are MBG galaxies which were classified as starburst based on narrow lines and small \[NII\]/H$`\alpha `$ ratios in one spectrum. The fact that these galaxies were not detected in IRAS suggests that the probability they host an AGN is practically null.

### 2.5 Comparison with LIRGs and uLIRGs

At this point of our analysis, it is convenient to compare our sample of galaxies with the 16 LIRGs previously classified as starburst in the study of Goldader et al. (1997a). In Figure 1a, we see that the LIRGs have spectroscopical characteristics similar to SBNGs. However, they are generally more metal rich, having solar metallicity on average. Their hot dust contribution (Figure 1b) is also higher than in SBNGs, which is consistent with their higher FIR luminosity.

In Figure 1c, we compare the FIR and B luminosities of our galaxies with those of the LIRGs. For comparison, we also include some ultra luminous infrared galaxies (uLIRGs) from the literature. The ratios LIR/LB can be used to distinguish galaxies with unusually high star formation rates (see Coziol 1996): normal late–type spiral galaxies with constant star formation rates over the last 3 Gyrs have $`1/3<`$LIR/LB $`<3`$. One can see that the ratio LIR/LB tends to increase with the B luminosity. The uLIRGs and LIRGs have both higher B luminosities and, therefore, higher LIR/LB ratios as compared to SBNGs. Most of the galaxies in our sample have ratios LIR/LB varying between 1 and 3. This comparison suggests that uLIRGs and LIRGs are generally more massive than SBNGs (having higher B luminosity) and may have more active burst regions or have younger bursts (as judged from their higher LIR/LB ratios).

One remarkable feature of Figure 1c is how the starburst galaxies in our sample differ from well known uLIRGs, like Arp 220, NGC 6240, NGC 1614 or NGC 3690. The nature of uLIRGs is still in debate. It is not sure if part of the activity observed in these galaxies is not due to an hidden AGN. This is not the case of the galaxies in our sample. The position, in Figure 1c, of the ”prototype“ starburst galaxy M82 seems, however, interesting. This galaxy looks more like an uLIRG than to one of our galaxies. This is consistent with the extreme properties observed in this object, which may also suggest some kind of AGN activity (Seaquist, Frayer & Frail 1997; Allen & Kronberg 1998; Kaaret et al. 2000). Two other archetype starburst galaxies in our sample are NGC 7714 (Mrk 538) and Mrk 710. These two galaxies are more representative of the starburst activity in our sample of galaxies.

The above analysis emphasizes the diversity of physical characteristics of galaxies hosting a starburst. Our observations should reveal if this variety of physical characteristics is reflected in their NIR spectral features.

## 3 Observation

The 2–2.5 $`\mu `$m spectra, studied in this paper, were obtained with the CGS4 spectrometer (Mountain et al. 1990) on UKIRT on the nights of 1992 November 5 and of 1993 November 24, 25, and 26. A 75 l/mm grating was used in first order, providing a resolving power ($`\lambda /\delta \lambda `$) of $``$300 in the $`K`$-band and a spectral coverage encompassing most of the $`K`$-band. The plate scale was 3 arc–second per pixel and the slit width was 1 pixel. This aperture was sufficient to cover the central and most luminous part (usually the nucleus) of the galaxies. The slit length was 30 pixels. For all galaxies, the slit was positioned in the north south direction on the brightest optical peak as seen from the offset guider camera. Integration times were generally 10–20 minutes and sky frames were obtained by nodding 10 pixels along the slit, hence keeping the object always on the slit. Several spectroscopic and photometric standards were also observed during each night. The spectrophotometric accuracy of our spectra is estimated to be $``$20%. The spectra were reduced under the IRAF environment using IRAF scripts written by one of us (RD). Details on the procedure followed for the reduction can be found in Doyon et al. (1995).

In Table 3, we present the fluxes and equivalent widths of the Br$`\gamma `$ and H2 emission lines as measured in our spectra. These quantities were estimated after rectifying the continuum by fitting a power law ($`F_\lambda \lambda ^\beta `$) from the featureless sections of the spectrum. The choice of a power–law is physically justified by the fact that late-stars show approximately Rayleigh–Jeans spectrum in the NIR (F$`{}_{\lambda }{}^{}\lambda ^4`$, between 2 and 2.9$`\mu `$m; see appendix A of Doyon, Joseph & Wright 1994). A $`\chi ^2`$ minimisation routine was then used to fit a Gaussian profile to the Br$`\gamma `$ emission line. In Table 3, values without an error estimate corresponds to 3 sigma upper limits.

We give in Table 4 the exponent ($`\beta `$) of the power law fit for the continuum, the CO spectroscopic indices (COspec), as defined in Doyon, Joseph & Wright (1994), and the spectroscopic K magnitudes. The COspec quoted include the uncertainty in the continuum level. In general, the error on COspec is dominated by this uncertainty, which is less than 0.02 mag. The second column of Table 4 gives the redshifts as deduced from the features observed. These redshifts are in relatively good agreement with the one found in NED. The last column gives the signal to noise ratios in our spectra.

For each galaxy in the two tables we give two sets of data. The first set corresponds to values as measured using an aperture covering 3$`\times 9`$ arc–second (Aperture 1). The second set (the last four columns of each tables) corresponds to values as measured using an aperture covering only 3$`\times 3`$ arc–second (Aperture 2). In our following analysis, the smaller aperture will be associated to the nucleus of the galaxies.

A few representative spectra are shown in Figure 2 and Figure 3. We regroup the spectra in two categories. In the first category, Figure 2, the galaxies possess strong Bracket $`\gamma `$ (2.166 $`\mu `$m) emission. These galaxies also show relatively strong HeI (2.059 $`\mu `$m) and H2 1-0 S(1) (2.122 $`\mu `$m) emission lines. They show, on the other hand, different strengths of the CO absorption band longward of 2.293 $`\mu `$m.

The most extreme case in our sample is the HII galaxy II Zw 40. This galaxy shows very strong emission lines and extremely weak CO band. This galaxy also happen to be one of the less chemically evolved galaxies in our sample. Two SBNGs with intense emission lines are Mrk 710 and Mrk 1089. Most galaxies in our sample resemble Kug 0720+335 and Mrk 538 (NGC 7714), which have medium intensity emission lines and relatively strong CO band.

In the second category, Figure 3, we find galaxies which have weaker emission lines than Mrk 538 and somewhat stronger CO band. In some galaxies, like Mrk 332, the NIR emission lines are barely visible, and only upper limits can be measured.

The NIR spectra of our sample of galaxies show significant variations. This result indicates that the diversity of physical characteristics of starburst galaxy hosts, as observed in the optical and far infrared, implies some variety of the NIR spectral features. Synthetic starburst models will now be used in order to understand what these variations mean in terms of the burst characteristics.

## 4 NIR Analysis

### 4.1 Synthetic starburst models: duration, age of the burst and IMF

Using synthetic starburst models, it is possible to interpret NIR features like EW(Br$`\gamma `$) and COspec in terms of the duration of the burst, its age and the form of the IMF (Puxley, Doyon & Ward 1997; Leitherer & Heckman 1995; Mayya 1997). The principal steps in the synthetic starburst model are the following (Leitherer & Heckman 1995): A) stars are formed at specified rates and distributed along the HR diagram; B) evolutionary models describe the time dependence of the individual physical parameters of the stars (mass, luminosity, etc.); C) the stellar number densities in the HR diagrams are determined and any desired synthetic quantities assigned to each stars. By summing over the entire stellar population the model yields the integrated properties of starburst galaxy.

In these models, star formation is represented by an exponential law: $`\xi (t)=\xi (t=0)\mathrm{exp}t/\tau `$, where $`t`$ is the age of the burst and $`\tau `$ its duration. Two limiting cases are of special interest: an instantaneous burst and a constant star formation rate. An instantaneous burst corresponds to the case where the duration of star formation is short as compared to the age of the galaxy ($`\tau <<t`$). All the stars are thus formed at the same epoch. The other limiting case is a constant star formation rate (CSFR). This mode of star formation is characteristic today of disc in giant spiral galaxies (Kennicutt 1983).

It is important to note that theoretically a CSFR can always be approximated as a series of instantaneous bursts. It means that a system having experienced a sequence of bursts over a time period comparable to $`\tau =t`$ would be observationally indistinguishable from a galaxy with a CSFR (Leitherer et. all 1999).

Another important parameter in synthetic starburst model is the IMF. This function is usually expressed as a power law: $`\varphi (M)=Cm^\alpha `$, where the constant $`C`$ is determined by the total mass converted into stars. To get the whole spectrum of star masses formed, the IMF is integrated between the upper and lower mass cutoff, Mup and Mlow. In starburst models the exponent of the IMF and the two mass cutoff are not very well constrained and it is therefore important to try different possible values.

We choose to compare our data with the models of Leitherer et al. (1999). These authors have made their grids of results directly accessible on the WEB222http://www.stsci.edu/science/starburst99. We consider only two limiting cases: the instantaneous burst and the CSFR (1 M per yr). For each of these cases, three different scenarios are tested: 1) a Salpeter IMF, $`\alpha =2.5`$, with M$`{}_{up}{}^{}=100`$ M; 2) a Salpeter’s IMF with M$`{}_{up}{}^{}=30`$ M; 3) an IMF with a steeper slope, $`\alpha =3.3`$ and M$`{}_{up}{}^{}=100`$ M. The models cover ages from $`10^6`$ to $`10^9`$ yr.

Comparisons of our data for EW(Br$`\gamma `$) and COspec with the instantaneous burst models are presented in Figure 4. From this figure, we conclude that none of the instantaneous burst models can fit the ensemble of our data. For solar metallicity (Z $`=0.02`$) the models generally underestimate the value of EW(Br$`\gamma `$). Some galaxies can be fitted by instantaneous bursts, but the predicted metallicity is 2 times solar, which is much higher than the gas metallicity we have estimated in these galaxies. Our results confirm those of Goldader et al. (1997b), extending their conclusion to starburst galaxies with different physical properties and FIR luminosity. It implies that the instantaneous burst scenario is not a good representation of the starburst phenomenon.

Our data are compared with the CSFR models in Figure 5. One can see in Figure 5a that the Salpeter’s IMF model with M$`{}_{up}{}^{}=100`$ M cannot fit the ensemble of our data. But a relatively good fit is obtained in Figure 5b, when the upper mass cutoff is equal to M$`{}_{up}{}^{}=30`$ M . We see no improvement in Figure 5c in adopting an IMF slope $`\alpha =3.3`$.

Once again our results confirm those of Goldader et al. (1997b), extending their conclusion to all the starburst types known. We conclude that the best parameters which describe starburst galaxies in the nearby Universe are a CSFR and a Salpeter’s IMF with an upper mass cutoff M$`{}_{up}{}^{}=30`$ M.

According to this solution, the bursts have ages between 10 Myr and 1 Gyr, the majority being older than 100 Myr. Comparing with LIRGs, we found, on average, younger bursts for these galaxies, with ages lower than 100 Myr. This is consistent with the difference in luminosity ratios noted in Section 2.5. The model reproduce correctly also the fact that LIRG are more metal rich than the SBNGs. The consistency with the metallicities deduced from their optical spectra (Section 2.2) is remarkably good.

According to the model, HII galaxies must have younger bursts than the SBNGs. The metallicities predicted by the models are also in good agreement with those measured. Young bursts and low metallicities are also consistent with the higher dust temperatures suggested by the FIR model in Section 2.3.

In our sample, the galaxy which have the youngest burst is II Zw 40. The position of this galaxy in Figure 5a suggests that it may be possible to fit an IMF with upper mass cutoff M$`{}_{up}{}^{}=100`$ M (although the predicted metallicity would then be wrong). The same conclusion may also apply to Mrk 1089. These observations may suggest varying IMF in some galaxies (Doyon, Puxley & Joseph 1992).

From the above discussion, we conclude that the solution of the synthetic starburst model is consistent with observations in the optical and FIR. In particular, it satisfactorily explains the differences between the different starburst types: HII galaxies have younger bursts and lower metallicities than SBNGs, while LIRGs have younger bursts but higher metallicities. These differences may be related to different histories of star formation or different stages in burst evolution.

Taken at face value, the above result confirms the studies of Doyon, Puxley & Joseph (1992), Goldader et al. (1997b) and Coziol (1996). The generality of our observation further suggests that this must be a characteristic of starburst galaxies in the nearby Universe. However, the above solution may not represent strong constraints on the duration of the burst or the IMF. This is because the synthetic starburst models we used cannot distinguished between real constant star formation and a series of instantaneous bursts (Leitherer et. all 1999). In order to verify this alternative, we will now reexamine our spectra and search for traces of previous bursts.

### 4.2 Spatial variation of NIR features: evidences for sequences of bursts

In order to reproduce CSFR using a sequence of bursts, one must assume that this sequence spreads over a relatively long period of time. The solution of synthetic starburst model suggests a period extending over $`10^8`$ up to 1 Gyr. If this sequence of burst is related to some self–regulated mechanism or to propagation of star formation, we may also assume that the different bursts do not happen in the same regions of space. This implies that the bursts regions should be heterogeneous in space and in time. With sufficient spatial resolution, we should then observe a variation in burst characteristics as we survey different regions. As a first test, therefore, we can check if a variation of aperture in our spectra induces some variation of the NIR spectral features.

Before doing the aperture test, it is important to verify that the obvious increase of scale with distance, implied by our two fixed apertures, do not introduce any systematic variations, which could be misinterpreted as variation of burst characteristics. Associating the smaller aperture ($`3\times 3`$ arc–second) to the nucleus, we verify in Figure 6 that the ratios of the Br$`\gamma `$ fluxes and equivalent widths, as measured in and out of the nucleus, do not show any relation with the redshift. Similar negative results were also obtained for H2, the K magnitudes, the COspec and for the slope of the continuum ($`\beta `$). These negative results suggest that what ever variations we may observed for these ratios, their origin must be intrinsic to each galaxy.

Having verified that the two fixed apertures do not introduce spurious effects, we now proceed to examine how the different ratios varied in each galaxy. In Figure 7a, we first compare the ratios of the fluxes in Br$`\gamma `$ with the ratios of the equivalent widths. In general, the ratios f(Br$`\gamma `$)/f(Br$`\gamma `$)N are higher than one, which indicate that the bursts of star formation generally extend over a few kpc around the nucleus. The equivalent widths, on the other hand, are generally wider in the nucleus, except in galaxies where the ratio f(Br$`\gamma `$)/f(Br$`\gamma `$)$`{}_{N}{}^{}>2`$, which correspond to extended bursts, where the equivalent width stays constant or increases out of the nucleus. According to synthetic starburst models, EW(Br$`\gamma `$) is an indicator of the age of the burst. From the above observation we then deduce that in the galaxies where the star formation is more concentrated, the bursts are younger in the nucleus. The contrary seems to be true in galaxies where the bursts are more extended. No difference is observed between HII galaxies and SBNGs.

The case for H2, Figure 7b, is slightly different. Very few galaxies in our sample show significant H2 emission. In these galaxies we find the same tendency than for Br$`\gamma `$, which is the H2 emission generally spreads out of the nucleus. But the H2 equivalent width tends to be wider out of the nucleus. In Figure 7c one can see that in galaxies where the star formation is concentrated in the nucleus (f(Br$`\gamma `$)/f(Br$`\gamma `$)$`{}_{N}{}^{}<2`$), the H2 emission is more intense than Br$`\gamma `$ out of the nucleus. Only two galaxies, Mrk 1089 and 1194, show the reverse tendency. For the galaxies in our sample with significant amount of H2 emission, the H2 emission seems, therefore, to be always more intense and wide spread in regions where Br$`\gamma `$ is less intense and less abundant. We verified also that 50% of the galaxies without H2 detection have detectable Br$`\gamma `$ emission, which is consistent with the general tendency observed. The signification of this variation is not obvious, however, and its possible interpretation will therefore be kept for later during our analysis.

For the K–band magnitude, we naturally expect it to increase as the number of stars included in the aperture increases. However, the mean variation observed in Figure 7d seem smaller than what we expected. While the gain in surface represented by the difference in aperture is a factor 3, we find a meager $`\mathrm{\Delta }`$K $`=0.7`$, which corresponds to an increase in flux by only a factor 1.8. The K–band emission, therefore, seems mostly concentrated in the nucleus. Indeed, one can see in Figure 7d that the difference in K–band magnitude is smaller in galaxies where the bursts are concentrated (f(Br$`\gamma `$)/f(Br$`\gamma `$)$`{}_{N}{}^{}<2`$) than in galaxies where the bursts are extended. The K–band emission seems, therefore, to tightly follow the region of the bursts. This observation suggests that the stars emitting in the K band must be tightly related to the bursts.

No trend was observed for COspec or $`\beta `$. These two parameters show random variations with standard deviations $`\pm 0.04`$ for the difference in COspec and $`\pm 0.15`$ for the ratios of $`\beta `$.

To illustrate what effects the variations of NIR features have on the burst characteristics of each galaxy, we show in Figure 8 the changes introduced by an increase in aperture in the burst model of Figure 5b. In this figure, an increase of the equivalent width transforms into a younger age while an increase of the COspec transforms into an increase of metallicity and a younger age. In Figure 8, one can see that some variations may imply significant differences in terms of the burst characteristics. No trend is observed, however, which suggests that the cause of these variations may depend on the particular history of star formation of each galaxy. Higher spatial resolution will be necessary to confirm this result.

From the above analysis we conclude that the variation of the NIR spectral features with aperture are consistent with a variation of burst characteristics in space. This result supports our hypothesis that the bursts regions are not homogeneous in space. However, the absence of systematic trends and the lack of spatial resolution impede us to conclude on the real cause of this variation and we must therefore search for other evidence in favour of the multiple burst hypothesis.

### 4.3 Luminosity–luminosity relations

Our previous analysis suggests that the stars emitting in the K–band are tightly related to the burst population. We should then expect to find a good correlation between the K–band absolute magnitude and parameters link to the burst. This is important to verify, because the stars producing the K–band luminosity are probably the same stars which produce the CO band. A correlation of K–band luminosity with parameters directly linked to the burst would thus favour massive and young stars, namely RSGs, over older stars like RGs.

We see in Figure 9a that L(Br$`\gamma `$) increases with the K–band absolute magnitude. A linear regression performed on our data (continuous curve in Figure 9a) gives a weak but statistically significant coefficient of correlation R$`=67`$%. (with 33 points, the probability to obtain higher correlation by chance is P$`{}_{33,0.67}{}^{}<0.05`$%; Pugh & Winslow 1966). The correlation improves to 82% when we include the LIRGs (dash curve in Figure 9a). The properties of our galaxies are in good continuity with those of the LIRGs, which form the upper limit of the distribution.

The correlation of L(Br$`\gamma `$) with the K–band absolute magnitude confirms that the stars producing this luminosity are related to the burst population. The relative weakness of this relation, however, may suggest that they are not correlated to the younger stars. But this result is consistent with what we expect if the stars emitting in the K–band are in majority RSGs: the number of RSGs increases with the intensity of the burst, but these stars also appear in great number only after the peak intensity of the bursts.

In Figure 9b, we find a reverse tendency for EW(Br$`\gamma `$), this parameter decreasing as MK increases. These two parameters are not strongly correlated (R$`=57`$%), although the correlation is still statistically significant (P$`{}_{33,0.57}{}^{}<0.05`$%). The obvious interpretation for this reverse tendency is that galaxies which are richer in burst regions naturally also contain the highest number of evolved stars.

What is interesting in this last figure is the high dispersion of the data. This high dispersion suggests that the evolution of the bursts depends on the particular star formation history of each galaxy. This is consistent with our analysis of the variation with aperture and suggests that the bursts may also be heterogeneous in time (the dispersion in EW(Br$`\gamma `$) is a dispersion in time). The behaviour of the LIRGs is particularly remarkable, suggesting younger bursts than the observed tendency would have predicted. This could be one indication of repetitive bursts.

Note that we do not reject, a priori, the alternative which is that the stars emitting in the K–band are old RGs. In this case, the correlation of MK with L(Br$`\gamma `$) would suggest some relation between mass and the intensity of the bursts, while the correlation between MK and EW(Br$`\gamma `$) would mean that the bursts evolve more rapidly in galaxies where they are more intense. However, it would be difficult in this case to understand why the correlation are not stronger. But such interpretations, in fact, are much more constraining than what our data can revealed (we do not have information on the masses or the intensity of the bursts, which, anyway, should be normalized by the surface or volume). To our knowledge, such relations were never reported before, and our data are obviously insufficient to defend them. Our favoured interpretation, on the other hand, do not imply any new mechanisms, relying only on standard star evolutionary scenarios. Another argument in favour of our interpretation may be the relatively strong correlation we find between L(H2) and MK, as we now show.

A very strong correlation, R$`=92`$%, is found between L(H2) and MK in Figure 10a. The lower number of points in this fit do not influence the results: with 16 points, the probability to obtain a higher correlation by chance is P$`{}_{16,0.92}{}^{}<0.05`$%. This good correlation do not depends on the inclusion of the LIRGs either, the coefficient increasing only to R$`=94`$% when we include them. For the equivalent widths, Figure 10b, we also find a better correlation than for Br$`\gamma `$, R$`=62`$%, P$`{}_{16,0.62}{}^{}<1.4`$%. This behaviour suggests that the H2 emission is better correlated to a more evolved phase of the bursts, being correlated to SRGs, than to a younger phase. In general, therefore, the behavior of H2 emission seems consistent with the hypothesis of heterogeneous burst in space and time, showing variation with aperture (section 4.2) and the coexistence of evolved components with younger ones. It would help, however, to determine what is the process producing this line, since the two most probable mechanisms, supernovae shocks remnants and UV-fluorescence, imply much different time scales.

In principle, it would be possible to determine what is the mechanism producing the H2 emission by comparing the two luminosities L(Br$`\gamma `$) and L(H2). In Figure 11 we find that L(H2) is relatively well correlated to L(Br$`\gamma `$) (R$`=74`$% with P$`{}_{16,0.62}{}^{}<0.4`$%). But the ratios L(Br$`\gamma `$)/L(H2) stay between 0.1 and 1, which is consistent with supernovae shocks remnants (Moorwood & Oliva 1990) or UV-fluorescence (Puxley, Hawarden & Mountain 1990). Therefore, although our observations suggest that the H2 emission phenomenon is related to a more evolved phase of the bursts, we are unable to say if this phase happen a few million years (the time scale of supernovae) or a few $`10^7`$ or $`10^8`$ yrs (the time scale of B stars, responsible for UV-fluorescence) after the maximum of the bursts.

### 4.4 Spectral types of ionising stars

In Coziol & Demers 1995 and in Coziol (1996), one evidence in favor of a sequence of bursts was the predominance of B type stars in starburst nucleus galaxies. We can repeat the analysis performed in Coziol (1996) to verify this result in the NIR. The advantage of working in the NIR is that the extinction in this part of the spectrum is less severe, and the fraction of obscured stars should therefore be less important (Calzetti et al. 1995), decreasing the uncertainties related to this method.

The level of extinction in our galaxies can be estimated (Figure 12a) by comparing L(Br$`\gamma `$) with the luminosity in H$`\alpha `$ (L(H$`\alpha `$)). For the theoretical relation between L(Br$`\gamma `$) and L(H$`\alpha `$) we use the one adopted by Leitherer & Heckman (1995). The theoretical ratio is traced in Figure 12a as a continuous curve. Lower ratios then must correspond to various magnitudes in extinction in the optical. In our galaxies, the extinction varied between 0 and 2 magnitudes , which is in good agreement with values estimated from the Balmer decrement method (Contini, Considère & Davoust 1998). This should not be surprising considering the low dust extinction in these galaxies. The LIRGs seem to suffer slightly higher extinction, the values varying between 3 and 4 magnitudes.

Once corrected for extinction, the flux ratios Br$`\gamma `$/LIR are proportional to the ionising flux produced by stars with various spectral types (Devereux & Young 1990). We show in Figure 12b the locus expected for ionising star clusters with different spectral types. This figure confirms the dominance of B type stars as found in Coziol (1996).

## 5 Discussion

The question that we need to answer is: can we accept the result of the synthetic starburst model? In order to answer this question we need first to better understand this result. The main reason why the instantaneous burst solution is rejected by our observation is because the predictions of the models are in contradiction with the values of the CO spectral indices we measured. In an instantaneous burst, the Br$`\gamma `$ equivalent width stays high for the first $`10^7`$ years. During this time, COspec is negligible and begins to increase only around $`10^7`$ year, when the most massive stars evolve as RSGs. It is because we observe both intense Br$`\gamma `$ emission and relatively strong CO lines that we need continuous star formation. The answer seems, therefore, to rely on the validity of the CO spectral index.

To obtain a good fit with any of the instantaneous burst models, the CO spectral indices would need to decrease by more than 50%. Obviously, this cannot be due to the method we used to measure these lines. The strength of the CO bands in our spectra is undeniable. A relatively strong value of COspec is consistent with a predominance of RSGs over RGs (Doyon, Joseph & Wright 1994). This is consistent with the correlation found between MK and L(Br$`\gamma `$), which suggests that the stars producing this luminosity, and probably also the CO band, are related to the burst population. Moreover, this relation is relatively weak because RSGs appear in great number only slightly after the culmination of the burst. The relatively strong CO bands in our spectra seems therefore to be real, and a constant star formation rate is consequently necessary to explain our data.

But what about the low upper mass cutoff? If we look at Figure 5, we see that the reason why we need such feature is because the Br$`\gamma `$ equivalent widths seem too weak as compared to the model with upper mass cutoff M$`{}_{up}{}^{}=100`$ M. Now, this result is far from obvious. It is therefore highly significant that when we determine the spectral types of the ionising stars in these galaxies we find a value which is consistent with this result: the dominant ionising stars are early–type B or late–type O which have masses of the order of 10 to 20 M (Panagia 1973), in good agreement with the upper mass cutoff suggested by the model. We have therefore to conclude that the solution suggested by the synthetic starburst model is fully consistent with our data.

But how could star formation be continuous? In order to sustain star formation in starburst galaxies, some mechanism other than interaction is necessary. Observations of nearby starburst galaxies suggests that most ($`70`$%) of these galaxies are relatively isolated and cannot be associated to on–going interaction events (Telles & Terlevich 1995; Coziol et al. 1997b). In our sample, only five galaxies have a peculiar morphology suggesting some sort of direct interaction. The best example is NGC 1089. The galaxy II Zw 40, on the other hand, which also has the youngest burst, cannot be easily connected to an interaction event. The same is true for most of the galaxies in our sample. More than two thirds (19) of our galaxies are isolated early–type spirals (earlier than Sbc) or galaxies with a compact appearance (14 galaxies). The rest (10 galaxies) looks like normal late–type spirals (Sbc or later). There is, consequently, no obvious dynamical cause for the bursts observed in most of these galaxies, which suggests that some ”internal“ regulating or self–sustained star formation mechanism must be active.

In order for star formation to be continuous over a significantly long period of time, star formation rates cannot be very high either. This may be in contradiction with some observations in very intense starburst galaxies. This is why a sequence of bursts may be a more interesting alternative. If the bursts are intense but do not last very long, the reservoir of gas will not be exhaust, allowing ignition of new bursts later on. Assuming some self-regulated mechanism, this process could probably extend over a relatively long period of time. One can already find in the literature different mechanisms capable of producing bursts in sequence, either by regulation or propagation of star formation from supernovae feedback (Gerola, Seiden & Schulman 1980; Krügel & Tutukov 1993) or multiple small merger events on a few Gyr period (Tinsley & Larson 1979). These two alternatives were already found to be consistent with the chemical properties of nearby starburst galaxies (see Coziol et al. 1997 and 1999 for example).

But what evidence do we have in favour of a sequence of bursts? In our spectra we have found some variation of burst characteristics with the aperture which suggests the bursts are not homogeneous in space. The fact that we do not observe systematic trends for these variations suggests that their origin may be related to the particular star formation history of each galaxy. This could be one signature of internal processes. Considering that these variations are observed over regions a few kpc wide, it is probably more realistic to assume also that different star formation events occurred during different time intervals (this would be the case for star formation propagation). The dispersion of the K–band absolute magnitude vs. EW(Br$`\gamma `$) relation is highly suggestive of some intermittent events, superimposed, perhaps, on a more global behaviour.

However, the strongest evidence in favour of a sequence of bursts may be the predominance of early–type B stars in the ionised regions of these galaxies. Such stars, with masses of the order of 10 to 20 M, have lifetime of a few $`10^7`$ yrs only. For one burst population, it is therefore difficult to understand how this particular composition of stars is possible. This cannot be explained solely in terms of an age effect. Indeed, the main sequence lifetime of these stars is only a factor 10 older than those of more massive stars and a factor 10 younger than the age of the bursts deduced from the synthetic starburst model, which would mean that we observe all these galaxies in a very particular and short phase of their evolution. Considering the generality of our observations, this makes it a very improbable event333It is doubtful that we can explain such rare events based on the obvious bias that these are starburst galaxies, considering that such galaxies form between 25 to 30% of all the galaxies in the nearby Universe..

A sequence of short bursts, on the other hand, without any limit on the upper mass cutoff and extending over a relatively long period of time (a few 108 or 1 Gyr) in order to mimic continuous star formation, may reproduce such phenomenon. Short bursts would produce few massive stars, which would then evolve rapidly and all disappear before another burst begin. Depending on the frequency of the bursts, the average population would then evolve statistically towards the predominance of early–type B stars. Note that it is probably impossible to reproduce such stellar population with constant star formation, unless the IMF upper mass cutoff is truncated towards low mass stars.

To finish, we have concluded that we cannot identify what is the mechanism responsible for the H2 emission in our galaxies. However, the different time scales over which these processes culminate should help us estimate which is the more probable one. Supernovae shocks remnants are predominant only when a high number of massive stars are produced and the bursts are very young (a few $`10^4`$ years). Infrared fluorescence, on the other hand, is produced by early B stars, which radiate ample photons in the Lyman–Werner band ($`\lambda =9121108`$ Å). These stars have longer life times and should therefore prevail only when the burst is old or, for some reason, they are predominant in a burst population. The fact that B stars do seem to be over abundant in our galaxies suggests that fluorescence must be the main mechanism responsible for the H2 emission. One would then expect to see an increase of fluorescent emission in regions where B stars dominate over O stars (Tanaka, Hasegawa & Gatley 1991). This would explain some of the variation we observed when we change the aperture, which would constitute, consequently, another evidence in favour of the multiple bursts scenario hypothesis.

## 6 Conclusion

Although the solution proposed by the application of the synthetic starburst models to our sample of starburst galaxies may look counter intuitive, we have shown that it seems to be fully consistent with our data. Our analysis confirms, therefore, the results previously obtained by Deutsch & Willner 1986, Doyon, Puxley & Joseph (1992), Coziol & Demers 1995, Coziol (1996) and Goldader et al. (1997b).

However, the solution proposed by the models may not represent strong constraints on the duration of the burst or the form of the IMF. This is because the models cannot distinguish between constant star formation or a sequence of short bursts over an extended period of time. This alternative, the sequence of short bursts, will have to be tested fully before we can conclude on the values of these two important parameters in starburst galaxies.

What ever is the solution, our observations are clear on one thing, which is that starburst must be a sustained or self–sustained phenomenon: either star formation is continuous in time or different bursts happen in sequence over a relatively long period of time. The generality of our observations suggests that this is a characteristic of starburst galaxies in the nearby Universe.

## Acknowledgments

R. C. would like to thank the Observatoire de Besançon for supporting his work. The authors would also like to thank Leitherer et al. for making their synthetic starburst models available to the community and an anonymous referee for interesting comments and useful suggestions. This research has made use of the NASA/IPAC Extragalactic Database (NED) which is operated by the Jet Propulsion Laboratory, Caltech, under contract with the National Aeronautics and Space Administration.


format: text

text:
# Muon Anomalous Magnetic Moment (𝑔-2)_𝜇 and the Randall-Sundrum Model

## Abstract

We study the effects of the Kaluza-Klein gravitons in the Randall-Sundrum model on the recent BNL measurements of the muon $`(g2)`$ deviation from the standard model prediction. By examining the $`J`$-partial wave unitarity bounds of the elastic process $`\gamma \gamma \gamma \gamma `$, the cut-off on the number of massive KK gravitons, $`n_c`$, has been introduced. We found that the recently measured $`\mathrm{\Delta }a_\mu `$ can be accommodated in the RS model, within the natural parameter space allowed by the perturbative unitarity. For example, dozens (hundreds) of the $`n_c`$ for $`\mathrm{\Lambda }_\pi =12`$ TeV (3 TeV) can explain the reported $`\mathrm{\Delta }a_\mu `$.

preprint:  hep-ph/0103127, YUMS-01-01, KIAS P01014

1. Introduction. As one of the most accurately measured quantities in particle physics, the anomalous magnetic moment of the muon, $`(g2)_\mu `$, has played an important role in providing constraints on various models for physics beyond the standard model (SM). The recent Brookhaven E821 results for the $`a_\mu (g2)_\mu /2`$ by a 2.6 standard-deviation from its SM prediction,

$$\mathrm{\Delta }a_\mu =a_\mu (\mathrm{exp})a_\mu (\mathrm{SM})=(43\pm 16)\times 10^{10},$$
(1)
therefore, is very likely to indicate that the SM should be extended just above the electroweak scale. A conservative view still exists such that the uncertainties in the SM calculations of the hadronic vacuum polarization may explain the deviation. The implications of the $`\mathrm{\Delta }a_\mu `$ on new physics have been extensively studied. Supersymmetric models are shown to accommodate this $`\mathrm{\Delta }a_\mu `$, as being consistent with all other relevant constraints. Interesting results are that the reported $`\mathrm{\Delta }a_\mu `$ value favors large $`\mathrm{tan}\beta `$ case and imposes observable upper bounds on the masses of supersymmetric particles. The effects from other new physics such as technicolor models, lepton flavor violation, composite models, leptoquark models, and others have been also studied.

Based on a theoretical motivation to explain the gauge hierarchy problem, a novel approach has been proposed by introducing extra dimensions: Arkani-Hamed, Dimopoulos and Dvali (ADD) proposed that the large volume of the extra dimensions with factorizable geometry can explain the observed largeness of Planck scale $`M_{\mathrm{Pl}}`$, whereas the fundamental string scale $`M_S`$ being maintained around TeV scale; based on two branes and a single extra dimension with non-factorizable geometry, Randall and Sundrum (RS) have proposed another high dimensional scenario, where the gauge hierarchy is attributed to a geometrical exponential factor. The phenomenologies of both extra-dimensional models have been extensively studied. In this paper, we examine if the RS model can explain the observed $`\mathrm{\Delta }a_\mu `$ deviation.

In the four-dimensional effective theory, both extra dimensional scenarios yield the Kaluza-Klein (KK) states of massive gravitons. The KK gravitons in the ADD case have almost continuous spectrum up to a cut-off scale $`M_S`$ with the Planck suppressed interaction; those in the RS case show discrete mass spectrum about TeV scale with the electroweak scale $`(\mathrm{\Lambda }_\pi )`$ interactions. In the ADD case, the calculation of the KK graviton contribution to the muon $`(g2)`$ has shown some interesting results such that a given KK graviton generates finite contribution to the $`\mathrm{\Delta }a_\mu `$; the universal coupling of each KK graviton leads to cancellation among logarithmic divergences of each Feynman diagram; the contribution of the KK graviton much heavier than the muon shows non-decoupling. The finiteness of the contributions of a graviton has been well known in supergravity models. In the ADD model, the summation of all the KK graviton contributions with the cut-off $`M_S`$ is compensated by the Planck scale suppression, which leaves finite contribution to the $`\mathrm{\Delta }a_\mu `$ proportional to $`(m_\mu /M_S)^2`$.

In the RS model, there exist an infinitely large number of KK graviton states in the view point of our four-dimensional world, whereas the interaction is suppressed by only the electroweak scale. A naive summation of the finite contributions shall generate infinite contribution to the $`\mathrm{\Delta }a_\mu `$. A cut-off, corresponding to the $`M_S`$ in the ADD case, is needed. We introduce $`n_c`$, the unitarity cut-off on the number of the KK graviton states. Then, we express the $`\mathrm{\Delta }a_\mu `$ in terms of $`n_c`$ and $`\mathrm{\Lambda }_\pi `$. In fact, the necessity of introducing a cut-off in the RS model has been already known in its phenomenological study: Unitarity violation happens at high energies. By examining the $`J`$-partial wave amplitudes of the elastic process $`\gamma \gamma \gamma \gamma `$, we shall derive the perturbative unitarity constraints on $`n_c`$ and $`\sqrt{s}/\mathrm{\Lambda }_\pi `$. It is to be shown that the recently measured $`\mathrm{\Delta }a_\mu `$ can be accommodated in the RS model, with the natural values of $`n_c`$ and $`\mathrm{\Lambda }_\pi `$ allowed by the unitarity constraint.

2. Randall-Sundrum Model and $`\mathrm{\Delta }a_\mu `$. For the hierarchy problem, Randall and Sundrum have proposed a five-dimensional non-factorizable geometry with the extra dimension compactified on a $`S_1/Z_2`$ orbifold of radius $`r_c`$. Reconciled with four-dimensional Poincar$`\stackrel{´}{\mathrm{e}}`$ invariance, the RS configuration has the following solution to Einstein’s equations:

$$ds^2=e^{2kr_c|\phi |}\eta _{\mu \nu }dx^\mu dx^\nu r_c^2d\phi ^2,$$
(2)
where $`0|\phi |\pi `$, and $`k`$ is $`AdS_5`$ curvature. Two orbifold fixed points accommodate two three-branes, the hidden brane at $`\phi =0`$ and our visible brane at $`|\phi |=\pi `$. The arrangement of our brane at $`|\phi |=\pi `$ renders a fundamental scale $`m_0`$ to appear as the four-dimensional physical mass $`m=e^{kr_c\pi }m_0`$. The hierarchy problem can be answered if $`kr_c12`$. From the four-dimensional effective action, the relation between the four-dimensional Planck scale $`M_{\mathrm{Pl}}`$ and the fundamental string scale $`M_5`$ is obtained by $`M_{\mathrm{Pl}}^2=M_5^3(1e^{2kr_c\pi })/k`$.

The compactification of the fifth dimension leads to the following four-dimensional effective Lagrangian ,

$$=\frac{1}{M_{\mathrm{Pl}}}T^{\mu \nu }h_{\mu \nu }^{(0)}\frac{1}{\mathrm{\Lambda }_\pi }T^{\mu \nu }\underset{n=1}{\overset{\mathrm{}}{}}h_{\mu \nu }^{(n)},$$
(3)
where $`\mathrm{\Lambda }_\pi e^{kr_c\pi }M_{\mathrm{Pl}}`$. The coupling of the zero mode KK graviton is suppressed by usual Planck scale $`M_{\mathrm{Pl}}`$, while those of the massive KK gravitons by the electroweak scale $`\mathrm{\Lambda }_\pi `$. The masses of the KK gravitons are also at electroweak scale,

$$m_n=x_n(k/M_{\mathrm{Pl}})\mathrm{\Lambda }_\pi ,$$

where $`x_n`$ is the $`n`$-th root of the first order Bessel function. And the total decay width of the $`n`$-th KK graviton is $`\mathrm{\Gamma }_n=\rho m_nx_n^2(k/M_{\mathrm{Pl}})^2`$, where the $`\rho `$, fixed to be one, is a model-dependent parameter which is different according to the decay pattern of the KK graviton. The condition $`k<M_{\mathrm{Pl}}`$ is to be imposed to maintain the reliability of the RS solution in Eq. (2).

The Lagrangian in Eq. (3) then makes new contributions to the $`\mathrm{\Delta }a_\mu `$. The relevant Feynman diagrams mediated by the KK gravitons are presented in Fig. 1. We refer readers to Ref.  for the corresponding Feynman rules of the graviton, photon and fermion. Note that the gravitational Ward identity renders almost negligible, to leading order, the contributions of the terms containing two or more $`k_\alpha `$’s in the massive graviton propagator. Here the $`k_\alpha `$ denotes the momentum of the graviton. A given $`n`$-th KK graviton state generates the $`\mathrm{\Delta }a_\mu `$, which is parameterized by

$$\mathrm{\Delta }a_\mu ^{(n)}=\frac{1}{16\pi ^2}\left(\frac{m_\mu }{\mathrm{\Lambda }_\pi }\right)^2\underset{f}{}a_f^G=\frac{5}{16\pi ^2}\left(\frac{m_\mu }{\mathrm{\Lambda }_\pi }\right)^2,$$
(4)
where the $`f`$ runs over five Feynman diagrams and the second equality holds for the case of $`m_nm_\mu `$. As discussed in the previous section, naive summation over all the KK states would yield infinite contribution, i.e., $`\mathrm{\Delta }a_\mu _n^{\mathrm{}}\mathrm{\Delta }a_\mu ^{(n)}\mathrm{}`$. A cut-off is needed.

This requirement of a cut-off is expected since the RS model described by Eq. (3) is an effective theory. According to the analysis of the process $`e^+e^{}\mu ^+\mu ^{}`$ with the KK graviton effects, in the large $`k/M_{\mathrm{Pl}}`$ case the partial wave unitarity seems to be violated even at TeV scale as the resonant peaks of the KK gravitons become very wide and the cross section is greatly enhanced . Since in the small $`k/M_{\mathrm{Pl}}<0.3`$ case the unitarity appears preserved at several TeV scales, possible unitarity violation of the RS model at high energy colliders has not been considered seriously yet. However, when we consider the loop effects by using an effective theory (the RS model) with an infinitely large number of states, we should determine the cut-off scale below which the perturbative calculations are reliable, and add only the contributions up to the cut-off scale. We introduce the cut-off on the effective number of KK graviton states, $`n_c`$. Then, the total RS contribution to the $`\mathrm{\Delta }a_\mu `$ becomes

$$\mathrm{\Delta }a_\mu ^{RS}=\frac{5}{16\pi ^2}\left(\frac{m_\mu }{\mathrm{\Lambda }_\pi }\right)^2n_c.$$
(5)
The scale of $`n_c`$ will be estimated by examining the partial wave unitary bound in the process of $`\gamma \gamma \gamma \gamma `$. Note that the parameters $`(\mathrm{\Lambda }_\pi ,k/M_{\mathrm{Pl}})`$ are constrained from the phenomenological studies of virtual exchange of the KK gravitons at LEP II and Tevatron Run I data; $`\mathrm{\Lambda }_\pi >1.1(0.65)`$ TeV for the $`k/M_{\mathrm{Pl}}=0.1(0.3)`$.

In Fig. 2, we plot the $`\mathrm{\Delta }a_\mu ^{RS}`$ as a function of $`n_c`$ for $`\mathrm{\Lambda }_\pi =`$1, 2, 3 TeV. It can be seen that the RS model can explain the recently reported $`\mathrm{\Delta }a_\mu `$ from the SM prediction with $`n_c10`$, 50, and 150, for $`\mathrm{\Lambda }_\pi =1`$, 2, and 3 TeV, respectively. As shall be shown in the next section, the scale of $`n_c10100`$ is natural in the sense that the partial wave unitarity is well preserved in this range. The scale of the heaviest KK graviton mass, $`m_{n_c}`$, then becomes of order 10 TeV, which can be regarded as a naive cut-off scale of the four-dimensional effective Lagrangian in the RS scenario. In the next section, we will show that this is consistent with the cut-off scale for the perturbative unitarity.

Some discussions on the other RS effects rather than the KK gravitons are in order here. First in the RS scenario where the modulus field (called the radion) is inherent, its stabilization is needed for the consistency in a cosmological context . According to the Goldberger and Wise stabilization mechanism, this radion can be an order of magnitude lighter than the $`\mathrm{\Lambda }_\pi `$ . The radion contributions to the $`\mathrm{\Delta }a_\mu `$, however, have been too small to explain alone the $`\mathrm{\Delta }a_\mu `$ of $`4\times 10^9`$. The effects of an extended RS model, where the SM gauge and fermion fields, are both in the bulk have been also examined. The dominant contributions from bulk gauge bosons, bulk fermions, and Higgs bosons, however, generate $`negative`$ contribution to the $`\mathrm{\Delta }a_\mu `$, which is difficult to be compatible with the recently reported positive deviation in $`a_\mu `$. Moreover, the graviton loop contributions are not well-defined since the cancellation among logarithmic divergences does not occur due to non-universal couplings between the KK gravitons and the bulk SM fields.

3. Unitary Bound from $`\gamma \gamma \gamma \gamma `$ Scattering. In order to see whether the $`n_c10100`$ can naturally explain the $`\mathrm{\Delta }a_\mu `$ in the effective theory interpretation, we consider another process to estimate the scale of $`n_c`$. The elastic process $`\gamma \gamma \gamma \gamma `$ is to be examined, focused on the unitarity bounds of the RS model: It has some merits such that the RS effects mediated by KK gravitons are dominant due to the absence of the SM contributions at tree level and thus the unitarity bounds of the RS model can be more sensitively probed compared to other processes. The requirement of partial wave unitarity shall constrain $`\sqrt{s}/\mathrm{\Lambda }_\pi `$ and the number of the KK graviton states, $`n_c`$. The $`J`$-partial wave amplitude is defined  by

$$a_{\mu \mu ^{}}^J=\frac{1}{64\pi }_1^1d\mathrm{cos}\theta d_{\mu \mu ^{}}^J(\mathrm{cos}\theta )\left[i_{\lambda _1\lambda _2\lambda _3\lambda _4}\right],$$
(6)
where the $`_{\lambda _1\lambda _2\lambda _3\lambda _4}`$ is the helicity amplitudes, $`\mu =\lambda _1\lambda _2`$, $`\mu ^{}=\lambda _3\lambda _4`$, and the $`d_{\mu \mu ^{}}^J`$ is the Wigner functions. Unitarity implies that the largest eigenvalue ($`\chi `$) of $`a_{\mu \mu ^{}}^J`$ is to be $`|\chi |1`$. The reliability of perturbative calculations is approximately guaranteed by the conditions $`|\chi |=1`$ and $`|\mathrm{Re}(\chi )|=1/2`$. The helicity amplitudes, of which the dominant contribution at high energies comes from the KK gravitons, are

$`_{++++}`$ $`=`$ $`_{}=i{\displaystyle \frac{s^2}{\mathrm{\Lambda }_\pi ^2}}{\displaystyle \underset{n=1}{\overset{n_c}{}}}\left[D_n(t)+D_n(u)\right],`$ (7)
$`_{++}`$ $`=`$ $`_{++}=i{\displaystyle \frac{u^2}{\mathrm{\Lambda }_\pi ^2}}{\displaystyle \underset{n=1}{\overset{n_c}{}}}\left[D_n(s)+D_n(t)\right],`$ (8)
$`_{++}`$ $`=`$ $`_{++}=i{\displaystyle \frac{t^2}{\mathrm{\Lambda }_\pi ^2}}{\displaystyle \underset{n=1}{\overset{n_c}{}}}\left[D_n(s)+D_n(u)\right],`$ (9)
where $`D_n(s)=1/(sm_n^2+im_n\mathrm{\Gamma }_n)`$ with $`m_n`$ being the mass of the $`n`$-th KK graviton state. Note that two parameters $`(\sqrt{s}/\mathrm{\Lambda }_\pi ,n_c)`$ with a given $`k/M_{\mathrm{Pl}}`$ determine the helicity amplitudes, and thus $`\chi `$. The odd $`J`$-partial wave amplitudes vanish due to Bose-Einstein statistics in the elastic $`\gamma \gamma `$ scattering. And we have $`a_{22}^2=a_{22}^2`$ and $`a_{22}^2=a_{22}^2`$ from the parity arguments. The non-vanishing eigenvalues $`\chi _i`$ are $`a_{00}^2`$ and $`2a_{22}^2`$.

In two limiting cases where $`\sqrt{s}/\mathrm{\Lambda }_\pi (k/M_{\mathrm{Pl}})x_n`$ and $`\sqrt{s}/\mathrm{\Lambda }_\pi (k/M_{\mathrm{Pl}})x_n`$, the two non-vanishing eigenvalues show converging behaviors. When $`a_{\mu \mu ^{}}^{J(n)}`$ denotes the contribution of the $`n`$-th KK state to $`a_{\mu \mu ^{}}^J`$, the $`a_{00}^{2(n)}`$ is approximated as

$`a_{00}^{2(n)}`$ $``$ $`{\displaystyle \frac{1}{384\pi }}{\displaystyle \frac{s^4}{\mathrm{\Lambda }_\pi ^8}}\left({\displaystyle \frac{k}{M_{\mathrm{Pl}}}}\right)^6{\displaystyle \frac{1}{x_n^6}},\text{ for }\sqrt{s}/\mathrm{\Lambda }_\pi (k/M_{\mathrm{Pl}})x_n,`$ (10)
$`a_{00}^{2(n)}`$ $``$ $`{\displaystyle \frac{1}{16\pi }}{\displaystyle \frac{s}{\mathrm{\Lambda }_\pi ^2}}\left[\mathrm{log}\left\{2{\displaystyle \frac{k^2}{M_{\mathrm{Pl}}^2}}{\displaystyle \frac{\mathrm{\Lambda }_\pi ^2}{s}}x_n^2\right\}+3\right],\text{ for }\sqrt{s}/\mathrm{\Lambda }_\pi (k/M_{\mathrm{Pl}})x_n.`$ (11)
There exist finite unitarity bounds $`n_c<1000`$ at small $`\sqrt{s}/\mathrm{\Lambda }_\pi `$, depending on the value of the parameter $`k/M_{\mathrm{Pl}}`$. Similarly, we calculate the approximate formula for $`a_{22}^{2(n)}`$ in two limiting cases:

$`a_{22}^{2(n)}`$ $``$ $`{\displaystyle \frac{1}{160\pi }}{\displaystyle \frac{s^2}{\mathrm{\Lambda }_\pi ^4}}{\displaystyle \frac{M_{Pl}^2}{k^2}}{\displaystyle \frac{1}{x_n^2}},\text{ for }\sqrt{s}/\mathrm{\Lambda }_\pi (k/M_{\mathrm{Pl}})x_n,`$ (12)
$`a_{22}^{2(n)}`$ $``$ $`{\displaystyle \frac{1}{64\pi }}{\displaystyle \frac{s}{\mathrm{\Lambda }_\pi ^2}}\left[{\displaystyle \frac{1}{16}}\mathrm{log}\left\{2{\displaystyle \frac{k^2}{M_{\mathrm{Pl}}^2}}{\displaystyle \frac{\mathrm{\Lambda }_\pi ^2}{s}}x_n^2\right\}+{\displaystyle \frac{2}{5}}\right],\text{ for }\sqrt{s}/\mathrm{\Lambda }_\pi (k/M_{\mathrm{Pl}})x_n.`$ (13)
The unitary bounds $`n_c`$ are around $`n_c10100`$. The eigenvalue $`2a_{22}^2`$ generates stronger constraints compared to that of $`a_{00}^2`$.

Figures 3 and 4 exhibit the exact numerical calculations of the unitarity bounds on $`(\sqrt{s}/\mathrm{\Lambda }_\pi ,n_c)`$ plane from the $`\chi =a_{00}^2`$ and $`\chi =2a_{22}^2`$ constraints, respectively. We consider three values for $`k/M_{\mathrm{Pl}}(=`$ 0.1, 0.3, 0.7). Note that the upper bounds on $`\sqrt{s}/\mathrm{\Lambda }_\pi `$ constrained from the perturbative unitarity are almost the same in the range of $`n_c10100`$ (such that $`\sqrt{s}<4\mathrm{\Lambda }_\pi `$, $`6.5\mathrm{\Lambda }_\pi `$, and $`11\mathrm{\Lambda }_\pi `$ for $`k/M_{\mathrm{Pl}}=0.1`$, 0.3, and 0.7 cases, respectively). When we regard this upper bounds on $`\sqrt{s}`$ as the cut-off scale of the effective RS model, we can see that this cut-off scale is compatible with the scale of $`m_{n_c}`$. Thus, the cut-off on the number of KK graviton states $`(n_c)`$ which explains the reported $`\mathrm{\Delta }a_\mu `$ is within the allowed region by the perturbative unitarity bounds.

4. Summary. To summarize, we have studied the effects of the Kaluza-Klein gravitons in the Randall-Sundrum model on the recent BNL measurements of the muon $`(g2)`$ deviation from the SM prediction. It is known that the individual contribution of a heavy KK graviton to $`\mathrm{\Delta }a_\mu `$ is finite. Since the four-dimensional effective theory in the RS model contains an infinitely large number of massive KK gravitons with the TeV suppressed couplings, a naive summation would yield infinite $`\mathrm{\Delta }a_\mu `$. A cut-off on the number of the KK gravitons has been introduced. Then the reported $`\mathrm{\Delta }a_\mu `$ can be attributed to the RS effects with dozens (hundreds) of the $`n_c`$ for $`\mathrm{\Lambda }_\pi =12`$ TeV (3 TeV). By examining the $`J`$-partial wave unitarity bounds in the elastic process $`\gamma \gamma \gamma \gamma `$, we have shown that this range of $`n_c`$ is natural in the effective theory interpretation and the scale of the heaviest graviton mass $`(m_{n_c})`$ is compatible with the cut-off scale for the perturbative unitarity. Therefore, we conclude that the recently measured $`\mathrm{\Delta }a_\mu `$ can be accommodated in the RS model.

Acknowledgments. The work of C.S.K. was supported in part by BK21 Program, SRC Program and Grant No. 2000-1-11100-003-1 of the KOSEF, and in part by the KRF Grants, Project No. 2000-015-DP0077. The work of J.D.K was supported by the Korea Research Foundation Grants (KRF-2000-DA0013 and 2000-015-DP0077).


text:
# The one-electron Green’s function of the half-filled Hubbard model on a triangular lattice

## I Introduction

As is well-known, the frustration effects associated with the triangular lattice often lead to non-trivial ground-state degeneracies as in the antiferromagnetic (AF) spin 1/2 Ising model . The classical Heisemberg model on the two-dimensional (2D) triangular lattice with nearest-neighbor AF coupling and easy-axis exchange anisotropy is another example where frustration leads to a novel ground-state degeneracy. This system has attracted much attention especially since Anderson suggested the possibility of a resonating-valence-bond ground state for the spin one-half case. Simply, the quantum liquid of radomly distributed spin-singlet pairs could be an efficient way to overcome the frustration of the Néel state in the triangular antiferromagnet. One more important source of interest in these lattices is the well-known diversity and richness of physical phenomena displayed by most transition metal compounds .

The experimental observation of low-temperature insulating phases in some $`\sqrt{3}`$-adlayer structures on (111) Si and Ge surfaces has only fostered the interest in these 2D triangular lattices. Thus, whereas the $`\sqrt{3}`$-overlayers of Sn and Pb on Ge(111) are both metallic at high temperature, their corresponding low-temperature counterparts are either metallic, as in the case of Sn, or weakly insulating as in the case of Pb . This latter system seems to go through some kind of reversible metallic to insulating transition whose precise nature is still controversial . A charge-density wave (CDW) has been invoked in the case of Pb (but not in the case of Sn) as the driving force for the destabilization of the high-temperature phase, a conjecture not universally accepted at the time of writing. Related isoelectronic systems, on the other hand, like the $`\sqrt{3}\times \sqrt{3}`$ adlayer of Si on SiC(0001) , or of K on Si(111):B show a clear insulating behavior with a large gap and no phase transitions. These systems have been studied theoretically both within the local-density (LDA) and the Hartree-Fock approximations. Quite recently the LDA+U approach has been used to include strong on-site repulsions in SiC(0001).

In this paper we carry out a model study of the triangular lattice in order to explore some general questions any realistic theory should comply with. For instance understanding the temperature behavior of the one-electron density of states (DOS) is essential to the development of a complete picture of these metal-insulator transitions. Hence we report the results of a Quantum Monte Carlo (QMC) simulation of the half-filled Hubbard model on such a triangular lattice in the grand canonical ensemble. The one-electron Green’s function is studied as a function of both temperature and coupling constant (Hubbard $`U`$). As the temperature is lowered, a pseudogap develops in the one-electron DOS for intermediate values of $`U`$. This pseudogap is accompanied by two weak peaks in the spin structure factor which signal the formation of a spiral spin-density-wave (SDW). For lower $`U`$, no gap at all is found even for low temperatures, the system remaining always paramagnetic. For higher $`U`$, on the other hand, a well developed gap appears at any temperature, accompanied by a strong peak in the spin structure factor. The system is then brought into a state very similar to the ground state of the triangular antiferromagnet (the three-sublattice model). We emphasize that these are not distinct phases, but only different regimes with smooth transitions among them, as characterized by the behavior of the one-electron DOS. Since the presence or absence of a gap or pseudogap is of fundamental importance in determining the properties of a system, we believe that this type of characterization is useful and can be of help in understanding the electronic properties of the more complex adsorption systems referred to above.

The paper is organized as follows: Section II recalls some of the basic properties of the triangular lattice as well as the Hubbard model, in order to fix the notation. Sec III summarizes very briefly the Quantum Monte Carlo algorithm in the grand canonical ensemble. The resulting one-electron DOS and spin structure factor are displayed and discussed in Sec IV and, finally, the paper closes with some concluding remarks in Sec V

## II Basic Properties of the Triangular Lattice and Hamiltonian

In the $`\sqrt{3}\times \sqrt{3}`$ R30o adlayer structures of Sn or Pb on Ge(111) at one third coverage, each adsorbate sits on top of a triangle of Ge atoms. With just one unpaired electron per adsorbate, the overlayer is half-filled and, therefore, metallic in the absence of electron-electron interactions. This overlayer can in turn be described as a $`3\times 3`$ lattice of adsorbate triangles (the three-sublattice model) with three unpaired electrons per triangle and, therefore, again metallic in the absence of interaction. The corresponding surface Brillouin zones (SBZ) are the large and small hexagon, respectively, in Fig 1. The adlayer $`\sqrt{3}\times \sqrt{3}`$ lattice has just one band in the large zone, $`\epsilon _k^0`$, which folds onto three bands in the small zone. Fig 2 shows these three bands, $`\epsilon _k^0,`$ $`\epsilon _k^+,`$ and $`\epsilon _k^{},`$ unfolded in the extended zone scheme in order to see the nesting properties. They are given by $`(t`$=hopping strength)

$$\epsilon _k^0=2t\mathrm{cos}kx+4t\mathrm{cos}\frac{1}{2}k_x\mathrm{cos}\frac{\sqrt{3}}{2}k_y$$
(1)
$$ϵ_k^\pm =\frac{1}{2}\epsilon _k^0\pm t\sqrt{3}\left(\mathrm{sin}k_x2\mathrm{sin}\frac{1}{2}k_x\mathrm{cos}\frac{\sqrt{3}}{2}k_y\right)$$
(2)
It is easy to see that $`ϵ_k^\pm `$ are just $`ϵ_k^0`$ for $`k=(k_x\pm \frac{2\pi }{3},k_y)`$. These bands cross at the points $`M`$’ and $`K`$’. The wavevector $`K`$=($`4\pi /3,0`$) turns out to be a nesting vector with the band folding around the $`M`$’=0.5 $`K`$ point ($`M`$$`KM`$$`\mathrm{\Gamma },`$ band $`\epsilon _k^+)`$. Likewise $`KM\mathrm{\Gamma }M`$’ (band $`\epsilon _k^{},)`$. Fig 3, finally, shows the resulting band along the small SBZ contour. Notice that the $`M`$$`K`$’ direction is obtained by folding the $`MK`$’ portion of $`\epsilon _k^0.`$

This is of no consequence for the interaction-free system at half filling since the Fermi surface is not anywhere close enough to either the large or the small SBZ boundaries. When the interaction is turned on, however, the nesting symmetry may come into play, although weakly, at the points $`M`$ and $`M`$’, closest to the Fermi surface. We shall see that, even at half-filling, this is indeed the case for the spin structure factor when $`U`$ is large enough.

In order to describe the interacting system, we adopt the Hubbard model, given by the standard Hamiltonian

$$H=t\underset{<ij>s}{}c_{is}^+c_{js}\mu \underset{is}{}n_{is}+U\underset{i}{}\left((n_i\frac{1}{2})(n_i\frac{1}{2})\right)$$
(3)
where $`t`$ is the hopping strength, $`U`$ the on-site repulsion and $`\mu `$ the chemical potential. The single sums run over all the $`N\times N`$ adlayer atoms and the symbol $`<`$$`>`$means summation over nearest neighbors ($`nn`$). As usual, $`c_{is}^+`$ creates, while $`c_{is}`$ destroys, an electron of spin $`s`$ at site $`i`$ with occupation number $`n_{is}=`$ $`c_{is}^+c_{js}.`$ We take $`t`$=0.055 eV so as to start with a narrow adlayer bandwidth ($`W`$=$`9t)`$ of around 0.5 eV at $`U`$=0. U is varied to cover different regimes of the triangular lattice and $`\mu `$ is adjusted so as to have always half filling. Recall that, unlike the case of bipartite lattices, $`\mu =U/2`$ does not necessarily correspond to half-filling since particle-hole symmetry does not hold in a triangular lattice.

This Hubbard model is now simulated by the QMC approach in the grand canonical ensemble as initially developed by Blankenbecler et al and supplemented by a discrete lattice version of the Hubbard-Stratonovich transformation by Hirsch. The whole approach has been explained at length by Hirsch and White et al and is briefly summarized in the following section.

## III Quantum Monte Carlo Algorithm

In a grand canonical simulation the imaginary time is discretized through the introduction of $`L`$ time slices separated by an interval $`\mathrm{\Delta }\tau `$ such that $`\beta `$=$`\mathrm{\Delta }\tau L`$. The partition function is then written as

$$Z=Tre^{\beta H}=Tre^{\mathrm{\Delta }\tau LH}Tr(e^{\mathrm{\Delta }\tau \widehat{K}}e^{\mathrm{\Delta }\tau \widehat{V}})^L,$$
(4)
where

$$\widehat{K}=t\underset{<ij>}{}c_{is}^+c_{js}\mu \underset{is}{}n_{is}=\underset{<ijs>}{}K_{ij}c_{is}^+c_{js}$$
(5)
and

$$\widehat{V}=U\underset{i}{}\left(n_{}\frac{1}{2}\right)\left(n_{}\frac{1}{2}\right).$$
(6)
The last step in Eq(4) follows from the Trotter formula which introduces systematic errors in the measured quantities of order $`\mathrm{\Delta }\tau ^2`$. One should therefore like to take $`\mathrm{\Delta }\tau `$ as small as possible, although keeping the number of time slices not too large. This poses a serious limitation at low temperatures (large $`\beta ).`$

A discrete Hubbard-Stratonovich (HS) transformation is now performed for each on-site interaction term within any time slice.

$$e^{\mathrm{\Delta }\tau U\left(n_i\frac{1}{2}\right)\left(n_i\frac{1}{2}\right)}=\frac{1}{2}e^{\mathrm{\Delta }\tau U/4}Tr_\sigma e^{\mathrm{\Delta }\tau \lambda \sigma _{il}(n_in_i)}$$
(7)
where a discrete, auxiliary Ising field $`\sigma _{il}=\pm 1`$ ($`l`$ runs through the $`L`$ time slices) has been introduced and $`\mathrm{cosh}\lambda \mathrm{\Delta }\tau =e^{U\mathrm{\Delta }\tau /2}`$. $`Tr_\sigma `$ means summation over the $`NL`$ Ising ”spins” $`\sigma _{il},`$ which define the auxiliary Ising field through the lattice sites and time slices. Defining now $`N\times N`$ matrices $`V(l)`$ such that

$$V_{ij}(l)=\frac{1}{2}\delta _{ij}\lambda s_{ij}\text{,}$$
(8)
we can write

$$Z=\left(\frac{1}{2}e^{\mathrm{\Delta }\tau U/4}\right)^{NL}Tr_\sigma Tr\mathrm{\Pi }_{s,l}D_l^s,$$
(9)
where

$$D_l^s=e^{\mathrm{\Delta }\tau _{ijs}K_{ij}c_{is}^+c_{js}}e^{\mathrm{\Delta }\tau _{is}V_{ii}(l)\alpha _sn_{is}}$$
(10)
where $`\alpha _s`$=$`\pm 1`$ for $`s`$=$``$ respectively. Notice carefully the difference between $`\sigma `$, Ising spin, and $`s`$, fermion spin.

The fermion degrees of freedom can be traced over to yield

$$Z=Tr_\sigma \mathrm{\Pi }_sdet0_s\left[\sigma \right]$$
(11)
where $`0_s\left[\sigma \right],`$ the fermion matrix, is given by

$$0_s=I+B_l^sB_{l1}^s\mathrm{}B_1^s$$
(12)
with

$$B_l^s=e^{\mathrm{\Delta }\tau K}e^{\mathrm{\Delta }\tau \alpha _sV(l)}$$
(13)
and $`I`$ being the unity matrix. Therefore $`\mathrm{\Pi }_sdet0_s`$ plays the role of a Botzmann weight for evaluating the average of any operator, $`<A>`$=$`TrAe^{\beta H}/Tre^{\beta H}`$. The above steps must now be repeated for $`TrAe^{\beta H}`$ .Thus for $`A`$=$`c_{is}c_{js}^+`$, the resulting expression is

$$Tr\left(c_{is}c_{js}^+e^{\beta H}\right)=Tr_\sigma \left(0_s^1\right)_{ij}\mathrm{\Pi }_sdet0_s\left[\sigma \right]$$
(14)
and hence an equal-time single particle Green’s function $`<c_{is}c_{js}^+>`$ is obtained by averaging the corresponding matrix elements of the inverse of the fermion matrix $`0_s`$. Time dependent Green’s functions can also be calculated and involve slightly more complicated matrices. For any HS field configuration, Wick’s theorem for operator products applies and one can therefore easily calculate time-dependent correlation functions in terms of single-particle Green’s functions for any given Ising configuration. These imaginary-time quantities are then to be analytically continued to real time and frequencies via the maximum-entropy method which yields the corresponding spectral weight functions.

The heat-bath algorithm is used to sample the HS field subject to the Boltzmann weight given above. This sampling is normally accomplished by single spin-flips $`\sigma _{il}`$-$`\sigma _{il}`$, the corresponding weight-ratio being related to equal-time Green’s function. If the new configuration is accepted, the corresponding Green’s functions is updated through simple operations. Calculating the weight ratio requires $`N^2`$ operations, and therefore $`N^3L`$ operations must be done for a sweep through the whole Hs field. Normally two hundred warm-up sweeps and one thousand measurements separated by two sweeps were performed for each set of parameters. At low temperatures the algorithm becomes unstable due to the large number of time slices required in order to make $`\mathrm{\Delta }\tau `$ small enough. The product of B matrices in Eq(12) is ill-conditioned and one has to resort to matrix factorization techniques.

## IV The Three Coupling Regimes

According to the Mermin-Wagner theorem, infinite-range magnetic order is forbidden in two dimensions at any $`T0`$. This is so because the Goldstone modes strongly disorder the system giving rise to a spin-spin correlation length which decays with temperature as $`\xi (T)\mathrm{exp}(A/T),`$ where $`A`$ is a constant. Thus no phase transitions of magnetic origin can take place in an infinite system except, perhaps, at $`T`$=$`0`$. Other kinds of phase transitions are outside the scope of this theorem. Such is the case, e.g., of the (Mott) paramagnetic metal - paramagnetic insulator transition. Let us specialize to the case of the triangular lattice.

### A The one-electron DOS

The one-electron DOS is given by

$$N(\omega )=\frac{1}{N}\underset{k}{}A(k,\omega )$$
(15)
where $`N`$ is the number of lattice sites and $`A(k,\omega )`$, the spectral-weight function, is the imaginary part of the retarded one-electron Green’s function.

Mean-field studies at $`T`$=$`0`$ have shown that the half-filled triangular lattice is a paramagnetic metal in the weak-coupling regime, in contrast with the SDW insulating character of the square lattice for small $`U/t`$. No gap in the one-electron DOS is, therefore, expected at any temperature for an $`infinite`$ triangular lattice. It has been shown, however, that size effects are very strong in this regime . A gap in the one-electron DOS develops as soon as the spin-spin correlations extend over the whole system. Thus, for lattices of increasing size $`N\times N`$, the system evolves from a situation where the correlation length $`\xi (T)>N`$, (with a gap) to one where $`\xi (T)<`$ $`N`$ (without a gap). One should be careful when drawing conclusions about the existence of gaps from small lattices.

Fig 4 shows the one-electron DOS of a half-filled $`4\times 4`$ triangular lattice with periodic boundary conditions. We have taken $`U/t`$ =$`5`$ ($``$ half the bandwidth, $`9t`$), which is a weak to moderate value, and several values of $`\beta `$, $`\beta t`$=$`5`$, $`10`$, $`15`$ and $`20.`$ Even for $`\beta t`$ as high as $`20`$, the system is far from having a fully developed gap. Since for the bigger lattices one expects weaker pseudogaps, it may be safely concluded that in the weak coupling regime a triangular lattice has no gaps at any temperature, in accordance with the Mermin-Wagner theorem.

In the strong-coupling regime at $`T`$=$`0`$ the system is brought into a commensurate, three-sublattice, $`120^o`$ twist SDW state (similar to the ground-state of the classical antiferromagnet) which is insulating and stable for increasing $`U`$. Quantum fluctuations about the classical antiferromagnetic solution lead to the essential qualitative physics of the Mott-Hubbard insulator at finite temperatures with a charge gap of order $`U`$ in the spectral-weight function. Fig 5 bears the same information as Fig 4, but with $`U/t`$=$`20`$ which is deep inside the strong-coupling regime. Since size effects are very small in this regime, it is fairly clear that a fully developed gap is present at any temperature.

We thus see that, for a given temperature, the system evolves from a gapless situation at small $`U`$ to a fully developed gap at large $`U`$. As $`U`$ increases through the intermediate-coupling regime, one should find a critical value $`U_c(T)`$ for which the gap first appears. Fig 6 displays, as Fig 4 and 5, the one-electron DOS for an intermediate value of $`U/t`$=$`10`$ ($``$ the bandwidth). As the temperature is lowered from $`\beta t`$= $`5`$ down to $`\beta t`$= $`20`$, an incipient pseudogap gradually evolves into a fully developed gap. This value of $`U`$ is clearly below the critical $`U`$ for all $`\beta t`$$`<`$$`20`$, i.e., $`U_c`$ =$`10t`$ for $`\beta t`$= $`20`$. The complementary view is given in Fig 7, which shows the one-electron DOS for $`\beta t`$= $`5`$ and $`U/t`$=$`5,10,15`$, and $`20`$. We see the system evolving from a gapless regime to a pseudogap, a deep pseudogap and finally a fully developed gap. Thus $`U_c`$=$`20t`$ for $`\beta t`$= $`5.`$ In this way one generates a temperature-dependent critical value of the coupling constant $`U_c(T).`$

### B  The spin structure factor

The spin structure factor, $`s(k)`$, is given by the $`k`$-Fourier transform of the static spin-spin correlation function

$$s_{ij}=<\sigma _{iz}(\tau ^+)\sigma _{jz}(\tau )>_{\tau =0}$$
(16)
where $`\tau ^+`$=$`\tau +o^+`$ in the imaginary-time domain, and $`\sigma _{iz}`$=$`n_in_i`$.

The peaks of $`s(k)`$ and corresponding widths in $`k`$-space give information, as is well-known, about the SDW’s sustained by the system and the spin-spin correlation length. Fig 8 shows $`s(k)`$ for $`\beta t`$= $`20`$ and $`U/t`$=$`5,10`$,and $`20`$, representative values of the three regimes. The almost featureless shape for $`U/t`$=$`5`$ evolves into two sharp, although small, peaks close to $`M`$ and $`M`$’ in the intermediate regime and, finally into a large peak at $`M`$’ in the strong-coupling regime. The system, correspondingly, evolves from a paramagnetic metal, through an incommensurate spiral SDW, into a Mott-Hubbard insulator

## V Concluding Remarks

The variation of the one-electron DOS with both temperature and coupling constant seems a useful tool for the purpose of identifying the different regimes of a given system. For the special case of the half-filled repulsive Hubbard model on a triangular lattice, we have identified an intermediate, temperature dependent coupling regime which interpolates smoothly between the weak-coupling (paramagnetic metal) and the strong-coupling (Mott-Hubbard insulator) regimes. As the temperature is lowered in this intermediate-coupling regime, the system evolves from metallic to insulating.

We conclude with a comment on the $`\sqrt{3}\times \sqrt{3}`$ adlayer structures on group fourth semiconductor surfaces. Although a close connection with the above model study is not claimed, these structures may constitute a physical realization of the three coupling regimes just described, Sn/Ge, Pb/Ge, and SiC being examples of the weak, intermediate and strong coupling regimes, respectively. Despite the added complexity due to electron-phonon interactions and atomic relaxation of both adsorbate and substrate atoms, the model study carried out here provides a general framework for the study of those systems.

Acknowledgments. This work was supported by the DGICYT (Spain) Project NoPB98-0683.

## VI
Figure Captions

> Fig 1. Large (outer exagon) and small (inner exagon) surface Brillouin zones (SBZ) of the triangular lattice. Shown are the especial points $`\mathrm{\Gamma },M^{},K,M,`$ and $`K`$’ which delimit the contours $`\mathrm{\Gamma }KM\mathrm{\Gamma }`$and $`\mathrm{\Gamma }M^{}K^{}\mathrm{\Gamma }`$ used in the text.
>
> Fig 2. Band structure of the triangular lattice for $`U`$=0 in the three-sublattice model. The three bands, $`\epsilon _k^0`$ (main band) and $`\epsilon _k^\pm ,`$ are displayed along the $`\mathrm{\Gamma }KM\mathrm{\Gamma }`$ contour of the large SBZ in order to show the band crossings and nesting symmetry.
>
> Fig 3. Same as Fig 2, but along the $`\mathrm{\Gamma }M^{}K^{}\mathrm{\Gamma }`$ contour of the small SBZ.
>
> Fig 4. One-electron density of states (DOS) of the triangular lattice for $`U/t`$=$`5`$ (weak coupling) and decreasing temperature, $`\beta t`$=$`5,10,15`$, and $`20`$
>
> Fig 5. Same as Fig 4, but for $`U/t`$=$`20`$ (strong coupling).
>
> Fig 6. Same as Figs 4 and 5, but for $`U/t`$=$`10`$ (intermediate coupling).
>
> Fig 7. One-electron DOS of the triangular lattice for increasing $`U/t`$=$`5,10,15`$, and $`20`$ at a fixed temperature $`\beta t`$=$`5`$.
>
> Fig 8. Low-temperature ($`\beta t`$=$`20)`$ spin structure factor, $`s(k)`$, of the triangular lattice in the weak ($`U/t`$=$`5`$ ), intermediate ($`U/t`$=$`10)`$ and strong ($`U/t`$=$`20`$ ) coupling regimes