text:
# The slope of the black-hole mass versus velocity dispersion correlation

## 1 Introduction

Observations of the centers of nearby early-type galaxies (ellipticals, lenticulars, and spiral bulges) show that most or all contain massive dark objects (hereafter “black holes”). The masses of these objects are consistent with the density of quasar remnants expected from energy arguments (Sołtan, 1982; Fabian & Iwasawa, 1999; Yu & Tremaine, 2002). There appears to be a strong correlation between the mass $`M_{}`$ of the black hole and the velocity dispersion $`\sigma `$ of the host galaxy, of the form111All logarithms in this paper are base 10.

$$\mathrm{log}(M_{}/M_{})=\alpha +\beta \mathrm{log}(\sigma /\sigma _0),$$
(1)
where $`\sigma _0`$ is some reference value (here chosen to be $`\sigma _0=200\text{km s}^1`$). The first published estimates of the slope $`\beta `$, $`5.27\pm 0.40`$ (Ferrarese & Merritt, 2000a) and $`3.75\pm 0.3`$ (Gebhardt et al., 2000a), differed by 3 standard deviations. Subsequently, Ferrarese & Merritt (hereafter FM) revised their slope downwards, to $`4.8\pm 0.5`$ (Ferrarese & Merritt, 2000b), $`4.72\pm 0.36`$ (Merritt & Ferrarese, 2001a), $`4.65\pm 0.48`$ (Merritt & Ferrarese, 2001b), and then $`4.58\pm 0.52`$ (Ferrarese, 2002). Although the discrepancy between the estimate by Gebhardt et al. (hereafter the Nukers) and the estimates by FM has declined monotonically with time, and is now only 1.4 standard deviations, it is still worthwhile to understand the reasons behind it. In particular, the slope is the most important point of comparison to theoretical models that attempt to explain the $`M_{}`$$`\sigma `$ relation (Adams, Graff, & Richstone, 2001; Burkert & Silk, 2001; Haehnelt & Kauffmann, 2000; Ostriker, 2000).

This paper has three main goals. (i) In §§24 we explore the reasons for the wide range in estimated slopes of the $`M_{}`$$`\sigma `$ relation. In §2 we focus on the statistical techniques used to estimate slopes by the two groups; we shall argue that the estimator used by the Nukers is more accurate, but that the choice of estimator cannot explain most of the differences in slope between FM and the Nukers. In §3 we describe the data sets used by the two groups. In §4 we examine several explanations that have been proposed for the slope range, including the neglect of random measurement errors in the dispersions, the dispersion used for the Milky Way, and differences in sample selection, and show that none of these is viable. We argue instead that the slope range reflects systematic differences in the velocity dispersion measurements used by the two groups. (ii) In §5 we present a new analysis of the $`M_{}`$$`\sigma `$ relation using recent data. (iii) Finally, in the Appendix we model the velocity-dispersion profile of the Milky Way bulge, which helps to fix the low-mass end of the $`M_{}`$$`\sigma `$ relation.

## 2 The fitting algorithm

The data consist of $`N`$ galaxies with measured black-hole masses, velocity dispersions, and associated uncertainties. We assume that there is an underlying relation of the form

$$y=\alpha +\beta x,$$
(2)
where $`y=\mathrm{log}(M_{}/M_{})`$, $`x=\mathrm{log}(\sigma /\sigma _0)`$. We assume that the measurement errors are symmetric in $`x`$ and $`y`$ with root-mean-square (rms) values $`ϵ_{xi}`$ and $`ϵ_{yi}`$ for galaxy $`i`$. The goal is to estimate the best-fit values of $`\alpha `$ and $`\beta `$ and their associated uncertainties.

The Nukers and FM use two quite different estimators. In this section we review the assumptions inherent in the two estimators and their respective advantages and disadvantages. In subsequent sections we shall usually give results for both estimators; we shall find that the differences are significant, but not large enough to explain the slope range.

The Nukers’ estimate is based on minimizing

$$\chi ^2\underset{i=1}{\overset{N}{}}\frac{(y_i\alpha \beta x_i)^2}{ϵ_{yi}^2+\beta ^2ϵ_{xi}^2},$$
(3)
(e.g., Press et al. 1992, whose procedures we use). The “1$`\sigma `$” uncertainties in $`\alpha `$ and $`\beta `$ are given by the maximum range of $`\alpha `$ and $`\beta `$ for which $`\chi ^2\chi _{\mathrm{min}}^21`$. An attractive feature of this approach is that the variables $`x`$ and $`y`$ are treated symmetrically; in other words, if we set $`\stackrel{~}{\beta }=1/\beta `$, $`\stackrel{~}{\alpha }=\alpha /\beta `$, equation (3) can be rewritten in the form

$$\chi ^2\underset{i=1}{\overset{N}{}}\frac{(x_i\stackrel{~}{\alpha }\stackrel{~}{\beta }y_i)^2}{ϵ_{xi}^2+\stackrel{~}{\beta }^2ϵ_{yi}^2},$$
(4)
which has the same form as equation (3) if $`xy`$, $`\alpha \stackrel{~}{\alpha }`$, and $`\beta \stackrel{~}{\beta }`$. This symmetry ensures that we are not assuming (for example) that $`y`$ is the dependent variable and $`x`$ is the independent variable in the correlation; this agnosticism is important because we do not understand the physical mechanism that links black-hole mass to dispersion. We shall call estimators of this kind “$`\chi ^2`$ estimators” and denote them by $`\alpha _\chi `$, $`\beta _\chi `$.

One limitation is that this approach does not account for any intrinsic dispersion in the $`M_{}`$$`\sigma `$ relation (i.e., dispersion due to the galaxies themselves rather than to measurement errors). Thus, for example, one or two very precise measurements with small values of $`ϵ_{xi}`$ and $`ϵ_{yi}`$ can dominate $`\chi ^2`$, even though the large weight given to these observations is unrealistic if the intrinsic dispersion is larger than the measurement errors. There are two heuristic approaches that address this concern. (i) Simply set $`ϵ_{yi}ϵ_y=\text{constant}`$, corresponding to the same fractional uncertainty in all the black-hole mass estimates. The value of $`ϵ_y`$ is adjusted so that the value of $`\chi ^2`$ per degree of freedom is equal to its expectation value of unity. This approach was adopted by Gebhardt et al. (2000a). (ii) Replace $`ϵ_{yi}`$ by $`(ϵ_{yi}^2+ϵ_0^2)^{1/2}`$, where the unknown constant $`ϵ_0`$, which represents the intrinsic dispersion, is adjusted so that the value of $`\chi ^2`$ per degree of freedom is unity. The second procedure is preferable if and only if the individual error estimates $`ϵ_{yi}`$ are reliable. We shall use both approaches in §5.2.

FM use the estimator

$$\beta _{\mathrm{AB}}=\frac{\underset{i=1}{\overset{N}{}}(y_iy)(x_ix)}{_{i=1}^N(x_ix)^2_{i=1}^Nϵ_{xi}^2},\alpha _{\mathrm{AB}}=y\beta _{\mathrm{AB}}x;$$
(5)
here $`xN^1_{i=1}^Nx_i`$ and $`yN^1_{i=1}^Ny_i`$ are the sample means of the two variables. This estimator is described by Akritas & Bershady (1996), who also provide formulae for the uncertainties in $`\alpha `$ and $`\beta `$. The Akritas-Bershady (hereafter AB) estimator accounts for measurement uncertainties in both variables, and is asymptotically normal and consistent. When $`ϵ_{xi}=0`$ and $`ϵ_{yi}ϵ_y=\text{constant}`$, the AB and $`\chi ^2`$ estimators give the same estimates for $`\alpha `$ and $`\beta `$ (but not their uncertainties).

Despite its merits, the AB estimator has several unsettling properties. (i) The measurement errors in velocity dispersion, $`ϵ_{xi}`$, only enter equation (5) through the sum $`ϵ_{xi}^2`$. Thus, for example, a single low-precision measurement can dominate both $`ϵ_{xi}^2`$ and $`_{i=1}^N(x_ix)^2`$, rendering the estimator useless, no matter how many high-precision measurements are in the sample. (ii) The errors in the black-hole mass determinations $`ϵ_{yi}`$ do not enter equation (5) at all: all observations are given equal weight, even if some are known to be much less precise than others. (iii) We have argued above that the variables $`x`$ and $`y`$ should be treated symmetrically, but this is not the case in equation (5). (iv) Even if the variables $`x_i`$ are drawn from a Gaussian distribution, there will occasionally be samples for which the denominator of equation (5) is near zero. In this case the estimator $`\beta _{\mathrm{AB}}`$ will be very large. These occasional large excursions are frequent enough that the variance of $`\beta _{\mathrm{AB}}`$ in a population of galaxy samples is infinite, no matter how large the number $`N`$ of data points may be. (v) Figure 1 shows the distribution of estimates of $`\beta _\chi `$ (solid line) and $`\beta _{\mathrm{AB}}`$ (dashed line) obtained from 100,000 Monte Carlo trials drawn from a population that has $`\beta =4.5`$ and other parameters similar to the sample FM1 defined below (for details see figure caption). The distribution of $`\beta _\chi `$ is substantially narrower than $`\beta _{\mathrm{AB}}`$ (note that values of either estimator outside the range of the histogram are plotted in the outermost bins). The estimator $`\beta _\chi `$ has a mean of $`4.52`$ and a standard deviation of $`0.36`$. The distribution of $`\beta _{\mathrm{AB}}`$ has a mean of $`4.69`$, and as stated above the standard deviation of this mean is infinite. Thus, in this example at least, $`\beta _{\mathrm{AB}}`$ is both biased and inefficient.

In this paper, we sometimes use a third fitting procedure, which is closely related to principal component analysis (Kendall, Stuart, & Ord, 1983). Suppose that the intrinsic distribution of $`x`$ and $`y`$ (the distribution that would be observed in the absence of measurement errors) is a biaxial Gaussian, with major and minor axes having standard deviations $`\sigma _a`$ and $`\sigma _b`$, respectively, and the major axis having slope $`\beta \mathrm{tan}\theta `$. If $`\sigma _b`$ were zero, all of the points would lie exactly on a line of slope $`\beta `$; thus $`\sigma _b`$ characterizes the intrinsic dispersion in the correlation between $`x`$ and $`y`$. Let us also assume that the measurement errors are Gaussian, with standard deviations $`ϵ_x`$ and $`ϵ_y`$ that are the same for all galaxies. The observed distribution of $`x`$ and $`y`$, which is obtained by convolving the intrinsic Gaussian with the measurement errors, is still Gaussian. The shape of this Gaussian is fully described by the three independent components of the symmetric $`2\times 2`$ dispersion tensor

$$\sigma _{ij}(x_ix_i)(x_jx_j),i=1,2,j=1,2,$$
(6)
where $`(x_1,x_2)(x,y)`$ and $``$ denotes a sample average. In this idealized but plausible model, at most three of the five parameters $`ϵ_x`$, $`ϵ_y`$, $`\sigma _a`$, $`\sigma _b`$, and $`\beta `$ can be determined from the data, no matter how many galaxies we observe. For example, if $`ϵ_x`$ and $`ϵ_y`$ are known, the other parameters can be estimated using the formulae

$`\mathrm{tan}2\theta `$ $`=`$ $`{\displaystyle \frac{2\sigma _{xy}}{\sigma _{xx}\sigma _{yy}+ϵ_y^2ϵ_x^2}},`$
$`\sigma _b^2`$ $`=`$ $`\sigma _{xx}ϵ_x^2\sigma _{xy}\mathrm{cot}\theta =\sigma _{yy}ϵ_y^2\sigma _{xy}\mathrm{tan}\theta ,`$
$`\sigma _a^2`$ $`=`$ $`\sigma _{xx}ϵ_x^2+\sigma _{xy}\mathrm{tan}\theta =\sigma _{yy}ϵ_y^2+\sigma _{xy}\mathrm{cot}\theta ;`$ (7)
there are two solutions for $`\theta `$ differing by $`\pi /2`$, and we choose the solution for which $`\sigma _a>\sigma _b>0`$. These equations, which we call Gaussian estimators, are related to the AB estimator (5), which in this notation is simply $`\mathrm{tan}\theta =\sigma _{xy}/(\sigma _{xx}ϵ_x^2)`$. However, the Gaussian estimators have the advantages that (i) they are symmetric in $`x`$ and $`y`$, and (ii) they account naturally for the possibility that there is an intrinsic dispersion $`\sigma _b`$ in the $`M_{}`$$`\sigma `$ correlation. The Gaussian estimators can easily be extended to include measurement errors that differ from galaxy to galaxy, and to provide uncertainties in the estimators (e.g., Gull 1989), and with these extensions they are likely to provide a more reliable slope estimator than either the $`\chi ^2`$ or AB estimators.

We close this section with a general comment on fitting linear relations such as (1). The choice of the reference value $`\sigma _0`$ affects the uncertainty in $`\alpha `$ and the covariance between the estimated values of $`\alpha `$ and $`\beta `$. A rough rule of thumb is that $`\sigma _0`$ should be chosen near the middle of the range of values of $`\sigma `$ in the galaxy sample to minimize the uncertainty in $`\alpha `$ and the correlation between $`\alpha `$ and $`\beta `$. As an example, Ferrarese & Merritt (2000b) use $`\sigma _0=1\text{km s}^1`$ and find an uncertainty in $`\alpha `$ of $`\pm 1.3`$. However, most of this uncertainty arises because errors in $`\alpha `$ and $`\beta `$ are strongly correlated at this value of $`\sigma _0`$ (correlation coefficient $`r=0.998`$). Simply by choosing $`\sigma _0=200\text{km s}^1`$ the uncertainty in $`\alpha `$ is reduced by a factor of more than ten, to $`\pm 0.09`$.

## 3 The data

The $`M_{}`$$`\sigma `$ relation has been explored in the literature using a number of distinct datasets:

1. Sample FM1 Much of FM’s analysis is based on a set of 12 galaxies with “secure” black-hole mass estimates (sample A, Table 1 of Ferrarese & Merritt 2000b). However, their definition of “secure” is not itself secure: in §5, we reject one of the galaxies in this sample (NGC 4374) because of concerns about the reliability of its mass estimate, and the best estimate of the mass of another (IC 1459) has recently increased by a factor of six. Half of the black-hole mass estimates in this sample come from gas kinematics, as determined by Hubble Space Telescope (HST ) emission-line spectra, and the remainder from stellar and maser kinematics. Unless otherwise indicated, when discussing this sample we shall use the upper and lower limits to the dispersion and black-hole mass given by Ferrarese & Merritt (2000b).222The error bars in $`x`$ and $`y`$ are given by $`(\mathrm{log}\sigma _{\mathrm{upper}}\mathrm{log}\sigma _{\mathrm{lower}})/2`$ and $`(\mathrm{log}M_{,\mathrm{upper}}\mathrm{log}M_{,\mathrm{lower}})/2`$, respectively. The slope estimators then yield

$$\beta _\chi =4.47\pm 0.44,\beta _{\mathrm{AB}}=4.81\pm 0.55.$$
(8)
The minimum $`\chi ^2`$ per degree of freedom is 0.69, which indicates an acceptable fit; thus there is no evidence for any intrinsic dispersion in this sample.
2. Sample G1 The sample used by Gebhardt et al. (2000a) contains 26 galaxies. Of these, the majority (18) of the mass estimates are from axisymmetric dynamical models of the stellar distribution function, based on HST and ground-based absorption-line spectra. All of the galaxies in sample FM1 are contained in this sample except for NGC 3115. The stated rms fractional uncertainty in the black-hole masses is 0.22 dex, but following Gebhardt et al. (2000a), we shall adopt $`ϵ_y=0.30`$, which yields a minimum $`\chi ^2`$ per degree of freedom equal to unity. Gebhardt et al. (2000a) take $`ϵ_x=0`$, corresponding to negligible uncertainties in the dispersions; this approximation is discussed in §4.1. The slope estimators then yield

$$\beta _\chi =3.74\pm 0.30,\beta _{\mathrm{AB}}=3.74\pm 0.23.$$
(9)
A maximum-likelihood estimate of the intrinsic dispersion in black-hole mass at constant velocity dispersion for this sample is $`0.22\pm 0.05`$ dex.
3. Sample FM2Merritt & Ferrarese (2001b) supplement sample FM1 with 10 additional galaxies, mostly taken from Kormendy & Gebhardt (2001), for a total of 22 galaxies. The stated rms fractional uncertainty in the black-hole masses is 0.24 dex. The slope estimators yield

$$\beta _\chi =4.78\pm 0.43,\beta _{\mathrm{AB}}=4.65\pm 0.49.$$
(10)
The minimum $`\chi ^2`$ per degree of freedom is 1.1, and there is no evidence for any intrinsic dispersion in the black-hole mass.
4. Sample G2 These are the 22 galaxies listed by Kormendy & Gebhardt (2001) that are also in sample FM2. By comparing samples FM2 and G2 we can isolate the effects of different treatments of the same galaxies. We shall assume 20% uncertainty in the dispersion of the Milky Way, and 5% uncertainty in the velocity dispersions of external galaxies (see §§4.1 and 4.3). Using G2’s stated uncertainties in the black-hole masses, the slope estimators yield $`\beta _\chi =3.70\pm 0.20`$, $`\beta _{\mathrm{AB}}=3.61\pm 0.31`$. The minimum $`\chi ^2`$ per degree of freedom is 2.8, which suggests that either the uncertainties in the black-hole masses are underestimated or there is an intrinsic dispersion in black-hole mass. Adding an intrinsic dispersion of 0.17 dex decreases the value of $`\chi ^2`$ per degree of freedom to unity, and reduces the best-fit slope to

$$\beta _\chi =3.61\pm 0.29,\beta _{\mathrm{AB}}=3.61\pm 0.31.$$
(11)
A maximum-likelihood estimate of the intrinsic dispersion in black-hole mass at constant velocity dispersion for this sample is $`0.16\pm 0.05`$ dex.

## 4 Why are the slopes different?

Our goal is to determine why different investigations yield such a wide range of slopes. In particular, the two samples from FM give slopes $`4.5`$ (“high” slopes) with both the $`\chi ^2`$ and AB estimators, while the two samples from the Nukers give slopes $`4.0`$ (“low” slopes) with both estimators. In §§4.14.4 we describe several explanations for the slope range that have been proposed in the literature, all of which are found to be inadequate. In §4.5 we suggest that systematic differences in the dispersions used by FM and the Nukers are responsible for most of the slope discrepancy.

### 4.1 Measurement errors in velocity dispersion

Merritt & Ferrarese (2001a) argue that random measurement errors in the velocity dispersion can have a significant effect on the slope of the $`M_{}`$$`\sigma `$ regression. In particular, they claim that the Nukers’ assumption of zero measurement error in $`\sigma `$ leads them to underestimate the slope. To test this claim, we plot in Figure 2 the slope $`\beta `$ derived from the G1 sample using both the AB and $`\chi ^2`$ estimators, as a function of the assumed rms error $`ϵ_x`$ in the log of the velocity dispersion.

For nearly all of the galaxies in sample G1, the data typically have signal-to-noise ratio around 100, and the formal uncertainties in the dispersions are around 2–3% ($`ϵ_x=0.009`$–0.013). However, at this level, stellar template variations, assumptions about the continuum shape, fitting regions used, and atmospheric seeing conditions all can have a noticeable effect on the estimated dispersion. To account crudely for these systematic errors, we double the uncertainties in the dispersions, to 5% ($`ϵ_x=0.021`$). The uncertainty is larger in the Milky Way (see §4.3), and in a few galaxies that we have not observed ourselves and that do not have accurate dispersion profiles in the literature. The statement of Merritt & Ferrarese (2001a) that velocity-dispersion errors are “easily at the 10% level” is indeed correct for the sample FM1, where the rms fractional error in the dispersions is 14% ($`ϵ_x=0.057`$), but their dispersions are mostly based on heterogeneous data that are 20–30 years old (Davies et al., 1987).

Figure 2 shows that the effect of random errors in the dispersions is negligible: at the 5% level, the change in $`\beta `$ for sample G1 is only 0.03 or 0.04 for the $`\chi ^2`$ and AB estimators respectively, and even at the 10% level the corresponding changes are only 0.12 and 0.16.

### 4.2 Measurement errors in black-hole mass

We next ask whether the combined effects of varying assumptions about measurement errors in both velocity dispersion and black-hole mass can explain the discrepancy between the high and low slopes. As usual, we parametrize these uncertainties by $`ϵ_x`$ and $`ϵ_y`$, the rms measurement error in the log of the velocity dispersion and black-hole mass. For simplicity, in this subsection these errors are assumed to be the same for all galaxies in each sample. The effects of these uncertainties on the slope $`\beta `$ can then be explored using the Gaussian estimators (7). These estimators have two advantages over the $`\chi ^2`$ or AB estimators for this purpose: (i) the slope estimator depends only on the difference $`ϵ_y^2ϵ_x^2`$ and hence is a function of only one variable; (ii) the condition that the derived intrinsic dispersion $`\sigma _b^2`$ be positive-definite provides an upper limit to the allowable errors.

The left and right panels of Figure 3 show the slope $`\beta `$ and the maximum allowed value of $`ϵ_x`$ for each of the galaxy samples in §3. For each sample there is a minimum slope $`\beta `$ and a maximum value of $`ϵ_y^2ϵ_x^2`$, beyond which the intrinsic dispersion $`\sigma _b^2`$ is negative. In particular, for sample FM1 the minimum allowable slope is $`\beta =4.39`$; thus there are no assumptions about the measurement errors that can lead to a slope in the low range. The slope vs. error lines in the left panel of Figure 3 are approximately parallel for all four samples; thus there is no single set of measurement errors that could remove the discrepancy between the high slopes found by FM and the low slopes found by the Nukers. Consistent slopes would require that $`(ϵ_y^2ϵ_x^2)_{\mathrm{Nuker}}(ϵ_y^2ϵ_x^2)_{\mathrm{FM}}0.07`$. This relation, combined with the constraint $`\sigma _b^2>0`$, cannot be satisfied with any plausible combination of measurement errors—note in particular that $`ϵ_y`$ should be similar for the two groups since they rely on many of the same black-hole mass determinations, and $`ϵ_x`$ should be smaller for the Nuker samples than the FM samples, since the Nukers employ high signal-to-noise ratio slit spectra while FM rely on central velocity dispersions from the pre-1990 literature. We conclude that random measurement errors cannot explain the slope discrepancy.

### 4.3 The dispersion of the Milky Way

Merritt & Ferrarese (2001a) also argue that the slope is strongly affected by the assumed dispersion for the Milky Way Galaxy, for which the Nukers’ estimated dispersion $`\sigma =75\text{km s}^1`$ should be increased to $`\sigma =100\text{km s}^1`$. We show in Figure 4 how the derived slope depends on the Milky Way dispersion, for both samples G1 and FM1. We see that in fact $`\beta `$ is quite insensitive to the Milky Way dispersion used in the G1 sample: increasing the dispersion from $`75\text{km s}^1`$ to $`100\text{km s}^1`$ as suggested by Merritt & Ferrarese (2001a) increases $`\beta `$ only by 0.13. The corresponding slope change is substantially larger for sample FM1—$`0.27`$ for the $`\chi ^2`$ estimator and $`0.44`$ for the AB estimator—but this strong sensitivity reflects the small size of that sample and is not relevant to conclusions drawn by Gebhardt et al. (2000a) from sample G1.

Despite this conclusion, it is worthwhile to determine a more accurate value for the Milky Way dispersion to use in the $`M_{}`$$`\sigma `$ relation. We review the data on the dispersion of the Galactic bulge in the Appendix, where our results are summarized in the dispersion profile of Figure 9 and equation (A3). We stress that the dispersion profile of the Milky Way is determined from a heterogeneous set of tracers with uneven spatial coverage, and by very different methods than the dispersions of the external galaxies discussed in this paper. We therefore assign our estimates of the Milky Way dispersion an uncertainty of 20%, much larger than the formal uncertainty, and much larger than the 5% uncertainty that we assume for the dispersions of external galaxies.

The conversion of the dispersion profile in equation (A3) to a characteristic dispersion is different for FM and the Nukers. FM define their dispersion to be the luminosity-weighted rms line-of-sight dispersion within a circular aperture of radius $`r_e/8`$, where $`r_e`$ is the effective radius. For $`r_e=0.7\text{kpc}`$ as derived in the Appendix, we find $`\sigma =95\text{km s}^1`$. Because the bulge is triaxial, we correct the dispersion that we measure from our particular location to the average over all azimuths in the Galactic plane. Binney, Gerhard, & Spergel (1997) model the bulge as a triaxial system with axis ratios 1 : 0.6 : 0.4 and long axis at an angle $`\varphi _0=20^{}`$ from the Sun-center line. If the density is stratified on similar ellipsoids, the ratio $`r^2\sigma ^2(\varphi _0=20^{})/\sigma ^2(\varphi _0)`$ depends only on the axis ratios (Roberts, 1962). For the axis ratios given by Binney et al., $`r=1.07`$. Thus our best estimate for the dispersion within $`r_e/8`$ is $`\sigma _{\mathrm{FM}}=90\pm 18\text{km s}^1`$; if we use this instead of FM’s estimate of $`\sigma =100\pm 20\text{km s}^1`$, the slope derived from sample FM1 is reduced from $`\beta _{\mathrm{AB}}=4.81\pm 0.55`$ to $`\beta _{\mathrm{AB}}=4.66\pm 0.42`$, and for sample FM2 from $`\beta _{\mathrm{AB}}=4.65\pm 0.49`$ to $`4.54\pm 0.40`$.

In contrast, the Nukers use the luminosity-weighted rms line-of-sight dispersion within a slit aperture of half-length $`r_e`$. This dispersion depends weakly on the slit width, which we take to be 70 pc (corresponding to 1 arcsec at Virgo). In this case we find $`\sigma =110\text{km s}^1`$; reducing this by a factor $`r`$ to account for triaxiality, we have $`\sigma =103\pm 20\text{km s}^1`$, close to the value advocated by FM. This change increases the slope derived by Gebhardt et al. (2000a) from $`\beta _\chi =3.74\pm 0.15`$ only to $`\beta _\chi =3.88\pm 0.15`$. Thus, improved estimates of the velocity dispersion of the Milky Way bulge reduce the slope discrepancy only slightly.

### 4.4 Different samples

Merritt & Ferrarese (2001b) argue that the shallower slope obtained by the Nukers arises in part from the inclusion of galaxies in which the black-hole sphere of influence is not well resolved. However, the samples FM2 and G2 contain exactly the same 22 galaxies, all of which are claimed by Merritt & Ferrarese (2001b) to have a well-resolved sphere of influence, and the difference in slope $`\beta _\chi `$ (eqs. 10 and 11) is actually larger than between the samples FM1 and G1.

### 4.5 Aperture and effective dispersions

Why, then, are the slopes different, particularly in the samples FM2 and G2, which contain the same galaxies? If we fit the dispersions in these samples to a relation of the form

$$\mathrm{log}\sigma _{\mathrm{G2}}=\gamma +(1+\delta )\mathrm{log}\sigma _{\mathrm{FM2}},$$
(12)
we find

$$\delta _\chi =0.13\pm 0.10,\delta _{\mathrm{AB}}=0.23\pm 0.10,$$
(13)
significantly different from the value $`\delta =0`$ that should obtain if there were no systematic differences between the dispersions (see Figure 5). A relation of this kind implies that the slopes $`\beta `$ determined from the FM2 and G2 samples will be related by $`\beta _{\mathrm{FM2}}/\beta _{\mathrm{G2}}=1+\delta `$. Then if $`\delta `$ is in the range 0.15–0.20, and the Nuker sample gives $`\beta =4`$, the sample from FM will give $`\beta =4.6`$–4.8, well inside the high range. Thus it appears that the major cause of the range of slopes is systematic differences in the dispersions: FM’s dispersions lead to high slopes, and the Nukers’ dispersions lead to low slopes.

This possibility was suggested by Gebhardt et al. (2000a), but was later rejected by Merritt & Ferrarese (2001a), who argued that systematic differences between dispersions are unimportant because there was “remarkably little difference on average” between the dispersions (they quote a mean ratio of 1.01 and a correlation coefficient in the logs of 0.97). However, these statistics have no bearing on the slope $`1+\delta `$ in equation (12).

There are several possible explanations of the difference in dispersions:

1. The Nukers use the rms dispersion within a slit aperture of length $`2r_e`$ (hereafter $`\sigma _1`$), while FM’s results are based on the rms dispersion within a circular aperture of radius $`r_e/8`$ (hereafter $`\sigma _8`$). The ratio $`\sigma _8/\sigma _1`$ could depend systematically on the velocity dispersion of the galaxy, for example,

$$\sigma _1\sigma _8^{1+\delta _1}.$$
(14)
In this case the difference in slope estimates would reflect the structural properties of the galaxies.
2. FM do not actually measure $`\sigma _8`$. Instead, they use the central velocity dispersion (hereafter $`\sigma _c`$), typically measured in an aperture of radius $`r_{\mathrm{ap}}2`$ arcsec (Davies et al., 1987), and correct this to a circular aperture of radius $`r_e/8`$ using the relation (Jørgensen, Franx, & Kjaergaard, 1995) $`\sigma _8^{}=\sigma _c(8r_{\mathrm{ap}}/r_e)^{0.04}`$ (the prime is used to distinguish this approximation to $`\sigma _8`$ from the actual value of $`\sigma _8`$). The ratio $`\sigma _8^{}/\sigma _8`$ could depend systematically on the velocity dispersion of the galaxy, for example,

$$\sigma _8(\sigma _8^{})^{1+\delta _2}.$$
(15)
Such a trend could arise if the correction factor depends on galaxy luminosity or type. In this case the difference in slope estimates would reflect a shortcoming in FM’s analysis rather than a real physical effect.
3. The dispersion measurements used by one or both of the two groups could be subject to dispersion-dependent systematic errors (e.g., one set of measurements is systematically low at high dispersions). In this case the difference in slope estimates would reflect problems with the data reduction.

To explore these possibilities, we have examined a sample of 40 early-type galaxies for which Faber et al. (1997) have compiled effective radii and central velocity dispersions, and have fitted HST photometry to a five-parameter “Nuker law” profile. Each galaxy is assumed to contain a central black hole, with mass given by the $`M_{}`$$`\sigma `$ relation in the form derived below (eq. 19). We use the Nuker law and the assumptions of spherical symmetry, constant mass-to-light ratio, isotropic velocity dispersion, and 1 arcsec slit width to compute the ratios $`\sigma _8/\sigma _1`$ and $`\sigma _8^{}/\sigma _8`$. This approach is model-dependent, but has the advantages that (i) the discussion is independent of observational errors in the dispersions, since the dispersion ratios are determined by a dynamical model; (ii) the sample is larger, since more galaxies have HST photometry than dispersion profiles. We find

$`\mathrm{log}\left({\displaystyle \frac{\sigma _8}{\sigma _1}}\right)`$ $`=`$ $`(0.004\pm 0.002)+(0.021\pm 0.010)\mathrm{log}\left({\displaystyle \frac{\sigma _c}{200\text{km s}^1}}\right),`$
$`\mathrm{log}\left({\displaystyle \frac{\sigma _8^{}}{\sigma _8}}\right)`$ $`=`$ $`(0.012\pm 0.003)(0.056\pm 0.014)\mathrm{log}\left({\displaystyle \frac{\sigma _c}{200\text{km s}^1}}\right).`$ (16)
The first of these equations suggests that there is a small but significant systematic trend in the ratio $`\sigma _8/\sigma _1`$ with dispersion, of the form (14) with $`\delta _10.02`$. However, this trend has the wrong sign and only a small fraction of the amplitude required to explain the systematic differences in equations (12) and (13); thus explanation 1 in the list above does not appear to be important.

The second of these equations suggests that there is a larger systematic trend in the ratio $`\sigma _8^{}/\sigma _8`$ with dispersion, of the form (15) with $`\delta _20.06`$. This trend is sufficient to explain about one-third of the systematic differences seen in equations (12) and (13). The origin of this trend, explanation 2 in the list above, is clear. Gebhardt et al. (1996) and Faber et al. (1997) show that the shape of the surface-brightness profile in the central parts of early-type galaxies depends on the galaxy luminosity (and hence on its velocity dispersion). Thus, the use of a single empirical formula to correct from $`\sigma _c`$ to $`\sigma _8`$ will lead to systematic errors that are correlated with velocity dispersion. It is always better to use the actual kinematic observations, as was done in G1, than to apply empirical correction factors.

Explanation 3, dispersion-dependent systematic measurement errors, is more difficult to assess. Hudson et al. (2001) compare dispersion measurements from 27 sources, including the catalog used by FM (Davies et al., 1987), and in most cases find no evidence for dispersion-dependent errors of the amplitude found in equation (13) (see Hudson et al.’s Figure 3). Nevertheless, it is striking that the data points in Figure 5 appear sharply lower for dispersions $`300\text{km s}^1`$ than for smaller dispersions. Measuring large dispersions is particularly difficult because the spectral lines blend together. The principal conclusion is that we badly need a systematic campaign of accurate HST and ground-based measurements of the radial velocity-dispersion profiles of early-type galaxies with black-hole candidates. A second conclusion is that the slope of the $`M_{}`$$`\sigma `$ relation should be estimated only from dispersion measurements at or within well-defined metric radii, rather than central velocity dispersions measured within apertures of a given angular radius.

The same sample of 40 galaxies can also be used to explore the degree of contamination of the dispersions by the dynamical influence of the central black hole. We computed spherical, isotropic dynamical models with and without a black hole of mass given by equation (19). We denote the ratio of the dispersion $`\sigma _1`$ with and without the black hole by $`f_1`$, and the analogous ratio for the dispersion $`\sigma _8^{}`$ by $`f_8^{}`$. The results are shown in Figure 6. In most galaxies the addition of the black hole raises either dispersion no more than 3–4%. However, in a few cases the contamination is much larger, more than 15%. In such galaxies the dispersion measures $`\sigma _c`$, $`\sigma _1`$, $`\sigma _8`$, or $`\sigma _8^{}`$ are all misleading. Future versions of the $`M_{}`$$`\sigma `$ relation should be based on dispersion measures that are less strongly weighted to the center.

## 5 Black-hole mass vs. velocity dispersion: a new estimate

In this section we present a new analysis of the $`M_{}`$$`\sigma `$ relation using the 31 galaxies in Table 1; over half of these have new or revised black-hole mass or dispersion determinations since the analysis by Gebhardt et al. (2000a).

### 5.1 Comments on individual galaxies

Milky Way: We use the black-hole mass estimate by Chakrabarty & Saha (2001), $`(1.8\genfrac{}{}{0pt}{}{+0.4}{0.3})\times 10^6M_{}`$. For comparison, Ghez et al. (1998) find $`(2.4\pm 0.2)\times 10^6M_{}`$, and Genzel et al. (2000) find (2.6–3.3)$`\times 10^6M_{}`$. The dispersion and its uncertainty are discussed in §4.3. The bulge mass-to-light ratio is taken from Kent (1992).

M32: The velocity dispersion is obtained from van der Marel et al. (1994) and the black-hole mass, $`M_{}=(2.5\pm 0.5)\times 10^6M_{}`$, is from Verolme et al. (2002). In estimating the dispersion, we have excluded the region near the center that is strongly perturbed by the black hole (cf. Figure 6). Other recent mass estimates, by van der Marel et al. (1998) and Joseph et al. (2001), give similar results: $`(3.9\pm 0.8)\times 10^6M_{}`$ and $`(3\pm 1)\times 10^6M_{}`$, respectively.

M31: The modeling is complicated by the double nucleus. Kormendy & Bender (1999) find $`M_{}=(3.0\pm 1.5)\times 10^7M_{}`$, although this result relies heavily on the small displacement between the center of light of the nucleus and bulge. Tremaine (1995) and Bacon et al. (2001) find $`M7\times 10^7M_{}`$, but without detailed model fitting. We adopt the range (2.0–8.5)$`\times 10^7M_{}`$.

NGC 1068: The black-hole mass is taken from Greenhill & Gwinn (1997); the error estimates are our own and are very approximate. The dispersion (Kobulnicky & Gebhardt, 2000) is somewhat uncertain because of contamination from the bright nucleus.

NGC 3115: The black-hole mass is based on stellar kinematics (Kormendy et al., 1996a). Although NGC 3115 does not have three-integral axisymmetric dynamical models, it does have a compact, high-contrast stellar nucleus, and the mass of the nucleus plus black hole can be estimated from the virial theorem. In estimating the dispersion, we have excluded the region near the center that is strongly perturbed by the black hole (cf. Figure 6).

NGC 3245: The velocity dispersion is obtained from Simien & Prugniel (1998).

NGC 4258: The velocity dispersion is obtained from Héraudeau & Simien (1998).

NGC 4486: The mass is the average of the values given by Harms et al. (1994) and Macchetto et al. (1997), corrected to a distance of 16.1 Mpc.

NGC 2787, NGC 4459, NGC 4596: The black-hole masses are based on Space Telescope Imaging Spectrograph (STIS) measurements of ionized-gas disks by Sarzi et al. (2001). The disk inclinations are determined from dust-lane morphology. Note that the distance and dispersion for NGC 2787, 7.5 Mpc and $`140\text{km s}^1`$, are much smaller than the values assumed by Sarzi et al. (2001). Our distance is from Tonry et al. (2001) and the dispersion was measured by one of us (Gebhardt). For NGC 4459 and NGC 4596 we have used the dispersions $`\sigma _8^{}`$ from Sarzi et al. (2001), since on average these are close to $`\sigma _1`$ (eq. 16).

IC 1459: The mass estimate that we use (Cappellari et al., 2002), based on stellar kinematics, is much larger than an earlier estimate by the same group from gas kinematics, (2–6)$`\times 10^8M_{}`$ (Verdoes Kleijn et al., 2000). The mass estimate from stellar kinematics is much more reliable, since the gas rotation curve is asymmetric and non-Keplerian.

We do not include the following galaxies in our sample:

NGC 4594 (Kormendy et al., 1996b), NGC 4486B (Kormendy et al., 1997), NGC 4350 (Pignatelli, Salucci, & Danese, 2001), NGC 3031=M81, and NGC 3998 (Bower et al., 2000) exhibit strong evidence from stellar dynamics for a black hole, but do not yet have three-integral dynamical models.

NGC 4374 (M84) has strong evidence for a black hole from gas dynamics, but the published estimates of the black-hole mass differ by far more than the stated errors: Bower et al. (1998) find (0.9–2.6)$`\times 10^9M_{}`$; Maciejewski & Binney (2001) find $`4\times 10^8M_{}`$, and Barth et al. (2001b) find $`10^9M_{}`$. The mass assigned to this galaxy is a factor of four larger in sample FM2 than in sample G2, which is by far the largest discrepancy between the two samples.

NGC 4945 has a mass estimate from maser emission (Greenhill, Moran, & Herrnstein, 1997) but no reliable dispersion.

NGC 5128 has a mass estimate from ground-based observations of a rotating gas disk (Marconi et al., 2001) but no HST spectroscopy; moreover, the galaxy has peculiar morphology, presumably because of a recent merger, and thus may not follow the same $`M_{}`$$`\sigma `$ relation as more normal galaxies.

Our sample contains eight galaxies with black-hole mass estimates based on gas kinematics. We have some concern that these results may have large systematic errors, due in part to uncertainties in the the spatial distribution of the gas (e.g., filled disk or torus configuration; uncertain inclination and thickness) and the large but uncertain correction for pressure support. We therefore urge caution when interpreting results from samples in which a large fraction of the black-hole mass estimates are based on gas kinematics. Eventually, galaxies with black-hole mass determinations from more than one technique will be invaluable for disentangling the systematic errors in different methods.

### 5.2 Slope estimation

We use the sample of galaxies and black-hole masses in Table 1 to estimate the logarithmic slope $`\beta `$ in the $`M_{}`$$`\sigma `$ relation. We assume 20% uncertainty in the dispersion of the Milky Way (cf. §4.3) and 5% uncertainties in the dispersions of external galaxies (cf. §4.1), although the uncertainties in the dispersions of a few galaxies that we have not observed ourselves may be larger. Initially we assume 0.33 dex rms uncertainties in the black-hole masses, which yields $`\chi ^2`$ per degree of freedom of unity. Using the $`\chi ^2`$ and AB estimators defined in §2, we find

$$\beta _\chi =4.03\pm 0.33,\beta _{\mathrm{AB}}=4.12\pm 0.34.$$
(17)
This approach does not account for the varying precision of the mass estimates for different galaxies. Therefore we have also computed the slope using the estimated errors in the black-hole masses in Table 1, adding to these in quadrature a common intrinsic dispersion with rms value $`ϵ_0`$ (i.e., $`ϵ_{yi}(ϵ_{yi}^2+ϵ_0^2)^{1/2}`$). We find that $`ϵ_0=0.27`$ gives

$$\beta _\chi =4.00\pm 0.31,\beta _{\mathrm{AB}}=4.12\pm 0.34,$$
(18)
with minimum $`\chi ^2`$ per degree of freedom of 1.00. A maximum-likelihood estimate of the intrinsic dispersion in black-hole mass at constant velocity dispersion for this sample is $`ϵ_0=0.23\pm 0.05`$ dex.

In both equations (17) and (18) the AB estimator for the slope is larger than the $`\chi ^2`$ estimator by about 0.1; since we have shown in §2 that the AB estimator may be biased, we prefer to rely on the $`\chi ^2`$ estimator. For our final answer we simply average $`\beta _\chi `$ from equations (17) and (18). Including results for the parameter $`\alpha `$ obtained in the same way, we have

$$\alpha =8.13\pm 0.06,\beta =4.02\pm 0.32;$$
(19)
the parameter $`\alpha `$ is evaluated for $`\sigma _0=200\text{km s}^1`$, for which the correlation coefficient between $`\alpha `$ and $`\beta `$ is only $`0.09`$.

Thus our best estimate (19) is just at the edge of the low range, $`\beta 4.0`$. To test the robustness of this result, we have tried culling the sample in several ways:

* If we consider only the 21 galaxies from Table 1 with masses determined from stellar kinematics, we find

$$\alpha =8.13\pm 0.09,\beta =4.02\pm 0.44;$$
(20)
the close agreement in the parameters in equations (19) and (20) implies that there is no significant systematic bias between masses determined by stellar kinematics and other methods.
* The dispersions for the Milky Way and for external galaxies are determined by quite different methods. The Milky Way also has one of the smallest and most accurate black-hole masses in our sample, and therefore has an unusually strong influence on the slope of the $`M_{}`$$`\sigma `$ relation. If we remove the Milky Way from our sample, the slope is reduced to $`\beta =3.88\pm 0.32`$, a change of 0.14 (0.4 standard deviations) towards even lower slopes.
* We have argued in §4.5 that high velocity-dispersion measurements may be subject to systematic errors. Thus we also estimate the slope using only the 25 galaxies in the sample with dispersion less than $`250\text{km s}^1`$. We find $`\beta =3.77\pm 0.49`$; once again the slope is even lower than our best estimate (19).
* The galaxy sample with the most homogeneous observations and analysis consists of the 10 galaxies analyzed by Pinkney et al. (2002) and Gebhardt et al. (2002). These all have HST spectra acquired with STIS as well as ground-based spectra, HST photometry, and axisymmetric orbit-based dynamical models, and were all reduced and analyzed in the same way. For this sample we find $`\beta =3.67\pm 0.70`$; once again the slope is consistent with and even lower than our best estimate.
* We have removed 9 galaxies from the sample which were subject to criticism: the Milky Way (uncertain dispersion), M31 (no accurate models of the double nucleus), NGC 1068 (both the dispersion and the interpretation of the maser kinematics are uncertain), NGC 2778 (the lowest signal-to-noise ratio in the Gebhardt et al. 2002 sample and a correspondingly large uncertainty in the black-hole mass), NGC 3115 (no three-integral dynamical models), NGC 3379 and NGC 5845 (these have only a single FOS pointing rather than STIS slit spectra at HST resolution; while there is no obvious problem with either measurement, other galaxies in the Gebhardt et al. 2002 sample have superior spatial coverage of the kinematics), NGC 4459 (the inclination of the gas disk is uncertain because the kinematic data comes from a single long-slit spectrum; also, the dispersion is uncertain because it is obtained from low-resolution data), and NGC 6251 (the most distant galaxy with a black-hole mass measurement; the sphere of influence of the black hole is poorly resolved and in addition there are the usual uncertainties—uncertain disk orientation, influence of random motions in the gas—associated with mass measurements from gas kinematics). The reduced sample of 22 galaxies has a slope $`\beta =3.79\pm 0.32`$, once again lower than our best estimate.

Since most of these culled samples have slopes that are smaller than our best fit (19), we suspect that our best fit may slightly overestimate the true slope by 0.1–0.3.

The data from Table 1 and the fit (19) are shown in Figure 7. In Figure 8 we show the residuals to the best-fit correlation.

The two largest residuals in Figure 7 belong to NGC 2778 ($`0.75`$ dex) and the Milky Way ($`0.72`$ dex); the largest positive residual belongs to NGC 3115 ($`+0.63`$ dex). The poor fit of the Milky Way probably arises because its dispersion profile has been determined by different methods than the other galaxies, using heterogeneous tracers and a variety of surveys; we have allowed for this by assigning the Milky Way dispersion an uncertainty of 20%, compared to 5% for external galaxies. The large residual in NGC 3115 may arise because its mass has been estimated by simply applying the virial theorem to its nucleus, rather than by dynamical modeling. The large residual in NGC 2778 may reflect the low signal-to-noise ratio of its kinematic data (Pinkney et al., 2002).

## 6 Conclusions

The masses $`M_{}`$ of dark objects (“black holes”) in the centers of nearby early-type galaxies are related to the velocity dispersion $`\sigma `$ by the log-linear relation (1). We have used the sample of 31 galaxies in Table 1 to determine the parameters in this relation, where $`\sigma `$ is defined to be the luminosity-weighted rms velocity dispersion in a slit extending to the effective radius. Our best estimate for the slope of this relationship is $`4.0\pm 0.3`$ (eq. 19), although several culled, and perhaps higher quality, samples give slopes that are lower by 0.1–0.3. There is no evidence for systematic differences in either slope or normalization between black-hole mass measurements based on stellar kinematics and gas kinematics. If the stated measurement errors in the black-hole masses are correct, or if they are underestimated because of systematic errors, the intrinsic dispersion in the $`M_{}`$$`\sigma `$ relation is no larger than about 0.3 dex in black-hole mass (i.e., less than a factor of two).

The range of slopes for the $`M_{}`$$`\sigma `$ relation found in the literature appears to arise mostly from systematic differences in the velocity dispersions used by different groups. We do not believe that these differences reflect the different definitions of dispersion used by the groups (FM use the dispersion within a circular aperture of radius $`r_e/8`$, and the Nukers use the dispersion within a slit aperture of half-length $`r_e`$). It appears that part of the difference results from Ferrarese & Merritt’s analysis, in which central velocity dispersions are extrapolated to $`r_e/8`$ using an empirical formula. However, another—and possibly larger—component appears to arise from poorly understood systematic errors in the dispersion measurements.

In a few galaxies, the influence of the central black hole may significantly affect the velocity dispersions—both the central dispersions used by FM and the slit dispersions used by the Nukers. Future analyses of the $`M_{}`$$`\sigma `$ relation should be based on velocity-dispersion measures that are less strongly weighted to the center. Other improvements in the analysis would include the use of statistical estimators that are more robust and that explicitly include an intrinsic dispersion in the black-hole mass, accounting properly for the asymmetric error bars in black-hole mass determinations, and estimating more accurately the uncertainties in individual dispersion measurements.

The investment of the astronomy community in the difficult task of measuring black-hole masses has not yet been matched by a commensurate investment in the much easier task of obtaining high-quality kinematic maps of galaxies containing black holes. A complete set of high-quality dispersion and rotation profiles for the galaxies in Table 1 would allow us to explore more deeply how the black-hole mass is related to the kinematic structure of its host galaxy.

We thank Michael Hudson and Tim de Zeeuw for discussions, and Tim de Zeeuw for communicating results in advance of publication. Support for proposals 7388, 8591, 9106, and 9107 was provided by NASA through a grant from the Space Telescope Science Institute, which is operated by the Association of Universities for Research in Astronomy, Inc., under NASA contract NAS 5-26555. This research was also supported by NSF grant AST-9900316.

## Appendix A The effective dispersion for the Milky Way

The Milky Way has one of the most accurate black-hole masses and anchors the low-mass end of the $`M_{}`$$`\sigma `$ relation. Therefore it is important to have an accurate value for the dispersion of the Milky Way bulge.

The first task is to estimate the effective or half-light radius $`r_e`$ of the bulge. Kent (1992) models Spacelab $`K`$-band observations of the bulge with a major-axis emissivity profile of the form

$$j(a)=\{\begin{array}{cc}j_ia^{1.85}\hfill & \text{for }a<0.94\text{kpc}\hfill \\ j_oK_0(a/a_0)\hfill & \text{for }a>0.94\text{kpc}\text{,}\hfill \end{array}$$
(A1)
where $`K_0`$ is a modified Bessel function, $`a_0=0.67\text{kpc}`$, and the constants $`j_i`$ and $`j_o`$ are chosen so that the emissivity is continuous. In a spherical galaxy described by equation (A1), the effective radius is $`1.50a_0`$ or 1.0 kpc; Kent’s model is oblate and axisymmetric, with axis ratio 0.6, so the geometric mean of the three effective semi-axes is smaller by $`(0.6)^{1/3}`$, yielding $`r_e=0.84`$ kpc.

Dwek et al. (1995) fit COBE measurements in several bands to a wide variety of triaxial models for the emissivity. Their best-fit model at $`K`$-band has a Gaussian emissivity profile with an effective semi-major axis of 1.86 kpc; the axis ratios are 1 : 0.4 : 0.3, so our best estimate for the effective radius is $`r_e=1.86\text{kpc}(0.4\times 0.3)^{1/3}=0.92\text{kpc}`$. Their second-best model (E3) has $`j(a)K_0(a/a_0)`$ and an effective radius $`r_e=0.56\text{kpc}`$.

Binney, Gerhard, & Spergel (1997) use COBE $`L`$-band photometry to perform a disk/bulge decomposition. Their equation (1b) describes an analytic model for the bulge emissivity that fits the data “very well”:

$$j(a)=j_0\frac{e^{a^2/a_m^2}}{(1+a/a_0)^{1.8}},$$
(A2)
where $`a`$ is the semi-major axis and $`a_m=1.9`$ kpc. They quote $`a_0=100`$ pc, but this value reflects the fact that the data have been smoothed to an angular resolution of 1.5 or 200 pc, and photometry at higher resolution suggests that $`a_0`$ is less than 1 pc (e.g., Genzel et al. 1996). The effective semi-major axis for equation (A2) is $`0.48a_m`$ or $`0.91\text{kpc}`$; the corresponding geometric mean of the effective semi-axes (1 : 0.6 : 0.4) is $`r_e=0.57\text{kpc}`$.

Based on these estimates, we shall adopt $`r_e=0.7\pm 0.2\text{kpc}`$. The much larger estimate $`r_e=2.7\text{kpc}`$ given by Merritt & Ferrarese (2001a) is based on a table in Gilmore, King, & van der Kruit (1990), which in turn appears to be based on the galaxy model of Bahcall & Soneira (1980), which in turn is based on comments by G. de Vaucouleurs in the 1970s that $`r_e`$ is about one-third of the distance of the Sun from the Galactic Center.

The next task is to estimate the velocity dispersion as a function of radius. We are interested in the rms line-of-sight velocity $`v_{\mathrm{los}}^2^{1/2}`$ measured relative to the Local Standard of Rest, since this is the closest analog to the dispersions used in the $`M_{}`$$`\sigma `$ relation for external galaxies. This quantity differs from the usual dispersion quoted in bulge studies, which is relative to the local mean velocity, $`\sigma =(v_{\mathrm{los}}\overline{v})^2^{1/2}`$, where $`\overline{v}=v_{\mathrm{los}}`$. When papers quote values for $`\sigma `$ and $`\overline{v}`$ we set $`v_{\mathrm{los}}^2=\overline{v}^2+\sigma ^2`$. We use the following sources:

1. Due to the interest in the black hole in our Galaxy, the kinematics in the central few parsecs have been investigated much more thoroughly than the kinematics at larger radii (Genzel et al. 2000, especially their Figure 16). The entries at radii $`<5\text{pc}`$ in Table 2 are taken from Genzel et al.’s Table 4; at these radii corrections for rotation are negligible.
2. OH/IR stars are mass-losing asymptotic giant branch stars, which are detected by hydroxyl maser emission from their circumstellar envelopes. They are old enough to represent a phase-mixed population and are unaffected by obscuration, and hence should be good tracers of the kinematics of the bulge. The survey by Lindqvist et al. (1992a,b) lists 133 OH/IR stars within $`1^{}`$ or 140 pc of the Galactic Center. We have divided these into three equal groups by projected distance from the center and computed the dispersion for each group. One limitation of this survey is that its radial-velocity coverage was relatively small, $`|v_{\mathrm{los}}|217\text{km s}^1`$, so that high-velocity OH/IR stars might have been missed. We have corrected for this cutoff, assuming that the distribution of line-of-sight velocities is Gaussian, in the two bins where the correction is less than 10%, and have discarded the third bin. At larger distances, Sevenster et al. (1997) have located 307 OH/IR stars in the region $`|\mathrm{}|<10^{}`$, $`|b|<3^{}`$. The minimum velocity range in this survey was $`330\text{km s}^1<v_{\mathrm{los}}<402\text{km s}^1`$ so velocity selection effects are negligible. We have discarded all sources not having a standard double-peaked profile and all sources with expansion velocity $`>17\text{km s}^1`$, which appear to represent a younger, more rapidly rotating population (Winnberg et al., 1998). The remaining 208 stars were divided into five equal groups by projected distance, and the mean projected distance and dispersion were computed for each group.
3. Beaulieu, Dopita, & Freeman (1999) have conducted an H$`\alpha `$ survey for new planetary nebulae and remeasured the velocities of many known planetary nebulae. Their databases contain 183 planetary nebulae within $`10^{}`$ of the Galactic Center. We have divided these into four equal groups by projected distance, and the mean projected distance and dispersion were computed for each group. Beaulieu et al. estimate that their velocity errors are $`\pm 11\text{km s}^1`$, which is negligible.
4. Blum et al. (1994, 1995) have measured the dispersion of samples of M giants in four fields between 160 and 300 pc from the Galactic Center. Terndrup, Sadler, & Rich (1995) have measured the dispersion of K giants in Baade’s window (0.56 kpc from the Galactic Center). We include only stars with $`V>16.0`$, which they believe restricts the sample to bulge stars and eliminates the foreground disk.

The data at $`r>0.1\text{pc}`$ shown in Figure 9 and Table 2 have been fit to the functional form

$$v_{\mathrm{los}}^2^{1/2}=c\frac{(r/r_0)^\alpha }{1+b(r/r_0)^\beta }+d(r/r_0)^{1/2}.$$
(A3)
For $`r_0=500\text{pc}`$ the best-fit values are $`c=633\text{km s}^1`$, $`\alpha =0.67`$, $`\beta =1.14`$, $`b=4.64`$, $`d=2.52`$. The general features of this curve—a minimum in the dispersion near 5 pc and a maximum of $`130\text{km s}^1`$ at a few hundred pc—are not new (Kent, 1992).

A possible concern is that the bulge is flattened, with an axis ratio of about 0.5, so the dispersion at a given radius may depend on the angle between the radius vector and the Galactic plane. To address this concern, we have divided the data points from outside 4 pc from the Galactic Center into those biased towards the minor axis, plotted with filled symbols (the criterion is $`|\mathrm{}|>|b|`$, where $`\mathrm{}`$ and $`b`$ are the Galactic longitude and latitude; these are objects such as planetary nebulae and late-type giants that are found optically), and those biased towards the major axis (the OH/IR stars, found in surveys along the Galactic plane), which are plotted with open symbols. There is no obvious systematic difference between the dispersion curves defined by the filled and open symbols.

We employ the fit (A3) at the end of §4.3 to estimate the appropriate Milky Way dispersion to use in the $`M_{}`$$`\sigma `$ relation.


format: text

text:
# Measuring 𝛼 in the Early Universe: CMB Temperature, Large-Scale Structure and Fisher Matrix Analysis

## I Introduction

The search for observational evidence for time or space variations of the ‘fundamental’ constants that can be measured in our four-dimensional world is an extremely exciting area of current research, with several independent claims of detections in different contexts emerging in the past year or so, together with other improved constraints Murphy et al. (2001a); Webb et al. (2001); Murphy et al. (2001b, c); Avelino et al. (2001); Fujii (2002); Ivanchik et al. (2001). We will review these in Sect. II.

Most of the current efforts have been concentrating on the fine-structure constant, $`\alpha `$, both due to its obviously fundamental role and due to the availability of a series of independent methods of measurement. Noteworthy among these is the Cosmic Microwave Background (CMB) Avelino et al. (2000a, b); Battye et al. (2001); Avelino et al. (2001). The latest available CMB results Avelino et al. (2001) yield a one-sigma indication of a smaller $`\alpha `$ in the past, but are consistent with no variation at the two-sigma level. However, these results are somewhat weakened by the existence of various important degeneracies in the data, and furthermore not everybody agrees on which and how strong these degeneracies are Avelino et al. (2000b); Battye et al. (2001); Huey et al. (2002).

Here we aim to clarify this issue by analyzing these possible degeneracies in some detail, mainly by means of a Fisher Matrix Analysis (FMA), see Sect. IV. We will emphasize that there are crucial differences between ‘theoretical’ degeneracies (due to simple physical mechanisms) and ‘experimental’ degeneracies (due to the fact that each CMB experiment only probes a limited range of scales, and that the experimental errors are scale-dependent). We will also show how such degeneracies can be eliminated either by using complementary data sets (such as large-scale structure constraints, see Sect. III) or by acquiring better data (such as that to be obtained by MAP and Planck). We present our conclusions in Sect. V.

In a companion paper Rocha et al. (2002) we will discuss a further way in which these degeneracies can be broken, namely by including information from CMB polarization.

## II The Present Observational Status

The recent explosion of interest in the study of varying constants is mostly due to the results of Webb and collaborators Murphy et al. (2001a); Webb et al. (2001); Murphy et al. (2001b, c) of a $`4\sigma `$ detection of a fine-structure constant that was smaller in the past,

$$\frac{\mathrm{\Delta }\alpha }{\alpha }=(0.72\pm 0.18)\times 10^5,z0.53.5;$$
(1)
indeed, more recent work Webb (2001) provides an even stronger detection. These results are obtained through comparisons of various transitions (involving various different atoms) in the laboratory and in quasar absorption systems, using the fact that the size of the relativistic corrections goes as $`(\alpha Z)^2`$. A number of tests for possible systematic effects have been carried out, all of which have been found either not to affect the results or to make the detection even stronger if corrected for.

A somewhat analogous (though simpler) technique uses molecular hydrogen transitions in damped Lyman-$`\alpha `$ systems to measure the ratio of the proton and electron masses, $`\mu =m_p/m_e`$ (using the fact that electron vibro-rotational lines depend on the reduced mass of the molecule, and this dependence is different for different transitions). The latest results Ivanchik et al. (2001) using two systems at redshifts $`z2.3`$ and $`z3.0`$ are

$$\frac{\mathrm{\Delta }\mu }{\mu }=(5.7\pm 3.8)\times 10^5$$
(2)
or

$$\frac{\mathrm{\Delta }\mu }{\mu }=(12.5\pm 4.5)\times 10^5,$$
(3)
depending on which of the (two) available tables of ‘standard’ laboratory wavelengths is used. This implies a $`1.5\sigma `$ detection in the more conservative case, though it also casts some doubts on the accuracy of the laboratory results, and on the influence of systematic effects in general.

We should also mention a recent re-analysis Fujii (2002) of the well-known Oklo bound Damour and Dyson (1996). Using new Samarium samples collected deeper underground (aiming to minimize contamination), these authors again provide two possible results for both $`\alpha `$ and the analogous coupling for the strong nuclear force, $`\alpha _s`$,

$$\frac{\dot{\alpha }}{\alpha }\frac{\dot{\alpha }_s}{\alpha _s}=(0.4\pm 0.5)\times 10^{17}yr^1$$
(4)
or

$$\frac{\dot{\alpha }}{\alpha }\frac{\dot{\alpha }_s}{\alpha _s}=(4.4\pm 0.4)\times 10^{17}yr^1.$$
(5)
Note that these are given as rates of variation, and effectively probe timescales corresponding to a cosmological redshift of about $`z0.1`$. Unlike the case above, these two values correspond to two possible physical branches of the solution. See Fujii (2002) for a discussion of why this method yields two solutions (and also note that these results have opposite signs relative to previously published ones Fujii et al. (2000)). While the first of these branches provides a null result, (5) is a strong detection of an $`\alpha `$ that was larger at $`z0.1`$, that is a relative variation that is opposite to Webb’s result (1). Even though there are some hints (coming from the analysis of other Gadolinium samples) that the first branch is preferred, this is by no means settled and further analysis is required to verify it.

Still we can speculate about the possibility that the second branch turns out to be the correct one. Indeed this would definitely be the most exciting possibility. While in itself this wouldn’t contradict Webb’s results (since Oklo probes much smaller redshift and the suggested magnitude of the variation is smaller than that suggested by the quasar data), it would have striking effects on the theoretical modelling of such variations. In fact, proof that $`\alpha `$ was once larger than today’s value would sound the death knell for any theory which models the varying $`\alpha `$ through a scalar field whose behaviour is akin to that of a dilaton. Examples include Bekenstein’s theory Bekenstein (1982) or simple variations thereof Sandvik et al. (2002); Olive and Pospelov (2002). Indeed, one can quite easily see Damour and Nordtvedt (1993); Santiago et al. (1998) that in any such model having sensible cosmological parameters and obeying other standard constraints $`\alpha `$ must be a monotonically increasing function of time. Since these dilatonic-type models are arguably the simplest and best-motivated models for varying alpha from a particle physics point of view, any evidence against them would be extremely exciting, since it would point towards the presence of significantly different, yet undiscovered physical mechanisms.

Finally, we also mention that there have been recent proposals Braxmaier et al. (2001) of more accurate laboratory tests of the time independence of $`\alpha `$ and the ratio of the proton and electron masses $`\mu `$ using monolithic resonators, which could improve current bounds by an order of magnitude or more.

However, given that there are both theoretical and experimental reasons to expect that any recent variations will be small, it is important to develop tools allowing us to measure $`\alpha `$ in the early universe, as variations with respect to the present value could be much larger then.

In what follows we focus on the analysis of CMB data allowing for possible variations of the fine-structure constant. In our previous work Avelino et al. (2001), we have carried out a joint analysis using the most recent CMB (BOOMERanG and DASI) and big-bang nucleosynthesis (BBN) data, finding evidence at the one sigma level for a smaller alpha in the past (at the level of $`10^2`$ or $`10^3`$), though at the two sigma level the results were consistent with no variation. However, as can be seen by comparing with earlier work Avelino et al. (2000b); Battye et al. (2001) (and has also been discussed explicitly in these papers), these results are quite strongly dependent on both the observational datasets and the priors one uses.

Regarding this latter issue, we point out that a recent Coc et al. (2002) improved analysis of standard BBN (focusing mostly on nuclear physics aspects) suggests that $`{}_{}{}^{7}Li`$ could lead to more stringent constraints on the baryonic density of the universe ($`\mathrm{\Omega }_b`$) than deuterium. The point made by the authors is that $`{}_{}{}^{7}Li`$ is effectively a better baryometer than $`D`$, because of difficulties in obtaining (extrapolated) primordial abundances of the latter. They then obtain values for $`w_b\mathrm{\Omega }_bh^2`$ that are considerably lower than the standard ones. These results are also corroborated by Cyburt et al. (2001). Using these results as a prior would transform our previous result Avelino et al. (2001) into a detection of a varying $`\alpha `$ at more than two sigma.

In any case, previous analyses of CMB data allowing for a varying $`\alpha `$ Avelino et al. (2000b); Battye et al. (2001); Avelino et al. (2001) have revealed some interesting degeneracies between $`\alpha `$ and other cosmological parameters, such as $`\mathrm{\Omega }_b`$ or $`H_0`$. On the other hand, a recent ‘brute-force’ exploration of a particular sector of parameter space (including quintessence models) Huey et al. (2002) seems to claim results on degeneracies between the various parameters Avelino et al. (2000b); Battye et al. (2001); Avelino et al. (2001).

While the two approaches are not really comparable (Huey et al. (2002) being rather more simplistic, as it uses no actual data and has somewhat unclear criteria for the presence of a degeneracy), this discrepancy begs the question of whether the degeneracies found in Avelino et al. (2000b); Battye et al. (2001); Avelino et al. (2001) are real ‘physical’ and fundamental degeneracies, which will remain at some level, no matter how much more accurate data one can get, or if they are simply degeneracies in the data, which won’t necessarily be there in other (better) datasets. And a related question is, of course, assuming that the degeneracies are significant, how can one get around them. We will address these issues in the following sections.

## III Current CMB and Large-Scale Structure Constraints

Here we present an up-to-date analysis of the Cosmic Microwave Background constraints on varying $`\alpha `$ as well as, for the first time, an analysis of its effects on the large-scale structure (LSS) power spectrum.

Even though this may not be entirely obvious, a varying $`\alpha `$ will have an effect on the matter power spectrum. The simplest way to understand this is to interpret the variation in $`\alpha `$ as being due to a variation in the speed of light $`c`$ (which one is always free to do Avelino and Martins (1999)).

A variation in $`\alpha `$ affects the matter power spectrum to the extent that it changes the horizon size, hence the turnover scale in the matter power spectrum. Allowing for a variation in $`\alpha `$, this is not only a function of $`\mathrm{\Omega }_m`$, $`\mathrm{\Omega }_B`$ and $`h`$ but of $`\alpha `$ as well, through the dependence of the recombination epoch on $`\alpha `$. Therefore varying alpha will produce a change in the turnover point position $`k_{rec}`$ of the matter power spectrum, hence a shift of the curve sideways, and therefore a change on the value of $`\sigma _8`$. For example a decrease in $`\alpha `$ shifts this turnover scale to smaller $`k`$, hence allowing for a decrease in $`\sigma _8`$.

By plotting the transfer functions (generated with a modified Avelino et al. (2000b) version of the CMBFAST Seljak and Zaldarriaga (1996) code which includes the effects of a varying $`\alpha `$) we find that this effect is fairly small. For $`\mathrm{\Omega }_m=0.3`$ and $`\mathrm{\Omega }_B=0.05`$ and keeping all other cosmological parameters fixed a variation of $`\alpha `$ by say $`10\%`$ from its standard value produces variations in the transfer function which are at most $`5\%`$ in restricted regions of $`k`$ (the effect on the value of $`\sigma _8`$ is even smaller).

On the other hand a change in $`\alpha `$ will modify the height of the first peak of the CMB power spectrum through the (early) ISW effect. This effect also depends on $`\mathrm{\Omega }_mh^2`$. This illustrates the interplay between a varying $`\alpha `$, $`\mathrm{\Omega }_m`$ and the value of $`\sigma _8`$. Further effects of a varying $`\alpha `$ in the CMB are a slight change in the position of the first peak due to the aforementioned change in the horizon size, plus a variation in the high-$`\mathrm{}`$ damping (due to the finite thickness of the last-scattering surface) which are also dependent on a number of cosmological parameters other than $`\alpha `$.

It should be emphasized that although these CMB and LSS constraints are in some sense complementary, and can help break degeneracies by determining other cosmological parameters, they certainly can not be blindly combined together, since the range of cosmological epochs (or redshifts) to which they are sensitive is somewhat different.

### III.1 CMB data analysis

We compare the recent CMB observations with a set of flat models with parameters sampled as follows (the value in brackets is the step size):

$$\mathrm{\Omega }_m=0.1\mathrm{}(0.1)\mathrm{}1.0$$
(6)
$$\mathrm{\Omega }_b=\mathrm{\Omega }_m\mathrm{\Omega }_{cdm}=0.009\mathrm{}(0.003)\mathrm{}0.036$$
(7)
$$\frac{\mathrm{\Delta }\alpha }{\alpha }=0.80\mathrm{}(0.01)\mathrm{}1.10$$
(8)
$$h=0.40\mathrm{}(0.05)\mathrm{}0.90$$
(9)
$$n_s=0.70\mathrm{}(0.05)\mathrm{}1.30.$$
(10)
We rescale the amplitude of fluctuations by a pre-factor $`C_{10}`$, in units of $`C_{10}^{COBE}`$, with $`0.50<C_{10}<1.40`$. Finally, we assumed a negligible re-ionization and an optical depth $`\tau _c0`$. This is in agreement with recent estimates on the redshift of re-ionization $`z_{re}6\pm 1`$ (see e.g. Gnedin (2001)).

The theoretical models are computed using a modified version of the publicly available CMBFAST program Seljak and Zaldarriaga (1996), accounting for the effects of a varying $`\alpha `$, and are compared with the recent BOOMERanG-98, DASI and MAXIMA-1 results. The power spectra from these experiments were estimated in $`19`$, $`9`$ and $`13`$ bins respectively, spanning the range $`25\mathrm{}1150`$.

For the DASI and MAXIMA-I experiment we use the publicly available correlation matrices and window functions. For the BOOMERanG experiment we assign a constant value for the spectrum in each bin $`\mathrm{}(\mathrm{}+1)C_{\mathrm{}}/2\pi =C_B`$, we approximate the signal $`C_B`$ inside the bin to be a Gaussian variable and we consider $`10\%`$ correlations between contiguous bins. The likelihood for a given cosmological model is then defined by

$$2\mathrm{l}\mathrm{n}=(C_B^{th}C_B^{ex})M_{BB^{}}(C_B^{}^{th}C_B^{}^{ex}),$$
(11)
where $`M_{BB^{}}`$ is the Gaussian curvature of the likelihood matrix at the peak. We consider $`10\%`$, $`4\%`$ and $`5\%`$ Gaussian distributed calibration errors for the BOOMERanG-98, DASI and MAXIMA-1 experiments respectively and we included the beam uncertainties by the analytical marginalization method presented in Bridle et al. (2001). We also include the COBE data using Lloyd Knox’s RADPack packages.

### III.2 LSS data analysis

In what follows, we will add to the CMB data the real-space power spectrum of galaxies in the 2dF 100k galaxy redshift survey using the data and window functions of the analysis of Tegmark et al. (2001).

To compute $`^{2dF}`$ we, evaluate $`p_i=P(k_i)`$, where $`P(k)`$ is the theoretical matter power spectrum and $`k_i`$ are the $`49`$ k-values of the measurements in Tegmark et al. (2001). Therefore we have

$$2ln^{2dF}=\underset{i}{}[P_i(Wp)_i]^2/dP_i^2,$$
(12)
where $`P_i`$ and $`dP_i`$ are the measurements and error bars in Tegmark et al. (2001) and $`W`$ is the reported $`27\times 49`$ window matrix. We restricted the analysis to a range of scales where the fluctuations are assumed to be in the linear regime ($`k<0.02h^1Mpc`$). When combining with the CMB data, we marginalize over a bias $`b`$ considered as additional free parameter.

We will also include information on $`\sigma _8`$, the RMS mass fluctuation in spheres of $`8h^1Mpc`$, obtained from local cluster number counts. There is presently no consensus on the correct value of this observable, mainly because of systematics in the calibration between cluster virial mass and temperature. For convenience of analysis, we consider $`2`$ values: an high value $`\mathrm{\Omega }_m^{0.6}\sigma _8=0.50\pm 0.05`$ in agreement with the results of Pierpaoli et al. (2001); Eke et al. (1996) and a lower one, $`\mathrm{\Omega }_m^{0.6}\sigma _8=0.40\pm 0.05`$ following the analysis of Seljak (2001); Viana et al. (2001).

We attribute a likelihood to each value of $`\delta \alpha /\alpha `$ by marginalizing over the nuisance parameters. We then define our $`68\%`$ ($`95\%`$), confidence levels to be where the integral of the likelihood is $`0.16`$ ($`0.025`$) and $`0.84`$ ($`0.975`$) of the total value (see e.g. Melchiorri et al. (2000)).

### III.3 Results

In a previous work Avelino et al. (2001) we produced likelihood contours in the $`\mathrm{\Omega }_bh^2\alpha /\alpha _0`$ plane by analyzing the recent BOOMERanG and DASI CMB datasets and by including two priors to the analysis: flatness and $`h=0.65\pm 0.2`$. Our results were consistent with the baryon abundance obtained from big-bang nucleosynthesis (BBN) and we constrained variations in $`\alpha `$ at $`z1000`$ at level of about $`10\%`$. In Fig. 1 we plot constraints on this plane, as well as on the $`n_s\alpha /\alpha _0`$ and $`h\alpha /\alpha _0`$ planes with a similar analysis, but including this time the MAXIMA-I dataset.

From these results we can see that the inclusion of the MAXIMA-I data doesn’t significantly change our previous constraints. In fact, even if the analysis of the MAXIMA-I data alone suggests a higher value of the baryon fraction ($`\mathrm{\Omega }_bh^20.030\pm 0.005`$ Stompor (2001)), the combined analysis with DASI and BOOMERanG still suggests a low value of $`\mathrm{\Omega }_bh^20.023`$. This result is in agreement with previous analysis, e.g. Tegmark et al. (2001).

From last two panels of Fig. 1 we also see that, in the set of models we are considering, there is a clear correlation between variations in $`\alpha `$ and changes in the scalar spectral index $`n_s`$ and the Hubble parameter $`h`$. We will see in the next section that this degeneracy can be broken by the future and more accurate measurements from satellite experiments like MAP MAP (2002) or the Planck Surveyor Planck (2002).

However, since variations in $`n_s`$ and $`h`$ affect the shape, position and amplitude of the matter power spectrum (irrespective of changes in $`\alpha `$), we can in principle use the data from galaxy clustering and local cluster abundances in order to break these degeneracies and infer stronger constraints on $`\alpha `$. Regarding the RMS amplitude of mass fluctuations on scales of $`8h^1Mpc`$, the two effects are competitive: an increase in $`n_S`$ will increase $`\sigma _8`$, while lowering $`h`$ decreases it. As we can see from the first panel of Fig. 2, the effect from $`h`$ is stronger and the final result is that a decrease in $`\alpha `$ generally allows for lower amplitudes of $`\sigma _8`$.

Furthermore, as we can see from Fig. 2, a certain degree of degeneracy is present in the CMB data between $`\alpha `$ and the shape parameter $`\mathrm{\Omega }_mh`$. This degeneracy can be optimally broken by incorporating the data from the 2dF galaxy survey. As we can see in the bottom panel, including the 2dF data shrinks the contours around $`\alpha /\alpha _01`$ and $`\mathrm{\Omega }_mh0.2`$, In other words, there is a clear distinction between the CMB and LSS data: while for the CMB data a negative variation of $`\alpha `$ is preferred, the opposite happens for the LSS data. When the two datasets are combined, the best-fit model, in the $`\mathrm{\Omega }_mh\alpha /\alpha _0`$ plane, is quite close to the standard one.

We should emphasize at this point that in doing this we are not combining direct constraints on the parameter $`\alpha `$ itself, obtained through both methods, to obtain a tighter constraint. As mentioned above this can not be done, since the CMB and LSS analyses are sensitive to the values of $`\alpha `$ at different redshift ranges, so there is no reason why these values should be the same. Additionally, there is no well-motivated theory that could relate such variations at different cosmological epochs. All that one could do at this stage would be to assume some toy model where a certain behaviour would occur, but this would mean introducing various additional parameters, thus weakening the analysis. Hence we chose not to pursue this path and leave the analysis as model-independent as possible.

What we are doing is using additional information (which is also sensitive to $`\alpha `$) to better constrain other parameters in the underlying cosmological model, such as $`n_s`$, $`h`$ and the densities of various matter components, which we can reliably assume are unchanged throughout the cosmological epochs in question. In other words, we are simply selecting more stringent priors for our analysis, in a self-consistent way.

The constraints obtained by combined analysis are reported in Table 1. Our main result from this table is that, as one would expect, when constraints from other and independent cosmological datasets are included in the CMB analysis, the constraints on variations on $`\alpha `$ become significantly stronger.

## IV Fisher Matrix Analysis

The precision with which the forthcoming satellite experiments MAP MAP (2002) and Planck Planck (2002) will be able to determine variations in $`\alpha `$ can be readily estimated with a Fisher Matrix Analysis (FMA). Some authors have already performed such an analysis in the past Hannestad (1999); Kaplinghat et al. (1999): however, their analysis was based on a different set of cosmological parameters and assumed cosmic variance limited measurements. In our FMA we also take into account the expected performance of the MAP and Planck satellites and we make use of a cosmological parameters set which is well adapted for limiting numerical inaccuracies. Furthermore, the FMA can provide useful insight into the degeneracies among different parameters, with minimal computational effort.

### IV.1 Analysis Setup

We characterize the cosmological model by a 7 dimensional parameter set, given by

$$𝚯=(\alpha ,\omega _b,\omega _m,\omega _\mathrm{\Lambda },,n_s,Q),$$
(13)
where $`\omega _b\mathrm{\Omega }_bh^2`$ is the physical baryonic density, $`\omega _m(\mathrm{\Omega }_{\mathrm{cdm}}+\mathrm{\Omega }_b)h^2`$ the energy density in matter and $`\omega _\mathrm{\Lambda }`$ the energy density due to a cosmological constant. Here $`h`$ denotes the Hubble parameter today, $`H_0100h`$ km s<sup>-1</sup> Mpc<sup>-1</sup>. The quantity $`\mathrm{}_{\mathrm{ref}}/\mathrm{}`$ is the ‘shift’ parameter (see Melchiorri and Griffiths (2001); Bowen et al. (2001) and references therein), which gives the position of the acoustic peaks with respect to a flat, $`\mathrm{\Omega }_\mathrm{\Lambda }=0`$ reference model, $`n_s`$ is the scalar spectral index and $`Q=<\mathrm{}(\mathrm{}+1)C_{\mathrm{}}>^{1/2}`$ denotes the overall normalization, where the mean is taken over the multipole range $`2\mathrm{}2000`$.

The shift parameter $``$ depends on $`\mathrm{\Omega }_m`$, on the curvature $`\mathrm{\Omega }_\kappa 1\mathrm{\Omega }_\mathrm{\Lambda }\mathrm{\Omega }_m\mathrm{\Omega }_{\mathrm{rad}}`$ through

$``$ $`=`$ $`2\left(1{\displaystyle \frac{1}{\sqrt{1+z_{\mathrm{dec}}}}}\right)`$ (14)
$`\times {\displaystyle \frac{\sqrt{|\mathrm{\Omega }_\kappa |}}{\mathrm{\Omega }_m}}{\displaystyle \frac{1}{\chi (y)}}\left[\sqrt{\mathrm{\Omega }_{\mathrm{rad}}+{\displaystyle \frac{\mathrm{\Omega }_m}{1+z_{\mathrm{dec}}}}}\sqrt{\mathrm{\Omega }_{\mathrm{rad}}}\right],`$
where $`z_{\mathrm{dec}}`$ is the redshift of decoupling, $`\mathrm{\Omega }_{\mathrm{rad}}`$ is the energy parameter due to radiation ($`\mathrm{\Omega }_{\mathrm{rad}}=4.1310^5/h^2`$ for photons and 3 neutrinos) and

$`y`$ $`=`$ $`\sqrt{|\mathrm{\Omega }_\kappa |}{\displaystyle _0^{z_{\mathrm{dec}}}}𝑑z`$
$`[\mathrm{\Omega }_{\mathrm{rad}}(1+z)^4+\mathrm{\Omega }_m(1+z)^3+\mathrm{\Omega }_\kappa (1+z)^2+\mathrm{\Omega }_\mathrm{\Lambda }]^{1/2}.`$
The function $`\chi (y)`$ depends on the curvature of the universe and is $`y`$, $`\mathrm{sin}(y)`$ or $`\mathrm{sinh}(y)`$ for flat, closed or open models, respectively. Inclusion of the shift parameter $``$ into our set of parameters takes into account the geometrical degeneracy between $`\omega _\mathrm{\Lambda }`$ and $`\omega _m`$ Efstathiou (2001). With our choice of the parameter set, $``$ is an independent variable, while the Hubble parameter $`h`$ becomes a dependent one.

We assume throughout purely adiabatic initial conditions and we do not allow for a tensor contribution. In the FM approach, the likelihood distribution for the parameters $`𝚯`$ is expanded to quadratic order around its maximum. We denote this maximum likelihood (ML) point by $`𝚯_\mathrm{𝟎}`$ and call the corresponding model our “ML model”, with parameters $`\omega _b=0.0200`$ (hence $`\mathrm{\Omega }_b=0.0473`$), $`\omega _m=0.1267`$ (hence $`\mathrm{\Omega }_m=0.3000`$), $`\omega _\mathrm{\Lambda }=0.2957`$ (hence $`\mathrm{\Omega }_\mathrm{\Lambda }=0.7000`$ and $`h=0.65`$), $`=0.9628`$, $`n_s=1.00`$, $`Q=1.00`$. For the value of $`z_{\mathrm{dec}}`$ in eq. (IV.1)(which is weakly dependent on $`\omega _b`$ and $`\omega _{\mathrm{tot}}`$) we have used the fitting formula from Hu and Sugiyama (1995). For the ML model we have $`z_{\mathrm{dec}}=1115.52`$.

Proceeding as described in Bowen et al. (2001), we then calculate the Fisher information matrix

$$F_{ij}=\underset{\mathrm{}=2}{\overset{\mathrm{}_{\mathrm{max}}}{}}\frac{1}{\mathrm{\Delta }C_{\mathrm{}}^2}\frac{C_{\mathrm{}}}{\mathrm{\Theta }_i}\frac{C_{\mathrm{}}}{\mathrm{\Theta }_j}|_{𝚯_\mathrm{𝟎}}$$
(16)
The quantity $`\mathrm{\Delta }C_{\mathrm{}}`$ is the standard deviation on the estimate of $`C_{\mathrm{}}`$:

$$\mathrm{\Delta }C_{\mathrm{}}^2=\frac{2}{(2\mathrm{}+1)f_{\mathrm{sky}}}(C_{\mathrm{}}+B_{\mathrm{}}^2)^2.$$
(17)
The first term is the cosmic variance, arising from the fact that we exchange ensemble average with a spatial average. The second term takes into account the expected error of the experimental apparatus Knox (1995); Efstathiou (2001):

$$B_{\mathrm{}}^2=\underset{c}{}w_ce^{\mathrm{}(\mathrm{}+1)/\mathrm{}_c^2}.$$
(18)
The sum runs over all channels of the experiment, with the inverse weight per solid angle $`w_c^1(\sigma _c\theta _c)^2`$ and $`\mathrm{}_c\sqrt{8\mathrm{ln}2}/\theta _c`$, where $`\sigma _c`$ is the sensitivity (in $`\mu `$K) and $`\theta _c`$ is the FWHM of the beam (assuming a Gaussian profile) for each channel. Furthermore we can neglect the issues arising from point sources, foreground removal and galactic plane contamination assuming that once they have been taken into account we are left with a “clean” fraction of the sky given by $`f_{\mathrm{sky}}`$.

The experimental parameters are summarized in Table 2. We use the 3 higher frequency MAP channels and the first 3 channels of the Planck High Frequency Instrument (HFI). Adding the 3 higher frequency channels of the HFI and the 3 channels of Planck’s Low Frequency Instrument leaves the expected errors unchanged: therefore they can be used for foreground removal, consistency checks, etc, leaving the HFI channels for cosmological use.

For Gaussian fluctuations, the covariance matrix is then given by the inverse of the Fisher matrix, $`C=F^1`$ Bond et al. (1999). The $`1\sigma `$ error on the parameter $`\mathrm{\Theta }_i`$ with all other parameters marginalised is then given by $`\sqrt{C_{ii}}`$. If all other parameters are held fixed to their ML values, the standard deviation on parameter $`\mathrm{\Theta }_i`$ reduces to $`\sqrt{1/F_{ii}}`$ (conditional value). Other cases, in which some of the parameters are held fixed and others are being marginalized over can easily be worked out.

A case of interest is the one in which all parameters are being estimated jointly: then the joint error on parameter $`i`$ is given by the projection on the $`i`$-th coordinate axis of the 7-dimensional hyper-ellipse which contains a fraction $`\gamma `$ of the joint likelihood. The equation of the hyper-ellipse is

$$(𝚯𝚯_\mathrm{𝟎})𝐅(𝚯𝚯_\mathrm{𝟎})^t=q_{1\gamma },$$
(19)
where $`q_{1\gamma }`$ is the quantile for the probability $`1\gamma `$ for a $`\chi ^2`$ distribution with 7 degrees of freedom. For $`\gamma =0.683`$ ($`1\sigma `$ c.l.) we have $`q_{1\gamma }=8.18`$.

The FMA assumes that we are expanding the likelihood function at the right point, ie that the parameters values of the true model are in the vicinity of $`𝚯_\mathrm{𝟎}`$. The validity of the results depends on this assumption, as well as on the assumption that the $`a_\mathrm{}m`$’s are independent Gaussian random variables. If the FM predicted errors are small enough, the method is self-consistent and we can expect the FM prediction to reproduce in a correct way the exact behaviour. This is indeed the case for the present analysis, with the notable exception of $`\omega _\mathrm{\Lambda }`$, which suffers from the geometrical degeneracy (see next section).

Special care must be taken in computing the derivatives of the power spectrum with respect to the cosmological parameters. Numerical errors in the spectra can lead to larger derivatives, which would artificially break degeneracies among parameters. In the present work we implement double–sided derivatives, which diminish the truncation error from second order to third order terms. The choice of the step size is a trade-off between truncation error and numerical inaccuracy dominated cases. For an estimated numerical precision of the computed models of order $`10^4`$, the step size should be approximately 5% of the parameter value Press et al. (1992). It turns out that for derivatives in direction $`\alpha `$ and $`n_s`$ the step size can be chosen to be as small as $`0.1\%`$. As for the other parameters, the accuracy is limited by the fact that differentiating around a flat model requires computing open and closed models, which are calculated using different numerical techniques. The relative numerical noise is therefore much larger. After several tests, we chose step sizes varying from $`1\%`$ to $`5\%`$ for $`\omega _b,\omega _m,\omega _\mathrm{\Lambda }`$, and $``$. This choice gives derivatives with an accuracy of about $`0.5\%`$. The derivatives with respect to $`Q`$ are exact, being the power spectrum itself.

### IV.2 Analysis results

#### IV.2.1 FMA forecast

Table 3 summarizes the results of our FMA. MAP will be able to constrain variations in $`\alpha `$ at the time of last scattering to within $`2\%`$ ($`1\sigma `$, all others marginalised). This corresponds to an improvement of a factor of 3 relative to the limits presented in the previous section. Planck will narrow it down to about half a percent. If all other parameters are supposed to be known and fixed to their ML value, then a factor of 10 is to be gained in the accuracy of $`\alpha `$ (compare the columns labelled “fixed” in table 3). However, if all parameters are being estimated jointly, the accuracy on variations in $`\alpha `$ will not go beyond $`1\%`$, even for Planck (column “joint”).

The parameters $`\omega _b,`$ and $`n_s`$ suffer from partial degeneracies with $`\alpha `$, which are discussed in more detail in the next section. This is only partially reflected in the marginalized errors of table 3. Correlations among the parameters play an important role: within the limit of the quadratic order approximation, they are fully described by the FM.

The geometrical degeneracy limits the accuracy on $`\omega _m`$ and $`\omega _\mathrm{\Lambda }`$. The degeneracy is so severe that the error on $`\omega _\mathrm{\Lambda }`$ is very unsensitive to the experimental details. From the FMA point of view, this happens because the derivative of the spectrum with respect to $`\omega _\mathrm{\Lambda }`$ vanishes for $`\mathrm{}50`$. Therefore probing higher multipoles does not help for the purpose of better constraining the cosmological constant. We emphasize once more that such a large error cannot be trusted to be accurate in any respect: it just signals a very large inaccuracy in $`\omega _\mathrm{\Lambda }`$. The errors on all other parameters, however, are small enough to justify the self-consistency of the FMA approach.

The power of an experiment can be roughly assessed by looking at the eigenvalues $`\lambda _i`$ and eigenvectors $`𝐮^{(𝐢)}`$ of its FM: the error along the direction in parameter space defined by $`𝐮^{(𝐢)}`$ (principal direction) is proportional to $`\lambda _i^{1/2}`$. But we are interested in determining the errors on the physical parameters rather then on their linear combinations along the principal directions. Therefore in the ideal case we want the principal directions to be as much aligned as possible to the coordinate system defined by the physical parameters. We display in Table 4 eigenvalues and eigenvectors of the FM for MAP and Planck. Planck’s errors, as measured by the inverse square root of the eigenvalues, are smaller by a factor of about 4 on average. For 6 of the 7 eigenvectors Planck also obtains a better alignment of the principal directions with the axis of the physical parameters. This is established by comparing the ratios between the largest (marked with an asterisk in Table 4) and the second largest (marked with a dagger) cosmological parameters’ contribution to the principal directions. This is of course in a slightly different form the statement that Planck will measure the cosmological parameters with less correlations among them.

#### IV.2.2 Degeneracies with other parameters

In previous work Avelino et al. (2000b) some of the present authors observed a degeneracy in the Boomerang-98 and Maxima-1 data between $`\alpha `$ and $`\omega _b`$. This degeneracy also shows up in the present analysis (see Fig. 1, top panel). The question we ask is: is this a fundamental degeneracy, or is it only in the data?

We have performed a FMA with experimental parameters chosen as to mimic a Boomerang-type experiment ($`f_{\mathrm{sky}}=0.02,\sigma =15\mu \mathrm{K},\theta =9.2^{},\mathrm{}_{\mathrm{max}}=1000`$). If the degeneracy is due to the limited precision of the present-day experimental data, we expect the degeneracy to disappear as we move from Boomerang, to MAP, to Planck. Fig. 3 (top panel) shows $`1\sigma `$ joint confidence curves (all other parameters marginalized) in the $`\alpha /\alpha _0\omega _b`$ plane for the FM simulated Boomerang, MAP and Planck (from the outside to the center, respectively). The curve for Boomerang is to be compared with the $`1\sigma `$ contour of the data analysis (Figure 1, top panel). Although the FM ellipse is centred by construction at the ML model value, it is in qualitative agreement with the result of the data analysis. As we move to MAP, the degeneracy shrinks but is still there: only higher multipole measurements from Planck can break it.

The same behaviour is observed in the $`\alpha /\alpha _0n_s`$ plane (Fig. 3, middle panel; compare with Fig. 1, middle panel). Again, the observed degeneracy between $`\alpha `$ and $`n_s`$ is clearly revealed by the FMA for Boomerang.

In the bottom panel of Fig. 3 we investigate the important degeneracy between $``$ and $`\alpha /\alpha _0`$. These two parameters are very highly correlated (correlation $`0.99`$ for all experiments), because an increase of $`\alpha `$ displaces the acoustic peaks to higher multipoles. This effect is mainly due to the increased redshift of last scattering Kaplinghat et al. (1999); Avelino et al. (2000b). On the contrary, an increase of $``$ shifts the peaks toward smaller $`\mathrm{}`$ values, because of the change in the angular diameter distance relation. However, an increase in $`\alpha `$ also produces a decrease in the damping at high multipoles, which can be used to break the degeneracy, as it is the case for Planck. This is reflected in the different amplitude for the two derivatives, which are otherwise perfectly in phase, as can be seen in Fig. 4. This degeneracy can clearly be identified because of our choice of the parameter set $`𝚯`$, which includes the shift parameter rather than the Hubble or curvature parameters. This emphasizes the importance of a correct choice of the parameter set in the context of a FMA.

Comparison to previous works is only partially possible, because of the differences in the analysis discussed above. Our detailed analysis confirms however the conclusions in refs. Hannestad (1999); Kaplinghat et al. (1999), which found that a cosmic variance limited experiment could obtain a precision on $`\alpha `$ of order $`10^310^2`$. We have also shown that there is much to be gained from using prior knowledge about the other parameters in the determination of $`\alpha `$ via CMB measurements. The improvement in accuracy is about a factor $`50`$ for both MAP and Planck.

## V Conclusions

We have provided an up-to-date analysis of the effects of a varying fine-structure constant $`\alpha `$ in the CMB, focusing on the issue of the degeneracies with other cosmological parameters, and of how these can be broken.

We have shown that the currently available data is consistent with no variation of $`\alpha `$ from the epoch of recombination to the present day, though interestingly enough the CMB and LSS datasets seem to prefer, on their own, variations of $`\alpha `$ with opposite signs. Whether or not this statement has any physical relevance (beyond the results of the statistical analysis) is something that remains to be investigated in more detail. In any case, any such (relative) variation is constrained to be less than about $`4\%`$, so a best-fit or ‘concordance’ model with $`\alpha `$ exactly constant will require, at most, some slight deviations of other cosmological parameters from the ‘standard’ values obtained from analyses which don’t allow for $`\alpha `$ variation.

On the other hand, the prospects for the future are definitely bright. In the short term, the imminent VSA and CBI data should be able to provide some improvement on the current results. The dataset we have used (BOOMERanG, MAXIMA and DASI) all have the common feature that their error bars are smallest for data points around the first Doppler peak and larger for smallest angular scales. Now, as we have explicitly shown above (and was already suggested in Avelino et al. (2000b)), the first Doppler peak is not a vary accurate ‘$`\alpha `$-meter’, due to the degeneracy with the shift parameter. Thus datasets in which points around the first peak will somewhat dominate the statistical analysis are not optimal for $`\alpha `$ estimation. In this regard, VSA and CBI should be useful because they can provide a significant number of data points on small angular scales with relatively small error bars, hence minimizing this problem.

In the longer term, the forthcoming satellite experiments will provide a dramatic improvement on these results. We have performed a Fisher Matrix Analysis using a well adapted parameter set and realistic experimental characteristics for the upcoming MAP and Planck satellite missions. The results of our forecast are that MAP and Planck will be able to constrain variations in $`\alpha `$ within $`2.2\%`$ and $`0.4\%`$, respectively ($`1\sigma `$ c.l., all other parameters marginalized). If all parameters are being estimated simultaneously, then this limits increase to about $`6.4\%`$ and $`1.1\%`$, respectively. The analysis of the presently observed degeneracies between $`\alpha `$ and $`\omega _b`$, $`n_s`$ comes to the conclusion that measurement of higher multipoles will allow to break it. We have also identified an important degeneracy between $`\alpha `$ and the shift parameter.

To conclude, we have provided a thorough analysis of the effects of cosmological parameter degeneracies in CMB measurements of the fine-structure constant $`\alpha `$, and quantified the importance of these degeneracies. We have also explicitly discussed two ways in which these degeneracies can be circumvented, namely acquiring better data (the easy solution, at least from the theorists’ point of view) or combining the CMB data with other cosmological datasets which can provide constraints on other cosmological parameters (the ‘brute-force’ solution)

In a follow-up paper, we will discuss a third way in which these degeneracies can be lifted, namely including CMB polarization data Rocha et al. (2002) (the more elegant solution in principle, though it’s yet to be realized in practice). These tools, together with other measurements coming from BBN Avelino et al. (2000b) and quasar and related data Murphy et al. (2001a); Webb et al. (2001) offer the exciting prospect of being able to map the value of $`\alpha `$ at very many different cosmological epochs, which would allow us to impose very tight constraints on higher-dimensional models where these variations are ubiquitous.

###### Acknowledgements.

We are grateful to Ruth Durrer, Stefano Foffa, Yasmin Friedmann, Arthur Kosowsky, Lyman A. Page, Paul Shellard, David N. Spergel, Carsten van de Bruck and John Webb for many useful discussions. This work is partially supported by the European Network CMBNET. C.M. is funded by FCT (Portugal), under grant no. FMRH/BPD/1600/2000. A.M. and R.B. are supported by PPARC. R.T. is partially supported by the Schmidheiny Fundation. G.R. is funded by a Leverhulme Fellowship. This work was performed on COSMOS, the Origin2000 owned by the UK Computational Cosmology Consortium, supported by Silicon Graphics/Cray Research, HEFCE and PPARC.