text:
# Halo Cores and Phase Space Densities: Observational Constraints on Dark Matter Physics and Structure Formation

## 1 Introduction

Recent work has drawn attention to the apparent conflict between predictions of collisionless Cold Dark Matter (CDM) on small scales and observations of rotation curves of dark matter dominated galaxies. Numerical simulations suggest that in a CDM cosmology, dark matter halos should have steeply rising central cusps ($`\rho r^{1.5}`$) and high densities. While the observational conclusions are somewhat ambiguous on the innermost profile shapes (e.g. Swaters et al. 2000, van den Bosch et al. 2000, van den Bosch & Swaters 2000, Borriello & Salucci 2000, Dalcanton & Bernstein 2000, Burkert 1997), rotation curves consistently imply low characteristic halo densities in the central regions. Because $`\rho (<r)(V(r)/r)^2`$, the characteristic slope of the rotation curve is proportional to the square root of the mean enclosed density. Observations of dark matter dominated galaxies consistently find rotation curves which rise with $`V/R1020\mathrm{km}/\mathrm{s}`$, suggesting much lower characteristic densities than implied by simulations of halos in viable CDM cosmologies, where $`V/R3050\mathrm{km}/\mathrm{s}`$ (Moore et al. 1998, 1999, Navarro et al. 1996, 1997).

The growing belief that there truly is a conflict between theory and observations has led to a renaissance in exploring alternative models for dark matter. By violating either the “collisionless” or “cold” properties of traditional CDM, or by considering additional exotic properties, many authors have sought to preserve the successes of CDM on large scales, while modifying the manifestations of dark matter on small scales (Spergel & Steinhart 1999, Hogan & Dalcanton 2000, Sommer-Larson & Dolgov 1999, Mohapatra & Teplitz 2000, Peebles 2000, Goodman 2000, Riotto & Tkachev 2000, Hu et al. 2000, Shi & Fuller 1999, Colin et al. 2000.)

In this paper, we place broad constraints upon these alternatives to CDM, by revisiting observations of the structure and phase space density of halos over a wide range of scales. Many of the above alternative dark matter scenarios make specific predictions for the sizes of dark matter cores as a function of mass scale (see Spergel & Steinhart 1999, Hogan & Dalcanton 2000, Hannestad 1999, Burkert 2000, Kochanek & White 2000, Yoshida et al. 2000, Moore et al. 2000, although some of these are in conflict with each other, particularly regarding the long term stability of self-interacting cores). We confront these predictions with existing limits on the scale of inner halo cores in §2, and argue that neither packing of phase space nor highly collisional dark matter can be primarily responsible for the observed behavior of dark matter cores at all scales.

In addition to discussing the scaling behavior of dark matter cores, we focus upon trends of phase space density $`Q`$ as a probe of structure formation history. In §3.1 we summarize the statistical and dynamical behavior of $`Q`$ in hierarchical clustering, including a simple argument predicting the decrease of $`Q`$ with mass and velocity dispersion. Current observational limits on the variation of phase space density with mass are reviewed and summarized in §3.2. We show in §4 that the phase space density of the dark matter halos is a very strongly declining function of mass (consistent with the nearly constant density seen across a similar mass scale in Firmani et al. 2000). This behavior is exactly as predicted by models where halos formed hierarchically, with successive mergers leading to phase mixing and dilution of the coarse-grained distribution function. We also note that the observed decline in phase space density for dark matter halos and their predicted density profiles resembles that observed in the baryon-dominated central regions of giant elliptical galaxies, and that by analogy, the structure of dark matter halos may also suffer from the systematic departures from homology similar seen in ellipticals. Finally, we use the maximum observed phase space densities to derive a limit on particle mass of $`>700\mathrm{eV}`$ for relativistically decoupled thermal relics.

## 2 Cores in Collisionless and Collisional Solutions

Nonsingular dark matter central halo profiles appear in a wide range of environments, a fact which already argues against the simplest explanation of cores based on “phase packing.” We illustrate this point using two such scenarios for limiting the maximum density of cores in dark matter halos (Hogan & Dalcanton 2000), namely generalized warm collisionless dark matter and warm collisional dark matter.

In the first scenario, dark matter particles have some primordial velocity dispersion, leading to a “phase space density” $`Q\rho /v^2^{3/2}`$ whose coarse-grained value is then either preserved or decreased during subsequent epochs of structure formation (see §3.1, equation 2). For halo material with a roughly isothermal, isotropic velocity dispersion $`\sigma `$, the primordial phase density therefore sets a minimum core size corresponding to “phase packing” the material:

$$r_{c,min}^2=\frac{\sqrt{3}}{4\pi GQ_0}\frac{1}{\sigma ^2^{1/2}}.$$
(1)
If the halo has gone through a period of violent relaxation or shock heating which thoroughly heats the matter ($`Q_0Q^{}<Q_0`$ everywhere), then the core may be larger than $`r_{c,min}`$.

In the second scenario, the dark matter particles are highly collisional, and thus they behave as a gas111In Hogan & Dalcanton (2000), we argued that the moderately collisional case would not be stable. A constant-density core requires a temperature gradient for support. However, if dark matter were only moderately collisional, particles would diffuse outwards in less than a Hubble time, erasing the necessary temperature gradient. This diffusive heat conduction would runaway to form a dense central cusp. Thus, we restrict ourselves to the highly collisional case, where the diffusion time is sufficiently long for cores to be stable over a Hubble time.. The equilibrium configuration of the halo will therefore be the solutions of a classical, self-gravitating, ideal gas. For a polytropic equation of state ($`p\rho ^\gamma `$) at all radii (i.e. constant entropy), and assuming the system is non-relativistic and adiabatic ($`\gamma =5/3`$), the density profile of the halo becomes that of a Lane-Emden polytrope like a giant degenerate dwarf star. (The material here is not degenerate but is on an adiabat again limited by the initial $`Q_0`$.) For a total mass M and radius R, solutions of the Lane-Emden equation give a central density $`\rho =1.43M/R^3`$ and central pressure $`p=0.77GM^2/R^4`$. Using the equation of state, $`MR^3`$, and thus the characteristic velocity of the sphere is $`\sigma ^2=GM/\sqrt{3}RR^4`$, which is a similar scaling to equation 1.

For both of these cases, we predict a simple scaling relationship between the size of dark matter cores and the characteristic velocity dispersion of the system. In general, higher mass, high velocity dispersion systems should have smaller cores, with $`r_{core}1/\sqrt{\sigma }`$. For dwarf spheroidals ($`\sigma 10\mathrm{km}/\mathrm{s}`$), rotating dwarf and LSB galaxies ($`\sigma 50100\mathrm{km}/\mathrm{s}`$), and clusters ($`\sigma 1000\mathrm{km}/\mathrm{s}`$), we expect core sizes to scale like roughly 10:2-3:1, if in fact the core size is set more by primordial conditions than by subsequent heating/relaxation. If we take a fiducial core size of $`1\mathrm{kpc}`$ for dwarf spheroidals, then rotating dwarfs and LSBs would have 300–500$`\mathrm{pc}`$ cores, and primordial galaxy cluster cores would be microscopic (100$`\mathrm{pc}`$) and observationally undetectable. Likewise, if we set the fiducial scale at clusters, with $`r_{core}50\mathrm{kpc}`$, then dwarf spheroidals would have implausibly large cores (0.5$`\mathrm{Mpc}`$, comparable to the separation between giant spirals in the Local Group, and inconsistent with possible detections of extra-tidal stars in nearby spheroidals; e.g. Majewski et al. 2000).

Even if we are detecting cores limited by primordial phase space density in dwarf spheroidals, similar cores in larger systems would be undetectable. Detectable cores in more massive galaxies and clusters must therefore be due to other processes, for example heating and/or violent relaxation during formation. This implies that the properties of warm or self-interacting dark matter would be most directly probed by the properties of dwarf spheroidals, not rotating dwarf and low surface brightness galaxies. On the other hand, if the cores seen in rotating LSBs are due to the effects of self-interaction, then the dense halos of dwarf spheroidals may be the end result of core collapse and may indeed be singular. We will return to many of these points below, when we consider the phase space density of dark matter cores.

## 3 Evolution and Scaling of Phase Space Density

### 3.1 Predictions of Phase Space Density in Gravitational Clustering

We may characterize systems by their mass per volume of phase space, or “phase space density” $`Q`$, a quantity which obeys important symmetries and in some circumstances admits detailed observational constraints. For a collisionless, dissipationless gas, the fine-grained value of $`Q`$ does not change, and the evolution of the system consists of various distortions of the “phase sheet” occupied by particles, in such a way that the coarse grained phase space density can only decrease. This is related in a straightforward way to the increase of thermodynamic entropy; for a uniform monatomic ideal thermal gas of $`N`$ particles,

$$S=kN[\mathrm{ln}(Q)+\mathrm{constant}].$$
(2)
The value of the fine-grained phase space density is fixed when the dark matter particles become microscopically collisionless. This quantity $`Q_0`$ is therefore a primordial relic reflecting the interactions and masses of the dark matter particles. Unfortunately, the primordial value of the fine-grained $`Q_0`$ cannot be directly measured;222In some models $`Q_0`$ could be measured in a direct laboratory detection of dark matter particles, or from effects of gravitational lensing of projected catastrophes of surface density (Hogan 1999). the astronomically observable quantity is the mean coarse-grained phase space density, which can be estimated dynamically from rotation curves, stellar velocity dispersions, gas emission or gravitational lensing, using the measured rms velocity and density. We adopt units for $`Q`$ most closely related to these observable quantities: $`Q\rho /v^2^{3/2}`$. This coarse-grained phase space density is a strict lower limit to the primordial $`Q`$, and we may use observations to set physically interesting constraints on primordial conditions.

In addition to probing initial conditions, observations of $`Q`$ can be made as a function of mass, allowing us to follow the evolution of the coarse-grained phase space density in hierarchical clustering, i.e. to study the process of the clustering itself. We may make a first order prediction for the expected evolution of $`Q`$ using the following simple argument.

Suppose that structures form by hierarchical merging of systems, each of which is in approximate virial equilibrium. Without gravity or dissipation, a merged system could be carefully, adiabatically assembled from parts to almost eliminate any increase in entropy or decrease of $`Q`$. Two blobs can be slowly merged into one, and if they don’t mix, the entropy of the new merged blob is just the sum of the two initial entropies, and therefore $`Q`$ is preserved. The total phase space volume is just the sum of the two initial volumes, since nothing in velocity space changes. The addition of gravitational dynamics to the picture, however, guarantees that $`Q`$ decreases steadily as a power of increasing mass, as the increase in mass requires an increase in velocity dispersion to maintain virial equilibrium.

We may place limits on the minimum possible decrease in $`Q`$ which is compatible with maintaining virial equilibrium during the merging hierarchy. Consider the merger of a blob 1 and a smaller blob 2 into a third blob 3, with the phase density of each $`Q_i=M_i/𝒱_i`$ depending on the volume $`𝒱_i`$ occupied in phase space. All three blobs have a homologous structure and are in virial equilibrium with characteristic size $`R_i`$ and velocity dispersion $`\sigma _i`$. Set $`M_2=ϵM_1`$, hence $`M_3=(1+ϵ)M_1`$. We assume that as the small blob 2 sinks into blob 1, its material is tidally stripped. The stripping of material with density $`\rho _2`$ occurs at a radius where $`\rho _1\rho _2`$, and thus this gentle merging process approximately preserves the physical space density as each layer is homologously added to form the larger system. Empirically, the density of dark matter halos is indeed observed to be approximately constant on mass scales from rotating dwarfs to clusters (Firmani et al. 2000), and thus this gentle merging assumption may not be a terrible deviation from the truth.

With our assumption of constant space density during merging, the added material leads to an increase in volume $`R_3=(1+ϵ)^{1/3}R_1`$. However, in order to preserve virial equilibrium $`\sigma ^2M/R`$ it is also necessary to grow the size of the blob in velocity space, $`\sigma _3=(1+ϵ)^{1/3}\sigma _1`$. Thus, the system responds by a symmetrical fractional increase in $`R`$ and $`\sigma `$. The phase volume increases by the factor $`𝒱_3=R_3^3\sigma _3^3=(1+ϵ)^2𝒱_1`$, implying that systems assembled from this hierarchy follow $`𝒱M^2`$, and hence they obey333Note that this is a fundamentally different assumption than made by Hernquist et al. 1993 for a similar phase space evolution calculation. Following Hausman & Ostriker 1978, and assuming parabolic orbits for merging of identical galaxies, they argued that the velocity dispersion of the merged remnant is identical to the velocity dispersions of the progenitors, and thus that the gravitational radius of the virialized remnant must increase in proportion to the mass. This implies that the both the physical density and the phase density of the remnants should decrease as $`M^2`$. We argue that this is not the quietest limiting hierarchy, although some such mergers certainly occur.

$$QM^1\sigma ^3R^3.$$
(3)
No matter how gradually the assembly is done a certain amount of extra phase wrapping is needed to achieve this equilibrium, decreasing $`Q`$.

While not rigorously proved, we argue that equation 3 must be close to the slowest possible decline in the coarse-grained phase space density with increasing mass. We have assumed the quietest form of merging, invoking only enough phase mixing to bring the system into virial equilibrium. Other models for the evolution of $`Q`$ may be invoked (e.g. Hernquist et al. 1993), but in general these should involve more phase-mixing, and thus more sharply declining values of $`Q`$. For example, although the predicted scaling for $`Q`$ is the same as equation 3 even when the two masses are comparable (i.e. $`ϵ`$ is not much less than one), in such a situation one expects “violent relaxation” rather than tidal stripping, leading to additional phase mixing and steeper evolution in $`Q`$. Likewise, while the assumption of homology has not been justified in detail, and indeed no account has been taken of changes in density profile shapes and other degrees of freedom available to real systems, these extra degrees of freedom must come with a net lowering of coarse-grained $`Q`$ relative to the quiet, homologous case used in our derivation. A shallower decrease than equation 3 would require the hierarchical merging to create a systematic increase in physical density — a situation which we consider unlikely. We conjecture that the simple constant-density scaling may have a rigorous dynamical basis as a limiting case.

As an aside, we note that similar constant space density behavior may exist for systems which formed via monolithic top-hat collapse, instead of through the merging hierarchy. For very low mass halos formed from CDM-like power spectra, the characteristic overdensities $`\delta \rho /\rho `$ are roughly constant over a range of mass scales, leading to similar collapse epochs and similar final halo densities. This will naturally lead to a $`QM^1`$ scaling indistinguishible from the merging hierarchy. Thus, if these low mass systems survive and retain the densities imprinted at formation they could be indistinguishable from the rest of the merging hierarchy.

### 3.2 Observational Constraints on Phase Space Densities

We have argued above that considerable information on primordial conditions, dark matter physics, and the merging hierarchy may be contained in the scaling of $`Q`$ with mass. With this in mind, we now consider the most recent observational data and derive the best current measurements of the phase space density of dark matter on mass scales from $`10^8\mathrm{M}_{}`$ to $`10^{15}\mathrm{M}_{}`$. We will discuss the interpretation of these results in §4.

#### 3.2.1 Phase Space Densities in Dwarf Spheroidals

The lowest mass systems which can be used to measure dark matter densities are the dwarf spheroidal galaxies in the Local Group. These galaxies are extremely diffuse and low mass, and are supported by velocity dispersion rather than rotation. They have typical stellar velocity dispersions of order $`10\mathrm{km}/\mathrm{s}`$ and luminosities $`10^4`$ times fainter than bright spiral galaxies (see review by Mateo 1998). Dynamical mass-to-light ratios for these systems in some cases are very high ($`M_{tot}/L_V100`$), suggesting that they are completely dark matter dominated444Alternatively, dwarf spheroidals may not be in virial equilibrium, and instead their high velocity dispersions may be due to tidal disruption (e.g. Kuhn & Miller 1989). However, there is a relatively tight relationship between $`M/L`$ and luminosity (Mateo et al. 1999), suggesting that the values of $`M/L`$ are intrinsic to the galaxy and are not a product of environment. Further support for the existence of dark matter in dwarf spheroidals comes from: simulations by Oh et al. (1995) which show that dwarfs retain their equilibrium velocity dispersion even when being tidally disrupted; Burkert’s (1997) calculations that the properties of extra-tidal stars in Sextans are consistent with a high dark matter content; and Mateo et al’s (1998) arguments that Leo I (which has one of the highest values of $`M/L`$) is sufficiently isolated that it can not have been strongly affected by tidal heating. While there are clear cases of tidal disruption (i.e. Saggitarius), we consider it to be unlikely that tides are universally responsible for the high velocity dispersions in dwarf spheroidals..

We have derived the dark matter density of the dwarf spheroidals assuming that the dwarfs are dark matter dominated, and that the stars effectively behave like test particles in the halo potential. The dark matter is not required to have the same structure as the stars, given that current observations are not quite sufficient to distinguish between the cases of mass following light and of the dwarves being embedded in a larger halo (e.g. Klenya et al. 1999). However, the possible detection of “extra-tidal” stars in Carina (Majewski et al. 2000, Ibata & Hatzidimitriou 1995, Kuhn et al. 1996) beyond a clear break in the profile at a radius of $`30\mathrm{}`$, suggests that the dark matter core radius is unlikely to be more than a factor of 2 larger than the observed core radius for this particular case.

Assuming that the stars have an isotropic velocity dispersion and following Pryor & Kormendy 1990, the central density of the halo is $`\rho _{0D}=\frac{3\mathrm{ln}2}{2\pi }\frac{\sigma _{}^2}{Gr_c^2}`$, where $`\sigma _{}`$ is the observed one-dimensional central velocity dispersion of the stars and $`r_c`$ is the observed “core” radius where the surface density of stars falls to half the central value. The corresponding phase space density of the halo of dwarf spheroidals is therefore

$$Q_{DS}\frac{\rho _{0D}}{(3\eta _{}^2\sigma _{}^2)^{3/2}}=\frac{\mathrm{ln}2}{3^{1/2}2\pi }\frac{1}{G\eta _{}^3r_c^2\sigma _{}}$$
(4)
where the scaling factor $`\eta _{}\sigma /\sigma _{}`$ accounts for the fact that the dark matter particles do not necessarily have the same velocity dispersion as the stars (which formed from presumably dissipative baryonic processes). For an isothermal model for the dark matter halo, $`\rho _{0D}=\frac{9}{4\pi G}\frac{\sigma ^2}{r_0^2}`$ (where $`r_0`$ is the King radius), suggesting that $`\eta _{}=0.48\frac{r_0}{r_c}`$. We will take $`\eta _{}1`$, and thus implicitly assume that the core radius of the dark matter is roughly twice that of the stellar surface density profile. Note that because of the current inability to trace the density profiles of dwarf spheroidals, $`Q_{DS}`$ is not necessarily a central phase space density, but instead is representative of the mean phase space density within a core radius.

In calculating $`Q_{DS}`$, we have used data for the eight Local Group dwarf spheroidals from Mateo (1998) which are fainter than $`M_V=14`$ and which have internal kinematic measurements, excluding the tidally disrupting dwarf Saggitarius. We have used values for $`\sigma _{}`$ and $`r_c`$ given in the compilation of Mateo (1998), but have supplemented these with newer values for Ursa Minor and Draco from Klenya et al. (1999) and for Leo I from Mateo et al. (2000).

#### 3.2.2 Phase Densities in Rotationally Supported Galaxies

On somewhat larger mass scales than the dwarf spheroidals, the stars and gas in galaxies tend to become rotationally supported. Measurements of the rotation speed as a function of radius $`V_c(r)`$ can be used to derive the mass interior to $`r`$, assuming that the disk is in centrifugal equilibrium. Assuming that the dark matter halo is spherical and dominates the mass at all radii, we can approximate the mean density of the halo within a radius $`r`$ as

$$\rho _{gal}(<r)\frac{3}{4\pi G}\frac{V_c^2(r)}{r^2}.$$
(5)
There is a growing body of evidence that the assumptions which go into the above equation are not strictly true. For example, recent measurements from the flaring of HI disks, the shapes of x-ray isophotes, warps in galactic disks, and the dynamics of polar ring galaxies all suggest that galaxy halos are not spherical, but are somewhat flattened (i.e. oblate; see the summary figure in Olling & Merrifield 1999, and discussion in Sackett 1999). However, models by Olling (1995) show that for the observed range of flattenings, the true central density is not more than a factor of $`50`$% greater what would be derived in the spherical case.

A much larger uncertainty comes from the contribution which the baryons in the disk make to the dynamics. The mass of atomic gas is usually easily determined through the distribution of HI (including a correction for helium). However, the molecular gas phase can be the dominant contributor in the inner disk of massive spiral galaxies, but is rarely observed, and then is only detected indirectly through the CO tracer. The mass in stars is also uncertain, given that for most stellar populations, most of the mass is due to stars which make little contribution to the total light. Thus, the density given by equation 5 represents an upper limit to the enclosed density of the dark matter halo. Fortunately, for many rotating dwarfs and low surface brightness galaxies, the baryonic disk is sufficiently diffuse that for all reasonable stellar mass-to-light ratios, the enclosed mass is dominated by the dark matter halo. Unfortunately, this limits our analysis to galaxies with relatively low rotation speeds ($`V_{c,max}100\mathrm{km}/\mathrm{s}`$, vs $`V_{c,max}250\mathrm{km}/\mathrm{s}`$ for bright spirals). While some “Malin-like” low surface brightness disks are known to have higher rotation speeds, these galaxies typically have large central bulges as well, and thus are likely to be baryon dominated in the central regions.

To calculate $`\rho _{gal}`$, we have chosen to use the rotation curve decompositions for NGC 247 ($`V_{c,max}100\mathrm{km}/\mathrm{s}`$; Carignan & Puche 1998), DDO 154 ($`V_{c,max}45\mathrm{km}/\mathrm{s}`$; Carignan & Beaulieu 1989), and NGC 3109 ($`V_{c,max}60\mathrm{km}/\mathrm{s}`$; Jobin & Carignan 1990), as compiled and analyzed in van den Bosch et al. (2000). These were three cases found in the literature where the HI observations were sufficiently resolved to accurately trace the rotation curve in the inner halo. In this analysis, the core radius $`r_0`$ is taken as the radius where the density profile of the dark matter halo changes slope: $`\rho (r)r^\alpha (r+r_0)^{\alpha 3}`$

To calculate the phase space density, we derive the halo velocity dispersion by considering the circular velocity measured at the core radius. For an isothermal distribution, the core radius is $`r_0=\sqrt{9\sigma ^2/4\pi G\rho }`$, suggesting that $`V_c^2(r_0)3\sigma ^2`$. Note, however, that the density distribution of the halo is not necessarily that of an isothermal sphere, and thus that our approximation of $`\sigma `$ will necessarily be uncertain. For NGC 3109 and NGC 247, the core radius derived by van den Bosch et al. (2000) is greater than the last measured point in the rotation curve555One complication is that for NGC 247, the core does not seem to be constant density (unlike NGC 3109 and DDO 154), and instead has a density rising inwards like $`r^1`$., so we take $`\sigma =V_{c,max}/\sqrt{3}`$ for these two cases. For DDO 154, the best fit core radius is $`3\mathrm{kpc}`$, well within the last measured radius. The final velocity dispersions are then $`\sigma _{247}=62\mathrm{km}/\mathrm{s}`$, $`\sigma _{3109}=38\mathrm{km}/\mathrm{s}`$, and $`\sigma _{154}=22\mathrm{km}/\mathrm{s}`$. However, because of the difficulty in securely identifying the halo core radius, these velocity dispersions are probably uncertain by $`50`$%.

The resulting average phase space density within $`r`$ is therefore

$$Q_D(r)\frac{9\sqrt{3}}{4\pi Gr^2}\frac{1}{V_c(r_0)}\left(\frac{V_c(r)}{V_c(r_0)}\right)^2.$$
(6)
#### 3.2.3 Cluster Phase Space Densities

Clusters of galaxies provide the best laboratories for measuring the phase space density of collapsed dark matter halos on the largest mass scales. Analyses of their x-ray properties and internal dynamics show that many (though not all) clusters are well-relaxed systems, and thus are appropriate for studying the equilibrium state of high-mass dark matter halos.

On the other hand, clusters are far from ideal. Unlike dwarf spheroidals, the centers of clusters have a significant mass contribution from baryons, mostly in the form of hot x-ray emitting gas. Current estimates are that 10-25% of the cluster mass within $`r_{500}`$ is in the gas phase (c.f. Ettori & Fabian 1999, where $`r_{500}`$ is the radius within which the density constrast is 500). This problem only intensifies in the very centers of clusters, where the density of the intracluster gas is the highest, and where the dynamical mass may be dominated by the stellar population of a central giant elliptical galaxy. There are also fewer dynamical probes in the centers of clusters, simply due to limited sampling volume for velocity tracers. The best dynamical studies to date, which incorporate velocity anisotropy for an ensemble of clusters, do not probe much within a radius of $`50h^1\mathrm{kpc}`$ (Carlberg et al. 1997, van der Marel et al. 2000). The mass profiles of the centers of clusters are also difficult to probe with x-rays. The historically low resolution of x-ray telescopes has not allowed measurements of the temperature of the gas to be spatially resolved at very small scales (though the experimental situation is rapidly improving). Finally, clusters are among the most massive bound structures seen today, and thus many are still in the process of formation. We therefore may expect to see some degree of variation in their properties, reflecting incomplete relaxation.

Of all the methods of constraining the central densities of clusters, the most secure estimates come from observations of strong gravitational lensing within cluster cores (i.e. arcs). For a spherical mass distribution, the mean density $`\rho _{arc}`$ within the radius of the lensed arc $`r_{arc}`$ is

$$\rho _{arc}=\frac{3c^2}{16\pi G}(\frac{D_s}{D_lD_{ls}})\frac{1}{r_{arc}}.$$
(7)
The use of elliptical mass distributions can reduce this density by typically 20%. To account for the baryonic contribution within $`r_{arc}`$, we reduce the above measurement of $`\rho _{arc}`$ by a factor $`(1f_{baryon})0.8`$ to estimate the dark matter density.

The resulting average phase space density within $`r_{arc}`$ is therefore

$$Q_C\frac{(1f_{baryon})\rho _{arc}}{(3\eta _{gal}^2\sigma _{gal}^2)^{3/2}}.$$
(8)
We have again included a scaling factor $`\eta _{gal}`$ to allow for systematic differences between 1-d velocity dispersion of the galaxies ($`\sigma _{gal}`$) and the dark matter dark matter. However, weak lensing and dynamical studies (c.f. Tyson et al. 1998, Carlberg et al. 1997) all suggest that mass traces light on $`>100\mathrm{kpc}`$ scales within clusters, and thus $`\eta _{gal}1`$.

To calculate $`Q_C`$ for specific clusters, we have restricted ourselves to lensing clusters which also host strong cooling flows. Allen (1998) argues persuasively that cooling flow clusters are the most likely to be fully relaxed, and demonstrates that they are the only clusters whose x-ray and lensing mass estimates are consistent. We have plotted $`Q_c`$ for the six cooling flow clusters with giant arcs which were analyzed in Allen (1998). For the three cases where more detailed mass models have been developed to fit the strong lensing data, we have revised the spherical estimate of $`\rho _{arc}`$ accordingly (MS2137.3-2353, Mellier et al. 1993; PKS0745-191, Allen et al. 1996; Abell 2390, Pierre et al. 1996).

Our calculation for $`Q_C`$ gives the mean phase space density with the radius of strongly lensed arcs. However, we wish to compare $`Q_C`$ with the phase space densities derived for dwarfs and rotating disks. These latter quantities are calculated within a core radius, and thus there is some inherent uncertainty in treating $`Q_C`$ as a “core” phase space density. At the radii where these arcs are typical, detailed dynamical studies by Carlberg et al. (1997) and van der Marel et al. 2000) show that the cluster density profiles are still rising like $`r^1`$ at the innermost measurable radii ($`35h^1\mathrm{kpc}`$); if this behavior continues towards the center where it terminates in a smaller constant density core, then the core phase space density could be a factor of 10-100 times higher than the above estimate of $`Q_C`$ on scales of $`1\mathrm{kpc}`$. Work by Williams et al. (1999) argues that shallow inner cores must be rare but on the other hand, Tyson et al. 1998 reconstruct the density profile one cooling-flow cluster with a constant density core ($`r_c35h^1\mathrm{kpc}`$, derived from fitting eight images of a multiply lensed background galaxy), suggesting that in some cases $`Q_C`$ may be close to the true core phase density. If clusters do not have a smaller constant density core within $`r_{arc}`$, and instead the most appropriate assignment for $`r_{core}`$ is the typically larger radius where the density profile changes from $`r^1`$ to $`r^3`$, then the phase space density which should be compared to $`Q_{DS}`$ and $`Q_D`$ will be substantially smaller than calculated for $`Q_C`$. Considering these uncertainties, our calculated values of $`Q_C`$ are uncertain by possibly as much as a factor of 10.

## 4 Interpretation

We have plotted the characteristic phase space densities for dwarf spheroidals, rotationally supported galaxies, and clusters of galaxies in Figures 7 & 7. It is immediately apparent that there is a systematic decrease in the phase density as a function of scale, with more massive systems having dramatically lower phase space densities666While the cluster measurements of $`Q`$ are uncertain, they are unlikely to compensate for the observed factor of $`10^8`$ variation in phase density.. A similar trend was noted by Burkert (1995) and Sellwood (2000), although over a much smaller range in scale.

### 4.1 Hierarchical Assembly

Immediately, the factor of $`10^210^3`$ difference in phase density between dwarf spheroidals and rotating galaxies in Figures 7 & 7 suggests that the cores in rotating dwarfs cannot be due to “phase packing” of material with a primordial phase density. This agrees with our analysis in §2 about the behavior of core sizes as a function of velocity dispersion.

Figure 7 also demonstrates that at the same physical scale ($`1\mathrm{kpc}`$), the phase space densities of rotating dwarfs are substantially smaller than for dwarf spheroidals777As a caveat, the amount of beam smearing in the inner parts of the rotating dwarf measurements may be considerable, leading to artificially low central densities (van dan Bosch et al. 2000). These three cases have been chosen to minimize this concern, however. This suggests that the cores of more massive halos cannot be made up purely of “sinking satellites” (e.g. Syer & White 1998); most accreted objects must have undergone substantial disruption and phase mixing while being incorporated.

Aside from the above two points, Figures 7 & 7 strongly suggest that halos formed as the result of a merging hierarchy. As discussed above, if larger systems build up from smaller chunks, relaxation processes necessarily lead to substantial phase mixing while reaching virial equilibrium, and thus with each successive merger there is a dilution of the coarse-grained phase space density. Figures 7 & 7 are strong support (if any were needed) for hierarchical formation of galaxies and clusters (i.e. bottom-up, rather than top down); if less massive objects were to fragment from larger mass objects, their phase space density would be lower, not higher as is observed, unless there were substantial dissipation involved in the fragmentation.

Moreover, the behavior seen in Figure 7 is close to the minimal decrease $`Q1/\sigma ^3`$ predicted from our simple scaling argument in §3.1. It is a profound fact that dark matter dominated systems obey a scaling relatively close to this limit, suggesting that they formed in a fairly quiet collisionless hierarchy.888Since we can only measure $`Q`$ where there are stars and gas (i.e. in the central regions), it is not surprising that they sample the lowest entropy envelope of the hierarchy. The maximum $`Q`$ on all scales remembers the initial $`Q`$ at the start of the hierarchy (the top left of Figures 7 & 7); the $`Q`$ of the first systems to collapse sets a maximum $`Q`$ for all the systems which form from subsequent clustering. We discuss the origin of this “seed point” in §4.2 below.

While we believe that cores were built hierarchically on scales from galaxies to clusters, it is possible that lower mass halos resulted from early monolithic collapse. Remarkably, such systems would still follow the $`Q\sigma ^3`$ scaling. For small mass halos formed from a CDM-like power spectra, the density contrast $`(\delta \rho /\rho )_M^2k^3P(k)^2`$, is nearly constant with mass, and thus all perturbations collapse at the same time. The synchronous formation of low mass halos will lead them to have similar densities, (if the densities in the final virialized halos tend to track the density of the universe at their formation – e.g. as claimed by Navarro et al. 1996,1997), yielding the same $`Q\sigma ^3`$ scaling derived for “quiet” merging. If these monolithically collapsed systems survive till the present, or if they subsequently merge, their descendents will be indistinguishable from the rest of the $`Q`$-$`\sigma `$ hierarchy.

We note that the observed scaling of $`Q`$ is not expected for highly collisional dark matter. The prediction of $`Q\sigma ^3`$ was derived assuming that merging material settles where its density matches the density of the enveloping galaxy. In contrast, highly collisional dark matter is compressed to higher density as it responds to higher local pressure, then sinks to where it reaches local pressure and density equilibrium, and finally stops when the entropy matches. Thus matter in a quiet merger tends to be stripped at constant $`Q`$ rather than constant density. This preserves $`Q`$ during merging, leading to $`Q\sigma ^0`$, i.e. constant phase space density at all mass scales. An exception occurs if collisions are rare enough to allow rapid heat conduction, which leads to a core-collapse instability and high central densities. This case is of course dissipational, allowing the $`Q`$ of a fluid element to increase. It is possible (though not likely) that the high $`Q`$ of the low mass systems arises this way.

### 4.2 The Seed Point of the Phase Space Hierarchy

While the arguments given in §3.1 set the slope of the $`Q`$-$`\sigma `$ relation, the origin of the overall normalization remains unclear. Within the framework of a merging hierarchy, the normalization seen in Figure 7 is fixed by the phase space density of the lowest mass systems – the “seeds” of the merging hierarchy – lying somewhere upwards and to the left on Figure 7. These early systems will have the highest observed phase space densities, and will set the phase space density of all systems further down the merging hierarchy999Note that in CDM there is no upper limit to $`Q`$ and dark matter halos are predicted to exist along a continuation of the $`Q(\sigma )\sigma ^3`$ relation indefinitely to low mass. Moore et al. (1999) have speculated that it is this cold initial phase space density of CDM which leads to the very dense cores in numerical simulations, regardless of the power spectrum of initial perturbations. It is possible, however, that extra heating could be provided by smaller-scale dynamical effects not yet resolved.. Thus the phase space density of the first generation of collapsed objects fixes the normalization of the entire $`Q`$-$`\sigma `$ relation. What sets the masses of these early halos and more importantly, what sets their phase space densities?

There are three physical parameters which can alter the phase space density of the first collapsed objects. First is the dark matter particle microphysics, which sets a maximum value for the fine-grained primordial $`Q_0`$, via equation 9 below; no collapsed dark matter halos can have a coarse-grained phase space density higher than the primordial value of $`Q_0`$, unless the dark matter is dissipational. Second is the density of the universe during the epoch when matter first collapses and virializes, if lower initial densities lead to lower phase densities in the virialized halos (Navarro et al. 1996,1997). Third is the efficiency101010By “efficiency”, we mean both the fraction of matter which undergoes phase mixing, and the degree to which that matter is mixed. of phase mixing during collapse itself. Through violent relaxation and phase-wrapping, the primordial phase sheet is mixed to a lower coarse-grained phase space density. The process of relaxation and virialization during the first halos’ collapse can set the phase density of the lowest mass cores, and thus the normalization of the entire $`Q`$-$`\sigma `$ relation.

The phase space density of the first collapsed objects will be set by a combination of these three processes. For example, if violent relaxation and phase mixing is inefficient (meaning, that much of the matter is not phase wrapped and is left at high $`Q`$), and/or the velocity dispersion of the first collapsed halos is comparable to the primordial velocity dispersion of the dark matter, then the maximum observed phase space density will be fixed at the primordial fine-grained value $`Q_0`$. Alternatively, if the original phase sheet with density $`Q_0`$ is well phase mixed during collapse of the first objects, then the maximum observed phase space density will be diluted from $`Q_0`$ to a lower value. Or finally, if the overall density of the universe is lower when the first halos form, then the early halos will possibly virialize to lower space densities and lower $`Q`$. In either of the these three cases, at fixed mass (or velocity dispersion) the seeds for the merging hierarchy will have lower phase densities than CDM, and the normalization of the $`Q`$-$`\sigma `$ relation will be reduced. On the other hand, CDM simulations by Moore et al. (1999) show that introducing an arbitrary cutoff in small-scale power produces no detectable changes in the density profiles of the dark matter halos. This raises questions about the degree of coupling between the final and initial densities of collapsing halos; it may turn out that the epoch of collapse has little direct impact on $`Q`$ in the cores of the first halos.

These three contributors to the $`Q`$-$`\sigma `$ normalization – primordial $`Q_0`$, violent relaxation & phase mixing, and the density at the epoch of collapse – are not necessarily independent. In some cases, the latter two are coupled directly or indirectly to $`Q_0`$. For example, the physics of the dark matter sets the primordial phase space density $`Q_0`$ via equation 9 below. This initial phase space density corresponds to a characteristic velocity dispersion for dark matter in the early universe. Given that the velocity dispersion of the initial conditions may affect the degree of violent relaxation and the growth of angular momentum, it is possible that the efficiency of phase mixing during violent relaxation may be indirectly set by the primordial $`Q_0`$; in other words, dark matter particle physics may be imprinted upon the normalization of the $`Q`$-$`\sigma `$ relation, even if $`Q_0`$ does not fix the phase space density of the first halos directly. Likewise, the epoch at which the first objects collapse depends upon the power spectrum of initial fluctuations, a function which is in turn affected by $`Q_0`$. In warm dark matter models, for example, free streaming creates a mass filtering scale $`M_{filter}`$ (which can be related to $`Q_0`$), that suppresses the power spectrum at low masses, and delays the collapse of the first objects.

If the first objects which collapse have sufficiently low mass, we expect little change in the normalization for changes in the dark matter properties. The density contrast $`(\delta \rho /\rho )_M^2k^3P(k)^2`$ is nearly constant at small mass for CDM-like power spectra, so that all perturbations collapse at the same time, as discussed above in §4.1. Changing the mass scale of the first collapsed objects will therefore not change the initial epoch of structure formation, and will leave the normalization of $`Q`$ vs $`\sigma `$ unchanged, provided that the masses of the new seeds are still in the regime where $`(\delta \rho /\rho )_M^2`$ is constant. If the core phase space densities of these first small halos are limited by the primordial phase space density $`Q_0`$, then we also expect no shift in the $`Q`$-$`\sigma `$ normalization – at small mass, the filtering mass scale varies like $`M_{filter}Q_0^1`$.

On the other hand, if the filtering is on a scale where $`(\delta \rho /\rho )^2`$ decreases with $`M`$, then the first collapsed objects have both larger masses and smaller primordial phase densities. In this case, for a given normalization of the power spectrum on large scales, the first collapse is sufficiently delayed such that $`Q`$ for the first objects drops faster than $`M^1`$, reducing the normalization of the $`Q`$-$`\sigma `$ relation. Filtering on these larger mass scales may also change the expected degree of scatter in the $`Q`$-$`\sigma `$ relation; with additional observational and theoretical investigation, one could potentially use the slope and scatter of the $`Q`$-$`\sigma `$ relation to place limits on the mass scale of the first objects. However, stronger constraints will probably come from considering the abundance of low mass halos.

### 4.3 Links to Elliptical Galaxies

We note that qualitatively similar behavior to Figure 7 is seen within other collisionless systems, namely giant elliptical galaxies, where the dynamics are dominated by stars within the half-light radius. Like dark matter halos, these ellipticals may also have formed via successive mergers of largely collisionless systems (particularly in clusters, where progenitors are largely gas-poor). Paralleling early work by Carlberg (1986) and Lake (1989), Hernquist et al. (1993) used data on elliptical galaxies from Bender et al (1992) to show a systematic decrease in elliptical galaxy phase density with increasing luminosity (roughly $`QL_B^{1.5}`$). Assuming a Faber-Jackson (1976) relationship of $`L_B\sigma _0^4`$, the central phase space densities of ellipticals must scale with the central velocity dispersion as roughly $`Q\sigma _0^\nu `$, with $`\nu 6`$.

The true velocity scaling may be shallower than implied by Hernquist et al. (1993), however. Recent analyses of elliptical galaxies’ light profiles suggest that ellipticals are non-homologous on large scales, such that they differ from deVaucouleurs profiles systematically with increasing mass (Caon et al. 1993, Graham & Colless 1997), such that lower mass ellipticals have shallower inner profiles111111Note that we are referring to light profiles and phase densities measured on the scale of the half light radii, not at the innermost points measured by HST, where the densities may be substantially affected by the presence of central black holes.; Hjorth & Madsen (1995) argue that taking this non-homology into account should lead to a shallower relationship between $`Q`$ and $`L`$, and thus the scaling of $`Q`$ with $`\sigma _0`$ should be likewise shallower ($`\nu <6`$), and possibly compatible with the $`\nu 34`$ behavior seen in Figure 7. If a proper reanalysis shows that the true relationship is steeper than observed in dark matter halos, then the difference is likely to reflect a more disruptive, less quiescent merger history offering more opportunities for breaking homology.

Given the similarities of the global phase space density behavior shared by both dark matter halos and ellipticals (particularly in the stellar-dominated inner regions), they may also share the more detailed internal phase space evolution. It is plausible that even with their possibly quieter hierarchy, dark matter halos may eventually reveal the same homology breaking seen in ellipticals. The non-homology in elliptical galaxy light profiles is typically parameterized in terms of the Sérsic function, with surface brightness $`\mathrm{\Sigma }(r)\mathrm{exp}[(r/r_0)^{1/n}]`$ (where $`n=4`$ corresponds to a deVaucouleurs’ profile). Work by Caon et al. (1993) and Graham & Colless (1997) shows that the Sérsic index $`n`$ tends to increase systematically with the projected half-light radius $`r_e`$. The Sérsic index $`n`$ is 1–2 for the smallest ellipticals ($`r_e1\mathrm{kpc}`$), and increases systematically to 5–10 for the largest ellipticals ($`r_e10\mathrm{kpc}`$). In Figure 7 we plot the corresponding density profiles for $`n=17`$, using Márquez et al. (2000)’s fitting formula for the deprojected density. The lower mass ellipticals with $`n1`$ have density profiles with shallower cores than the higher mass ellipticals (although the overall density is higher). This may be analogous to the detection of relatively shallow inner profiles in the cores of rotating dwarfs and steeper profiles in massive clusters (although see caveats in §3.2.3). We note also that the density profile which corresponds to a projected deVaucouleur’s profile ($`n=4`$) is quite similar to the density profile found by Navarro et al. (1996) for simulated dark matter halos, both plotted in Figure 7.

### 4.4 Constraints on the Mass of Dark Matter Candidates

In addition to giving us clues about the relaxation processes involved in galaxy formation, the above observations of the phase space density can be used to place strong limits on the masses of possible particle dark matter candidates. If dark matter has a primordial velocity dispersion, then its initial phase space density is lowered over CDM, which presumably leads to lower overall densities in the final virialized halo. Generalizing the Tremaine & Gunn (1979) argument for massive neutrinos (see also Gerhard & Spergel 1992), Hogan & Dalcanton (2000) showed that the primordial phase density $`Q_0`$ of dark matter particles $`X`$ can be simply related to the mass $`m_X`$ (of particles which decouple when their momentum distribution is relativistic) through

$$Q_0=q_Xg_Xm_X^4$$
(9)
where the coefficient from the distribution function integral $`q_X`$ is 0.00196 for thermal particles and 0.0363 for degenerate fermions, and $`g_X`$ is the number of effective photon degrees of freedom of the particle $`X`$.

In the absence of dissipation, the coarse-grained phase space density can only decrease from its primordial value. The maximum observed phase space density therefore places a lower limit on the mass of the $`X`$ particle. Figure 7 shows that the highest observed phase space densities are found for dwarf spheroidals, with $`Q_{obs}10^4\mathrm{M}_{}\mathrm{pc}^3(\mathrm{km}/\mathrm{s})^3`$. This lower limit on $`Q_0`$ implies that

$$m_X>669\mathrm{eV}\left(\frac{Q_{obs}}{10^4\mathrm{M}_{}\mathrm{pc}^3(\mathrm{km}/\mathrm{s})^3}\right)^{1/4}\left(\frac{0.00196}{q_X}\right)^{1/4}\left(\frac{2}{g_X}\right)^{1/4}$$
(10)
For thermal particles with 2 degrees of freedom, the data suggest that to $`m_X>669eV`$. For degenerate fermions, with 2 degrees of freedom, $`m_X>322\mathrm{eV}`$.

These lower limits on particle mass correspond to the largest $`Q`$ actually observed. There may well be dark matter halos with smaller velocity dispersions and larger $`Q`$ halos; indeed, these are expected in CDM. However, such systems would never form stars, and thus would remain undetected. At velocity dispersions below 7 km/sec, the collisional cooling of zero-metal atomic gas becomes very inefficient. The specific binding energy of the shallowest observed halo potentials is close to to the minimum temperature expected for the protogalactic medium during early galaxy formation.

For standard collisionless warm matter, if the dark matter saturates the $`Q_0`$ limit from dwarf spheroidals (that is, $`m_X700`$ eV), there is a corresponding filtering scale at the masses of dwarf galaxies (see Sommer-Larsen and Dolgov 1999, Hogan and Dalcanton 2000). This effect may already be indicated by the paucity of dwarf galaxies relative to standard CDM predictions. We do not yet know enough about the predictions of WDM to compare with halo mass functions in detail, however.

## 5 Conclusion

The above examinations of the existing data on the structure and phase space density of dark matter halos yields a number of conclusions which may be relevant to constraining the nature of the dark matter.

First, the behavior of halo core size with increasing mass suggests that it is unlikely that either phase-space packing or highly collisional dark matter is sufficient for simultaneously explaining the dark matter cores of dwarf spheroidals, rotating dwarf galaxies, and clusters of galaxies. The generic $`r_{core}1/\sqrt{\sigma }`$ behavior of these scenarios for core formation would predict far larger cores for the dwarf spheroidals than for larger systems, in contrast to observational evidence.

Second, if there were still any doubt, the dramatic decrease in the characteristic phase space density with increasing mass is extremely strong evidence for a “bottom-up” hierarchical buildup of bound structures. Given that the phase space density can never increase with successive mergers (in the absence of dissipation), smaller structures cannot generically have fragmented from larger ones. The trend points to dwarf spheroidals as the lowest entropy, and therefore dynamically most primitive observable systems.

Third, the observed dependence of $`Q\sigma ^3`$ approximately agrees (over eight orders of magnitude in $`Q`$!) with a simple scaling relation that assumes the minimal decrease compatible with a gentle merging hierarchy with virial equilibrium and homologous tidal stripping at each stage. The same scaling is also compatible with a simple synchronous collapse of different mass systems to constant virial density. The two descriptions are both appropriate, at different stages, for the collisionless hierarchy predicted in CDM models. We also discuss the physics which sets the normalization of the $`Q`$-$`\sigma `$ scaling.

Fourth, examination of the phase space density for dark matter halos suggests some parallels to elliptical galaxies. In most scenarios, both dark matter halos and cluster elliptical galaxies are thought to be formed through collisionless merging and accretion. We show that observationally, both systems show similar decreases in the coarse-grained phase space density with increasing velocity dispersion. They also show surprisingly similar density profiles. If elliptical galaxies can be used as a rough analog to dark matter profiles, then they suggest that the structure of dark matter halos may undergo subtle, systematic deviations from homology, leading to somewhat flatter inner cores (and steeper fall-off at large radii) for low mass halos. The amplitude of these deviations are sufficiently small that they are unlikely to be well resolved in current numerical simulations, though they may have already been detected observationally. This non-homology could help to alleviate some of the discrepancies between observations of rotating dwarfs and predictions of dark matter simulations on the smallest, most poorly resolved scales. This parallel also provides suggestive, although not conclusive, evidence that dark matter halos are indeed collisionless on the scale of halo cores.

Fifth, the very high phase space densities of dwarf spheroidals can be used to place constraints on the masses of potential dark matter candidates. For dark matter particles with 2 degrees of freedom, decoupled while still relativistic, masses of $`m_X>700\mathrm{eV}`$ (thermal fermions) or $`m_X>300\mathrm{eV}`$ (degenerate fermions) are preferred. Because systems with smaller masses and higher phase space densities than dwarf spheroidals may exist, the actual particle masses may be substantially higher than these limits. This seems likely since smaller $`\sigma `$ halos would be invisible even if they exist; in other words we see halos populated with stars right up to the cooling limit, which would be a coincidence if they also correspond to the phase density limit. This interpretation is again consistent with the view that dwarf spheroidals are the most primitive bound systems so far observed.

Finally, we conjecture that the addition of primordial velocity dispersion can help to reconcile the discrepancies between numerical predictions of dense central cores in hierarchical clustering (e.g. Navarro et al. 1996,1997, Fukushige & Makino 1997, Moore et al. 1998&1999, Ghinga et al. 1999, Jing & Suto 2000) and observations of much lower central densities (e.g. Flores & Primack 1994, Moore 1994, Burkert 1995, Navarro et al. 1996, Stil 1999). Regardless of the origin of the “universal” density profile, simulations routinely predict a denser, more concentrated halo than is actually observed; in other words, the predicted central phase space density for CDM is too high, which in turn suggests that the CDM initial conditions themselves have too high a phase density. Instead, if the primordial phase space density is lower than the CDM case, then even if the final density structure is set by merging and relaxation, the final phase space density should be lowered as well, provided the primordial $`Q_0`$ is not much higher than that which occurs as a result of virialization at the earliest nonlinear collapse in CDM. Numerical simulations by Huss et al. (1999) approximately explore this conjecture through studying monolithic collapse of spherical overdensities with varying velocity dispersion. However, numerical relaxation is clearly a problem for these simulations, and we draw no conclusion from them at this time. Future numerical work will certainly shed light on this hypothesis.

We are happy to acknowledge discussions with Tom Quinn, David Spergel, Frank van den Bosch, Arif Babul, Simon White, Julio Navarro, George Lake, and Ben Moore. JJD gratefully acknowledges the hospitality of the Institute of Theoretical Physics in Santa Barbara, where some of this work was done. The research was supported in part by the National Science Foundation under Grant No. PHY94-07194, and at the University of Washington by NASA and NSF.

## 6 References

Allen, S. W. 1998, MNRAS, 296, 392

Allen, S. W., Fabian, A. C., & Kneib, J.-P. 1996, MNRAS, 279, 615

Bender, R., Burstein, D., & Faber, S. M. 1992, ApJ, 399, 462

Bento, M. C., Bertolami, O, Rosenfeld, R., & Teodoro, L. 2000, preprint, (astro-ph/0003350)

Blumenthal, G. R., Faber, S. M., Flores, R., & Primack, J. R. 1986, ApJ, 301, 27

Borriello, A., & Salucci, P. 2000, MNRAS, submitted (astro-ph/0001082)

Burkert, A. 1995, ApJ, 447, L25

Burkert, A. 1997, ApJ, 474, L99

Burkert, A. 2000, ApJ, submitted (astro-ph/0002409)

Caon, N, Capaccioli, M, & D’Onofrio, M. 1993, MNRAS, 265, 1013

Carignan, C., & Beaulieu, S. 1989, ApJ, 347, 760

Carignan, C., & Puche, D. 1990, AJ, 100, 641

Carlberg, R. G. 1986, ApJ, 310, 593

Carlberg, R. G., Yee, H. K. C., Ellingson, E., Morris, S. L., Abraham, R. , Gravel, P., Pritchet, C. J., Smecker-Hane, T., Hartwick, F. D. A., Hesser, J. E., Hutchings, J. B., & Oke, J. B. 1997, ApJ, 485, L13

Colin, P., Avila-Reese, V., & Valenzuela, O. 2000, ApJ, submitted (astro-ph/0004115)

Dalcanton, J. J., & Bernstein, R. A. 2000, in XVth IAP Meeting Dynamics of Galaxies: from the Early Universe to the Present, ed. F. Combes, G. A. Mamon, & V. Charmandaris

Ettori, S., & Fabian, A. C. 1999, MNRAS, 305, 834

Firmani, C., D’Onghia, E., Aliva-Reese, V., Chincarini, G., & Hernańdez, X. 2000, MNRAS,, in press

Flores, R. A., & Primack, J. R. 1994, ApJ, 427, L1

Flores, R. A., & Primack, J. R. 1996, ApJ, 457, L5

Fukushige, T., & Makino, J. 1997 ApJ, 477, L9

Gerhard, O. E. & Spergel, D. N. 1992, AJ, 389, L9

Goodman, J. 2000, preprint, (astro-ph/0003018)

Graham, A., & Colless, M. 1997, MNRAS, 287, 221

Hannestad, S., & Scherrer, R. J. 2000, preprint, (astro-ph/0003046)

Hannestad, S., 1999, astro-ph/9912558

Hjorth, J., & Madsen, J. 1995, ApJ, 445, 55

Hernquist, L., Spergel, D. N., & Heyl, J. S. 1993, ApJ, 416, 415

Hogan, C. J. 1999, ApJ, 527, 42

Hogan, C. J., & Dalcanton, J. J. 2000, Phys. Rev. D, in press

Hu, W., Barkana, R., & Gruzinov, A. 2000, Phys. Rev. Lett., submitted, (astro-ph/0003365)

Irwin, M., & Hatzidimitriou, D. 1995, MNRAS, 277, 1354

Jing, Y. P. & Suto Y. 2000, ApJ, 529, 69

Jobin, M., & Carignan, C. 1990, AJ, 100, 648

Kamionkowski, M., & Liddle, A. R. 1999, preprint, (astro-ph/9911103)

Klenya, J., Geller, M., Kenyon, S., & Kurtz, M. 1999, AJ, 117, 1275

Kravtsov, A. V., Klypin, A. A., Bullock, J. S., & Primack, J. R. 1998, ApJ, 502, 48

Kochanek, C. S., & White, M. 2000, ApJ, submitted, (astro-ph/0003483)

Kuhn, J. R., & Miller, R. H. 1989, ApJ, 341, L41

Kuhn, J. R., Smith, H. A. & Hawley, S. L. 1996, ApJ, 469. L93

Lake, G. 1989, AJ, 97, 1312

Lake, G. 1989, AJ, 98, 1253

Majewski, S. R., Ostheimer, J. C., Patterson, R. J., Kunkel, W. E., Johnston, K. V., & Geisler, D. 2000, ApJ, 119, 760

Mateo, M. 1998, ARA&A, 36, 435

Mateo, M., Olszewski, E. W., Vogt, S. S., & Keane, M. J. 1998, 116, 2315

Mellier, Y., Fort, B., & Kneib, J.-P. 1993, ApJ, 407, 33

Mohapatra, R. N., & Teplitz, V. L. 2000, preprint, (astro-ph/0001362)

Moore, B., 1994, Nature, 370, 629

Moore, B., Quinn, T., Governato, F., Stadel, J., & Lake, G. 1999, MNRAS, 310, 1147

Moore, B., Governato, F., Quinn, T., Stadel, J., & Lake, G. 1998, ApJ, 499, L5

Moore, B., Gelato, S., Jenkins, A., Pearce, F. R., & Quilis, V. 2000, preprint, (astro-ph/002308)

Navarro, J. F. 1998, preprint, (astro-ph/9807084)

Navarro, J. F., Frenk, C. S., & White, S. D. M. 1996, ApJ, 462, 563

Navarro, J. F., Frenk, C. S., & White, S. D. M. 1997, ApJ, 490, 493

Oh, K. S., Lin., D. N. C., & Aarseth, S. J. 1992, ApJ, 442, 142

Olling, R. P. 1995, AJ, 110, 591

Olling, R. P., & Merrifield, M. R. 1999, MNRAS, in press

Peebles, P. J. E. 2000, preprint, (astro-ph/0002495)

Pierre, M., Le Borgne, J. F., Soucaile, G., & Kneib, J.-P. 1996, å, 311, 413

Pryor, C., & Kormendy, J. 1990, AJ, 100, 127

Riotto, A. & Tkachev, I. 2000, preprint, (astro-ph/0003388)

Sackett, P. D. 1999, in Galaxy Dynamics, eds. Merritt, D., Sellwood, J. A., & Valluri, M., ASP

Sellwood, J. A. 2000, ApJ, submitted, (astro-ph/0004352)

Sommer-Larsen, J., and Dolgov, A. 1999, preprint, (astro-ph/9912166)

Spergel, D. N., & Steinhardt, P. J. 1999, Phys. Rev. Lett., submitted (astro-ph/9909386)

Schwarz, D. J., & Hofmann, S. 1999, preprint, (astro-ph/9912343)

Shi, X., and Fuller, G. M. 1999, Phys. Rev. Lett., 82, 2832

Swaters, R. 2000, Ph.D. thesis, University of Groningen

Syer, D., & White, S. D. M. 1998, MNRAS, 293, 337

Tremaine, S. and Gunn, J. E. 1979, Phys. Rev. Lett., 42, 407

Tyson, J. A., Kochanski, G. P., & Dell’Antonio, I. P. 1998, ApJ, 498, L107

van den Bosch, F. C., Robertson, B. E., Dalcanton, J. J., & de Blok, W. J. G. 2000, AJ, 119, 1579

van den Bosch, F. C., & Swaters, R. 2000, AJ, submitted

van der Marel, R. P. et al. 2000, AJ, in press (astro-ph/9910494)

Williams, L. L. R., Navarro, J. F., & Bartelmann, M. 1999, ApJ, 527, 535

Yoshida, N., Springel, V., White, S. D. M., & Tormen, preprint, (astro-ph/0002362)

## 7 Figure Captions


format: text

text:
# Static and dynamic coupling transitions of vortex lattices in disordered anisotropic superconductors

\[

## Abstract

We use three-dimensional molecular dynamics simulations of magnetically interacting pancake vortices to study vortex matter in disordered, highly anisotropic materials such as BSCCO. We observe a sharp 3D-2D transition from vortex lines to decoupled pancakes as a function of relative interlayer coupling strength, with an accompanying large increase in the critical current remniscent of a second peak effect. We find that decoupled pancakes, when driven, simultaneously recouple and order into a crystalline-like state at high drives. We construct a dynamic phase diagram and show that the dynamic recoupling transition is associated with a double peak in $`dV/dI`$.

\]

In highly anisotropic superconductors such as BSCCO, the vortex lattice is composed of individual pancake vortices . These pancakes, which interact both magnetically and through Josephson coupling, align under certain conditions into elastic lines resembling those found in isotropic superconductors. Three-dimensional (3D) line-like behavior has been observed in transformer geometry measurements , muon-spin-rotation  and neutron scattering . Under different conditions, however, the pancake vortices in each layer move independently of the other layers, and the system acts like a stack of independent thin film superconductors. Such two-dimensional (2D) behavior has also been seen in transformer experiments . Thus in layered superconductors two different effective dimensionalities of the vortex pancake lattice may appear, each with different characteristic properties.

Layered superconductors exhibit a striking second peak in magnetization measurements , corresponding to an abrupt increase in the critical current of the material as the applied field is increased. This second peak is especially sharp in BSCCO, as shown in local Hall probe measurements  and recent Josephson plasma frequency measurements . The lattice appears ordered at fields below the transition and disordered above. There seems to be no widely accepted agreement on the mechanism behind this transition, although numerous scenarios have been suggested, including vortex entanglement , dislocation proliferations, dynamic effects , or a 3D to 2D transition in the vortex pancake lattice . The effect of strong disorder on a possible 3D-2D transition is unclear, and also it is not known how the transport properties would be affected by a 3D-2D transition.

In 2D systems with uncorrelated pinning, the vortex lattice can be dynamically reordered by an applied driving current, passing from a glassy state at zero drive, through plastic flow , to a reordered state at high current . In layered systems, when the between plane interactions of pancakes is weak enough that the pancakes are decoupled, the pancakes on each plane should reorder when a high enough driving current is applied. This reordering may also be accompanied by a dynamically driven recoupling transition , but it is unclear where this transition could be located in relation to the 2D reordering transition, and how it is affected by the externally applied magnetic field.

To address the issue of possible 3D-2D transitions in disordered anisotropic materials, we have developed a simulation which allows for decoupling by incorporating the correct magnetic interactions between vortex pancakes. We show that a sharp 3D-2D transition occurs when the relative strength of interlayer and intralayer pancake interactions is varied, and that this transition is associated with a sharp increase in the critical current. Furthermore, the system exhibits a rich array of dynamic 3D phases when driven by a current . We show that 2D decoupled pancakes can be dynamically recoupled in a transition that occurs simultaneously with the dynamic ordering in each plane. We construct a phase diagram as a function of interlayer coupling and applied driving force, and show that a sharp, experimentally observable second peak in $`dV/dI`$ is associated with the dynamic recoupling transition.

The magnetic interactions between pancakes in a layered material have the same logarithmic form present in thin films, but are highly anisotropic . We have performed simulations in which pancakes in all layers interact magnetically with long range interactions , in contrast to other simulations, which treated only nearest layer interactions . Our approach compliments calculations based on Lawrence-Doniach models . The pancake interactions are long range both in and between planes, and are treated according to Ref. . Josephson coupling is neglected as a reasonable approximation for materials in which the anisotropy $`\gamma `$ is sufficiently large .

The overdamped equation of motion, at $`T=0`$, for vortex $`i`$ is given by $`𝐟_i=_{j=1}^{N_v}_i𝐔(\rho _{i,j},z_{i,j})+𝐟_i^{vp}+𝐟_d=𝐯_i`$, where $`N_v`$ is the number of vortices, and $`\rho _{i,j}`$ and $`z_{i,j}`$ are the distance between pancakes $`i`$ and $`j`$ in cylindrical coordinates. The system has periodic boundaries in-plane and open boundaries in the $`z`$ direction. The magnetic energy between pancakes is

$`𝐔(\rho _{i,j},0)=2dϵ_0\left((1{\displaystyle \frac{d}{2\lambda }})\mathrm{ln}{\displaystyle \frac{R}{\rho }}+{\displaystyle \frac{d}{2\lambda }}E_1\right)`$ (1)
$`𝐔(\rho _{i,j},z)=s_m{\displaystyle \frac{d^2ϵ_0}{\lambda }}\left(\mathrm{exp}(z/\lambda )\mathrm{ln}{\displaystyle \frac{R}{\rho }}E_2\right)`$ (2)
where $`E_1=_\rho ^{\mathrm{}}𝑑\rho ^{}\mathrm{exp}(\rho ^{}/\lambda )/\rho ^{}`$, $`E_2=_\rho ^{\mathrm{}}𝑑\rho ^{}\mathrm{exp}(R/\lambda )/\rho ^{}`$, $`R=22.6\lambda `$, the maximum radial distance, $`ϵ_0=\mathrm{\Phi }_0^2/(4\pi \lambda )^2`$, $`\lambda `$ is the London penetration depth, $`d=0.005\lambda `$ is the interlayer spacing, and $`\xi `$ is the coherence length. We vary the relative strength of the interlayer coupling using the prefactor $`s_m`$. The uncorrelated pins are modeled by parabolic traps that are randomly distributed in each layer. The vortex-pin interaction is given by $`𝐟_i^{vp}=_{k=1}^{N_{p,L}}(f_p/\xi _p)(𝐫_i𝐫_k^{(p)})\mathrm{\Theta }((\xi _p|𝐫_i𝐫_k^{(p)}|)/\lambda )`$, where the pin radius $`\xi _p=0.2\lambda `$, the pinning force $`f_p=0.02f_0^{}`$, and $`f_0^{}=ϵ_0/\lambda `$. The case of stronger pins is considered in . The driving current must be increased slowly enough for the system to equilibrate at each drive . Here $`f_d`$ is increased by $`0.00025f_0^{}`$ every 35000 time steps. We have simulated a $`16\lambda \times 16\lambda `$ system with a vortex density of $`n_v=0.35/\lambda ^2`$ and a pin density of $`n_p=1.0/\lambda ^2`$ in each of $`L=8`$ layers. This corresponds to $`N_v=89`$ vortices and $`N_p=256`$ pins per layer, with a

total of 712 pancake vortices. We have checked for finite size effects on systems containing up to 42 layers and 3738 pancakes.

In the equilibrium state of the vortex lattice at zero drive, we find a sharp 3D-2D transition from vortex pancakes to vortex lines when the strength $`s_m`$ of the interlayer coupling is decreased, as shown in Fig. 1. We quantify the transition by measuring the spatial correlation of pancakes in neighboring planes, $`C_z=1(|(𝐫_{i,L}𝐫_{i,L+1})|/(a_0/2))\mathrm{\Theta }(a_0/2|(𝐫_{i,L}𝐫_{i,L+1})|)`$, where $`a_0`$ is the vortex lattice constant. A clear sharp drop in $`C_z`$ with decreasing $`s_m`$ appears in Fig. 1 at $`s_m=2.0`$ for a sample with $`L=8`$ layers. The 3D-2D transition is accompanied by a large increase in the critical current $`f_c`$, as seen in Fig. 1. For weak interlayer coupling, $`s_m2.0`$, the different layers of the sample depin independently and $`f_c`$ is close to the value it would have in a 2D sample. Here, $`f_c`$ is insensitive to the number of layers in the system, as can be seen by comparing the data from samples with $`L=`$4 to 16 in the inset of Fig. 1. In contrast, coupled lines of pancakes at $`s_m>2.0`$ average the random pinning over their length and become much less effectively pinned. Therefore $`f_c`$ decreases with increasing number of layers as seen in the inset of Fig. 1.

When the magnetic field $`H`$ increases, pancakes within a plane are brought closer together, but the distance between planes is unchanged. Thus increasing $`H`$ corresponds to weakening the coupling between planes by decreasing $`s_m`$. Therefore, our results support the suggestion that the sharp second peak observed in magnetization measurements results from a dimensional change in the vortex lattice from weakly pinned 3D vortex lines to strongly pinned 2D pancakes as the magnetic field is

increased . The behavior in $`f_c`$ that we observe indicates that a sharp change in transport properties can occur in a disordered system as a result of a change in the effective dimensionality. Furthermore, the sharpness of the transition found both here and in experiments suggests that the 3D-2D transition is first order.

We next consider the question of a possible dynamic recoupling transition by applying a driving force. When the vortex lattice begins to move for $`f_d>f_c`$, it undergoes plastic tearing due to the strong pinning in our sample. For decoupled samples with $`s_m2.0`$ each plane performs independent 2D plastic flow, as seen by the different vortex positions and trajectories in the top and bottom layer of the sample shown in Fig. 3(a,b). When high drives are applied, however, the pancakes recouple into lines and all planes begin to move in unison \[Fig. 3(c)\].

The reordering transition is shown in more detail in Fig. 2(a-c). Here, for a sample with $`s_m=1.5`$, the recoupling transition in $`C_z`$ \[Fig. 2(c)\] is sharp and occurs simultaneously with the in-plane reordering transition indicated by $`P_6`$ \[Fig. 2(b)\]. Furthermore, a sharp peak in $`dV/dI`$ appears at the transition, which will be discussed in more detail below. In contrast, as seen in Fig. 2(d-f), a sample with $`s_m=4.0`$ that is above the static 2D-3D transition contains vortices that move as stiff 3D lines, and shows the same reordering transitions seen in previous work on effectively two-dimensional systems .

We summarize the behavior of the system in the phase

diagram of Fig. 3. The coupled vortices with $`s_m>2.0`$ undergo plastic flow of stiff lines, pass through a smectic state, and finally reorder into a crystalline-like state at high drives. The plastic flow and smectic regions shrink as the number of layers is increased, and finally disappear, so that for $`L=16`$ and $`s_m=5`$ we observe elastic depinning directly into the 3D ordered state, with no plastic flow. Decoupled vortices with $`s_m2.0`$ depin into a 2D plastic flow phase in which each layer moves independently. The vortices switch abruptly from 2D to 3D behavior at the recoupling transition line, so they directly enter the 3D ordered state. The dynamic recoupling transition line and in-plane reordering transition line fall on top of each other in the phase diagram.

As shown in Fig. 4(b), the dynamic recoupling and simultaneous reordering are associated with a sharp peak in the $`dV/dI`$ curve. This peak is distinct from the broader peak in $`dV/dI`$ associated with plastic flow of the vortex lattice, as indicated in Fig. 4(d). The sharp peak disappears into the background value of $`dV/dI`$ and is not observed when the interlayer coupling becomes too weak. The height of the reordering peak increases rapidly as the static 2D-3D transition value of $`s_m`$ is approached from below, as indicated in Fig. 4(c), and simultaneously the current at which the reordering peak appears shifts downwards towards the location of the broad plastic peak. To understand the reordering peak, note that the 2D decoupled lattice depins at the high $`f_c`$ associated with the 2D limit. When the lattice recouples, it must cross from the low $`V_x`$ response in the 2D plastic limit up to the higher 3D elastic $`V_x`$ response curve at the recoupling transition. The location of the depinning transition is not affected by the value of $`s_m`$ since it is purely a 2D effect; however, the recoupling transition is shifted to lower driving currents as $`s_m`$ increases. As a result, the rising V(I) curve becomes steeper as the static transition point is approached from below, as can be seen for $`s_m=`$1.0, 1.5, and 2.0 in Fig. 4(a). The maximum value of $`dV/dI`$, which we call $`dV/dI_0`$, correspondingly increases, as shown in Fig. 4(c). When the static 2D-3D transition is crossed at $`s_m=2.0`$, the second sharp peak disappears. The behavior of this sharp peak in $`dV/dI`$, associated with the dynamic recoupling transition, should be experimentally observable in transport measurements performed at fields approaching the second peak from above, when the vortex pancakes are expected to be decoupled.

In summary, we have used a 3D molecular dynamics simulation employing the magnetic interactions of pancake vortices to study the dynamic phases of vortex matter in disordered highly anisotropic materials such as BSCCO. As a function of the relative interlayer coupling strength, we observe a sharp 3D-2D transition from vortex lines to decoupled pancakes. We find an abrupt large increase in the critical current as the 3D-2D line is crossed in a direction corresponding to increasing $`H`$, with decoupled pancakes being much more strongly pinned. At driving currents well above depinning, we find that the decoupled pancakes simultaneously recouple and reorder into a crystalline-like state at high drives. We construct a phase diagram as a function of interlayer coupling and show that the recoupling transition coincides with the single-layer recrystallization transition. We show that the recrystallization is associated with an experimentally observable double peak in $`dV/dI`$ and that the peak height grows rapidly as the static recoupling transition point is approached from below.

We acknowledge helpful discussions with L. N. Bulaevskii, D. Dominguez, C. Reichhardt, and R.T. Scalettar. This work was supported by CLC and CULAR (LANL/UC), by the NSF DMR 9985978, and by the Director, Office of Adv. Scientific Comp. Res., Div. of Math., Information, and Comp. Sciences, U.S. DoE contract DE-AC03-76SF00098.


text:
# TGRS OBSERVATIONS OF POSITRON ANNIHILATION IN CLASSICAL NOVAE

## I INTRODUCTION

Theoretical models of classical novae imply that large quantities of $`\beta `$-unstable proton-rich nuclei are formed during a thermonuclear runaway, with half-lives of the order minutes to hours. The resulting positrons undergo annihilation in the expanding nova envelope, giving rise to a pulse of annihilation $`\gamma `$-rays which is blueshifted and broadened by the nova velocity and lasts for up to $`6`$ hr (2).

All space-borne instruments are subject to a strong background emission line at 511 keV, arising from annihilation of cosmic ray induced $`\beta `$-decay positrons in the instrument, which hinders detection of the same line from cosmic sources. However, a spectrometer with sufficient energy resolution could distinguish between cosmic and background lines because the former is blueshifted, typically by 2–5 keV. The Ge spectrometer TGRS (FWHM resolution 3 keV at the beginning of its mission) fulfills this requirement. Its other advantages as a detector and monitor of the nova 511 keV line are its broad aperture (nearly $`2\pi `$ sterad) and its high altitude high inclination orbit, in which background count rates are rather low, due to the lack of interaction with Earth’s trapped radiation belts, and to the virtual absence of Earth albedo background radiation. The background count rate and spectrum has also been extremely stable throughout the mission.

A disadvantage of Ge detectors for long-term missions is progressive degradation of performance due to cosmic-ray impact damage. For this reason most of the data used here were obtained during 1995–1997, before this effect became serious. In an attempt to avoid the effects of degradation, such as low-energy ”tailing”, we fitted only energies on the high-energy (blue) side of the background 511 keV line. One such fitted background spectrum is shown in Fig. 1. The dashed line is the strong background 511 keV line. The dot-dashed line is the fit to a theoretical nova line emitted in a 6-hr interval after the event (3). Although the amplitude of this line is significant, we will see shortly that it is characteristic of all our background spectra — in other words, there is a systematic constant positive offset in this measurement. For scale, the dotted line shows how the same line would appear from a model nova at a distance 1 kpc. This line would be detected at a level $`8\sigma `$.

## II SURVEY OF THE SOUTHERN SKY, 1995–1997

The global rate of classical novae in the Galaxy is highly uncertain, since the two methods used to derive it both suffer from serious systematic problems. One involves correcting the rate of actual discoveries ($`3`$ yr-1) for highly uneven sky coverage and interstellar extinction for poorly-known nova distances. The other involves correcting the nova rates measured in external galaxies (according to some parameter such as $`M_B`$), which runs into the problem of differences in the stellar populations between galaxies.

The method used here is potentially superior, being free from these biases. The 511 keV $`\gamma `$-rays are not subject to interstellar extinction, and TGRS’s sky coverage was almost uniform during the period 1995–1997. Between 1995 January and 1997 October TGRS accumulated about $`7.7\times 10^7`$ s of background spectra (88% overall). Nor are much data lost due to changing sensitivity across the aperture, which amounts to a factor $``$25% between the zenith and the ecliptic plane.

This entire period was divided into 6 hr intervals during which background spectra were accumulated, which were fitted to models of the type shown in Fig. 1. The parameters of the nova line in these fits were fixed at values predicted by theory at epochs $`6`$ hr and $`12`$ hr after the explosion (widths 8 keV FWHM, blueshifts 5 and 2 keV, from ref. 2 and Hernanz 1997, private communication). The fits to the count spectra were normalized by the effective area for the zenith angle of an average nova, which is $`60^{}`$. The fitted amplitudes for the 6-hr and 12-hr lines were combined in order to maximize the signal.

The fitted nova line amplitudes in the 4005 resulting background spectra are shown in Fig. 2. Since the positive systematic amplitude is constant to a very good approximation, we are justified in subtracting it from all measurements. We then see that there is no single 6-hr interval which, after this subtraction, yields a convincing nova signal. We chose a level $`4.6\sigma `$ as the threshold for detection of a candidate signal (Fig. 2, dashed line), which is appropriate for the expected number of chance detections in 4005 events. From Poisson statistics, our measured nova rate is therefore $`<1`$ at the 63% level. The sensitivity of this survey at the $`4.6\sigma `$ level is about $`3.8\times 10^3`$ photon cm-2 s-1 (4).

If a given nova model predicts a 511 keV line flux $`\varphi _{pred}`$ at 1 kpc then this survey could have detected it out to a distance $`r_{det}=\sqrt{\varphi _{pred}(M)/3.8\times 10^3}`$ kpc. Unfortunately the most recent self-consistent hydrodynamic models (5) predict much smaller fluxes ($`<2\times 10^3`$ photon cm-2 s-1 at 1 kpc) than earlier versions (2), although parametrized models (6) can predict values values as high as $`7\times 10^3`$ photon cm-2 s-1. Thus in the most optimistic case $`r_{det}=1.4`$ kpc, for an ONeMg nova of model type HH5 (6). Using the Bahcall-Soneira Galactic mass model (1) we can extrapolate our limit $`<1`$ event per $`7.7\times 10^7`$ s within 1.4 kpc (which includes $``$0.8% of the Galaxy) to a Galactic rate $`<54`$ events yr-1 of this type.

## III NOVA VELORUM 1999

Five novae occurred within TGRS’s aperture during the period 1995–1997, before the instrument performance began to deteriorate. Searches for 511 keV line emission around the times of these outbursts were unsuccessful (3), the sensitivities achieved being similar to the values given above for the overall survey. Unfortunately a very bright nova V382 Vel ($``$ N Vel 1999) was discovered in 1999 May, by which time the deterioration of the detector performance had advanced.

Three main effects contribute to this deterioration (7): a distortion of the response to a narrow line (starting with low-energy tailing, culminating in the formation of a separate low-energy peak); a poorly-quantified loss of efficiency; and an accelerated rate of change of the instrument gain. In order to apply our search method to Nova Vel we attempted approximate solutions to these problems. In the count spectrum fitting, the Gaussian line models (Fig. 1) were replaced by more complex shapes reflecting the current detector response (evaluated from the response to the 511 keV and other background lines). Fortunately, during the rather narrow time interval (15–23 May) which we fitted the response did not change significantly, nor did the gain. We estimated the detector effective area by assuming constancy of the true TGRS background line strengths, which we had found to be a very good approximation during the period of stability 1995–1997; a measurement of the 511 keV line intensity, compared with the average from 1995–1997, then enabled an efficiency correction to be made.

The preliminary results are shown in Fig. 3. Note that, compared to Fig. 2, both the error bars and the systematic positive offset are much worse. The rise to visual maximum probably started at least 1 d before discovery (arrow), and the explosion may have occurred up to 3 d before that. Our fitted background nova line amplitude should show a single 6-hr peak at that epoch, over and above the average background level (solid line). No such event was observed, so that we can only place upper limits on the line flux and a lower limit on the distance to V382 Vel. The average $`3\sigma `$ upper limit above the solid line is $`<1.3\times 10^2`$ photon cm-2 s-1; however, the constant positive offset level is much less well-defined than it was earlier in the mission (Fig. 2), so that a systematic error estimate for this must be introduced. We also estimated the systematic error due to the uncertainty in the nova line position arising from the broad and irregular shape of the background 511 keV line relative to which it is measured. We thus obtained an upper limit on the line flux of order $`<1.6\times 10^2`$ photon cm-2 s-1. V382 Vel is believed to have occurred on an ONeMg white dwarf (8), in which case we can combine our flux limit with the best-case theoretical flux at 1 kpc of $`7\times 10^3`$ photon cm-2 s-1 (6) to obtain a lower limit of 0.66 kpc on the distance.