text:
# Study of the ammonia emission in the NGC 6334 region

## 1 Introduction

The NGC 6334 region, located at a distance of 1.74 kpc from the Sun (Neckel nec78 (1978)), is a complex of radio and infrared sources (Shaver & Goss shgo70 (1970); Haynes et al. hay78 (1978); McBreen et al. mcb79 (1979)). One of its components, NGC 6334 I(N), is also the strongest source in the NH3(J,K) = (1,1) transition (e.g., Schwartz et al. sch78 (1978)). It is also a strong continuum source at 400, 1000 and 1300 $`\mu `$m (Cheung et al. che78 (1978); Gezari gez82 (1982); Loughran et al. lou86 (1986)) but it is below the detection limit at wavelengths smaller than 137 $`\mu `$m. It has an H2O maser source, CH3OH masers (Moran & Rodríguez moro80 (1980); Kogan & Slysh kosl98 (1998); Walsh et al. wal98 (1998)), six reddened sources at the JHK bands (Tapia et al. tap96 (1996)), a bipolar molecular outflow in SiO and eight knots of H2 emission (Megeath & Tieftrunk meti99 (1999)). However, no H ii region associated to this source was detected.

The source NGC6334 I, located approximately 2′ south of NGC 6334 I(N), has H2O, OH and CH3OH masers (Moran & Rodríguez moro80 (1980); Gaume & Mutel gamu87 (1987); Forster & Caswell foca89 (1989); Menten & Batrla meba89 (1989)), an ultra-compact H ii region (Rodríguez et al. rod82 (1982); DePree et al. dep95 (1995)) and a cluster of stars detected at the JHK bands (Tapia et al. tap96 (1996)). It was studied with high angular resolution in the NH3(J,K) = (1,1) transition by Jackson et al. (jac88 (1988)) (hereafter, JHH88), who interpreted the observed structure as a circumstellar molecular disk rotating around a 30 $`M_{\mathrm{}}`$ O star. On the other hand, CO and H2 observations suggest that the structure arises from a molecular bipolar outflow (Bachiller & Cernicharo bace90 (1990); Persi et al. per96 (1996)). Due to these characteristics, it is probable that this region is in a more advanced stage of stellar formation than NGC 6334 I(N).

Both regions have been studied in several transitions of 12CO, 13CO and CS (Kraemer & Jackson krja99 (1999)), the meta-stable transitions of the ammonia molecule, including NH3(J,K) = (1,1), (2,2), (3,3) and (6,6) \[e.g., Schwartz et al. sch78 (1978); Forster et al. for87 (1987); Vilas-Boas et al. vil88 (1988); Jackson et al. jac88 (1988); Kuiper et al. kui95 (1995); Kraemer & Jackson krja95 (1995), krja99 (1999)\], and the non-meta-stable transition NH3(J,K) = (2,1) (Kuiper et al. kui95 (1995)). The physical parameters of these regions were always derived from the NH3(J,K) = (1,1) and (2,2) observations, assuming LTE conditions. In fact, the intensity anomalies in the nuclear quadrupole hyperfine structure of the NH3(J,K) = (1,1) spectrum are usually neglected, although they are observed in many warm molecular clouds (Stutzki et al. stu82 (1982); Batrla et al. bat83 (1983); Stutzki et al. stu84 (1984)).

Matsakis et al. (mat77 (1977)) interpreted the anomalous spectrum as the consequence of non-thermal population in the hyperfine states, induced by selective trapping in the hyperfine transitions of NH3(J,K) = (2,1) $``$ (1,1). This effect is relevant only when the width of the hyperfine lines lie between 0.3 and 0.6 km s-1. Due to this restriction, they assumed that the molecular cloud is formed by clumps, each one producing a narrow line spectrum, so that the observed spectrum would be the superposition of individual clump spectra. Based on this model, Stutzki & Winnewisser (stwi85 (1985)) (hereafter, SW85) elaborated a numerical algorithm to calculate the brightness temperature ratios between the hyperfine satellites and the main line in the NH3(J,K) = (1,1) spectrum, and also the brightness temperature ratio between the (2,2) to the (1,1) main lines, as a function of the NH3 column density, H2 density and kinetic temperature of each clump.

Gaume et al. (gau96 (1996)) questioned this model, based on observations of the NH3(J,K) = (1,1) absorption spectra towards the continuum source DR21. Their argument was based on an anti-correlation in the degree of LTE departure of the inner and outer hyperfine components in different regions of the cloud. They proposed instead, inflows and outflows of matter to explain the observations, but did not present any quantitative calculation to support their claim.

In this work, we used our high quality observations of the NGC 6334 region in the NH3(J,K) = (1,1) transition, which allowed the precise determination of the intensity anomalies in the hyperfine structure, to obtain with good reliability its physical parameters assuming non-LTE conditions. The data allowed us also to separate in the spectra the contribution of at least three different sources: NGC 6334 I, NGC 6334 I(N)w and NGC 6334 I(N)e.

## 2 Observations

The observations were made during July 1996 using the 13.7 m radome enclosed radio telescope of the Itapetinga Radio Observatory111Operated by CRAAE, Centro de Radio Astronomia e Aplicações Espaciais, São Paulo, Brazil. The receiver front-end consisted of a circularly polarized corrugated horn connected to a cooled HEMPT. An acusto-optical spectrometer was used in the back-end, with spectral resolution and total bandwidth of 70 kHz and 41 MHz respectively. The total system temperature was about 200 K. The angular resolution of the radio telescope at the frequency of NH3(J,K) = (1,1) transition was 4.2′. The observations were made using the ON-OFF total-power technique, switching between positions every 20 s with amplitude of 20′ in azimuth, enough to guarantee that there was no source in the OFF position. The effective integration time of each observation was 210 s. A 15 K noise source and a room temperature load were used in the calibration to obtain the gain and to correct for atmospheric attenuation (Abraham & Kokubun abko92 (1992)). The continuum point source Virgo A was used as a primary calibrator. The NGC 6334 region was mapped in the NH3(J,K) = (1,1) transition, covering 6′ in right ascension and 12′ in declination, with a spacing of 2′. The central position of the map, with equatorial coordinates $`\alpha (1950)=17^\mathrm{h}17^\mathrm{m}32^\mathrm{s}`$ and $`\delta (1950)=35\mathrm{°}42\mathrm{}`$, was observed several times during the mapping period and was used as secondary calibrator. This position was also observed in the NH3(J,K) = (2,2) transition.

A polynomial baseline was subtracted from the observations and the five Gaussian functions were fitted to the hyperfine NH3(J,K) = (1,1) spectra. The velocity separation between the Gaussians was fixed and the line-width of the hyperfine satellites constrained to be equal to the main line. This last assumption is well justified, since the optical depth of the main line is small ($`\tau 1.4`$).

The results from the observations are presented in Table 1. The first column shows the position in the map relative to the central coordinate, columns 2-6 exhibit the antenna temperature of each transition corrected for atmospheric attenuation, columns 7 and 8 the velocity and line-width of the main line and the last column the number of observations made in each position. The NH3(J,K) = (1,1) spectra of some points of special interest, as well as the NH3(J,K) = (2,2) spectrum of the central position are shown in Figure 1.

The NH3(J,K) = (1,1) spectra in all the observed positions exhibit systematic differences between the line intensities in each pair of hyperfine transitions (F=1$``$0, F=0$``$1) and (F=1$``$2, F=2$``$1), which are, at least in the strongest points, much larger than the rms measured at the baseline. These intensity differences can be interpreted as the signature of non-LTE conditions. However, we should point out that this conclusion cannot be obtained by looking only at the residuals of the line fittings, which present larger rms than the baseline, both under LTE and non-LTE assumption. This is expected an expected result, since the superposition of sources and the existence of velocity gradients can make the line shapes differ from Gaussian functions.

The antenna temperature distribution and gradients of velocity and line-width are presented in Figure 2. They agree with the results of Forster et al. (for87 (1987)) and Kuiper et al. (kui95 (1995)), who mapped with high spatial resolution a small region (about 100″ in diameter) around the central position in our map in the NH3(J,K) = (3,3) transition. However, the distribution of velocities differs from that found by Schwartz et al. (sch78 (1978)), since their map do not show an increase in the velocity’s modulus towards negative declinations, possibly due to the low signal-to-noise ratio in their spectra.

## 3 Results

### 3.1 Identification of the sources NGC 6334 I, NGC 6334 I(N)w and NGC 6334 I(N)e

Although the map in Figure 2 seems to be consistent with the presence of a single source centered in NGC 6334 I(N), high resolution observations (JHH88; Kraemer & Jackson krja99 (1999)) showed the existence of at least two ammonia features, with angular sizes of the order and smaller than 1′, centered at the positions of the continuum sources NGC 6334 I(N) and NGC 6334 I respectively. Taking into account the half-power beam width of the Itapetinga radio telescope, we find that the expected contribution of these two sources at the position ($`\mathrm{\Delta }\alpha ,\mathrm{\Delta }\delta `$) = (2′,6′) \[hereafter, just (2′,6′)\] should fall below the detection limit given by the rms of the observations. Since this is not the case, as can be seen in Figure 1(d), we conclude that another source is present in the region. We will describe, in the following paragraphs, the procedure used to separate the contribution of each component.

NGC 6334 I, when observed with high resolution (JHH88), exihibts two main structures, with velocities centered in -6.6 and -9.4 km s-1 and widths of 2.0 and 3.5 km s-1 respectively. The maximum flux densities of these structures are 0.42 and 0.48 Jy beam-1 respectively which, once diluted into the 4.2′ Itapetinga radio telescope beam, correspond to antenna temperatures of 0.20 and 0.23 K respectively. We calculated the contribution of this source to the NH3(J,K) = (1,1) spectra in our map as the convolution of the antenna beam and two point sources, with the characteristics mentioned above. The intensity of the hyperfine satellite lines was calculated assuming LTE in an optically thin source. Although the antenna temperature of NGC 6334 I is small compared to the intensity of NGC 6334 I(N), its effect on the line-width is significant, and its subtraction decreased the observed gradient along the map.

Since the high resolution observations of Kraemer & Jackson (krja99 (1999)) have revealed NGC 6334 I(N) as an extended source, although smaller than the Itapetinga radio telescope beam, it was difficult to subtract its contribution to expose the third and much weaker source. For this reason we used the spectrum at position (2′,6′), where we did not expect any contribution from the strongest source, as representative of the spectrum of the weakest. We labeled these sources as NGC 6334 I(N)w and NGC 6334 I(N)e \[hereafter, I(N)w and I(N)e\] respectively. We assumed no velocity gradients and subtracted a fraction of the (2′,6′) spectrum from all the others. This fraction was chosen in such a way that the resultanting spectra had the same departure from LTE as found at position (0′,0′). Moreover, since there is a velocity gradient along the region, the subtracted fraction was also limited in order to avoid fake absorptions in the resulting spectra.

In Figure 3, we show the superimposed antenna temperature maps of the three regions, NGC 6334 I, I(N)w and I(N)e. We can see that I(N)w is aproximately 2.5 and 6.5 times brighter than I(N)e and NGC 6334 I, respectively. The source I(N)w turned out to be smaller than the beam size, as expected from the observations of Kraemer & Jackson (krja99 (1999)). On the other hand, the I(N)e region looks more elongated in declination.

The subtraction procedures affected the magnitude of the gradients of velocity and line-width, but not their directions. In fact, the gradients became softer than before, showing that the other sources were responsible in some amount for the variations of these properties along the region I(N)w. As it can be seen in Figure 4, the velocity gradient is aproximately perpendicular to the Galactic plane. According to the model adopted here, the gradient in line-width is related to gradient in turbulence, which turned out to be larger in the direction of the more evolved sources.

### 3.2 The physical parameters of I(N)w and I(N)e regions in non-LTE conditions

We used the antenna temperature of the hyperfine satellites relative to the main line in the NH3(J,K) = (1,1) spectrum and the ratio between the antenna temperatures of the main lines of NH3(J,K) = (2,2) and (1,1), shown in Table 2, to calculate the relevant physical parameters for the I(N)w and I(N)e regions. We applied the model of SW85 to obtain the NH3 column density ($`N_{\mathrm{NH}_3}`$), the kinetic temperature ($`T_\mathrm{K}`$) and the H2 density ($`n_{\mathrm{H}_2}`$) of the clumps in the two regions. In region I(N)w we used the spectrum from the (0′,0′) position, which corresponds to the highest intensity, and in I(N)e we used the (2′,6′) position, where we believe there is not important contribution from I(N)w.

To determine the physical parameters of the clumps we looked into the ($`N_{\mathrm{NH}_3}`$, $`n_{\mathrm{H}_2}`$, $`T_\mathrm{K}`$) space for the point at which, according to the model of SW85, the curves representing the observed line intensity ratios intercept themselves. We had to extrapolate these curves up to H2 densities of 108 cm-3 to obtain a good crossing point, which occured at $`n_{\mathrm{H}_2}10^{7.5}`$ cm-3.

In Figure 5, we show the curves of constant ratio between the hyperfine satellites and the main line in the NH3(J,K) = (1,1) spectrum, as well as the ratio between the main line intensities of the NH3(J,K) = (2,2) and (1,1) transitions for a kinetic temperature of 24 K, which corresponds to the best fit in both regions. In Table 3, we present the values of the physical parameters extracted from this figure: the brightness temperature for the NH3(J,K) = (1,1) and (2,2) transitions \[$`T_\mathrm{B}(1,1)`$ and $`T_\mathrm{B}(2,2)`$, respectively\], the kinetic temperature, the NH3 column density and the H2 density. We can see that the physical conditions of the clumps in both regions are pratically the same. It is important to stress that the values listed in Table 3 would not change substantially if we use the original spectra, including the contribution of the three sources, since I(N)w is much brighter than the others. On the other hand, the dispersion of the contours around the intersection point in Figure 5(a) decreased appreciably after the separation of the three components, resulting in a better accuracy in the determination of the physical parameters of the region.

### 3.3 Physical parameters of the individual clumps

As it was previously mentioned, to explain the hyperfine structure intensity anomalies in the NH3(J,K) = (1,1) transition by the superposition of individual clump spectra, their dispersion velocities must be small, implying that they should be gravitationally stable. Therefore, their individual masses ($`M_{\mathrm{cl}}`$) cannot be larger than the Jeans mass ($`M_\mathrm{J}`$) and their diameters ($`L_{\mathrm{cl}}`$) must be smaller or equal than the Jeans length ($`L_\mathrm{J}`$). Arguments in favor of these hypotheses have been presented by Curry & McKee (cumc00 (2000)) based on numerical simulations. Assuming spherical clumps with a ratio of helium to molecular hydrogen of 0.25, we calculated their diameter and mass, using the following expressions:

$`L_{\mathrm{cl}}=L_\mathrm{J}=\sqrt{{\displaystyle \frac{\pi kT_\mathrm{K}}{7.2Gm_\mathrm{H}^2n_{\mathrm{H}_2}}}}`$ (1)
$`M_{\mathrm{cl}}=M_\mathrm{J}={\displaystyle \frac{\pi }{2}}m_\mathrm{H}n_{\mathrm{H}_2}L_{\mathrm{cl}}^3`$ (2)
where $`k`$ is the Boltzmann’s constant, $`m_\mathrm{H}`$ the mass of hydrogen atom and $`G`$ the gravitational constant.

The beam filling factor ($`f_{\mathrm{JK}}`$) can be calculated from the brightness temperature, $`T_\mathrm{B}(J,K)`$, derived from the model and $`T_\mathrm{B}^{obs}(J,K)`$, obtained from the observations, which is proportional to the antenna temperature:

$`f_{\mathrm{JK}}={\displaystyle \frac{T_\mathrm{B}^{obs}(J,K)}{T_\mathrm{B}(J,K)}}={\displaystyle \frac{2c^2}{\pi k\nu ^2}}{\displaystyle \frac{A}{\theta _\mathrm{B}^2}}{\displaystyle \frac{T_\mathrm{A}(J,K)}{T_\mathrm{B}(J,K)}}`$ (3)
where $`\theta _\mathrm{B}`$ is the half-power beam width of the telescope and $`A`$ is the ratio between flux density and antenna temperature of a continuum point source. In this work, we used the radio galaxy Virgo A as the calibrating source.

Using the definition of beam filling factor, we can determine the number $`K_{\mathrm{cl}}`$ of clumps inside the beam’s solid angle:

$`K_{\mathrm{cl}}=f_{\mathrm{JK}}\left({\displaystyle \frac{D\theta _\mathrm{B}}{L_{\mathrm{cl}}}}\right)^2`$ (4)
where $`D`$ is the distance from the cloud.

To obtain the density of clumps ($`\zeta _{\mathrm{cl}}`$), we can consider two different situations. The first one corresponds to the clumps spread over a solid angle larger than the beam. In this case, the density of clumps is given by:

$`\zeta _{\mathrm{cl}}={\displaystyle \frac{4}{\pi }}{\displaystyle \frac{K_{\mathrm{cl}}}{D^3\theta _\mathrm{B}^2\sqrt{\theta _\alpha \theta _\beta }}}`$ (5)
where $`\theta _\alpha `$ and $`\theta _\beta `$ are, respectively, the extension of the region in right ascension and declination. In the second one, if the source is smaller than the antenna beam, we can only estimate either an upper limit, considering that the clumps are grouped side by side and without superposition, or a lower limit, supposing they are spread over the whole beam:

$`{\displaystyle \frac{6}{\pi }}{\displaystyle \frac{K_{\mathrm{cl}}}{\left(D\theta _\mathrm{B}\right)^3}}\zeta _{\mathrm{cl}}{\displaystyle \frac{6}{\pi }}{\displaystyle \frac{K_{\mathrm{cl}}}{\left(D\theta _\mathrm{B}\sqrt{f_{\mathrm{JK}}}\right)^3}}`$ (6)
Once the density of clumps and their diameters are known, we can calculate their collision time ($`t_{\mathrm{coll}}`$):

$`t_{\mathrm{coll}}={\displaystyle \frac{8\sqrt{ln2}}{\pi L_{\mathrm{cl}}^2\zeta _{\mathrm{cl}}\mathrm{\Delta }v_{\mathrm{FWHM}}^{JK}}}`$ (7)
We can also calculate the mean H2 density ($`\overline{n}_{\mathrm{H}_2}`$), which is the density of an homogeneous cloud with mass equal to the total mass of the clumps filling the telescope beam:

$`\overline{n}_{\mathrm{H}_2}={\displaystyle \frac{M_{\mathrm{cl}}}{3m_\mathrm{H}}}\zeta _{\mathrm{cl}}`$ (8)
The ammonia abundance is calculated from the following expression:

$`X_{\mathrm{NH}_3}={\displaystyle \frac{N_{\mathrm{NH}_3}}{L_{\mathrm{cl}}n_{\mathrm{H}_2}}}`$ (9)
where it was assumed that there is not superposition between the clumps, so that the line of sight extension is equal to the diameter of each clump (SW85).

The physical parameters of the clumps in regions I(N)w and I(N)e and their respective uncertainties are listed in Table 4. The quoted errors for the mass and diameter of the clumps represent the uncertainties in the kinetic temperature and H2 density.

The clumps in both regions have similar diameters (0.007 pc), masses (0.2 M) and ammonia abundances (7$`\times `$10-9). They also seem to be gravitationally bounded, since the total virial mass is about a factor of ten lower than the total mass of the clumps which are contained in the beam’s solid angle \[7600 and 2100 M for I(N)w and I(N)e respectively\]. The 36000 clumps of I(N)w are spread in such a way that their density is between 6000 and 33000 clumps per pc-3, much larger than the 720 clumps per pc-3 in the I(N)e region.

The beam filling factor for the I(N)w region is larger than expected from the NH3(J,K) = (3,3) data presented in Kraemer & Jackson (krja99 (1999)) (around 0.10 for a beam size of 4′). The reason is that we had assumed no superposition between the clumps along the line of sight. The physical superposition required by the smaller filling factor is small enough to guarantee very little velocity superposition, in which case the spectra of the superposed clumps can be simply added.

## 4 Discussion

### 4.1 Ammonia abundance for the regions I(N)w and I(N)e

The study of ammonia abundance is important because it provides observational constrains for the chemical models that try to explain how this molecule is formed in the interstellar medium, and it may be used as an age indicator of the molecular cloud (e.g., Suzuki et al. suz92 (1992)). Although the chemical models calculate the evolution of the ammonia abundance as a function of time with a limited confidence, due mainly to the uncertainties concerning the formation of this molecule in the interstellar medium, they agree that the time necessary to reach the equilibrium abundance is approximately 106 years. If depletion effects are not consider in the model, the equilibrium abundance for the ammonia molecule is 10-7. If there is depletion, this value might be 10-8 (Bergin et al. ber95 (1995)) or 10-9 (Hasegawa & Herbst hahe93 (1993)), even though in this last case the abundance reaches equilibrium after 108 years.

Comparing the ammonia abundances listed in Table 4 with the equilibrium abundance, we conclude that either the regions I(N)w and I(N)e have an age lower than 106 years, or that ammonia depletion is important in both sources. This result was also obtained by Kuiper et al. (kui95 (1995)), assuming LTE conditions. Another relevant aspect that must be taken into account is the dependence of the equilibrium abundance on the nitrogen abundance in the interstellar medium, and therefore the gradient of nitrogen with galactocentric distance must be also considered (Vilas-Boas & Abraham viab00 (2000)).

### 4.2 Comparison with LTE models

In this sub-section, we will derive, under assumptions of LTE, the physical parameters of the source I(N)w and compare them with the parameters derived in section 3 under non-LTE conditions. We calculate these parameters at the (0′,0′) position because it is the only point in which the NH3(J,K) = (2,2) transition was observed. In the LTE calculation we considered a three-level system, in order to obtain the rotational and kinetic temperatures ($`T_\mathrm{R}`$ and $`T_\mathrm{K}`$ respectively) through the equations (Ho & Townes hoto83 (1983); Walmsley & Ungerechts waun83 (1983)):

$`T_\mathrm{R}=41.5\times \left[\mathrm{ln}\left({\displaystyle \frac{9}{20}}{\displaystyle \frac{\tau _{22}}{\tau _{11}}}{\displaystyle \frac{\mathrm{\Delta }v_{\mathrm{FWHM}}^{22}}{\mathrm{\Delta }v_{\mathrm{FWHM}}^{11}}}\right)\right]^1`$ (10)
$`T_\mathrm{K}={\displaystyle \frac{T_\mathrm{K}}{1+\left(T_\mathrm{K}/41.7K\right)\mathrm{ln}\left[1+\frac{C\left(2221\right)}{C\left(2211\right)}\right]}}`$ (11)
where $`\tau _{11}`$ and $`\tau _{22}`$ are respectively the optical depths in the transitions NH3(J,K) = (1,1) and (2,2), C(22 $``$ 21) and C(22 $``$ 11) are the collision rate coefficients given by Danby et al. (dan88 (1988)).

To determine the optical depth, we assumed that the cloud has a spherical shape, that the excitation temperature ($`T_{\mathrm{ex}}`$) is the same in all hyperfine lines of the NH3(J,K) = (1,1) spectrum (Ho & Townes hoto83 (1983)). Thus, using the ratio between the (1,1) main line and its inner hyperfine satellites, $`\tau _{11}`$ can be calculated from:

$`{\displaystyle \frac{T_\mathrm{A}^\mathrm{m}(1,1)}{T_\mathrm{A}^\mathrm{s}(1,1)}}={\displaystyle \frac{1e\left(\tau _{11}^\mathrm{m}\right)}{1e\left(\tau _{11}^\mathrm{s}\right)}}`$ (12)
where $`\tau _{11}^\mathrm{m}`$ and $`\tau _{11}^\mathrm{s}`$ are, respectively, the optical depth for the main line and the inner satellites, with $`\tau _{11}^\mathrm{s}`$ = 0.28 $`\tau _{11}^\mathrm{m}`$ (Ho & Townes hoto83 (1983)) and $`e(\tau )=2\tau ^2[1e^\tau (1+\tau )]`$ (SW85). To obtain the optical depth for the NH3(J,K) = (2,2) transition, it is necessary to substitute in (12) $`T_\mathrm{A}^\mathrm{s}(1,1)`$ by $`T_\mathrm{A}(2,2)`$, the antenna temperature of the (2,2) main line.

As we have less equations than unknown parameters, it is necessary to make some assumptions to solve the system. There are at least two clear alternatives: we may assume that either the excitation and rotational temperatures have the same value, or that the source fills completely the antenna beam. Here, we adopted the first alternative since high resolution observations indicate filling factors smaller than one for these sources (Kuiper et al. kui95 (1995); Kraemer & Jackson krja99 (1999); Megeath & Tieftrunk meti99 (1999)).

We present in Table 5 the physical parameters obtained under LTE and non-LTE conditions. The NH3 column density and the ammonia abundance are not sensitive to LTE departures. The largest discrepancies are found in the kinetic and rotational temperatures. In fact, the kinetic temperature estimated under LTE conditions is about twice the value obtained in non-LTE. The optical depth of I(N)w, in both transitions, is slightly smaller than what it was found in previous studies (e.g., Forster et al. for87 (1987); Vilas-Boas et al. vil88 (1988); Kuiper et al. kui95 (1995)). We believe that the differences can be attributed to the lower signal-to-noise ratio in previous observations. The beam filling factor is larger under LTE conditions. This was already expected because under non-LTE conditions the gas is confined in clumps and not spread in a homogeneous cloud.

### 4.3 Evolutionary state of the regions

We will try to determine now the evolutionary state of the three regions identified in our ammonia observations. A possible way to explore this question is comparing the distribution of indicators of early stages of star formation. In Figure 3, we show the distribution of infrared sources, detected in the JHK bands by Tapia et al. (tap96 (1996)) in a small region of the mapped area. We can see a decrease in the number of infrared sources towards the region I(N)w and also the existence of a cluster of sources around NGC 6334 I, indicating that it is more evolved than I(N)w. Other observational results favorable to this scenario are the detection of an UCH ii region and the identification of a molecular bipolar outflow associated to this source (Rodríguez et al. rod82 (1982); Bachiller & Cernicharo bace90 (1990)). There are no infrared observations towards I(N)e, to indicate its evolutionary stage.

Our observations, together with the non-LTE model (SW85), show that the physical properties of the clumps in I(N)w and I(N)e are similar. However, the volume density of clumps in I(N)w ($``$ 6000 pc-3) is at least a hundred times higher than in I(N)e, resulting in a collision rate which is hundred times higher in I(N)w than I(N)e. Thus, assuming that collisions result in coagulation of the fragments, which finally collapse to form stars after several collision times (SW85; Scalo & Pumphrey scpu82 (1982)), we expect that I(N)e is in a earlier star formation stage than I(N)w.

The evolution scenario proposed by SW85 and the results presented in this work indicate that the regions I(N)w and I(N)e could form in the future star clusters, similar to those detected in NGC 6334 I.

## 5 Conclusions

The NH3(J,K) = (1,1) spectra from the NGC 6334 region exhibit hyperfine structure intensity anomalies, suggesting departures from LTE conditions. We have used the numerical model of SW85, based on the qualitative model of Matsakis et al. (mat77 (1977)) to calculate the physical parameters of the ammonia clouds. This model takes into account the non-LTE effects through selective trapping in the far IR transition NH3(J,K) = (2,1) $``$ (1,1) and hyperfine selective collisional excitation. Its main limitation is that it does not include the effects of an IR continuum at the NH3(J,K) = (2,1) $``$ (1,1) transition frequency. As it was stressed by SW85, its influence on the anomalies is difficult to predict, but they argued that if the IR radiation field had significant influence, variations in their strength with the distance from the exciting source would be seen, which is not supported by observations, at least in the regions S106 and W48.

The existence of clumps with very small dispersion velocity was questioned by Gaume et al. (gau96 (1996)), based on the NH3 absorption spectra in the direction of the continuum source DR21. They proposed, instead, inflows and outflows of matter in different regions of the cloud. However, the mechanism producing the non-LTE level population is the same as in Matsakis et al. (mat77 (1977)), that is, the selective trapping of photons in the NH3(J,K) = (2,1) $``$ (1,1) transition. Since this model was not developed quantitatively, it is not clear if it will work to explain the observed anomalies.

Although our observations do not have high angular resolution, our high quality spectra allow the identification of three distinct sources. The first one is NGC 6334 I, a compact source mapped in NH3(J,K) = (1,1) transition by JHH88, reveling the presence of a possible bipolar molecular outflow in NH3 and CO (Bachiller & Cernicharo bace90 (1990)), having also a cluster of IR sources (Tapia et al. tap96 (1996)). The second region is NGC 6334 I(N)w, which also presents a bipolar molecular outflow in SiO (Megeath & Tieftrunk meti99 (1999)), is the most intense ammonia source in the sky, but is not spatially resolved in our observations. It is formed by clumps with diameters of 0.007 pc, masses of 0.2 M, kinetic temperatures of 24 K and with a total mass of about 7600 M. This result suggests larger masses for this region than estimated by Kuiper et al. (kui95 (1995)), assuming LTE conditions. It also exihibits velocity gradients almost perpendicular to the Galactic plane, and line-width gradients parallel to the Galactic plane, showing larger line-widths towards the more evolved sources. Finally, the region NGC 6334 I(N)e is formed by clumps with physical conditions similar to those observed in I(N)w, and spread through an area larger than the telescope beam.

The ammonia abundances in both regions are similar. Their values indicate that the regions have either ages smaller than 106 years or that depletion effects are important.

Among the three regions identified, NGC 6334 I is the most evolved while NGC 6334 I(N)e is in a very early star formation stage.

Comparison between physical parameters determined under LTE and non-LTE conditions showed that the NH3 column density and ammonia abundance are the only quantities which are similar between them. Other parameters, as rotational and kinetic temperatures, exhibit differences that can reach a factor of three.

###### Acknowledgements.

This work was supported by the Brazilian Agencies FAPESP and CNPq. CRAAE is formed by agreement between INPE, UNICAMP, USP and University of Mackenzie. We would like to thank the referee, Dr. E. A. Bergin, for careful readings of the manuscript and for useful suggestions.


format: text

text:
# A Deep Multicolor Survey. VI. Near-Infrared Observations, Selection Effects, and Number CountsBased on observations obtained at MDM Observatory, operated by Columbia University, Dartmouth College, the University of Michigan, and the Ohio State University

## 1 Introduction

The surface density of galaxies as a function of integrated brightness, also known as the number–magnitude relation, is one of the classic tests of observational cosmology. This test has a long, distinguished history as a tool for studying the nature of the universe and the evolution of galaxies (e.g. Hubble, 1934; Sandage, 1961; Tinsley, 1977). Surveys to progressively deeper apparent magnitude limits have measured the number–magnitude relation at a range of wavelengths, and this work has been reviewed by Koo & Kron (1992) and Ellis (1997). The observed number–magnitude relations are compared to the predictions of galaxy evolution models, such as those by Yoshii & Takahara (1988), Guiderdoni & Rocca-Volmerange (1990), and Gardner (1998) in order to determine cosmological parameters and study galaxy formation and evolution.

Near-infrared (1 – 2.5$`\mu `$m, hereafter NIR) galaxy counts are a valuable addition to studies of galaxy formation and evolution with the number–magnitude relation. The spectral energy distributions (SEDs) of most galaxies are relatively constant as a function of wavelength in the NIR because this wavelength regime is dominated by light from old, evolved stars. The shape and uniformity of galaxy SEDs in the NIR results in relatively small $`k`$-corrections for galaxies up to $`z1`$ and therefore the NIR luminosity is representative of the total stellar luminosity. A recent burst of star formation, in contrast, can significantly increase the brightness of a galaxy at visible and $`UV`$ wavelengths. This may cause a galaxy with a large amount of current star formation to appear as bright as a more quiescent galaxy with a much larger stellar mass. The presence of a great deal of dust will also attenuate the brightness of galaxies at these wavelengths and further complicate the relation between $`UV`$ or visible luminosity and total stellar luminosity. NIR number counts to a limit of $`20`$ mag, which are dominated by galaxies at $`z<1`$, provide a less uncertain means of evaluating models of galaxy formation. The old, evolved stars that dominate the NIR light are also good tracers of the total stellar mass and therefore NIR measurements are an excellent method for direct study of the stellar mass evolution in galaxies in the context of hierarchical galaxy formation (Brinchmann & Ellis, 2000).

NIR number counts to date have mostly been obtained at $`K`$ (Gardner et al., 1993; Cowie et al., 1994; Djorgovski et al., 1995; McLeod et al., 1995; Gardner et al., 1996; Moustakas et al., 1997) as this is the longest-wavelength atmospheric transmission window with sufficient sensitivity to detect high-redshift galaxies in a reasonable time. Faint counts have also been obtained at $`H`$ (Teplitz et al., 1998; Yan et al., 1998) with NICMOS on the Hubble Space Telescope (HST), though given its small field of view, there are few bright galaxies in this sample. Some observations have also been obtained at both $`J`$ and $`K`$ (Bershady et al., 1998; Teplitz et al., 1999; Väisänen et al., 2000) and these studies have used the mean NIR colors as a function of apparent magnitude as an additional constraint for galaxy evolution models.

I have obtained $`J`$, $`H`$, and $`K`$ observations of 185 square arcminutes of high galactic latitude fields which extend to $`J21`$, $`H20`$, and $`K18.5`$. The median redshift to $`K=18.5`$ is $`z0.5`$ in the $`K`$-selected Hawaii Redshift Survey (Cowie et al., 1996). These observations are made up of 21 subfields of the Deep Multicolor Survey (DMS) fields observed by Hall et al. (1996a). The DMS covers a total area of 0.83 square degrees in six filters: $`UBVRI_{75}I_{86}`$. The DMS is comprised of 6 different fields and to date this rich dataset has been used to study the quasar luminosity function (Hall et al., 1996b; Kennefick et al., 1997), the evolution of blue galaxies (Liu et al., 1998), and the M dwarf distribution, mass, and luminosity function (Martini & Osmer, 1998). The selection criteria for the DMS fields were relatively low galactic foreground extinction and avoidance of very bright stars. Five of the fields are equatorial to facilitate follow-up spectroscopy from both hemispheres.

In §2 I describe the NIR observations, while in §3 I discuss the image processing, photometric solution, and catalogs. I discuss the detection efficiencies in §4, galaxy photometry in §5, and star/galaxy separation in §6. The galaxy number counts and NIR colors of these galaxies are presented in §§7 and 8. In a separate paper (Martini, 2000) I combine these NIR observations with the $`UBVRI_{75}I_{86}`$ CCD data to compute photometric redshifts for this NIR sample to study the visible-NIR colors of these galaxies, particularly the Extremely Red Objects.

## 2 Observations

A total of 21 randomly-selected subfields of the DMS were observed through the NIR $`J`$, $`H`$, and $`K`$ filters. These images were obtained with TIFKAM111TIFKAM: The Instrument Formerly Known As MOSAIC. This instrument is also known as ONIS at Kitt Peak. (Pogge et al., 1998), a NIR imager/spectrograph with a 512x1024 InSb detector on the 2.4m Hiltner telescope of the MDM Observatory. The f/7 camera was used for all of the observations; this camera gives a plate scale of $`0.3^{\prime \prime }`$ pix-1 on the Hiltner telescope. Table 1 is a log of the observations, listing the date each field was observed in each filter and the seeing measured in that field/filter combination. Each of the 21 fields was observed for a total of 60 minutes on-source integration time per filter. The observations were obtained in a six-point dither pattern with a 10′′ – 15′′ offset between positions. Individual exposures in the dither pattern were one to two minutes and comprised a combination of integration time and coadds such that the sky level in the frames was less than 10000 counts. Tests of the detector linearity showed that the array is less than 1% nonlinear at this count level. Dark frames were taken nearly every night with the same combination of exposure time and number of coadds as the science frames.

Standard stars from Persson et al. (1998) were observed each night at a range of airmass. A series of red stars was also observed to determine the transformation of the instrumental magnitude system to the CTIO/CIT photometric system (Elias et al., 1982). Red stars were observed at least one night per observing run as there were several changes to the filters over the course of the observations for this program. These changes are discussed below in §3.2. The standard stars were observed in a five or six point dither pattern with 15′′ offsets, though with only 15 – 30 seconds on-source integration per dither position.

## 3 Image Processing

### 3.1 Survey Fields

All of the data were reduced with IRAF222IRAF is distributed by the National Optical Astronomy Observatories, which are operated by the Association of Universities for Research in Astronomy, Inc., under cooperative agreement with the National Science Foundation. and many of the data processing steps took advantage of the PHIIRS333Available at http://iraf.noao.edu/iraf/web/contrib.html package, a collection of IRAF routines compiled by Pat Hall. The first step in the data processing was subtraction of a dark frame from each image. The individual images were then flat-fielded and sky-subtracted using “running” flat and sky frames. The running flats were created by averaging the six neighboring images using a percentile clipping algorithm to remove bright objects. This image was then normalized to unity to create a skyflat for each image. The running sky frames were similarly created by averaging the six neighboring frames. At this stage the images were inspected and images with anomalously high noise or serious tracking/guiding errors (generally two to three per field) were removed from further processing. The good images were then shifted and coadded together using an offset table of integer shifts to create a final mosaic image. This image was used to identify objects and, together with the offset table, was used to make an object mask for each of the individual frames. The flat field and sky subtraction steps were then repeated using these individual object masks.

TIFKAM has a noise pattern in the lower half of the array due to vibration of the mechanical cryocooler. This pattern is a few counts in amplitude and runs along rows in the form of a sine wave with a period six times the width of the array. Because the period is much longer than the array width and the amplitude is less than the sky noise, simply subtracting the average of each row, after masking out objects, removes most of this feature. After the cryocooler pattern was subtracted, the individual images were shifted and added into the final mosaic. This region was then trimmed to only include the area of the sky present in more than 83% of the individual frames. As the offsets were generally small and consistent, this step produced a final mosaic of relatively constant noise with minimal loss of field. Several of the fields in Table 1 were initially observed under nonphotometric conditions. Calibration images of these fields were obtained on photometric nights and these images were reduced in the same manner as above. Three to six bright, stellar objects were used to calibrate the fields observed under nonphotometric conditions.

### 3.2 Standards

The standard stars were processed in an identical manner to that outlined above, though with two exceptions. First, the cryocooler pattern was not subtracted from these frames. Tests of the images with artificial stars showed that this pattern did not affect the photometry of bright stars. Secondly, the five to six observations of each standard were photometered individually, rather than one measurement of a shifted and added frame. The standard stars were measured with PHOT using a $`15^{\prime \prime }`$ radius aperture and a mean sky measured in an annulus extending from $`25^{\prime \prime }`$ to $`35^{\prime \prime }`$ radius.

To convert the instrumental magnitudes to the CTIO/CIT system, the following equations were solved:

$$\begin{array}{c}J=j+j_0+j_1(X1.3)+j_2(JK1.4)\\ H=h+h_0+h_1(X1.3)+h_2(JK1.4)\\ K=k+k_0+k_1(X1.3)+k_2(JK1.4)\end{array}$$
(1)
where $`j,h,k`$ are the instrumental magnitudes, $`j_0,h_0,k_0`$ are the photometric zeropoints, $`j_1,h_1,k_1`$ are the airmass coefficients, and $`j_2,h_2,k_2`$ are the color terms. $`X=1.3`$ is the mean airmass of the fields observed in this survey and $`JK=1.4`$ is approximately the mean $`JK`$ color of galaxies in the range $`K=1619`$ reported by Saracco et al. (1999). While the photometric zeropoints and airmass coefficients may vary from night to night, the color terms should not vary so long as the filters remain unchanged. To take advantage of the observations of red stars over several nights, I therefore performed a multidimensional fit (e.g. Gould, 1995) to all of the nights of a given observing run simultaneously. For each filter, this allowed the zeropoint and airmass coefficient to vary but not the color term. Over a given observing run, the photometric zeropoint varied by a few percent. The airmass coefficient was more variable, with large changes generally temporally correlated with changes in the weather (as discussed by Frogel, 1998). If standards at large airmass were not observed on a particular night, the average coefficient of the temporally adjacent nights on that observing run was used. Most of the fields were observed at $`X<1.5`$ (and all at less than two) so uncertainties in the airmass coefficients translate to less than one percent uncertainties in the photometry. All of the photometric solution coefficients are listed in Table 2, along with the root-mean-square variation in these solutions.

As mentioned above, there were several changes to the filters in TIFKAM over the course of this project. The $`J`$ filter used in 1997 October had considerable structure in the wings of the point spread function (PSF) and therefore it was replaced before the 1998 April observing run. The new $`J`$ filter had a red leak such that the measured sky brightness was $`\mathrm{\Sigma }_J13`$ mag arcsec-2, compared to measurements of $`\mathrm{\Sigma }_J15.516`$ mag arcsec-2 on the other observing runs. After 1998 April this $`J`$ filter was only used in conjunction with a piece of PK50 glass in the prefilter wheel, which serves as a red-blocking filter. PK50 is opaque longward of $`2.7\mu `$m, but transparent at shorter wavelengths. A piece of PK50 was also added to the same filter cell as the $`H`$ filter between the 1998 April and 1998 September observing runs due to the possible presence of a slight red leak in this filter. Because of the changes in the $`J`$ and $`H`$ filters, the photometric solution has three separate color terms for the $`J`$ filter and two separate color terms for the $`H`$ filter. The $`K`$ filter was unchanged over the course of this observing program and therefore only one color term was sought in this equation.

### 3.3 Object Catalogs

I cataloged the positions of all objects in these fields with the SExtractor package (Bertin & Arnouts, 1996) and the default object detection filter. This choice was motivated by the data reduction procedure, which creates images with “dead” regions, corresponding to areas with low exposure times and higher sky noise in the final shift-and-add step. The weight map option in SExtractor is an efficient way to insure detections on only the useful part of each image file. In addition, the noise varies somewhat over our images, and specifically the northern half of the array is on average noisier than the southern half. I compared SExtractor to FOCAS (Jarvis & Tyson, 1981; Valdes, 1982) and found that SExtractor did a better job of detecting all of the objects in the fields without introducing significant spurious detections.

## 4 Detection Efficiencies

The limiting magnitude of a survey is typically expressed as the apparent magnitude where the probability of detecting an object is equal to some percentage, commonly 50% or 90%. To obtain an accurate census of all objects to such limiting magnitudes, one can use an estimate of the incompleteness vs. apparent magnitude to correct the observed number of objects to the true value. This simple characterization of the detection efficiency is sufficient for the study of objects that all have the same size and surface brightness profile and is simplest to apply to point sources such as stars or quasars. I discuss the related issue of star–galaxy separation below in §6.

In studies of galaxies, more compact and higher surface brightness objects are generally easier to detect. Not taking the relative “visibility” (Disney, 1976; Disney & Phillipps, 1983; Phillipps et al., 1990; Davies, 1990) of different types of objects into account can lead to erroneous conclusions about the intrinsic distribution of observed galaxy properties. The detection efficiency of a survey should therefore be characterized as not just a function of apparent magnitude, but also as a function of surface brightness profile and angular size. That is, surveys should take into account the multivariate nature of the galaxy population, rather than collapsing variations in profile shape, size, and surface brightness into just a dependence on integrated brightness. Bershady et al. (1998), for example, separate objects into stars, small, and large galaxies and calculate the detection efficiency of each class separately.

The surface brightness profile of galaxies is commonly parameterized by either an exponential profile

$$\mathrm{\Sigma }(r)=\mathrm{exp}\left(1.673\frac{r}{r_h}\right)$$
(2)
or a de Vaucouleurs ($`r^{1/4}`$) profile

$$\mathrm{\Sigma }(r)=\mathrm{exp}\left(7.67\left[\left(\frac{r}{r_h}\right)^{1/4}1\right]\right),$$
(3)
where $`\mathrm{\Sigma }(r)`$ is the surface brightness at radius $`r`$ and $`r_h`$ is the scale radius for each profile that encloses half of the total light. Most galaxies can be fit by either one of these profiles or, for disk galaxies with a bulge component, a superposition of both. These parameterizations are useful for examining how the detection efficiency varies as a function of galaxy size and profile shape.

The sensitivity of each of the fields in this survey is different due to variations in the atmospheric conditions and instrument setup. The likelihood of detecting an object of a given brightness, size, and profile will vary from field to field and can not be accurately represented by a single number for the entire survey, even for stellar objects. To characterize the detection limits of this survey, I added artificial objects produced with the IRAF ARTDATA package to each of the fields in each filter and measured the fraction of recovered objects with SExtractor. In addition to stellar profiles, I added galaxies with both exponential and $`r^{1/4}`$ profiles with $`r_h`$ equal to 0.25′′, 0.5′′, 0.75′′, and 1′′ over a range of apparent magnitude. Studies of the HST Medium Deep Survey (Roche et al., 1997), the NICMOS parallel survey (Teplitz et al., 1998; Yan et al., 1998), and HST imaging of the CFRS and LDSS galaxies (Lilly et al., 1998) have shown that nearly all galaxies to the apparent magnitude limits of this survey are well fit by either exponential or $`r^{1/4}`$ profiles and most will have $`r_h<1^{\prime \prime }`$ and exponential profiles. An alternative to using artificial objects to measure the detection efficiency is to extract bright objects from the survey fields, artificially dim them, place them at random back into the original images, and attempt to recover them. However, as the median galaxy size declines at fainter apparent magnitudes, this may lead to an underestimate of the detection efficiency.

I convolved the model galaxy profiles used in this analysis with a Moffat (1969) profile as this profile is a much better fit to the observed profiles of bright stars than a Gaussian profile. A Moffat profile has the functional form

$$I(r)=\left[1+\left(r/\alpha \right)^2\right]^\beta .$$
(4)
After fitting Moffat profiles to bright stars in a number of fields and filters that spanned the range of PSF sizes, I fixed $`\beta `$ to be 2.5. The parameter $`\beta `$ determines the strength of the power-law tail of the radial intensity distribution, while $`\alpha `$ parameterizes the width of the profile peak and is similar to $`\sigma `$ in a Gaussian function. This procedure set the convolution kernel for the model galaxy profiles in ARTDATA for each image and takes the variation in PSF from image to image into account in the determination of the detection efficiency. The functional form of the Moffat profile used by the ARTDATA package is slightly different from equation 4, but MOFFAT profiles produced by ARTDATA are in reasonable agreement with this standard form. The default dynamic range of the convolution kernel in ARTDATA for galaxy profiles (ARTDATA.DYNRANG) is only ten, which causes an artificial truncation in galaxy profiles. I changed the value of this parameter to 10000, which matches the dynamic range of the PSF computation. While this is more computationally intensive, it is not prohibitive.

In Figure 1 I show the relative detection efficiency for these two profiles at a range of $`r_h`$ in a field of fixed sky brightness. The sky noise and magnitude zeropoint in this figure is from one of the $`K`$ band frames, although the relative detection efficiencies for different sizes, profiles, and seeing are independent of the actual filter used. Throughout this paper I will use $`r_h`$ to parameterize the angular half-light radius, rather than associate it with a physical scale. The top two panels of Figure 1 show an exponential disk model and the bottom two panels show a $`r^{1/4}`$ profile. The left panels correspond to a seeing FWHM of $`1^{\prime \prime }`$ and the right panels correspond to $`1.5^{\prime \prime }`$ seeing. This figure shows that the detection efficiency for stellar objects is 0.5 – 1 mag fainter than the detection efficiency for marginally resolved galaxies and that the detection efficiency is similar for exponential disks and $`r^{1/4}`$ profiles with the same $`r_h`$. As angular size and surface brightness vary with redshift, the detection efficiency for different classes of objects can be even more important for selecting galaxies for a redshift survey that will be used to compute the luminosity function (Dalcanton, 1998a; Petrosian, 1998).

Figure 2 illustrates the variation in detection efficiency from field to field of this survey as a function of object size. The histogram represents the number of square arcminutes with a given 90% completeness limit vs. $`K`$ magnitude for the 185 square arcminutes of this survey. The top panel shows the distribution for stellar profiles, while the lower panels show exponential disks with $`r_h=0.25^{\prime \prime }`$, 0.5′′, 0.75′′, and 1′′. The average, area-weighted 50% completeness limits of the survey are $`J=20.5`$, $`H=19.5`$, and $`K=18`$ mag for exponential disks with $`r_h=0.75^{\prime \prime }`$. For stellar objects these limits are $`J=21`$, $`H=20`$, and $`K=18.5`$ mag.

## 5 Photometry

The unknown intrinsic morphology of galaxies makes galaxy photometry considerably more challenging than measuring the apparent magnitude of a star. In this paper, the goal is to measure the total, integrated brightness of galaxies to compare to models of galaxy surface density as a function of integrated brightness. Most of the galaxies in this sample have small angular sizes, where small in this context corresponds to $`r_h`$ less than or equal to the seeing FWHM. Due to the finite signal-to-noise ratio and the lack of numerous bright stars in each field to exactly determine the PSF, the true morphology of the galaxies can not be accurately deconvolved. Photometric techniques for marginally-resolved galaxies include aperture magnitudes, isophotal magnitudes, metric magnitudes, and various slight modifications of these techniques.

Aperture magnitudes are the simplest to define as they just involve computing the total flux within some given size aperture. The problem with aperture magnitudes, however, is that they will always exclude some fraction of a galaxy’s light unless the aperture is made very large. While using a very large aperture is a way of avoiding systematic errors in galaxy photometry, it is impractical to apply to observations within several magnitudes of the survey limit due to nonnegligible sky noise. A pragmatic approach is therefore to measure the brightness of the galaxy within some small aperture and then to correct this aperture magnitude to compensate for the lost light, usually based on some measure of the galaxy size (Glazebrook et al., 1994; Soifer et al., 1994; Cowie et al., 1994; Bershady et al., 1998; Saracco et al., 1999). For any size aperture, the fraction of lost light will depend on both the scale size and surface brightness profile of the galaxy. These quantities are difficult to measure with great accuracy for galaxies near their detection limit.

An isophotal magnitude is measured by summing all of a galaxy’s flux out to some limiting isophote, usually the limiting isophotal magnitude of the survey. A variation of the isophotal magnitude called the “TOTAL” magnitude is more commonly used (McLeod et al., 1995; Hall et al., 1998; Minezaki et al., 1998b) and is one of the quantities calculated by FOCAS. The FOCAS TOTAL magnitude is defined as the sum of the flux within twice the isophotal area, rather than just the isophotal area. The isophotal magnitude is well-known to be biased towards underestimating the flux of all but very bright and high surface-brightness galaxies. Using the FOCAS TOTAL magnitude decreases the tendency of isophotal magnitudes to underestimate galaxy size and integrated brightness by effectively increasing the aperture size, but it is fundamentally still an isophotal magnitude and therefore underestimates the integrated light of galaxies (Dalcanton, 1998b).

Metric magnitudes appear to offer a physically motivated solution to the shortcomings of these two techniques. The most common metric magnitude is measured within the radius introduced by Petrosian (1976). The Petrosian magnitude is defined to be the flux interior to radius $`r_p`$, where the surface brightness at $`r_p`$ is equal to some fraction of the average surface brightness interior to $`r_p`$. The great attraction of using a Petrosian magnitude is that it is independent of the spacetime geometry by virtue of being the ratio of two surface brightnesses. $`r_p`$ therefore corresponds to the same physical scale in two galaxies with the same surface-brightness profile at different redshifts. The corresponding integrated brightness within that radius will correspond to the same fraction of the total light (any systematic underestimate will be identical for the two galaxies). However, the fraction of the light missed by a Petrosian magnitude for two galaxies at different redshifts will only be the same if two conditions hold: the surface brightness profiles of the galaxies are the same and neither of the galaxies are small enough that they are significantly affected by the PSF. Thus while the Petrosian magnitude is ideal for cosmological tests (Sandage & Perelmuter, 1990; Petrosian, 1998), it may not offer the ideal choice for measuring the integrated light of galaxies with a diverse range of surface brightness profiles and sizes comparable to the PSF of the observations.

The main goal of the magnitude measurements in this paper is to measure the number of galaxies per unit magnitude. These measurements, in conjunction with theoretical predictions, can then be used to constrain models of galaxy formation and evolution. The output of the theoretical models to which observed number counts are compared is the sum of galaxies in a given apparent magnitude bin. To compare observational data to these theoretical predictions therefore requires an unbiased measurement of the number of galaxies at a given integrated brightness, which in turn requires a good means of estimating the integrated brightness of individual galaxies.

I evaluated different techniques for measuring the integrated brightness of the galaxies in the sample including aperture magnitudes measured with PHOT in IRAF, TOTAL magnitudes measured with FOCAS, and “BEST” magnitudes measured by SExtractor. The SExtractor BEST magnitude is defined to be either a Kron magnitude (Kron, 1980), which is measured in an elliptical aperture whose size is determined by the object’s profile, or an isophotal magnitude in very crowded regions. To evaluate the accuracy of these photometry techniques I used ARTDATA in IRAF (as described in §4) to add artificial galaxies to the images. The metric to evaluate the accuracy of each of these techniques is the size of the difference between the input magnitude and the measured magnitude as a function of $`r_h`$ for exponential and $`r^{1/4}`$ profiles. I performed these tests for different images to sample the range of seeing in the data and at a range in apparent magnitude from the 50% completeness limit for stellar profiles to four magnitudes brighter. This magnitude range includes more than 80% of the sources in the sample. The investigation showed that IRAF PHOT aperture magnitudes, FOCAS TOTAL magnitudes, and SExtractor BEST magnitudes all underestimate the integrated brightness of galaxies. The FOCAS and SExtractor magnitudes were taken directly from the output of these two packages, while I added a stellar-profile aperture correction, based on measurements of several stellar objects in the field, to the aperture magnitudes measured by IRAF PHOT.

Figure 3 shows the result of one of these experiments for an image with 1.25′′ FWHM. The measured magnitude is either the IRAF PHOT aperture magnitude (left panels), FOCAS TOTAL magnitude (middle panels), or SExtractor BEST magnitude (right panels) and the true magnitude is the input magnitude assigned to the artificial object. As in Figure 1, the sky noise and apparent magnitude for this simulation are based on one of the $`K`$ band frames; however, the reliablility of these different photometric techniques relative to the completeness limit are independent of the filter. This figure shows the results for stellar profiles (top panels), $`r_h=0.5^{\prime \prime }`$ exponential disks (middle panels), and $`r_h=1^{\prime \prime }`$ (bottom panels). The data points are the mean offset between the measured and true magnitudes from $`200`$ artificial objects per apparent magnitude and the errorbars are the $`1\sigma `$ dispersion in these measurements at that magnitude. At a fixed input galaxy brightness more light is missed in galaxies with larger $`r_h`$, while at fixed $`r_h`$, more light is missed in an $`r^{1/4}`$ profile than in an exponential profile. For fainter apparent magnitudes, the size of the brightness underestimate increases for the FOCAS TOTAL and SExtractor BEST magnitudes at fixed $`r_h`$, but remained constant for the aperture magnitude. The reason for the systematic increase in the underestimate for the FOCAS TOTAL magnitudes is that the isophotal area of a galaxy of fixed size decreases as the integrated brightness decreases. Though the FOCAS TOTAL magnitude doubles the isophotal area, this still leads to an underestimate of the true size of the galaxy for the photometric measurement. The SExtractor BEST magnitude suffers from the same systematic increase in error as the size of the elliptical aperture is set by a fit to the observed galaxy profile. This offset is also due to the fact that SExtractor BEST magnitudes assume the PSF is Gaussian when computing the elliptical aperture for photometry and a Gaussian profile is more centrally concentrated than a MOFFAT profile (Dalcanton, 1998b). At a higher galaxy surface density than probed in this survey, the SExtractor BEST magnitudes will be isophotal magnitudes, rather than Kron magnitudes, and the brightness underestimate may be more severe. For fainter galaxies at fixed size and profile shape, a smaller fraction will be above the noise and the tendency is for a detected object to appear more compact, which will further shrink the aperture size used in the photometry. While I have not evaluated Petrosian magnitudes here, Dalcanton (1998a) included them in her study of systematic biases in measuring the integrated light of galaxies for computing the luminosity function. She found that Petrosian magnitudes underestimate the true brightness of galaxies to a lesser extent than aperture or isophotal magnitudes (see also Dalcanton, 1998b), although they too are subject to large errors for objects near the detection limit. This is because the surface brightness profile becomes increasingly noisy. In addition, as stated above, the Petrosian radius is difficult to determine for marginally resolved objects.

Saracco et al. (1999) also compare aperture magnitudes to isophotal magnitudes and SExtractor AUTO magnitudes, which are similar to SExtractor BEST magnitudes. Their figure 1 shows a difference of several tenths of a magnitude between photometric measurements with these three techniques of the galaxies in their fields. These differences increase to on order one magnitude for the last two magnitudes below their S/N = 5 detection limit for compact objects. They find that both isophotal and SExtractor AUTO magnitudes underestimate the flux of faint objects and therefore chose to use aperture magnitudes. Väisänen et al. (2000) have also evaluated the relative merits of aperture, isophotal, and SExtractor BEST magnitudes. They concluded that SExtractor BEST magnitudes do not underestimate the integrated brightness of galaxies near the detection limit, in contrast with the simulations presented here and the results of Dalcanton (1998b).

My analysis has shown that only aperture magnitudes avoid errors in the photometry that are a function of integrated brightness, and I have chosen to use them. The brightness underestimate of aperture photometry will still depend on the intrinsic size and profile shape, but avoids the additional systematic uncertainty due to the dependence on integrated brightness inherent in SExtractor BEST and FOCAS TOTAL magnitudes. Another consideration in the decision to use aperture magnitudes is the unique nature of these data. I have 21 fields observed in three filters, each with a different PSF shape. Aperture photometry is the best way to account for the heterogeneous nature of the dataset. The size of the error in the SExtractor BEST and FOCAS TOTAL magnitudes will vary from field to field with the seeing. While aperture magnitudes will suffer from the same problem, they can be relatively easily scaled to take seeing variations into account. In addition, the aperture magnitudes can be easily measured through a sufficiently large aperture to minimize the size of the galaxy aperture correction and therefore the associated uncertainty.

Given the range of galaxy sizes and typical seeing, aperture magnitudes with a radius 1.5 times the FWHM of the seeing are a good compromise between minimizing the size of the galaxy aperture correction and maximizing the galaxy signal in the aperture. At $`z=0.5`$, 1′′ corresponds to a physical size of 4.3 $`h^1`$ kpc (for $`\mathrm{\Omega }_M=0.3`$, $`\mathrm{\Omega }_\mathrm{\Lambda }=0.7`$) and therefore these typical aperture sizes of 1.5′′ – 2.5′′ include most of the galaxies’ light. The galaxy aperture correction is the difference between the true integrated brightness of the galaxy and the measured aperture magnitude plus the stellar aperture correction. The stellar aperture correction only depends on the shape of the PSF and, because the aperture is scaled to the PSF size, this correction was always measured to be $`0.1`$ mag. The galaxy aperture correction depends on the typical galaxy size and was measured with the same code developed to measure the detection efficiency for objects of different size, surface brightness profile, and integrated brightness. The distribution of size and surface brightness profile as a function of integrated brightness have not been completely determined in the NIR. However, a range of studies of galaxy morphology at visible (Smail et al., 1995; Roche et al., 1997, 1998) and NIR (Yan et al., 1998; Bershady et al., 1998; Saracco et al., 1999) wavelengths, combined with color information (Thompson et al., 1999) shows that most of the faint galaxies in this survey will have exponential profiles with typical half-light radii of $`0.5^{\prime \prime }`$$`0.75^{\prime \prime }`$. At the magnitude limits of this survey the average galaxy size is $`r_h0.6^{\prime \prime }`$ in these three filters. Any bulge component for these galaxies, though more prominent at NIR wavelengths, will fall inside the aperture. For this range of sizes of galaxies, I computed the galaxy aperture correction for each frame as a function of integrated brightness. The size of this correction is always in the range 0.1 – 0.2 mag. The correction is generally smaller for fields with poorer seeing as the aperture size is larger in these fields, although the detection limit for a given galaxy size and profile shape is lower for these fields as well.

## 6 Star/Galaxy Separation

The calculation of number counts of galaxies at faint magnitudes must deal with stellar contamination, which affects faint galaxy surveys in two ways. First, the typical galaxy size decreases at fainter apparent magnitudes and therefore galaxies are progressively more likely to be morphologically indistinguishable from stars. Second, stars (and compact galaxies) are the easiest objects to detect and accurately photometer near the survey limits. This is illustrated in Figure 1, which shows that the limiting magnitude at fixed detection efficiency is on order one magnitude brighter for exponential disks or $`r^{1/4}`$ profiles with $`r_h=1^{\prime \prime }`$ than for stars. Multicolor separation is one way to remove stellar contamination from faint galaxy samples. This process is most effective with small photometric errors, although it can be effective with larger errors with a longer wavelength baseline (e.g. visible–NIR colors, Gardner, 1995). The removal of stellar contamination from this survey is particularly important as stars contribute on order 20 – 30% to even the faintest magnitude bins of this survey. In even deeper surveys stellar contamination is less of a problem as galaxies begin to significantly outnumber stars.

The stellar content of the DMS fields has already been cataloged as part of the search for quasars (Hall et al., 1996a; Osmer et al., 1998). The selection criterion for identifying stellar objects was that they must be indistinguishable from the PSF in at least three of the six CCD filters (Hall et al., 1996a). I used the stellar catalog from Osmer et al. (1998) to identify and filter out stars from the NIR catalogs. Given that the CCD stellar catalog extends several magnitudes fainter than this survey, all but the very reddest stars should be included in the stellar catalog. In contrast, the CCD stellar catalog includes some contamination from compact galaxies near its magnitude limit. From the stellar model counts presented in Minezaki et al. (1998a) and galaxy counts in the literature, I expect stars comprise approximately 10 – 20 % of all objects at the limiting magnitude of this survey. Any missed fraction will therefore be a minor contamination to the galaxy number counts.

## 7 Number Counts

The calculation of NIR galaxy number counts in the DMS is complicated by the variation in detection efficiency from field to field. To combine all of the fields I have used a technique analogous to the one outlined in the appendix of Bershady et al. (1998). However, the same variations in seeing that contribute to the different field sensitivities make it difficult to reliably bin these objects into different size classes. I have therefore not separated the counts into different sizes (and profile types), effectively collapsing the galaxy distribution from a multivariate distribution (size, profile shape, and integrated brightness) to a function of integrated brightness alone. As most number–magnitude models for galaxies only predict surface density vs. integrated brightness (though see Im et al., 1995), this is the most useful format to compare the data with theoretical predictions.

For each apparent magnitude bin I used the typical galaxy size and galaxy aperture correction discussed in §5 to correct the measured brightness of each galaxy. All of the galaxies in each field were then placed into 0.5 mag bins in apparent magnitude. The detection efficiency for each bin is the average of the detection efficiency vs. magnitude measured in 0.1 mag increments and weighted by the expected slope of the number–magnitude relation $`\alpha 0.5`$, where $`\mathrm{log}N\alpha m`$. In practice, the detection efficiency drops off significantly faster than the number–magnitude relation increases and so the detection efficiency of a magnitude bin is relatively insensitive to $`\alpha `$. If the detection efficiency of the faint end of a bin was less than 50%, this bin was not included in the calculation of the number–magnitude relation. The detection efficiencies described in §4 multiplied by the field size yields the effective area of each field. I then summed the total differential counts and effective area of each field per 0.5 magnitude bin to compute the differential number counts per unit magnitude per square degree. The number counts for $`J`$, $`H`$, and $`K`$ are shown in Figures 4, 5, and 6, respectively. The raw and corrected counts, along with the effective area, are listed in Tables 3, 4, and 5. The raw counts are the total counts in 0.5 mag bins with only the stellar ($`0.1`$ mag) aperture correction applied to the photometry. The corrected counts are the counts per 0.5 mag bin after application of the galaxy aperture correction. The errorbars on the number counts are $`1\sigma `$ upper and lower confidence intervals calculated using the formulae in Gehrels (1986).

The $`J`$ number counts shown in Figure 4 agree well with the number counts of Teplitz et al. (1999), Väisänen et al. (2000) and the (small+large) counts of Bershady et al. (1998), though they are somewhat higher than the Saracco et al. (1999) counts. The Väisänen et al. (2000) survey overlaps the most with this survey, although they use SExtractor BEST magnitudes rather than aperture magnitudes. Their survey covers a substantially larger area than this one, though its faint limit is nearly one magnitude brighter. Bershady et al. (1998) and Saracco et al. (1999) extend much fainter than this survey and their brightest counts only overlap with the faintest magnitudes of this survey and in this regime they have approximately five to ten times fewer galaxies due to their smaller area. The Saracco et al. (1999) surface density is probably lower in this range due to small number statistics as they also use the Persson et al. (1998) standards, aperture magnitudes, and the agreement appears to improve at fainter magnitudes where they have a larger sample. All of the $`J`$ counts in the range 16 – 20.5 mag have essentially the same slope as these data, $`\alpha 0.54`$.

The slope of the $`H`$ band counts in Figure 5 over the 16.5 – 19 mag range is $`\alpha =0.47`$. These counts overlap with the brightest magnitude bin from Yan et al. (1998), but the surface density vs. magnitude relation shows an apparent “break” at $`H=19`$ – 20 mag. The Yan et al. (1998) measurements are based on NICMOS parallel data with Camera 3 over a total area of 8.7 arcmin2 and therefore they have relatively few galaxies in the region of overlap. However, the additional NICMOS parallel data studied by Teplitz et al. (1998), which includes images from Camera 2 and extends several magnitudes brighter, also shows an apparent break at $`H=19`$ – 20 mag. Teplitz et al. (1998) transformed several deep $`K`$ galaxy counts studies to the NICMOS F160W filter and these transformed $`K`$ counts appear to match the break in the $`H`$ galaxy number–magnitude relation. This break may therefore be the change in slope seen in $`K`$ band counts at $`K20`$ mag.

Most published NIR number counts to date have been obtained in the $`K`$ band (Gardner et al., 1993; Cowie et al., 1994; Glazebrook et al., 1994; Soifer et al., 1994; Djorgovski et al., 1995; McLeod et al., 1995; Moustakas et al., 1997; Minezaki et al., 1998b; Szokoly et al., 1998; Bershady et al., 1998; Saracco et al., 1999; Väisänen et al., 2000; McCracken et al., 2000). Gardner et al. (1996) and Väisänen et al. (2000) have the largest magnitude range of overlap with the DMS counts shown in Figure 6 and the number–magitude relations from these studies are in good agreement. The slope of the $`K`$ counts from 14 – 18 mag is $`\alpha =0.54`$. At fainter magnitudes there is a great deal of dispersion in the surface density measurements, though given the small areas of these surveys they are generally consistent with one another. As noted by several authors (e.g. Bershady et al., 1998; Saracco et al., 1999), this dispersion may also be due to different measurement techniques, variations in magnitude systems, and filters, in addition to small number statistics and cosmic variance. The most significant differences are likely due to the method of magnitude measurement, completeness corrections, and at least at bright magnitudes, star–galaxy separation. Of the faint counts presented in Figure 6, Saracco et al. (1999) and Moustakas et al. (1997) apply a uniform galaxy aperture correction for all objects, Bershady et al. (1998) compute completeness and a galaxy aperture correction as a function of object size, Minezaki et al. (1998b) use FOCAS TOTAL magnitudes, and Djorgovski et al. (1995) only use a stellar aperture correction.

I computed the four models shown in Figures 46 with ncmod, a publically available code for computing galaxy number counts and colors by Gardner (1998). The models correspond to no evolution (solid lines), passive evolution (dotted lines), merging plus passive evolution (short-dashed lines), and dust plus passive evolution (long-dashed lines). For all of the models I have assumed $`\mathrm{\Omega }_M=0.3`$ and $`\mathrm{\Omega }_\mathrm{\Lambda }=0.7`$, in agreement with results on Type Ia supernovae (Riess et al., 1998; Perlmutter et al., 1999), the Lyman-$`\alpha `$ forest, COBE-DMR (Phillips et al., 2000), as well as number counts in the Hubble Deep Field (Totani & Yoshii, 2000). I chose to fix the cosmological parameters as the uncertainties in $`\mathrm{\Omega }_M`$ and $`\mathrm{\Omega }_\mathrm{\Lambda }`$ are less than the uncertainties in models of galaxy evolution.

All of these models are based on the $`K`$ luminosity function and mix of galaxy types from Gardner et al. (1997) and galaxy spectral energy distributions from GISSEL96 (Bruzual & Charlot, 1993). To compute these models I also used the filter transmission curves for the TIFKAM filters plus a measurement of the atmospheric transmission. The no evolution model is a pure $`k`$-correction model. In the passive evolution model, all galaxies form at $`z=15`$ except for the latest galaxy type, which have constant star formation and are always 1 Gyr old. The merging model has the same galaxy evolution as the passive evolution mode, but it also includes number evolution in the form $`\varphi ^{}(1+z)^\eta `$ with $`\eta =1.5`$ and conservation of luminosity density. This model is based on the parameterization of Rocca-Volmerange & Guiderdoni (1990) and it produces similar results to the Broadhurst et al. (1992) merging model. This value of $`\eta `$ is higher than the constraint $`\eta 1`$ derived by Totani & Yoshii (2000) in their $`\mathrm{\Lambda }`$ model for the HDF, but is illustrative of the effect of adding merging to passive evolution. Finally, the dust model also contains the same galaxy evolution parameterization as the passive evolution model, but with the addition of dust in the plane of the galaxies. This model, based on the work of Bruzual et al. (1988) and Wang (1991) was introduced to number count models to correct for their tendency to overproduce the $`UV`$ flux in galaxies, though it has a minimal effect on the NIR number–magnitude relation.

The $`J`$, $`H`$, and $`K`$ observations presented here all agree well with the passive evolution model. They are less consistent with the number density evolution we assumed in the merging model, and clearly have a higher surface density of objects than the no evolution prediction. The observed $`J`$ counts are all higher than the models, but this may be due to the extrapolation of the $`K`$ galaxy luminosity function to the $`J`$ band. As noted above, the HST counts for $`H>20`$ mag (Teplitz et al., 1998; Yan et al., 1998) have systematically lower surface density. This difference does not appear to be due to differences in the transmission profiles of the two filters. When I calculated the model galaxy counts with the F160W transmission profile, rather than the $`H`$ filter, the F160W galaxy count models were offset by less than 0.1 mag fainter than the ground-based $`H`$ model counts, whereas an offset of 0.3 – 0.4 mag is needed to explain the difference in surface density.

## 8 Mean Colors

Color as a function of apparent magnitude can provide an additional constraint on galaxy evolution models, particularly when severals colors are available. I have used the Gardner (1998) model to compute the median $`JH`$ color vs. $`J`$ magnitude and the median $`JK`$ color vs. $`K`$ magnitude for the four models discussed above. Figure 7 shows these two color–magnitude relations for all of the galaxies detected in these filters. For the magnitude range of this survey, all of these models predict similar median colors and they agree well with the data and $`JK`$ colors reported by Bershady et al. (1998) and Saracco et al. (1999) in the region of overlap.

To these apparent magnitude limits, NIR colors do not provide a strong constraint on galaxy evolution as most galaxies are low redshift ($`z<1`$) objects. In this redshift range, the $`JHK`$ filters all sample the old stellar population, which has a relatively flat spectral energy distribution independent of galaxy type. The overlap of the four models and the lack of change in the median color with apparent magnitude reflects the similarity of these galaxies at NIR wavelengths. Bershady et al. (1998) and Saracco et al. (1999) also report $`JK`$ colors vs. $`K`$ and these results agree well with their colors.

## 9 Summary

I have presented NIR $`JHK`$ galaxy counts and colors and described in detail the detection efficiencies and selection effects that affect this survey. I have studied several popular methods for measuring the integrated brightness of galaxies as a function of apparent magnitude and found that only aperture magnitudes miss a consistent fraction of a galaxy’s light as a function of apparent magnitude for a range of galaxy sizes and profile shapes. This survey is the largest area observed to these depths in all three NIR filters that has been published to date. The $`J`$, $`H`$, and $`K`$ number–magnitude relations and colors are in good agreement with a simple model of passive galaxy evolution with at most a small amount ($`\eta 1.5`$) of merging in an $`\mathrm{\Omega }_M=0.3`$, $`\mathrm{\Omega }_\mathrm{\Lambda }=0.7`$ cosmology.

Current model predictions for the number–magnitude relation at NIR wavelengths are very similar to one another and larger surveys at the faintest magnitudes are needed to discriminate between different models with this diagnostic tool. The models shown in Figures 46 suggest that number counts to $`H24`$ mag or $`K23`$ mag are necessary to distinguish between the passive evolution and merging model. High angular resolution imaging to measure the mix of galaxy sizes and morphological types as a function of brightness, as well as color information, could prove to be a more efficient means to discriminate between galaxy formation and evolution models with deep, NIR imaging surveys.

I would like to thank Darren DePoy, Patrick Osmer, and David Weinberg for helpful discussions and suggestions as well as the referee, Jonathan Gardner, for helpful comments that have improved this presentation. I am grateful to Christopher J. Burke and Adam Steed for assistance with some of the observations, and to Mike Lannon for supplying the TIFKAM filter curves in digital form. I would also like to thank the MDM staff for their support and the OSU TAC for a generous allocation of telescope time. I was supported in part by a Presidential Fellowship from Ohio State University and received additional travel and other support from a PEGS grant and the Department of Astronomy at Ohio State University. TIFKAM was funded by the Ohio State University, the MDM consortium, MIT, and NSF grant AST-9605012. NOAO and USNO paid for the development of the ALADDIN arrays and contributed the array currently in use in TIFKAM.