text:
# Galaxy-dark matter correlations applied to galaxy-galaxy lensing: predictions from the semi-analytic galaxy formation models

## 1 Introduction

One of the main goals of modern cosmology is to understand the processes of structure and galaxy formation in the universe. It is widely believed that large scale structure emerged from small matter density fluctuations through the gravitational instability process. Once the fluctuations became large the process of gravitational collapse led to the formation of first dark matter haloes, which subsequently merged to form larger and larger haloes. Gas initially followed the dark matter until it reached sufficiently high densities that it was able to cool efficiently and condense at the centres of the haloes. Subsequent star formation from the cold gas lead to the creation of galaxies. While the broad picture of this scenario is widely accepted its details are still poorly understood.

Observational constraints on the processes of structure and galaxy formation can be divided into three categories. In the first are observations that depend only on the distribution of the dark matter. Examples of these are velocity flows (e.g. Strauss & Willick 1995), weak lensing (e.g. Mellier 1999) or X-ray temperature function (e.g. Blanchard et al. 2000). Another example at high redshift are cosmic microwave background anisotropies (e.g. Hu et al. 2000). These direct probes of dark matter are our best hope to determine the cosmological parameters and the clustering of dark matter. In the second category are observations that only trace galaxies. Examples are galaxy luminosity function, colours, morphologies, clustering etc. These observations also depend on the underlying dark matter distribution to some extent, but the relation is often a complicated one and difficult to interpret in the absence of dynamical information. In the third category are observations that relate the properties of the galaxies to those of the dark matter. An example is the Tully-Fisher relation (Tully & Fisher 1977), which relates the luminosity of the spiral galaxies to their maximal rotational velocity, which is related to the mass of the halo in which the galaxies reside. Observations in this category are specially important in relating the process of galaxy formation to the process of structure formation. Galaxy-dark matter correlations, which can be measured through the weak lensing effects such as galaxy-galaxy lensing, galaxy foreground-background correlations or galaxy-quasar correlations, offer another example that fall into this category. The goal of this paper is to analyze these in the context of the galaxy formation models and make predictions for such observations.

Physics of galaxy formation remains a complicated and poorly understood subject and various approaches have been developed to address it. Two most commonly used approaches are hydrodynamic simulations (Blanton et al. 1999; Pearce et al. 1999) and N-body simulations coupled to the semi-analytical models (SAMs; Benson et al. 1999; Kauffmann et al. 1999; Somerville et al. 1998). First approach has the advantage of better modelling the physics, but is currently limited by the resolution. Second approach is less ab-initio and therefore may miss some important physics, but currently has a larger dynamic range. In this paper we focus on the dark matter and galaxy distribution which is quantified in terms of the cross-correlation function and the respective cross-power spectrum. Since we are particularly interested in the galactic scales which are only poorly resolved with hydrodynamic simulations we will use SAMs coupled to the N-body simulations. These are still limited in resolution and so we compare the results with analytic modelling (Seljak 2000), which under certain reasonable assumptions allows one to extend the resolution limit of the current N-body simulations.

Theoretical predictions on how galaxies trace mass, how is galaxy luminosity related to the surrounding dark matter and how can we extract these relations from the observations are essential if we are to take advantage of the large amount of data available in the near future from surveys such as Sloan Digital Sky Survey (SDSS) or 2dF. SDSS has already obtained the first detection of the galaxy-galaxy lensing with only a tiny fraction of the total data (Fischer et al. 2000). This has been based on a rough separation between the foreground and the background galaxy sample. In the future this will be done with a better precision by using the photometric redshift information, which will significantly increase the signal to noise. Photometric redshifts will also allow one to combine the signal as a function of the physical separation rather than the angular one. Larger data set will allow one to split the galaxies as a function of luminosity, colour or morphology. As we show in this paper these give quantitatively different predictions for galaxy-galaxy lensing, as well as for galaxy-quasar correlations and foreground-background galaxy correlations. This would allow one to extract the information on the environment of different types of galaxies.

The outline of the paper is as follows. In Section 2 we present the formalism, simulations and computational methods. In Section 3 we present results in terms of the cross-power spectra and respective cross-correlation functions for different galaxy samples. We also show their evolution with redshift. In Section 3.3 predictions for galaxy-galaxy lensing are given and compared with some of the existing measurements of the effect (Fischer et al. 2000). In Section 4 we consider the cross-correlation coefficient dependence on scale and galaxy type and in Section 5 we address the prospects of interpreting the cross-correlations in terms of the dark matter halo profiles. Conclusions and prospects for future work are presented in Section 6.

## 2 Formalism and Simulations

### 2.1 Cross-correlation analysis

We are interested in the distribution of the dark matter around the galaxies of a selected type, averaged over all the galaxies in the sample. We can quantify this as an excess of the dark matter density above the average as a function of the radial separation $`r`$. This is described by the galaxy-dark matter cross-correlation function,

$$\xi _{\mathrm{g},\mathrm{dm}}(r)=\delta _\mathrm{g}(\stackrel{}{x})\delta _{\mathrm{dm}}^{}(\stackrel{}{x}+\stackrel{}{r}).$$
(1)
Here $`\delta _\mathrm{g}`$ and $`\delta _{\mathrm{dm}}`$ are the overdensities of galaxies and dark matter, respectively, and we used the continuous description of a random field (see Peebles 1980 and Bertschinger 1992 for a discussion of the relation between the continuous and discrete realization and interpretation of $`\xi `$ in terms of the excess probability). We can define similarly the galaxy correlation function $`\xi _{\mathrm{g},\mathrm{g}}`$ and dark matter correlation function $`\xi _{\mathrm{dm},\mathrm{dm}}`$ by replacing one $`\delta _\mathrm{g}`$ or $`\delta _{\mathrm{dm}}`$ in the above equation.

A quantity we extensively use throughout the paper is the power spectrum of the random process, which is defined as a Fourier transform of the correlation function. In the case of the cross-power spectrum we have

$$P_{\mathrm{g},\mathrm{dm}}(k)=\frac{1}{(2\pi )^3}\xi _{\mathrm{g},\mathrm{dm}}(r)e^{i\stackrel{}{k}\stackrel{}{r}}d^3r,$$
(2)
where the power spectrum depends only on the module of $`\stackrel{}{k}`$ because of isotropy,

$$P_{\mathrm{g},\mathrm{dm}}(k)\delta (\stackrel{}{k}\stackrel{}{k^{}})=\stackrel{~}{\delta }_\mathrm{g}(\stackrel{}{k})\stackrel{~}{\delta }_{\mathrm{dm}}^{}(\stackrel{}{k^{}}),$$
(3)
where $`\stackrel{~}{\delta }(\stackrel{}{k})`$ is the Fourier transform of $`\delta (\stackrel{}{x})`$. Often the dark matter power spectrum is written in terms of the power contributing to the variance of the density field per logarithmic interval in Fourier space

$$\sigma ^2(k)_{\mathrm{g},\mathrm{dm}}=4\pi k^3P_{\mathrm{g},\mathrm{dm}}(k).$$
(4)
### 2.2 Cosmological simulations and power spectrum extraction

In our work we use GIF high resolution N-body simulations carried out by the Virgo collaboration (Jenkins et al. 1998). Our analysis is constrained to the currently popular $`\mathrm{\Lambda }`$CDM cosmological model with matter density $`\mathrm{\Omega }_m=0.3`$, cosmological constant $`\mathrm{\Omega }_\mathrm{\Lambda }=0.7`$ and the Hubble parameter $`H_0=70\mathrm{km}\mathrm{s}^1\mathrm{Mpc}^1`$. The N-body simulations have $`256^3`$ particles, each of mass $`1.410^{10}h^1M_{}`$ in a comoving box of size $`L=141h^1\mathrm{Mpc}`$. Variance of mass fluctuations on a scale of $`8h^1\mathrm{Mpc}`$ was $`\sigma _8=0.9`$, in agreement with the observed abundance of clusters (Eke, Cole & Frank 1996). N-body simulations are accurate on scales above the gravitational softening scale $`r_s=30h^1\mathrm{kpc}`$ and dark matter clumps with more than 10 particles were identified as haloes. For the most of the present work we apply the absolute magnitude cut to the galaxy selection which eliminates the very small haloes from the sample.

For the distribution of galaxies and their physical properties we use mock catalogues from the semi-analytic model of galaxy formation (Kauffmann et al. 1999a,b). These contain information on positions of galaxies, their absolute and apparent magnitude in several bands, star formation rate and mass contained in stars or gas. They also include the information on the halo in which they are placed, including its mass and virial radius. Empirically motivated dust correction has been applied to model the extinction for the $`z=0`$ sample. As shown in Kauffmann et al. 1999a,b (see also Benson et al. 1999, 2000, Somerville et al. 1998 and van den Bosch 1999) these models have been successful in reproducing a number of the observational constraints, including galaxy luminosity function, Tully-Fisher relation, pairwise velocity dispersion and galaxy correlation function.

Despite this success of the SAMs it is clear that parameterizing the complex process of galaxy formation with a few parameters is an oversimplification. It is likely that as the new data become available new parameters will need to be introduced to account for this complexity. Moreover, at present the number of predictions is small and one would like to have more independent predictions that can be verified with the new observations. Galaxy-dark matter correlations as measured through the galaxy-galaxy lensing and magnification bias provide an example of this kind. These have not been explored so far with SAMs or hydrodynamic simulations and could provide important constraints for such models.

N-body simulations together with the galaxy catalogues allow us to extract the galaxy-dark matter cross-spectra. To increase the dynamic range of the power spectrum we use the technique proposed by Jenkins et al. 1998. Instead of taking the Fourier transform of the particle distribution in a whole simulation box we divide it into $`m^3`$ cubic boxes where $`m=2^n,n=0,1,\mathrm{},6`$. We interpolate all the particles onto the small box and perform a $`128^3`$ Fourier transform. We verified that increasing the grid does not affect the resulting power spectra. This method recovers exactly all the modes periodic on scale $`L/m`$. Assuming these modes are statistically representative of all the modes it allows one to exploit the entire dynamic range of the simulation. To perform the FFT on a grid in the real space we use the Nearest-Grid-Point (NGP) mass assignment scheme (Hockney & Eastwood 1981), which involves the least amount of smoothing. Mass assignment however still suppresses the power and we divide each Fourier mode by a window function suitable for the NGP mass assignment (Hockney & Eastwood 1981). The averaging for a given amplitude of $`k`$ is taken over spherical shells in the Fourier space. For each $`n`$ the longest modes are sparsely sampled and it is better to use the larger box (with $`n1`$) to extract those. For these reasons we only use dynamic range of 2 from one box to the next. All of the spectra are corrected for the shot noise term which is due to the discrete nature of galaxies and dark matter particles. This term is negligible for the cross-power spectrum and dark matter spectrum because of the large number of dark matter particles, except for the galaxy samples with very few galaxies in the case of cross-spectrum. However, for galaxy power spectra shot noise dominates on small scales with the amplitude inversely proportional to the number of galaxies in the sample. For rare galaxy samples such as the very bright galaxies shot noise dominates already on scales of order $`1h^1\mathrm{Mpc}`$. This limits the accuracy of the cross-correlation coefficient extraction in such samples.

### 2.3 Shear and magnification derivation

Gravitational lensing of distant objects by intervening mass distribution between the source and the observer distorts the size and shape of sources as seen in the image plane. In the regime of small image distortions (weak lensing regime) observationally relevant quantities are described by the convergence $`\kappa `$ and the two-component shear $`\stackrel{}{\gamma }=(\gamma _1,\gamma _2)`$, which are related to the projected gravitational potential of the lens $`\psi (\theta )`$ via the following relations $`2\kappa (\theta )=_1_1\psi (\theta )+_2_2\psi (\theta )`$, $`2\gamma _1(\theta )=_1_1\psi (\theta )_2_2\psi (\theta )`$, $`\gamma _2(\theta )=_1_2\psi (\theta )`$, where $`\theta `$ is a two-dimensional position in the lens plane. The convergence $`\kappa `$ and may be regarded as a projected mass density along the line of sight (ignoring photon displacement effects, see Jain, Seljak and White 1999 for justification of this assumption),

$$\kappa =\frac{3}{2}\left(\frac{H_0}{c}\right)^2\mathrm{\Omega }_\mathrm{m}_0^{\chi _s}g(\chi ^{},\chi _s)\frac{\delta }{a}𝑑\chi ^{}.$$
(5)
where $`g(\chi ^{},\chi )=r(\chi ^{})r(\chi \chi ^{})/r(\chi )`$ is the radial window function, $`\chi `$ is the radial distance with $`\chi _s`$ is the distance to the source galaxy, $`r(\chi )`$ is the angular comoving distance (equal to $`\chi `$ in a flat universe discussed here), $`H_0`$ is the Hubble constant, $`\mathrm{\Omega }_\mathrm{m}`$ the matter density and $`a=(1+z)^1`$ the expansion factor. If most of the cross-correlation signal is associated with the galaxy then we can define the comoving surface matter density $`\mathrm{\Sigma }=\rho 𝑑\chi `$, in terms of which $`\kappa =\mathrm{\Sigma }/\mathrm{\Sigma }_{\mathrm{crit}}`$. Here $`\mathrm{\Sigma }_{\mathrm{crit}}^1(\chi _l,\chi _s)=4\pi Gr(\chi _l)r(\chi _s\chi _l)/ar(\chi _s)`$ is critical surface density, where $`\chi _l`$ is the comoving radial distances to the lens.

The quantity we are most interested in is tangential shear $`\gamma _t(\theta )`$ which describes elongation of images perpendicularly to the line connecting the image and the lens. It may be expressed as the 2-d shear rotated to the frame defined by the image and the lens, $`\gamma _t(\theta )=\gamma _1(\theta )\mathrm{cos}2\beta +\gamma _2(\theta )\mathrm{sin}2\beta `$, where $`\beta `$ is the relative position angle between the image and the lens. In the weak lensing regime shear is directly measurable from the image ellipticities. The relation between the mean convergence $`\overline{\kappa }(\theta )`$ inside a given circular aperture of radius $`\theta `$ and the mean tangential shear along the aperture boundary $`\gamma _t(\theta )`$ is given by (Kaiser 1993, Squires & Kaiser 1996, Miralda-Escudé 1996)

$$\gamma _t(\theta )=\frac{1}{2}\frac{d\overline{\kappa }(\theta )}{d\mathrm{ln}\theta }.$$
(6)
Averaged projected matter density can be expressed in terms of the galaxy-dark matter cross-correlation function. Here we present both the case where the redshifts of lens galaxy and background distorted galaxy are known, as well as the case when only their distributions are. From the definition of the correlation function we have that mean number density $`n(r)`$ of dark matter particles at a distance $`r`$ from a chosen galaxy is proportional to the product of the mean number density $`\overline{n}`$ if they were randomly distributed in space and respective cross-correlation function. This relation may be written as

$$n(r)=\overline{n}[1+\xi _{\mathrm{g},\mathrm{dm}}(r)].$$
(7)
Thus at an angular separation $`\theta `$ from a galaxy of a given type the projected matter density is on average given by

$$\mathrm{\Sigma }(\theta )=\overline{\rho }\left(1+\xi _{\mathrm{g},\mathrm{dm}}(r)\right)𝑑\chi ,$$
(8)
where the integration is taken along the line of sight with $`\chi `$ the radial distance and $`\overline{\rho }`$ the average matter density. Keeping $`\theta `$ constant requires $`r^2=\chi _l^2+\chi ^22\chi _l\chi \mathrm{cos}\theta `$. We can then use $`\kappa =\mathrm{\Sigma }/\mathrm{\Sigma }_{\mathrm{crit}}`$ to determine $`\overline{\kappa }`$ and $`\gamma _t`$. This relation does not include the change in the focusing strength along the line of sight. A more general expression is

$$\kappa =_0^{\chi _s}\frac{\overline{\rho }}{\mathrm{\Sigma }_{\mathrm{crit}}(\chi ,\chi _s)}\left(1+\xi _{\mathrm{g},\mathrm{dm}}(r)\right)𝑑\chi .$$
(9)
Except for the largest angles the dominant contribution is coming from the scales closest to the lens galaxy, in which case the approximation in equation 8 is a valid one. This simplifies the analysis, since for each pair of foreground and background galaxies we only need to compute the corresponding $`\mathrm{\Sigma }_{\mathrm{crit}}`$.

When the source and lens redshifts are known (or at least can be estimated with the photometric redshift techniques) the optimal way to combine the data requires to calculate for each pair of lens and source galaxies the corresponding critical density $`\mathrm{\Sigma }_{\mathrm{crit}}(\chi _l,\chi _s)`$ and to average over the product $`\mathrm{\Sigma }_{\mathrm{crit}}\gamma _t`$. This can then be related directly to the integral of the correlation function in equation 8.

Often the redshifts of the sources and lenses are unknown and the signal has to be averaged over the redshift distribution of both the source and the lens galaxies. This is necessary for example when the data sample is split into the bright and faint sample (Brainerd et al. 1996, Griffiths et al. 1996, Fischer et al. 2000). The relation in this case is (Moessner & Jain 1998)

$`\kappa (\theta )`$ $`=`$ $`6\pi ^2\left({\displaystyle \frac{H_0}{c}}\right)^2\mathrm{\Omega }_m{\displaystyle _0^{\chi _0}}𝑑\chi W_1(\chi ){\displaystyle \frac{f(\chi )}{a(\chi )}}`$ (10)
$`\times `$ $`{\displaystyle 𝑑kkP_{\mathrm{g},\mathrm{dm}}(k,\chi )J_0[kr(\chi )\theta ]},`$
where $`W_1(\chi )`$ is the normalized radial distribution of foreground galaxies of a given type and $`f(\chi )=_\chi ^{\chi _0}g(\chi ,\chi ^{})W_2(\chi ^{})𝑑\chi ^{}`$ is the average of $`g(\chi ,\chi ^{})`$ over the radial distribution of background galaxies $`W_2(\chi ^{})`$.

Having $`\kappa (\theta )`$ associated with foreground galaxies of a given type we perform an average in a circular aperture

$`\overline{\kappa }(\theta )`$ $`=`$ $`6\pi ^2\left({\displaystyle \frac{H_0}{c}}\right)^2\mathrm{\Omega }_m{\displaystyle _0^{\chi _0}}𝑑\chi W_1(\chi ){\displaystyle \frac{f(\chi )}{a(\chi )}}`$ (11)
$`\times `$ $`{\displaystyle 𝑑kkP_{\mathrm{g},\mathrm{dm}}(k,\chi )\frac{2J_1[kr(\chi )\theta ]}{kr(\chi )\theta }},`$
thus we receive the mean tangential shear as a function of lens-image separation

$`\gamma _t(\theta )`$ $`=`$ $`6\pi ^2\left({\displaystyle \frac{H_0}{c}}\right)^2\mathrm{\Omega }_m{\displaystyle _0^{\chi _0}}𝑑\chi W_1(\chi ){\displaystyle \frac{f(\chi )}{a(\chi )}}`$ (12)
$`\times `$ $`{\displaystyle 𝑑kkP_{\mathrm{g},\mathrm{dm}}(k,\chi )J_2[kr(\chi )\theta ]}.`$
## 3 Galaxy-dark matter correlations

### 3.1 Correlations at z=0

In this section we investigate the cross-power spectra and the cross-correlation functions for several galaxy samples. We choose galaxies of different absolute luminosity in B-band from $`\mathrm{M}_B=18`$ at the completeness limit of the sample to the very rare bright galaxies with $`\mathrm{M}_B<21`$. We also select red galaxies with $`\mathrm{M}_B\mathrm{M}_V>0.8`$ and blue galaxies with $`\mathrm{M}_B\mathrm{M}_V<0.8`$ in addition to the absolute magnitude cut.

Examples of several power spectra for redshift $`z=0`$ are shown in Fig. 1. In Fig. 1a we show the power spectra for galaxies brighter than $`\mathrm{M}_B<18`$, for which the sample is complete in the sense that most galaxies brighter than this limit form in haloes more massive than $`1.410^{11}h^1\mathrm{M}_{}`$, which is the smallest mass halo that can form in our simulations. The galaxy power spectrum is a power law up to a large $`k`$, while the dark matter power spectrum is continuously changing slope and becoming less steep. The galaxies are unbiased on large scales where the two spectra agree. The cross-spectrum has a similar amplitude to the galaxy and dark matter spectrum on large scales, as expected if the galaxies are an unbiased tracer of the dark matter. Cross-spectrum grows above both galaxy and dark matter spectrum on smaller scales. This can be understood with a model where for a given halo mass range there are more individual galaxies (which contribute to the cross-spectrum) than galaxy pairs (which contribute to the galaxy spectrum, see Seljak 2000). In addition, in a smaller halo with typically just one galaxy in it the galaxy is at the halo centre with dark matter particles distributed as the halo profile. In contrast, for the dark matter particles the number of pairs at a given separation is suppressed because not all dark matter particles are at the centre. This suppresses the auto power spectrum of the dark matter relative to the cross-spectrum.

In Fig. 1a the transition between the large and small scales in the cross-spectrum is continuous for the sample selected only by absolute magnitude. Figure 1b shows the same for the star-forming galaxies. These are mostly in the field and show a more prominent transition in the cross-spectrum between the correlations between haloes for $`k<3h\mathrm{Mpc}^1`$ and the profile of the haloes themselves at $`k>3h\mathrm{Mpc}^1`$. For galaxies selected only by magnitude groups and clusters also contribute to correlations, which fills in the transition region between the two (see also fig. 6 in Seljak 2000), while for star forming galaxies which are predominantly in the field groups and clusters do not contribute.

In Fig. 1c,d red and blue samples of the galaxies are shown. Red galaxies are quite biased on large scales, indicating that they form predominantly in haloes more massive than the nonlinear mass scale (i.e. groups and clusters). On smaller scales the cross-correlation drops off more rapidly than the corresponding spectra in Fig. 1a and b. This is because red galaxies in this model typically do not reside at the halo centres and thus the power spectrum suffers similar suppression as the dark matter. In contrast, blue galaxies in Fig. 1d are unbiased on large scales. The galaxy power spectrum is flat on small scales indicating there are very few haloes with more than one blue galaxy. Cross-spectrum shows similar features on large scales, but begins to rise on small scales where the galaxies that are at the halo centres become dominant and the cross-spectrum reflects their own dark matter halo profiles. Overall, the blue galaxies show similar features to the star forming galaxies, although the latter show more prominent transition between intra and inter halo correlations.

Fig. 2 shows the cross-spectra and the cross-correlation functions for galaxies in narrow magnitude bands in addition to the galaxies in Fig. 1. On large scales the spectra of narrow-band magnitude galaxies agree, indicating that the galaxies on average reside in unbiased haloes around or somewhat below the nonlinear mass. Significant fraction of them reside in groups and clusters, which explains the smooth transition between the large and the small scales. On small scales however there are significant differences in the amplitude of correlations, which increases with the brightness of the sample. This is similar to the Tully-Fisher relation, where the luminosity of spiral galaxies is correlated with the maximal circular velocity of the disc, which is correlated with the circular velocity of the halo in the SAMs used here. The differences between the models are significant already at $`100h^1\mathrm{kpc}`$, where the complications because of the baryonic effects on the dark matter distribution are negligible. This shows the potential of galaxy-galaxy lensing, which can provide independent information on the relation between the galaxy luminosity and its dark matter environment and can probe regions where baryon effects are negligible. Because of the weakness of the signal this can only be done statistically for a large sample of the galaxies and one must understand the complications caused by effects such as the contribution from groups and clusters, number of galaxies inside the halo as a function of halo mass, central versus satellite galaxies inside the halo etc. These are discussed further below.

### 3.2 Redshift evolution of cross-correlation

From the high z catalogs of the galaxy and dark matter positions we extract the evolution of the galaxy-dark matter correlation function for galaxies brighter than $`\mathrm{M}_B<19.5`$ and brighter than $`\mathrm{M}_B<21`$. No dust correction has been applied to these data so we focus only on the amplitude of the correlation function at different scales as a function of redshift. Figure 3 shows the redshift evolution of the amplitude of the correlation function at $`2h^1\mathrm{Mpc}`$, $`3h^1\mathrm{Mpc}`$ and $`8h^1\mathrm{Mpc}`$ for all three spectra. The amplitude of the dark matter correlation function decreases with redshift because of gravitational instability, which causes density perturbations to grow in time. On the other hand, the amplitude of the galaxy correlation function first decreases for the same reason, but then increases again because at higher redshifts bright galaxies are only found in the rare massive haloes, which are biased with respect to the dark matter (Kauffmann et al. 1999b). For brighter galaxies (right panels) the biasing at higher redshift is more important and leads to a larger amplitude than for fainter galaxies. Amplitude of the cross-correlation function falls in-between the amplitudes of dark matter and galaxy correlation function. From the model with correlation coefficient of unity discussed below one predicts it to be the geometric mean, which is a good description even for the galaxies that are highly biased, since we are looking at scales above $`1h^1\mathrm{Mpc}`$, where correlation coefficient is close to unity (see next section). The cross-correlation amplitude typically does not increase at higher z, but becomes flat.

### 3.3 Galaxy-galaxy lensing predictions

In this subsection we present predictions of galaxy-galaxy lensing effect for samples of galaxies discussed in previous section. For simplicity we model source and lens galaxies as being at fixed redshifts $`z_l=0.16`$ and $`z_s=0.32`$, so that the critical surface density corresponds to that of SDSS sample (Fischer et al. 2000). We use $`z=0`$ dust corrected sample to extract the galaxy-galaxy lensing signal, but we verified that for a shallow surveys such as the SDSS this assumption does not significantly affect the results. Results presented in Fig. 4 give the mean tangential shear as a function of the observed angular separation from the lens. For the considered cosmological model $`10\mathrm{arcsec}`$ corresponds to a physical distance in the lens plane of $`22.4h^1\mathrm{kpc}`$. At the redshift $`z=0.16`$ galaxy of absolute luminosity $`\mathrm{M}_B=20`$ corresponds to an apparent magnitude $`\mathrm{m}_B=19.4`$.

Plots in the upper panel of Fig. 4 are for the same luminosity intervals as in Fig. 2. For the bright galaxies tangential shear is larger than for the faint ones as predicted from the results in Fig. 2. Note also that the slope increases with luminosity. Fitting the spectra to a power law gives the slopes varying from $`0.26`$ at the faint end to $`0.54`$ at the bright end, but in general single power law provides a poor fit to the spectra (the slope steepens at larger angles). The slope is considerably shallower than the singular isothermal sphere (SIS) distribution with the slope $`1`$, predicting that with better data one should be able to see the deviations from SIS. To compare it to the Tully-Fisher relation one can compare the predictions for $`\gamma _t`$ at a given angle, which we choose to be $`\theta =100\mathrm{arcsec}`$ to maximize the signal to noise in SDSS. This corresponds to $`220h^1\mathrm{kpc}`$ transverse distance from the galaxy. Assuming $`\gamma _tv_c^2`$ where $`v_c`$ is the circular velocity and fitting this to the relation $`Lv_c^\alpha `$ one finds $`\alpha =3.5`$, which is in a good agreement with the Tully-Fisher relation. This agreement is however partially fortuitous, because the slope of $`\gamma _t`$ is not well fit with the SIS model and so the $`Lv_c`$ relation must change with the angle at which the comparison is made.

In the lower part of Fig. 4 we show the results for $`\mathrm{M}_B<18`$, $`\mathrm{M}_B<19`$, star forming galaxies, red and blue galaxies. Since the luminosity function peaks at $`\mathrm{M}_B=19`$ galaxies brighter than $`\mathrm{M}_B<18`$ to $`\mathrm{M}_B<19`$ should be compared to the SDSS apparent magnitude selected sample. It is encouraging that the predictions are in a good agreement with the observed signal. We should caution that this is not meant to be a quantitative analysis of SDSS results, because we do not simulate SDSS colour bands, selection criteria and galaxy redshift distribution, all of which depend also on the luminosity function of the sample and not just on the relation between galaxy luminosity and its dark matter halo environment. With photometric redshift information the dependence on the luminosity function can be eliminated and observations will provide a more direct constraint on the dynamical environment of the galaxies.

In Fig. 5 we show mean tangential shear dependence on the angular scale for red, blue and star forming galaxies, all with the luminosity cut $`\mathrm{M}_B<18`$. Also shear for all galaxies brighter than $`\mathrm{M}_B=18`$ is shown. Squares in this figure are observational points from SDSS (Fischer et al. 2000). Here we include the evolution of the cross-correlation function with redshift. Realistic lens and source distributions provided by the SDSS team are used in equation (12). They were derived as approximation of the respective SDSS distributions by the power-law with exponential cut-off of the form

$`W_i(z)={\displaystyle \frac{\beta _iz^2}{(z_i)^3\mathrm{\Gamma }\left(3/\beta _i\right)}}e^{\left(\frac{z}{z_i}\right)^{\beta _i}}`$ (13)
with $`z_1=0.17`$, $`\beta _1=2.3`$ and $`z_2=0.35`$, $`\beta _2=1.7`$ for lens and source samples respectively.

Our galaxy samples are roughly complete for galaxies brighter than $`\mathrm{M}_B=18`$ so we miss some lensing signal from the nearby galaxies ($`z<0.02`$) when we use SDSS window functions $`W_1(\chi )`$ and $`W_2(\chi )`$. But uncertainties related to measured source redshift distribution are likely to be more important in this comparison.

Qualitatively the predictions follow the observations, but we notice that predicted lensing signal is above that detected by SDSS specially on large scales. One of the reasons for this behaviour is the discrepancy between the luminosity function from simulations we base on (Kauffmann et al. 1999a) and that obtained from observations (Lin et al. 1997). Modelled luminosity function in the $`\mathrm{\Lambda }CDM`$ model lacks less luminous galaxies, fainter than $`\mathrm{M}_B=21`$, and has an excess of very luminous galaxies, brighter than $`\mathrm{M}_B=23`$, when compared to the present day observations. Since as shown in Fig. 2 there is a strong luminosity bias expected for the cross-correlation amplitude this enhances the theoretical prediction for the shear amplitude. The other uncertainty is already mentioned poorly known redshift distribution function. Both of these currently limit the interpretation of results, but will be eliminated once the photometric redshift techniques become realiable. This is specially promising in the case of SDSS where 5 colour photometry should enable one a very accurate redshift determination.

## 4 Cross-correlation coefficient

If the bias between the galaxies and the dark matter is not constant, as indicated from the results in Fig. 1, then one cannot use its determination at a given scale and apply it at another scale. This complicates the extraction of the dark matter power spectrum from the galaxy power spectrum and additional information is necessary to break the degeneracy. One can parameterize the ignorance with a scale dependent bias (van Waerbeke 1998, Dolag & Bartelmann 1998), where the relation between the galaxy, cross and matter power spectra is $`P_{\mathrm{g},\mathrm{dm}}(k)=b(k)P_{\mathrm{dm},\mathrm{dm}}(k)`$ and $`P_{\mathrm{g},\mathrm{g}}(k)=b^2(k)P_{\mathrm{dm},\mathrm{dm}}(k)`$. In this case from a measurement of galaxy and cross-spectrum one can determine both $`b(k)`$ and $`P_{\mathrm{dm},\mathrm{dm}}(k)`$. Even this relation is however not general, since the bias factor that relates one pair of spectra may not be the same as that for another pair. One can generalize this by introducing the cross-correlation coefficient between the galaxy and the dark matter power spectrum

$$r^2(k)=\frac{P_{\mathrm{g},\mathrm{dm}}^2(k)}{P_{\mathrm{g},\mathrm{g}}(k)P_{\mathrm{dm},\mathrm{dm}}(k)}.$$
(14)
This definition implies that in the case of a scale dependent but linear bias discussed above the cross-correlation coefficient $`|r(k)|=1`$.

In Fig. 8 we present the correlation coefficient as a function of scale for a set of galaxy samples selected by luminosity, colour or star formation rate criteria. In the right panels we show the correlation coefficient without the subtraction of the shot noise term arising form the discreteness of galaxies. In this case the correlation coefficient is constrained to $`|r(k)|1`$. More commonly however the shot noise is subtracted from the galaxy spectrum, which is shown in the left panels of Fig. 8. In this case for all of the considered galaxy samples the correlation coefficient remains close to unity up to $`k1h\mathrm{Mpc}^1`$. This is encouraging because when $`r(k)=1`$ one can extract the bias parameter $`b(k)`$ and the dark matter power spectrum $`P_{\mathrm{dm},\mathrm{dm}}(k)`$ directly by measuring the galaxy power spectrum and the cross-power spectrum from the galaxy-galaxy lensing. This means that at least on large scales there is hope that bias could be determined with this method.

The correlation coefficient remains closer to unity on small scales when the shot noise is subtracted as opposed to when it is not. This is related to the fact that for a given halo mass the pair weighted number of galaxies inside the halo $`N(N1)^{1/2}`$ agrees with the linearly weighted number $`N`$ for large halo masses (Seljak 2000) and that for large haloes there are many galaxies inside the halo so the central galaxy, if present, does not have a dominant contribution. Since the power spectrum on large (but still nonlinear) scales is dominated by large haloes this gives $`r(k)1`$. For small haloes typically $`N>N(N1)^{1/2}`$ because there are many haloes with just one galaxy in it, which gives rise to $`r(k)>1`$ when shot noise is subtracted from the galaxy spectrum. In addition, for haloes with just one galaxy this galaxy is usually at the centre of the halo, which enhances the cross-correlation relative to the case where the galaxies are distributed like dark matter inside the halo (the distinction between the central galaxy and all galaxy cross-correlation is discussed further in the next section). Both of these effects become prominent when $`N<1`$ and so are more important for galaxies that are rarer and/or on smaller scales where small haloes dominate. This agrees with the results in Fig. 8, where for the brightest and rarest sample of galaxies, $`\mathrm{M}_B<21`$, the correlation coefficient increases monotonically already from $`k0.5h\mathrm{Mpc}^1`$, whereas for the faintest and most abundant sample $`\mathrm{M}_B<18`$ the cross-correlation coefficient is unity up to $`k3h\mathrm{Mpc}^1`$.

It is interesting to note that for the sample of red galaxies ($`\mathrm{M}_B\mathrm{M}_V>0.8`$ and $`\mathrm{M}_B<18`$), which exhibit a strong scale dependent bias as seen from Fig. 1, $`r(k)`$ is unity over several orders of magnitude in $`k`$. In small haloes where $`N1`$ one has $`NN(N1)^{1/2}`$ (Seljak 2000). These galaxies therefore cannot be the central galaxies inside the haloes which would give $`N>N(N1)^{1/2}`$. For the same reason the cross-spectrum is also not enhanced relative to the dark matter, both of which explains why $`r(k)1`$ over the entire range of scales. While this is suggestive we should be cautious not to overinterpret these results since SAMs may not be accurate in such detailed properties. It will be interesting to examine whether this prediction can be confirmed from the observations directly.

For galaxies with star formation rate (SFR) above $`3\mathrm{M}_{}/\mathrm{yr}`$ the cross-correlation coefficient $`r(k)`$ has a strong decrease on scales of order $`24h\mathrm{Mpc}^1`$. This is caused by a decrease in $`P_{\mathrm{g},\mathrm{dm}}(k)`$ (and $`P_{\mathrm{g},\mathrm{g}}(k)`$) relative to $`P_{\mathrm{dm},\mathrm{dm}}(k)`$. The reason for this decrease is the transition from the correlations between the haloes to correlations inside the halo, which is prominent for star forming galaxies which are predominantly in the field, but is washed out by the group and cluster contribution in the dark matter power spectrum.

## 5 Dark matter halo profiles

In previous sections we have presented the predictions for the galaxy-dark matter cross-correlations for various galaxy selections. We have shown that the predictions strongly depend on the galaxy luminosity, colour and star forming rate. For example, we observe a strong luminosity dependence with more luminous galaxies showing a stronger cross-correlation on small scales because such galaxies reside in more massive haloes. These haloes are still below the nonlinear mass scale and no luminosity bias caused by halo bias is observed on large scales, nor is any luminosity bias observed for galaxy auto-correlation. This shows that galaxy-galaxy lensing can be a sensitive probe of halo masses for different galaxy types. Such interpretation was the basis for most of the galaxy-galaxy lensing analysis so far (Brainerd et al. 1996, Griffiths et al. 1996 , Hudson et al. 1998). Still, the relation between the cross-correlation spectrum and the halo mass profile may be a complicated one and in this section we explore this in more detail.

In the hierarchical clustering picture galaxies form in the dark matter haloes which subsequently merge into larger structures. Galaxies which are not members of a cluster or a group at the time of observation should have more extended haloes than those of same luminosity already swollen by a cluster because tidal forces in a potential well of a cluster tend to strip the galaxies of their haloes. Present observations of the weak gravitational lensing within the clusters seem to support this picture (Natarajan et al. 1999). However, such galaxies are nevertheless embedded in larger haloes and galaxy-galaxy lensing will be sensitive also to the gravitational lensing effect of these more massive and more extended structures. This effect is specially important on scales above $`100h^1\mathrm{kpc}`$ where the haloes of individual galaxies have only a weak signal which can easily be dominated by a sub-population of galaxies in groups and clusters (Seljak 2000). This effect is also important on smaller scales, because there is a range of halo masses that contribute to galaxies selected by their (absolute or apparent) luminosity and haloes of different mass can dominate on different scales. This makes the interpretation of the cross-correlation in term of the halo profile more complicated than simply an averaged dark matter profile of a typical galaxy.

Plots in Fig. 6 show the cross-power spectra for several galaxy types selected by their intrinsic luminosity as a function of halo mass. Smaller haloes give lower amplitude of the cross-power spectrum as expected. Note that for most galaxy types the total cross-spectrum shown with a dotted line does not agree with any individual mass range. The logarithmic slope $`\alpha (k)=d\mathrm{ln}\sigma ^2(k)/d\mathrm{ln}k`$ of the total cross-spectrum is shallower that the slope of individual mass intervals (Fig. 7). On larger scales the cross-spectrum is dominated by the high mass range of the haloes, while on smaller scales smaller haloes become dominant and since these give lower amplitude of the cross-spectrum the overall slope is shallower. The only exception are the very bright galaxies with $`M_B<21`$ which seem to reside predominantly in haloes more massive than $`10^{13}h^1M_{\mathrm{}}<M<10^{14}h^1M_{\mathrm{}}`$ and their cross-correlation function agrees well with the cross-correlation for this mass interval.

Second complication which is important for large haloes such as groups and clusters is that when there are many galaxies inside the halo most of them are not at the halo centre. When a galaxy is at the outskirts of a halo the lensing effect will on average be reduced. If the galaxies are distributed like the dark matter then one can account for this effect. However, galaxies may not be distributed in the same way as the dark matter inside the haloes (Diaferio et al. 1999) and this becomes difficult to model analytically. This effect is shown by comparing the first plot of Fig. 6, where the cross-spectra of central galaxies are shown as a function of halo mass intervals, to the other plots in that figure, where all the galaxies inside the halo are selected. For the two most massive intervals there is a significant difference between the central galaxy and all galaxies cross-spectra. The all galaxy spectra are substantially lower in amplitude as expected if most galaxies are not central. For lower mass intervals the difference between the two is smaller, since such haloes contain on average at most one bright galaxy, which is usually the central one.

It is instructive to investigate the slope of the cross-spectrum. For central galaxies this gives the slope of the dark matter profile as a function of scale. On small scales the slope approaches $`\alpha 1.5`$ for the largest haloes with mass range $`10^{14}h^1M_{\mathrm{}}<M<10^{15}h^1M_{\mathrm{}}`$ for which the force resolution in units of virial radius is the highest. It is possible that the slope would become even shallower if higher resolution simulation are used, although some of such higher resolution simulations suggest the slope remains close to $`\alpha 1.5`$ down to very small scales (Moore et al. 1999). This slope steepens to $`\alpha 2.5`$ at the virial radius, beyond which the contribution from the other haloes becomes important and makes the slope less steep again. The behaviour for smaller halo mass intervals is similar, but shifted to smaller scales. At the resolution limit ($`k20h\mathrm{Mpc}^1`$) the slope decreases with the decreasing halo mass, caused by the finite force resolution. For the lowest mass interval the transition between the halo profile and other halo correlations is quite prominent, but this is at least in part caused by the finite mass resolution because there are no very small haloes very close to the virial radius of such haloes. For all the galaxies, except the very brightest ones, the cross-spectrum slope is always shallower than for the central galaxies when viewed as a function of the halo mass.

## 6 Conclusions

We have investigated the galaxy-dark matter cross-correlation function and power spectrum using the semi-analytic galaxy formation models coupled with a high resolution N-body simulation. These cross-correlation spectra are the necessary ingredients to interpret the observed correlations based on weak lensing and galaxy positions. Examples are galaxy-galaxy lensing, QSO-galaxy correlations and foreground-background galaxy correlations. The advantage of the approach used here is that it includes many complicating effects such as the variety of the dark matter environments around the galaxies, the distribution of galaxies inside the dark matter haloes, correlations between the galaxies and the dark matter in the neighbouring haloes. The limitation of this approach is the mass and force resolution, which limits our study to haloes more massive than $`10^{11}h^1M_{\mathrm{}}`$ (limiting the completeness of galaxy sample to $`M_B<18`$) and scales larger than $`50h^1\mathrm{kpc}`$ . This can be overcome to some extent by using the analytic model, which can extend these results to smaller scales (Seljak 2000).

The cross-correlation spectra on large scales fall between the dark matter and galaxy auto-correlation spectra. As a function of redshift the correlation strength of bright galaxies first decreases to $`z1`$ and then remains approximately flat because such galaxies become rare and biasing increases above unity, even if the dark matter correlation strength continues to decrease. In terms of the cross-correlation coefficient we find it is close to unity on large scales. This is good news for attempts to determine the scale dependent bias and the dark matter power spectrum directly from the galaxy correlations and the galaxy-galaxy lensing. It breaks down on small scales where effects such as the average number of galaxies inside the halo dropping below unity and presence of central galaxies cause the cross-correlation spectrum to exceed the auto-correlation spectra. This happens on larger scales for rarer galaxies, which are therefore less suitable for the determination of the dark matter power spectrum.

While on large scales the cross-correlations reflect clustering of large scale structure, on small scales they reflect more the dark matter environments around galaxies. These have been often interpreted as the dark matter profiles of haloes, but such interpretation is complicated by the effects of multiple galaxies inside clusters and by having different halo mases dominate on different scales. For example, the observed signal at $`1h^1\mathrm{Mpc}`$ by SDSS (Fischer et al. 2000) does not mean that the haloes of galaxies such as our own can be observed at such large distances, but rather that groups and clusters extend to these distances or that haloes are correlated with neighbouring haloes. Still, the predicted signal on small scales does increase with the galaxy luminosity, as predicted by the Tully-Fisher relation if the dark matter makes a significant contribution to the maximal rotation velocity. This shows that the cross-correlations can become a complementary tool to study the dark matter environments of the galaxies. This can be potentially a powerful probe of the dark matter around the galaxies, because it is sensitive to larger distances from the centre which are less affected by the baryonic component. The upcoming surveys such as SDSS and 2dF should be able to extract this information with a high statistical precision, making galaxy-galaxy lensing an important observational tool in connecting galaxies to their dark matter environment.

## Acknowledgments

We thank Guinevere Kauffmann and Antonaldo Diaferio for providing results of GIF N-body simulations and semi-analytic simulations and for help with them. We also thank Phil Fischer for providing us with the galaxy redshift distributions as measured by the SDSS team. J.G. was supported by grants 2P03D00618 and 2P03D01417 from Polish State Committee for Scientific Research. U.S. acknowledges the support of NASA grant NAG5-8084.


format: text

text:
# Learning and generation of long-range correlated sequences

## I Introduction

Real-life (temporal) sequences are characterized by a certain degree of correlation. It is known that a wide range of systems in nature displays long-range correlations, e.g., biological - DNA sequences and heartbeat intervals, natural - languages, etc., see . Since long-range correlations can appear in many forms, we restrict the analysis to the case of power-law correlations in a random sequence, e.g., the correlation function for a 1D sequence $`x_i`$ is given by

$$C(l)=x_ix_{i+l}l^\gamma (l\mathrm{},\gamma >0),$$
(1)
where the angular brackets denote an average over the randomness. This type of random sequence is also termed “colored” or correlated-noise. The general form of description allows us to investigate the capability of the network to capture statistical properties of a sequence.

The theory of learning from examples by a neural network, and in particular on-line learning, has been developed almost exclusively for uncorrelated patterns, see . Though some particular cases of correlated patterns were treated, they were limited to simple spatial correlations within each pattern, or to temporal correlations of each input unit, e.g., . The case of long-range correlations is absent. Clearly, the problem of extracting a feature from a correlated sequence whose length is much larger than the network’s size, cannot be treated under the same assumptions. Rather than dealing with the question of the generalization error (average over a distribution of patterns), we focus on the capability of the network to asymptotically capture the correlations within the sequence and its ability to generate a sequence with similar properties.

As shown previously, the sequence generator (SGen), a continuous-valued feed-forward network in which the next state vector is determined from past output values, exhibits (quasi) periodic attractors in the stable regime, regardless of the complexity of the weights, both in the case of a perceptron as well as multilayer SGen’s, e.g., ; the unstable, chaotic regime is studied in . Therefore, it is obvious that the perceptron-SGen, or its extension to multilayer networks, are not suitable candidates for learning and generating correlated noise. The natural way to overcome this limitation is to increase the complexity of the feedback in the architecture. In this paper we study a fully connected, asymmetric network. The updating rule for the network’s state is either sequential or parallel, namely, each unit is updated on its turn or all units are updated simultaneously. Unit $`i`$ $`(i=1,\mathrm{},N)`$ is updated as follows

$`S_i^{t+1}`$ $`=`$ $`\mathrm{tanh}\left(\beta \left[{\displaystyle \underset{j=1}{\overset{i1}{}}}W_{ij}S_j^{t+1}+{\displaystyle \underset{j=i}{\overset{N}{}}}W_{ij}S_j^t\right]\right)`$ (2)
$`S_i^{t+1}`$ $`=`$ $`\mathrm{tanh}\left(\beta {\displaystyle \underset{j=1}{\overset{N}{}}}W_{ij}S_j^t\right)`$ (3)
where eq. 2(3) refers to the sequential(parallel) rule; $`𝐖`$ is an \[NxN\] weight matrix and $`\beta `$ is a gain parameter. The network generates iteratively an infinite sequence $`\{\sigma _m\}`$ starting from an initial state, $`𝐒^\mathrm{𝟎}`$, as follows:

$$\sigma _m=S_i^{t+1},m=tN+i=1,2,\mathrm{},$$
(4)
where $`i=1,\mathrm{},N,t=0,1,2\mathrm{}`$.

Two complementary issues are discussed in this paper: (a) Given a training sequence characterized by long-range correlations, can we train a network in an on-line scheme to generate a sequence with the same asymptotic statistical properties? (b) The inverse problem, is there an interplay between a network whose weight matrix follows a power-law correlation function, and the sequence it generates?

It is important to stress that the model we investigate is not proposed as a practical method for generating long-range correlated sequences, rather, it is motivated by the issues raised above.

In the next section we investigate the first question. An “on-line”, gradient-based learning rule is applied where each example is presented to the network only once. The sequences that constitute the basis for the training patterns and the correlated weight matrices, are generated using an algorithm for re-shaping the power spectrum of an uncorrelated sequence; the method has been developed for investigating various stochastic processes, see . In section III the inverse problem is analyzed. A method for constructing the weight matrix is presented based on the findings obtained from section II regarding the correlation properties of the weights in trained networks. A simple analytical derivation support these findings.

## II Generalizing the rule of a colored sequence

Suppose a source generating sequences that obey eq. 1; the question we address in this section focuses on the possibility of learning the statistical properties of the source, and in particular the exponent $`\gamma `$ of the power-law correlation function. For a network defined by its weights $`𝐖`$, a gain $`\beta `$ and a nonlinear function $`f`$, the response of the ith unit, given the current state $`\xi ^t`$, is

$`S_i^{t+1}=f(\xi ^t,𝐖_i),`$

where $`𝐖_i`$ denotes the vector of weights connected to the ith unit. The on-line learning algorithm minimizes a quadratic error function

$$ϵ_i(\xi ^t,𝐖_i)=[S_i^{t+1}\tau _i^{t+1}]^2/2,$$
(5)
where $`\tau _i^{t+1}`$ is the desired response of the ith unit given the state $`\xi ^t`$. The weights are updated according to

$$𝐖_i^{t+1}=𝐖_i^t\frac{\eta }{N}_Wϵ_i(\xi ^t,𝐖_i^t),$$
(6)
i.e., a gradient descent rule with a learning rate $`\eta `$ (similar results were obtained using the Hebbian learning rule).

The training patterns are defined as follows. Let $`D_L=\{x_1,x_2,\mathrm{},x_L\}`$ be a 1D sequence obeying eq. 1. A training pattern (the pair $`(\xi ^m,\tau _i^m)`$ $`0<mL2N+1`$, $`i=1,\mathrm{},N`$) is defined by

$$\{\begin{array}{ccc}\xi ^m& =& (x_m,x_{m+1},\mathrm{},x_{m+N1})\hfill \\ \tau _i^m& =& x_{m+N+i1}\hfill \end{array},$$
(7)
where each weight vector $`𝐖_i`$ is updated with the corresponding desired output $`\tau _i^m`$, and the same vector $`\xi ^m`$. Updating all $`N`$ weight vectors for a given pattern, $`(\xi ^m,\tau _i^mi=1,\mathrm{},N)`$, accounts for a single training cycle. The patterns for consecutive training cycles are achieved via sliding a window of size $`N`$ by one site along the sequence $`D_L`$; e.g., given a current pattern starting at site $`m`$ along the training sequence, $`\xi ^m`$ (eq. 7), the next pattern is $`\xi ^{m+1}=(x_{m+1},x_{m+2},\mathrm{},x_{m+N})`$. The training patterns can be obtained in a different scheme by sliding the window $`N`$ sites (non-overlapping windows); i.e.,

$$\{\begin{array}{cccc}\xi ^{m+1}& =& (x_{mN+1},x_{mN+2},\mathrm{},x_{mN+N})\hfill & \\ \tau _i^{m+1}& =& x_{(m+1)N+i}\hfill & \hfill m=0,1,\mathrm{}\end{array}.$$
(8)
Results obtained in both schemes are similar, however, the length of the sequence ($`D_L`$) used in the second scheme (to obtain the same results) is about $`N`$ times larger.

Let us now describe our numerical investigation. An ensemble of long sequences, $`D_L`$ (of size $`LN`$), that obey eq. 1 with a given $`\gamma `$ is generated. A randomly chosen network is trained using a part of the sequence. Taking the last pattern from the training process as an initial state, the trained network is used to generate iteratively a long sequence $`\{\sigma _m\}`$, of size $`MN`$ with $`M=10`$, following the sequential rule, eq. 2 and eq. 4. The correlation function of this sequence can be calculated in the two following ways: spatial and temporal. The spatial correlation is obtained by averaging the correlation function, calculated on ($`M`$) sequences of size $`N`$ after updating all ($`N`$) units, over the $`M`$ iteration cycles, whereas the temporal correlation is simply the correlation function of the long sequence, i.e.,

$$\begin{array}{ccc}C_{spat}(l)& =& 1/M_{j=0}^{M1}\left[1/N_{i=1}^N\sigma _{jN+i}\sigma _{jN+(i+lmodN)}\right]\hfill \\ C_{temp}(l)& =& (MN)^1_{i=1}^{MN}\sigma _i\sigma _{i+l},\hfill \end{array}$$
(9)
where we take periodic boundary conditions. Next, the same weight matrix is further trained and after each $`\alpha N`$ patterns ($`\alpha =10`$) , the same process of “generating a sequence and calculating its correlation function”, is repeated for a better statistical estimation. We found that both definitions of the correlation function yield similar results, therefore, in the sequel we omit the subscript from eq. 9 and refer to the temporal correlation function

$$C(l)=C_{temp}(l).$$
(10)
Note that the range of correlations is bounded by the number of degrees of freedom $`\{S_i\}_{i=1}^N`$, as in ; therefore, the correlation function is calculated in the range $`0<l<N/2`$ (a symmetric function). The whole procedure is applied for all members of the ensemble. This extensive averaging is necessary since the patterns $`\xi ^m`$ taken from the long sequences $`D_L`$, exhibit large fluctuations (recall that $`NL`$ and the variance decreases linearly with the size of the sequence).

Figure 1 depicts the results of the above procedure for $`\gamma =0.4,0.6,0.8(N=200)`$, with $`L10^5`$ and an ensemble of 50–100 samples. For comparison, we show the results of training with patterns obtained by sliding the window N-sites each cycle (non-overlapping windows, eq. 8); in this case, the sequence is much larger, $`L10^7`$. The data points are the average values with relative error-bars that vary from $`5\%`$ for small $`l`$ to $`40\%`$ (less than $`20\%`$ for non-overlapping windows) of the data point for $`lN/2`$, hence, we omit them to preserve the clarity of the figure. The learning rate used, $`\eta =2`$ (eq. 6), is not optimal; it is obvious that an optimization can reduce the fluctuations in the correlation function since it affects the relative change in the weights. We note that the fluctuations inspected in the sequences $`\{\sigma _m\}`$ generated by the trained networks are similar to those of the training patterns $`\xi `$, indicating a finite size effect. This has been confirmed for several network sizes.

It is interesting to examine how the network (learning algorithm) has embedded the relevant information associated with the correlations. At the end of the training process described above, we measured the correlation function of the weight vectors (averaged over realizations of $`D_L`$) in two directions, horizontal (over rows) and vertical (over columns), as follows:

$$\begin{array}{cccc}C_h(l)& =& _{i,j=1}^NW_{i,j}W_{i,j+lmodN}\hfill & \text{- horizontal}\hfill \\ C_v(l)& =& _{i,j=1}^NW_{i,j}W_{i+lmodN,j}\hfill & \text{- vertical}\hfill \end{array}$$
(11)
Results are presented in Fig. 2 for a training sequence obeying a power-law correlation function with exponent $`\gamma =0.6`$ and a network of size $`N=300`$. Clearly, the vertical correlations follow a rule similar to that of the sequence, $`C_v(l)l^{0.625}`$, while the horizontal correlations decay much faster, with an exponential fit $`C_h(l)\mathrm{exp}(0.03l)`$. The case of training with patterns obtained from non-overlapping windows is presented for comparison by the opaque circles. In this case, the vertical correlations are similar, $`C_v(l)l^{0.61}`$, however there are no horizontal correlations. We conclude that sliding one site each cycle induces horizontal correlations, however they decay exponentially fast.

The issue of learning the rule from a teacher is not treated in this framework, however, we add the following (numerical) observation: The overlap between two (initially random) networks, $`R=𝐖^1𝐖^2`$, learning from the same rule (training sequence) decreases with the size of the network ($`N`$) and remains very low even for $`LN^2`$, although the two networks generate sequences with similar correlation functions, asymptotically. The same holds when each network learns from a different sequence. We conclude that the network indeed learns the statistical properties of the sequence, but not its values. Clearly, in batch learning one expects that the network would learn the rule when $`L=N^2`$, since the number of free parameters (weights) equals the number of examples.

## III Generating colored sequences by a colored network

So far, we demonstrated the capability of the network to capture statistical properties from the training sequence. Let us now consider the inverse problem, i.e., that of constructing a network that is capable of generating correlated sequences. The information obtained from the trained networks in the previous section regarding the structure of the weight matrix, suggests that the significant correlation is present between the elements of a column, i.e., vertical direction. Therefore, we would like to compare the correlation function of sequences generated by networks with the same vertical correlation function (power-law), and various horizontal decay forms, i.e., power-law with an increasing exponent, $`\gamma `$. The weights are constructed as follows. Start by generating a random matrix of normally distributed elements. Each column is treated as a 1D sequence and is “colored” following the process described above for generating a 1D correlated sequence. After this stage, the rows are still uncorrelated. To achieve a different power-law function for the rows, we treat each row independently as a 1D sequence and follow the same procedure as above, this time with (possibly) a different exponent. This process generates a weight matrix with pronounced correlations in the vertical and horizontal directions only. We normalize the weights, $`_{i,j=1}^NW_{(i,j)}^2=N`$, such that $`\beta =𝒪(1)`$ (independent of $`N`$). The value of $`\beta `$ in the dynamic equations, eqs. 2-3, is taken well above bifurcation to increase the probability of non-periodic attractors  (we carefully avoid the periodic attractors in our measurements). In the analysis described below, each sample network (colored $`𝐖`$) is initialized at random ($`𝐒^0`$) and the correlation function of the sequence generated is calculated at long times.

Figure 3 depicts two cases for which the vertical correlation function of the weights decays polynomially with exponents $`\gamma _v=0.4`$ and $`\gamma _v=0.6`$. For each case, the horizontal correlation function takes one of the following three values: $`\gamma _h=\gamma _v`$, $`\gamma _h=2\gamma _v`$ or uncorrelated. The results were obtained for a network of size $`N=2048`$ and averaged over 50 realizations of the weight matrix. Additional averaging is done by starting from several initial conditions for each matrix. It is apparent that the symmetric case, $`\gamma _h=\gamma _v`$, gives rise to a relatively poor long-range correlations in the generated sequence. The other two cases exhibit much longer-range correlations. Although we are not trying to determine the optimal correlation function, it seems that weak correlations are better than lack of horizontal correlations. This is in agreement with our findings regarding the trained networks, see Fig. 2.

To conclude, we propose a naive calculation which should serve as a starting point to the analytical investigation of the model. The quantity of interest in our calculation is the asymptotic correlation function,

$$C(l)=ZS_i^tS_{i+l}^t_{t,W},$$
(12)
where $`Z`$ is a normalization factor ($`C(0)=1`$). The average is taken over the time $`t`$ and the realizations of the weights, expecting $`C(l)`$ to be independent of the site $`i`$. For simplicity we use the parallel updating rule, eq. 3, hence, the stationary state of the correlation function may be given by

$$C(l)=Z\mathrm{tanh}\left(\beta \underset{j=1}{\overset{N}{}}S_j^tW_{i,j}\right)\mathrm{tanh}\left(\beta \underset{j=1}{\overset{N}{}}S_j^tW_{i+l,j}\right)_{t,W}.$$
(13)
The approximation of eq. 13 consists of linearizing the r.h.s. and assuming $`𝐒`$ independent of the realization of $`𝐖`$, leading to

$$C(l)\underset{j,k=1}{\overset{N}{}}S_j^tS_k^t_tW_{i,j}W_{i+l,k}_W.$$
(14)
Next, we identify the averages as the correlation functions, defined above, that depend on the distance only, and rewrite eq. 14 using $`mkj`$ in the form

$$C(l)=\widehat{Z}\underset{m=1}{\overset{N}{}}C(m)C_W(l,m),$$
(15)
where $`\widehat{Z}`$ is a normalization factor, $`C_W(l,m)`$ denotes the 2D correlation function of the weights, and $`C(l)`$ is the 1D function which is the quantity of interest. In the scenario described above, $`C_W`$ is known a priori (independent of time) since the weights are constructed. If we assume a power-law vertical correlations for $`C_W`$ and no horizontal correlations, the correlation function of the sequence, $`C(l)`$, simply follows the vertical correlations of the weights, i.e., $`C(l)=\widehat{Z}_mC_W(l,m)\delta _{m,0}=\widehat{Z}C_W(l,0)`$, supporting the above findings. We remark that this decomposition of $`C_W`$ into independent vertical and horizontal functions is still a good approximation when $`C_h`$ is not a delta function, as long as $`C_h`$ decays much faster than $`C_v`$, which enables us to neglect other correlations. In this case eq. 15 can be formulated as an “Eigen-value problem” of the matrix $`C_W`$. Few such cases have been solved numerically for which the assumption of decomposition was found consistent, see . When this decomposition is no longer valid, we observe a breakdown of the long-range power-law behavior, see Fig. 3 \- the case $`\gamma _h=\gamma _v`$.

## IV Discussion

In this paper we analyzed the capability of a neural network model to learn the rule of a long-range correlated sequence on the one hand, and a method for constructing a network that is able to generate such sequences on the other hand. We demonstrated that a simple on-line learning algorithm can be used to extract the rule, provided that the sequence is long enough. The fluctuations observed in the training patterns are manifested in the generated sequences as well. The investigation of the weights that were obtained during the learning process indicates that the vertical correlations ($`C_v`$, eq. 11) play the most important role in generating correlated sequences by the network. Employing this finding, we were able to construct networks (without training) that are capable of generating sequences with a predefined power-law correlation function. Indeed, we found that additional significant horizontal correlations in the constructed networks’ weights corrupt this property. These observations were confirmed by a naive analytical treatment of the stationary correlation function.

The question of the optimal learning rate ($`\eta `$) was not treated although it seems to be an important parameter in the convergence of the training process. Another question which deserves further research regards the analytical derivation of the correlation function. Taking into account the nonlinearity of the transfer function is necessary to close the naive calculation self-consistently, and to obtain the corrections to the correlation function.

## Acknowledgments

We thank W. Kinzel for fruitful discussions and critical reading of the manuscript. IK acknowledges the support of the Israel Ministry of Science.