text:
# Gamma-ray burst beaming: a universal configuration with a standard energy reservoir?

## 1. Introduction

Recently, several independent approaches have led to the conclusion that long gamma-ray bursts (GRBs) have a standard energy reservoir of several $`10^{50}\mathrm{ergs}`$ (Frail et al. 2001, hereafter F01; Panaitescu & Kumar 2001, hereafter PK01; Piran et al. 2001, hereafter P01). An important ingredient of this argument is that the putative jet opening angles, $`\theta _j`$, as inferred from the afterglow lightcurve breaking times, $`t_b`$, have a broad distribution, but just of the right form to compensate for the wide dispersion of the “isotropic” energy emitted in $`\gamma `$-rays, $`E_{\gamma ,\mathrm{iso}}`$, so that $`E_\gamma =(E_{\gamma ,\mathrm{iso}}/4\pi )(\theta _j^2/2)`$ is essentially invariant (F01). The total energy of the fireball should be $`E_{\mathrm{tot}}E_\gamma +E_0`$, where $`E_0`$ is the initial kinetic energy of the fireball in the afterglow phase assuming an adiabatic evolution, and the inequality takes into account the possible energy loss during the radiative regime in the early afterglow phase that has evaded the present observations, as well as energy losses outside the $`\gamma `$-ray band (e.g. the BATSE window) or in non-electromagnetic forms (e.g. neutrinos and gravitational waves) during the prompt phase. Writing $`E_\gamma =\eta E_{\mathrm{tot}}`$ where $`\eta `$ is the gamma-ray emission efficiency, $`E_{\mathrm{tot}}`$ could be mainly contributed by $`E_0`$ if $`\eta `$ is small (e.g. $`<0.1`$). PK01 and P01 found that $`E_0`$ is also distributed in a narrow range. For a uniform jet, this leads to the inference that $`E_0=(dE/d\mathrm{\Omega })(\theta _j^2/2)`$ is also essentially invariant. However, in the above analysis, and in the current afterglow jet models which are used to determine $`\theta _j`$, it is generally assumed that the jets are uniform, with sharp cut-offs at the edges, and that the line-of-sight cuts right across the jet axis. None of these assumptions are necessarily true in general (Mészáros , Rees & Wijers 1998; MacFadyen & Woosley 1999; Woods & Loeb 1999; Nakamura 1999; Paczyński 2001; Salmonson 2001; Dai & Gou 2001). On the other hand, although it is not difficult to construct a central engine model which makes GRBs with a standard energy reservoir but with quite different beaming angles, it would be more elegant to have a model that all the GRB beams share a standard energy reservoir as well as a quasi-universal beaming configuration (M. J. Rees, 2001, private communication). Here we show that such a model can be constructed by taking account of the off-axis anisotropic jet effects, without violating the present observational constraints.

## 2. The Model

Our assumption is that all the long GRBs have a quasi-universal beam configuration, with a strong anisotropy of the angular distribution of the fireball energy around an axial symmetry. The jet axis is physically related to the rotational axis of the central engine, so it is reasonable to assume that initially the closer to the jet axis, the higher the energy concentration. The actual angular distribution of the fireball energy is unknown, and we model it as (e.g. Mészáros et al. 1998)

$$dE/d\mathrm{\Omega }=ϵ(\theta ,\varphi )=ϵ(\theta )=ϵ_0\theta ^k,$$
(1)
within the range $`\theta _m\theta \mathrm{\Theta }`$, where $`\theta _m`$ is a very small angle within which some deviation from (1) is necessary to avoid the divergence at $`\theta =0`$, and $`\mathrm{\Theta }`$ is some large angle which exceeds the presently measured $`\theta _j`$ by at least a factor of two (for the simplification of the discussions below). The real angular energy distribution may differ from the power law (1), but most of our discussions below can be generalized to other forms of distribution functions (e.g. see  and relevant discussions below). The adoption of (1) is for the simplicity of the discussions. The angular-dependence of the baryon loading rate is uncertain, and we assume that it is weak so that the Lorentz factor angular distribution follows a similar law, i.e., $`\mathrm{\Gamma }(\theta )\theta ^k`$ (of course, the law should be modified when $`\mathrm{\Gamma }(\theta )`$ approaches unity). We make furthermore the assumption that

$$(1.5)<k\stackrel{<}{}2$$
(2)
in (1). The reason for this requirement will become evident later. The main conjecture of the model is that the dispersion in the afterglow data of the breaking time, $`t_b`$, is a manifestation of the diversity of viewing angles of the observers, rather than to the diversity of intrinsic opening angles of the jets themselves. In other words, that what were inferred by Frail et al. (2001) as $`\theta _j`$ are essentially $`\theta _v`$ in our model, where $`\theta _v`$ is the observer’s viewing angle with respect to the jet axis. We will test whether the above hypothesis is able to pass the following three criteria: (i) When $`\mathrm{\Gamma }(\theta _v)1/\theta _v`$, the jet dynamics along the line-of-sight satisfies the isotropic law $`\overline{\mathrm{\Gamma }}(\theta _v,t)t^{3/8}`$ (for simplicity, we only discuss an adiabatic fireball running into an interstellar medium with a constant density), where $`t`$ is the observer time, and $`\overline{\mathrm{\Gamma }}(\theta _v,t)`$ is an effective Lorentz factor assuming an isotropic fireball which could mimic the emission in the direction $`\theta _v`$ at the time $`t`$; (ii) When $`\mathrm{\Gamma }(\theta _v)\stackrel{<}{}1/\theta _v`$, the dynamics changes so that the lightcurves steepen; (iii) The total jet energy $`E_{\mathrm{tot}}`$ is essentially a universal value.

For an isotropic adiabatic fireball running into a uniform medium, $`\mathrm{\Gamma }(t)ϵ^{1/8}n^{1/8}(at)^{3/8}`$, where $`ϵ=dE/d\mathrm{\Omega }`$ is the energy per solid angle, $`n`$ is the ambient medium number density, and the blastwave radius is written in a general form as $`R=a\mathrm{\Gamma }^2ct`$, where the factor $`a`$ effectively takes into account the surface of equal-arrival-time as well as the thickness of the emitting region. To test the criteria (i), the key is to estimate the effective energy per solid angle, $`\overline{ϵ}(\theta _v,\varphi _v,t)=\overline{ϵ}(\theta _v,t)`$, in the direction $`(\theta _v,\varphi _v)`$, and to evaluate the possible time-dependence of this value. When $`\mathrm{\Gamma }(\theta _v,t)=\mathrm{\Gamma }1/\theta _v`$, the observer can only observe a solid angle around $`(\theta _v,\varphi _v)`$ with a half opening angle of order $`1/\mathrm{\Gamma }`$ due to the relativistic beaming effect111Strictly speaking, the observer will see a smaller half cone on the close side of the jet axis, and a larger half cone on the far side to the axis, due to different Lorentz factors in different directions. This will modify the integral limits in (3), but does not influence the conclusion in (4) and the relevant discussions.. By definition, the effective energy per solid angle in the direction $`(\theta _v,\varphi _v)`$ is

$$\overline{ϵ}(\theta _v,\varphi _v,t)=\overline{ϵ}(\theta _v,t)=\frac{_{\theta _v1/\mathrm{\Gamma }}^{\theta _v+1/\mathrm{\Gamma }}ϵ(\theta ,t)\mathrm{sin}\theta d\theta }{_{\theta _v1/\mathrm{\Gamma }}^{\theta _v+1/\mathrm{\Gamma }}\mathrm{sin}\theta d\theta }$$
(3)
due to the axial symmetry. In the small angle approximation, which is relevant to the present discussions222The largest “jet” angle in F01 is 0.411, and the approximation is good within 3%., one has $`\mathrm{sin}\theta \theta `$. When $`\mathrm{\Gamma }1/\theta _v`$ and noticing (1), this gives

$$\overline{ϵ}(\theta _v,t)\frac{_{\theta _v1/\mathrm{\Gamma }}^{\theta _v+1/\mathrm{\Gamma }}ϵ_0\theta ^{1k}𝑑\theta }{_{\theta _v1/\mathrm{\Gamma }}^{\theta _v+1/\mathrm{\Gamma }}\theta 𝑑\theta }ϵ_0\theta _v^k=ϵ(\theta _v).$$
(4)
This is a time-independent quantity, since the sideways expansion effect is not important at the same stage (see discussions below). We then have

$$\overline{\mathrm{\Gamma }}(\theta _v,t)[\overline{ϵ}(\theta _v,t)]^{1/8}n^{1/8}(at)^{3/8}t^{3/8},\mathrm{\Gamma }1/\theta _v.$$
(5)
This indicates that the observer does not feel the anisotropy of the fireball when the relativistic beaming angle $`1/\mathrm{\Gamma }`$ is much smaller than the viewing angle $`\theta _v`$, but observes the fireball as if it were isotropic. This is the same conclusion as drawn in the on-axis uniform jet model. The conclusion (4) does not require (2) and holds for any $`k`$ value. In fact, it even holds for some other forms of angular energy distributions, e.g. the Gaussian distribution,

$$ϵ(\theta )=ϵ_0\mathrm{exp}[(1/2)(\theta /\theta _0)^2],$$
(6)
as long as the first order Taylor expansion term of these functions lead to the same result. Notice that the factor $`a`$ of various forms may deviate from the conventional value (e.g. $`4`$), since the shape of the equal-arrival-time surface will be distorted due to the anisotropic distribution of the fireball energy. However, its time-dependence, if any, would be very small. Therefore, although it may influence the absolute values of the afterglow flux levels, such an effect does not change the blastwave dynamics in the viewing direction.

In principle, the jet configuration is time-dependent, due to the effects such as the energy redistribution and the lateral expansion. In the lab frame, the causally connected region subtends an angle of $`\mathrm{\Delta }\theta c_st^{}/R=c_s/c\mathrm{\Gamma }(\theta )`$, where $`t^{}=R/c\mathrm{\Gamma }(\theta )`$ is the comoving time since the explosion along the $`\theta `$ direction, $`c_s`$ is the expansion speed, which may be either the relativistic speed of sound $`c/\sqrt{3}`$ (Rhoads 1999), or simply the speed of light (Sari et al. 1999). As $`\mathrm{\Gamma }(\theta _v)1/\theta _v`$, the line-of-sight direction is causally disconnected from other regions, so the dynamics evolves essentially independently, so that the description in (3) holds. When the blastwave decelerates so that the line-of-sight bulk Lorentz factor $`\mathrm{\Gamma }(\theta _v)`$ drops close to and below $`1/\theta _v`$, the dynamics along the line-of-sight starts to change, and $`\overline{\mathrm{\Gamma }}(\theta _v)`$ will deviate from the $`t^{3/8}`$ dependence. There are several effects that play a role. First, as $`1/\mathrm{\Gamma }(\theta _v)`$ exceeds $`\theta _v`$, the observer starts to feel the energy deficit due to the drop of the energy distribution, i.e. deviation of the power law (1), on the other side of the jet axis. Although the calculation of $`\overline{ϵ}(\theta _v,t)`$ is no longer straightforward, this deficit effect should mimic that in the uniform jet model as long as $`k`$ is not too flat, say, $`k>1.5`$. Second, the anisotropic jet has a trend to resume the isotropic shape. As $`\mathrm{\Gamma }(\theta _v)1/\theta _v`$, the viewing direction starts to connect the jet axis causally. The energy outflow from the cone defined by $`\theta _v`$ becomes prominent, and this equivalently decreases $`\overline{ϵ}(\theta _v)`$ in the viewing direction. In the meantime, the initial material within the $`\theta _v`$ cone starts to spread into a wider cone, and the observer would feel a stronger deceleration, although the global sideways expansion will become evident only when $`\mathrm{\Gamma }(\theta _v)`$ drops below $`1/\mathrm{\Theta }`$. All these effects tend to steepen the afterglow lightcurve, although the degree of steepening is unclear without detailed numerical dynamical calculations. In any case, in the asymptotic phase of sideway expansions, $`R`$ is essentially a constant, and one would eventually have $`\overline{\mathrm{\Gamma }}(\theta _v,t)t^{1/2}`$ (from $`tR/\overline{\mathrm{\Gamma }}^2`$, Rhoads 1997, 1999; Sari et al. 1999). In this regime, the temporal indices of the lightcurves in the various spectral regimes would follow closely the same predictions as in the uniform jet models (Sari et al. 1999; Rhoads 1999). For example, in the slow-cooling regime (which is usually the case after the viewing or “jet” break), for spectral regimes both below and above the cooling frequency, the asymptotic spectral flux is $`F_\nu t^p`$, where $`p`$ is the power-law index of the electron number distribution. For reasonable values of $`p`$ (e.g. $`2.2`$), this is consistent with several GRB afterglow observations. The above discussion should also hold for other distribution functions such as (6), mainly because eventually all the initial configurations will be smeared out. However, to address the lightcurves properly within different models, including the relevant gradual transition between asymptotic regimes, a detailed dynamical description and numerical calculation is necessary, and we postpone this to a future work.

We have shown that the present model can reproduce the key features of the on-axis uniform jet model, with an arbitrary $`k`$ value as long as it is not too flat. The next question is whether the model can also retain the merit of a standard energy reservoir invoked in the conventional jet model. In principle, one does not have to fulfill this constraint, but just wishes so for the sake of elegance. By definition, the total energy in a fireball with an energy distribution given by (1) is

$$E_{\mathrm{tot}}=2\pi _0^\mathrm{\Theta }ϵ(\theta )\mathrm{sin}\theta d\theta 2\pi _{\theta _m}^\mathrm{\Theta }ϵ_0\theta ^{1k}𝑑\theta .$$
(7)
For $`k<2`$ and $`\mathrm{\Theta }\theta _m`$, we get

$$E_{\mathrm{tot}}\frac{2\pi }{2k}b^{2k}ϵ(\theta _v)\theta _v^2,$$
(8)
where we have parameterized $`\mathrm{\Theta }=b\theta _v`$. We can see that the quantity $`2\pi ϵ(\theta _v)\theta _v^2`$ (which is essentially the $`E_\gamma `$ of F01, or $`E_0`$ of PK01 and P01) is quasi-invariant, if $`k`$ and $`E_{\mathrm{tot}}`$ are constant (or have a small scatter). The only extra scatter is introduced through the scatter of $`b`$, which is introduced by the scatter of $`\theta _v`$ (assuming the same $`\mathrm{\Theta }`$ for all GRBs). However, for the index $`(2k)`$ this scatter is greatly reduced if $`k`$ is not much smaller than 2. This is another reason why we require, say, $`k>1.5`$, in (2). A smaller $`\mathrm{\Theta }`$ can also reduce the $`b`$ scatter. Notice that the $`b`$ scatter tends to raise $`E_{\mathrm{tot}}`$ in GRBs with smaller $`\theta _v`$’s (and hence larger $`b`$’s), which seems to be helpful to reduce the $`E_0`$ scatter in PK01. An important implication of equation (8) satisfying such a constraint is that the total energy reservoir is standard, but the absolute value need no longer necessarily be several times $`10^{50}`$ ergs, but would depend on the value of $`k`$ and the typical value of $`b`$. Given reasonable values, $`E_{\mathrm{tot}}`$ could be one order of magnitude higher than that of F01 and PK01, but this could be still well accommodated within conventional central engine models (Mészáros , Rees & Wijers 1999). The closer $`k`$ approaches 2, the larger the standard energy reservoir one requires. At $`k=2`$, equation (8) should be modified in a form containing a logarithmic term, and the energy requirement is the highest (see discussions in Rossi, Lazzati & Rees 2001). Also the scatter of $`\theta _m`$ must be very small for $`k=2`$, while for $`k<2`$, the actual value of $`\theta _m`$ is not important. For $`k2`$, generally $`E_{\mathrm{tot}}`$ (eq.) can not be expressed in terms of $`ϵ(\theta _v)\theta _v^2`$, since most of the energy is distributed at small angles. The standard energy budget argument no longer holds333However, if $`\theta _m`$ is not too small (e.g. a not very small fraction of $`\theta _v`$), the case $`k\stackrel{>}{}2`$ could still retain the feature of a standard, finite (but even larger) energy reservoir.. A quasi-universal beaming configuration as well as a standard energy reservoir is however in general obtained if the requirement (2) is satisfied for an energy distribution such as (1).

For other forms of energy distributions, a standard energy reservoir is also attainable. For example, for the Gaussian distribution (6), one has $`E_{\mathrm{tot}}ϵ_0\theta _0^2`$.

## 3. Discussion

We have shown that an off-axis anisotropic jet with an energy distribution with angle given by equation (1) (or other forms such as ), is able to reproduce the key observational features of a conventional on-axis uniform jet model, e.g. such as producing a “jet break” signature in the light curve. The novelty here is that the achromatic break time $`t_b`$ in the broadband afterglow lightcurves no longer corresponds to the time when the relativistic beaming angle is equal to the jet half opening angle, $`\theta _j`$. Rather, it corresponds to the time when the relativistic beaming angle is roughly equal to the observer’s viewing angle $`\theta _v`$ relative to the jet axis. In this model, the broad distribution of $`t_b`$ in the data is no longer due to the intrinsic scatter of the jet opening angles among different bursts, but is attributed to the distribution of the observer’s lines of sight. For a power law energy distribution (1) (or a Gaussian energy distribution ), if the constraint (2) is satisfied, all the GRBs may have a quasi-universal beaming configuration, besides a quasi-standard energy reservoir. We deem this to be a more elegant picture than the conventional on-axis uniform jet model. In addition, the homogeneous nature of the conventional model is more idealized, and the present inhomogeneous model is likely to be a closer representation of what could be expected in nature.

The predictions of this inhomogeneous model for the afterglow lightcurves are not completely equivalent to those of the uniform jet model. The key difference should occur around the “jet break” time. Our model should give a more gradual variation at the break than the uniform jet model, which assumes a sharp drop off at the jet edge. The so far sparsely studied sideways expansion effect in an anisotropic jet may further complicate the problem. The shape of the break should also depend on the angular energy distribution function and some unknown parameters, such as $`k`$. Detailed modeling is necessary in order to address these questions. In any case, the gradual break expected in our model is not inconsistent with several well studied afterglow lightcurves, and some simulations have shown that the conventional jet models usually also give gradual and smooth jet breaks (e.g. Panaitescu & Mészáros , 1999; Moderski, Sikora & Bulik 2000; Huang et al. 2000). Both models are compatible with the present data, but this situation may change as better data becomes available and as more detailed simulations are performed.

Recently, Rossi et al. (2001) have independently discussed the power-law model (1) in more detail. They plotted the afterglow lightcurves for the $`k=2`$ case which mimic those of the on-axis uniform jet model, and also discussed the more general cases of $`k2`$. Here we have presented a general analytical argument, showing that the blastwave dynamics at the line of sight is identical to the uniform jet model in the asymptotic regime for a locus of models of the general form of equation (1), as long as $`k`$ is not too flat. With this particular form of the angular dependence of the energy, in order to have a standard, finite energy reservoir for all bursts one requires the constraint (2). The upper end $`k\stackrel{<}{}2`$ of the constraint (2) ensures that the total energy can be expressed in terms of $`ϵ(\theta _v)\theta _v^2`$ and does not diverge. (However, the case $`k\stackrel{>}{}2`$ could also have the same virtue if $`\theta _m`$ is not too small compared with $`\theta _v`$). The lower end of the constraint, $`k>1.5`$, ensures that the scatter introduced by $`\theta _v`$ is not too large, and that the energy-deficit effect at the other side of the jet is not too small. We have also found that the main features in the power-law model are also applicable to some other angular energy distributions, e.g. such as the Gaussian form (6).

For ease of discussion, we have here assumed that the upper limit of validity of the assumed angular distribution is $`\mathrm{\Theta }>2\theta _v`$. This is to avoid that the observer feels the energy deficit beyond $`\mathrm{\Theta }`$ before the relativistic beaming angle exceeds $`\theta _v`$. Indeed, if $`\mathrm{\Theta }<2\theta _v`$, $`\overline{\mathrm{\Gamma }}(\theta _v)`$ starts to deviate from the value predicted by the adiabatic law $`t^{3/8}`$ after it is less than $`(\mathrm{\Theta }\theta _v)^1`$. In this regime, the upper limits for $`\theta `$-integration in the numerators of both (3) and (4) should be replaced by $`\mathrm{\Theta }`$. Thus the maximum correction factor with respect to the $`\mathrm{\Theta }>2\theta _v`$ case is a factor of $`\mathrm{\Theta }/2\theta _v`$. Even for $`\mathrm{\Theta }=\theta _v`$ (i.e., the line of sight marginally cuts the jet edge), the deviation is at most a factor of 1/2. We therefore conclude that the $`\mathrm{\Theta }`$ effect may in most cases not be important. The main reason is that the large angles contribute a small portion of the total energy in the beam due to the distribution of the form (1).

In our model, the “isotropic” luminosity function will be determined by the assumed angular distribution, $`N(ϵ)dϵ=N(\theta )d\theta \mathrm{sin}\theta d\theta \theta d\theta `$ (the latter being for small $`\theta `$). From equations (1), (6) and substituting $`ϵ`$ by $`L`$, we get the luminosity function predictions in our model, e.g.,

$$N(L)dLL^{12/k}dL$$
(9)
for the power-law model, and

$$N(L)dLL^1dL$$
(10)
for the Gaussian model. To test these luminosity functions, redshift measurements are needed. Using only the bursts for which optical redshifts have been determined so far, e.g. as compiled in F01, PK01, the above luminosity distributions are not consistent. However, this discrepancy could be due to small number statistics ($`20`$ in all or $`10`$ on each side of the mid-point). The fact that the small sample size or other selection effects related to the afterglow detections could lead to a spurious inconsistency is also suggested, for example, by the clear deficit of low luminosity (e.g. possibly due to large viewing angle) bursts at higher redshifts ($`z>1`$) in F01’s data set. Alternatively, some other distance indicators have been proposed, such as spectral time-lags, e.g. Norris, Marani & Bonnell 2000, or variability measures, e.g. Fenimore & Ramirez-Ruiz 2002, Reichart et al. 2001 (and interpretations of these indicators in terms of the viewing angle have been discussed by, e.g., Salmonson & Galama 2002 and Norris et al 2002). If one accepts such distance indicators and their inferred redshifts at face value, the observational GRB luminosity function inferred for a much larger bursts sample (e.g. Schaefer, Deng & Band 2001) is not inconsistent with the theoretical distribution (9). And, using a different approach, Schmidt (2001) obtained a flatter observational GRB luminosity function, which over a large range of luminosities is compatible with the model distribution (10).

A natural consequence of this model is that the distribution of break times $`t_b`$, and hence the $`\theta _v`$ distribution, should be related to the statistical distribution of viewing angles and to the shape of the beam distribution. The present data and the preliminary calculations are not sufficient to draw firm constraints on parameters of such models. However, the comparison of such predictions or more detailed versions of them against future data in the Swift era should provide interesting constraints, as a larger quantity of more accurate redshift measurements become available.

We are grateful to M.J. Rees for discussions which stimulated this research, to E. Rossi for sending us their paper, and to B. Paczyński, J. Granot, E. Waxman, L. J. Gou for valuable comments or discussions. This work is supported by NASA (NAG5-9192 and NAG5-9153).


format: text

text:
# An Improved Bound on the VC-Dimension of Neural Networks with Polynomial Activation Functions

## 1 Introduction

We examine neural networks with polynomial activation functions. The specific architecture of the neural networks is described in detail in the next section. Such neural networks have been the subject of active investigation for several years, since powerful tools from algebraic geometry can be brought into play in analyzing the VC-dimension of such networks. Perhaps \[GJ93\] was the first paper to connect these two subjects. For several years (see for example \[KM97\]) it has been known that every bound on the number of connected components of a semi-algebraic set can be readily translated into a corresponding bound on the VC-dimension of a neural network architecture. Practically all of the known bounds on the VC-dimension of neural networks with polynomial activation bounds make use of a classical result discovered by Milnor, Oleinik, Petrovsky, and Thom \[OP49, Mil64, Tho65\].111Actually, this result bounds the sum of the Betti numbers of a semi-algebraic set, and this quantity is always at least as large as the number of connected components. In practice, one usually only needs an upper bound on the number of connected components. This bound, while easy to use, is usually much larger than necessary, since it only uses coarse information about the underlying set such as the number of variables and the maximum degree of the input polynomials. More recently, sharper bounds using more refined data from the input polynomials have been discovered. In the present note, we use a result due to Rojas \[Roj00\] that is particularly well-suited to neural networks with polynomial activation functions. The present bound is, in all cases, sharper than the earlier bound of Goldberg and Jerrum \[GJ93\]. Moreover, it is intuitively appealing, as the improvement can be quantified as the relative entropy of two probability vectors, whose dimension equals the number of layers in the neural network. This shows that the problem of bounding the VC-dimension of a neural network architecture continues to be interesting, and that we should strive to derive even tighter upper bounds.

Our main result is stated in theorem 4 of section 4. The recent semi-algebraic bound it is based on is stated as theorem 3 of section 3. However, let us first review a bit of background and some of the earlier bounds.

## 2 Known Results

The following definition of the VC-dimension is standard; see for example the books by Vapnik \[Vap95\] or Vidyasagar \[MV97\].

###### Definition 1

Suppose $`X`$ is a set and $``$ is a collection of $`\{0,1\}`$-valued functions on $`X`$. A set $`S=\{x_1,\mathrm{},x_n\}X`$ is said to be shattered by $``$ if each of the $`2^n`$ functions mapping $`S`$ into $`\{0,1\}`$ is the restriction to $`S`$ of some function in $``$. The Vapnik-Chervonenkis (VC)-dimension of $``$ is the largest integer $`n`$ such that there exists a set of cardinality $`n`$ that is shattered by $``$. $``$

By identifying a $`\{0,1\}`$-valued function with its support set, it is also possible to speak of the VC-dimension of a collection of sets. In the sequel, we shall use both notions interchangeably.

Following by now familiar approaches, we view a neural network as a verifier of formulas. Specifically, let $`𝐰^k`$ denote the “weight vector” or the vector adjustable parameters in a neural networks. A neural network with input space $`X^N`$ and weight vector $`𝐰`$ evaluates a logical proposition $`\varphi (𝐱,𝐰)`$ which is a Boolean combination of $`s`$ atomic expressions of the form $`\tau _i(𝐱,𝐰)=0`$ or $`\tau _j(𝐱,𝐰)>0`$. Letting $`1`$ (resp. $`0`$) denote “true” (resp. “false”), we can thus think of $`\varphi `$ as a function from $`^{k+N}`$ to $`\{0,1\}`$. So for each weight vector $`𝐰`$, define

$$A_𝐰:=\{𝐱^N:\varphi (𝐰,𝐱)=1\}.$$

The objective is to obtain an upper bound on the VC-dimension of the collection of sets $`𝒜:=\{A_𝐰|𝐰^k\}`$ or, equivalently, the VC-dimension of the collection of $`\{0,1\}`$-valued functions $`\mathrm{\Phi }:=\{\varphi (𝐰,)|𝐰^k\}`$.

To state the result, we need one final bit of notation: Let $`𝐱_1,\mathrm{},𝐱_v^N`$, and suppose $`svk`$. From the $`sv`$ polynomials $`\tau _j(,𝐱_i)`$ determined by all $`(i,j)\{1,\mathrm{},v\}\times \{1,\mathrm{},s\}`$, choose $`rk`$ polynomials, and label them $`\theta _1(),\mathrm{},\theta _r():^k`$. Define

$$𝐟(𝐰):=[\theta _1(𝐰)\mathrm{}\theta _r(𝐰)]^r.$$

Finally, let $`B`$ denote the maximum number of connected components of any pre-image $`f^1(y)`$ with $`y𝐑^r`$, for any choice of $`r`$ and $`\theta _1,\mathrm{},\theta _r`$ as above. With the above set-up, the following result is proved in \[KM97\]. For further background on our setting below see \[KM97\] or \[MV97\], p. 329.

###### Theorem 1

Following the notation above, assume further that restrict to those $`y𝐑^r`$ that are regular values of $`f`$. Then

$$\mathrm{VCDIM}(\mathrm{\Phi })2\mathrm{lg}B+2k\mathrm{lg}(2es).$$

That $`B`$ is in fact finite and admits an explicit upper bound is obtained by appealing to the aforementioned classical result of Milnor, Oleinik, Petrovsky, and Thom \[OP49, Mil64, Tho65\], which we now state as follows:

###### Lemma 1

Suppose $`\theta _1,\mathrm{},\theta _r`$ are polynomials in $`k`$ variables, with degree no larger than $`d`$. Then whenever $`𝐲`$ is a regular value of $`𝐟`$ as defined above, the preimage $`𝐟^1(𝐲)`$ contains no more than $`d(2d)^{k1}`$ connected components.

Note that Milnor actually proves the theorem in the case where $`𝐲=\mathrm{𝟎}`$, but we can clearly perturb the constant terms of the $`\theta _i`$ to enforce this assumption. If we replace the quantity $`d(2d)^{k1}`$ by the larger number $`(2d)^k`$ and substitute $`B=(2d)^k`$ into the upper bound (1), we get the following result.

###### Theorem 2

Suppose $`\varphi `$ is a Boolean formula involving a total of $`s`$ polynomial equalities and inequalities, where each polynomial has degree no larger than $`d`$ with respect to $`𝐰`$. Then $`\mathrm{VCDIM}(\mathrm{\Phi })2k\mathrm{lg}(4eds)`$.

The above result is the same as that derived in \[GJ93\]. It should be noted, however, that Goldberg and Jerrum actually consider neural networks with piecewise polynomial activation functions. With more elaborate notation, their results can be derived as special cases of Theorem 1.

Theorem 1 shows the importance of deriving tight upper bounds on the number of connected components of a semi-algebraic set. This is a long-standing problem in real algebraic geometry that has received considerable attention from the research community. It is obvious from the bound (1) that any improvement over Milnor’s upper bound translates directly into a corresponding improvement in the estimate of the VC-dimension of a neural network architecture with polynomial activation functions. This leads us to the next topic.

## 3 Improved Upper Bound on the Number of Connected Components

In \[Roj00\], an improvement is provided over Milnor’s bound. To state this improved result, a bit of notation is introduced.

Let $`\mathrm{\Delta }_n`$ denote the standard $`n`$-simplex in $`𝐑^n`$, with vertices the standard basis vectors and the origin. Note that

$$d\mathrm{\Delta }_n:=\left\{(x_1,\mathrm{},x_n)^n\right|x_i0\text{ for all }i\text{ and }\underset{i=1}{\overset{n}{}}x_id\}.$$

Let $`\mathrm{Vol}_n()`$ denote the renormalization of the usual volume in $`𝐑^n`$ satisfying $`\mathrm{Vol}_n(\mathrm{\Delta }_n)=1`$. (Since the usual $`n`$-dimensional volume is multiplicative for orthogonal subspaces, it is easy to prove by induction that $`\mathrm{Vol}_n`$ is just $`n!`$ times the usual $`n`$-dimensional volume.)

###### Theorem 3

Suppose $`\tau _1,\mathrm{},\tau _r`$ are polynomials in $`(w_1,\mathrm{},w_k)`$, and let $`𝐞_1,\mathrm{},𝐞_k`$ and $`𝐎`$ denote the standard basis vectors and the origin of $`^k`$. Also, let $`V`$ denote the convex hull of the union of $`\{𝐎,e_1,\mathrm{},e_k\}`$ with the set of all $`(i_1,\mathrm{},i_k)`$ such that $`w_1^{i_1}\mathrm{}w_k^{i_k}`$ is a monomial of some $`\theta _j()`$. Then

$$B2^k\mathrm{Vol}_k(V).$$

In the special case where every $`k`$-tuple with $`_{j=1}^ki_jd`$ occurs in $`V`$, we recover the (adjusted) Milnor bound $`(2d)^k`$. However, the whole point of the preceding refined bound is that there are many instances where the input polynomial are far more sparse, and this can be exploited.

## 4 Improved Upper Bound on the VC-Dimension

In this section, we derive an improved upper bound on the VC-dimension of neural networks with polynomial activation functions. The improved bound is a direct consequence of coupling Theorems 1 and 3.

Let us begin by describing the class of neural networks under study. It is assumed that the network has $`N`$ real inputs denoted by $`x_1,\mathrm{},x_N`$. There are $`l`$ levels in the network, and at level $`i`$ there are $`q_i`$ output neurons; however, at the output layer (level $`l`$) there is only a single neuron (see below). Let $`k_i`$ denote the number of adjustable parameters, or “weights,” at level $`i`$, and let $`k=_{i=1}^lk_i`$ denote the total number of adjustable parameters. Let $`𝐰_i:=(w_{i,1},\mathrm{},w_{i,k_i})`$ denote the weight vector at level $`i`$, and $`𝐰=(𝐰_1\mathrm{}𝐰_l)`$ denote the total weight vector. The input-output relationship of each neuron at level $`i`$ is of the form

$$y_{i,j}=\tau _{i,j}(𝐰_i,y_{i1,1},\mathrm{},y_{i1,q_{i1}}),j=1,\mathrm{},q_i.$$

where $`y_{i,j}`$ is the output of neuron $`j`$ at level $`i`$, and $`\tau _{i,j}`$ is a polynomial of degree no larger than $`\alpha _i`$ in the components of the weight vector $`𝐰_i`$, and no larger $`\beta _i`$ in the components of the vectors $`y_{i1,j}`$. At the final layer, there is a simple perceptron device following the polynomial activation function.

With this class of neural networks, it is clear that the output will equal one if and only if a polynomial inequality of the form

$$y_l(𝐰,𝐱)0,$$

is satisfied, where $`𝐰`$ is the weight vector and $`𝐱=(x_1\mathrm{}x_N)`$ is the input vector. Thus we can apply Theorem 1 with $`s=1`$. The issue now is to determine the number of connected components $`B`$ of the semi-algebraic set defined by $`y_l(𝐰,𝐱)=𝐲`$.

Now we are in a position to state the main result. To facilitate the statement, we introduce a bit of notation. Define

$$d_l=\alpha _l,d_{l1}=\alpha _{l1}\beta _l,\mathrm{},d_i=\alpha _i\underset{j=i+1}{\overset{l}{}}\beta _j,i=1,\mathrm{},l1.$$

Recall that $`k_i`$ denotes the number of adjustable parameters at level $`i`$, and that $`k`$ denotes the total number of adjustable parameters. Define the probability vectors

$$𝐯:=(k_1/k\mathrm{}k_l/k),𝐮:=(d_1/d\mathrm{},d_l/d),$$

and define the “binary” relative entropy $`H(𝐯|𝐮)`$ as

$$H(𝐯|𝐮):=\underset{i=1}{\overset{l}{}}v_i\mathrm{lg}\left(\frac{v_i}{u_i}\right)=\frac{1}{k}\underset{i=1}{\overset{l}{}}k_i\mathrm{lg}\left(\frac{dk_i}{kd_i}\right).$$

Note that the above is the same as the conventional relative entropy of two probability vectors, except that we use base-$`2`$ logarithms instead of natural logarithms. Following standard convention, we take $`0\mathrm{lg}(0/0)=0`$.

###### Theorem 4

With the above notation, we have

$`B`$ $``$ $`2^kk!{\displaystyle \underset{i=1}{\overset{l}{}}}{\displaystyle \frac{d_i^{k_i}}{k_i!}}.`$ (4.1)
$``$ $`\left({\displaystyle \frac{2d}{e^{7/8}}}\right)^k2^{kH(𝐯|𝐮)}.`$ (4.2)
where $`d:=_{i=1}^ld_i`$ and we assume $`k_1,\mathrm{},k_l2`$ in the last inequality. Consequently, when $`k_1,\mathrm{},k_n2`$, the VC-dimension of the neural network architecture is bounded above by

$$2k(\mathrm{lg}(4ed)H(𝐯|𝐮)).$$

Remark The above theorem shows that the reduction in the VC-dimension estimate over that of Theorem 2 is precisely $`2k`$ times the (binary) relative entropy of the two probability vectors $`v`$ and $`u`$ defined above. Thus if $`k_i/k=d_i/d`$ for all $`i`$, there will not be any reduction at all. In general, the fraction by which the older VC-dimension estimate is reduced is precisely the ratio $`H(𝐯|𝐮)/(\mathrm{lg}(4ed))`$. Note also that the assumption that there are at least $`2`$ adjustable parameters at each levels is a reasonably mild assumption. $``$

Proof of Theorem 4: The proof depends on a careful book-keeping of the degree of $`y_l(𝐰,𝐱)`$ with respect to the various components of $`𝐰`$. From the architecture of the neural network, it is clear that at the first level, each of the $`y_{1,j}`$ is a polynomial in the components of $`𝐰_1`$ of degree no larger than $`\alpha _1`$. At the second level, each of the $`y_{2,j}`$ is a polynomial, whose monomials are of (combined) degree no larger than $`\alpha _2`$ in the components of $`𝐰_2`$, and of (combined) degree no larger than $`\beta _2\alpha _1`$ in the components of $`𝐰_1`$. Thus, while each $`y_{2,j}`$ could have a total degree of $`\alpha _2+\beta _2\alpha _1`$ in the components of $`𝐰_1`$ and $`𝐰_2`$, the total degree of the monomial terms involving the components of $`𝐰_1`$ does not exceed $`\beta _2\alpha _1`$, while the total degree of the monomial terms involving the components of $`𝐰_2`$ does not exceed $`\alpha _2`$. A simple argument by induction then tells us that at the output layer (level $`l`$), the single output $`y_l`$ is a polynomial whose monomials have total degree no larger than $`d_l=\alpha _l`$ in the components of $`𝐰_l`$, no larger than $`d_{l1}=\beta _l\alpha _{l1}`$ in the components of $`𝐰_{l1}`$, and so on. With the $`d_i`$’s defined as above, the components of each $`𝐰_i`$ appear with total degree no larger than $`d_i`$. Thus the total degree of $`y`$ could be as large as $`d`$, but the monomial terms involving the components of $`𝐰_i`$ have total degree no larger than $`d_i`$. So the set $`V`$ defined in Theorem 3 satisfies the following containment:

$$V\underset{i=1}{\overset{l}{}}S_{d_i}^{k_i}.$$

Because of this containment, it follows that

$$\mathrm{Vol}_n(V)k!\underset{i=1}{\overset{l}{}}\frac{d_i^{k_i}}{k_i!}.$$

Combining this with the bound (3) establishes the first estimate (4.1).

To prove the second estimate, we use Stirling’s approximation. In particular, \[Rud76, ex. 20, pg. 200\] tells us that for all $`t\{2,3,4,\mathrm{}\}`$, we have

$$e^{7/8}(t/e)^t\sqrt{t}<t!<e(t/e)^t\sqrt{t}.$$

Consequently, we easily obtain

$$\frac{k!}{k_1!\mathrm{}k_l!}<e^{1\frac{7}{8}k}\frac{k^k}{k_1^{k_1}\mathrm{}k_l^{k_l}\sqrt{k_1\mathrm{}k_l}}.$$

Dropping the square root term on the bottom can of course be done, and then an elementary calculation yields $`2^kk!_{i=1}^l\frac{d_i^{k_i}}{k_i!}\left(\frac{2d}{e^{7/8}}\right)^k2^{kH(𝐯|𝐮)}`$, provided $`k_1,\mathrm{},k_n2`$.

The VC-dimension estimate (4) now follows readily from Theorem 1. $`\mathrm{}`$

## 5 Numerical Example

Consider a network with four inputs, five hidden-layer neurons at the first level and an output-layer neuron. As is common, let us suppose that $`\alpha _i=1`$ for all $`i`$. This means that all the adjustable parameters enter linearly into the corresponding activation function. Suppose $`\beta _1=2,\beta _2=3`$. This means that the hidden-layer neurons have quadratic activation functions, whereas the output-layer neuron has a cubic activation function. It remains to specify the integers $`k_1`$ and $`k_2`$, representing the number of adjustable parameters. Let us assume that practically all of the monomial terms are present in each neural characteristic. Thus it is reasonable to assume $`k_1=50,k_2=20`$. Finally, $`d_1=3,d+2=1`$. With these figures, one has

$$𝐯=(5/7,2/7),𝐮=(0.25,0.75),$$

$$H(𝐯|𝐮)0.684033,\mathrm{lg}(4ed)5.4427,\frac{H(𝐯|𝐮)}{\mathrm{lg}(4ed)}0.12567.$$

Thus, in this case, the improved bound is roughly 12.5% sharper.

## 6 Conclusions


text:
# On the Graviton Mass

## 1 What’s wrong with a non-zero graviton mass?

Zakharov (1970) and van Dam & Veltman (1970) (ZvDV) suggested that observations of the solar system tell us that the graviton mass must be rigorously zero. One would usually expect that observations can give just an upper limit, but in the linearized gravity the ZvDV theorem is indeed correct as we explain below using a non-covariant linearized massive gravity. In §2 a covariant version of the ZvDV theorem is given in the quasi-massive (induced) gravity of Dvali, Gabadadze & Porrati (2000).

Consider linearized massive gravity on the flat background: the metric tensor is $`g_{ik}+h_{ik}`$, $`g_{ik}=\mathrm{diag}(1,1,1,1)`$, $`|h_{ik}|1`$. The Lagrangian should be of the form

$$=_E\frac{1}{4}m^2(h_{ik}h^{ik}+ah^2).$$
(1)
Here $`_E`$ is the quadratic part of the Einstein Lagrangian and $`h=h_i^i`$. Energy is positive if and only if $`a=1`$, which is the Pauli-Fierz mass term. But then, the graviton propagator is

$$P_{iknm}=\frac{\frac{1}{2}g_{in}g_{km}+\frac{1}{2}g_{im}g_{kn}\frac{1}{3}g_{ik}g_{nm}}{k^2m^2},$$
(2)
while the standard massless propagator is

$$P_{iknm}=\frac{\frac{1}{2}g_{in}g_{km}+\frac{1}{2}g_{im}g_{kn}\frac{1}{2}g_{ik}g_{nm}}{k^2}.$$
(3)
The two propagators are different by order one even in the limit $`m=+0`$ — the ZvDV theorem.

In the solar system, gravity is weak, and it appears that the use of the linearized approximation is justified. But then observations (light deviation and Mercury perihelion advance) rule out massive gravity. The graviton mass should be rigorously zero. That observations can measure the graviton mass to an absolute accuracy is strange, and indeed Vainshtein (1972) showed that non-linear corrections restore the continuity in the graviton mass.

Both the ZvDV theorem, and its non-perturbative refutation are based on a non-covariant theory. The physical interpretation of these results in the GR framework is unclear. One could argue that the absence of a covariant massive gravity theory indicated that $`m=0`$. Also, Vainshtein (1972) had to calculate non-linear corrections using a partly linearized action – a questionable procedure. In our view, the problem of the graviton mass remained open.

Recently, Dvali, Gabadadze & Porrati (2000) introduced the first covariant theory of (quasi) massive gravity, and showed that the ZvDV theorem holds in their theory too (§2). Nevertheless, in §3 we show that this massive gravity theory is consistent with the solar system observations if the graviton is light enough. We show that corrections to the Schwarzschild metric of the Sun vanish for very light gravity (a similar result holds for a string, Lue (2001)). Interestingly, the resulting limit on the graviton mass does not rule out possible cosmological effects of this massive gravity (Deffayet et al, 2001).

## 2 ZvDV for Schwarzschild metric in induced gravity

The action of induced gravity is (Dvali, Gabadadze & Porrati 2000):

$$d^4x\sqrt{g}RL^1d^5x\sqrt{\mathrm{g}}.$$
(4)
Here the first term is the Einstein action, $`L`$ is a constant with dimension of length, the second term is a 5D Einstein action, and $`g_{ik}=\mathrm{g}_{ik}`$ at $`x^4=0`$, $`i,k=0\mathrm{}3`$. This action is covariant, that is covariant in the 4D space-time $`x^4=0`$, and this is all we need. For our purposes, reality of the fifth dimension is irrelevant. We will be interested only in the resulting 4D geometry of $`x^4=0`$. The last term is then analogous to the mass-term, with $`m^2(Lr)^1`$.

The empty-space Einstein equation becomes

$$R^{ik}\frac{1}{2}Rg^{ik}+\frac{2\sqrt{\mathrm{g}}}{L\sqrt{g}}\{^{ik}\frac{1}{2}\mathrm{g}^{ik}\}=0,$$
(5)
where the last term should be understood as follows. The Jacobian $`\sqrt{\mathrm{g}}`$ is calculated at $`x^4=0`$. The Einstein tensor $`^{ik}\frac{1}{2}\mathrm{g}^{ik}`$ is calculated at $`x^4=+0`$, and the operation $`\{\mathrm{}\}`$ leaves only the $`\delta `$-function part of the 5D piece. This operation is defined as follows: $`\{a_4_4A+b_4B+C\}=a_4A`$, where expressions $`a`$, $`A`$, $`b`$, $`B`$, and $`C`$ do not contain derivatives over $`x^4`$. For $`x^4>0`$,

$$_{IK}=0,$$
(6)
where $`I,K=0\mathrm{}4`$. $`\mathrm{g}_{IK}`$ are symmetric functions of $`x^4`$.

Consider modifications of the Schwarzschild metric in induced gravity. Take the line element of the form

$$ds^2=\mathrm{e}^\nu dt^2\mathrm{e}^\lambda dr^2\mathrm{e}^\mu (\mathrm{sin}^2\chi d\theta ^2+\mathrm{sin}^2\chi \mathrm{sin}^2\theta d\varphi ^2+d\chi ^2),$$
(7)
with $`x^4=r\mathrm{cos}\chi `$. The three unknowns $`\nu `$, $`\lambda `$, and $`\mu `$ are functions of $`r`$ and $`\chi `$.

The nontrivial Ricci components are

$$2_0^0=\mathrm{e}^\lambda \left(\nu ^{\prime \prime }+\frac{1}{2}\nu ^{}(\nu ^{}\lambda ^{}+3\mu ^{})\right)+\mathrm{e}^\mu \left(\ddot{\nu }+\frac{1}{2}\dot{\nu }(\dot{\nu }+\dot{\lambda }+\dot{\mu }+4\mathrm{cot}\chi )\right),$$
(8)
$$2_1^1=\mathrm{e}^\lambda \left(\nu ^{\prime \prime }+3\mu ^{\prime \prime }+\frac{1}{2}\nu ^{}(\nu ^{}\lambda ^{})+\frac{3}{2}\mu ^{}(\mu ^{}\lambda ^{})\right)+\mathrm{e}^\mu \left(\ddot{\lambda }+\frac{1}{2}\dot{\lambda }(\dot{\nu }+\dot{\lambda }+\dot{\mu }+4\mathrm{cot}\chi )\right),$$
(9)
$$2_2^2=2_3^3=\mathrm{e}^\lambda \left(\mu ^{\prime \prime }+\frac{1}{2}\mu ^{}(\nu ^{}\lambda ^{}+3\mu ^{})\right)+\mathrm{e}^\mu \left(\ddot{\mu }+\frac{1}{2}\dot{\mu }(\dot{\nu }+\dot{\lambda }+\dot{\mu })+\mathrm{cot}\chi (\dot{\nu }+\dot{\lambda }+3\dot{\mu })4\right),$$
(10)
$$2_4^4=\mathrm{e}^\lambda \left(\mu ^{\prime \prime }+\frac{1}{2}\mu ^{}(\nu ^{}\lambda ^{}+3\mu ^{})\right)+\mathrm{e}^\mu \left(2\ddot{\mu }+\ddot{\nu }+\ddot{\lambda }+\frac{1}{2}\dot{\nu }(\dot{\nu }\dot{\mu })+\frac{1}{2}\dot{\lambda }(\dot{\lambda }\dot{\mu })+2\mathrm{cot}\chi \dot{\mu }4\right),$$
(11)
$$2_{14}=\dot{\nu }^{}2\dot{\mu }^{}+\dot{\lambda }(\frac{1}{2}\nu ^{}+\mu ^{})+\frac{1}{2}\dot{\nu }(\mu ^{}\nu ^{}),$$
(12)
where prime and dot are derivatives over $`r`$ and $`\chi `$.

The 4D Ricci tensor is

$$2R_0^0=\mathrm{e}^\lambda \left(\nu ^{\prime \prime }+\frac{1}{2}\nu ^{}(\nu ^{}\lambda ^{}+2\mu ^{})\right),$$
(13)
$$2R_1^1=\mathrm{e}^\lambda \left(\nu ^{\prime \prime }+2\mu ^{\prime \prime }+\frac{1}{2}\nu ^{}(\nu ^{}\lambda ^{})+\mu ^{}(\mu ^{}\lambda ^{})\right),$$
(14)
$$2R_2^2=2R_3^3=\mathrm{e}^\lambda \left(\mu ^{\prime \prime }+\frac{1}{2}\mu ^{}(\nu ^{}\lambda ^{}+2\mu ^{})\right)2\mathrm{e}^\mu .$$
(15)
In the linear approximation, we take $`\mu 2\mathrm{ln}r+\mu `$ and linearize in $`\nu `$, $`\lambda `$, and $`\mu `$. Then equations (6) are solved by

$$_{(5)}^2\nu \nu ^{\prime \prime }+\frac{3}{r}\nu ^{}+\frac{1}{r^2}(\ddot{\nu }+2\mathrm{cot}\chi \dot{\nu })=0,$$
(16)
$$\mu =a+b\mathrm{cos}\chi \frac{\nu }{2},$$
(17)
$$\lambda =ra^{}+a+rb^{}\mathrm{cos}\chi \frac{\nu }{2},$$
(18)
where $`a(r)`$ and $`b(r)`$ are arbitrary functions of $`r`$.

Write the linearized equations (5) in the form

$$2R_0^0+\frac{2r}{L}\{2_0^0+_4^4\}=0,$$
(19)
and similar equations for 11 and 22 components. We get, at $`\chi =\pi /2`$,

$$\nu ^{\prime \prime }+\frac{2}{r}\nu ^{}\frac{3}{2Lr}\dot{\nu }+\frac{1}{L}(b^{}+\frac{2b}{r})=0,$$
(20)
$$\frac{1}{r}\nu ^{}\frac{3}{2Lr}\dot{\nu }\frac{1}{L}(3b^{}+\frac{2b}{r})=0,$$
(21)
$$\nu ^{\prime \prime }+\frac{1}{r}\nu ^{}\frac{3}{Lr}\dot{\nu }\frac{2}{L}(b^{}+\frac{4b}{r})=0.$$
(22)
These equations are solved by

$$\nu ^{\prime \prime }+\frac{2}{r}\nu ^{}\frac{2}{Lr}\dot{\nu }=0,$$
(23)
$$b=\frac{L}{4}\nu ^{}.$$
(24)
Now consider the small graviton mass limit, which corresponds to large $`L`$ in induced gravity. Equations (16), (23) give, for $`\chi \pi /2`$:

$$\nu =\frac{2\chi }{\pi \mathrm{sin}\chi }\frac{r_g}{r},$$
(25)
where $`r_g`$ is a constant. Then (24) gives $`b=\frac{Lr_g}{4r^2}`$. Taking $`a=\frac{r_g}{2r}`$, so as to make $`\mu =0`$ in the 4D space-time, we get $`\lambda =\frac{r_g}{2r}`$ in 4D. This is to be compared with the linearized Schwarzschild’s $`\lambda =\frac{r_g}{r}`$. Thus ZvDV holds in induced gravity.

But: is the linearization justified? At the very least, all metric components should be much less than unity. This is violated by the linearized solution. Indeed, for large $`L`$, the full 5D solution (given by (17), (18), (24)) is, for $`\chi \pi /2`$:

$$\lambda =\frac{\chi }{\pi \mathrm{sin}\chi }\frac{r_g}{r}+\mathrm{cos}\chi \frac{Lr_g}{2r^2},$$
(26)
$$\mu =\left(\frac{\chi }{\pi \mathrm{sin}\chi }\frac{1}{2}\right)\frac{r_g}{r}\mathrm{cos}\chi \frac{Lr_g}{4r^2}.$$
(27)
In 5D, $`\mu `$ and $`\lambda `$ are large for large $`L`$. Even though we are only interested in 4D, the linearized theory is not applicable at large $`L`$. What happens at large L is explained in §3. It turns out that the linear theory works only for $`b^2a`$, that is for $`L^2r_gr^3`$.

## 3 Schwarzschild solution at large L

To analyze the large $`L`$ limit of the weak ($`rr_g`$) gravity, the full non-linear theory is not needed. It is sufficient to check that the following substitution

$$\mu =2\mathrm{ln}r+b\mathrm{cos}\chi +\frac{1}{8}b^2\mathrm{cos}2\chi +a\frac{\nu }{2},$$
(28)
$$\lambda =rb^{}\mathrm{cos}\chi +\frac{1}{8}rb^{}(2brb^{})\mathrm{cos}2\chi +ra^{}+a+\frac{1}{8}(b^2r^2b^2)\frac{\nu }{2},$$
(29)
where $`a`$, $`b`$ are arbitrary functions of $`r`$, and $`\nu `$ satisfies

$$_{(5)}^2\nu =0,$$
(30)
solves $`_{IK}=0`$ up to $`O(b^2)=O(a)=O(\nu )`$.

Now choosing $`\nu `$ given by (25), and $`b=\sqrt{2r_g/r}`$, $`a=r_g/(4r)`$, we recover the weak-field Schwarzchild in the 4D space-time. And thus we also solve the $`L=\mathrm{}`$ limit of equations (5).

The weak-field Schwarzschild with matter is also recovered in the large $`L`$ limit of induced gravity, if the proper energy density $`ϵ`$ is positive. The weak-field Schwarzschild with matter is $`\mu =2\mathrm{ln}r`$, $`r\nu ^{}=\lambda `$, $`r\lambda =𝑑rr^2ϵ`$. In induced gravity we choose $`a`$ so as to give $`\mu =2\mathrm{ln}r`$. Then $`\lambda =r\nu ^{}/2+b^2/4`$. Thus, the $`L=\mathrm{}`$ limit of induced gravity recovers the weak-field Schwarzschild with matter if $`\lambda r\nu ^{}/2`$, or $`\lambda 0`$, which is true for $`ϵ0`$.

The finite $`L`$ corrections to the Schwarzschild solution are given by (5) (that is by (20)):

$$\frac{\delta \nu }{\nu }=\left(\frac{8r^3}{L^2r_g}\right)^{1/2}.$$
(31)
For the solar system, assuming that fractional corrections at about $`5`$AU are $`10^8`$ (Talmadge et al 1988), we get $`L100`$Mpc. But gravity can be massive.

I thank my NYU colleagues: Gia Dvali for discussions that initiated this study, Cedric Deffayet, and Arthur Lue for useful discussions, and Massimo Porrati for advice on how to derive the Pauli-Fierz mass term (use rest frame).


text:
# Herd Formation and Information Transmission in a Population: Non-universal behaviour

## I Introduction

Herd behaviour and information transmission among agents are obviously important in financial markets. With the newly established area of econophysics, various multi-agent models of financial markets have been studied. In agent-based models, markets are typically modelled by a population of competing agents. Through information transmission and opinion sharing, these agents may not act independently. The collective behaviour of clusters of agents, also referred to as crowds, in which there is efficient information sharing among the agents is an important factor in both real and simulated markets. These crowds are dynamic in nature in that there is a continual process of crowd formation and dissociation.

Recently, Eguíluz and Zimmermann proposed and studied a simple model for stochastic opinion cluster formation and information dispersal. It is a dynamical model (henceforth referred to as the EZ model) in which there is a continual grouping and re-grouping of agents to form clusters. A cluster or crowd of agents act together (either buying or selling) and then dissolve after the transaction has occurred. When a cluster of agents decides not to trade, i.e., an inactive state, it may combine with another cluster of agents to form a bigger cluster. Detailed numerical studies and mean field analysis revealed several interesting features of the model. For example, it was observed that the model could lead to a fat-tail distribution of price returns qualitatively similar to that observed in real markets. In addition, the cluster size distribution $`n_s`$ shows a scaling behaviour of the form $`n_ss^{5/2}`$ for a range of cluster size $`s`$, followed by an exponential cutoff. The EZ model represents a dynamical version of the static percolation-type model of Cont and Bouchaud in which herd formation is described by random connection between agents, and the cluster size distribution is found to follow the same scaling behaviour and with the same value of $`5/2`$ for the exponent. Several variations on the model have been proposed and studied. These variations include the spreading of opinion to multiple clusters  and inhomogeneous dissociation of clusters. Interestingly, it was found that the values of the exponent characterizing the cluster size distribution seem to be robust against these variations, i.e., these values remain unchanged for the different variations proposed so far. We note that similar scaling behavior has been found in the size distribution of businesses, although the value of the exponent is different.

Here we introduce two generalized versions of the EZ model. One version has a size-dependent probability for the dissociation of clusters after opinion sharing. The distributions of cluster sizes and price returns are found to be characterized by model-dependent exponents. Another version has a size-dependent transaction rate. It leads to a price returns distribution with a model-dependent exponent, while keeping the exponent for the cluster-size distribution to be $`5/2`$. Both models can be treated analytically via a mean field approach. The mean field results are found to be in good agreement with numerical results. The EZ model is re-covered as a special case of our generalized models.

The plan of the paper is as follows. We define our generalized model with size-dependent cluster dissociation probability in Sec. II and present the numerical results of the model. Section III provides a mean field analysis. The analytic results are compared with numerical simulations. In Sec. IV, we define another generalized model with a size-dependent transaction rate and present numerical results. The results are summarized in Sec. V.

## II Model I: Size Department Cluster Dissociation Rate

We consider a model with a total of $`N`$ agents. Following Ref., a cluster or crowd is a group of agents who can exchange information efficiently. These agents make the same decision at a given moment in time. Initially, all the agents are isolated. As time evolves, an agent belongs to a cluster of a certain size. At each timestep an agent, say the $`i`$-th one, is chosen at random. Let $`s_i`$ be the size of the cluster to which the agent belongs. Since the agents within a cluster have a common opinion, all agents in such a cluster tend to imitate each other and hence act together. With probability $`a`$ the agent, and hence the whole cluster, decides to make a transaction, e.g. to buy or to sell with equal probability. After the transaction, the cluster is then broken up into isolated agents with a probability $`s_i^\delta `$, with $`0\delta <1`$. With probability $`(1a)`$ the agents decide not to make a transaction, i.e., they wait and try to gather more information. The other agents in the cluster follow. In this case, another agent $`j`$ is chosen at random. The two clusters of sizes $`s_i`$ and $`s_j`$ then either combine to form a bigger cluster with probability $`s_i^\delta s_j^\delta `$, or the two clusters remain separate with probability $`(1s_i^\delta s_j^\delta )`$. Here $`a`$ can be treated as a parameter reflecting the investment rate showing how frequent a transaction is made. Our model thus represents a generalization of the basic EZ model to the case in which a cluster of agents may stay together to form a group after making a transaction. The probability of dissociation $`s_i^\delta `$ implies that larger clusters have a larger tendency to remain grouped while smaller clusters are easier to break up. For the special case of $`\delta =0`$, our model reduces to the EZ model.

In the EZ model ($`\delta =0`$), clusters of agents break up after a transaction. Here, our model includes a dissociation probability depending on the cluster size - this feature may be invoked to mimic practical aspects of a financial market, such as the effect of news arrival. Imagine one of the agents in a cluster of size $`s_i`$ receives some external news with probability $`a`$ at a given timestep. This external news suggests that he, and hence the other members of his cluster, should immediately trade (buy or sell). Since the news is external, the crowd act together in this one moment, leaving the cluster with a finite probability of subsequently dissociating. Suppose that they sense, e.g. from the resulting price-movement, that they are a member of a large crowd of like-minded agents: in practice many traders like to feel part of a larger crowd for reassurance. We therefore assume that the crowd breaks up with a size-dependent probability $`p(s_i)`$, where $`p(s_i)=1`$ for $`s_i=1`$ and $`p(s_i)`$ decreases monotonically as $`s_i`$ increases. By contrast, with probability ($`1a`$) there is no news arrival from outside. The agent in the chosen cluster, uncertain about whether to buy or sell, makes contact with an agent in another cluster of size $`s_j`$. The agents share information and come up with a new opinion. Each of them then separately tries to persuade the other members of his cluster of the new opinion. With probability $`p(s_i)`$ ($`p(s_j)`$) the opinion of cluster $`i`$ ($`j`$) changes to the new opinion. Thus, the two clusters combine with probability $`p(s_i)p(s_j)`$. It turns out that this particular form of the two combined modifications to the EZ model, can be treated analytically using our mean field analysis. As a specific example, our numerical simulations are carried out for the case in which $`p(s)s^\delta `$.

Let $`n_s`$ be the number of clusters of size $`s`$. Figure 1 shows the results of numerical simulations on the cluster size distribution in the steady state, for various values of the parameter $`\delta `$. The results are obtained for a system with $`N=10^4`$ agents and $`a=0.3`$ . Averages are taken over a time window of $`10^6`$ time steps after the transient behavior has disappeared, together with a configuration average over 100 different runs with different initial conditions. The $`\delta =0`$ results give the features in the EZ model. For a range of $`s`$, $`n_ss^\beta `$ with $`\beta =5/2`$ . Deviation from the scaling behavior sets in at a value of $`s`$ depending on the value of the parameter $`a`$. For smaller values of $`a`$, the scaling region enlarges. These features are consistent with previous numerical and analytical studies. For $`0\delta <1`$, it is observed that the size distribution $`n_s`$ still scales with $`s`$ in a range of $`s`$ as in the EZ model. However, the exponent becomes model dependent and hence non-universal. The data shows that the exponent $`\delta `$ is consistent with the behavior $`n_ss^{\beta (\delta )}`$, where $`\beta (\delta )=5/2\delta `$. A mean field analysis can be used to extract this scaling behavior, as will be described in the next section.

It is interesting to note that several attempts have been made to modify the EZ model. These extensions include, for example, the study by d’Hulst and Rodgers on democracy versus dictatorship by incorporating an inhomogeneous investment rate in the population and also allowing rumor to spread to multiple clusters in one time step after a chosen cluster decides not to make a transaction. All the extensions proposed so far give $`\beta =5/2`$, hence the value seems to be robust. The present model incorporates a size-dependent dissociation probability of a cluster after a transaction and leads to a tunable and model-dependent $`\beta (\delta )`$. Thus our model actually gives a set of models with different values of $`\beta `$, similar to the case of changing a system from one universality class to another in problems in critical phenomena. In fact, the situation is reminiscent of the non-universal exponent of conductivity in continuum percolation. In percolation problems, it is known that the effective conductivity for a system consisting of insulators and conductors exhibits the scaling behavior $`\sigma _e(pp_c)^t`$ near the percolation threshold $`p_c`$. The exponent $`t`$ is universal in that its value depends only on the dimension of the system, regardless of other details, e.g. lattice type. However, if the conductances $`\sigma _e`$ of the conductors follow a distribution of the form $`P(\sigma _e)\sigma _e^\delta `$ with $`0<\delta <1`$, the $`t`$-exponent and other related properties become non-universal with exponents taking on a value depending on $`\delta `$. It should be noted that it is not so surprising to see a connection between percolation and model for herd behavior. In the model of Cont and Bouchaud, the EZ model and their variations, an agent could be connected to any one of $`(N1)`$ other agents to form a cluster. These models hence represent a problem of connectivity in high dimensions. Several other percolation type models have also been proposed to explain the phenomena observed in real markets. It is also interesting to note that value of power law exponents in statistical physics could also be made non-universal by introducing long-range interactions. By making the larger clusters harder to dissociate, it can be thought of as effectively introducing some long range interaction among agents in time.

Eguíluz and Zimmermann applied their model to study the distribution of price returns. A price can be generated according to

$$P(t+1)=P(t)\mathrm{exp}(s^{}/\lambda ),$$
(1)
where $`\lambda `$ is a parameter for the liquidity of the market. The price return $`R(t)=\mathrm{ln}P(t)\mathrm{ln}P(t1)`$ is defined to be the relative number of agents buying or selling at a time with $`s^{}=s`$ for a cluster of agents deciding to buy, and $`s^{}=s`$ for a cluster deciding to sell at a given timestep. Numerical results for the EZ model showed that the distribution of returns $`P(R)R^\alpha `$ with $`\alpha =3/2`$. We have carried out similar calculations for our model. Figure 2 shows the price return distributions for different values of $`\delta `$ on a log-log scales. As for the cluster size distribution, the exponent $`\alpha `$ is now non-universal and takes on the value $`3/2\delta `$, which is also the value of $`\beta (\delta )1`$ .

## III Mean field analysis

The cluster size distribution in the EZ model can be studied via a mean field analysis. The treatment can be extended to the generalized model to extract the scaling behavior of $`n_s`$, though the algebra is more complicated. Let $`n_s(t)`$ be the number of clusters of size $`s`$ at time $`t`$. The dynamics of $`n_s(t)`$ is governed by the following master equation describing the result of collection action of the members of the cluster. The equation is

$$N\frac{n_s}{t}=as^{1\delta }n_s+\frac{(1a)}{N}\underset{r=1}{\overset{s1}{}}r^{1\delta }n_r(sr)^{1\delta }n_{sr}\frac{2(1a)s^{1\delta }n_s}{N}\underset{r=1}{\overset{\mathrm{}}{}}r^{1\delta }n_r$$
(2)
for $`s>1`$. Each of the terms on the right hand side of Eq.(2) represents the consequence of a possible action of the agent. The first term describes the dissociation of a cluster of size $`s`$ after a transaction is made. The second term represents coagulation of two clusters to form a cluster of size $`s`$. The third term represents the combination of a cluster of size $`s`$ with another cluster. For clusters of size unity ($`s=1`$), we have

$$N\frac{n_1}{t}=a\underset{r=2}{\overset{\mathrm{}}{}}r^{2\delta }n_r\frac{2(1a)n_1}{N}\underset{r=1}{\overset{\mathrm{}}{}}r^{1\delta }n_r.$$
(3)
Here, the first term comes from the dissociation of any clusters into isolated agents and the second term describes the combination of a cluster of size unity with another cluster. In the steady state, $`\frac{n_s}{t}=0`$, we have

$$s^{1\delta }n_s=A\underset{r=1}{\overset{s1}{}}r^{1\delta }(sr)^{1\delta }n_rn_{sr}$$
(4)
for $`s>1`$, and

$$n_1=B\underset{r=2}{\overset{\mathrm{}}{}}r^{2\delta }n_r$$
(5)
for $`s=1`$, where

$$A=\frac{1a}{Na+2(1a)_{r=1}^{\mathrm{}}r^{1\delta }n_r},$$
(6)
and

$$B=\frac{Na}{2(1a)_{r=1}^{\mathrm{}}r^{1\delta }n_r}.$$
(7)
The aim here is to extract the scaling behaviour. Invoking a generating function approach, we let

$$G(\omega )=\underset{r=0}{\overset{\mathrm{}}{}}r^{1\delta }n_re^{\omega r}=g(\omega )+n_1e^\omega ,$$
(8)
where $`g(\omega )=_{r=2}^{\mathrm{}}r^{1\delta }n_re^{\omega r}`$. It follows from Eq.(4) that the function $`g(\omega )`$ satisfies the equation

$$g^2(\omega )+(2n_1e^\omega \frac{1}{A})g(\omega )+n_1^2e^{2\omega }=0.$$
(9)
Note that $`A`$ can be expressed in terms of $`n_1`$ and $`g(0)`$, and

$$g(\omega )=\frac{1}{4A}(1\sqrt{14n_1Ae^\omega })^2.$$
(10)
The number of clusters of size $`s`$ can be found formally by

$$n_s=\frac{1}{s^{1\delta }s!}\frac{^sG}{z^s}|_{z=0},$$
(11)
where $`z=e^\omega `$. The resulting expression for $`n_s`$ is

$$n_s=\frac{(2s2)!(1a)^{s1}(_{r=1}^{\mathrm{}}r^{1\delta }n_r)^s[(1a)_{r=1}^{\mathrm{}}r^{1\delta }n_r+Na]^s}{(s!)^2s^\delta [Na+2(1a)_{r=1}^{\mathrm{}}r^{1\delta }n_r]^{2s1}}.$$
(12)
Invoking Sterling’s formula yields

$$n_sN\left[\frac{4(1a)[(1a)+\frac{Na}{_{r=1}^{\mathrm{}}r^{1\delta }n_r}]}{[\frac{Na}{_{r=1}^{\mathrm{}}r^{1\delta }n_r}+2(1a)]^2}\right]^ss^{(\frac{5}{2}\delta )}.$$
(13)
For $`\delta =0`$, the sum $`_{r=1}^{\mathrm{}}r^{1\delta }n_r=N`$ and the previous results of Refs. are recovered. For $`\delta 0`$, it is difficult to solve for $`n_s`$. Since the summations in Eqs.(12) and (13) give a number, our result shows that $`n_ss^{\beta (\delta )}`$ with $`\beta (\delta )=5/2\delta `$ for the present model. This $`\delta `$-dependent exponent is also indicated in Fig. 1 (lines are a guide to the eye). We note that the scaling behaviour is masked by the behaviour of the term in the squared brackets in Eq.(13) for large values of $`s`$, similar to the situation for the EZ model.

## IV Model II: Size-dependent transaction rate

In this generalization of the EZ model, we introduce a size-dependent transaction rate instead of size-dependent dissociation and combination rates of clusters. It is motivated by the fact that it is hard to measure the structural properties of herds, e.g. the size distribution. Instead, the resulting properties of the herd effect, e.g. price returns, are easier to obtain. Empirically, it was found that the distribution of price returns, while taking on a power law behaviour, has an exponent $`\alpha `$ that is deviated from the value of $`3/2`$ in the EZ model. It is therefore interesting to investigate models in which the exponent for the distribution of the price returns becomes tunable. Here, we introduce a model in which the EZ dynamics is used for cluster formation and dissociation, i.e., the cluster always dissolves after an action is taken. However, when an agent $`i`$ is randomly chosen, there are three possible actions. With probability $`a(1s_i^\sigma )`$, the cluster of agents decide not to take an action (make a transaction) and the cluster simply dissolves. With probability $`as^\sigma `$, the cluster of agents act collectively to either buy or sell, and the cluster dissolves after the transaction is made. With probability $`(1a)`$, the chosen cluster combines with another randomly picked cluster. For $`\sigma =0`$, the present model reduces to the EZ model. For $`\sigma 0`$, the model reflects that while agents may have established connections to exchange information through different means, they may tend to look for common opinion before taking actions. Reaching a consensus becomes difficult as the cluster size increases. Therefore the effective transaction rate is lower than $`a`$ and becomes size-dependent.

Since the combination and dissociation probabilities are identical to those in the EZ model, the distribution of cluster sizes also follows $`n_ss^{5/2}`$, with an exponential cutoff as in the EZ model. Figure 3 shows the distribution of price returns in Model II for different values of $`\sigma `$ in a system with $`N=10^4`$ agents and $`a=0.1`$. It is noted that the return $`P(R)`$ follows a power law $`P(R)R^\alpha `$, where the exponent $`\alpha `$ has become model-dependent. The value of $`\alpha `$ is well described by $`\alpha =3/2+\sigma `$. The result can be understood as the distribution of returns is given by the product of the probability of selecting a cluster of size $`s`$ times the probability of making a transaction, i.e., $`P(R)R^\alpha s^{5/2}s^\sigma s^{(3/2+\sigma )}`$. Since the price returns are generated by the collective action of the whole cluster, we have $`P(R)R^{(3/2+\sigma )}`$.

## V summary

In summary, we have proposed and studied the cluster size distribution, and the price returns distribution, of two generalized versions of the EZ model for herd behavior and information sharing in a population. We have carried out numerical simulations on our models. The two generalizations can be treated analytically within a mean field approach. By introducing a probability for dissociation of a cluster depending on its size, the exponent characterizing the cluster size distribution takes on a model-dependent non-universal value, with a corresponding shift in the exponent characterizing the price returns from the value of the EZ model. It is also possible to make the exponent for the distribution of price returns model-dependent while keeping the exponent of the cluster-size distribution unchanged by imposing a size-dependent transaction rate. Our models thus illustrate that it is possible to tune the power law behaviour. It is interested to note that observed values for the exponent $`\alpha `$ in markets can be as large as $`4`$ , which is quite different from the robust value of $`3/2`$ in the EZ model. Within our model, a value of $`\alpha `$ with $`\alpha >3/2`$ can be attributed to the hesitation among agents in a cluster in making a transaction.

Several extensions of our models are immediately possible. Our particular choices of the form of the probability for dissociation of clusters in Model I and the probability for making a transaction in Model II allow us to tune the exponent. Exploring other functional forms of the probabilities may alter the range of values in which the exponent can be tuned. Our model can also be extended to study the size distribution of businesses. A model similar to the EZ model has already been proposed in this context. However the scaling behavior seems to be non-universal for data from different countries. Our work thus provides a possible extension to cope with this observed non-universality.

ACKNOWLEDGMENTS

One of us (DFZ) acknowledges the support from the China Scholarship Council and the Natural Science Foundation of Guangdong Province, China. Work at CUHK was supported in part by a Grant (CUHK4241/01P) from the Research Grants Council of the Hong Kong SAR Government.

FIGURE CAPTIONS

Figure 1: The size distribution $`n_s/n_1`$ as a function of the size $`s`$ on a log-log scale for different values of $`\delta `$ obtained by numerical simulations (symbols) of Model I. The values of $`\delta `$ used in the calculations are: $`\delta =0,0.25,0.50,0.75`$. The solid lines are a guide to the eye corresponding to exponents $`\beta =2.5,2.25,2.0,1.75`$ respectively.

Figure 2: The distribution of price returns $`P(R)/P(1)`$ as a function of $`R`$ on a log-log scale for different values of $`\delta `$ (symbols) in Model I. The values of $`\delta `$ used in the calculations are: $`\delta =0,0.25,0.50,0.75`$. The solid lines are a guide to the eye corresponding to exponents $`\alpha =1.5,1.25,1.0,0.75`$ respectively.

Figure 3: The distribution of price returns as a function of $`\sigma `$ (symbols) in Model II. The values of $`\sigma `$ used in the calculations are: $`\sigma =0,0.5,1.5,2.5`$. The solid lines are a guide to the eye corresponding to exponents $`\alpha =1.5,2,3,4`$ respectively.