text:
# Kalman filter control in the reinforcement learning framework

## 1. Motivation

Kalman filters and their various extensions are well studied and widely applied tools in both state estimation and control. Recently, there is an increasing interest in Kalman-filters or Kalman-filter like structures as models for neurobiological substrates. It has been suggested that Kalman-filtering (i) may occur at sensory processing , (ii) may be the underlying computation of the hippocampus, and may be the underlying principle in control architectures . Detailed architectural similarities between Kalman-filter and the entorhinal-hippocampal loop as well as between Kalman-filters and the neocortical hierarchy have been described recently . Interplay between the dynamics of Kalman-filter-like architectures and learning of parameters of neuronal networks has promising aspects for explaining known and puzzling phenomena, such as priming, repetition suppression and categorization .

As it is well known, Kalman-filter provides an on-line estimation of the state of the system. On the other hand, optimal control cannot be computed on-line, because it is typically given by a backward recursion (the Ricatti-equations). For on-line parameter estimations without control aspects, see .

The aim of this paper is to derive an on-line control method for the Kalman-filter and achieve optimal performance asymptotically. Slight modification of the linear-quadratic-Gaussian (LQG) Kalman-filter model is introduced for treating the LQG model as a reinforcement learning (RL) problem.

## 2. the Kalman filter and the LQG model

Consider a linear dynamical system with state $`𝐱_t^n`$, control $`𝐮_t^m`$, observation $`𝐲_t^k`$, noises $`𝐰_t^n`$ and $`𝐞_t^k`$ (which are assumed to be Gaussian and white, with covariance matrix $`\mathrm{\Omega }^w`$ and $`\mathrm{\Omega }^e`$, respectively), in discrete time $`t`$:

(1) $`𝐱_{t+1}`$ $`=`$ $`F𝐱_t+G𝐮_t+𝐰_t`$
(2) $`𝐲_t`$ $`=`$ $`H𝐱_t+𝐞_t,`$
the initial state has mean $`\widehat{𝐱}_1`$ and covariance $`\mathrm{\Sigma }_1`$. Executing the control step $`𝐮_t`$ in $`𝐱_t`$ costs

(3)
$$c(𝐱_t,𝐮_t):=𝐱_t^TQ𝐱_t+𝐮_t^TR𝐮_t,$$

and after the $`N`$th step the controller halts and receives a final cost of $`𝐱_N^TQ_N𝐱_N`$.

This problem has the well known solution

(4) $`\widehat{𝐱}_{t+1}`$ $`=`$ $`F\widehat{𝐱}_t+G𝐮_t+K_t(𝐲_tH\widehat{𝐱}_t)`$
(5) $`K_t`$ $`=`$ $`F\mathrm{\Sigma }_tH^T(H\mathrm{\Sigma }_tH^T+\mathrm{\Omega }^e)^1`$
(6) $`\mathrm{\Sigma }_{t+1}`$ $`=`$ $`\mathrm{\Omega }^w+F\mathrm{\Sigma }_tF^TK_tH\mathrm{\Sigma }_tA^T\text{(state estimation)}`$
and

(7) $`𝐮_t`$ $`=`$ $`L_t\widehat{𝐱}_t`$
(8) $`L_t`$ $`=`$ $`(G^TS_{t+1}G+R)^1G^TS_{t+1}F`$
(9) $`S_t`$ $`=`$ $`Q_t+F^TS_{t+1}FF^TS_{t+1}GL_t.\text{(optimal control)}`$
Unfortunately, the optimal control equations are not on-line, because they can be solved only by stepping backward from the final, $`N`$th step.

## 3. Kalman Filtering in the Reinforcement Learning Framework

First of all, we slightly modify the problem: the run time of the controller will not be a fixed number $`N`$. Instead, after each time step, the process will be stopped with some fixed probability $`p`$ (and then the controller incurs the final cost $`c_f(𝐱_f):=𝐱_f^tQ_f𝐱_f`$).

### 3.1. The cost-to-go function

Let $`V_t^{}(𝐱)`$ be the optimal cost-to-go function at time step $`t`$, i.e.

(10)
$$V_t^{}(𝐱):=\underset{𝐮_t,𝐮_{t+1},\mathrm{}}{inf}E\left[c(𝐱_t,𝐮_t)+c(𝐱_{t+1},𝐮_{t+1})+\mathrm{}+c_f(𝐱_f)|𝐱_t=𝐱\right].$$

Clearly, for any $`𝐱`$,

(11)
$$V_t^{}(𝐱)=pc_f(𝐱)+(1p)\underset{𝐮}{inf}\left(c(𝐱,𝐮)+E_w\left[V_{t+1}^{}(F𝐱+G𝐮+w)\right]\right)$$

It can be easily shown that the optimal cost-to-go function is time-independent, furthermore, it is a quadratic function of $`𝐱`$, that is, it is of the form

(12)
$$V^{}(𝐱)=𝐱^T\mathrm{\Pi }^{}𝐱.$$

Our task is to estimate $`V^{}`$ (in fact, the parameter matrix $`\mathrm{\Pi }^{}`$) on-line. This will be done by value iteration.

### 3.2. Value iteration, greedy action selection and the temporal differencing error

Value iteration starts with an arbitrary initial cost-to-go function $`V_0(𝐱)=𝐱^T\mathrm{\Pi }_0𝐱`$. After this, control actions are selected according to the current value function estimate, the value function is updated according to the experience, and these two steps are iterated.

The $`t`$th estimate of $`V^{}`$ is $`V_t(𝐱)=𝐱^T\mathrm{\Pi }_t𝐱`$. The greedy control action according to this is given by

(13) $`𝐮_t`$ $`=`$ $`\mathrm{arg}\underset{𝐮}{\mathrm{min}}\left(c(𝐱_t,𝐮)+E\left[V_t(F𝐱_t+G𝐮+w)\right]\right)`$
(14) $`=`$ $`\mathrm{arg}\underset{𝐮}{\mathrm{min}}\left(𝐮^TR𝐮+(F𝐱_t+G𝐮)^T\mathrm{\Pi }_t(F𝐱_t+G𝐮)\right)`$
(15) $`=`$ $`(R+G^T\mathrm{\Pi }_tG)^1(G^T\mathrm{\Pi }_tF)𝐱_t.`$
For the sake of simplicity, the cost-to-go function will be updated by using the 1-step temporal differencing (TD) method. Naturally, it can be substituted with more sophisticated methods like multi-step TD or eligibility traces. The TD error is

(16)
$$\delta _t=\{\begin{array}{cc}V_t(𝐱_t)c_f(𝐱_t)\hfill & \text{if the controller was stopped at the }t\text{th time step},\hfill \\ \left(c(𝐱_t,𝐮_t)+V_t(𝐱_{t+1})\right)V_t(𝐱_t),\hfill & \text{otherwise}.\hfill \end{array}$$

and the update rule for the parameter matrix $`\mathrm{\Pi }_t`$ is

(17) $`\mathrm{\Pi }_{t+1}`$ $`=`$ $`\mathrm{\Pi }_t+\alpha _t\delta _t_{\mathrm{\Pi }_t}V_t(𝐱_t)`$
(18) $`=`$ $`\mathrm{\Pi }_t+\alpha _t\delta _t𝐱_t𝐱_t^T,`$
where $`\alpha _t`$ is the learning rate. Note that value-estimation error weighted Hebbian learning rule has emerged.

## 4. Concluding remarks

The Kalman-filter control problem was slightly modified to fit the RL framework and an on-line control rule was achieved. The well-founded theory of reinforcement learning ensures asymptotic optimality for the algorithm. The described method is highly extensible. There are straightforward generalizations to other cases, e.g., to extended Kalman filters, dynamics with unknown parameters, non-quadratic cost functions, or more advanced RL algorithms, e.g. eligibility traces. For quadratic loss functions, we have found that learning is Hebbian and it is weighted by the error of value-estimation.


format: text

text:
# Contextual objectivity and quantum holism.

## I Introduction

In a previous preprint , we introduced and discussed a “physical” (as opposed to mathematical) definition of a (pure) quantum state, that reads in the following way:

The quantum state of a physical system is defined by the values of a complete***The set of quantities is complete in the sense that the value of any other quantity which satisfies the same criteria is a function of the set values, see  for details. set of physical quantities, which can be predicted with certainty and measured repeatedly without perturbing in any way the system.

As discussed in detail in ref. , this definition is in full agreement with the usual formalism of QM. It is also implies that some “objectivity” can be attached to the quantum state, because the quantum state is defined from a fully predictable course of events, that is independent of the observer. In  we tried to exploit this definition, together with some ideas about the system dimensionality, to propose a new axiomatic approach to QM, that is an attempt to spell out how the “quantum reality” is related with the “macroscopic reality”. While  is a straightforward rewriting of usual QM,  is much more tentative and its goal is mostly to stimulate some thinking. The FAQ below is a continuation of ref. .

## II More Questions (and answers)

Q: What about the following statement, taken from : “If you believe that the quantum state is rigidly (or even loosely) connected to reality, then you will never find a way out of the conundrum of “unreasonableness” associated with state-vector collapse at a distance. I.e., our community will always be left with a search for the mechanism that makes it go. Our community will always be left with the embarrassing questions to do with its clash with Lorentz invariance. And, maybe most importantly, we will be left with the nagging question of why we can’t harness this mechanism for more useful purposes.”

A: If we simply admit that there is nothing like hidden variables, then Bell’s inequalities (BI) can be violated without invoking anything like an action at a distance, because without the “$`\lambda `$’s” the standard demonstration of these inequalities does not hold. Then in a classical view there should be no correlations either, but this only proves that classical theory does not work.

The apparent “action at a distance” only results from the effort of attributing a \- non-existent - “reality” to the state of a sub-part of a system. The conclusion taken in the quotation above is that “one cannot connect the quantum state to reality”. But this is not correct : the correct conclusion is that a subpart of the system has no quantum state, while the overall system (the pair) certainly has one, because it has a full set of properties that can be predicted with certainty. To state it again:

\- let us admit that there is a “reality” attached to the pair of particles, but that there is no “reality” attached to each particle (here “reality” simply means a quantum state according to our definition).

\- the “reality” attached to the pair makes no problem with Lorentz invariance, because it was created when the two particles interacted, and it simply follows them if they move very far away. The same conclusion apply to more fancy schemes like entanglement swapping, that requires classical communications to effectively prepare the remote entangled state.

\- since there is no “reality” attached to each particle, Bell’s inequalities do not hold ! All the job is done by the fact that the individual particles have no quantum state, or no other property whatsoever that would decide on the result of the measurement. Then “action at a distance” simply vanishes: if a measurement if performed on one side, absolutely nothing happens on the other side.

As far as “action at a distance” is concerned, the situation is just the same as in the classical example where I mail a red and a white paper in two envelopes, one to New Jersey and the other one to Australia : if everybody trust my “state preparation” (many witnesses can swear that it is “real”), the guy in Bell Labs will know immediately the colour of the other paper when opening his envelope. The shocking question is : but “what” carries the correlation if the “papers” have no “colour”, as it is the case in QM ? The answer is : the correlation is carried by the quantum state of the pair, that is real and that propagates causally. All the trouble here comes from the desperate attempt to map this quantum evidence on a classical picture, that is out of the game.

Actually, what I am saying is old hat : there is a well known quotation by Einstein (in approximate form) “There is no doubt that the physicists who adhere to QM will react in the following way : they will give up the requirement of the separate existence of physical reality in different parts of space; they will be right to say that QM makes no explicit use of this requirement”. Then Einstein wrote that this was unacceptable to him, and maybe this is still so difficult to accept because it was said many time that the price to pay is giving up physical reality. But this is wrong : in our point of view local realism was an inheritage from classical physics, that can be abandoned without giving up neither locality (in the sense of relativistic causality), nor realism (in the sense of the existence of an objective physical reality). We can thus admit now that local realism is dead, but that physical realism can do quite well without it.

What next ? QM is a fantastic theory, and it seems relevant to ask: why is it working so well ? Actually, QM was invented 75 years ago in a somehow anarchic way, as an attempt to understand atomic spectra. We may thus speculate the following: QM is actually the answer to a question that was never clearly formulated. We have the answer, what about finding the question ?

Q: What about nonlocality without entanglement  ?

A: In  the authors present a set of orthogonal product states of two three-states particles, that cannot be reliably distinguished by a pair of separate observers who are given one particle each, and who are ignorant of which of the states has been presented to them (the separate observers are allowed any sequence of local operations and classical communications). This was termed “quantum nonlocality without entanglement” in , and “EPR paradox without entanglement” in . This latter paper states that “negation of local realism is roughly called non-locality”, and that “informational local realism” is the concept that is at stake in this example.

First we don’t fully agree with the statement “negation of local realism is roughly called non-locality”. As said above, the negation of local realism is that it is not possible to define objectively the quantum state of a subpart of a quantum system; there is no need, and it may even be misleading, to call this non-locality. We fully acknowledge however that the most straightforward way to demonstrate that local realism is contradictory with QM is by using Bell’s inequalities, that use entangled states (on the quantum side) and locality (on the classical side). But once it is admitted that QM has nothing to do with local realism, there is no reason to restrict this property to entangled states. The situation considered in  gives a striking example that the joint state of a quantum system can be given more “reality” (i.e. more predictability and reproducibility) than the “states” of its parts. The state of the joint system can indeed be determined with certainty by performing a joint measurement (in the appropriate measurement basis), while separate measurements done on the parts cannot provide an equivalent result, whatever is done on both sides.

Q: Can contextual objectivity say anything about the quantum state of the universe in a cosmological sense ?

A: Quantum cosmology is a subject that is far beyond my area of knowledge. However, here is an example of statement (taken from ), that enters in contradiction with contextual objectivity: “The first important question is this: is the universe a complete quantum system or not ? By complete we mean that there are no external observers or classical agencies not subject to the laws of quantum mechanics. The observation of Bell-type correlations and violations of Bell-type inequalities together with a vast amount of empirical evidence for the universal correctness of quantum principles strongly suggests that the answer to the first question is yes. This answer excludes from further discussion all paradigms and models based on any sort of classical hidden variables. We assume henceforth that the entire universe is a vast, self-contained quantum automaton”.

In our view, the authors’ conclusion is unwarranted. Bell’s theorem proves that there are no “classical hidden variables” to be looked for “below” or “inside” quantum mechanics (QM). But it does not prove that QM is an “embedding theory”, i.e., that “the universe a complete quantum system” in the authors’ sense. There is another possible conclusion, i.e. to say that the universe has both a classical structure and a quantum sub-structure. But it may have neither a classical sub-sub-structure (hidden variables), nor a quantum super-structure (multiverse or other versions of a fully quantum universe).

This may be shocking in the sense that we as physicists are strongly tempted to say : “if QM is right, then it must be universal”. But one has too look carefully at what “universal” means. It certainly means that every time one looks at a quantum system, i.e. a system where quantization plays an important role, then this system appears to be quantum. But it certainly does not mean that linear superpositions and so forth are an obvious feature of our everyday’s environment. It may be possible to claim that “we cannot see such superpositions, but even if they have undergone a decoherence process they must remain around there, because the overall evolution must be unitary”. Our approach is still different, since we claim that QM is universal, in the sense that it provides the only way to relate our classical environment, that is essentially based upon continuity, and its quantized sub-structure . In other terms, QM is basically an interface tool.

We don’t conclude either that a quantum state is “purely subjective”, or “a state of our mind”, or whatever similar. A (pure) quantum state is a fairly objective status for a quantum system, in particular because there is no “ignorance” left (a pure state has zero entropy).

So there is clearly some conflict between contextual objectivity and the more theoretical view that “QM is either everything or nothing”. I am not able to spell out whether or not the present approach may have some implications for cosmology or quantum gravity, but hopefully it may interest people competent in these fields.

Q: What about “quantum holism” ?

A: For explaining to the layman what is quantum entanglement, you may say either :

(i) when two remote particles are in an entangled state, there is some kind of mysterious influence or instantaneous action-at-a distance between them, as it is proven by the observed violation of Bell’s inequalities. Though this action-at-a-distance cannot be used for superluminal signalling, it is a clear evidence for quantum non-locality.

or :

(ii) quantum entanglement is the fact that in quantum mechanics it is possible to attribute definite predictable properties to an ensemble of several particles, while it is impossible to do so for each of the particles taken separately. This impossibility contradicts the classical assumption that the state of a given composite system can be defined from the state of its parts.

It should be clear to the reader that in our approach, (i) is misleading and even wrong, while (ii) is perfectly fine, and moreover is in perfect agreement with the mathematical content of the quantum formalism. But for an unknown reason, (i) usually generates a lot of excitement, while (ii) will generate no excitement at all - simply a glance of deep misunderstanding (you may try it, I did).

Therefore I propose to use the words “quantum holism” to convey the idea that (ii) is strange, interesting and very far from classical physics. This wording expresses the ability of the quantum formalism to attribute “more reality” to a composite system than to its parts. Then (ii) can be rewritten as :

(iii) quantum entanglement is the fact that quantum mechanics can attribute fully predictable properties to an ensemble of several particles, in such a way that knowing these properties is contradictory with knowing predictable properties for each of the particles. This “quantum holism” strongly contradicts the classical assumption that the state of a given composite system can always be reconstructed from knowing the states of its parts.

From the experience of anyone who has been reviewer and editor of scientific papers, or simply reader of broad-audience essays about quantum mechanics, the number of irrelevant or even wrong discussions that have been triggered by the wording “quantum non-locality” is simply astonishing. So it may be worthwhile to give a try to “quantum holism”, that has essentially the same content for those who know and use quantum mechanics, but may convey quite different ideas. And it does say what we wish to say : quantum mechanics has no conflict with causality or relativity, but it gives an holistic description of the state of a physical system. Quantum holism is the true “quantum scandal”, that is at the heart of entanglement, EPR paradox, violation of Bell’s inequalities, and decoherence. After 75 years of quantum mechanics it probably deserves some recognition.

Q: Are not you unfair to Chris Fuchs’ position ?

A: The sentence from  used as the first question of the present paper may be quoted in an inappropriate way indeed : its initial purpose was to illustrate the state of affair before Bell’s theorem, and not to express Chris Fuchs’ personal views on quantum mechanics (thanks to David Mermin for pointing that out).

Nevertheless, if I remember well a discussion about Chris Fuchs’ position , the conclusion was that a quantum state is not objective, but that it is not less objective than a classical state. If this is the case, the debate vanishes : my point is not to do philosophy about the ultimate nature of reality, neither to be unfair to Chris Fuchs, but to criticize and if possible to dismiss the often-heard statement that classical physics is the domain of realism and objectivity, while quantum physics would be the domain of fogginess and subjectivity .

I do that by claiming that the EPR criterion of reality is (almost) valid in QM, with the caveat that it cannot be applied to all physical quantities simultaneously, but only to a subset of them. Nevertheless, this is strong enough to attribute (contextual) objectivity to a quantum state. The characteristic feature of an entangled state is that “reality” can be attributed only to joint physical properties, and not to individual ones. This is why I think that the usual wording of “quantum non-locality” that is attributed to entanglement is very misleading, and should be replaced by “quantum holism”.

Actually, Bell’s theorem proves that if a “classical” theory (i.e., a theory where all particles have definite physical properties, that may be unknown to us) wants to reproduce QM and be in agreement with the experiments, then it must be non-local (i.e., it must include “instantaneous action at a distance”). But does that imply that QM itself requires anything happening instantaneously at a distance ? My answer is definitely no.

It is certainly generally admitted that QM does not allow super-luminal communications. But then it is often said that QM include mysterious “non-local influences”, often by invoking the “instantaneous reduction of the wave packet”. This is nonsense. Bell’s inequalities are violated because the first Bell’s hypothesis fails, i.e. that, in contradistinction with classical physics, QM is able to describe global properties that are not carried by properties of individual particles : this is what I call quantum holism.

But is not quantum holism simply the same thing as “quantum non-locality” ? Yes indeed, if “quantum non-locality” is understood correctly, recognizing that it has nothing to do with anything “instantaneous at a distance”. But if the meaning of “quantum non-locality” is unclear to you, quantum holism has the huge advantage that it does not suggest any kind of “instantaneous transmission”, but explicitly refers to the existence of global properties that are not contained in the properties of the subparts. This is in my opinion the correct physical view that is underlying entanglement, EPR paradox, Bell’s inequalities and even decoherence.


text:
# Evolution of an afterglow with a hard electron spectrum

## 1. Introduction

GRB afterglow emission is usually modelled with a single power-law electron energy spectrum:

$$N(\gamma _e)\gamma _e^p\gamma _m<\gamma _e<\mathrm{}$$
(1)
with the power-law index $`p>2`$ and $`\gamma _e`$ being the lorentz factor of the electron (see Piran 1999 for a review). Relativistic shock acceleration theory predicts an universal $`p2.3`$ which fits many afterglow spectra (Waxman 1997; Galama et al 1998; Wijers and Galama 1999).

In some GRB afterglows, harder electron energy spectrum $`p<2`$ appears to be needed (Panaitescu and Kumar 2001a; Sagar et al 2001; Cowsik et al 2001; Bhattacharya 2001). Such a spectrum must have an upper cutoff, an injection break $`\gamma _i`$, beyond which $`p`$ must exceed $`2`$ in order to keep the total energy in the electron distribution finite. The evolution of the afterglow emission is influenced by the evolution of $`\gamma _i`$ (Bhattacharya 2001; Panaitescu and Kumar 2001b; Dai and Cheng 2001)

The minimum lorentz factor $`\gamma _m`$ of the distribution is determined by the fraction $`ϵ_e`$ of the postshock thermal energy that is injected into electrons. For $`p>2`$ this is given by

$$\gamma _m=ϵ_e\left(\frac{p2}{p1}\right)\left(\frac{m_p}{m_e}\mathrm{\Gamma }_{\mathrm{sh}}\right)$$
(2)
where $`\mathrm{\Gamma }_{\mathrm{sh}}`$ is the bulk lorentz factor of the blast wave shock and $`m_p`$, $`m_e`$ are proton and electron masses respectively (Sari, Piran and Narayan 1998).

## 2. An injection break

Relativistic shock acceleration theory predicts that the universal spectrum with $`p2.3`$ can extend only down to a minimum $`\gamma _{\mathrm{acc}}=(m_p/m_e)\mathrm{\Gamma }_{\mathrm{sh}}`$. Below this energy the electron Larmor radius becomes smaller than the shock thickness, the shock is no longer perceived as a discontinuity and the acceleration efficiency drops (Hoshino et al 1992; Gallant et al 2000; Kirk 2002). However for a low $`ϵ_e`$ the $`\gamma _m`$ estimated from eq. 2, and routinely used in GRB afterglow modelling, works out to be much lower than $`\gamma _{\mathrm{acc}}`$. This creates a contradiction.

In the region between $`\gamma _m`$ and $`\gamma _{\mathrm{acc}}`$, some special pre-acceleration mechanisms are needed (Kirk 2002; Gallant et al 2000). One candidate for this is a variant of the ion magnetosonic wave absorption mechanism proposed for electron-positron-proton plasma (Hoshino et al 1992). This mechanism produces a hard energy spectrum, but has a natural upper cutoff at exactly the $`\gamma _{\mathrm{acc}}`$ defined above. We conjecture that for electrons in GRB afterglows, a $`p_2>2`$ universal spectrum is valid down to $`\gamma _{\mathrm{acc}}`$, and the energy spectrum below this, down to an appropriate $`\gamma _m`$, is harder, with a $`p_1<2`$. We identify the injection break $`\gamma _i`$ with $`\gamma _{\mathrm{acc}}`$. This would also introduce a corresponding injection break $`\nu _i`$ in the radiation spectrum. Here

$$\gamma _m=\left[ϵ_e\frac{(2p_1)(p_22)}{(p_2p_1)(p_11)}\right]^{1/(p_11)}\gamma _i$$
(3)
## 3. Evolution of the radiation spectrum

We have calculated the evolution of the afterglow radiation spectrum based on this premise. Writing the flux at a frequency $`\nu `$ at time $`t`$ as $`F_\nu (t)t^\alpha \nu ^\beta `$, we find the values of $`\alpha `$ and $`\beta `$ in different spectral regimes to be as follows:

In the table, the third column gives the light curve slopes after the lateral spreading of the collimated outflow dominates the dynamics, and the second column those before this phase. The transition between them is often called the jet break. The quantity $`\nu _c`$ stands for the synchrotron cooling break frequency in the radiation spectrum.

## 4. GRB010222

The afterglow of GRB010222 has been modelled previously as a burst in a very high density medium undergoing a transition to a non-relativistic expansion (Masetti et al 2001), which cannot explain the early appearance of radio emission; or as a hard electron spectrum afterglow that underwent an early jet break (Sagar et al 2001; Cowsik et al 2001; Panaitescu and Kumar 2001a), which had difficulty explaining the spectral slope observed in X-ray bands.

We suggest that this afterglow had an injection break $`\nu _i`$ in the X-ray band $`1`$ day after the burst, had a jet break transition at $`t_j0.5`$ day, and evolved in a normal ISM. With an assumed $`p_1=1.3`$, $`p_2=2.1`$, and a smooth joining of the power-law segments, we are able to obtain good fits to the spectrum and the light curve of this afterglow, as shown in the figures.

## 5. Summary

We have computed the evolution of a GRB afterglow with a hard electron energy spectrum up to an injection break at $`m_p/m_e`$ times the bulk lorentz factor of the shock. Above the injection break, relativistic shock acceleration predicts an universal, relatively softer, electron energy spectrum. We obtain good fits to the observed evolution of GRB010222 afterglow using this model. We suggest that hard spectrum emission is more likely to be seen in GRB afterglows that have a relatively low $`ϵ_e`$.

## References

Bhattacharya, D. 2001, BASI, 29, 107

Cowsik, R., et al 2001, BASI, 29, 157

Frail, D.A., et al. 2002, ApJ, 565, 829

Galama, T.J., Wijers, R.A.M.J., Bermer, M., et al. 1998, ApJ, 500, L101

Gallant, Y.A., Achterberg, A., Kirk, J.G., & Guthmann, A.W. 2000, in ASP Conf. Ser. No. 526: 5th Huntsville Symposium on Gamma Ray Bursts (NewYork:ASP), p. 524

Hoshino, M., Arons, J., Gallant, Y.A., & Langdon, A.B. 1992, ApJ, 390, 454

in’t Zand, J.J.M., et al. 2001, ApJ, 559, 710

Kirk, J.G. 2002, in Sackler Symposium on ’Gamma Ray Bursts: The Brightest Explosions in the Universe’, http://cfa-www.harvard.edu/grbconf/

Masetti, N., et al. 2001, A&A, 374, 382

Panaitescu, A., Kumar, P. 2001a, ApJ, 571, 779

Panaitescu, A., Kumar, P. 2001b, ApJ, 560, L49

Piran, T. 1999, Phys. Rep., 314, 575

Sagar, R., et al. 2001, BASI, 29, 91

Sari, R., Piran, T., & Narayan, R. 1998, ApJ, 497, L17

Waxman, E. 1997, ApJ, 485, L5


text:
# Hadronization in heavy ion collisions: Recombination and fragmentation of partons

## Abstract

We argue that the emission of hadrons with transverse momentum up to about 5 GeV/$`c`$ in central relativistic heavy ion collisions is dominated by recombination, rather than fragmentation of partons. This mechanism provides a natural explanation for the observed constant baryon-to-meson ratio of about one and the apparent lack of a nuclear suppression of the baryon yield in this momentum range. Fragmentation becomes dominant at higher transverse momentum, but the transition point is delayed by the energy loss of fast partons in dense matter.

Data from the Relativistic Heavy Ion Collider (RHIC) have shown a strong nuclear suppression of the pion yield at transverse momenta larger than 2 GeV/$`c`$ in central Au + Au collisions, compared with $`p+p`$ interactions PHENIX . The emission of protons and antiprotons does not appear to be similarly suppressed, and the $`p`$/$`\pi ^+`$ ratio reaches or even exceeds unity for transverse momenta above 2 GeV/$`c`$ PHENIX-B ; STAR-B ; STAR-L ; PHENIX-L . These results lack a consistent explanation in the standard picture of hadron production at high transverse momentum, which assumes that hadrons are created by the fragmentation of energetic partons. Whereas the observed suppression of the pion yield is attributed to the energy loss of partons during their propagation through the hot and dense matter created in the nuclear collision – a phenomenon commonly referred to as jet quenching GyulWang:94  – the absence of a similar effect in the proton spectrum is puzzling Science .

We propose that hadron production at momenta of a few GeV/$`c`$ in an environment with a high density of partons occurs by recombination, rather than fragmentation, of partons. Below we show that recombination always dominates over fragmentation for an exponentially falling parton spectrum, but that fragmentation wins out eventually, when the spectrum takes the form of a power law. We also show that recombination can explain some of the surprising features of the RHIC data, as first suggested by Voloshin Voloshin:02 .

In the fragmentation picture CoSo:81  the single parton spectrum is convoluted with the probability for a parton $`i`$ to hadronize into a hadron $`h`$, which carries a fraction $`z<1`$ of the momentum of the parent parton. It has been argued that the fragmentation functions $`D_{ih}(z)`$ can be altered by the environment GW:00 . The dominant modification mechanism is the energy loss of the propagating parton in the surrounding medium, which leads, in first approximation, to a rescaling of the variable $`z`$. This would affect all produced hadrons in the same way, in contradiction with the observations at RHIC.

Another mechanism of hadron production is quark recombination. In the recombination picture, three quarks or a quark/antiquark pair in a densely populated phase space can form a baryon or meson respectively. The amplitude for this process is determined by the hadron wave function. This mechanism has recently been identified as the source of unnatural isospin ratios in the production of $`D`$-mesons in the fragmentation region in $`\pi ^{}+A`$ interactions at Fermilab BJM:02 . Hadron production in heavy ion collisions by recombination of quarks has been considered before recomb , primarily at small transverse momentum. Quark recombination has recently been invoked to explain some aspects of the RHIC data, such as the flavor pattern of elliptic flow LinKo:02 , and in the context of a scaling model HwaYa:02 .

In the formalism of perturbative QCD, recombination is a more exclusive process, and falls off faster than fragmentation with increasing transverse momentum. On the other hand, for a meson of given momentum $`P`$ fragmentation starts out with a parton with much higher momentum $`p=P/z`$, $`(z<1)`$, whereas recombination requires a quark/antiquark pair where each parton carries only about $`P/2`$ in average. However, the spectrum of high-momentum partons is steeply falling and further reduced by the energy loss in dense matter. Our main result is that the momentum range, in which recombination processes can successfully compete with fragmentation, may extend up to 5 GeV/$`c`$ in the favorable environment of central heavy ion collisions at RHIC. We will show below that recombination is effective whenever the phase-space distribution of a system of partons has thermal character.

Let us consider an expanding system of quarks and antiquarks. We assume that the recombination of these partons into hadrons occurs on a space-like hypersurface $`\mathrm{\Sigma }_\text{f}`$. The RHIC experiments indicate that the freeze-out is very rapid. The measured two-particle correlation functions are consistent with an extremely short emission time in the local rest frame, suggesting a sudden transition after which individual hadrons interact only rarely RHIC-HBT . In our treatment, we assume that the dense parton matter is devoid of dynamical thermal gluons and predominantly composed of quarks and antiquarks at the moment of hadronization. We neglect a possible effective quark mass, because we are here interested in hadron production at high transverse momentum.

Denoting the density matrix of the parton system on $`\mathrm{\Sigma }_\text{f}`$ as $`\widehat{\rho }_\text{f}`$, we can express the number of mesons at freeze-out as $`EdN_M/d^3P=(2\pi )^3M;P|\widehat{\rho }_\text{f}|M;P`$ . Here $`|M;P`$ is a meson state with momentum $`P`$. We now introduce the single-particle Wigner functions for quarks and antiquarks, $`w_a(r;p)`$ and $`\overline{w}_b(r;p)`$, respectively, and neglect multi-particle correlations in the density matrix. Further introducing the time-like, future oriented unit vector $`u^\mu (r)`$ orthogonal to the freeze-out hypersurface at $`r\mathrm{\Sigma }_\text{f}`$ CoFr:74 ; DHSZ:91 , we obtain the following expression for the meson emission spectrum:

$$E\frac{dN_M}{d^3P}=\underset{\mathrm{\Sigma }_\text{f}}{}𝑑\sigma \frac{Pu(r)}{(2\pi )^3}\underset{a,b}{}\underset{0}{\overset{1}{}}𝑑z|\psi _{ab}^M(z)|^2w_a(r;zP^+)\overline{w}_b(r;(1z)P^+).$$
(1)
Because we are considering the emission of a meson with high momentum, it is most convenient to use a distribution amplitude in terms of light-cone coordinates in a frame where $`P^+P^{}`$ and the transverse momentum of the meson vanishes. $`z`$ and $`1z`$ are the momentum fractions carried by the quark and antiquark, respectively, and $`a,b`$ denote the internal quantum numbers of the quarks (spin, flavor, color). For baryons, a similar calculation yields:

$$E\frac{dN_B}{d^3P}=\underset{\mathrm{\Sigma }_\text{f}}{}𝑑\sigma \frac{Pu(r)}{(2\pi )^3}\underset{a,b,c}{}\underset{0}{\overset{1}{}}𝑑z_1\underset{0}{\overset{1z_1}{}}𝑑z_2|\psi _{abc}^B(z_1,z_2)|^2w_a(r;z_1P^+)w_b(r;z_2P^+)w_c(r;(1z_1z_2)P^+).$$
(2)
The overall normalization is fixed by the number of quarks on the hypersurface $`\mathrm{\Sigma }_\text{f}`$:

$$N_a=_{\mathrm{\Sigma }_\text{f}}𝑑\sigma \frac{d^3p}{(2\pi )^3p^0}pu(r)w_a(r;p).$$
(3)
It is important to note that our results are not really sensitive to the model used for the recombination process. A complete dynamical description in terms of QCD is hard to achieve. However, for the observables that we discuss below, it is essential to use two common features: the probability for the emission of a meson (baryon) is proportional to the single parton distribution squared (cubed), and the parton momenta sum up to the hadron momentum. At this level of sophistication a description with the light-cone wave functions replaced by three-dimensional, spatial quark wave functions leads to identical results.

To make further progress, we need to specify the parton Wigner functions. We first consider the case that these are described by the exponential tail of a thermal distribution $`w_a(r;p)=\mathrm{exp}((pu\mu )/T)`$ with local temperature $`T(r)`$ and chemical potential $`\mu (r)`$, independent of the internal quantum numbers. The constraint that the momentum fractions of all partons in the hadron wave function must add up to the total momentum $`P`$ of the hadron then insures that the product of all Wigner functions entering into the hadron yield is solely dependent on the hadron momentum:

$`w_a(r;zP^+)\overline{w}_b(r;(1z)P^+)=\mathrm{exp}(Pu/T),`$
$`w_a(r;z_1P^+)w_b(r;z_2P^+)w_c(r;(1z_1z_2)P^+)`$
$`=\mathrm{exp}((Pu\mu _B)/T)`$ , (4)
where $`\mu _B=3\mu `$. One can then perform the integrations over the momentum fractions in (1) and (2) and obtains the result that the baryon-to-meson ratio is independent of the momentum $`P`$ and simply given by the ratio of the number of quark degrees of freedom contributing to the emission of the specific hadrons: $`dN_B/dN_M=_{a,b,c}e^{\mu _B/T}/_{a,b}`$.

The summations over color, flavor, and helicity give rise to degeneracy factors for mesons and baryons, $`C_M`$ and $`C_B`$, respectively. In order to derive an upper limit to the nucleon-to-pion ratio, we make the simplifying assumption that all decuplet baryon states $`\mathrm{\Delta }`$ contribute to the nucleon yield through decay, but neglect the contributions to the high-$`P_T`$ pion spectrum from the decay of unstable hadrons. If we require the partons to form color singlets at recombination, we obtain $`C_{\pi ^+}=1`$ and $`C_p=20/(2\times 3!)=5/3`$, yielding the result

$$dN_p/dN_{\pi ^+}=e^{\mu _B/T}C_p/C_{\pi ^+}=\frac{5}{3}e^{\mu _B/T}.$$
(5)
Decay of unstable hadron states is likely to reduce this value, but will not change the prediction that the $`p/\pi ^+`$ ratio is constant as a function of particle momentum and significantly larger than the value expected from quark fragmentation. We note that this result is in good agreement with the RHIC data, which show $`dN_p/dN_{\pi ^+}`$ reaching a plateau around unity in the range 2 GeV/$`c`$ $`P_T4`$ GeV/$`c`$ RHIC-ppi . For the $`\mathrm{\Lambda }`$-hyperon yield, we need to include the channels $`\mathrm{\Lambda }`$, $`\mathrm{\Sigma }^0`$ and $`\mathrm{\Sigma }^0`$, leading to $`C_\mathrm{\Lambda }=4/3`$. This would predict a plateau at 4/3 in the $`\mathrm{\Lambda }/K_s^0`$ yield, which will be diluted by kaons from the decay of $`K^{}`$. The $`\overline{p}`$/$`p`$ ratio is also predicted to be independent of momentum and equal to $`\mathrm{exp}(2\mu _B/T)`$, again in rather good agreement with the RHIC data STAR-B .

The predictions are radically different, when one considers a power law spectrum as it is characteristic in perturbative QCD at large transverse momentum:

$$w_a^{\text{pert}}(r;p)=A_a\left(1+\frac{p_T}{B}\right)^\alpha .$$
(6)
One then finds that mesons always dominate over baryons at large momentum, $`dN_B/dN_M=(27/4P)^\alpha C_BA_q^2/C_MA_{\overline{q}}`$, and that eventually parton fragmentation wins out over quark recombination. For pions, the local ratio of the two contributions is $`dN_\pi ^{\mathrm{frag}}/dN_\pi ^{\mathrm{rec}}P^\alpha `$. On the other hand, for an exponential quark spectrum, fragmentation is always suppressed with respect to recombination.

This result constitutes the main insight gained from our considerations: Hadron emission from a thermal parton ensemble is always dominated by parton recombination; only when the thermal distribution gives way to a perturbative power law at high momentum, does fragmentation become the leading hadronization mechanism. The threshold between the two domains depends on the size of the emitting system and the hadron species.

That the meson spectrum from recombination is determined, on the average, by $`w(P/2)`$, whereas the baryon spectrum depends on $`w(P/3)`$, implies that those kinematic properties of the hadron spectrum, which are due to collective flow of partons, extend to higher values of the transverse momentum for baryons than for mesons LinKo:02 . This effect is clearly visible in the RHIC data RHIC-ppi ; RHIC-v2 , which exhibit a linear rise of the elliptic flow velocity $`v_2`$ with $`P_T`$, which continues further in $`P_T`$ for protons and hyperons than for pions and kaons.

Realistic calculations require the specification of the freeze-out hypersurface $`\mathrm{\Sigma }_\text{f}`$ and the parton spectrum. Here we assume boost invariance of $`\mathrm{\Sigma }_\text{f}`$ according to $`\tau _\text{f}=\sqrt{t_\text{f}^2z_\text{f}^2}=`$ const. DHSZ:91 . For the thermal part of the quark spectrum we use an axially and longitudinally expanding thermal source:

$$w_a^{\text{th}}(\eta ;y,p_T)=A_{\text{th}}e^{p_T\mathrm{cosh}(\eta y)/T}e^{y^2/2\mathrm{\Delta }^2},$$
(7)
characterized by an effective temperature $`T350`$ MeV, which includes the blue shift caused by the radial expansion, and a rapidity width $`\mathrm{\Delta }2`$. Here $`\eta `$ and $`y`$ are the rapidities in space-time and momentum. The form of this spectrum agrees with the results of the parton cascade VNI/BMS PCM , which yields a parton distribution exhibiting an exponential shape at low transverse momentum and a power law shape for high transverse momentum. For the power law tail of the parton spectrum we choose the results given by a lowest-order perturbative QCD calculation FMS:02 , shifted by $`\mathrm{\Delta }p_T=\sqrt{\lambda p_T}`$ with $`\lambda =1\text{ GeV}`$ to account for the energy loss of fast partons, and standard fragmentation functions KKP:00 . The normalization of the thermal part of the spectrum is adjusted to fit the measured inclusive spectrum of charged hadrons from PHENIX Jia:02 , as shown in the upper frame of Fig. 1. The contributions from recombination and fragmentation are shown separately to exhibit the location of the rapid crossover between these two mechanisms at about 5 GeV/$`c`$. We note that in the parton spectrum $`w^{\text{th}}+w^{\text{pert}}`$ itself, the crossover between the thermal and the perturbative part occurs already at about 3 GeV/$`c`$, consistent with parton cascade predictions PCM . The recombination mechanism shifts this point to higher values of $`p_T`$ in the hadron spectrum.

The lower frame of Fig 1 shows our prediction for the $`p/\pi ^+`$ ratio. The rapid drop of its value in the range 4–5 GeV/$`c`$ is an unambiguous prediction of our model. Experiments at RHIC have not yet been able to probe this $`P_T`$ range, because the identification of protons has not been feasible beyond 4 GeV/$`c`$. The identification of hyperons is possible to higher $`P_T`$, and indications of a rapid drop in the $`\mathrm{\Lambda }/K_s^0`$ ratio have been found STAR-L .

Figure 2 shows the scaled ratio of particle yields in Au + Au and $`p+p`$ collisions, called $`R_{\text{AA}}`$. (We use fits to data for the $`p+p`$ yields and our predictions for Au + Au.) The energy loss of fast partons leads to a nuclear suppression in the fragmentation region ($`P_T>5`$ GeV/$`c`$). For low $`P_T`$ this suppression is counteracted by the recombination mechanism, which is absent in $`p+p`$ reactions. Recombination is more important for protons than for pions, resulting in much less nuclear suppression for protons.

RHIC data RHIC-v2  exhibit a strong increase of the anisotropic flow parameter $`v_2`$ for mesons and baryons at small $`P_T`$, which finally saturates. This happens earlier for mesons than for baryons. It has been argued that the flow anisotropy originates in the partonic phase Kolb:00 . In the recombination region mesons at transverse momentum $`P_T`$ reflect the properties of partons with an average transverse momentum $`P_T/2`$, while baryons reflect those of partons with $`P_T/3`$. It follows that $`v_2`$ saturates later for baryons than for mesons Voloshin:02 . The transition to the fragmentation region would provide such a mechanism, but it occurs at too high momentum. The observed saturation of $`v_2`$ must, therefore, be due to some other mechanism or require a more realistic description of the space-time evolution of the system.

In summary, we propose a two component behaviour of hadronic observables in heavy ion collisions at RHIC. These components include fragmentation of high-$`p_T`$ partons and recombination from a thermal parton distribution. The competition between recombination and fragmentation of partons can explain several of the surprising features of the published data. In particular, the proton excess at intermediate $`P_T`$, the different nuclear suppression observed in pion and proton spectra, and the different saturation thresholds in the elliptic flow, are easily explained. We predict that all baryon spectra will exhibit a rapid transition around 5 GeV/$`c`$ to a region dominated by parton fragmentation. Finally, our scenario requires the assumption of a thermalized partonic phase characterized by an exponential momentum spectrum. Such a phase may be appropriately called a quark-gluon plasma.

Note added: We draw attention to the closely related recent work of Greco et al. GreKoLe:03 , who also propose that parton recombination can explain the large baryon/meson ratio observed at RHIC.

###### Acknowledgements.

We thank T. Mehen and M. Asakawa for very helpful discussions. This work was supported by RIKEN, Brookhaven National Laboratory, DOE grants DE-FG02-96ER40945 and DE-AC02-98CH10886, and by the Alexander von Humboldt Foundation.


text:
# Red giant depletion in globular cluster cores

## 1 Introduction

Observations by Djorgovski et al. (1991) have shown that clusters with central cusps (presumably collapsed cores) have colour and population gradients. Their observations show that a number of post-core-collapse (PCC) clusters become bluer toward their centres, which is an effect of the demise of the red giant and/or sub-giant populations, and possibly an increase in the number of faint, blue objects. The effect is a few per cent of the visible light.

The cores of PCC clusters have been imaged using the high spatial resolution of the Hubble Space Telescope (Bailyn 1994; Shara et al. 1998). The observations show a depletion of the red giant branch stars relative to the horizontal branch. In addition a population of supra-horizontal-branch stars (SHBs) have been observed. These are stars which lie in the Hertzsprung-Russell diagram as a distinct population brighter than normal horizontal branch stars. Bailyn (1994) found SHBs in the core of 47 Tuc which were about one magnitude brighter, while Shara et al. (1998), in images of NGC 6522, found SHBs which were one and a half magnitudes brighter in the $`B`$ band. The formation of these objects is not well understood.

In PCC clusters more stellar encounters are expected to occur than for the non-PCC clusters. These encounters could deplete the red giant population through two methods. Firstly, collisions between red giants and binaries could either eject the red giant core from the surrounding envelope or result in a common envelope phase (Adams, Sills & Davies 2003). This effect could at most deplete a few per cent of the red giant population. Secondly stellar encounters would modify the binary population through hardening (encounters making the binary tighter) and exchanges (the third star is exchanged with the lower mass binary component if it is more massive). This would result in a different binary population to the non-PCC clusters. This modified population evolves such that the red giants, which would be observed today, undergo mergers with their binary companions.

As red giants evolve into horizontal branch stars any evolutionary model which is considered needs to explain why the horizontal branch is not depleted in the same manner as the red giants. We investigate whether such an evolutionary scenario is possible. Such a scenario would require binary interactions to turn/accelerate the evolution of the red giants into becoming horizontal branch stars (see Djorgovski et al. 1991). In this paper we discuss an evolutionary scenario which naturally explains both the observed depletion of red giants and the non-depletion of horizontal branch/faint, blue stars in PCC clusters. In this scenario, the binary consists of two low-mass (below helium-flash limit) stars and evolves through two common envelope phases. In the first common envelope phase the primary is a red giant and loses its envelope to leave a helium white dwarf–main sequence star binary. During the second common envelope phase , in which the secondary has evolved into a red giant, the two degenerate helium cores coalesce resulting in a helium burning core and the remainder of the common envelope forming an envelope around the merged object.

We examine the required parameter space (initial masses and separation) to explain why the depletion is more likely to occur in PCC clusters. Stellar encounters are expected to play a significant role in PCC clusters. After a number of encounters the original binary will have had its separation ground down and its components will have higher masses due to exchanges (see Davies 1995). The initial mass range which would produce red giants today is between 0.75 M and 0.95 M depending on the age of the cluster and hence the turn-off mass. We show that without encounters this mass range is more likely to be found in single stars and the higher mass binary components (the primaries). After a number of encounters more are found in binaries and in particular as the secondaries of binaries. It is these secondaries which lose their envelopes in the second common envelope phase explaining why red giant depletion is only observed in PCC clusters.

Our evolutionary scenario is discussed in section 2. In section 3 we investigate the effects stellar encounters have in modifying the binary population of PCC clusters. Section 4 contains a discussion of our results, in particular the observational consequences and the other products of our evolutionary scenario. We give our conclusions in section 5.

## 2 Evolutionary scenario

The evolutionary scenario for depleting the red giant branch is shown in Fig. 1. We consider the evolution of binary stars where convective mass transfer occurs when the primary evolves off the main sequence and fills its Roche lobe. A common envelope phase follows. The secondary and the degenerate helium core spiral together as the envelope is ejected. The secondary then evolves to fill its Roche lobe and convective mass transfer again occurs forming a second common envelope phase. During this common envelope phase the degenerate helium cores spiral together.

If the cores coalesce during the second common envelope phase then the combined core masses may be enough to ignite helium and the remainder of the common envelope is left around the merged helium burning core and resembles a horizontal branch star. If the hydrogen envelope mass is small then the product may resemble an sdB star. Alternatively, after the second common envelope phase a tight double degenerate helium white dwarf system could be formed. This system could be close enough that inspiral due to gravitational radiation brings them into contact.

To model the binary evolution we use the binary evolution code of Hurley, Tout & Pols (2002). This code uses analytical formulae for stellar evolution (Hurley, Pols & Tout 2000) and includes the effects of tides, gravitational radiation, mass transfer and mass loss due to winds. For the common envelope the code assumes the inspiral is given by an energy balance between the envelope binding energy and the difference in orbital energy of the inspiraling components. The envelope binding energy is given by

$$E_{\mathrm{bind}}=\frac{GM_{\mathrm{c1}}M_{\mathrm{env1}}}{\lambda R_1},$$
(1)
where $`M_{\mathrm{c1}}`$ and $`M_{\mathrm{env1}}`$ are the core and envelope mass of the red giant respectively, $`R_1`$ is its radius and $`\lambda `$ is a constant. The change in orbital binding energy is given by

$$\delta E_{\mathrm{orb}}=\frac{1}{2}\left(\frac{GM_{\mathrm{c1}}M_2}{a_\mathrm{f}}\frac{GM_{\mathrm{c1}}M_2}{a_\mathrm{i}}\right),$$
(2)
where $`a_\mathrm{i}`$ and $`a_\mathrm{f}`$ are the initial and final separations respectively and $`M_2`$ is the mass of the secondary. The energy balance is then given by

$$E_{\mathrm{bind}}=\alpha _{\mathrm{CE}}\delta E_{\mathrm{orb}},$$
(3)
where $`\alpha _{\mathrm{CE}}`$ is the common envelope efficiency. The ratio of initial to final separation only depends on the product $`\alpha _{\mathrm{CE}}\lambda `$ and not on their separate values. We take the value of $`\alpha _{\mathrm{CE}}\lambda =1`$ in our calculations.

### 2.1 Constraints on initial masses

In order for the observed red giant population to be depleted the secondaries in the binaries require masses near the turn-off mass for the cluster. Fig. 2 shows the age of a star at the end of the red giant branch as function of initial mass and for metallicities z = 0.0001, 0.001, 0.002.

From Fig. 2 it is clear that a large range of cluster ages of 10 – 20 Gyr can be represented if we consider a mass range for the secondary of 0.75 – 0.95 M. This represents the range of masses of secondaries which would be observed as red giants in globular clusters today.

### 2.2 Constraints on initial separations

The initial separation needs to be such that both binary components overflow their Roche lobes as red giants. If the separation is too small then the secondary will overflow its Roche lobe before it reaches the red giant branch. This will deplete both the red giant and horizontal branch. Alternatively if the separation is too large then the binary will not come into contact for a second time. In this case no depletion of either the red giant or the horizontal branch occurs.

Fig. 3 is a plot of initial primary masses and separations and shows the constraints for the formation of a horizontal branch star via the coalescence of two degenerate helium cores during the second common envelope phase. We assume a metallicity of 0.001 and a secondary mass of 0.8 M. The solid line shows the separation above which coalescence occurs in the second common envelope. Below this separation the secondary fills its Roche lobe before it has reached the red giant branch. The short-dashed line represents the separation at which a double degenerate helium white dwarf system is formed as opposed to coalescence occurring during the common envelope. The long-dashed line represents the separation at which a helium white dwarf and carbon-oxygen white dwarf is formed due to the primary not filling its Roche lobe until the asymptotic giant branch. The dot-dashed line represents the separation above which the secondary does not fill its Roche lobe.

Fig. 4 is a plot of initial secondary masses and separations and shows the constraints for the formation of a horizontal branch star via the coalescence of two degenerate helium cores during the second common envelope phase. For the evolutionary calculation we assumed a metallicity z = 0.001 and a primary mass of 1 M. The lines are the same as in Fig. 3. Fig. 4 shows that the constraints for formation of the merged object are only weakly dependent on the secondary mass.

In all of our calculations in which two degenerate helium cores merge during the second common envelope we find that the combined core mass is sufficient to ignite helium. In all the cases which form a horizontal branch star, the primary overflows its Roche lobe near the end of the giant branch. The core mass of the primary in this case is around 0.4 M. For the second common envelope phase we require the secondary to be a red giant and consequently its core mass is at least 0.2 M. Consequently the combined core masses are always sufficiently large enough to ignite helium (which requires $`M_\mathrm{c}>0.48\mathrm{M}_{\mathrm{}}`$).

We have run the evolution code with different values for the common envelope efficiency and neglecting the mass loss due to the wind. Neglecting the wind did not significantly change the evolutionary outcome of the systems and the range of parameter space was found to be similar for each model. The common envelope efficiency, however, determines the separations at which coalescence during the second common envelope phase will occur. The dot-dashed line above which the primary overflows its Roche lobe on the asymptotic giant branch is unaffected but the separation after the first common envelope phase is. This in turn affects whether a merger will occur in the second common envelope phase as this will depend on both the common envelope efficiency and the initial separation. It was found that decreasing the efficiency by a factor two shifted the parameter space in which coalescence occurs upwards by a factor 0.2 dex but without significantly changing its area. The common envelope efficiency consequently mainly affects the proportion of double degenerates which are formed relative to the horizontal branch stars. This demonstrates that the evolutionary scenario is a common outcome and not a consequence of the selection of certain parameters.

## 3 Role of the stellar encounters in a post-core-collapse cluster

### 3.1 Binary hardening due to stellar encounters

During stellar encounters the orbital separation is hardened as the third body interacts with the system (Davies 1995). Initially binaries will have a range of orbital separations but the wide binaries will become tighter pushing more binaries into the parameter space where they are likely to undergo two common envelope phases and merge to form a horizontal branch star. As an example in the cluster 47 Tuc a 1000 R binary can be ground down to 100 R (Davies & Benz 1995).

### 3.2 Modification of binary masses due to stellar encounters

When a star encounters a binary, the lowest mass star is usually ejected (Davies 1995). This could be either the third star or one of the binary components. The number of encounters depends on the encounter lifetime ($`\tau _{\mathrm{enc}}`$) which is given by

$$\tau _{\mathrm{enc}}=7\times 10^4\frac{n}{10^5/\mathrm{pc}^3}\left(\frac{\mathrm{M}_{\mathrm{}}}{M_1+M_2}\right)\frac{\mathrm{R}_{\mathrm{}}}{b_{\mathrm{min}}}\frac{v_{\mathrm{}}}{10\mathrm{km}\mathrm{s}^1}\mathrm{Myr},$$
(4)
where $`n`$ is the number density of stars, $`b_{\mathrm{min}}`$ is the distance of closest approach and $`v_{\mathrm{}}`$ is the stellar velocity. For typical parameters in the core of a PCC cluster, $`\tau _{\mathrm{enc}}`$ is approximately 400 Myr. This implies that during the main sequence lifetime of a 0.8 M star we can expect 20–40 encounters. To determine how encounters change the population of binary components we calculated the effect of a number of stellar encounters on a binary population. We assumed an initial population of stars with an initial mass function given by Eggleton, Fitchett & Tout (1989)

$$\frac{M}{M_{\mathrm{}}}=\frac{0.19x}{(1x)^{0.75}+0.032(1x)^{0.25}}$$
(5)
where $`x`$ is a random number between 0 and 1. This appears as a power-law for masses greater than 1 M and with a turnover at low masses (with a peak at $`M0.18`$M). Of our initial population of stars, we assumed 10% of them were in binaries. To mimic encounters we assumed that a binary has encounters with a star randomly drawn from the single star population. During an encounter if the third star is more massive than one of the binary components they will be exchanged (see Davies 1995) i.e., the remaining binary always consists of the two most massive stars. We simulated a number of encounters for each binary to build up an image of how the population changes with the number of encounters. The mass range which would have evolved onto the red giant branch in the recent past is 0.8 – 0.81 M. Fig. 5 shows the fraction of stars with this mass range which are the primary and secondary components of binary systems as a function of the number of encounters the binary population has experienced. The results did not significantly change if a different range of masses was chosen.

Table 1 shows how the proportions of primary and secondary components after 0, 20 and 40 encounters per binary. In addition the table shows the proportion of stars which may form red giant or horizontal branch stars. We assume only the single stars are likely to form red giants while horizontal branch stars will form from both the single stars and the secondaries of binary systems (through mergers). In the case of the horizontal branch these numbers are an upper limit and depend on the proportion of the binary population which have the necessary masses and separations to undergo a merger during the second common envelope phase. From Table 1 and Fig. 5 it is clear that in a non-PCC cluster the population of stars would be the same as the field with roughly 8% of the stars in the mass range of interest as the primaries of binaries and almost none as the secondaries. Consequently the red giant population will not be significantly depleted through binaries undergoing two common envelope phases in a non-PCC cluster. After 20 encounters, however, the situation is reversed. The secondary population has increased up to 16% while the primary component has dropped to 5%. Consequently more systems can form horizontal branch stars than red giants in a cluster whose members have had a number of stellar encounters. After 40 encounters, the secondary population has decreased to 7% and the primary population to 1%. However, the red giant branch will still be depleted relative to the horizontal branch.

This naturally explains why the depletion of red giants is only observed in PCC clusters. In non-PCC clusters there is not a significant number of binaries which have secondaries of the cluster turn-off mass. After a number of encounters, however, the situation is reversed and a number of turn-off mass stars are the secondaries in binaries. These binaries may undergo two common envelope phases with coalescence in the second common envelope. These mergers form horizontal branch stars and deplete the red giant population.

## 4 Discussion

We have shown that if a binary system undergoes two common envelope phases it is possible for a merger of the two cores to occur. We have assumed that this results in a horizontal branch star. The masses of the cores will add up to greater than the mass required for helium ignition. If all the helium ignited simultaneously then the nuclear energy generated would be greater than the binding energy of the star. However, as in the case of degenerate helium ignition at the end of the red giant branch, the burning timescale is large enough ($``$100 Myr) that this does not occur and the envelope remains bound.

If the cores merge near the end of the common envelope phase then the final star will have ejected a large portion of the hydrogen envelope and so the merged object may resemble an sdB star. Han et al. (2002) have investigated the formation channels for sdB stars and although one of the routes considered is dynamical mass transfer followed by a common envelope phase, two common envelope phases are not discussed (although the merger of two white dwarfs is one of the formation channels considered). Whether or not this is a main evolution channel is unknown as it may depend strongly on the initial separation.

The merged objects in our calculations will have a higher core mass and a lower envelope mass than normal horizontal branch stars. The higher core mass will result in a larger luminosity and so the SHBs observed by Bailyn (1994) and Shara et al. (1998) (or a subset of them) may be the merged objects which would form after coalescence during the second common envelope phase.

We have shown that in the cores of PCC clusters the conditions are such that depletion of the red giant population is possible. We have not, however, performed detailed binary population synthesis of the cluster as a whole. This is complicated due to the necessity of including stellar encounters, stellar evolution and binary interactions. The solution to this may be the use of N-body modelling codes. However all we are showing here is that this evolutionary route is both possible and likely in PCC cluster cores. In this paper we do not attempt the determination of the depletion due to common envelope mergers including the effects of encounters, exchanges and binary evolution simultaneously.

In addition to the horizontal branch stars formed through mergers our calculations predict the existence of a substantial double degenerate population. The double degenerate population we find in our calculations have two main components. The first is where both are helium white dwarfs. In this case the primary will have overflowed its Roche lobe near the end of the red giant branch while the secondary will typically overflow near the start of the red giant branch. This is a consequence of the inspiral during the common envelope phase resulting in the radii of the two stars at Roche lobe overflow being significantly different. The mass ratio in this case is typically between 1.5 and 2 and the final orbital period is a few hours.

The second component to the double degenerate population comes from carbon-oxygen and helium white dwarf binaries. These are formed when the primary overflows its Roche lobe on the asymptotic giant branch resulting in both the carbon-oxygen white dwarf-helium white dwarf binary and a larger separation. The mass ratio in these cases are between 1 and 1.5 with larger periods of a couple of days upwards. Comparing to the observed population of Maxted, Marsh & Moran (2002) we find that these most closely resemble their population in which the mass ratios are around one and the periods are from a few hours to slightly greater than a day. However, there is a significant difference in the ages determined from cooling models. In their models the white dwarfs are within a few 100 Myr of each other while in our models the more massive primary overflows its Roche lobe typically a few Gyr before the secondary. A possible remedy to this scenario may be binaries which are closer in mass to each other or which initially were much more massive. In both these cases they would become giants in similar time-scales allowing the white dwarfs to be formed closer in age to each other. The other populations of double degenerates, however, should still be present and it may be that there are selection effects which determine which binaries are observed or that the double degenerate population in the absence of a number of encounters is different..

Han (1998) has modelled the formation of double degenerates including two common envelope phases. This evolutionary route was found to produce a significant portion of the total double degenerate population. Nelemans et al. (2001) have also modelled the population of double degenerates although they use different assumptions concerning the inspiral during the first common envelope phase and the mass loss due to a wind.

Of the double degenerates which do form a number of them are close enough that they inspiral and mass transfer occurs. In the case of helium-helium white dwarfs the mass ratio is such that the mass transfer is always stable. In this case as mass is transferred the binary becomes wider again. The final binary in this case has a very low mass secondary and a $``$0.6 M helium white dwarf primary which could undergo ignition to form a carbon oxygen white dwarf.

## 5 Conclusions

We have described an evolutionary scenario in which the red giants are depleted in the cores of PCC clusters. The binary undergoes two common envelope phases during the second of which coalescence of two degenerate helium cores occurs. The combined mass of these cores is sufficient for helium burning to commence resulting in the formation of horizontal branch stars. The binary separations required for this to occur are around 100 R for a primary of mass 1 M and a range of secondary masses.

We have shown that this evolutionary scenario will be favoured in the cores of PCC due to the effects of stellar encounters. These encounters both harden the binary towards the required orbital separation and increase the population of secondaries which have the cluster turn-off mass.

## Acknowledgements

We thank Jarrod Hurley for providing his binary evolution code. Theoretical astrophysics at Leicester is supported by a PPARC rolling grant. MEB gratefully acknowledges a UK Astrophysical Fluids Facility (UKAFF) fellowship.