text:
# The Quantum Monty Hall Problem

## I Introduction

The well-known classical Monty Hall problem, also known under various other names ctk , is set in the context of a television game show. It can be seen as a two person game, in which a player P tries to win a prize, but a show master (or Quiz master) Q tries to make it difficult for her feminine . Like other games of this sort Wilkens1 ; Wilkens2  it can be “quantized”, i.e., its key elements can be formulated in a quantum mechanical context, allowing new strategies and new solutions. However, such quantizations are rarely unique, and depend critically on what is seen as a “key element”, and also on how actions which might change the system are formalized, corresponding to how, in the classical version, information is gained by “looking at something”. The Monty Hall problem is no exception, and there are already quantizations Hefei ; FA . The version we present in this paper was drafted independently, and indeed we come to a quite different conclusion. We discuss the relation between these two approaches and ours in more detail in Sec. VII below.

Our paper is organized as follows. In Sec. II we review the classical game, whose various steps are translated to the quantum context in Sec. III. In Sec. VII we discuss some alternative ideas to quantize the game. As in the classical version, computer simulations help to understand the structure of the game and its basic strategic options. We provide a Java simulation applet  of our version from the point of view of player P. There is a basic strategy for P in our version, which we call the classical strategy (Sec. IV), which guarantees a gain for her at least as good as in the classical game. The further analysis depends on how Q records the information about the initial preparation, which he needs later on in the game: if he uses a classical notepad, he has to choose his strategy very carefully, or P can improve her odds beyond the classical expectation (Sec. V). These possibilities for P are illustrated especially nicely by the simulation applet . The case of a quantum notepad is discussed in Sec. VI, and gives Q a simpler way to restrict P’s expectations to the classical value.

## II The Classical Monty Hall Problem

The classical Monty Hall problem is set in the context of a television game show. In the last round of the show, the candidates were given a chance to collect their prize (or lose it) in the following game:

1. Before the show the prize is hidden behind one of three closed doors. The show master knows where the prize is but, of course, the candidate does not.
2. The candidate is asked to choose one of the three doors, which is, however, not opened at this stage.
3. The show master opens another door, and shows that there is no prize behind it. (He can do this, because he knows where the prize is).
4. The candidate can now open one of the remaining doors to either collect her prize or lose.

Of course, the question is: should the candidate stick to her original choice or “change her mind” and pick the other remaining door? As a quick test usually shows, most people will stick to their first choice. After all, before the show master opened a door the two doors were equivalent, and they were not touched (nor was the prize moved). So they should still be equivalent. This argument seems so obvious that trained mathematicians and physicists fall for it almost as easily as anybody else embarass .

However, the correct solution by which the candidates can, in fact, double their chance of winning, is to always choose the other door. The quickest way to convince people of this is to compare the game with another one, in which the show master offers the choice of either staying with your choice or opening both other doors. Anybody would prefer that, especially if the show master courteously offers to open one of the doors for you. But this is precisely what happens in the original game when you always change to the other door.

## III The Quantum Monty Hall Problem

We will “quantize” only the key parts of the problem. That is, the prize and the players, as well as their publicly announced choices, will remain classical. The quantum version can even be played in a game show on classical TV.

The main quantum variable will be the position of the prize. It lies in a $`3`$-dimensional complex Hilbert space $``$, called the game space. We assume that an orthonormal basis is fixed for this space so that vectors can be identified by their components, but apart from this the basis has no significance for the game. A second important variable in the game is what we will call the show master’s notepad. This might be classical information describing how the game space was prepared, or it might be a quantum system, entangled with the prize. In the latter case, the show master is able to do a quantum measurement on his notepad, providing him with classical information about the prize, without moving the prize, in the sense that the player’s information about the prize is not changed by the mere fact that the show master “consults his notepad”. A measurement on an auxiliary quantum system, even if entangled with a system of interest, does not alter the reduced state of the system of interest. After the show master has consulted his notepad, we are in the same situation as if the notepad had been a classical system all along. As in the classical game, the situation for the player might change when the show master, by opening a door, reveals to some extent what he saw in his notepad. Opening a door corresponds to a measurement along a one dimensional projection on $``$.

The game proceeds in the following stages, closely analogous to the classical game:

1. Before the show the game space system is prepared quantum mechanically. Some information about this preparation is given to the show master Q. This can be in the form of another system, called the notepad, which is in a state correlated with the game space.
2. The candidate chooses some one dimensional projection $`p`$ on $``$.
3. The show master opens a door, i.e., he chooses a one dimensional projection $`q`$, and makes a Lüders/von Neumann measurement with projections $`q`$ and $`(1\mathrm{I}q)`$. In order to do this, he is allowed first to consult his notebook. If it is a quantum system, this means that he carries out a measurement on the notebook. The joint state of prize and notebook then change, but the traced out or reduced state of the prize does not change, as far as the player is concerned. Two rules constrain the show master’s choice of $`q`$: he must choose “another door” in the sense that $`qp`$; and he must be certain not to reveal the prize. The purpose of his notepad is to enable him to do this. After these steps, the game space is effectively collapsed to the two-dimensional space $`(1\mathrm{I}q)`$.
4. The player P can now choose a one dimensional projection $`p^{}`$ on $`(1\mathrm{I}q)`$, and the corresponding measurement on the game space is carried out. If it gives “yes” she collects the prize.

As in the classical case, the question is: how should the player choose the projection $`p^{}`$ in order to maximize her chance of winning? Perhaps it is best to try out a few options in a simulation, for which a Java applet is available applet . For the input to the applet, as well as for some of the discussion below it is easier to use unit vectors rather than one-dimensional projections. As standard notation we will use for $`p=|\mathrm{\Phi }\mathrm{\Phi }|`$ for the door chosen by the player, $`q=|\chi \chi |`$ for the door opened by $`Q`$, and $`r=|\mathrm{\Psi }\mathrm{\Psi }|`$ for the initial position of the prize, if that is defined.

From the classical case it seems likely that choosing $`p^{}=p`$ is a bad idea. So let us say that the classical strategy in this game consists of always switching to the orthogonal complement of the previous choice, i.e., to take $`p^{}=1\mathrm{I}qp`$. Note that this is always a projection because, by rule 3, $`p`$ and $`q`$ are orthogonal one dimensional projections. We will analyze this strategy in Sec. IV, which turns out to be possible without any specification of how the show master can guarantee not to stumble on the prize in step 3.

There are two main ways the show master can satisfy the rules. The first is that he chooses randomly the components of a vector in $``$, and prepares the game space in the corresponding pure state. He can then just take a note of his choice on a classical pad, so that in stage 3 he can compute a vector orthogonal to both the direction of the preparation and the direction chosen by the player. Q’s strategies in this case are discussed in Sec. IV. The second and more interesting way is to use a quantum notepad, i.e., another system with three dimensional Hilbert space $`𝒩`$, and to prepare a “maximally entangled state” on $`𝒩`$. Then until stage 3 the position of the prize is completely undetermined in the strong sense only possible in quantum mechanics, but the show master can find a safe door to open on $``$ by making a suitable measurement on $`𝒩`$. Q’s strategies in this case are discussed in Sec. VI.

## IV The classical strategy

To explain why the classical strategy works almost as in the classical version of the problem, we look more closely at the situation, when Q has just reached the decision which door $`q`$ he wants to open. His method of arriving at that decision is irrelevant. It must take into account the player’s choice $`p`$, as well as Q’s information about the preparation, and may involve some further randomness or even a quantum measurement on an auxiliary quantum system called the notepad. The choice of $`q`$ is announced publicly by “opening” that door, and needs to be known as classical information to P for her to choose a door $`p^{}`$ orthogonal to it. So we shall consider $`q`$ as a classical random variable. What we need to study are the correlations of this variable with outcomes of later measurements made on the game system. That is to say, we have a hybrid system described by a combination of classical and quantum variables.

There is a standard form for states on such systems, which we will need to use here. Let us consider first the classical part. Since $`p`$ was arbitrary and since the procedure used by the show master might involve classical randomization or quantum randomness (through measurement of a quantum notepad), $`q`$ has a probability distribution, denoted by $`w`$, which typically will depend on $`p`$. Thus the expectation of any real valued function $`f`$ of $`q`$ is to be computed as $`w(dq)f(q)`$. Next we are interested in the joint response of a classical detector (described by a characteristic function $`f`$) and a quantum detector, e.g., given by a one dimensional projection $`p^{}`$. By the theorem of Ozawaozawa  the additional information needed to compute such expectations is encoded by *conditional density operators* $`\rho _q`$. Then

$$w(dq)\mathrm{tr}(\rho _qp^{})f(q)$$
(1)
is the joint response probability for $`f`$ and $`p^{}`$. This too will typically depend on $`p`$, but we suppress that dependence also from the notation. Loosely speaking, $`\rho _q`$ is the density matrix which $`P`$ has to use for the game space after Q has announced his intention of opening door $`q`$. This is usually not the same conditional density operator as the one used by Q: Since Q has more classical information about the system, he may condition on that, leading to finer predictions. In contrast, $`\rho _q`$ is conditioned only on the publicly available information.

From $`w`$ and $`\rho _q`$ we can compute the marginal density operator for the quantum subsystem, describing measurements without consideration of the classical variable $`q`$. This is the mean density operator

$$\overline{\rho }=w(dq)\rho _q.$$
(2)
It will not depend on $`p`$ and it will be the same as the reduced density operator for the game space before the show master consults his notepad (he is not allowed to touch the prize), and even before the player chooses $`p`$ (which cannot affect the prize).

From the rules alone we know two things about the conditional density operators: firstly, that $`\mathrm{tr}(\rho _qq)=0`$: the show master must not hit the prize. Secondly, $`q`$ and $`p`$ must commute, so it does not matter which of the two we measure first. Thus a measurement of $`p`$ responds with probability $`w(dq)\mathrm{tr}(\rho _qp)=\mathrm{tr}(\overline{\rho }p)`$. Combining these two we get the overall probability $`w_c`$ for winning with the classical strategy as

$$w_c=w(dq)\mathrm{tr}(\rho _q(1\mathrm{I}pq))=1\mathrm{tr}(\overline{\rho }p).$$
(3)
If we assume that $`\overline{\rho }`$ is known to P, from watching the show sufficiently often, the best strategy for P is to choose initially the $`p`$ with the smallest expectation with respect to $`\overline{\rho }`$, just as in the classical game with uneven prize distribution it is best to choose initially the door least likely to contain the prize. If $`Q`$ on the other hand wants to minimize P’s gain ratings , he will choose $`\overline{\rho }`$ to be uniform, which in the quantum case means $`\overline{\rho }=\frac{1}{3}1\mathrm{I}`$, and hence $`w_c=2/3`$.

## V Strategies against classical notepads

In this section we consider the case that the show master records the prepared direction of the prize on a classical notepad. We will denote the one dimensional projection of this preparation by $`r`$. Then when he has to open a door $`q`$, he needs to choose $`qr`$ and $`qp`$. This is always possible in a three dimensional space. But unless $`p=r`$, he has no choice: $`q`$ is uniquely determined. This is the same as in the classical case, only that the condition “$`p=r`$”, i.e., that the player chooses exactly the prize vector typically has probability zero. Hence Q’s strategic options are not in the choice of $`q`$, but rather in the way he randomizes the prize positions $`r`$, i.e., in the choice of a probability measure $`v`$ on the set of pure states. In order to safeguard against the classical strategy he will make certain that the mean density operator $`\overline{\rho }=v(dr)r`$ is unpolarized ($`=\frac{1}{3}1\mathrm{I}`$). It seems that this is about all he has to do, and that the best the player can do is to use the classical strategy, and win $`2/3`$ of the time. However, this turns out to be completely wrong.

### V.1 Preparing along the axes

Suppose the show master decides that since the player can win as in the classical case, he might as well play classical as well, and save the cost for an expensive random generator. Thus he fixes a basis and chooses each one of the basis vectors with probability $`1/3`$. Then $`\overline{\rho }=\frac{1}{3}1\mathrm{I}`$, and there seems to be no giveaway. In fact, the two can now play the classical version, with P choosing likewise a projection along a basis vector.

But suppose she does not, and chooses instead the projection along the vector $`\mathrm{\Phi }=(1,1,1)/\sqrt{3}`$. Then if prize happens to be prepared in the direction $`\mathrm{\Psi }=(1,0,0)`$, the show master has no choice but to choose for $`q`$ the unique projection orthogonal to these two, which is along $`\chi =(0,1,1)`$. So when Q announces his choice, P only has to look which component of the vector is zero, to find the prize with certainty!

This might seem to be an artifact of the rather minimalistic choice of probability distribution. But suppose that $`Q`$ has settled for any arbitrary finite collection of vectors $`\mathrm{\Psi }_\alpha `$ and their probabilities. Then P can choose a vector $`\mathrm{\Phi }`$ which lies in none of the two dimensional subspaces spanned by two of the $`\mathrm{\Psi }_\alpha `$. This is possible, even with a random choice of $`\mathrm{\Phi }`$, because the union of these two dimensional subspaces has measure zero. Then, when Q announces the projection $`q`$, P will be able to reconstruct the prize vector with certainty: at most one of the $`\mathrm{\Psi }_\alpha `$ can be orthogonal to $`q`$. Because if there were two, they would span a two dimensional subspace, and together with $`\mathrm{\Phi }`$ they would span a three dimensional subspace orthogonal to $`q`$, which is a contradiction.

Of course, any choice of vectors announced with floating point precision is a choice from a finite set. Hence the last argument would seem to allow P to win with certainty in every realistic situation. However, this only works if she is permitted to ask for $`q`$ at any desired precision. So by the same token (fixed length of floating point mantissa) this advantage is again destroyed.

This shows, however, where the miracle strategies come from: by announcing $`q`$, the show master has not just given the player $`\mathrm{log}_23`$ bits of information, but an infinite amount, coded in the digits of the components of $`q`$ (or the vector $`\chi `$).

### V.2 Preparing real vectors

The discreteness of the probability distribution is not the key point in the previous example. In fact there is another way to economize on random generators, which proves to be just as disastrous for Q. The vectors in $``$ are specified by three complex numbers. So what about choosing them real for simplicity? An overall phase does not matter anyhow, so this restriction does not seem to be very dramatic.

Here the winning strategy for P is to take $`\mathrm{\Phi }=(1,i,0)/\sqrt{2}`$, or another vector whose real and imaginary parts are linearly independent. Then the vector $`\chi \mathrm{\Phi }`$ announced by Q will have a similar property, and also must be orthogonal to the real prize vector. But then we can simply compute the prize vector as the outer product of real and imaginary part of $`\chi `$.

For the vector $`\mathrm{\Phi }`$ specified above we find that if the prize is at $`\mathrm{\Psi }=(\mathrm{\Psi }_1,\mathrm{\Psi }_2,\mathrm{\Psi }_3)`$, with $`\mathrm{\Psi }_k\mathrm{I}\mathrm{R}`$, the unique vector $`\chi `$ orthogonal to $`\mathrm{\Phi }`$ and $`\mathrm{\Psi }`$ is connected to $`\mathrm{\Psi }`$ via the transformations

$`\chi `$ $``$ $`(\mathrm{\Psi }_3,i\mathrm{\Psi }_3,\mathrm{\Psi }_1+i\mathrm{\Psi }_2)`$ (4)
$`\mathrm{\Psi }`$ $``$ $`(\mathrm{}e\chi _3,\mathrm{}m\chi _3,\chi _1),`$ (5)
where “$``$” means “equal up to a factor”, and it is understood that an overall phase for $`\chi `$ is chosen to make $`\chi _1`$ real. This is also the convention used in the simulation applet , so Eq. (5) can be tried out as a universal cheat against show masters using only real vectors.

### V.3 Uniform distribution

The previous two examples have one thing in common: the probability distribution of vectors employed by the show master is concentrated on a rather small set of pure states on $``$. Clearly, if the distribution is more spread out, it is no longer possible for P to get the prize every time. Hence it is a good idea for Q to choose a distribution which is as uniform as possible. There is a natural definition of “uniform” distribution in this context, namely the unique probability distribution on the unit vectors, which is invariant under arbitrary unitary transformations uniform . Is this a good strategy for Q?

Let us consider the conditional density operator $`\rho _q`$, which depends on the two orthogonal projections $`p,q`$. It implicitly contains an average over all prize vectors leading to the same $`q`$, given $`p`$. Therefore, $`\rho _q`$ must be invariant under all unitary rotations of $``$ fixing these two vectors, which means that it must be diagonal in the same basis as $`p,q,(1\mathrm{I}pq)`$. Moreover, the eigenvalues cannot depend on $`p`$ and $`q`$, since every pair of orthogonal one dimensional projections can be transformed into any other by a unitary rotation. Since we know the average eigenvalue in the $`p`$-direction to be $`1/3`$, we find

$$\rho _q=\frac{1}{3}p+\frac{2}{3}(1\mathrm{I}pq).$$
(6)
Hence the classical strategy for P is clearly optimal. In other words, the pair of strategies: “uniform distribution for Q and classical strategy for P” is an equilibrium point of the game. We do not know yet, whether this equilibrium is unique, in other words: If Q does not play precisely by the uniform distribution: can P always improve on the classical strategy? We suppose that the answer to this question is yes; to find a proof of this conjecture has turned out, however, to be a hard problem which is still open.

## VI Strategies for Quantum notepads

### VI.1 Quantum notepad and entanglement

As briefly described in Sec. III, a notepad is quite generally another physical system, which by its initial preparation is brought into a state correlated with the game system, so that measurements on the notepad provide information about the game system as well. The classical notepad we were considering in the previous section can be formalized in this way, too, although it may appear a bit artificial to introduce a physical system on which some random variable is “written”. For the quantum notepads considered in this section, however, this view is mandatory.

The simplest form of a quantum notepad meeting Q’s requirements is another system with three dimensional Hilbert space $`𝒩`$. Denoting some bases in $``$ and $`𝒩`$ by ket vectors $`|1,|2,|3`$, and abbreviating $`|k|\mathrm{}|k\mathrm{}𝒩`$, we introduce the “maximally entangled vector”

$$\mathrm{\Omega }=\frac{1}{\sqrt{3}}\underset{k=1}{\overset{3}{}}|kk.$$
(7)
It clearly depends on the bases we have chosen, but not as much as one might expect. Its crucial property is the equation

$$(X1\mathrm{I})\mathrm{\Omega }=(1\mathrm{I}X^T)\mathrm{\Omega },$$
(8)
where the transpose $`X^T`$ is defined by $`k|X^T|\mathrm{}=\mathrm{}|X|k`$. Note that in this way an operator $`X`$ on $``$ becomes an operator $`X^T`$ on $`𝒩`$, and this becomes possible by using the same labels for the basis vectors in both spaces. If we now change the basis in $``$ by a unitary operator $`U`$, say, we find that there is a corresponding basis change on $`𝒩`$, leaving the vector $`\mathrm{\Omega }`$ invariant, namely $`(U^1U^T)\mathrm{\Omega }=\mathrm{\Omega }`$. Hence $`\mathrm{\Omega }`$ does not depend on the individual bases, but represents a particular way of identifying $``$ with $`𝒩`$ ccHilbert , and all bases accordingly. This is underlined by the observation that the restriction of this state to either factor is the completely unpolarized state, i.e.,

$$\mathrm{\Omega }|A1\mathrm{I}|\mathrm{\Omega }=\mathrm{\Omega }|1\mathrm{I}A|\mathrm{\Omega }=\frac{1}{3}\mathrm{tr}(A),$$
(9)
i.e., the unique state invariant under all unitary rotations.

Let us consider now a measurement on the notepad. It is described, like any measurement, by a positive operator valued measure. Let us take it discrete-valued for the moment to simplify the explanation. Thus the measurement is given by a collection of positive operators $`F_x`$ on $`𝒩`$, such that $`_xF_x=1\mathrm{I}`$. Here the labels “$`x`$” are the (classical) outcomes of the measurement, and $`\mathrm{tr}(\rho F_x)`$ is interpreted as the probability of getting this result, when measuring on systems prepared according to $`\rho `$. How could Q now infer from this result a safe door $`q`$ for him to open in the game? This would mean that $`F_x`$ measured on $`𝒩`$, and $`q`$ measured on $``$ never give a simultaneous positive response, when measured in the state $`\mathrm{\Omega }`$, i.e.,

$$0=\mathrm{\Omega }|qF_x|\mathrm{\Omega }=\mathrm{\Omega }|1\mathrm{I}F_xq^T|\mathrm{\Omega }=\frac{1}{3}\mathrm{tr}(q^TF_x).$$
(10)
Since $`F_x`$ and $`q^T`$ are both positive, this equivalent to $`F_xq^T=0`$. Of course, Q’s choice must also satisfy the constraint $`qp`$. There are different ways of arranging this, which we discuss in the following two subsections.

### VI.2 Equivalence if observable is chosen beforehand

Suppose Q chooses the measurement beforehand, and let us suppose it is discrete, as before. Then for every outcome $`x`$ and every $`p`$ he must be able to find a one dimensional projection satisfying both constraints $`F_x^Tq=0`$ and $`qp=0`$. Clearly, this requires that $`F_x`$ has at least a two dimensional null space, i.e., $`F_x=|\varphi _x\varphi _x|`$, with a possibly non-normalized vector $`\varphi _x𝒩`$. It will be convenient to take the vectors $`\varphi _x`$ to be normalized, and to define $`F_x=v_x|\varphi _x\varphi _x|`$ with factors $`v_x`$ summing to $`3`$, the dimension of the Hilbert space (take the trace of the normalization condition for $`F`$). We can further simplify this structure, by identifying outcomes $`x`$ with the same $`\varphi _x`$, since for these the same projection $`q`$ has to be chosen anyhow. We can therefore drop the index “$`x`$”, and consider the measure to be defined directly on the set of one dimensional projections. But this is precisely the structure we had used to describe a classical notepad. This is not an accidental analogy: apart from taking transposes as in equation (8) this measure has precisely the same strategic meaning as the measure of a classical notepad.

This is not surprising: if the observable is chosen beforehand, it does not matter whether the show master actually performs the measurement before or after the player’s choice. But if he does it before P’s choice, we can just as well consider this measurement with its classical output as part of the preparation of a classical notepad, in which the result is recorded.

### VI.3 Simplified strategy for Q

Obviously the full potential of entanglement is used only, when Q chooses his observable after P’s choice. Since the position of the prize is “objectively undetermined” until then, it might seem that there are now ways to beat the 2/3 limit. However, the arguments for the classical strategy hold in this case as well. So the best Q can hope for are some simplified strategies. For example, he can now get away with something like measuring along axes only, even though for classical notepads using “axes only” was a certain loss for Q.

We can state this in a stronger way, by introducing tougher rules for Q: In this variant P not only picks the direction $`p`$, but also two more projections $`p^{}`$ and $`p^{\prime \prime }`$ such that $`p+p^{}+p^{\prime \prime }=1\mathrm{I}`$. Then Q is not only required to open a door $`qp`$, but we require that either $`q=p^{}`$ or $`q=p^{\prime \prime }`$. It is obvious how Q can play this game with an entangled notepad: he just uses the transposes of $`p,p^{},p^{\prime \prime }`$ as his observable. Then everything is as in the classical version, and the equilibrium is again at $`2/3`$.

## VII Alternative versions and quantizations of the game

### VII.1 Variants arising already in the classical case

Some variants of the problem can also be considered in the classical case, and they tend to trivialize the problem, so that P’s final choice becomes equivalent to “Q has prepared a coin, and P guesses heads or tails”. Here are some possibilities, formulated in a way applying both to the classical and the quantum version.

* Q is allowed to touch the prize after P made her first choice. Clearly, in this case Q can reshuffle the system, and equalize the odds between the remaining doors. So no matter what P chooses, there will be a 50% chance for getting the prize.
* Q is allowed to open the door first chosen by P. Then there is no way P’s first choice enters the rules, and we may analyze the game with stage 2 omitted, which is entirely trivial.
* Q may open the door with the prize, in which case the game starts again. Since Q knows where the prize is, this is the same as allowing him to abort the round, whenever he does not like what has happened so far, e.g., if he does not like the relative position of prize and P’s choice. In the classical version he could thus cancel 50% of the cases, where P’s choice is not the prize, thus equalizing the chances for P’s two pure strategies. Similar possibilities apply in the quantum case.

### VII.2 Variants in which classical and quantum behave differently

* Q may open the door with the prize, in which case P gets the prize. In the classical version, revealing the prize is then the worst possible pure strategy, so mixing in a bit of it would seem to make things always worse for Q. Then although increasing Q’s options in principle can only improve things for $`Q`$, one would advise him not to use the additional options. This is assuming, though, that in the remaining cases Q sticks to his old strategy. However, even classically, the relaxed rule gives him some new options: He can simply ignore the notepad, and open any door other than $`p`$. Then the game becomes effectively “P and Q open a door each, and P gets all prizes”. Assuming uniform initial distribution of prizes this gives the same $`2/3`$ winning chance as in the original game.

The corresponding quantum strategy works in the same way. Assuming, for simplicity, a uniform mean density operator $`\overline{\rho }=\frac{1}{3}1\mathrm{I}`$, Q’s strategy of ignoring his prior information will give the classical $`2/3`$ winning chance for P. But this is a considerable improvement for Q in cases where a non-uniform probability distribution of pure states previously gave Q a 100% chance of winning. So in the quantum case, doing two seemingly stupid things together amounts to a good strategy for Q: firstly, sometimes revealing the prize for P, and secondly ignoring all prior information.

Note that this strategy is optimal for Q, because the classical strategy still guarantees the $`2/3`$ winning chance for P. This can be seen with the same arguments as in Section IV. The only difference is that $`\mathrm{tr}(\rho _qq)`$ can be nonzero, since Q may open the door with the prize. However in this case P wins and we get instead of Equation (3)

$`w_c`$ $`=`$ $`{\displaystyle w(dq)\mathrm{tr}(\rho _q(1\mathrm{I}pq))}+\mathrm{tr}(\rho _qq)`$ (11)
$`=`$ $`1\mathrm{tr}(\overline{\rho }p)={\displaystyle \frac{2}{3}}`$ (12)
* As Q opens the door he is allowed to make a complete von Neumann measurement. Classically, it would make no difference if the doors were completely transparent to the show master. He would not even need a pad then, because he could always look where the prize is. But “looking” is never innocent in quantum mechanics, and in this case it is tantamount to moving the prize around. So let us make it difficult for Q, by insisting that the initial preparation is along a fixed vector, known also to P, and that Q not only has to announce the direction $`q`$ of the door he opens, but also the projections $`q^{}q`$ and $`q^{\prime \prime }=1\mathrm{I}qq^{}`$ entering in the complete von Neumann measurement, which takes an arbitrary density operator $`\rho `$ to

$$\rho q\rho q+q^{}\rho q^{}+q^{\prime \prime }\rho q^{\prime \prime }.$$
(13)
Moreover, we require as before, that $`q`$ is orthogonal both to $`p`$ and to the prize. The only thing remaining secret is which of the projections $`q^{}`$ and $`q^{\prime \prime }`$ has detected the presence of the prize (This simply would allow P to open that door and collect). Q’s simple strategy is now to choose $`q`$ as before. The position of $`p`$ is irrelevant for his choice of $`p^{}`$ and $`p^{\prime \prime }`$: he will just take these directions at $`45^{}`$ to the prize vector. This will result in the unpolarized density operator $`(q^{}+q^{\prime \prime })/2`$, and no matter what P chooses, her chances of hitting the prize will be $`1/2`$. She will probably feel cheated, and she is, because even though she knows exactly where the prize was initially, the strategy “choose the prize, and stick to this choice” no longer works.

### VII.3 Two published versions

* The quantization proposed in Hefei  is probably Hefei2  closely related to the second variant of the last Subsection: After Q has opened one door he is allowed to perform an arbitrary von Neumann measurement on the remaining two-dimensional subspace – i.e. he “looks” where the prize is. In the classical case this is an allowed (but completely superfluous) step. In the quantum case, however, the prize is shuffled around and we end up with a variant from Subsection VII.1: $`Q`$ is allowed to move the prize. In other words, the final result of the game is completely independent of the steps prior to this measurement and the whole game is reduced to coin tossing – which is not very interesting.
* A completely different quantization of the game is given in FA . In contrast to our approach, the moves available to Q and P are here not preparations and mesurements but *operations* on a tripartite system which is initially in the pure state $`\psi _Q_P_O`$ (and different choices for $`\psi `$ lead to different variants of the game). The Hilbert spaces $`_Q,_P`$ and $`_O`$ describe the doors where $`Q`$ hides the prize, which P chooses in the second step and which Q opens afterwards and the gameplay is described by the unitary operator

$$U=(\mathrm{cos}\gamma U_S+\mathrm{sin}\gamma U_N)U_O(U_QU_P1\mathrm{I}).$$
(14)
$`U_Q`$ and $`U_P`$ are arbitrary unitaries, describing Q’s and $`P`$’s initial choice, $`U_O`$ is the (fixed) opening box operator and $`U_S`$ respectively $`U_N`$ are P’s “switching” and “not-switching” operators. The payoff is finally given as the expectation value of an appropriate observable $`\$`$ (for a precise definition of $`U_O`$, $`U_S`$, $`U_N`$ and $`\$`$ see FA ). The basic idea behind this scheme is quite different from ours and a comparison of results is therefore impossible. Nevertheless, this is a nice example which shows that quantizing a classical game is very non-unique.

## Acknowledgments

The idea for this paper came up in a bar during a very stimulating workshop “Qrandom”, conducted at the research center EURANDOM at Eindhoven, Feb. 12-16, 2001. Financial support by the center, is gratefully acknowledged. We whish to thank also M. Guta for useful discussion about the uniqueness problem discussed at the end of Section V.


format: text

text:
# 1 How Did We Get Here?

## 1 How Did We Get Here?

Cosmology is currently in its most exciting period ever. However, the past hundred years haven’t been too bad either. In 1916, Einstein introduced general relativity, the first theoretical framework up to describing the evolution of the Universe. In the next decade, Hubble established the extragalactic nature of the nebulae and put forth the first systematic evidence for the expansion of the Universe. This burst of activity was powered by ideas – general relativity – and technology – big telescopes on high mountains – and marked the beginning of modern cosmology.

Cosmology then stalled for some twenty years. During that time, the 100-inch Mt. Wilson telescope and its counterpart at the Lick Observatory charted only a few hundred galaxies, with redshifts less than $`z=0.1`$ . Most of the Universe lay beyond their reach. While some important new theoretical ideas were introduced, most notably the idea of a hot big bang by Gamow and his collaborators, theory did little to push things forward either.

First light at the 200-inch Hale telescope on Palomar and the steady state vs. hot big bang controversy signaled the beginning of a new period of discovery around 1950. Once again, it was the combination of theory and observation that propelled the field forward. The discovery of quasars and radio-source counts murdered cosmology’s most beautiful theory – the steady state – in the early 1960s. I note the important role played here by a strong theory – that is, one that makes sharp predictions and is therefore easily falsified. Because the steady state predicts a nonevolving Universe, any evidence of evolution can falsify it (here, the distribution of radio sources vs. their strength and the large abundance of quasars seen at high redshift).

The discovery of the cosmic background radiation in 1964 by Penzias and Wilson broke things wide open, and cosmology became a legitimate branch of physics. The details of Gamow’s nucleosynthesis scheme were worked out in detail and became big-bang nucleosynthesis, and the large mass fraction of 4He predicted by the hot big-bang model became its first success. In Chapter 15 of his influential text, Gravitation and Cosmology, Steven Weinberg laid out “The Standard Cosmology” and coined the term . The Standard Cosmology meant the hot big-bang model, from shortly before the epoch of big-bang nucleosynthesis to the present. A convergence of ideas (a better understanding of general relativity and the careful application of physics to the early Universe) and observations (the discovery of the cosmic microwave background) again had powered a significant advance our understanding of the Universe.

According to Weinberg’s same text, the period before BBN ($`t\begin{array}{c}<\hfill \\ \hfill \end{array}10^4`$sec) was “cosmos incognito” (cf, Section 15.11). He recognized quite correctly that the key to further progress was a better understanding of the elementary particles. A subatomic world of strongly interacting particles with exponentially rising numbers, which was the world view of subatomic physics then, leads to a maximum temperature and a breakdown of any simple statistical mechanical treatment.

The breakthrough came in the mid 1970s with the emergence of the standard model of particle physics and its point-like quark/lepton constituents with asymptotically weak interactions. The grander ideas about unification of the strong and electroweak interactions (GUTs) that came somewhat later were even more influential. Asymptotically free gauge theories, GUTs and the sturdy framework of the hot big-bang fueled a decade of very fertile speculation about the early Universe in the 1980s (“the go-go junk bond days of early-Universe cosmology”).

There was optimism that the marriage of ideas about unification with early-Universe cosmology could solve some of the most pressing questions in cosmology. The puzzles at the time included : What is the origin of the baryon asymmetry? Why is the Universe so smooth, flat, and old? How did the primeval lumpiness that seeded all structure arise? What is the dark matter? And the marriage brought a new problem: the glut of superheavy magnetic monopoles expected from the GUT phase transition (about one per baryon!).

Grand ideas, clever scenarios, and interesting schemes were suggested – topological defects, baryogenesis, particle dark matter, phase transitions, inflation, superheavy relics, superconducting cosmic strings, unstable relics, colored relics, relics produced by the decay of other relics, hot dark matter, cold dark matter, warm dark matter, shadow worlds, mirror worlds, parallel universes, and on and on. It was a fantastic time for theorists.

A handful of these ideas stood the test of time. In the go-go 80s, that meant surviving more than a month without being falsified, being replaced by a better idea, or going out of style. The survivors – particle dark matter, inflation, topological defects, and baryogenesis – began to form the basis of a new cosmological framework. However, only a few years later, the very beautiful idea that topological defects seeded large-scale structure was killed by cosmic microwave background measurements that began to show a series of well formed acoustic peaks. (Only adiabatic density perturbations lead to such a structure in the CMB power spectrum.)

Inflation and cold dark matter survived the cut. Moreover, they were expansive and eminently testable. Theorists like to think (and with some good reason) that the tremendous growth in observational cosmology over the past decade has something to do with how powerful their ideas are. Technology – the advent of large format CCDs, 6-meter, 8-meter, and 10-meter ground based telescopes, the Hubble Space Telescope, HEMTs, bolometers and large-scale computing – of course played a role too. An avalanche of data, which has made cosmological phenomenology possible and the term “precision cosmology” a reality, started in the mid 1990s. And there is no end in sight.

We are now in the midst of a great period of cosmic discovery. Once again, it came about through a convergence of ideas and technology. Thus far, we have determined the basic features of our Universe, which are pointing to a new standard cosmological model. While we learned much about the Universe, we still have much more to understand.

## 2 The New Cosmology

Over the past three years a New Cosmology has been emerging. It incorporates the highly successful standard hot big-bang cosmology  and may extend our understanding of the Universe to times as early as $`10^{32}`$sec, when the largest structures in the Universe were still subatomic quantum fluctuations.

This New Cosmology is characterized by

* Flat, critical density accelerating Universe
* Early period of rapid expansion (inflation)
* Density inhomogeneities produced from quantum fluctuations during inflation
* Composition: 2/3 dark energy; 1/3 dark matter; 1/200 bright stars
* Matter content: $`(29\pm 4)`$% cold dark matter; $`(4\pm 1)`$% baryons; $`\begin{array}{c}>\hfill \\ \hfill \end{array}0.3`$% neutrinos
* $`T_0=2.725\pm 0.001`$K
* $`t_0=14\pm 1`$Gyr
* $`H_0=72\pm 7\mathrm{km}\mathrm{s}^1\mathrm{Mpc}^1`$

The New Cosmology is not as well established as the standard hot big-bang cosmology. However, the evidence is growing.

### 2.1 Mounting Evidence: Recent Results

The position of the first acoustic peak in the multipole power spectrum of the anisotropy of the cosmic microwave background (CMB) radiation provides a powerful means of determining the global curvature of the Universe. With the recent DASI observations of CMB anisotropy on scales of one degree and smaller, the evidence that the Universe is at most very slightly curved is quite firm . The curvature radius of the Universe ($`R_{\mathrm{curv}}`$) and the total energy density parameter $`\mathrm{\Omega }_0=\rho _{\mathrm{TOT}}/\rho _{\mathrm{crit}}`$, are related:

$$R_{\mathrm{curv}}=H_0^1/|\mathrm{\Omega }_01|^{1/2}$$

The spatial flatness is expressed as $`\mathrm{\Omega }_0=1.0\pm 0.04`$, or said in words, the curvature radius is at least 50 times greater than the Hubble radius.

I will discuss the evidence for accelerated expansion and dark energy later.

The series of acoustic peaks in the CMB multipole power spectrum and their heights indicate a nearly scale-invariant spectrum of adiabatic density perturbations with $`n=1\pm 0.07`$. Nearly scale-invariant density perturbations and a flat Universe are two of the three hallmarks of inflation. Thus, we are beginning to see the first significant experimental evidence for inflation, the driving idea in cosmology for the past two decades.

The striking agreement of the BBN determination of the baryon density from measurements of the primeval deuterium abundance , $`\mathrm{\Omega }_Bh^2=0.020\pm 0.001`$, with those from from recent CMB anisotropy measurements , $`\mathrm{\Omega }_Bh^2=0.022\pm 0.004`$, make a strong case for a small baryon density, as well as the consistency of the standard cosmology ($`h=H_0/100\mathrm{km}\mathrm{sec}^1\mathrm{Mpc}^1`$). There can now be little doubt that baryons account for but a few percent of the critical density.

Our knowledge of the total matter density is improving, and becoming less linked to the distribution of light. This makes determinations of the matter less sensitive to the uncertain relationship between the clustering of mass and of light (what astronomers call the bias factor $`b`$) . Both the CMB and clusters of galaxies allow a determination of the ratio of the total matter density (anything that clusters – baryons, neutrinos, cold dark matter) to that in baryons alone: $`\mathrm{\Omega }_M/\mathrm{\Omega }_B=7.2\pm 2.1`$ (CMB) , $`9\pm 1.5`$ (clusters) . Not only are these numbers consistent, they make a very strong case for something beyond quark-based matter. When combined with our knowledge of the baryon density, one infers a total matter density of $`\mathrm{\Omega }_M=0.33\pm 0.04`$ .

The many successes of the cold dark matter (CDM) scenario – from the sequence of structure formation (galaxies first, clusters of galaxies and larger objects later) and the structure of the intergalactic medium, to its ability to reproduce the power spectrum of inhomogeneity measured today – makes it clear that CDM holds much, if not all, of the truth in describing the formation of structure in the Universe.

The two largest redshift surveys, the Sloan Digital Sky Survey (SDSS) and the 2-degree Field project (2dF), have each recently measured the power spectrum using samples of more than 100,000 galaxies and found that it is consistent with that predicted in a flat accelerating Universe comprised of cold dark matter . The SDSS will eventually use a sample of almost one million galaxies to probe the power spectrum. \[Interestingly enough, according to the 2dF Collaboration, bias appears to be a small effect, $`b=1.0\pm 0.09`$ .\]

All of this implies that whatever the dark matter particle is, it moves slowly (i.e., the bulk of the matter cannot be in the form of hot dark matter such as neutrinos) and interacts only weakly (e.g., with strength much less than electromagnetic) with ordinary matter.

The evidence from SuperKamiokande  for neutrino oscillations makes a strong case that neutrinos have mass ($`_im_\nu \begin{array}{c}>\hfill \\ \hfill \end{array}0.1`$eV) and therefore contribute to the mass budget of the Universe at a level comparable to, or greater than, that of bright stars. Particle dark matter has moved from the realm of a hypothesis to a quantitative question – how much of each type of particle dark matter is there in the Universe? Structure formation in the Universe (especially the existence of small scale structure) suggests that neutrinos contribute at most 5% or 10% of the critical density, corresponding to $`_im_nu=_im_\nu /90h^2\mathrm{eV}\begin{array}{c}<\hfill \\ \hfill \end{array}5`$eV .

Even the age of the Universe and the pesky Hubble constant have been reined in. The uncertainties in the ages of the oldest globular clusters have been better identified and quantified, leading to a more precise age, $`t_0=13.5\pm 1.5`$Gyr . The CMB can be used to constrain the expansion age, independent of direct measurements of $`H_0`$ or the composition of the Universe, $`t_{\mathrm{exp}}=14\pm 0.5`$Gyr .

A host of different techniques are consistent with the Hubble constant determined by the HST key project, $`H_0=72\pm 7\mathrm{km}\mathrm{s}^1\mathrm{Mpc}^1`$. Further, the error budget is now well understood and well quantified . \[The bulk of the $`\pm 7`$ uncertainty is systematic, dominated by the uncertainty in the distance to the LMC and the Cepheid period – luminosity relation.\] Moreover, the expansion age derived from this consensus Hubble constant, which depends upon the composition of the Universe, is consistent with the previous two age determinations.

The poster child for precision cosmology continues to be the present temperature of the CMB. It was determined by the FIRAS instrument on COBE to be: $`T_0=2.725\pm 0.001`$K . Further, any deviations from a black body spectrum are smaller than 50 parts per million. Such a perfect Planckian spectrum has made any noncosmological explanation untenable.

### 2.2 Successes and Consistency Tests

To sum up, we have determined the basic features of the Universe: the cosmic matter/energy budget; a self consistent set of cosmological parameters with realistic errors; and the global curvature. Two of the three key predictions of inflation – flatness and nearly scale-invariant, adiabatic density perturbations – have passed their first significant tests. Last but not least, the growing quantity of precision data are now testing the consistency of the Friedmann-Robertson-Walker framework and General Relativity itself.

In particular, the equality of the baryon densities determined from BBN and CMB anisotropy is remarkable. The first involves nuclear physics when the Universe was seconds old, while the latter involves gravitational and classical electrodynamics when the Universe was 400,000 years old.

The entire framework has been tested by the existence of the aforementioned acoustic peaks in the CMB angular power spectrum. They reveal large-scale motions that have remained coherent over hundreds of thousands of years, through a delicate interplay of gravitational and electromagnetic interactions.

Another test of the basic framework is the accounting of the density of matter and energy in the Universe. The CMB measurement of spatial flatness implies that the matter and energy densities must sum to the critical density. Measurements of the matter density indicate $`\mathrm{\Omega }_M=0.33\pm 0.04`$; and measurements of the acceleration of the Universe from supernovae indicate the existence of a smooth dark energy component that accounts for $`\mathrm{\Omega }_X0.67`$. \[The amount of dark energy inferred from the supernova measurements depends its equation of state; for a cosmological constant, $`\mathrm{\Omega }_\mathrm{\Lambda }=0.8\pm 0.16`$.\]

Finally, while cosmology has in the past been plagued by “age crises” – time back to the big bang (expansion age) apparently less than the ages of the oldest objects within the Universe – today the ages determined by very different and completely independent techniques point to a consistent age of 14 Gyr.

## 3 Mysteries

Cosmological observations over the next decade will test – and probably refine – the New Cosmology . If we are fortunate, they will also help us to make better sense of it. At the moment, the New Cosmology has presented us with a number of cosmic mysteries – opportunities for surprises and new insights. Here I will quickly go through my list, and save the most intriguing to me – dark energy – for its own section.

### 3.1 Dark Matter

By now, the conservative hypothesis is that the dark matter consists of a new form of matter, with the axion and neutralino as the leading candidates. That most of the matter in the Universe exists in a new form of matter – yet to be detected in the laboratory – is a bold and untested assertion.

Experiments to directly detect the neutralinos or axions holding our own galaxy together have now reached sufficient sensitivity to probe the regions of parameter space preferred by theory. In addition, the neutralino can be created by upcoming collider experiments (at the Tevatron or the LHC), or detected by its annihilation signatures – high-energy neutrinos from the sun, narrow positron lines in the cosmic rays, and gamma-ray line radiation .

While the CDM scenario is very successful there are some nagging problems. They may point to a fundamental difficulty or may be explained by messy astrophysics . The most well known of these problems are the prediction of cuspy dark-matter halos (density profile $`\rho _{\mathrm{DM}}1/r^n`$ as $`r0`$, with $`n11.5`$) and the apparent prediction of too much substructure. While there are plausible astrophysical explanations for both problems , they could indicate an unexpected property of the dark-matter particle (e.g., large self-interaction cross section , large annihilation cross section , or mass of around 1 keV). While I believe it is unlikely, these problems could indicate a failure of the particle dark-matter paradigm and have their explanation in a radical modification of gravity theory .

I leave for the “astrophysics to do list” an accounting of the dark baryons. Since $`\mathrm{\Omega }_B0.04`$ and $`\mathrm{\Omega }_{}0.005`$, the bulk of the baryons are optically dark. In clusters, the dark baryons have been identified: they exists as hot, x-ray emitting gas. Elsewhere, the dark baryons have not yet been identified. According to CDM, the bulk of the dark baryons are likely to exist as hot/warm gas associated with galaxies, but this gas has not been detected. \[Since clusters account for only about 5 percent of the total mass, the bulk of the dark baryons are still not accounted for.\]

### 3.2 Baryogenesis

The origin of quark-based matter is not yet fully understood. We do know that the origin of ordinary matter requires a small excess of quarks over antiquarks (about a part in $`10^9`$) at a time at least as early as $`10^6`$sec, to avoid the annihilation catastrophe associated with a baryon symmetric Universe . If the Universe underwent inflation, the baryon asymmetry cannot be primeval, it must be produced dynamically (“baryogenesis”) after inflation since any pre-inflation baryon asymmetry is diluted away by the enormous entropy production associated with reheating.

Because we also now know that electroweak processes violate $`B+L`$ at a very rapid rate at temperatures above $`100`$GeV or so, baryogenesis is more constrained than when the idea was introduced more than twenty years ago. Today there are three possibilities: 1) produce the baryon asymmetry by GUT-scale physics with $`BL0`$ (to prevent it being subsequently washed away by $`B+L`$ violation); 2) produce a lepton asymmetry ($`L0`$), which is then transmuted into the baryon asymmetry by electroweak $`B+L`$ violation ; or 3) produce the baryon asymmetry during the electroweak phase transition using electroweak $`B`$ violation .

While none of the three possibilities can be ruled out, the second possibility looks most promising, and it adds a new twist to the origin of quark-based matter: We are here because neutrinos have mass. \[In the lepton asymmetry first scenario, Majorana neutrino mass provides the requisite lepton number violation.\] The drawback of the first possibility is the necessity of a high reheat temperature after inflation, $`T_{\mathrm{RH}}10^5`$GeV, which is difficult to achieve in most models of inflation. The last possibility, while very attractive because all the input physics might be measurable at accelerator, requires new sources of $`CP`$ violation at TeV energies as well as a strongly first-order electroweak phase transition (which is currently disfavored by the high mass of the Higgs) .

### 3.3 Inflation

There are still many questions to be answered about inflation, including the most fundamental: did inflation (or something similar) actually take place!

A powerful program is in place to test the inflationary framework. Testing framework involves testing its three robust predictions: spatially flat Universe; nearly-scale invariant, nearly power-law spectrum of Gaussian adiabatic, density perturbations; and a spectrum of nearly scale-invariant gravitational waves.

The first two predictions are being probed today and will be probed much more sharply over the next decade. The value of $`\mathrm{\Omega }_0`$ should be determined to much better than 1 percent. The spectral index $`n`$ that characterizes the density perturbations should be measured to percent accuracy.

Generically, inflation predicts $`|n1|𝒪(0.1)`$, where $`n=1`$ corresponds to exact scale invariance. Likewise, the deviations from an exact power-law predicted by inflation , $`|dn/d\mathrm{ln}k|10^410^2`$ will be tested. The CMB and the abundance of rare objects such as clusters of galaxies will allow Gaussianity to be tested.

Inflationary theory has given little guidance as to the amplitude of the gravitational waves produced during inflation. If detected, they are a smokin’ gun prediction. Their amplitude is directly related to the scale of inflation, $`h_{\mathrm{GW}}H_{\mathrm{inflation}}/m_{\mathrm{Pl}}`$. Together measurements of $`n1`$ and $`dn/\mathrm{ln}k`$, they can reveal much about the underlying scalar potential driving inflation. Measuring their spectral index – a most difficult task – provides a consistency test of the single scalar-field model of inflation .

### 3.4 The Dimensionality of Space-time

Are there additional spatial dimensions beyond the three for which we have very firm evidence? I cannot think of a deeper question in physics today. If there are new dimensions, they are likely to be relevant for cosmology, or at least raise new questions in cosmology (e.g., why are only three dimensions large? what is going on in the bulk? and so on). Further, cosmology may well be the best means for establishing the existence of extra dimensions.

### 3.5 Before Inflation, Other Big-bang Debris, and Surprises

Only knowing everything there is to know about the Universe would be worse than knowing all the questions to ask about it. Without doubt, as our understanding deepens, new questions and new surprises will spring forth.

The cosmological attraction of inflation is its ability to make the present state of the Universe insensitive to its initial state. However, should we establish inflation as part of cosmic history, I am certain that cosmologists will begin asking what happened before inflation.

Progress in cosmology depends upon studying relics. We have made much of the handful we have – the light elements, the baryon asymmetry, dark matter, and the CMB. The significance of a new relic cannot be overstated. For example, detection of the cosmic sea of neutrinos would reveal the Universe at 1 second.

Identifying the neutralino as the dark matter particle and determining its properties at an accelerator laboratory would open a window on the Universe at $`10^8`$sec. By comparing its relic abundance as derived from its mass and cross section with its actual abundance measured in the Universe, one could test cosmology at the time the neutralino abundance was determined.

And then there may be the unexpected. Recently, a group reported evidence for a part in $`10^5`$ difference in the fine-structure constant at redshifts of order a few from its value today . I remain skeptical, given possible astrophysical explanations, other much tighter constraints to the variation of $`\alpha `$ (albeit at more recent times), and the absence of a reasonable theoretical model. For reference, I was also skeptical about the atmospheric neutrino problem because of the need for large-mixing angles.

## 4 Dark Energy: Seven Things We Know

The dark energy accounts for 2/3 of the stuff in the Universe and determines its destiny. That puts it high on the list of outstanding problems in cosmology. Its deep connections to fundamental physics – a new form of energy with repulsive gravity and possible implications for the divergences of quantum theory and supersymmetry breaking – put it very high on the list of outstanding problems in particle physics .

What then is dark energy? Dark energy is my term for the causative agent for the current epoch of accelerated expansion. According to the second Friedmann equation,

$$\frac{\ddot{R}}{R}=\frac{4\pi G}{3}\left(\rho +3p\right)$$
(1)
this stuff must have negative pressure, with magnitude comparable to its energy density, in order to produce accelerated expansion \[recall $`q=(\ddot{R}/R)/H^2`$; $`R`$ is the cosmic scale factor\]. Further, since this mysterious stuff does not show its presence in galaxies and clusters of galaxies, it must be relatively smoothly distributed.

That being said, dark energy has the following defining properties: (1) it emits/absorbs no light; (2) it has large, negative pressure, $`p_X\rho _X`$; (3) it is approximately homogeneous (more precisely, does not cluster significantly with matter on scales at least as large as clusters of galaxies); and (4) it is very mysterious. Because its pressure is comparable in magnitude to its energy density, it is more “energy-like” than “matter-like” (matter being characterized by $`p\rho `$). Dark energy is qualitatively very different from dark matter, and is certainly not a replacement for it.

### 4.1 Two Lines of Evidence for an Accelerating Universe

Two independent lines of reasoning point to an accelerating Universe. The first is the direct evidence based upon measurements of type Ia supernovae carried out by two groups, the Supernova Cosmology Project  and the High-$`z`$ Supernova Team . These two teams used different analysis techniques and different samples of high-$`z`$ supernovae and came to the same conclusion: the expansion of the Universe is speeding up, not slowing down.

The recent serendipitous discovery of a supernovae at $`z=1.76`$ bolsters the case significantly  and provides the first evidence for an early epoch of decelerated expansion. SN 1997ff falls right on the accelerating Universe curve on the magnitude – redshift diagram, and is a magnitude brighter than expected in a dusty open Universe or an open Universe in which type Ia supernovae are systematically fainter at high-$`z`$.

The second, independent line of reasoning for accelerated expansion comes from measurements of the composition of the Universe, which point to a missing energy component with negative pressure. The argument goes like this: CMB anisotropy measurements indicate that the Universe is nearly flat, with density parameter, $`\mathrm{\Omega }_0=1.0\pm 0.04`$. In a flat Universe, the matter density and energy density must sum to the critical density. However, matter only contributes about 1/3 of the critical density, $`\mathrm{\Omega }_M=0.33\pm 0.04`$. (This is based upon measurements of CMB anisotropy, of bulk flows, and of the baryonic fraction in clusters.) Thus, two thirds of the critical density is missing! Doing the bookkeeping more precisely, $`\mathrm{\Omega }_X=0.67\pm 0.06`$ .

In order to have escaped detection, this missing energy must be smoothly distributed. In order not to interfere with the formation of structure (by inhibiting the growth of density perturbations), the energy density in this component must change more slowly than matter (so that it was subdominant in the past). For example, if the missing 2/3 of critical density were smoothly distributed matter ($`p=0`$), then linear density perturbations would grow as $`R^{1/2}`$ rather than as $`R`$. The shortfall in growth since last scattering ($`z1100`$) would be a factor of 30, far too little growth to produce the structure seen today.

The pressure associated with the missing energy component determines how it evolves:

$`\rho _X`$ $``$ $`R^{3(1+w)}`$
$`\rho _X/\rho _M`$ $``$ $`(1+z)^{3w}`$ (2)
where $`w`$ is the ratio of the pressure of the missing energy component to its energy density (here assumed to be constant). Note, the more negative $`w`$, the faster the ratio of missing energy to matter decreases to zero in the past. In order to grow the structure observed today from the density perturbations indicated by CMB anisotropy measurements, $`w`$ must be more negative than about $`\frac{1}{2}`$ .

For a flat Universe the deceleration parameter today is

$$q_0=\frac{1}{2}+\frac{3}{2}w\mathrm{\Omega }_X\frac{1}{2}+w$$

Therefore, knowing $`w<\frac{1}{2}`$ implies $`q_0<0`$ and accelerated expansion. This independent argument for accelerated expansion and dark energy makes the supernova case all the more compelling.

### 4.2 Gravity Can Be Repulsive in Einstein’s Theory, But …

In Newton’s theory, mass is the source of the gravitational field and gravity is always attractive. In General Relativity, both energy and pressure source the gravitational field: $`\ddot{R}/R(\rho +3p)`$, cf., Eq. 1. Sufficiently large negative pressure leads to repulsive gravity.

While accelerated expansion can be accommodated within Einstein’s theory, that does not preclude that the ultimate explanation lies in a fundamental modification of Einstein’s theory. Lacking any good ideas for such a modification, I will discuss how accelerated expansion fits in the context of General Relativity. If the explanation for the accelerating Universe ultimately fits within the Einsteinian framework, it will be a stunning new triumph for General Relativity.

### 4.3 The Biggest Embarrassment in all of Theoretical Physics

Einstein introduced the cosmological constant to balance the attractive gravity of matter. He quickly discarded the cosmological constant after the discovery of the expansion of the Universe.

The advent of quantum field theory made consideration of the cosmological constant obligatory, not optional: The only possible covariant form for the energy of the (quantum) vacuum,

$$T_{\mathrm{VAC}}^{\mu \nu }=\rho _{\mathrm{VAC}}g^{\mu \nu },$$

is mathematically equivalent to the cosmological constant. It takes the form for a perfect fluid with energy density $`\rho _{\mathrm{VAC}}`$ and isotropic pressure $`p_{\mathrm{VAC}}=\rho _{\mathrm{VAC}}`$ (i.e., $`w=1`$) and is precisely spatially uniform. Vacuum energy is almost the perfect candidate for dark energy.

Here is the rub: the quantum zero-point contributions arising from well-understood physics (the known particles, integrating up to $`100`$GeV) sum to $`10^{55}`$ times the present critical density. (Put another way, if this were so, the Hubble time would be $`10^{10}`$sec, and the associated event horizon would be 3 cm!) This is the well known cosmological-constant problem .

While string theory currently offers the best hope for marrying gravity to quantum mechanics, it has shed precious little light on the cosmological constant problem, other than to speak to its importance. Thomas has suggested that using the holographic principle to count the available number of states in our Hubble volume leads to an upper bound on the vacuum energy that is comparable to the energy density in matter + radiation . While this reduces the magnitude of the cosmological-constant problem very significantly, it does not solve the dark energy problem: a vacuum energy that is always comparable to the matter + radiation energy density would strongly suppress the growth of structure.

The deSitter space associated with the accelerating Universe may pose serious problems for the formulation of string theory . Banks and Dine argue that all explanations for dark energy suggested thus far are incompatible with perturbative string theory . At the very least there is high tension between accelerated expansion and string theory.

The cosmological constant problem leads to a fork in the dark-energy road: one path is to wait for theorists to get the “right answer” (i.e., $`\mathrm{\Omega }_X=2/3`$); the other path is to assume that even quantum nothingness weighs nothing and something else with negative pressure must be causing the Universe to speed up. Of course, theorists follow the advice of Yogi Berra: “When you see a fork in the road, take it.”

### 4.4 Parameterizing Dark Energy: For Now, It’s $`w`$

Theorists have been very busy suggesting all kinds of interesting possibilities for the dark energy: networks of topological defects, rolling or spinning scalar fields (quintessence and spintessence), influence of “the bulk”, and the breakdown of the Friedmann equations . An intriguing recent paper suggests dark matter and dark energy are connected through axion physics .

In the absence of compelling theoretical guidance, there is a simple way to parameterize dark energy, by its equation-of-state $`w`$ .

The uniformity of the CMB testifies to the near isotropy and homogeneity of the Universe. This implies that the stress-energy tensor for the Universe must take the perfect fluid form . Since dark energy dominates the energy budget, its stress-energy tensor must, to a good approximation, take the form

$$T_{X}^{}{}_{\nu }{}^{\mu }\mathrm{diag}[\rho _X,p_X,p_X,p_X]$$
(3)
where $`p_X`$ is the isotropic pressure and the desired dark energy density is

$$\rho _X=2.7\times 10^{47}\mathrm{GeV}^4$$

(for $`h=0.72`$ and $`\mathrm{\Omega }_X=0.66`$). This corresponds to a tiny energy scale, $`\rho _X^{1/4}=2.3\times 10^3`$eV.

The pressure can be characterized by its ratio to the energy density (or equation-of-state):

$$wp_X/\rho _X$$

Note, $`w`$ need not be constant; e.g., it could be a function of $`\rho _X`$ or an explicit function of time or redshift. ($`w`$ can always be rewritten as an implicit function of redshift.)

For vacuum energy $`w=1`$; for a network of topological defects $`w=N/3`$ where $`N`$ is the dimensionality of the defects (1 for strings, 2 for walls, etc.). For a minimally coupled, rolling scalar field,

$$w=\frac{\frac{1}{2}\dot{\varphi }^2V(\varphi )}{\frac{1}{2}\dot{\varphi }^2+V(\varphi )}$$
(4)
which is time dependent and can vary between $`1`$ (when potential energy dominates) and $`+1`$ (when kinetic energy dominates). Here $`V(\varphi )`$ is the potential for the scalar field.

### 4.5 The Universe: The Lab for Studying Dark Energy

Dark energy by its very nature is diffuse and a low-energy phenomenon. It probably cannot be produced at accelerators; it isn’t found in galaxies or even clusters of galaxies. The Universe itself is the natural lab – perhaps the only lab – in which to study it.

The primary effect of dark energy on the Universe is determining the expansion rate. In turn, the expansion rate affects the distance to an object at a given redshift $`z`$ \[$`r(z)`$\] and the growth of linear density perturbations. The governing equations are:

$`H^2(z)`$ $`=`$ $`H_0^2(1+z)^3\left[\mathrm{\Omega }_M+\mathrm{\Omega }_X(1+z)^{3w}\right]`$
$`r(z)`$ $`=`$ $`{\displaystyle _0^z}𝑑u/H(u)`$
$`0`$ $`=`$ $`\ddot{\delta }_k+2H\dot{\delta }_k4\pi G\rho _M\delta _k`$ (5)
where for simplicity $`w`$ is assumed to be constant and $`\delta _k`$ is the Fourier component of comoving wavenumber $`k`$ and overdot indicates $`d/dt`$.

The various cosmological approaches to ferreting out the nature of the dark energy – all of which depend upon how the dark energy affects the expansion rate – have been studied . Based largely upon my work with Dragan Huterer , I summarize what we now know about the efficacy of the cosmological probes of dark energy:

* Present cosmological observations prefer $`w=1`$, with a 95% confidence limit $`w<0.6`$ .
* Because dark energy was less important in the past, $`\rho _X/\rho _M(1+z)^{3w}0`$ as $`z\mathrm{}`$, and the Hubble flow at low redshift is insensitive to the composition of the Universe, the most sensitive redshift interval for probing dark energy is $`z=0.22`$ .
* The CMB has limited power to probe $`w`$ (e.g., the projected precision for Planck is $`\sigma _w=0.25`$) and no power to probe its time variation .
* A high-quality sample of 2000 SNe distributed from $`z=0.2`$ to $`z=1.7`$ could measure $`w`$ to a precision $`\sigma _w=0.05`$ (assuming an irreducible systematic error of 0.14 mag). If $`\mathrm{\Omega }_M`$ is known independently to better than $`\sigma _{\mathrm{\Omega }_M}=0.03`$, $`\sigma _w`$ improves by a factor of three and the rate of change of $`w^{}=dw/dz`$ can be measured to precision $`\sigma _w^{}=0.16`$ .
* Counts of galaxies and of clusters of galaxies may have the same potential to probe $`w`$ as SNe Ia. The critical issue is systematics (including the evolution of the intrinsic comoving number density, and the ability to identify galaxies or clusters of a fixed mass) .
* Measuring weak gravitational lensing by large-scale structure over a field of 1000 square degrees (or more) could have comparable sensitivity to $`w`$ as type Ia supernovae. However, weak gravitational lensing does not appear to be a good method to probe the time variation of $`w`$ . The systematics associated with weak gravitational lensing have not yet been studied carefully and could limit its potential.

With the exception of vacuum energy, all the other possibilities for the dark energy cluster to some small extent on the largest scales . Measuring this clustering, while extremely challenging, could rule out vacuum or help to elucidate the nature of the dark energy. Hu and Okamoto have recently suggested how the CMB might be used to get at this clustering .

While the Universe is likely the lab where dark energy can best be attacked, one should not rule other approaches. For example, if the dark energy involves a ultra-light scalar field, then there should be a new long-range force .

### 4.6 The Nancy Kerrigan Problem

A critical constraint on dark energy is that it not interfere with the formation of structure in the Universe. This means that dark energy must have been relatively unimportant in the past (at least back to the time of last scattering, $`z1100`$). If dark energy is characterized by constant $`w`$, not interfering with structure formation can be quantified as: $`w\begin{array}{c}<\hfill \\ \hfill \end{array}\frac{1}{2}`$ . This means that the dark-energy density evolves more slowly than $`R^{3/2}`$ (compared to $`R^3`$ for matter) and implies

$`\rho _X/\rho _M`$ $``$ $`0\mathrm{for}t0`$
$`\rho _X/\rho _M`$ $``$ $`\mathrm{}\mathrm{for}t\mathrm{}`$
That is, in the past dark energy was unimportant and in the future it will be dominant! We just happen to live at the time when dark matter and dark energy have comparable densities. In the words of Olympic skater Nancy Kerrigan, “Why me? Why now?”

Perhaps this fact is an important clue to unraveling the nature of the dark energy. Perhaps not. I shudder to say this, but it could be at the root of an anthropic explanation for the size of the cosmological constant: The cosmological constant is as large as it can be and still allow the formation of structures that can support life .

### 4.7 Dark Energy and Destiny

Almost everyone is aware of the connection between the shape of the Universe and its destiny: positively curved recollapses, flat; negatively curved expand forever. The link between geometry and destiny depends upon a critical assumption: that matter dominates the energy budget (more precisely, that all components of matter/energy have equation of state $`w>\frac{1}{3}`$). Dark energy does not satisfy this condition.

In a Universe with dark energy the connection between geometry and destiny is severed . A flat Universe (like ours) can continue expanding exponentially forever with the number of visible galaxies diminishing to a few hundred (e.g., if the dark energy is a true cosmological constant); the expansion can slow to that of a matter-dominated model (e.g., if the dark energy dissipates and becomes sub-dominant); or, it is even possible for the Universe to recollapse (e.g., if the dark energy decays revealing a negative cosmological constant). Because string theory prefers anti-deSitter space, the third possibility should not be forgotten.

Dark energy is the key to understanding our destiny.

## 5 A Cosmic Perspective from Rovaniemi

After attending an exciting, enjoyable, and very productive COSMO-01, here is my summary of the state of cosmology:

* We have discerned the basic features of the Universe, as embodied in The New Cosmology.
* We can say confidently that events that took place during earliest moments have had a profound influence on the present state of Universe.
* We have a impressive program in place to test inflation and the particle dark-matter hypothesis, as well as their corollary, the Cold Dark Matter theory of structure formation.
* For some years, the front line in cosmology will continue to be phenomenology.
* No doubt, there will be surprises and we will need new ideas to replace (or supplement) those that are driving cosmology today.
* Schramm’s Razor – given two equally interesting new ideas, pursue the one that’s more testable – is more true than ever.

I am look forward to new results, new ideas and new (as well as old) faces at COSMO-02 in Chicago.