text:
# 1 Introduction and Overview

## 1 Introduction and Overview

Of all four-dimensional gauge field theories the one with the maximum number $`𝒩=4`$  of supersymmetries appears to be very special in many ways. At first sight this model appears to be incredibly complicated due to the very large number of fields of different kinds, but upon closer inspection many hidden simplicities appear. It is believed that the model is exactly conformally invariant at the quantum level  and thus a non-trivial example (as opposed to free massless field theory) of a *four*-dimensional conformal field theory (CFT). Given the huge successes in understanding CFT’s in *two* dimensions, one might hope that at least some of the aspects allowing their treatment in $`D=2`$ might fruitfully reappear in $`D=4`$. One of the many intriguing features of two-dimensional CFT’s is that they are intimately connected to integrable 2+0 dimensional lattice models in statistical mechanics, or, equivalently, to 1+1 dimensional quantum spin chains. Integrable models are even more important for the study of integrable massive deformations of these CFT’s. Thus, an optimistic mind could hope that integrability might also play a rôle in the putative simplicity of a theory such as $`𝒩=4`$ Yang-Mills. Excitingly, a number of recent discoveries lend much support to this idea. One might wonder about standard, naive no-go theorems that seem to suggest that integrability can never exist above $`D=2`$. These may be potentially bypassed by the fact that there appears to be a hidden “two-dimensionality” in $`\mathrm{SU}(N)`$ SYM4 when we look at it at large $`N`$, i.e. when we introduce $`\frac{1}{N}`$ as an additional coupling constant in the theory, aside from the ’t Hooft coupling $`\lambda =g_{\mathrm{YM}}^2N`$.

In fact, one might interpret the AdS/CFT correspondence , believed to be exact in the $`𝒩=4`$ case, as one important indication of the validity of this idea: Again following ’t Hooft, the worldsheet of the string is a clear candidate for explaining the hidden two dimensions. Interestingly, first indications that the world sheet theory, highly non-trivial due to the curved $`AdS_5\times S^5`$ background, might be integrable have recently appeared  (for the simpler but related case of pp-wave backgrounds see also ).

However, let us get back to the gauge theory and review how integrable structures have recently emerged directly, without resorting to the correspondence. Following the seminal BMN paper  it became clear that, despite much work on SYM4, not much was known about how to efficiently calculate the anomalous dimensions of local, non-protected conformal operators composed of an arbitrary number of elementary fields. In particular, in  effective vertex techniques were developed in order to efficiently calculate one-loop two-point functions of operators composed of an arbitrary numbers of scalars. From these two-point correlators the scaling dimensions could then be (indirectly) extracted. The main focus was on the case where the classical dimension of the operators tended to infinity, but the technique was applicable to the finite case as well. In particular  contained the full $`𝔰𝔬(6)`$ invariant effective vertex which was used in  to find the exact finite dimensional generalizations of the BMN operators. A significant simplification took place when it was realized  that the effective vertex could be directly interpreted as a *Hamiltonian* acting on a Hilbert space formed by all possible scalar operators of fixed classical dimension. Therefore it was no longer necessary to extract the anomalous dimensions from a two-point function; instead one could directly go about diagonalizing the Hamiltonian. This was done in  for a Hilbert space containing multi-trace gauge-invariant scalar operators; a natural split between a planar “free” part and a non-planar trace-splitting and trace-joining “interaction” part appeared, where the interaction is purely cubic with coupling constant $`\frac{1}{N}`$. This split turned out to be very reminiscent of similar structures appearing in cubic string field theories. In turn, Minahan and Zarembo focused on the planar part of the Hamiltonian and noticed the remarkable fact that it is identical to the one of an $`𝔰𝔬(6)`$ invariant integrable spin chain. The integrablity leads, via a hidden Yang-Baxter symmetry, to the appearance of an infinite number of charges commuting with the Hamiltonian. This allows to write down Bethe equations whose solution furnish the spectrum, i.e. the planar anomalous dimensions. For low dimensional operators (i.e. those containing only a few elementary fields) the approach is not necessarily easier than direct, brute force diagonalization of the Hamiltonian (however, it does lead to a very transparent and short derivation of the finite $`J`$ BMN operators found in ). Its real power lies in the fact that it allows to obtain anomalous dimensions in the limit of a large number of constituent fields.

This power was very recently illustrated in a study of scalar operators belonging to the $`𝔰𝔬(6)`$ representations $`[J_2,J_1J_2,J_2]`$ with $`\mathrm{\Delta }_0=J_1+J_2`$ in the limit where both charges $`J_1,J_2`$ become large . The ground state energy in this representation is argued to be related to the minimal energy solution of a string rotating in two planes in $`S^5`$, which can be calculated exactly . The result for this energy is given by inverting certain elliptic functions, it is a non-trivial *function* of the parameters $`J_1,J_2`$, and agrees on both sides. On the gauge theory side, we believe it can only be obtained by using the Bethe ansatz. This is arguably the most subtle dynamical, quantitative test of AdS/CFT existing to date.

Integrable spin chains had appeared before in gauge theories through the pioneering work of Lipatov on high energy scattering in planar QCD . The model was subsequently identified as a XXX Heisenberg $`𝔰𝔩(2)`$ spin chain of noncompact spin zero . More recently, and physically closely related to the present study, further integrable structures were discovered by Belitsky, Braun, Derkachov, Korchemsky and Manashov in the computation of planar one-loop anomalous dimensions of various types of quasi-partonic operators in QCD 111While QCD is surely not a conformally invariant quantum field theory, it still behaves like one as far as one-loop anomalous dimensions are concerned. We thank A. Belitsky, A. Gorsky and G. Korchemsky for pointing this out to us. In one-loop QCD, integrability is not complete; in particular it requires aligned helicities of the partonic degrees of freedom. . Subsequently, Kotikov and Lipatov  applied the corresponding integrable structure, originally found in the QCD context, to the computation of anomalous dimensions of twist-two operators (i.e. operators containing two scalar fields and an arbitrary number of symmetrized, traceless covariant derivatives acting on them) in SYM4. The result for the anomalous dimension is given in terms of harmonic numbers and was verified by Dolan and Osborn using totally different methods . This result was very useful in (qualitatively) comparing certain string states in $`AdS_5`$ to twist-two operators with a large spin in the gauge theory . Very recently Belitsky, Gorsky and Korchemsky  considered QCD composite light-cone operators, corresponding to operators of arbitrarily high twist, and discussed the hidden integrability of the associated dilatation operator.

Let us ask the question whether the integrable structures appearing in the planar gauge theory, when one investigates the first radiative corrections to anomalous dimensions, are accidental or rather more generic and thus indicative of a deeper underlying structure. The preliminary evidence from the correspondence with strings  certainly points to the second possibility. And indeed a first study of the two-loop corrections to the integrable Hamiltonian yields intriguing evidence that these do *not* break the integrability property . One indirect way to detect the hidden commuting charges is to look for unexpected degeneracies in the energy spectrum. We found that these degeneracies exist in planar SYM4, can be explained at one loop by the hidden integrability, and are not broken at the two-loop level. It is important to note that, while the one-loop Hamiltonian for scalar fields is exactly integrable , the combined one- and two-loop dilatation operator is not exactly integrable and requires the inclusion of three-loop terms, and so on . Interpreting these corrections in the spin chain language one finds that the higher-loop deformations of the charges are in a certain sense non-local, reminiscent of . The integrable system corresponding to the all-loop planar dilatation operator of $`𝒩=4`$ Yang-Mills, if it exists, should be some kind of long-range spin chain and has yet to be identified. In particular it is unclear how to extend the Bethe ansatz of  to include the higher loop effects.

All this suggests that, maybe, planar $`𝒩=4`$ Yang-Mills is integrable. If this is the case, the hidden symmetries should extend the known symmetries of SYM4. The full symmetry algebra of SYM4 is neither $`𝔰𝔬(6)`$ nor $`𝔰𝔩(2)`$, but the full superconformal algebra $`𝔭𝔰𝔲(2,2|4)`$.222The conjugation properties of the algebras are not relevant here, we will always refer to complex algebras, e.g. $`𝔰𝔩(4|4)=𝔰𝔲(2,2|4)`$, $`𝔰𝔬(6)=𝔰𝔩(4)`$, etc. If the integrable structures discovered to date are not accidental, we should expect that the $`𝔰𝔬(6)`$ one-loop results of Minahan-Zarembo and the $`𝔰𝔩(2)`$ results suggested from one-loop QCD  (see also ) can be combined and “lifted” to a full $`𝔭𝔰𝔲(2,2|4)`$ super spin chain, as first conjectured333In  it was conjectured that the $`𝔰𝔩(2)`$ and $`𝔰𝔬(6)`$ spin chains combine into a $`𝔰𝔬(2,4)\times 𝔰𝔬(6)=𝔰𝔲(2,2)\times 𝔰𝔲(4)`$ chain. As we shall see, we need the full $`𝔭𝔰𝔲(2,2|4)`$ algebra in order to achieve this “grand unification”. in . In this paper we will argue that this is indeed the case. In a companion paper, one of us worked out the complete one-loop dilatation operator of $`𝒩=4`$ Yang-Mills theory . That is, the effective vertex (not necessarily restricted to the planar case) allowing the computation of mixing matrices of composite operators containing arbitrary sequences of elementary fields (scalars, covariant derivatives, field strengths, and fermions) was derived. Upon diagonalization of this matrix (which in practice is however only possible for states of small classical dimension) its eigensystem yields the eigenoperators and their anomalous dimensions. Here we will establish that the restriction of this complete one-loop dilatation operator to the planar case yields the promised $`𝔰𝔲(2,2|4)`$ 444At the one-loop level the superconformal algebra $`𝔭𝔰𝔲(2,2|4)`$ may be extended by a $`𝔤𝔩(1)`$ charge to the semi-direct product $`𝔰𝔲(2,2|4)`$. spin chain. Our arguments contain some assumptions which should be filled in by experts on integrability; in particular, while we propose an R-matrix, we will assume its existence and uniqueness. At any rate we then proceed to write down the Bethe ansatz equations expected to describe our super spin chain. Again, it is not clear to us whether the validity of the ansatz has been rigorously proven in the spin chain literature. However, by analyzing the first few states in our superchain we find complete agreement with the gauge theory results. Therefore we are confident that the details of our Bethe ansatz are indeed correct. A subtlety, to be discussed below, is that for superalgebras one can write down *several* Bethe ansätze, which appear to be different and possess different (pseudo)vacuum states. They nevertheless yield an identical spectrum. We will discuss two specific ansätze which are both natural from different points of view.

The outline of this paper is as follows. We begin with a short review/preview on Bethe ansatz equations, and how they are determined by the Dynkin diagram of the symmetry algebra of the chain in conjunction with the highest weight representation of the degrees of freedom distributed along the chain. We will also introduce some useful, compact notation for writing the Bethe equations. We explain how to employ the latter in order to derive anomalous dimensions in $`𝒩=4`$ Yang-Mills. Next we present our arguments, via the existence of a unique R-matrix, why one expects the complete one-loop dilatation operator of  to become integrable in the large $`N`$ limit. The procedure naturally incorporates the $`𝔰𝔩(2)`$ (twist operators) and the $`𝔰𝔬(6)`$ (Minahan-Zarembo) integrable structures. For super chains we can write seemingly different sets of Bethe ansatz equations. We will pick two examples: The first (which we call “Beauty”) does not correspond to the standard way of choosing the fermionic root(s) of the Dynkin diagram. However, this formulation is particularly useful since the vacuum states of the chain correspond to the half-BPS states of SYM4. To illustrate the freedom of choosing various forms of the Bethe equations we then present a second version (“the Beast”) which employs the standard “distinguished” choice of one fermionic root in the middle of the diagram. As we will see, this shifts the vacuum to a different, high energy state containing only field strengths. It obscures certain well known results (e.g. it becomes hard to see that half-BPS states have vanishing energy) but instead leads to some further interesting spectroscopic results. We end with an outlook on interesting applications and extensions of this work.

## 2 Anomalous dimensions, (super) spin chains, and Bethe ansätze

Let us review how anomalous dimensions in YM4 are computed by employing the algebraic Bethe ansatz, once it has been established that the anomalous part of the dilatation operator is identical to the Hamiltonian of an integrable quantum spin chain . Let us explain this in the simplest case of an $`𝔰𝔩(2)`$ chain, the so-called XXXs/2 Heisenberg chain. (For a very pedagogical introduction, see ). The proposal is that the energy eigenvalues $`E`$ of the Hamiltonian are proportional to the anomalous dimensions $`\delta \mathrm{\Delta }`$ of conformal scaling operators:

$$\delta \mathrm{\Delta }=\frac{g_{\mathrm{YM}}^2N}{8\pi ^2}E.$$
(2.1)
For spin $`\frac{s}{2}=\frac{1}{2}`$ these scaling operators are, in the planar limit, linear combinations of a single trace of sequences of two of the three complex scalars of SYM4, say $`Z`$ and $`\varphi `$:

$$Tr\left(ZZZ\varphi Z\varphi \varphi \mathrm{}\right)$$
(2.2)
The single trace operator is then interpreted as a chain of spins, where $`Z`$ and $`\varphi `$ represent, respectively, up and down spins. The total number $`L`$ of complex scalars inside the trace corresponds to the length of the chain. Each eigenstate (alias scaling operator) of the Hamiltonian is uniquely characterized by a set of Bethe roots $`u_j`$, $`j=1,\mathrm{},n`$, and the energy $`E`$ of the state is given by

$$E=\pm \underset{j=1}{\overset{n}{}}\frac{s}{u_j^2+\frac{1}{4}s^2}.$$
(2.3)
(where for spin $`\frac{s}{2}=\frac{1}{2}`$ we should pick the plus sign in front of the sum). The Bethe roots are found by solving the Bethe equations for $`j=1,\mathrm{},n`$

$$\left(\frac{u_j+\frac{i}{2}s}{u_j\frac{i}{2}s}\right)^L=\underset{{\scriptscriptstyle \genfrac{}{}{0.0pt}{}{k=1}{kj}}}{\overset{n}{}}\frac{u_ju_k+i}{u_ju_ki}.$$
(2.4)
The vacuum state of energy $`E=0`$ is the half-BPS state

$$|0=TrZ^L,$$
(2.5)
i.e. it is interpreted as the ferromagnetic ground state (all spins are up, no Bethe roots) of the chain. The excitation number $`n`$, giving the total number of roots, counts the number of $`\varphi `$’s or down spins along the chain; it is naturally bounded by $`nL`$. There is an additional constraint on the Bethe roots:

$$1=\underset{j=1}{\overset{n}{}}\frac{u_j+\frac{i}{2}s}{u_j\frac{i}{2}s}.$$
(2.6)
For the spin chain, it means that we have periodic boundary conditions and we are only looking for zero-momentum states. In the gauge theory interpretation (2.2) it expresses the cyclicity of the trace.

The second simplest example concerns twist operators, closely related to quasi-partonic operators in one-loop QCD . Just as the fields in (2.2) form a closed subsector (i.e. they do not mix with any other types of fields) the following operators form another such subsector (see also  for further details):

$$Tr\left((𝒟^{m_1}Z)(𝒟^{m_2}Z)\mathrm{}(𝒟^{m_L}Z)\right)$$
(2.7)
where $`𝒟`$ is the “light-cone” covariant derivative $`𝒟_{1+i2}`$ (as in ). This is still a spin chain of length (twist) $`L`$, with non-compact spin $`\frac{s}{2}=\frac{1}{2}`$, as will be proven in the next chapter. Here the spins at each lattice site $`k`$ may take any value $`m_k=0,1,2,\mathrm{}`$, as we have an infinite $`s=1`$ representation of $`𝔰𝔩(2)`$. Also the total excitation number $`n=m_k`$ is not bounded as in the above example. The vacuum is still $`TrZ^L`$. Note that in this example the length of the chain is *not* equal to the classical dimension $`L+n`$ of the operator. Again, the anomalous dimensions of the scaling operators formed from the states (2.7) are given via eqs. (2.1),(2.3),(2.4),(2.6) ! (Here we have to choose the negative sign in the energy formula (2.3).)

In the above example the algebra is $`𝔰𝔩(2)`$ and thus of rank one. There is a beautiful extension of the Bethe equations to any algebra and any representation of the elementary constituents of the chain at each lattice site . This general form easily extends to the case of super algebras as well, see  and references therein. This is what we need when we are looking for a spin chain describing SYM4 at one loop, where we expect Bethe equations for the super algebra $`𝔰𝔩(4|4)`$. This general equation may be written in the following compact notation, based on knowing the Dynkin diagram of the algebra. The Dynkin diagram of $`𝔰𝔩(4|4)`$ contains seven dots corresponding to a choice of seven simple roots. Consider a total of $`n`$ excitations. For each of the corresponding Bethe roots $`u_i`$, $`i=1,\mathrm{},n`$, we specify which of the seven simple roots is excited by $`k_j=1,\mathrm{},7`$. The Bethe equations for $`j=1,\mathrm{},n`$ can then be written in the compact form

$$\left(\frac{u_j+\frac{i}{2}V_{k_j}}{u_j\frac{i}{2}V_{k_j}}\right)^L=\underset{{\scriptscriptstyle \genfrac{}{}{0.0pt}{}{l=1}{lj}}}{\overset{n}{}}\frac{u_ju_l+\frac{i}{2}M_{k_j,k_l}}{u_ju_l\frac{i}{2}M_{k_j,k_l}}.$$
(2.8)
Here, $`M_{kl}`$ is the Cartan matrix of the algebra and $`V_k`$ are the Dynkin labels of the spin representation. Furthermore, we still consider a cyclic spin chain with zero total momentum. This gives the additional constraint

$$1=\underset{j=1}{\overset{n}{}}\frac{u_j+\frac{i}{2}V_{k_j}}{u_j\frac{i}{2}V_{k_j}}.$$
(2.9)
The energy of a configuration of roots that satisfies the Bethe equations and constraint is now given by 555The Bethe equations determine the energy only up to scale and a shift $`cL`$ where c is a constant to be fixed such that the energy corresponds to planar anomalous dimensions of $`𝒩=4`$ SYM via (2.1).

$$E=cL\pm \underset{j=1}{\overset{n}{}}\left(\frac{i}{u_j+\frac{i}{2}V_{k_j}}\frac{i}{u_j\frac{i}{2}V_{k_j}}\right).$$
(2.10)
It is easily seen that restricting these equations to the Dynkin diagram of the algebra $`𝔰𝔬(6)`$ reproduces the Bethe equations of . It will turn out, see below, that these general equations, which are well known in the literature on integrable spin chains, indeed solve the entire problem of computing planar anomalous dimensions in SYM4, once we (*i*) identify the correct representations of the fundamental fields on the lattice sites, and (*ii*) after resolving certain subtleties concerning Dynkin diagrams for superalgebras. However, let us first show that we expect planar one-loop SYM4 to be completely integrable indeed.

## 3 The R-Matrix

In this section we derive the R-matrix for the integrable spin chain considered in this work. For this we make use of a special subsector of the spin chain with residual $`𝔰𝔩(2)`$ symmetry and show how to lift the universal $`𝔰𝔩(2)`$ R-matrix to an $`𝔰𝔩(4|4)`$ invariant R-matrix. The derived Hamiltonian is shown to agree with the complete one-loop planar dilatation generator of $`𝒩=4`$ SYM, thus proving the integrability of the latter.

#### The planar one-loop dilatation operator.

The complete one-loop planar dilatation generator of $`𝒩=4`$ SYM acting on two fields is given by

$$H_{12}=2h(J_{12}):=\underset{j=0}{\overset{\mathrm{}}{}}2h(j)P_{12,j}.$$
(3.1)
Here, $`h(j)`$ is the $`j`$-th harmonic number defined alternatively by a series or by the digamma function $`\mathrm{\Psi }(x)=\mathrm{log}\mathrm{\Gamma }(x)/x`$

$$h(n)=\underset{k=1}{\overset{n}{}}\frac{1}{k}=\mathrm{\Psi }(n+1)\mathrm{\Psi }(1).$$
(3.2)
The operator $`P_{12,j}`$ projects the spins at sites $`1,2`$ to the module $`V_j`$. Each spin belongs to the ‘fundamental’ module $`V_\mathrm{F}`$, consequently the modules $`V_j`$ arise in the tensor product of two fundamental modules

$$V_\mathrm{F}\times V_\mathrm{F}=\underset{j=0}{\overset{\mathrm{}}{}}V_j.$$
(3.3)
The primary weights of these modules are given by (see (4.4) for details)

$`w_\mathrm{F}`$ $`=`$ $`[1;0,0;0,1,0;0,1],`$
$`w_0`$ $`=`$ $`[2;0,0;0,2,0;0,2],`$
$`w_1`$ $`=`$ $`[2;0,0;1,0,1;0,2],`$
$`w_j`$ $`=`$ $`[j;j2,j2;0,0,0;0,2].`$ (3.4)
#### The $`𝔰𝔩(2)`$ subsector.

A crucial point in the derivation of the full Hamiltonian density  was the restriction to a subsector. This subsector is obtained by restricting the allowed weights of states to

$$w=[L+s;s,s;0,L,0;0,L].$$
(3.5)
For the construction of such states we must restrict all spins to the form

$$z^n:=\frac{1}{n!}𝒟^nZ,\text{with}𝒟:=𝒟_1+i𝒟_2,Z:=\mathrm{\Phi }_5+i\mathrm{\Phi }_6.$$
(3.6)
The residual symmetry algebra within the subsector is $`𝔰𝔩(2)`$ which can be represented by the generators

$$J_{}^{}=,J_3^{}=\frac{1}{2}+z,J_+^{}=z+z^2$$
(3.7)
where $``$ is the derivative with respect to $`z`$. It was then shown that the modules $`V_\mathrm{F},V_j`$ of the full theory correspond to the $`𝔰𝔩(2)`$ modules $`V_\mathrm{F}^{},V_j^{}`$ with highest weights

$$w_\mathrm{F}^{}=[1],w_j^{}=[22j].$$
(3.8)
Here, the weights are described by *twice* the spin. The Hamiltonian density (3.1) restricts within this subsector to

$$H_{12}^{}=2h(J_{12}^{}):=\underset{j=0}{\overset{\mathrm{}}{}}2h(j)P_{12,j}^{},$$
(3.9)
where $`P_{12,j}^{}`$ projects states of the tensor product $`V_\mathrm{F}^{}\times V_\mathrm{F}^{}`$ to the module $`V_j^{}`$. Interestingly, this is the Hamiltonian of the XXX-1/2 Heisenberg spin chain. In other words it is an integrable spin chain with unbroken $`𝔰𝔩(2)`$ symmetry, where each site transforms in a spin $`w_\mathrm{F}^{}/2=\frac{1}{2}`$ representation.

#### Integrability within the $`𝔰𝔩(2)`$ subsector.

To prove this statement we make use of the universal R-matrix of $`𝔰𝔩(2)`$ spin chains. This $`𝔰𝔩(2)`$ invariant operator can be decomposed into its irreducible components corresponding to the modules $`V_j^{}`$

$$R_{12}^{}(u)=\underset{j=0}{\overset{\mathrm{}}{}}R_j^{}(u)P_{12,j}^{}.$$
(3.10)
The eigenvalues $`R_j^{}(u)`$ of the $`𝔰𝔩(2)`$ universal R-matrix were determined in . In a spin $`w_j^{}/2=1j`$ representation the eigenvalue is

$$R_j^{}(u)=(1)^{j+1}\frac{\mathrm{\Gamma }(jcu)}{\mathrm{\Gamma }(j+cu)}f(cu).$$
(3.11)
The arbitrary function $`f(u)`$ and normalization constant $`c`$ reflect the trivial symmetries of the Yang-Baxter equation. The induced eigenvalues of the Hamiltonian density are obtained as the logarithmic derivative of $`R_j^{}(u)`$ at $`u=0`$ times the permutation operator $`P_{12}`$

$$H_{12}V_j^{}=P_{12}\frac{\mathrm{log}R_j^{}(u)}{u}|_{u=0}V_j^{}.$$
(3.12)
We note that for even (odd) $`j`$ the composite module $`V_j^{}`$ is a (anti)symmetric combination of two $`V_\mathrm{F}^{}`$, consequently the permutation acts as

$$P_{12}V_j^{}=(1)^jV_j^{}.$$
(3.13)
We choose the function and constant to be

$$f(cu)=\frac{\mathrm{\Gamma }(cu)}{\mathrm{\Gamma }(cu)},c=1.$$
(3.14)
Using the fact that $`j`$ is exactly integer we find the eigenvalues of the Hamiltonian density

$$(1)^j\frac{\mathrm{log}R_j^{}(u)}{u}|_{u=0}=2\mathrm{\Psi }(j+1)2\mathrm{\Psi }(1)=2h(j).$$
(3.15)
This proves that the Hamiltonian density (3.9) is integrable.

#### Integrability of the full $`𝔰𝔩(4|4)`$ R-matrix.

To derive an R-matrix for the full $`𝔰𝔩(4|4)`$ spin chain we will assume that for given representations of the symmetry algebra there exists a unique R-matrix which satisfies the Yang-Baxter equation (modulo the symmetries of the YBE). We are not aware whether this claim  has been proven. Let $`R_{12}`$ be this R-matrix for the $`𝔰𝔩(4|4)`$ integrable spin chain. The R-matrix is an invariant operator, thus it can be reduced to its irreducible components corresponding to the modules $`V_j`$

$$R_{12}(u)=\underset{j=0}{\overset{\mathrm{}}{}}R_j(u)P_{12,j}.$$
(3.16)
The restriction $`R^{}`$ of the R-matrix to the $`𝔰𝔩(2)`$ sector must also satisfy the Yang-Baxter equation. The unique solution for the eigenvalues of $`R^{}`$ is (3.11). Due to the one-to-one correspondence of modules $`V_j`$ and $`V_j^{}`$,

$$V_j^{}V_j$$
(3.17)
the eigenvalues of the unique $`𝔰𝔩(4|4)`$ R-matrix (3.16) must be

$$R_j(u)=(1)^j\frac{\mathrm{\Gamma }(jcu)}{\mathrm{\Gamma }(j+cu)}f(cu).$$
(3.18)
As in (3.15) this R-matrix yields $`H_{12}=2h(J_{12})`$ in agreement with (3.1). This in turn shows that the planar one-loop dilatation generator of $`𝒩=4`$ is integrable. Note however, that this proof is based on the assumption of the existence of a unique R-matrix.

#### The Minahan and Zarembo chain.

As an application let us investigate the restriction of the R-matrix to the $`𝔰𝔬(6)`$ subsector investigated by Minahan and Zarembo

$$w=[L;0,0;q_1,p,q_2;0,L],$$
(3.19)
i.e. where the spins are given by the $`𝒩=4`$ SYM scalars $`\mathrm{\Phi }_m`$. Two scalars $`\mathrm{\Phi }_p,\mathrm{\Phi }_q`$ can transform in three different irreducible representations of $`𝔰𝔬(6)`$, symmetric-traceless, antisymmetric and singlet. These correspond to the modules $`V_0,V_1,V_2`$, respectively.

We set the normalization function and constant to

$$f(u)=\frac{\mathrm{\Gamma }(3+cu)}{2\mathrm{\Gamma }(1cu)},c=1.$$
(3.20)
and compute the eigenvalues of the R-matrix (3.18) for $`V_0,V_1,V_2`$

$`R_{12,0}(u)`$ $`=`$ $`\frac{1}{2}(u+1)(u+2),`$
$`R_{12,1}(u)`$ $`=`$ $`\frac{1}{2}(u1)(u+2),`$
$`R_{12,2}(u)`$ $`=`$ $`\frac{1}{2}(u1)(u2).`$ (3.21)
We note the projectors to $`V_0,V_1,V_2`$ in the scalar sector

$`P_{12,0}`$ $`=`$ $`\frac{1}{2}+\frac{1}{2}P_{12}\frac{1}{6}K_{12},`$
$`P_{12,1}`$ $`=`$ $`\frac{1}{2}\frac{1}{2}P_{12},`$
$`P_{12,2}`$ $`=`$ $`\frac{1}{6}K_{12},`$ (3.22)
where $`K_{12}`$ is the $`𝔰𝔬(6)`$-trace operator defined by $`K_{12}\mathrm{\Phi }_{1,m}\mathrm{\Phi }_{2,n}=\delta _{mn}\mathrm{\Phi }_{1,p}\mathrm{\Phi }_{2,p}`$. Assembling (3) and (3) we obtain the R-matrix operator (3.16)

$$R_{12}(u)=\frac{1}{2}(u+2)P_{12}+\frac{1}{2}u(u+2)\frac{1}{2}uK_{12}.$$
(3.23)
This is precisely the R-matrix of the Minahan and Zarembo chain  which has previously been found by Reshetikhin .

## 4 Beauty …

In the last section we have established that the planar one-loop dilatation operator of $`𝒩=4`$ YM4 is integrable. We therefore expect the general Bethe ansatz equations (2.8) to hold. However, for them to be useful, we still need to specify the Dynkin labels, the Cartan matrix and precise form of the energy (2.10). Furthermore, we will perform a check of the validity of this $`𝔰𝔩(4|4)`$ Bethe ansatz which goes beyond the $`𝔰𝔬(6)`$ subsector.

#### Representations.

First, we need to specify the Cartan matrix, determined by the Dynkin diagram, and the Dynkin labels of the spin representation corresponding to the module $`V_\mathrm{F}`$. For classical semi-simple Lie algebra the Dynkin diagram is unique. In the case of superalgebras, however, there is some freedom to distribute the simple fermionic roots. In the context of $`𝒩=4`$ SYM one particular choice of Dynkin diagram turns out to be very useful :666We thank V. Dobrev for this hint, and for informing us about reference .

(4.1)
On top of the Dynkin diagram we have indicated the Dynkin labels of the spin representation. We write the Cartan matrix corresponding to this choice of Dynkin diagram and the representation vector as777In fact, the Cartan matrix is obtained from this by inverting some lines. The Bethe equations are invariant under the inversion and it is slightly more convenient to work with a symmetric matrix $`M`$.

$$M=\left(\begin{array}{ccccccc}2& +1& & & & & \\ & & & & & & \\ +1& & 1& & & & \\ & & & & & & \\ & 1& +2& 1& & & \\ & & 1& +2& 1& & \\ & & & 1& +2& 1& \\ & & & & & & \\ & & & & 1& & +1\\ & & & & & & \\ & & & & & +1& 2\end{array}\right),V=\left(\begin{array}{c}\hfill 0\\ \\ \hfill 0\\ \\ \hfill 0\\ \hfill 1\\ \hfill 0\\ \\ \hfill 0\\ \\ \hfill 0\end{array}\right).$$
(4.2)
The energy corresponding to a solution to the Bethe equations is

$$E=\underset{j=1}{\overset{n}{}}\left(\frac{i}{u_j+\frac{i}{2}V_{k_j}}\frac{i}{u_j\frac{i}{2}V_{k_j}}\right).$$
(4.3)
#### Excitation numbers.

Finally, we need to obtain the number of excitations $`n_k`$, $`k=1,\mathrm{},7`$ of the individual simple roots for a state with a given weight

$$w=[\mathrm{\Delta }_0;s_1,s_2;q_1,p,q_2;B,L].$$
(4.4)
A weight of (the classical) $`𝔰𝔩(4|4)`$ is described by various labels. The (bare) dimension is indicated by $`\mathrm{\Delta }_0`$. The $`𝔰𝔩(4)`$ and $`𝔰𝔩(2)\times 𝔰𝔩(2)`$ Dynkin labels are given by $`[q_1,p,q_2]`$ and $`[s_1,s_2]`$, respectively, where $`s_1,s_2`$ equal *twice* the spin. Furthermore, $`B`$ describes the $`𝔤𝔩(1)`$ hypercharge or chirality in the semi-direct product $`𝔰𝔩(4|4)=𝔤𝔩(1)𝔭𝔰𝔩(4|4)`$. Finally, $`L`$, which is not a label of $`𝔰𝔩(4|4)`$, gives the length or number of spins. In $`𝒩=4`$ SYM the chirality and length of an operator are not good quantum numbers, they are broken at higher loops by the Konishi anomaly. Nevertheless, at one-loop order they are conserved and their leading order values can be used to describe a state.

This is most easily seen in the oscillator picture in  using the physical vacuum $`|Z`$. We write down the action of the generators corresponding to the simple roots in terms of creation and annihilation operators

(4.5)
It is now clear that $`n_1=n_{𝐚_2^{}}`$, $`n_2=n_{𝐚_1^{}}+n_{𝐚_2^{}}`$ and so on. Using the formulas in  we write down the corresponding excitation numbers of the simple roots

$$n_k=\left(\begin{array}{c}\frac{1}{2}\mathrm{\Delta }_0\frac{1}{2}(LB)\frac{1}{2}s_1\hfill \\ \mathrm{\Delta }_0(LB)\hfill \\ \mathrm{\Delta }_0\frac{1}{2}(LB)\frac{1}{2}p\frac{3}{4}q_1\frac{1}{4}q_2\hfill \\ \mathrm{\Delta }_0p\frac{1}{2}q_1\frac{1}{2}q_2\hfill \\ \mathrm{\Delta }_0\frac{1}{2}(L+B)\frac{1}{2}p\frac{1}{4}q_1\frac{3}{4}q_2\hfill \\ \mathrm{\Delta }_0(L+B)\hfill \\ \frac{1}{2}\mathrm{\Delta }_0\frac{1}{2}(L+B)\frac{1}{2}s_2\hfill \end{array}\right).$$
(4.6)
Not all excitations of the simple roots correspond to physical states. Obviously, the excitation numbers of the oscillators must be non-negative, this gives the bounds 888 Superconformal primaries reside in the fundamental Weyl chamber defined by the bounds $`2n_1+n_2>1`$, $`n_22n_3+n_4>1`$, $`n_32n_4+n_5+L>1`$, $`n_42n_5+n_6>1`$, $`n_62n_7>1`$. Together with (4.7) this implies, among other relations, (4.8). Solutions of the Bethe equations outside the fundamental domain apparently correspond to mirror images of primary states due to reflections at the chamber boundaries (We thank J. Minahan and K. Zarembo for this insight).

$$0n_1n_2n_3n_4n_5n_6n_70.$$
(4.7)
Furthermore, each fermionic oscillator cannot be excited more than once, this gives the bounds

$$n_2+2Ln_3+Ln_4n_5+Ln_6+2L.$$
(4.8)
Certainly, we should obtain the $`𝔰𝔬(6)=𝔰𝔩(4)`$ subsector studied by Minahan and Zarembo  when we remove the outer four simple roots from the Dynkin diagram in (4.1). When we restrict to the states (3.19) of this subsector the number of excitations (4.6) of the outer four roots is trivially zero. They become irrelevant for the Bethe ansatz and can be discarded. Thus all solutions to the $`𝔰𝔬(6)`$ Bethe equations are also solutions to the $`𝔰𝔩(4|4)`$ Bethe equations. What is more, we can apply this Bethe ansatz to a wider range of operators, in fact, to *all* single-trace operators of $`𝒩=4`$ SYM.

#### Multiplet splitting.

Now we can write down and try to solve the Bethe equations for any state in $`𝒩=4`$ SYM. Note, however, that the Bethe equations need to be solved only for highest weight states. All descendants of a highest weight state are obtained by adding Bethe roots at infinity, $`u_i=\mathrm{}`$. In other words, the solutions to the Bethe equations corresponding to highest weight states are distinguished in that they have no roots $`u_i`$ at infinity. Nevertheless, there is one subtlety related to this point which can be used to our advantage. Namely this is multiplet splitting at the unitarity bounds . We assume that the spin chain of $`L`$ sites transforms in the tensor product of $`L`$ spin representations. The corrections $`\delta \mathrm{\Delta }`$ to the scaling dimension induced by the Hamiltonian $`H`$ are not included in this picture. Thus, in terms of the spin chain only the classical $`𝔰𝔩(4|4)`$ algebra applies where the scaling dimension is exactly $`\mathrm{\Delta }_0`$. Long multiplets of the interacting $`𝔰𝔩(4|4)`$ algebra close to one or both of the unitarity bounds

$`\mathrm{I}:`$ $`\mathrm{\Delta }_0=2+s_1+p+\frac{3}{2}q_1+\frac{1}{2}q_2,`$
$`\mathrm{II}:`$ $`\mathrm{\Delta }_0=2+s_2+p+\frac{1}{2}q_1+\frac{3}{2}q_2.`$ (4.9)
split up into several semi-short or BPS multiplets. The weights of the additional primary states are offset by

$`\delta w_\mathrm{I}`$ $`=`$ $`\{\begin{array}{ccc}[+0.5;1,0;+1,0,0;0.5,+1],\hfill & & \text{for }s_1>0,\hfill \\ [+1.0;0,0;+2,0,0;0.0,+1],\hfill & & \text{for }s_1=0,\hfill \end{array}`$ (4.12)
$`\delta w_{\mathrm{II}}`$ $`=`$ $`\{\begin{array}{ccc}[+0.5;0,1;0,0,+1;+0.5,+1],\hfill & & \text{for }s_2>0,\hfill \\ [+1.0;0,0;0,0,+2;0.0,+1],\hfill & & \text{for }s_2=0.\hfill \end{array}`$ (4.15)
The unitarity bounds can also be expressed in terms of excitations of simple roots, we find

$`\mathrm{I}:`$ $`n_1+n_3=n_2+1,`$
$`\mathrm{II}:`$ $`n_7+n_5=n_6+1.`$ (4.16)
The corresponding offsets translate into

$`\delta n_\mathrm{I}`$ $`=`$ $`\{\begin{array}{ccc}(0,1,1,0,0,0,0),\hfill & & \text{for }n_2>n_1,\hfill \\ (0,0,1,0,0,0,0),\hfill & & \text{for }n_2=n_1,\hfill \end{array}`$ (4.19)
$`\delta n_{\mathrm{II}}`$ $`=`$ $`\{\begin{array}{ccc}(0,0,0,0,1,1,0),\hfill & & \text{for }n_6>n_7,\hfill \\ (0,0,0,0,1,0,0),\hfill & & \text{for }n_6=n_7,\hfill \end{array}`$ (4.22)
together with an increase of the length $`L`$ by one. We thus see that in the case of multiplet shortening the primaries of higher submultiplets have less excitations. In a calculation this may reduce the complexity of the Bethe equations somewhat as we shall see in an example below.

Multiplet splitting is an extremely interesting issue from the point of view of integrability. Let us consider some operator acting on a spin chain. Assume the operator is invariant under the classical algebra $`𝔰𝔩(4|4)`$. In the most general case, this operator can assign a different value to all irreducible multiplets of states. In particular this is so for the submultiplets of a long multiplet at the unitarity bound. Now, if we impose integrability on the operator all these submultiplets become degenerate.999This is because the unique integrable operator is equivalent to the one-loop planar correction to the dilatation operator of $`𝒩=4`$ SYM. In $`𝒩=4`$ SYM the submultiplets must rejoin into a long multiplet which would be inconsistent if their one-loop anomalous dimensions were different. A priori, this seems like a miracle. Why should integrability imply *this* degeneracy? It almost seems as if integrability selects that scalar operator which is suitable as a consistent deformation of the dilatation generator! Then, clearly the miracle would turn into the condition for integrability. If this can be made sense of, maybe it also helps to understand higher loop corrections in the light of integrability .

#### Example.

Let us apply the Bethe ansatz to the twist-two operator with primary weight

$$w=[4;2,2;0,0,0;0,2].$$
(4.23)
Using (4.6) we find the excitation numbers and length

$$n_{0,k}=(0,2,3,4,3,2,0),L_0=2.$$
(4.24)
This weight is on both unitarity bounds, (4), the excitation numbers of the highest submutiplet, (4.19), are

$$n_k=(0,1,2,4,2,1,0),L=4.$$
(4.25)
We therefore configure the simple roots as follows

$$k_j=(2,3,3,4,4,4,4,5,5,6).$$
(4.26)
Now we note that the twist-two operators are unpaired states. Therefore the configurations of Bethe roots must be invariant under the symmetry $`u_ju_j`$ of the Bethe equations. This tells us

$$u_1=u_{10}=0,u_2=u_3,u_4=u_5,u_6=u_7,u_8=u_9,$$
(4.27)
which automatically satisfies the momentum constraint (2.9). Furthermore the excitations (4.24) are invariant under flipping the Dynkin diagram, $`n_kn_{8k}`$. This can be used to set

$$u_2=u_8.$$
(4.28)
The Bethe equations (2.8) are then solved exactly by

$$u_2=\sqrt{\frac{5}{7}},u_{4,6}=\sqrt{\frac{65\pm 4\sqrt{205}}{140}},$$
(4.29)
which yields the energy (4.3)

$$E=\frac{25}{3},\delta \mathrm{\Delta }=\frac{g_{\mathrm{YM}}^2N}{8\pi ^2}E=\frac{25g_{\mathrm{YM}}^2N}{24\pi ^2}.$$
(4.30)
This is the energy of the twist-two state at dimension four , see also Tab. 1.

## 5 …& the Beast

The alert reader will have noticed that the Dynkin diagram in (4.1) is not the standard one found in textbooks. In fact, for superalgebras there are alternative choices for the Dynkin diagram. In this subsection we will consider the ‘distinguished’ Dynkin diagram with one simple fermionic root

(5.1)
Note, that for a different choice of Dynkin diagram, we get different Dynkin labels for the spin representation. What is more, the highest weight has actually changed as well as we will see below. The Cartan matrix and the representation vector read

$$M=\left(\begin{array}{ccccccc}+2& 1& & & & & \\ 1& +2& 1& & & & \\ & 1& +2& 1& & & \\ & & & & & & \\ & & 1& & +1& & \\ & & & & & & \\ & & & +1& 2& +1& \\ & & & & +1& 2& +1\\ & & & & & +1& 2\end{array}\right),V=\left(\begin{array}{c}\hfill 0\\ \hfill 3\\ \hfill 2\\ \\ \hfill 0\\ \\ \hfill 0\\ \hfill 0\\ \hfill 0\end{array}\right)$$
(5.2)
The Bethe equations and momentum constraint are still given by (2.8),(2.9), while the energy of a solution to the Bethe equations is now

$$E=3L\underset{j=1}{\overset{n}{}}\left(\frac{i}{u_j+\frac{i}{2}V_{k_j}}\frac{i}{u_j\frac{i}{2}V_{k_j}}\right).$$
(5.3)
In this form of the Bethe equations, the vacuum has an energy, it is a pseudovacuum and not the ground state of the theory. The vacuum state is therefore *not* the half-BPS state $`TrZ^L`$ in (2.5), but instead

$$|0=Tr^L$$
(5.4)
i.e. a composite operator of classical dimension $`\mathrm{\Delta }_0=2L`$ and energy $`E=3L`$  built from the field strength component $`=_{1+i2,3+i4}`$. This vacuum configuration is the major difference to the Bethe ansatz discussed in the previous section. What used to be a rather trivial state in one Bethe ansatz, becomes a highly excited state in the other. Nevertheless there is a duality between both sets of Bethe equations in that they must both yield the same spectra of energies. This remarkable fact can be made use of in the usual sense of dualities. To determine the energy of some set of states we apply that Bethe ansatz which seems most suitable for the problem. For example, the Bethe ansatz of the previous section is useful for states with a low energy. In contrast, the Bethe ansatz discussed in this section seems most suitable for states of a large chirality $`B`$ or of an energy around $`3L`$.

The number of excitations can be determined as above. We note the action of the simple roots in the oscillator picture of

(5.5)
The vacuum corresponds to the configuration $`𝐚_1^{}𝐚_1^{}|0`$. The numbers of excitations for a given weight are

$$n_k=\left(\begin{array}{c}\frac{1}{2}(LB)+\frac{1}{2}\mathrm{\Delta }_0\frac{1}{2}s_2L\hfill \\ \frac{2}{2}(LB)+\mathrm{\Delta }_02L\hfill \\ \frac{3}{2}(LB)+\frac{1}{2}\mathrm{\Delta }_0\frac{1}{2}s_1\hfill \\ \frac{4}{2}(LB)\hfill \\ \frac{3}{2}(LB)\frac{1}{2}p\frac{1}{4}q_1\frac{3}{4}q_2\hfill \\ \frac{2}{2}(LB)p\frac{1}{2}q_1\frac{1}{2}q_2\hfill \\ \frac{1}{2}(LB)\frac{1}{2}p\frac{3}{4}q_1\frac{1}{4}q_2\hfill \end{array}\right)$$
(5.6)
We note the bounds

$$0n_1n_2,n_3n_4n_5n_6n_70,$$
(5.7)
as well as

$$n_3n_22L,n_4n_5,n_5n_4,n_6n_7,n_7L.$$
(5.8)
We defer the discussion of highest weight states and multiplet splitting in the context of the ‘distinguished’ choice of Dynkin diagram to App. A. There we also compare the energies of all dimension four operators to the results of SYM.

#### Example.

We see that the vacuum of this choice of Bethe equations has energy $`3L`$. Clearly, this is not the ground state, the ground state is half-BPS and has zero energy. This situation is similar to the antiferromagnetic spin chain, where the ground state is described by a highly excited state (in terms of Bethe roots). In this spin chain the first half-BPS multiplet with primary weight

$$w_0=[2;0,0;0,2,0;0,2]$$
(5.9)
has the highest weight (in the sense of highest chirality $`B`$)

$$w=[4;0,0;0,0,0;2,2].$$
(5.10)
This corresponds to the excitations and length

$$n_k=(0,0,2,0,0,0,0),k_j=(3,3),L=2.$$
(5.11)
The Bethe equations and momentum constraint are solved exactly by

$$u_{1,2}=\pm \frac{i}{\sqrt{3}}.$$
(5.12)
Reassuringly, the energy due to the excitations precisely cancels off the vacuum energy

$$E=0.$$
(5.13)
#### Spin Waves.

For a low number of excitations but arbitrary length of the spin chain, the Bethe ansatz can quickly provide good approximations to the energies. Especially in the case of two excitations it is often trivial to solve the Bethe equations exactly. We will now discuss these spin wave solutions, which correspond to small fluctuations of the vacuum. The major difference to the previous investigation of spin waves in the context of $`𝒩=4`$ SYM  is the modified vacuum. The solution is quite similar and we will follow along the lines of .

Let us consider the following excitations of simple roots

$$n_k=(0,1,1,0,0,0,0),k_j=(2,3).$$
(5.14)
The momentum constraint implies

$$u_1=\frac{3}{2}u_2.$$
(5.15)
and the two Bethe equations then collapse into

$$\left(\frac{i+u_2}{iu_2}\right)^{L1}=1.$$
(5.16)
This equation has $`L1`$ independent solutions

$$u_2=\mathrm{cot}\frac{\pi n}{L1},\frac{L1}{2}<n\frac{L1}{2}$$
(5.17)
with energy

$$E=3L\frac{2}{3}\mathrm{sin}^2\frac{\pi n}{L1}.$$
(5.18)
Note that the symmetry $`u_ju_j`$ translates into $`nn`$. The states $`n`$ and $`n`$ therefore have degenerate energies and form a pair unless $`n=0`$ or $`n=(L1)/2`$. Then the solutions $`u_{1,2}=\mathrm{}`$ or $`u_{1,2}=0`$, respectively are invariant and do not pair up.

We can also consider the thermodynamic limit of a large $`L`$. In leading order we can approximate the positions of the Bethe roots and the corresponding energy. An excitation of simple root 2 yields

$$u_{2,n}\frac{3L}{2\pi n},\delta E_{2,n}=\frac{4\pi ^2n^2}{3L^2}$$
(5.19)
whereas for an excitation of simple root 3 we find

$$u_{3,n}\frac{2L}{2\pi n},\delta E_{3,n}=\frac{4\pi ^2n^2}{2L^2}.$$
(5.20)
The sum of all mode numbers $`n`$ must vanish due to the momentum constraint. Using this we quickly find an approximation to the above exactly solved spin wave.

$`u_1`$ $``$ $`{\displaystyle \frac{3L}{2\pi n}},u_2{\displaystyle \frac{2L}{2\pi (n)}},`$
$`E`$ $``$ $`3L+{\displaystyle \frac{4\pi ^2n^2}{3L^2}}{\displaystyle \frac{4\pi ^2n^2}{2L^2}}=3L{\displaystyle \frac{2\pi ^2n^2}{3L^2}}.`$ (5.21)
It would be interesting if some string theory solution describing this Bethe vacuum could be found. The spectrum of fluctuations, somewhat reminiscent of the plane wave spectrum, could then be compared.

## 6 Outlook

We are confident that our proposal furnishes the tool for very comprehensive “spectroscopic” studies of $`𝒩=4`$ Yang-Mills theory. The recent progress with semi-classical approaches on the string side  allows for very interesting dynamical comparisons with gauge theory. In particular our superchain and its associated Bethe equations should be very useful for string motions that also involve the AdS part of the background, or even joint motion in AdS and on the five sphere. In all these applications string theory is able, in principle, to predict anomalous dimensions for large dimension gauge theory operators, to which the Bethe ansatz is ideally suited: On the level of the spin chain these situations correspond to the thermodynamic limit of the chain, in which the Bethe equations often become tractable .

It would be helpful to classify all possible ways to write Bethe equations for the above $`𝔰𝔩(4|4)`$ super spin chain, and identify the corresponding pseudovacua. Correspondingly one should then find further spin wave solutions, describing the simplest fluctuations around the pseudovacua. It would be interesting to see whether the large dimension limits of the pseudovacuum states and their elementary excitations play a special rôle in string theory.

One puzzling aspect of the observed integrable structures is that they really correspond to “hidden” symmetries appearing in large $`N`$ SYM4. Clearly we would like to understand their presence from the point of view of the gauge theory. Are they a technical consequence of what one could call “planar supersymmetry”, or is there a deeper reason, possibly related to string theory?

However, the most exciting open problem remains to identify the integrable deformation of the super spin chain, with deformation parameter $`\lambda =g_{\mathrm{YM}}^2N`$, that correctly includes the effects of all quantum loops. This presumably would be a $`𝔭𝔰𝔩(4|4)`$ super spin chain with long-range interactions. A related question concerns the closure of the interacting algebra and the consistency requirements imposed by it. How are these related to integrability and can they be used to fix higher-loop corrections?

### Acknowledgements

We would like to thank Gleb Arutyunov, Vladimir Dobrev, Stefano Kovacs, Joe Minahan, Jan Plefka and Kostya Zarembo for interesting discussions, and A. Belitsky, V. Braun, A. Gorsky and G. Korchemsky for email correspondence concerning the integrable structures previously found in QCD. N.B. dankt der *Studienstiftung des Deutschen Volkes* für die Unterstützung durch ein Promotionsförderungsstipendium.

## Appendix A Dissection of the Beast

In this appendix we present some more details on the Bethe ansatz of Sec. 5. We will start by an investigation of highest weight states. Next we will apply this to the Konishi state and its descendants, for which we then solve the Bethe equations. Finally, we present the set of all solutions corresponding to states up to classical dimension four.

### A.1 Highest Weights

A primary state is defined as a state annihilated by the boosts $`K,S,\dot{S}`$. This condition is identical to highest weight condition in the sense of Sec. 4. However, when we change the Dynkin diagram of the superalgebra, we also change the notion of positive roots and highest weights. For the ‘distinguished’ Dynkin diagram of Sec. 5 not all of the boosts $`K,S,\dot{S}`$ can be positive roots simultaneously. Therefore primary weights (where primary weight shall refer to highest weight in the sense of Sec. 4) are not highest weights. Instead, a highest weight state is annihilated by $`K,S,\dot{Q}`$.

For a given primary weight we need to find the corresponding highest weight. For a generic long multiplet with primary weight $`w`$ this is achieved by adding

$$\delta w=[4;0,0;0,0,0;4,0].$$
(A.1)
It corresponds to acting with $`\dot{Q}^8`$ on the primary state. A further application of $`\dot{Q}`$, as well as $`K,S`$ which commute with $`\dot{Q}^8`$, will therefore kill the state.

In the classical $`𝔰𝔩(4|4)`$ there are various shortening conditions that affect the highest weight. For a half-BPS state we add

$$\delta w_{1/2}=[2;0,0;0,2;0;2,0]$$
(A.2)
instead. The fundamental multiplet $`V_\mathrm{F}`$ with primary weight $`[1;0,0;0,1,0;0,1]`$ is subject to an additional shortening condition. The highest weight is $`[2;2,0;0,0,0;1,1]`$ and corresponds to the field $`_{1+i2,3+i4}`$ of $`𝒩=4`$. The vacuum is constructed from exactly this field.

There are two shortening conditions for unprotected multiplets, see (4). When shortening condition I applies, the multiplet splits in two with highest weights

$$w+\delta w+\delta w_\mathrm{I},w+\delta w_{\mathrm{I}+}.$$
(A.3)
where

$`\delta w_\mathrm{I}`$ $`=`$ $`[0;0,0;0,0,0;1,+1],`$
$`\delta w_{\mathrm{I}+}`$ $`=`$ $`\{\begin{array}{c}[+3.5;+1,0;1,0,0;+3.5,0],\hfill \\ [+3.0;+2,0;0,1,0;+3.0,0],\hfill \\ [+2.5;+3,0;0,0,1;+2.5,0],\hfill \\ [+2.0;+4,0;0,0,0;+2.0,0].\hfill \end{array}`$ (A.8)
For $`\delta w_{\mathrm{I}+}`$ it is understood that the topmost line applies for which the resulting labels are positive. When shortening condition II applies, the highest weights are

$$w+\delta w,w+\delta w_{\mathrm{II}}+\delta w.$$
(A.9)
with

$$\delta w_{\mathrm{II}}=\{\begin{array}{c}[+0.5;0,1;0,0,+1;+0.5,+1],\hfill \\ [+1.0;0,0;0,0,+2;0.0,+1].\hfill \end{array}$$
(A.10)
Finally, if both shortening conditions apply, the four highest weights are

$$\begin{array}{c}w+\delta w+\delta w_\mathrm{I},\hfill \\ w+\delta w_{\mathrm{I}+},\hfill \\ w+\delta w_{\mathrm{II}}+\delta w+\delta w_\mathrm{I},\hfill \\ w+\delta w_{\mathrm{II}}+\delta w_{\mathrm{I}+}.\hfill \end{array}$$
(A.11)
Note that to determine the last highest weight, it is necessary to choose the correct $`\delta w_{\mathrm{I}+}`$ for $`w+\delta w_{\mathrm{II}}`$. The submultiplets join in the interacting theory. In each of the three cases (A.3),(A.9),(A.11) we have listed the highest weight of the long multiplet first.

### A.2 Konishi Submultiplets

The Konishi state has primary weight

$$w_0=[2;0,0;0,0,0;0,2].$$
(A.12)
This is at both unitarity bounds. According to the rules of the previous section we find the highest weights of the four submultiplets

$`w_1`$ $`=`$ $`[4.0;4,0;0,0,0;2.0,2],`$
$`w_2`$ $`=`$ $`[6.0;0,0;0,0,0;3.0,3],`$
$`w_3`$ $`=`$ $`[5.5;3,0;0,0,1,2.5,3],`$
$`w_4`$ $`=`$ $`[7.0;0,0;0,0,2;3.0,4].`$ (A.13)
The excitation numbers are determined by (5.6)

$`n_{1,k}`$ $`=`$ $`(0,0,0,0,0,0,0),L=2,`$
$`n_{2,k}`$ $`=`$ $`(0,0,3,0,0,0,0),L=3,`$
$`n_{3,k}`$ $`=`$ $`(0,0,2,1,0,0,0),L=3,`$
$`n_{4,k}`$ $`=`$ $`(0,0,5,2,0,0,0),L=4.`$ (A.14)
The first configuration is the vacuum, we find the energy $`E=3L=6`$ straight away.

For the second configuration with $`k_j=(3,3,3)`$ we note that the roots must be invariant under the map $`u_ju_j`$ because the Konishi state is unpaired. Therefore one root is zero while the other two sum to zero. The only way to satisfy the momentum constraint is to pick the singular configuration

$$u_{1,2}=\pm i,u_3=0.$$
(A.15)
This has to be regularized in the usual way, we again find the energy $`E=6`$. As an aside, this state has the highest weight for the interacting multiplet. At leading order, it corresponds to the operators $`\epsilon ^{\beta \gamma }\epsilon ^{\delta \epsilon }\epsilon ^{\eta \alpha }Tr_{\alpha \beta }_{\gamma \delta }_{\epsilon \eta }`$ of $`𝒩=4`$ SYM.

For the third configuration with $`k_j=(3,3,4)`$ it is straightforward to find and verify

$$u_{1,2}=\pm \frac{1}{\sqrt{3}},u_3=0$$
(A.16)
as a solution to the Bethe equations and momentum constraint with energy $`E=6`$.

For the last configuration with $`k_j=(3,3,3,3,3,4,4)`$ we again require a singular solution. Taking the roots (A.15) as an ansatz, the remaining roots can be found

$$u_{1,2}=\pm i,u_3=0,u_{4,5}=\pm \frac{1}{\sqrt{3}},u_{6,7}=\pm \sqrt{\frac{7}{20}}.$$
(A.17)
They satisfy the momentum constraint and yield, not surprisingly, $`E=6`$.

We have seen that all submultiplet of the Konishi multiplet have the same energy. This is somewhat remarkable, as they have rather different configurations of roots.

### A.3 Bethe Roots for Dimension Four States

Here we will present the Bethe roots for operators up to dimensions four. This demonstrates that also the ‘distinguished’ choice of Dynkin diagram with a pseudovacuum leads to the correct results. We present the complete list of operators and anomalous dimensions in Tab. 1. The Bethe roots which solve the Bethe equations and momentum constraint are given below. The energies agree with Tab. 1, we do not list them again.

#### \[2;0,0;0,2,0;0,2\].

$`k_j=(3,3)`$, $`L=2`$. See (5.12).

$$u_{1,2}=\pm \frac{i}{\sqrt{3}}.$$
(A.18)
#### \[2;0,0;0,0,0;0,2\].

$`k_j=()`$, $`L=2`$. Trivial.

#### \[3;0,0;0,3,0;0,3\].

$`k_j=(3,3,3,3,4,4,5)`$, $`L=3`$.

$$u_{1,2,3,4}=\pm \sqrt{\frac{3\pm i\sqrt{15}}{12}},u_{5,6}=i\sqrt{\frac{5}{12}},u_7=0.$$
(A.19)
#### \[3;0,0;0,1,0;0,3\].

$`k_j=(3,3)`$, $`L=3`$.

$$u_{1,2}=\pm \frac{i}{\sqrt{5}}.$$
(A.20)
#### \[4;0,0;0,4,0;0,4\].

$`k_j=(3,3,3,3,3,3,4,4,4,4,5,5)`$, $`L=4`$. This is left as an exercise for the reader. Good luck!

#### \[4;0,0;0,2,0;0,4\].

$`k_j=(3,3,3,3,4,4,5)`$, $`L=4`$. Two states distinguished by $`\pm ^{}`$.

$$u_{1,2,3,4}=\pm \sqrt{\frac{\pm ^{}\sqrt{5}\pm i(5\pm ^{}3\sqrt{5})}{15}},u_{5,6}=\pm \sqrt{\frac{15^{}2\sqrt{5}}{36}},u_7=0.$$
(A.21)
#### \[4;0,0;1,0,1;0,4\].

$`k_j=(3,3,3,3,4)`$, $`L=4`$.

$$u_{1,2,3,4}=\frac{\pm 1\pm i}{\sqrt{2\sqrt{3}}},u_5=0.$$
(A.22)
#### \[4;0,0;0,0,0;0,4\].

Two states distinguished by $`\pm ^{}`$. $`k_j=(3,3,3,3)`$, $`L=4`$.

$$u_{1,2,3,4}=\pm \sqrt{\frac{9^{}\sqrt{41}}{12}\pm \sqrt{\frac{173\pm ^{}33\sqrt{41}}{360}}}.$$
(A.23)
#### \[4;2,0;0,0,0;1,3\].

$`k_j=()`$, $`L=3`$. Trivial.

#### \[4;0,2;0,0,0;$`1`$,3\].

$`k_j=(2,2,3,3,3,3)`$, $`L=3`$.

$$u_{1,2}=\pm \sqrt{\frac{17}{60}},u_{3,4,5,6}=\pm \sqrt{\frac{39\pm \sqrt{993}}{30}}.$$
(A.24)
#### \[4;1,1;0,1,0;0,3\].

$`k_j=(2,3,3)`$, $`L=3`$. A pair of states distinguished by $`\pm ^{}`$.

$$u_1=\pm ^{}\sqrt{\frac{5}{28}},u_{2,3}=^{}\sqrt{\frac{1\pm i\sqrt{399}}{30}}.$$
(A.25)
#### \[4;2,2;0,0,0;0,2\].

$`k_j=(2,2)`$, $`L=2`$.

$$u_{1,2}=\pm \sqrt{\frac{9}{28}}.$$
(A.26)


format: text

text:
# On the cross correlation between the arrival direction of ultra-high energy cosmic rays, BL Lacertae, and EGRET detections: A new way to identify EGRET sources?

## 1. Introduction

Farrar and Biermann (1998) pointed out the existence of a directional correlation between compact radio quasars (QSOs) and ultra-high energy cosmic rays (UHECRs). Their claim was supported by two significant factors: (i) there was an a priori postulated theoretical reason by which to expect such an alignment, i.e. the existence of a new neutral hadron that would travel unscathed all the way to the Earth (Farrar 1996, Chung et al. 1998) or neutrinos producing $`Z`$-bursts (Weiler 1999; Fargion, Mele & Salis 1999; Fodor, Katz & Ringwald 2002), and (ii) all events at the high end of the spectrum observed by that time, with energy at least 1$`\sigma `$ above $`10^{19.9}`$ eV, were aligned with high redshifted quasars, a phenomenon with a chance probability of occurrence less than 0.5% (Farrar & Biermann 1998). This report quickly opened a large debate on whether UHECRs can evade interactions with cosmic microwave background photons, and arrive at Earth evading the Greisen (1966) – Zatsepin–Kuzmin (1966), GZK, cutoff. Most of the alternative explanations to evade the GZK cutoff require physics beyond the standard model (for an exception see Kalashev et al. 2001), including violation of local Lorentz invariance (Coleman & Glashow 1998), photon-axion mixing (Csaki et al. 2003), and neutrinos interacting strongly in the atmosphere (due to physics beyond the –perturbative– Standard Model, Fodor et al. 2003; or because of TeV-scale gravity, Domokos & Kovesi-Domokos 1999, Jain et al. 2000). The latter, however, is severely constrained by observations (Anchordoqui et al. 2001). In this Letter, we first comment on the status of the correlation between QSOs and UHECRs, and then analyze more recent strong claims for a correlation between UHECRs and BL Lacertae (BL Lacs), a subgroup of the previously studied QSOs. Finally, we scrutinize a newly proposed classification technique of EGRET sources, based on the cross correlation of BL Lacs, UHECRs, and $`\gamma `$-ray catalogs.

## 2. QSOs and UHECRs

The possible correlation between UHECRs and QSOs was subject to a great deal of scrutiny. Hoffman (1999) stated that one of the 5 events used in the Farrar and Biermann (1998) study, the highest energy event observed by the Fly’s Eye experiment (Bird et al. 1995), should not be included in the UHECR sample under analysis, because this very same event was previously considered to introduce the hypothesis. Without this event, the positive alignment with random background probability is increased to $`<3\%`$ (Farrar & Biermann 1999). Using an updated event list (twice the size of the previous) from the Haverah Park (Ave et al. 2000) and the AGASA (Hayashida et al. 2000) experiments, Sigl et al. (2001) showed that the statistical significance of the alignment is lowered to 27%. More recently, Virmani et al. (2002) favored the earlier proposed alignment. However, it should be stressed that most of the Virmani et al. correlation signal comes from events with large uncertainty both in energy and in position: they considered events from the SUGAR experiment, but it is not clear whether these events were above the GZK cut-off (see, e.g., Anchordoqui et al. 2003).

Very recently, the Haverah Park energy estimates have been re-assessed (Ave et al. 2003). For the cosmic rays in question, the energy of the 2 events observed by this array with incident zenith angle $`<45^{}`$, that was previously quoted as $`>10^{19.9}`$ eV at 1$`\sigma `$, is now shifted $`30\%`$ downwards, below the energy cut chosen by Farrar and Biermann (1998). Hence, independently of the statistical test used, when considering only the highest energy ($`>10^{19.9}`$ eV at 1$`\sigma `$) events111Those events would be most interesting for new physics, because they have no contamination from the expected proton pile-up around the photopion production threshold. the correlation between UHECRs and QSOs is consistent with a random distribution at the $`1\sigma `$ level.

## 3. BL LACs and UHECRs

In a series of recent papers, Tinyakov and Tkachev (2001, 2002, 2003) claim a correlation between the arrival directions of UHECRs and BL Lacs, a subgroup of the QSO sample previously considered. Specifically, the BL Lacs chosen were those identified in the (9th-Edition) Veron-Cetty and Veron (2000) catalogue of Quasars and Active Galactic Nuclei, with redshift $`z>0.1`$ or unknown, magnitude $`m<18`$, and radio flux at 6 GHz $`F_6>0.17`$ Jy.222The catalogue of Quasars and Active Galactic Nuclei is regularly updated, see Veron-Cetty and Veron (2001) for the 10th-Edition. The 9th-Edition is electronically available at http://www.obs-hp.fr/www/catalogues/veron2$`\mathrm{\_}`$9/veron2$`\mathrm{\_}9`$.html. Only 22 objects fulfill such restrictions. In this analysis there is no buffer against contamination by mismeasured protons piled up at the GZK energy limit. The CR sample of Tinyakov and Tkachev consists of 26 events measured by the Yakutsk experiment with energy $`>10^{19.38}`$ eV (Afanasiev et al. 1996), and 39 events measured by the AGASA experiment with energy $`>10^{19.68}`$ eV (Hayashida et al. 2000). The evidence supporting their claim is based on 6 events reported by the AGASA Collaboration (all with average energy $`<10^{19.9}`$ eV), and 2 events recorded with the Yakutsk experiment (both with average energy $`<10^{19.6}`$ eV), which were found to be within $`2.5^{}`$ of 5 BL Lacs contained in the restricted sample of 22 sources. The chance probability for this coincidence set-up is found to be $`2\times 10^5.`$

One drawback of the claim made by Tinyakov and Tkachev (2001) is that the data set used to make the initial assertion is also being used in the hypothesis testing phase. Note that if enough searches are performed on a finite data set which is sampled from an isotropic distribution, some highly significant positive results are certain to occur due to the statistical fluctuations that necessarily arise in any finite sampling. Evans, Ferrer and Sarkar (2002) already called into question whether the selection criteria for the subset of brightest BL Lacs are unbiased. Strictly speaking, Tinyakov and Tkachev imposed arbitrary cuts on the BL Lac catalogue so as to maximize the signal-to-noise ratio, compensating a posteriori the different cut adjustments by inclusion of a penalty factor. Without these arbitrary cuts, the significance of the correlation signal is reduced at the 1$`\sigma `$ level (Evans, Ferrer & Sarkar 2002). Moreover, even in acceptance of this a posteriori approach, the estimated value of the penalty factor is subject to debate (Evans, Ferrer & Sarkar 2002; Tinyakov & Tkachev 2003).

Given the pivotal role played by the penalty factor in testing the hypothesis with a single set of data, it is of interest to circumvent this ambiguity by performing a blind analysis. We have at our disposal the cosmic ray arrival directions of the Haverah Park (Stanev et al. 1995) and Volcano Ranch (Linsley 1980) experiments, which, although not useful to distinguish a positive correlation (because the penalties involved are probably already as large as the signal which one expects to test), they provide the framework to disregard the correlation if none is found in the data.

Surface arrays in stable operation have nearly continuous observation over the entire year, yielding a uniform exposure in right ascension. However, the declination distribution is different for each experiment, because the relative efficiency of the detection of events depends upon the latitude of the array and detector type. As shown by Uchihori et al. (2000), the field of view of AGASA + Yakutsk is roughly equal to that of Volcano Ranch + Haverah Park. It is noteworthy that even though the energy of the Haverah Park events has been reduced by about 30% (Ave et al. 2003), the 27 events contained in our sample, originally with energy $`>10^{19.6}`$ eV (Lawrence, Reid & Watson 1991), are well above the energy cut for Yakutsk’s events selected by Tinyakov and Tkachev. Combined with the 6 events recorded at the Volcano Ranch with energy $`>10^{19.6}`$ eV (Linsley 1980), we have a virgin data-set of 33 events, amounting to half of the cosmic-ray arrival directions used to make the claim.

In Fig. 1 we plot the position on the sky in galactic coordinates of both the UHECRs and the selected BL Lacs. There are no positional coincidences between these two samples up to an angular bin $`>5^{}`$. Such an angular scale is well beyond the error in arrival determination, which is found to be $`3^{}`$ (Uchihori et al. 2000). On the basis of the strongly correlated sample analyzed by Tinyakov and Tkachev, one expects the distribution describing the correlation between the set of BL Lacs and any UHECR data-set with 33 entries to be Poisson with mean $`4.06.`$ Taking the data at face value, this implies a $`2\sigma `$ deviation effect. Moreover, the 95% CL interval of the distribution which samples the correlation between the BL Lacs and cosmic rays recorded by Volcano Ranch + Haverah Park is (0, 3.09) (see, e.g. Feldman & Cousins 1998). Therefore, the probability to measure the expected mean value $`4.06`$ is $`5\%`$. All in all, the 8 coincidences in the Tinyakov and Tkachev (2001) analysis do not represent a statistically significant effect.

## 4. UHECRs and EGRET AGNs

On a similar track, Gorbunov et al. (2002) claimed that a set of $`\gamma `$-ray loud BL Lac objects can be selected by intersecting the EGRET and BL Lacs catalogs. The only requirement that Gorbunov et al. considered for a BL Lac to be physically associated with an EGRET source is that the angular distance between the best estimated position of the pair does not exceed $`2R_{95}`$, where $`R_{95}`$ is the 95% CL contour of the EGRET detection.

Their claim was based on a positional correlation analysis (using the doubled size for EGRET sources) between the Third EGRET Catalog (3EG, Hartman et al. 1999) and the objects identified as BL Lac in the Veron-Cetty & Veron (2000) Catalog. This results in 14 coincidences, 4 of which are further found to be part of the 5 BL Lacs located within 2.5 of UHECRs discussed above.

The typical $`R_{95}`$ radius for EGRET sources is 0.5–1. Because of such large uncertainties, a standard practice in $`\gamma `$-ray studies aiming to give preliminary associations between EGRET sources and possible counterparts is to study, in addition to the object being proposed, any other coincident system able to generate photons in the EGRET range (100 MeV–10 GeV). All of the latter should be discarded as the origin of the high energy radiation in order for the association claim to persist. This process usually involves theoretical modelling and multiwavelength observations (see e.g. Caraveo 2002, Reimer et al. 2001, Torres et al. 2003a, and references therein).

The case of active galactic nuclei (AGNs) as EGRET counterparts has been analyzed by Mattox, Hartman and Reimer (2001), who provided a spatial-statistical assessment. They list 46 EGRET sources with high probability of being identified with known blazars, confirming 45 AGN identifications made in the 3EG (Hartman et al. 1999). A further 39 EGRET sources have been listed as plausible AGN identifcations. In Fig. 2 (left panel) we show the positional offset between the maximum-likelihood-algorithm for the $`\gamma `$-ray source position (Mattox et al. 1996) and the radio position of the respective AGN identification. We show separately those AGN considered as high-confidence and plausible counterparts.

Independent support for some of these AGN identifications results from dedicated multifrequency counterpart observations, including spectroscopic confirmation of their blazar nature (Sowards-Emmerd, Romani & Michelson 2003, Halpern, Eracleous & Mattox 2003). In contrast to the offset distribution between $`\gamma `$-ray and radio position of confirmed (or at least, most probable) AGN identifications, the offset distribution of the newly suggested AGN counterparts by Gorbunov et al. (2002), shown in Fig. 2 (middle panel), has a completely different shape. We attribute this to the inappropriate consideration of source localization uncertainties of EGRET-detected $`\gamma `$-ray sources, which lead Gorbunov et al. to suggest counterparts well into the range of $`2R_{95}`$ of an individual EGRET source. This extension of the EGRET angular uncertainty was motivated in a discrepancy between the radio and the $`\gamma `$-ray position of the Vela pulsar, which will be discussed below. For comparison, Fig. 2 (right panel) gives the offset distribution for the radio and $`\gamma `$-ray positions of identified pulsars in the EGRET data. Clearly, the range of absolute offset values, based on phase selected $`\gamma `$-ray events, is minimal in the case of the pulsars.

The 3EG source related to Vela is a very special case. It is the strongest known $`\gamma `$-ray source, and one of the best localized, $`R_{95}=0.021^{}`$. The 95% CL contour of the EGRET detection and the offset with Vela are both one order of magnitude less than the typical values of these quantities in the 3EG. The misplacing for Vela occurs because the analysis technique privileges the discovery and correct detection of weaker sources, and it is applied to all EGRET sources in the 3EG (2/3 of which are unidentified with no obvious candidates) identically. The offset of the Vela position and, in general, of bright sources, is minimized by using map bins smaller than the standard 0.5 used in the 3EG. This increases the computation time greatly; and since all of the most significant sources were identified with objects whose positions were well known, the smaller bin size was not adopted to give source positions in the 3EG. See the comment on the source 3EG J0834-4511 (Vela) in the section of particular detections of the 3EG Catalog (Hartman et al. 1999). Systematics, then, do not pose a major problem for the source location capability of EGRET, even in regions of significant diffuse emission or strong nearby sources (Hartman et al. 1999). Most importantly, the error contours for many of the AGNs show that the location capability improves for regions away from the Galactic plane, where most of the blazars are.

In addition, some of the 3EG-associated AGNs could be false positives (i.e. AGNs that are mis-associated with EGRET sources by a failure of the statistical methods used in the classification). This fact is particularly important for statistical methods based only on the relative positions between the candidate and the EGRET source center (see Torres 2003b for a review). Working with 114 sources above $`|b|>10^\mathrm{o}`$, Punsly (1997) has estimated the number of random coincidences as a function of the field radius: $`2`$ (10) quasars with more than 1 Jy of 5 GHz flux are expected to correlate by random chance if the size of the typical EGRET angular uncertainty is 0.7o (1.7o). The number of random coincidences increases as the radio-loudness of the AGN decreases (since there are more AGNs with smaller flux). This sheds additional doubt on the correlations found beyond the 95% location contours of EGRET sources.

## 5. Concluding Remarks

Available statistics on the arrival directions of the UHECRs reveals no significant correlations above random with BL Lacs nor with any other type of quasars, including EGRET blazar detections. Furthermore, identifying EGRET sources with BL Lacs just by positional pairing within twice the EGRET error grossly underestimates the goodness of existing gamma-ray data. 333In closing, we comment on the apparent correlation between nearby dormant quasar remnants (“dead quasars”) and the arrival directions of the 38 UHECRs measured by the AGASA experiment with energy $`>10^{19.6}`$ eV and $`|b|>20^{}`$ (Torres et al. 2002). We note that NGC 2300 (which is the only quasar remnant hosted in a non-elliptical galaxy) is actually beyond the reach of the AGASA experiment, which improves a little the correlation presented by Torres et al. (2002). Since there are 25 events with $`|b|>20^{}`$ (some of them with energy $`<10^{19.6}`$ eV) in the data sample which combines measurements of the Volcano Ranch and the Haverah Park experiments, the dead quasar hypothesis (Boldt & Ghosh 1999) is limited by the statistics of small numbers (the expected mean value being $`1.31`$ is just on the verge of the 68.27% CL interval, Feldman & Cousins 1998), and the hypothesis awaits further testing with larger sets of data.

We gratefully acknowledge Elihu Boldt, Francesc Ferrer, Haim Goldberg, Tom McCauley, Andreas Ringwald, Subir Sarkar, Peter Tinyakov, and Alan Watson, for useful discussions/email correspondence. We further acknowledge Alan Watson for his kind permission to use the Haverah Park data. The work of D.F.T. was performed under the auspices of the U.S. D.O.E. (NNSA), by University of California Lawrence Livermore National Laboratory under contract No. W-7405-Eng-48. The research of S.R. and L.A.A was partially supported by the US National Science Foundation (NSF) under grant No. PHY-0140407.


text:
# Predicting Crystal Structures with Data Mining of Quantum Calculations

## Abstract

Predicting and characterizing the crystal structure of materials is a key problem in materials research and development. It is typically addressed with highly accurate quantum mechanical computations on a small set of candidate structures, or with empirical rules that have been extracted from a large amount of experimental information, but have limited predictive power. In this letter, we transfer the concept of heuristic rule extraction to a large library of ab-initio calculated information, and demonstrate that this can be developed into a tool for crystal structure prediction.

Ab-initio methods, which predict materials properties from the fundamental equations of quantum mechanics, are becoming a ubiquitous tool for physicists, chemists, and materials scientists. These methods allow scientists to evaluate and pre-screen new materials “in silico”, rather than through time-consuming experimentation, and in some cases, even make suggestions for new and better materials . One inherent limitation of most ab-initio approaches is that they do not make explicit use of results of previous calculations when studying a new system. This can be contrasted with data-centered methods, which mine existing data libraries to help understand new situations. The contrast between data-centered and traditional ab-initio methods can be seen clearly in the different approaches used to predict the crystal structure of materials. This is a difficult but important problem that forms the basis for any rational materials design. In heuristic models, a large amount of experimental observations are used in order to extract rules which rationalize crystal structure with a few simple physical parameters such as atomic radii, electronegativities, etc.. The Miedema rules for predicting compound forming , or the Pettifor maps  which can be used to predict the structure of a new binary material by correlating the position of its elements in the periodic table to those of systems for which the stable crystal structure is known, are excellent examples of this. Ab-initio methods differ from these data-centered methods in that they do not use historic and cumulative information about previously studied systems, but rather try to determine structure by optimizing from scratch the complex quantum mechanical description of the system, either directly (as in ab-initio Molecular Dynamics), or in coarse-grained form (as in lattice models ). Here, we merge the ideas of data-centered methods with the predictive power of ab-initio computation. We propose a new approach whereby ab-initio investigations on new systems are informed with knowledge obtained from results already collected on other systems. We refer to this approach as Data Mining of Quantum Calculations (DMQC), and demonstrate its efficiency in increasing the speed of predicting the crystal structure of new and unknown materials. Using a Principal Component Analysis on over 6000 ab-initio energy calculations, we show that the energies of different crystal structures in binary alloys are strongly correlated between different chemical systems, and demonstrate how this correlation can be used to accelerate the prediction of new systems. We believe that this is an interesting new direction to address in a practical manner the problem of predicting the structure of materials.

Using Density Functional Theory we have calculated a library of ab-initio energies for 114 different crystal structures in each of 55 binary metallic alloys. About 1/3 of the crystal structures in the library were chosen from the most common binary crystal structures in the CRYSTMET database for intermetallics . The rest are superstructures of the fcc, bcc, and hcp lattices. The alloys include all 45 binaries that can be made from row 4 transition metals, as well as ScAl, AgMg, and 8 binary Ti alloys (AgTi, CdTi, MoTi, PdTi, RhTi, RuTi, TcTi, TiZr). The formation energy for each structure is determined with respect to the most stable structure of the pure elements. Energy calculations were done using density functional theory, in the local density approximation, with the Ceperley-Alder form for the correlation energy as parameterized by Perdew-Zunger  with ultrasoft pseudopotentials, as implemented in VASP . Calculations are at zero temperature and pressure, and without zero-point motion. The energy cutoffs in an alloy was set to 1.5 times the larger of the suggested energy cutoffs of the pseudopotentials of the elements of the alloy (suggested energy cutoffs are derived by the method described in ). Brillouin zone integrations were done using 2000/(number of atoms in unit cell) $`𝐤`$-points distributed as uniformly as possible on a Monkhorst-Pack mesh. We verified that with these energy cutoffs and k-points mesh the absolute energy is converged to better then 10 meV/atom. Energy differences between structures are expected to be converged to much smaller tolerances. Spin polarization was not used as no magnetic alloys were studied. All structures were fully relaxed.

For each alloy $`i`$, consider the 114 structural formation energies as the components of a vector $`𝐄_i`$ in a 114-dimensional space. If the energies of the structures are linearly dependent then the vectors for each alloy will not be distributed randomly in the 114 dimensional space, but confined in a subspace of reduced dimension. To look for such approximate linear dependencies we use Principal Component Analysis  (PCA). This allow us to express the energy vector of an alloy as an expansion in a basis of reduced dimension $`d`$, $`𝐄_i=_{j=1}^d\alpha _{ij}𝐞_i+ϵ_i(d)`$, where $`ϵ_i(d)`$ is the error vector for the alloy $`i`$. PCA consists of finding the proper basis set $`\left\{𝐞_j(d)\right\}`$ that minimizes the remaining squared error $`_iϵ_i^Tϵ_i`$ for a given dimension $`d`$. These optimum basis vectors $`\left\{𝐞_j(d)\right\}`$ are called the Principal Components (PC’s), and form a set of orthogonal vectors ordered by the amount of variation of the original sample they can explain. More intuitively, they are a new set of axes in the 114 dimensional space, ordered according to the fraction of the data lying along that axis. As an extreme example, if the energies of all 55 alloys were proportional each other, then all the alloy vectors would lie along a single line, and the first PC would be a subspace that encompassed all the data $`(d=1)`$.

A Principal Components Analysis of our ab-initio data set (Figure 1) shows that significant dimension reduction is possible in the space of structural energies. The solid curve, labelled “55”, in Figure 1 shows the remaining unexplained Root Mean Square (RMS) error (average error in the 114 structural energies of the 55 alloys), as function of the number of principal components $`d`$. All quantities are given as energy per atom. The number of relevant dimensions depends on the error one can tolerate. For example, to describe the energies with a 50 meV RMS error, only 9 dimensions are required, much less than the original 114. The implication is that it is possible to perform far fewer than 114 calculations to parameterize the 9 dimensional subspace, and then derive the other 105 energies through linear relationships given by the PCA.

Dimension reduction holds only because the energy differences of structures are strongly correlated between alloys with different chemistry. In fact, if we perform a PCA analysis in which the structural energies for each alloy are randomly permuted, and hence destroy their relations, there is little opportunity for dimension reduction, as the dashed curve in Figure 1 shows. Given an acceptable accuracy, dimension reduction does not depend on the dimension of the library, once the library is bigger than a certain size. Figure 1 shows the PCA analysis for 35, 45, and 55 alloys. For subspaces defined by up to $`20`$ PC’s (27 meV RMS accuracy) the variance is essentially independent of the number of alloys, indicating that the dimension reduction we obtain can be expected to apply to new alloy systems.

These correlations are further confirmation that the success of heuristic methods is not accidental, and that with relatively few parameters it can be possible to predict the structure of a binary alloy. In fact, these correlations can be used to develop an ab-initio-data-mining algorithm that rapidly searches through the available space of possible structures.

Given a library of $`N_a`$ alloys, $`N_s`$ structures, and a new alloy where the first $`n`$ energies have been calculated, we predict the energy for structure $`i>n`$ of the new alloy as follows. Define $`𝐗`$ as the $`(n,N_a)`$ matrix of energies for structures $`\{1\mathrm{}n\}`$ in the library. Define $`𝐲`$ as the $`n`$component vector of known energies for the new alloy and $`𝐗^{}`$ as the $`N_a`$ component vector of energies for structure $`i`$ for all alloys in the library. The scalar $`y^{}`$ represents the unknown energy of structure $`i`$ for the new alloy. We regress $`𝐲`$ on $`𝐗`$ using the Partial Least Squares method  implemented with the SIMPLS algorithm. The resulting regression coefficients are used to predict $`y^{}`$ from $`𝐗^{}`$. This is done for every structure of the new alloy for which the energy has not yet been calculated.

The ground states for an alloy are found through an iterative scheme. At each step, the PLS regression is used to find the most probable ground state, which is then calculated with quantum mechanics and added to the data. The algorithm is started with only the pure element energies for the two elements of the alloy in the bcc, fcc, and hcp structures, and then proceeds as follows.
Step 1 (prediction). The regression algorithm given above is used to predict all unknown structural energies in the new system. We found that for early iterations ($`<10`$) the RMS error can be reduced by preclustering the library into ordering and phase-separating systems and regressing only within the library subcluster in which the system is predicted to fall. Physically, this means that for early stage of the iterative procedure, new alloys regress better with similar alloys than with the complete library.
Step 2 (suggestion). With the available calculated energies we determine the ground-state energy versus composition curve (convex-hull). The convex hull, which is the set of tie lines that connects the lowest energy ordered phases, represents the free energy of the alloy at 0K. Structures with energy above a tie line, are unstable with respect to mixtures of the two structures that define the vertices of the tie line. Hence, the convex hull determines the phase stability of the system at zero temperature. The structure with predicted energy farthest below the convex hull of calculated energies is calculated with quantum mechanics and added to the database. If no structure breaks the hull, we look for the structure predicted to be closest to the hull. For early iterations ($`<13`$ in Figure 2), if no such structure can be found within 80 meV of the ground state hull, we consider the prediction to have failed in this step, and instead add the most frequent and not yet calculated ground state structure of the database.
Step 3 (calculation). The candidate suggested structure is then calculated with Quantum Mechanics and added to the list, and the entire process is iterated (prediction $``$ suggestion $``$ calculation). With each step, more energetic information for the new alloy is incorporated and a better prediction of the ground state can be expected.

Any structure in the library can be predicted and there are no preconceived biases as to the symmetry or underlying superlattice of the structure as is the case for methods that work with lattice model approaches. For example, in the Ti-Pt alloy, our method correctly finds the A15  structure to be a ground state for Ti3Pt after only 20 steps in the algorithm, even though this structure is not a superstructure of fcc (the structure of Pt) or hcp (the structure of Ti), and is therefore not an obvious structure to investigate for this system. To study in a more statistically significant way how this iterative scheme converges we tested how well the library minus one alloy can perform predictions on the alloy left out. A key property is whether the alloy is immiscible (no ordered compounds) or has intermediate compounds (compound-forming). Empirical schemes like the one developed by Miedema  have been particularly successful in classifying this difference. We find that DMQC can predict whether an alloy excluded from the library is compound-forming with 95%, 98%, and 100% accuracy using 3, 6, and 13 calculations, respectively. Note that here and below, we do not count the initial pure element calculations, since these are only performed once for each element. For comparison, if one randomly picked trial structures from the list of 114 structures, predictions with 95%, 98%, and 100% accuracy require 7, 21, and 98 calculations, respectively. The DMQC method performs extremely well, far better than a naive random choice of structures, and gives almost perfect prediction with a small amount of computation.

A more stringent evaluation is whether the correct stable crystal structures are predicted for the system left out. Figure 2 (solid line) shows the number of calculations required as a function of the percentage of ground states predicted correctly (averaged over all alloys). For our purpose, “correct” is what would be obtained from the direct quantum mechanical calculations on all 114 structures. Ninety percent accuracy can be achieved with less than 26 calculations for an alloy. To achieve the same confidence level with random structure selection (dashed line) one needs to calculate almost the complete database (98 calculations).

Even though it is generally believed that the binary alloys are well characterized experimentally, our approach can be used to quickly predict previously unknown stable structures in some systems. For example, with only 26 calculations we predict Ag3Cd and Ag2Cd respectively to have the DO24 and C37 structure. In addition, we predict the previously unidentified structure for CdZr3 to be A15 (Cr3Si-type). This prediction takes only 21 iterations and is particularly interesting since A15 does not share the hcp parent lattice of Cd and Zr. These predictions were confirmed by calculation of all the structures in the library. A more detailed analysis of the predictions made from our database in a large number of systems will be published elsewhere.

More structures will need to be added to the library to give the method better applicability to many unknown systems. It is therefore important to assess how the number of required calculations scales with the number of structures in the library. This scaling is shown in Figure 3 for various required confidence levels. As the library grows, more calculations are needed to select between the increasing number of possibilities. Fortunately, the number of calculations increases less than linearly with the number of structures in the database, demonstrating that efficiency increases as the library grows.

Our current DMQC approach has the limitation that structure types must already be in the database to be predicted. However a concerted effort to develop a public database, analogous to those used in biology, may make this limitation less important. Our work has also focussed on a simple test library of binary alloys. The real payoff will come with the inclusion of multicomponent systems, where fewer than 10% of all intermetallic systems have been characterized . A library of ternary structures can be integrated with the binary libraries and extensions of the formalism are not required, besides adding an extra composition variable. Although the data mining methods discussed here are centered around dimension reduction and linear correlation approaches, including nonlinear methods (e.g., neural nets, clustering, learning machines, etc.) will certainly be more effective in extracting information from the library. For example, problems associated with using a single set of regression coefficients for the whole heterogeneous data set can be avoided by preclustering the data and using linear regression only within each cluster.

In summary, by data mining quantum mechanical calculations (DMQC) we have established that there exist significant correlations among ab-initio energies of different structures in different materials. The correlations we found can be seen as a formal extension of the heuristic structure-properties selection rules that have been established in the past on the basis of large amounts of experimental structure information . Our approach differs from the previous classifications in that we correlate on calculated information (structural energies in our particular example), and hence our description can be used when there is limited experimental data, and can be extended to arbitrary accuracy.

The data-mined correlations form the basis for an efficient algorithm for structure prediction which has all the capacities of ab-initio energy methods, but extracts information from previous calculations on other systems in order to efficiently propose candidate structures. Because structures are not found through optimization of some physical variable space (e.g. atomic coordinates), it has none of the problems with time-scale and equilibration common to other approaches. We believe that the integration of data mining techniques with ab-initio methods is a promising development towards the practical prediction of crystal structure.

The research was supported by the Department of Energy, Office of Basic Energy Science under Contract No. DE-FG02-96ER45571.