text:
# Complex Grid Computing

## 1 Introduction

Among the many implications of the scientific and technological advances in microelectronics along the last decades, the availability of microprocessors characterized by ever diminishing size, cost and power consumption (per operation), together with increasing computing power Culler99 , has led to an unprecedented opportunity for parallel computing, allowing simulations of non-linear and complex physical systems. More recently, the advent of the Internet paved the way for using a network of computers to obtain a very large and powerful computing system, defining the new research areas of *grid computing* Skillicorn01 ; Foster01 ; Fox01  and *parasitic computing* Barabasi01 .

A parallel computing system consists of a set of processing elements connected by some kind of communication network. A parallel program runs on the system by partitioning the work to be done in several pieces that are executed on the available processing elements. To collaborate in the execution of the program, each piece must, as a rule, communicate with others through the communication network. To show a good performance in the execution of a program, a parallel system must then: (i) make a large number of processing elements available to the application; and (ii) enable fast communications between these processing elements.

For grid computing, the first condition can be met by the large amount of computers available in the Internet, as long as their owners agree to make their processing power available. But the second condition is difficult to meet: even if high bandwidth networks are common, the latency to deliver a message to a site geographically far from the origin is large. The increased processing power of microprocessors due to Moore’s law (i.e. the number of transistors in a chip doubles each 1.5 years) contrasts with the typically slow interconnection between processing elements, thus undermining the performance of parallel systems due to the relatively large time spent to send data for processing elsewhere in comparison with the time taken to process that data. A program will only have good performance on the grid if the amount of computations done is large in comparison with the time needed to send a message between two processors. Thus, the effective use of grid computing remains a challenge, demanding a delicate balance between computation and interprocess communication workloads. As a rule, the weaker the coupling (i.e. the amount of communication needed) between different processing tasks, the higher the overall efficiency in a given parallel system.

Generally, a more densely connected processing network favors faster data transmission, as the mean distance between nodes tends to decrease, but at the expense of additional communication resources. Moreover, the specific kind of processing to be performed and the availability of the computer resources for collaborative (or parasitic) computing also play an important role in defining the overall grid execution performance.

The novel area of complex networks (e.g. Albert02 ; Dorogovtsev02 ; Newman03 ) has drawn increasing interest of the physics community. A large part of the success of such an approach derives from the fact that such networks have been found to adhere, at varying degrees, with important real phenomena such as transmission of infectious diseases, social and ecological interactions, and the Internet. The fact that today most computers are interconnected through the Internet has contributed further to promote the systematic investigation of the Internet characteristics, a task that can greatly benefit from physical modeling approaches.

As several features are shared by grid computing systems and complex networks, much can be gained through integrative and comparative approaches, allowing cross-fertilization between those two important areas. The underlying idea in the current work is to study the efficiency of parallel/distributed architectures whose interconnections are defined in terms of complex network models. Such an investigation therefore focus on the integration between topology and function of the networks, an important aspect of complex network research Newman03 . Regular networks, which are often used in parallel computing, are also considered as a reference for comparison. It is particularly interesting to verify how the specific properties of these interconnecting schemes —such as the average vertex-vertex distance and cluster size— affect the processing time and efficiency for different network configurations. Such a possible dependence between the topological and functional properties of the networks is backed by recent works which verified that the emergent features of complex networks, such as associative memory recall in neuronal networks, can be strongly affected by the network interconnecting scheme and phase transitions Stauffer03 ; Costa03 . In related works, the complex network paradigm has also been explored from the perspective of search algorithms Kleinberg00 ; Adamic02 ; Guimera02  and information transfer in graphs Sole01 ; Tadic04 ; Tadic05 .

## 2 Network Models Used

The main purpose of this article is to study the influence of some network topological features in the efficiency of a grid system for the execution of a suitable parallel program. Therefore, we will not try to use network models that reproduce the characteristics of the Internet; we instead restrict ourselves to some simple models which are described below. The models are undirected, reflecting the bidirectional transfer of packets on the Internet.

Let a complex network be represented as a graph with $`n`$ nodes, identified as $`i`$, $`i=0,\mathrm{},n1`$, and unweighted, undirected edges represented as $`(i,j)`$. The first model considered is the Erdős-Rényi (ER) model with a fixed number $`c`$ of edges. In this model, for each connection, two nodes $`i`$ and $`j`$ are chosen uniformly among all the nodes to establish the connection $`(i,j)`$. Self-connections (connections of a node with itself) and duplicate connections (connections between already connected nodes) are avoided in this and all the following models.

ER graphs have a fast decaying degree distribution, with very small probability for nodes with high degree (also called *hubs*). To widen the degree distribution and increase the probability of high degree nodes, a *preferential attachment* (PA) model is used. Networks in this model are generated as described in Stauffer03 : starting with all nodes without connections and choosing two nodes to connect by drawing nodes from a list of node numbers represented in amount proportional to their respective number of connections (plus one, to account for the unconnected nodes). Note that this network model, although having preferential attachment, has no growth and so does not lead to scale-free networks (see Section 3.1).

Hubs of larger degrees can be found in scale-free networks. A simple model for scale-free networks was proposed by Barabási and Albert Barabasi99 . In their model, the network starts with $`m_0`$ connected nodes and grows by the addition of one node at a time. When a node is added, $`m`$ new connections from the new node to already existing nodes are made, and each already existing node can be chosen to receive a connection with probability proportional to its degree $`k`$. In this model, all nodes in the network form a single large connected component; as we are interested in studying the influence of percolation with growing connectivity, their model is not adequate due to the nonexistence of a percolation transition and to the impossibility of specifying an average connectivity that is not an even integer (the average degree is always $`2m`$). For this reason, their model is here generalized as described in the following. In the model used in this work, which we call *scale-free* (SF) model, instead of having a fixed number of connections for each new node $`i`$, a random number $`m_i`$ is chosen using a Poisson distribution with mean $`m`$, and $`m_i`$ connections from this node to the already existing nodes are made. As some nodes may have $`m_i=0`$, the network will have unconnected nodes; to enable these nodes to receive connections with the addition of new nodes, each already existing node is chosen with probability proportional do $`k+1`$ instead of $`k`$. In this model, as $`m`$ is only a mean value, it can be any real number (instead of only an integer number, as in the Barabási-Albert model); also, as new nodes are not necessarily connected to the already existing nodes, the network consists of many connected components.

For grid computing, as for all kinds of collaborative work between the agents represented by a network, the binding of the agents to other agents of the network is a necessity. There is, therefore, a tendency to bind nodes to the network as new communication resources are available, instead of using them to bind already connected nodes. This suggests a different kind of random network construction: connecting new nodes to the network should be given preference while there are still isolated nodes. For that purpose, we introduce here two new models of random networks. The parameters for their construction are: the number of nodes $`n`$ and the number of connections $`c`$.

In the first model, one end of each new edge is chosen with uniform distribution among the isolated nodes and the other end with uniform distribution among all nodes (isolated or not). If at some point no more isolated nodes exist, both ends of the remaining edges are drawn with uniform distribution among all nodes.

In the second model, one end of the new edge is drawn randomly from the set of isolated nodes, as for the previous model, but the other end is drawn among all nodes with probability proportional to the node degrees, as in the previously discussed preferential attachment network.

We call these networks *insertion networks*, because the connections are used to insert the nodes in the network; the first model is called *uniform-uniform* (UU) insertion network, and the second model *uniform-preferential* (UP) insertion network.

For comparison, we study also three regular network structures common in parallel systems Stojmenovic96 : the hypercube and the 2D and 3D tori. The construction of an hypercube with $`n`$ nodes can be explained as follows. For node $`i`$, represent the value of $`i`$ in binary using $`\mathrm{log}_2n`$ bits; there is a link $`(i,j)`$ iff the binary representation of $`j`$ differ in exactly one bit from that of $`i`$. For example, in an $`n=8`$ network, node 3 (binary 011) is connected to nodes 2 (010), 1 (001) and 7 (111). In the bi-dimensional torus the nodes are distributed in a grid of size $`(n_x,n_y)`$ (with $`n_xn_y=n`$), each node receiving a label $`(x,y)`$, where $`x=i/n_y`$ and $`y=imodn_y`$; node $`(x,y)`$ is connected with node $`(x^{},y^{})`$ iff $`x^{}=x`$ and $`(y^{}=y\pm 1)modn_y`$ or $`y^{}=y`$ and $`(x^{}=x\pm 1)modn_x`$. For example, in a $`n=8`$ bi-dimensional torus organized as a $`2\times 4`$ grid, node 3 corresponds to coordinates $`(0,3)`$ and is connected to nodes 2 $`(0,2)`$, 0 $`(0,0)`$, and 7 $`(1,3)`$. The three-dimensional torus is similarly constructed.

## 3 Results

The results are divided in two parts. First, some results concerning the properties of the network models described are presented. Then, results obtained by using these network models as communication infrastructure for the simulation of the execution of a parallel program on a grid are shown.

### 3.1 Network properties

A thorough analysis of the network models described is out of the scope of this paper. Here only some properties of interest to the analysis of the following grid simulation results (Section 3.2) are presented.

The topological properties of those networks are quantified in terms of the following measures: (a) node degree $`k`$; (b) mean vertex-vertex distance $`\mathrm{}=\frac{2}{n(n1)}_{i>j}d_{ij}`$ where $`d_{ij}`$ is the geodesic distance (distance, in number of links, of the shortest path) between nodes $`i`$ and $`j`$ and the summation includes only pairs $`ij`$ that have a path connecting them; and (c) mean cluster size $`s`$. A cluster, also know as a connected component, is a set of directly or indirectly connected nodes, i.e., nodes that can be reached from all the other nodes on the cluster by a path. To evaluate the mean cluster size, we compute for each node of the network the number of nodes on its cluster and take the average of these values for all nodes on the network. Note that this average includes the largest cluster, and is dominated by it after percolation. To enable the comparison between the different complex networks models with respect to their connectivity, we use the parameter $`z`$ defined as $`z=2m`$ for the SF model and $`z=2c/n`$ for the other models. In the limit of large $`n`$ we have $`z=k`$ for all network models.

In the following, network characteristics are quantified as averages for 50 random networks for each set of parameters. Error bars display the 99% confidency interval for the computed average, considering normal distribution of the averaged values.

The degree distributions for each of the considered models for $`n=100000`$ and $`z=3`$, compared with the ER model, are presented in Figure 1, where $`P(k)`$ is the cumulative probability distribution (probability of finding a node in the network with degree larger than $`k`$). The UU model is almost indistinguishable from the ER model in terms of degree distribution. Due to the preferential attachment rule, the PA and UP models have broader degree distributions. For the UP model this effect is not so marked due to the preference given to newly added nodes in new connections. As expected, the SF model follows a power law (with a finite-size cutoff), given rise to a large probability of nodes with high degree.

Fig. 2 gives the average network cluster size $`s`$ normalized by the network size $`n`$, and Fig. 3 shows the average node distance $`\mathrm{}`$ for the networks. These results are shown as functions of $`z`$ comparatively to the ER model. As Figs. 2(a–d) show, there is an abrupt transition (a percolation transition) from small to large cluster sizes as the connectivity grows. In the small cluster size region, the mean distance tends to grow with the connectivity (Fig. 3), as new links result in new connections between previously unconnected nodes; in the large cluster region, increased connectivity reduces mean distance, as most of the nodes are already connected in the largest cluster. A striking feature of the results for the PA model (and to some extent for the SF model) is the small cluster sizes even for high connectivity. The reason is that, as the number of nodes connected in the largest cluster grows, the probability of linking an unconnected node is very small, due to the preferential attachment rule used to choose the ends of new links, resulting in a relatively large number of isolated nodes or small clusters. In the SF model, this problem is minimized by the fact that one end of each new connection always go to a new (previously unconnected) node. The insertion (UU and UP) models show very similar behavior in terms of average distance and cluster size. The formation of a cluster spanning most of the nodes occurs for these models for higher connectivities than for the ER model. The explanation is that links that could be used to connect small clusters to form a larger one are being used to link new nodes. On the other hand, the size of the resulting largest cluster tends to be larger, as isolated nodes are less likely to appear. Distances in these models tend to be higher than in the ER model, because the same number of links is used to connect a larger number of nodes. For sufficiently higher connectivities, this last property is compensated in the UP model by the onset of high degree nodes (hubs), that shorten the mean distance between the nodes of the cluster. The presence of hubs is also the explanation for the smaller distances shown by the PA and SF models; this advantage vanishes as the connectivity grows.

More detail about the size of clusters is given in the left side of Figs. 4(a–e), which display the cumulative probability distribution of cluster sizes (i.e. the probability $`P(s)`$ that a randomly selected node is part of a cluster of size greater or equal to $`s`$) against $`s/n`$ for some values of $`z`$. These figures show that for small average connectivities the probability of finding a large cluster is negligible. On the other hand, for sufficiently large average connectivities almost all nodes are found in large clusters. The models without preferential attachment (ER, Fig. 4(a), and specially the insertion models, UU Figs. 4(c) and UP Fig. 4(d)) show a sharp transition from a regime with low probability of large clusters and high probability of small clusters to a regime with high probability of large clusters and small probability of small clusters. On the preferential attachment models (PA, Fig. 4(b), and SF, Fig. 4(e)), this transition is more gradual; they also display a larger probability of medium sized clusters before the percolation.

In the regular networks, all nodes are connected, and so $`s/n=1`$. Also, the value of $`z`$ is fixed for each network type, given $`n`$. The third column of Table 1 shows the values of the average distance for these network types.

### 3.2 Grid Simulations

The parallel computing systems were obtained by assigning a processing unity to each network node, while messages flow along the network edges. The distributed application considered follows the *master/slave* paradigm (also known as *manager/worker* or *bag of tasks*), where a master delivers processing tasks on demand to slave computers. The computational tasks are assumed to be completely independent, in the sense that each node can proceed without additional communications after receiving the work packet. This arrangement is similar to many grid computing efforts, like SETI@home Anderson02 . The computations are partitioned into $`M`$ work packets (tasks), each requiring the same amount $`L`$ of computing time. The communication cost is taken to correspond to the minimum number of edges between the master and the slave requesting the data. *The edge communication overhead is therefore equal for all edges and adopted as time unit.* Taking into account this communication cost model, the very small number of short cycles present in all the considered network models (compared with the Internet) does not represent an additional limitation.

Given a network, each node $`i`$ at a time is considered as master. The nodes that are part of the same cluster as the master start requesting tasks. The nodes that are not part of the same cluster as the master cannot contribute to the computation because of the lack of connection to the master. After receiving a task from the master, the slave computes the result, taking time $`L`$, and sends it, together with a request for another task, to the master. When all $`M`$ tasks have been delivered and their results received, the master terminates the execution and computes its total execution time $`T_i`$. Isolated nodes cannot take part on a distributed computation and so, when chosen as master nodes, their execution time is considered infinite (they will wait forever to receive a work request from a slave).

To quantify the suitability of the network models for grid computing we compute the average speedup achieved by the execution of the application on the networks. The speedup is defined as the ratio between sequential and parallel execution times. For the problem considered, the parallel execution time for master node $`i`$ is the value of $`T_i`$ discussed in the previous paragraph and the sequential execution time is $`ML`$, so that $`S_i=\frac{ML}{T_i}`$ is the speedup. The mean speedup of a network is the mean value $`\frac{1}{n}S_i`$. Note that, for isolated nodes, as discussed, $`T_i=\mathrm{}`$ and so $`S_i=0`$; as these nodes are anyway considered in the average, if a network has many isolated nodes its average speedup is low. Another equivalent measure, used in our results, is the normalized speedup $`E=S/n`$, also known as *parallel efficiency*. The averages $`S`$ and $`E`$ are then taken for 50 different random networks for each model and parameter set.

If master $`i`$ sends a task to slave $`j`$, the time to complete the task (get the result back) is the sum of the computation time of the task with the time taken to send the task to the slave node and receive it back, that is, $`L+2d_{ij}`$. At the same time, the number of nodes computing tasks is $`s_i1`$ ($`s_i`$ is the size of the cluster of master node $`i`$ and we subtract one because the master does not compute tasks). This indicates that, for the problem considered, two important network metrics are the average cluster size $`s`$ (Fig. 2) and average distance $`\mathrm{}`$ between nodes with a path connecting them (Fig. 3) .

The number of tasks $`M`$ was chosen so that each node has some tasks to process, but avoiding too large number of tasks, because the simulation time is proportional to $`M`$; the size of the tasks $`L`$ was chosen to achieve computation times larger than the average communication time (two times the average distance; see also Figs. 3(a–d)), but small enough such that the communication time has a detectable effect. If $`L`$ is too large, the average communication time, and then the network structure, is not important; if it is too small, the grid system is not a good choice for the execution of the application. Fig. 5(a) shows the dependence of average efficiency $`E`$ with the number of tasks $`M`$ (for fixed $`L=100,`$ $`n=1000,`$ and $`z=3`$) for the five models. If $`M`$ is small, there will be not enough computational work to distribute evenly among the nodes, resulting in very small efficiencies; after some value of $`M,`$ the efficiency is not much affected by a further increase of $`M.`$ The relation of the efficiency with $`L`$ is very similar: a larger value o $`L`$ helps reduce the importance of the communication costs, increasing the efficiency; Fig. 5(b) plots this relation (for $`M=5000,`$ $`n=1000,`$ and $`z=3`$). It can be seen that after $`L100`$ there is no significant increase in efficiency for larger $`L.`$ The following results assume $`M=5000`$ and $`L=100;`$ results for other values show no qualitative differences.

Figs. 6(a–d) present the average parallel efficiency as a function the $`z`$ for $`M=5000`$ and $`L=100`$. As can be easily seen by comparing Figs. 2(a–d) and 6(a–d), the average parallel efficiency tends to closely reflect the normalized average cluster size for all considered models.

This is further substantiated by a comparison of the left and right graphs of Figs. 4(a–e). The right graphs show the cumulative probability distribution $`P(E)`$ of a randomly selected node having efficiency greater or equal to $`E`$ when chosen as master node. It can be seen that $`P(E)`$ closely follows $`P(s).`$ The main differences are that the $`P(E)`$ curves are smoother and the largest achievable values of $`E`$ smaller than that of $`s/n.`$ The latter difference is a result of the fact that a cluster with $`s`$ nodes cannot achieve speedup $`S`$, because the master node does not compute and the communication costs increase the execution time of each task (in comparsion to sequential execution); the former difference is due to the fact that different clusters with the same number of nodes $`s`$ have different interconnection topology, resulting in different communication costs.

As a consequence, similar conclusions can be drawn for the efficiency of a grid computing network as for the mean cluster sizes of the corresponding network model. Particularly, we note a percolation transition of the efficiency with increasing connectivity from a regime of very small efficiency to a regime of high efficiency. This transition is more abrupt for the ER and particularly for the UU and UP models, and somewhat slower for the PA and SF models. The efficiency of the models with preferential attachment is restricted due to the already discussed higher number of isolated nodes, that contribute to a speedup of zero to the average speedup. The insertion models need a higher connectivity to reach the percolation point, but after that show a higher efficiency, demonstrating the advantage of using network resources to connect new nodes.

The strong correlation between the cluster size and parallel efficiency is further substantiated in Fig. 7, where a scatter plot of efficiency against normalized cluster size is shown, that demonstrates the almost linear correlation between efficiency and normalized cluster sizes. Another interesting result observed from this figure is that the PA and SF networks tend to provide slightly better efficiencies than the other models. In other words, if clusters of similar size are considered, the presence of hubs that “short-circuit” the distances tends to enhance the speed-up of the computations in the networks.

In order to further investigate such a possible effect of the intrinsic connectivity properties of the considered models over the respective performance, the largest clusters of the four models were considered in isolation in the grid simulations. That is, networks with the same connectivity were generated and only their largest clusters where considered. Special care was taken so as to obtain such connected clusters with equivalent number of nodes (about 1000 nodes). The results are shown in Figure 8(a–d), where the efficiency of the parallel execution on the largest clusters, $`E_{\mathrm{largest}}`$, are plotted. As we are considering here only the largest clusters, the efficiency is computed with respect to the number of nodes of the cluster, and not of the whole network. The results indicate a definite tendency of the SF model, and to a lesser extent of the PA model, to outperform the others. Such an effect is possibly a consequence of the shorter average lengths usually observed for this model —see Figure 3(a–d)— and the presence of *hubs* which act as message distribution nodes.

The results obtained for the regular topologies (hypercube and 2D and 3D torus) for $`n=1000`$ are shown in Table 1. Interestingly, the 2D and 3D torus regular architectures led to rather low efficiencies, despite their relatively large node degrees, reflecting the respective large average distances, due to the absence of shortcut links in the regular structure of these networks. The hypercube topology allowed efficiencies comparable to the maximum obtained for the network models, but at the expense of almost 10 connections per node, which implies a high network cost.

## 4 Conclusions

While the scale-free and preferential attachment models allowed better efficiency considering only the largest cluster, the Erdős-Rényi model tended to provide better average speedup when all clusters where condered, as a consequence of the smaller number of isolated clusters implied by this type of network. The insertion models resulted in even better efficiencies, due to the inclusion of more nodes in the largest cluster after percolation. The random models had better efficiencies than the regular ones, due to the implied smaller average distances.

The results show that a network is of little use for grid computing before the percolation point is reached, that is, for values of $`z`$ before the formation of a cluster spanning most of the nodes, because of the very small resulting efficiencies. In other words, the percolation of the network used as a grid computing resource is of fundamental importance to the utility of the grid. Although the Internet already connects a very large number of computers, the use of these computers for grid computing is subject mainly to two limitations: a consent from the part of their owner and the installation of a grid computing software on them. It is therefore appropriate to consider the grid network as distinct from the Internet, and two computers in the Internet as connected in the grid network only if their owners have given permission to use *and* installed an appropriate set of protocols and compatible software. In this aspect, the obtained results motivate the grid community to achieve convergence in protocols and software, as the presence of many incompatible software platforms represent the presence of unconnected clusters of nodes in the network. This conclusion is related with the efficiency of the network as a whole for grid computing; for users in isolated clusters, the execution of an application in these clusters can be nevertheless of interest, if enough speedup is achieved.

Future work should extend the analysis to other types of distributed systems and applications. Network models closer to some real Internet characteristics, as in Tadic01 , should be considered. The inclusion of measures to quantify the load of intermediate nodes in packet transmission Goh01 , like betweenness centrality, can improve the results, altough centrality is non-trivially related to traffic flow if congestion is considered Holme03 . If the assumption of communication through shortest paths (that requires global knowledge) is relaxed, the use of local search algorithms Adamic02  will result in stronger dependence of efficiency with the network structure. A further refinement in this direction is to consider queuing of packets on the nodes and congestion. Recent works Tadic04 ; Tadic05  have shown that transmission times are in this case not directly related to shortest distances, but have a much richer behavior, depending also on the total communication load carried by the network.

Another important generalization is to consider complex networks presenting links with different communication speeds. This can be modeled using weighted networks, in which the weight of the links reflect the bandwidth or inverse latency of the interconnecting links. Also the nodes can display different processing powers. These generalizations make the network models closer to real interconnection networks.

The generalization of the parallel application model to include communications between the tasks is of interest, expanding the classes of applications modeled and due to the importance of the network topology to the efficiency of these communications, resulting in an interesting interplay between network and application characteristics.

We conjecture that even with the generalization of the network, routing and application models, as suggested above, the efficiency will remain strongly related with cluster size, altough the correlation with shortest distance may be reduced, and other network features may increase their importance, resulting in stronger influence of the network model used.

###### Acknowledgements.

L. da F. Costa is grateful to FAPESP (processes 99/12765-2 and 96/05497-3) and CNPq for financial support. G. Travieso was financially supported by FAPESP grant 98/14681-8.


format: text

text:
# Delays, connection topology, and synchronization of coupled chaotic maps

(*Physical Review Letters* 92, 144101 (2004). DOI: 10.1103/PhysRevLett.92.144101)

## Abstract

We consider networks of coupled maps where the connections between units involve time delays. We show that, similar to the undelayed case, the synchronization of the network depends on the connection topology, characterized by the spectrum of the graph Laplacian. Consequently, scale-free and random networks are capable of synchronizing despite the delayed flow of information, whereas regular networks with nearest-neighbor connections and their small-world variants generally exhibit poor synchronization. On the other hand, connection delays can actually be conducive to synchronization, so that it is possible for the delayed system to synchronize where the undelayed system does not. Furthermore, the delays determine the synchronized dynamics, leading to the emergence of a wide range of new collective behavior which the individual units are incapable of producing in isolation. (http://link.aps.org/abstract/PRL/v92/e144101)

Recent years have witnessed a growing interest in the dynamics of interacting units. Particularly, a large number of studies have been devoted to synchronization in a variety of systems (see Pikovsky et al. (2001) and the references therein), including the coupled map lattices introduced by Kaneko Kaneko (1984). Usually, such systems have been investigated under the assumption of a certain regularity in the connection topology, where units are coupled to their nearest neighbors or to all other units. Lately, more general networks with random, small-world, scale-free, and hierarchical architectures have been emphasized as appropriate models of interaction Gade et al. (1995); Gade (1996); Watts and Strogatz (1998); Barabasi and Albert (1999); Cosenza and Tucci (2001). On the other hand, realistic modeling of many large networks with non-local interaction inevitably requires connection delays to be taken into account, since they naturally arise as a consequence of finite information transmission and processing speeds among the units. Some numerical studies have regarded synchronization under delays for special cases such as globally coupled logistic maps Jiang (2000) or carefully chosen delays Masoller et al. (2003). In this Letter we consider synchronization of coupled chaotic maps for general network architectures and connection delays. Because of the presence of the delays, the constituent units are unaware of the present state of the others; so it is not evident *a priori* that such a collection of chaotic units can operate in unison, i. e. synchronize. Based on analytical calculations, we show that this is indeed possible, and in fact may be facilitated by the presence of delays. Moreover, while the connection topology is important for synchronization, the delays have a crucial role in determining the resulting collective dynamics. As a result, the synchronized system can exhibit a plethora of new behavior in the presence of delays. We illustrate the results by numerical simulation of large networks of logistic maps.

We consider a finite connected graph $`\mathrm{\Gamma }`$ with nodes (vertices) $`i`$, writing $`ij`$ when $`i`$ and $`j`$ are neighbors, that is, connected by an edge, and with the number of neighbors of $`i`$ denoted by $`n_i`$. On $`\mathrm{\Gamma }`$, we have a dynamical system with discrete time $`t𝐙`$, with the state $`x_i`$ of $`i`$ evolving according to

$$x_i(t+1)=f(x_i(t))+\epsilon \left(\frac{1}{n_i}\underset{\begin{array}{c}j\\ ji\end{array}}{}f(x_j(t\tau ))f(x_i(t))\right).$$
(1)
Here $`f`$ is a differentiable function mapping some finite interval, say $`[0,1]`$, to itself, $`\epsilon [0,1]`$ is the coupling strength, and $`\tau 𝐙^+`$ is the transmission delay between vertices. A synchronized solution is one where the states of all vertices are identical,

$$x_i(t)=x(t)\text{for all }i.$$
(2)
Thus $`x(t)`$ satisfies

$$x(t+1)=(1\epsilon )f(x(t))+\epsilon f(x(t\tau )).$$
(3)
In order to investigate the stability of the synchronized solution (see Pikovsky et al. (2001) for a description of the stability concept employed here), we consider orthonormal eigenmodes $`u^k`$ of the graph Laplacian $`\mathrm{\Delta }_\mathrm{\Gamma }`$, defined by $`(\mathrm{\Delta }_\mathrm{\Gamma }v)_i:=(1/n_i)_{ji}(v_jv_i)`$, with corresponding eigenvalues $`\lambda _k`$. The spectrum of $`\mathrm{\Delta }_\mathrm{\Gamma }`$ reflects the underlying connection topology, with the zero eigenvalue $`\lambda _0`$ corresponding to the constant eigenfunction 111We follow here the conventions of Jost and Joy (2002) where also further references can be found. The eigenvalues are labelled in increasing order, the smallest one being the trivial eigenvalue $`\lambda _0=0`$.. Since the eigenfunctions yield an $`L^2`$-basis for functions on $`\mathrm{\Gamma }`$, it suffices to consider perturbations of $`x`$ of the form

$$x_i(t)=x(t)+\delta \alpha _k(t)u_i^k(t)$$
(4)
for some small $`\delta `$, and for $`k>0`$, that is non-constant ones. The solution $`x`$ is stable against such a perturbation when $`\alpha _k(t)0`$ for $`t\mathrm{}`$. Expanding about $`\delta =0`$ yields

$`\alpha _k(t+1)`$ $`=`$ $`(1\epsilon )f^{}(x(t))\alpha _k(t)`$ (5)
$`+`$ $`\epsilon (1\lambda _k)f^{}(x(t\tau ))\alpha _k(t\tau ).`$
The sufficient local stability condition is

$$\underset{T\mathrm{}}{lim}\frac{1}{T}\mathrm{log}\frac{|\alpha _k(T)|}{|\alpha _k(0)|}<0.$$
(6)
In the case without delay, that is, $`\tau =0`$, this is rewritten as

$$\underset{T\mathrm{}}{lim}\frac{1}{T}\mathrm{log}\underset{t=0}{\overset{T1}{}}\frac{|\alpha _k(t+1)|}{|\alpha _k(t)|}<0$$
(7)
and, by (5), becomes

$$\mathrm{log}|1\epsilon \lambda _k|+\underset{T\mathrm{}}{lim}\frac{1}{T}\underset{t=0}{\overset{T1}{}}\mathrm{log}|f^{}(x(t))|<0,$$
(8)
that is

$$|e^\mu (1\epsilon \lambda _k)|<1$$
(9)
where $`\mu =lim_T\mathrm{}\frac{1}{T}_{t=0}^{T1}\mathrm{log}|f^{}(x(t))|`$ is the Lyapunov exponent of $`f`$.

Essentially the same reasoning works for the case of non-zero delay $`\tau `$. Depending on whether $`|\alpha _k(t)|`$ is larger than $`|\alpha _k(t\tau )|`$ or not, in (7), we either keep the quotient $`|\alpha _k(t+1)/\alpha _k(t)|`$ or replace it by $`|\alpha _k(t+1)/\alpha _k(t\tau )|`$. This reduces the number of factors in (7) by $`\tau `$, but since we also get correspondingly fewer factors in the term leading to the first term in (8), the left hand side of (8) is changed by a positive multiplicative factor which does not affect the inequality. The terms get slightly more complicated, as instead of

$$(1ϵ\lambda _k)f^{}(x(t))\alpha _k(t)$$
(10)
yielding the two terms in (8), we now have

$$(1ϵ)f^{}(x(t))\alpha _k(t)+\epsilon (1\lambda _k)f^{}(x(t\tau ))\alpha _k(t\tau )$$
(11)
from (5). When comparing (10) and (11), we see that in (11), we may get additional partial cancellations due to different signs of $`f^{}(x(t))`$ and $`f^{}(x(t\tau ))`$ or $`\alpha _k(t)`$ and $`\alpha _k(t\tau )`$, so that the absolute value can be significantly smaller, thereby making the synchronization condition easier to achieve. On the other hand, in a chaotic regime the values of $`f^{}(x(t))`$ and $`f^{}(x(t\tau ))`$ should be essentially uncorrelated so that we still obtain the Lyapunov exponent $`\mu `$ as the relevant parameter coming from $`f`$. A detailed analysis will appear elsewhere, but the foregoing ideas suggest that synchronization in the delayed case is at least not more difficult, but potentially easier (due to cancellations in (11)) to achieve than in the non-delayed one. Moreover, the connection topology, as reflected by the eigenvalues $`\lambda _k`$, still plays an important role in synchronization.

As an application, we take $`f`$ to be the logistic map

$$f(x)=\rho x(1x)$$
(12)
which possesses a rich dynamical structure depending on the value of the parameter $`\rho `$ Elaydi (2000). Figure 1 shows the synchronization regions in the parameter space for several common network architectures at the value $`\rho =4`$ for which $`f`$ is fully chaotic. The gray-scale encoding represents the degree of synchronization of the network after an initial transient of 10,000 time steps, as measured by the fluctuations $`\sigma ^2(t)=_{i=1}^N(x_i(t)\overline{x}(t))^2`$, where $`N`$ is the size of the network and $`\overline{x}(t)=(1/N)_{i=1}^Nx_i(t)`$. Thus, $`\sigma ^2(t)0`$ as $`t\mathrm{}`$ if the system synchronizes. Darker colors in the figure correspond to smaller values of $`\sigma ^2`$, with black indicating that $`\sigma ^2(t)<10^{25}`$ after the transients. The networks used in the simulations have the same size, $`N=10,000`$, and the same number of average connections, even though the architectures may be different 222 The regular networks used in simulations have circular arrangement with each node coupled to its nearest $`k`$ neighbors, where $`k`$ is an even integer. Such networks are known to synchronize only for sufficiently small system sizes in the undelayed case Jost and Joy (2002). Small-world architecture is obtained by replacing randomly selected edges with long-range connections Watts and Strogatz (1998). Scale-free networks are constructed using the algorithm of Barabasi and Albert Barabasi and Albert (1999), starting with 10 isolated nodes and adding one node and 10 links at each step. Each network consists of 10,000 nodes and about $`10^5`$ links.. The effects of the network topology are clearly seen in Figure 1: Scale-free and random networks can synchronize for a large range of parameters whereas more regular networks with nearest-neighbor and small-world type coupling do not. In this respect, the similarities between scale-free and random networks (Figure 1a-b) are noteworthy. A closer inspection reveals some common features for synchronizing networks. For strong coupling (roughly for $`\epsilon >0.6`$) synchronization is achieved regardless of the actual value of the delay, as long as it is positive. For intermediate coupling in the range $`0.4<\epsilon <0.6`$, the value of the delay becomes decisive for synchronization. In this range one may also observe long transients, clustering, and on-off intermittency Pikovsky et al. (2001), in the gray regions where $`\epsilon `$ is slightly below the synchronization value. There are also smaller regions of synchronization that exist for weaker coupling ($`0.15<\epsilon <0.20`$) and only for odd delays. Note that for zero delay synchronization can occur only for a rather limited range ($`\epsilon >0.85`$). Also, the small region of synchronization in Figure 1c-d occurs for nonzero delay. Therefore, the presence of delays can indeed facilitate synchronization.

While the connection topology is important for the synchronizability of the coupled system, connection delays are significant in determining the resulting synchronized dynamics. In particular, there is an important difference with the undelayed case, for which (3) reduces to

$$x(t+1)=f(x(t));$$
(13)
i.e., when the synchronized system can exhibit nothing different from the exact dynamics of the individual isolated unit. Hence, an important implication of connection delays is the possibility of the emergence of new collective phenomena. Indeed, the solutions of (3) exhibit a much richer range of dynamics when $`\tau `$ is nonzero. Figure 2 shows the bifurcation diagram for several values of $`\tau `$ when $`f`$ is given by (12). Of course, for $`\tau =0`$ the familiar bifurcation diagram of the logistic map is obtained, displaying the period-doubling route to chaos. By contrast, when $`\tau >0`$ Neimark-Sacker type bifurcations Elaydi (2000) are prevalent, immediately resulting in high-period solutions followed by more complex behavior. Other values of odd and even delays result in pictures similar to the cases $`\tau =1`$ and $`\tau =2`$, respectively. Since Neimark-Sacker bifurcations cannot arise in one-dimensional maps, the difference with the undelayed case is fundamental. Similarly, a consideration of the Lyapunov exponents shows that the character of the chaotic attractor is also different. For instance, as seen in Figure 3, the synchronized system can have two positive Lyapunov exponents when connection delays are present, thus exhibiting hyperchaos.

Connection delays make the coupling strength $`\epsilon `$ an important factor in the dynamics of the synchronized solutions. To demonstrate, we take the logistic map (12) with $`\rho =4`$, for which it has a chaotic attractor. Thus, (3) is chaotic for all $`\epsilon `$ when $`\tau =0`$. By contrast, when $`\tau =1`$ the attracting solutions of (3) display a wider variety depending on the value of $`\epsilon `$, as depicted in Figure 4. Chaos is interrupted by windows of stable periodic solutions, for instance near $`\epsilon =0.68`$ and $`\epsilon =0.71`$. There is a large window of period-5 solutions for $`0.82<\epsilon <0.89`$, which also contain other stable solutions of higher periods for certain $`\epsilon `$ in this range. A periodic attractor of period 3 exists near $`\epsilon =0.945`$, which transforms to intermittent behavior at $`\epsilon =0.95`$, shown in Figure 5. All these dynamics are exhibited by the coupled system (1), since Figure 1 implies that for these parameter values the system is synchronized (provided it has an appropriate connection topology), so the collective behavior is described by (3). Clearly, delays can make the dynamics of the synchronized system quite sensitive to the coupling strength, a feature that is absent in undelayed networks.

Networks with connection delays arise naturally in many areas of science, including biology, population dynamics, neuroscience, economics, and so on. In neural systems, for instance, delays result from e. g. finite axonal transmission speeds Atay and Hutt (2003). Such networks lack an intrinsic notion of simultaneity since the present state of the system is inaccessible to the constituent units. These networks are nevertheless capable of operating in synchrony, even when complex units and significantly large delays are involved, as our findings indicate. In fact, networks with delays can actually synchronize more easily. This synchronizability property is especially relevant for neural networks, circumventing the difficulties in establishing a concept of collective or simultaneous information processing in the presence of delayed information transmission 333See also U. Ernst, K. Pawelzik, and T. Geisel, Phys. Rev. Lett. 74, 1570 (1995), and W. Gerstner, Phys. Rev. Lett. 76, 1755 (1996), in the context of coupled integrate-and-fire neurons.. Furthermore, the delays shape the dynamics of the synchronized system, leading to the emergence of a variety of new dynamics which the individual units are not capable of producing 444For further discussion along this line, see F. M. Atay and J. Jost, Max Planck Institute for Mathematics in the Sciences, Preprint Series Nr. 102 (2003), and arXiv:nlin.AO/0312026.. The observation that a wide range of different behavior can be accessed by varying the coupling strength has important implications for neural networks, where synaptic coupling strengths can change through learning. This interesting connection provides additional motivation for investigating the role of delays in complex adaptive systems.


text:
# 1 Fit for the heavy triplet Higgsino mass as a function of adjoint-Higgs corrections (ϵ) in order to accomplish gauge coupling unification. Here we define 𝑚₀ (universal scalar mass) and 𝑀_{1/2} (universal gaugino mass) at the GUT scale, and 𝜇 and 𝑚_𝐴 at the weak scale without imposing a radiative electro-weak symmetry breaking condition. One expects ϵ∼few%, and thus 𝑀_𝐻_𝑐>10¹⁷GeV can be naturally achieved as is required by proton decay constraints. The width of each band is primarily due to the current uncertainty in 𝛼_𝑠(𝑚_𝑍).

## Acknowledgements

This work was supported in part by the Department of Energy and the Alfred P. Sloan Foundation.


text:
# 1 Solitons with 𝐵={1,2} and 3. Horizontal axis: r (in fm).

Soliton solutions of Chiral Born-Infeld Theory and baryons.

O.V. Pavlovskiĭ

Institute for Theoretical Problems of Microphysics, Moscow State University

Moscow, 119992, Russian Federation.

Abstract.

Finite-energy topological spherically symmetrical solutions of Chiral Born-Infeld Theory are studied. Properties of these solution are obtained, and a possible physical interpretation is also given. We compute static properties of baryons (mass,main radius, magnetic main radius, axial coupling constant) whose solutions can be interpreted as the baryons of QCD.

The concept of the baryon as a chiral soliton has a long history. The idea of a unified theory for baryons and mesons that is formulated in terms of chiral field was only proposed for the first time by Skyrme . This theory has a set of topological static solutions and one can classify these solutions by the value of the topological (baryon) charge. In this model such solutions are associated with baryon states with different baryon charges, and soliton with $`B=1`$ (skyrmeon) is treated as a nucleon. Such kind of solutions and quantum fluctuations about them is well studied now and results of such analysis are in agreement with the experiment data with accuracy of 30-40 percents for baryonic masses and another static properties of baryons.

But there are many other ways for stabilization of chiral soliton, because from the methodological point of view, arising of the Skyrme part in the lagrangian of effective meson field theory is an ”ad hoc” procedure, and there are no real physical bases for this procedure. Of course, one can treat this part in the effective lagrangian as a leading high derivative expansion of effective chiral meson action, but this approach leads to numerous questions so far. Two most important ones concern the physical nature of the scale parameter in this theory and the influence of another terms of expansion on existence and stability of chiral solitons.

The consistent effective low-energy meson’s theory construction is a very complicated task that is closely connected to the quark’s confinement problem and to the problem of spontaneous breaking of chiral symmetry. It is desirable that such theory should be Lorentz and chiral invariant and should have a set of finite energy stable solutions. In the case of electro-magnetic fields such kind of an effective low-energy theory is the well-known Born-Infeld model . In our work  we study the direct analogue of the Born-Infeld action for chiral fields. By construction our model has no singular solutions and looks very attractive as a possible effective action for mesons fields. This model has a set of stable topological solitons. These solutions can be treated as baryons states in our model.

In the well-known paper , Born and Infeld proposed a non-linear covariant action for electro-magnetic fields with very attractive features. Firstly, in the framework of BI theory the problem of singular self-energy of electron can be solved. In this theory the electron is a stable finite energy solution of the BI field equation with electric charge. Second, the BI action has a scale parameter $`\beta `$. Using expansion by this parameter one reduces the BI action to the usual Maxwell form in the low-energy limit.

We want to perform a very similar procedure with the chiral prototype lagrangian

$$_{pr}=\frac{f_\pi ^2}{4}\mathrm{Tr}L_\mu L^\mu ,$$
(1)
where $`L_\mu =U^+_\mu U`$ is a Cartan left-invariant form and $`U=\mathrm{exp}(i\frac{\stackrel{}{\varphi }_\pi \stackrel{}{\tau }}{f_\pi })`$ where the vector field $`\stackrel{}{\varphi }_\pi `$ is associated with $`\pi `$ \- mesons and $`f_\pi =93`$ MeV is the pion decay constant. Like in the case of BI action, our chiral model must have a set of finite energy solutions with integer values of charge (topological or baryon), and in the low-energy limit such a theory must reproduce the prototype lagrangian (1). The model must be Lorentz and chiral invariant. Finally, the form of such theory follows from the analogy with the action of a relativistic particle, the BI action for EM and YM field.

Arguing as above, let us consider a theory with lagrangian

$$_{ChBI}=f_\pi ^2\mathrm{Tr}\beta ^2\left(1\sqrt{1\frac{1}{2\beta ^2}L_\mu L^\mu }\right)\frac{f_\pi ^2}{4}\mathrm{Tr}L_\mu L^\mu ,$$
(2)
where $`\beta `$ is a mass dimensional scale parameter of our model. It is easily shown that the expansion of the lagrangian (2) gives us the prototype theory as the leading order theory by parameter $`\beta `$.

Now we consider the spherically symmetrical field configuration $`U=\mathrm{exp}(iF(r)(\stackrel{}{n}\stackrel{}{\tau }))`$, $`\stackrel{}{n}=\stackrel{}{r}/|r|`$. Using the variation principle, we get the equation of motion

$$(r^2\frac{1}{\beta ^2}\mathrm{sin}^2F)F^{\prime \prime }+(2rF^{}\mathrm{sin}2F)$$

$$\frac{1}{\beta ^2}(rF^3F^2\mathrm{sin}2F+3\frac{1}{r}F^{}\mathrm{sin}^2F\frac{1}{r^2}\mathrm{sin}2F\mathrm{sin}^2F)=0.$$
(3)
The numerical investigation of the solutions of equation (3) is presented in Fig.1 (solitons with $`B=1,2`$ and $`3`$). The scale parameter $`\beta =807\text{MeV}`$ is defined from the hypothesis that the soliton with $`B=1`$ is a nucleon. Indeed, using now a scale transformation, one gets $`\beta =8\pi f_\pi ^2E(B=1,\beta =1)/m_p=807\text{MeV}`$ , where $`E(B=1,\beta =1)=3.487`$ is the energy of this soliton solution for $`B=1`$ and $`\beta =1`$.

Using this value of the $`\beta `$, one can find another physical properties of the hadron. The baryonic density (see Fig.2) and the main radius of the hadron can be calculate by

$$\rho _B=\frac{2}{pi}\mathrm{sin}^2FF^{},<r^2>=_0^{\mathrm{}}r^2\rho _B(r)𝑑r.$$

For Chiral Born-Infeld model we have $`<r^2>^{\text{ChBI}}0.51\text{fm}`$ (Skyrme model: $`<r^2>^{\text{Sk}}0.59\text{fm}`$); experiment data: $`<r^2>^{\text{exp}}0.72\text{fm}`$  ). The magnetic moment density (see Fig.2) and magnetic main radius of the hadron are

$$\rho _M=\frac{r^2\mathrm{sin}^2FF^{}}{_0^{\mathrm{}}r^2\mathrm{sin}^2FF^{}dr},<r^2>_M=_0^{\mathrm{}}r^2\rho _M(r)𝑑r.$$

For Chiral Born-Infeld model we have $`<r^2>_M^{\text{ChBI}}0.74\text{fm}`$ (Skyrme model: $`<r^2>^{\text{Sk}}0.92\text{fm}`$); experiment data: $`<r^2>^{\text{exp}}0.81\text{fm}`$ ).

Using asymptotic of our solutions at infinity, one can find a prediction for the value of axial coupling constant $`g_A^{\text{ChBI}}=0.79`$. Such prediction lies much close to the experiment data $`g_A^{\text{exp}}=1.23`$ then the prediction of Skyrme model $`g_A^{\text{Sk}}=0.61`$ .

We do not give a comprehensive investigation of the Chiral Born-Infeld theory. The questions about non-spherical solutions, quantum fluctuations about such solutions and corresponding properties of baryons or about the nucleon-nucleon interaction are not clear now. But maybe the most important question in such investigation is about physical substantiation of such theory. All of these questions should be the themes for a future investigation.


text:
# A Case of Almost Redundant Components in the Three-Alpha Faddeev Equations

## Abstract

The three-alpha orthogonality condition model using the Pauli-forbidden bound states of the Buck, Friedlich and Wheatly $`\alpha \alpha `$ potential can yield a compact three-alpha ground state with a large binding energy, in which a small admixture of the redundant components can never be eliminated.

preprint: APS/123-QED

As a typical example of quantum-mechanical three-body systems, the three-alpha-particle ($`3\alpha `$) model for $`{}_{}{}^{12}\text{C}`$ has been extensively studied from various viewpoints. SU68  It is well known that the microscopic structure of the $`\alpha `$-cluster plays an important role to create damped inner oscillations in the relative wave functions of the two $`\alpha `$-clusters, which can be described phenomenologically as the short-range repulsive core in the $`2\alpha `$ system. In the $`3\alpha `$ system, the amplitudes of these inner oscillations can be enhanced by the existence of the third $`\alpha `$-cluster, resulting in the formation of the compact shell-model like ground state of $`{}_{}{}^{12}\text{C}`$. HO75  On the other hand, the loosely bound nature of the $`2\alpha `$ pair is still preserved in the excited $`0^+`$ state at $`E_x=7.65`$ MeV, for which much interest is recently paid as a possible candidate of the $`\alpha `$ condensation state. TO01 ; SU02  This structure change of the $`3\alpha `$ system can most easily be simulated in the orthogonality condition model (OCM), first proposed by Saito SA68 .

In a previous publication ocm03 , we have discussed a new type of the Faddeev formalism for the $`3\alpha `$ system, in which pairwise $`\alpha `$-clusters interact via the Buck, Friedlich and Wheatly potential (BFW potential) BF77 . In this model, the Pauli-forbidden states between the two $`\alpha `$-clusters are composed of the lowest two $`S`$-wave bound states and one $`D`$-wave bound state of the BFW potential. We found that the $`3\alpha `$ ground-state energy is $`19.897`$ MeV for this potential, which is contradictory to the very small binding energy, $`E_{3\alpha }=0.26`$ MeV, from the variational calculations carried out by Tursunov, Baye, Descouvemont and Daniel in Refs. TB03 ; DE03 . The same situation also happens when we neglect the Coulomb force between $`\alpha `$-clusters. Namely, we have obtained $`E_{3\alpha }=27.748\text{MeV}`$, while theirs $`6.003\text{MeV}`$ tur03 . These authors comment that our result for the $`3\alpha `$ ground-state solution, using the BFW bound-state Pauli-forbidden states, does not completely eliminate the Pauli-forbidden components. Unlike their work our Faddeev solution contains a small admixture of the redundant components. Suppose $`\mathrm{\Psi }=\phi _\alpha +\phi _\beta +\phi _\gamma `$ be the total wave function of the $`3\alpha `$ system, composed of the three Faddeev components $`\phi _\alpha `$, $`\phi _\beta `$ and $`\phi _\gamma `$. If one sets $`f_uu|\mathrm{\Psi }`$ with $`u`$ being one of the Pauli-forbidden bound-state solutions, $`N_R=_uf_u|f_u`$ for all three Pauli-forbidden states is only $`(2.62.7)\times 10^4`$. This is a big contrast to the result for the harmonic oscillator (h.o.) Pauli-forbidden states $`|u`$, since in this case $`N_R10^{12}`$ ocm03 . The purpose of this brief report is to show that, if one wants to keep the shell-model like compact $`3\alpha `$ ground state, one cannot help but allowing a small admixture of the redundant components. In other words, it is impossible to eliminate this small admixture in the present framework without giving up the solution with the dominant shell-model like (04) component with the total h.o. quanta $`N=8`$. This implies that their solution corresponds to our second (excited-state) solution. The energy of this second solution is about $`6`$ MeV, and it has a small (04) component and $`N_R10^6`$.

A main problem arises from the second -symmetric component in the $`3\alpha `$ Faddeev equation, which now becomes no longer an exact redundant component, but “an almost redundant component” of the Faddeev equation. Here we use the Faddeev terminology and the notations used in our previous publications, Refs. RED  and ocm03 , but the same analysis is also possible in the various variational approaches. In Ref. RED , we first solve the eigenvalue equation of the rearrangement matrix

$$u|S|uf^\tau =\tau |f^\tau ,$$
(1)
where $`S=(123)+(123)^2`$ and $`|f^\tau `$ is normalized as $`f^\tau |f^\tau ^{}=\delta _{\tau ,\tau ^{}}`$. The solution $`|f^\tau `$ with $`\tau =1`$ gives a -symmetric redundant solution $`\phi _\tau =G_0|uf^\tau `$ of the Faddeev equation, where $`G_0`$ is the free 3-body Green function. The Faddeev component $`\phi _\tau `$ trivially satisfies

$$\lambda (E)\phi =G_0\stackrel{~}{T}S\phi \text{with}\lambda (E)=1,$$
(2)
due to the the orthogonality property, $`\stackrel{~}{T}G_0|u=|u`$, of the redundancy-free $`\stackrel{~}{T}`$-matrix and the commutability $`G_0S=SG_0`$. For this reason we add an extra term as in Eq. (9) below, and determine the bound-state energy $`E`$ with $`\lambda (E)=1`$. After $`E`$ is determined in this way, we again solve the Faddeev equation Eq. (2) without this second term. Then we get three $`\lambda (E)=1`$ solutions; one is a real solution and the others are the dual complex solutions having $`\mathrm{}e\{\lambda (E)\}1`$ with a small imaginary part of the order of $`10^2`$. The appearance of the complex eigenvalues having opposite signs in the imaginary part is not excluded since we are working with an eigenvalue problem of the non-symmetric kernel, $`G_0\stackrel{~}{T}S`$. These three solutions are characterized by the following three $`SU_3`$ components; $`[21]2(20)`$, $`[21]4(40)`$, and $`[3]8(04)`$, in the notation $`|[f]N(\lambda \mu )`$ or $`[f](\lambda \mu )`$ with $`N=\lambda +2\mu `$. However, this classification is for the Faddeev component $`\phi `$. If we make $`\mathrm{\Psi }(1+S)\phi `$, all of these three $`\mathrm{\Psi }`$’s become -symmetric total wave functions for the $`3\alpha `$ system, as long as they are non-zero. If $`|u`$ is the h.o. $`(0s)`$, $`(1s)`$ and $`(0d)`$ states, the first two (20) and (40) states exactly vanish by the $`(1+S)`$ operation, which means that these are trivial solutions of Eq. (2) with $`\lambda (E)=1`$. However, if we use the bound-state $`|u`$ of the BFW potential, the latter (40) state becomes almost redundant. (The same situation also happens for the first (20) state, but the residual component after the $`(1+S)`$ operation is very small and less than $`10^5`$.) In such a case, we can construct the normalized state

$$\varphi _\tau ^{[3]}=\frac{(1+S)|uf^\tau }{\sqrt{3(1+\tau )}}.$$
(3)
(Note that $`(1+S)^2=3(1+S)`$.) This becomes (04) dominant state. This can be confirmed by expanding $`|uf^\tau `$ in the h.o. basis and calculating the overlap of $`\varphi _\tau ^{[3]}`$ with the shell-model state, $`|[3]8(04)`$, in the $`3\alpha `$-cluster representation RED . Here we use a rather compact $`\alpha `$-cluster with the h.o. width parameter $`\nu =0.28125\text{fm}^2`$. In Table 1, the $`\varphi _\tau ^{[3]}`$ state generated from the second 4(40) dominant solution $`|uf^\tau `$ with $`\tau =0.999037`$ involves the 8(04) component with the amplitude 0.865401. This overlap is obtained from the third overlap in Table 1 through

$`[3]8(04)|\varphi _\tau ^{[3]}`$ $`=`$ $`{\displaystyle \frac{[3]8(04)|(1+S)|uf^\tau }{\sqrt{3(1+\tau )}}}`$ (4)
$`=`$ $`\sqrt{{\displaystyle \frac{3}{1+\tau }}}[3]8(04)|uf^\tau ,`$
since $`S^{}=S`$. The normalization factor $`1/\sqrt{3(1+\tau )}`$ is the reason of this large overlap. This immediately reminds us our old experience of the almost forbidden state SA73  in 2-cluster systems. In that case, the almost forbidden state is the cluster excited state, but in the present case it is the real (04) state, which is generated by the $`3\alpha `$ symmetrization from the almost redundant solution of Eq. (1), with the dominant -symmetric configuration $`|[21]4(40)`$.

It is interesting to note that the transition from $`N=4`$ to 8 takes place, since (04) is the only Pauli-allowed state with the lowest h.o. quanta $`N=8`$. The reason for this transition is naturally understood if we recall how we construct the Pauli-forbidden states in the pairwise OCM for the $`3\alpha `$ system. Let us assume for the time being that $`|u`$ is the h.o. Pauli-forbidden states. We first enumerate the translationally invariant -symmetric h.o. states by the Moshinsky rule MO96 . The elimination of the Pauli-forbidden state by the diagonalization procedure for the projection operator $`P=_\alpha |u_\alpha u_\alpha |`$ gives that the lowest Pauli-allowed state of the $`3\alpha `$ system is only (04) for $`N=8`$, and (62) and (24) for $`N=10`$, etc. TRGM  On the other hand, the construction of the $`3\alpha `$ Pauli-forbidden states in Eq. (1) is exactly equivalent to this elimination procedure of the Pauli-forbidden state, as long as the -symmetric basis states are concerned. Since the -symmetric Pauli-forbidden states are already enumerated by Eq. (3) for the solutions of Eq. (1) with $`\tau >1`$ in the h.o. limit, $`\varphi _\tau ^{[3]}`$ with $`\tau 1`$ should be an extra state which is orthogonal to all of these -symmetric Pauli-forbidden states. Therefore, $`\varphi _\tau ^{[3]}`$ in Eq. (3) generated from the small deviation from the pure h.o. limit should be the -symmetric allowed state with the smallest number of oscillations, namely, $`N=8`$ (04) state.

To be more specific, let us expand the Faddeev component $`\phi `$ by the following basis states:

1. -symmetric basis: $`\varphi _1^{[21]}=|uf^1`$ and the other orthonormalized basis $`\varphi _\alpha ^{[21]}`$.
2. -symmetric basis: $`\varphi _\tau ^{[3]}`$ with $`\tau >1`$, given in Eq. (3), and the other orthonormalized basis $`\varphi _\beta ^{[3]}`$. Here it is important to construct these as

$$\varphi _\tau ^{[3]}|\varphi _\beta ^{[3]}=0\text{for}\tau >1\text{and}\beta .$$
(5)

We expand $`\phi `$ in Eq. (2) with $`\lambda (E)=1`$ as

$`\phi `$ $`=`$ $`C_1^{[21]}|uf^1+{\displaystyle \underset{\alpha }{}}C_\alpha ^{[21]}\varphi _\alpha ^{[21]}`$ (6)
$`+{\displaystyle \underset{\tau >1}{}}C_\tau ^{[3]}\varphi _\tau ^{[3]}+{\displaystyle \underset{\beta }{}}C_\beta ^{[3]}\varphi _\beta ^{[3]},`$
and multiply the resultant equation by $`u|`$ from the left. Then, because of the basic relationship, $`u|G_0\stackrel{~}{T}=u|`$, the -symmetric part vanishes by $`u|(1+S)|[21]=0`$, and we obtain

$`0`$ $`=`$ $`u|\mathrm{\Psi }=u|(1+S)|\phi `$ (7)
$`=`$ $`{\displaystyle \underset{\tau >1}{}}C_\tau ^{[3]}\sqrt{3(1+\tau )}|f^\tau +{\displaystyle \underset{\beta }{}}3C_\beta ^{[3]}u|\varphi _\beta ^{[3]}.`$
Here we further take the matrix element with some particular $`f^\tau |`$ with $`\tau >1`$. Then the basis construction in Eq. (5) gives that the last term of Eq. (7) disappears and we are left with

$$C_\tau ^{[3]}\sqrt{3(1+\tau )}=0\text{for}\tau >1.$$
(8)
This implies that the exact solution of Eq. (2) with $`\lambda (E)=1`$ should not contain any of the $`\varphi _\tau ^{[3]}`$ components with $`\tau >1`$; namely, the $`3\alpha `$ Pauli-forbidden components. However, this is correct only within the accuracy of numerical calculations. For the solution with $`\tau `$ far apart from $`1`$, $`C_\tau ^{[3]}0`$ is certainly true. But, for the second solution of Table 1 with $`\tau =0.999037`$, $`C_\tau ^{[3]}`$ could be appreciably large, since $`C_\tau ^{[3]}\times 0.054=0`$. (Note that the imaginary part of the dual complex eigenvalues for the Faddeev equation is of the order of $`10^2`$.) In fact, we have a good reason to believe that our ground-state solution has a dominant $`\varphi _\tau ^{[3]}`$ component with $`\tau =0.999037`$, since both of them have a large (04) component. In a practical calculation, we can classify this $`|uf^\tau `$ solution to the complete redundant state with $`\tau =1`$ and solve a “modified” Faddeev equation

$`\lambda (E)\phi =G_0\left[\stackrel{~}{T}S{\displaystyle \underset{\tau 1}{}}|uf^\tau {\displaystyle \frac{1}{uf^\tau |G_0|uf^\tau }}uf^\tau |\right]\phi .`$
(9)
(Otherwise, we obtain unstable complex solutions and the energy with $`\lambda (E)=1`$ is not precisely determined.) The $`3\alpha `$ ground-state energy obtained by this prescription is $`E_{3\alpha }=27.625\text{MeV}`$, which is very close to the exact value $`27.748\text{MeV}`$ obtained by solving an improved equation Eq. (14) or Eq. (22). In this case, the relationship in Eq. (8) is modified to

$`uf^\tau |\mathrm{\Psi }`$ $`=`$ $`uf^\tau |1+S|\phi `$ (10)
$`=`$ $`C_\tau ^{[3]}\sqrt{3(1+\tau )}=uf^\tau |\phi .`$
For the normalized $`\phi `$ with $`3\phi |1+S|\phi =1`$, the last matrix element of Eq. (10) for the ground state is found to be $`0.1604\times 10^1`$. This leads to the value $`C_\tau ^{[3]}=0.2984`$, which yields the amplitude of the $`\varphi _\tau ^{[3]}`$ component contained in the total wave function $`\mathrm{\Psi }`$ as

$$\varphi _\tau ^{[3]}|\mathrm{\Psi }=\sqrt{\frac{3}{1+\tau }}uf^\tau |\mathrm{\Psi }=3C_\tau ^{[3]}=0.8952.$$
(11)
If we assume $`\mathrm{\Psi }3C_\tau ^{[3]}\varphi _\tau ^{[3]}`$, we can approximate the redundant amplitudes as

$$|f_u=u|\mathrm{\Psi }3C_\tau ^{[3]}u|\varphi _\tau ^{[3]}=C_\tau ^{[3]}\sqrt{3(1+\tau )}|f^\tau ,$$
(12)
and the redundant component admixed in the ground state is given by

$$f_u|f_uC_{\tau }^{[3]}{}_{}{}^{2}3(1+\tau )=uf^\tau |\phi ^2=0.26\times 10^3,$$
(13)
which agrees very well with the number $`(2.62.7)\times 10^4`$, obtained by solving Eq. (22).

From the definition of $`\varphi _\beta ^{[3]}`$ in Eq. (5), it is apparent that none of the $`\varphi _\beta ^{[3]}`$ has the large (04) component. Therefore, if one rejects the second $`\varphi _\tau ^{[3]}`$ in the -symmetric model space, one misses the dominant (04) component, and consequently one obtains a broad solution with a smaller binding energy. This is the situation which happens in Refs. TB03  and DE03 .

In order to formulate a precise $`3\alpha `$ OCM equation with the almost redundant Faddeev components, we write $`\varphi _\tau ^{[3]}`$ with $`\tau 1`$ as $`\mathrm{\Psi }_0`$, and define a new projection operator $`\stackrel{~}{P}=|\mathrm{\Psi }_0\mathrm{\Psi }_0|+P`$ with $`P=_{\lambda =0}|\mathrm{\Psi }_\lambda \mathrm{\Psi }_\lambda |`$. Here $`\mathrm{\Psi }_\lambda `$ with $`\lambda =0`$ are the -symmetric Pauli-allowed $`3\alpha `$ states and $`P|\mathrm{\Psi }_0=0`$ is satisfied. The $`3\alpha `$ OCM equation solved in the present formalism is

$`\stackrel{~}{P}[EH_0{\displaystyle \underset{\alpha }{}}V_\alpha ^{\mathrm{BFW}}]\stackrel{~}{P}\mathrm{\Psi }=0.`$ (14)
(On the other hand, the original equation with $`\stackrel{~}{P}P`$ is solved in Refs. TB03  and DE03  in the method of orthogonalizing pseudo-potentials.) This equation is equivalent with the following two equations:

$`\mathrm{\Psi }_0|EH_0{\displaystyle \underset{\alpha }{}}V_\alpha ^{\mathrm{BFW}}|\stackrel{~}{P}\mathrm{\Psi }=0,`$ (15a)
$`P[EH_0{\displaystyle \underset{\alpha }{}}V_\alpha ^{\mathrm{BFW}}]|\stackrel{~}{P}\mathrm{\Psi }=0.`$ (15b)
From Eq. (15a), we find

$`\mathrm{\Psi }_0|H|P\mathrm{\Psi }=(EE_0)\mathrm{\Psi }_0|\mathrm{\Psi },`$ (16)
where $`H=H_0+_\alpha V_\alpha ^{\mathrm{BFW}}`$ and $`E_0=\mathrm{\Psi }_0|H|\mathrm{\Psi }_0`$. By multiplying Eq. (15b) by $`\mathrm{\Psi }|`$ from the left and using Eq. (16), we immediately obtain

$`EP\mathrm{\Psi }|P\mathrm{\Psi }P\mathrm{\Psi }|H|P\mathrm{\Psi }={\displaystyle \frac{|\mathrm{\Psi }_0|H|P\mathrm{\Psi }|^2}{EE_0}}.`$ (17)
If $`|\mathrm{\Psi }_0`$ is an approximate eigenstate of the full Hamiltonian $`H`$, the coupling term in Eq. (16) is almost zero. In this case, we find two solutions for $`E`$ from a simple illustration of the graph for Eq. (17) with respect to $`E`$; namely, one is the $`\mathrm{\Psi }_0`$-dominant ground state with $`EE_0`$ and the other the excited state with $`EP\mathrm{\Psi }|H|P\mathrm{\Psi }/P\mathrm{\Psi }|P\mathrm{\Psi }`$ and a small admixture of the $`\mathrm{\Psi }_0`$ component.

It is also possible to derive a Faddeev equation which is completely equivalent to Eq. (14). We assume $`|u`$ the bound-state solution of $`V^{\mathrm{BFW}}`$ with the energy eigenvalue $`\epsilon _B`$. For $`𝒱_\alpha (E)=EH_0\mathrm{\Lambda }_\alpha (EH_0V_\alpha ^{\mathrm{BFW}})\mathrm{\Lambda }_\alpha `$ with $`\mathrm{\Lambda }_\alpha =1|u_\alpha u_\alpha |`$, one can prove

$`𝒱_\alpha (E)V_\alpha ^{\mathrm{BFW}}=|u_\alpha u_\alpha |(Eh_{\overline{\alpha }}\epsilon _B)|u_\alpha u_\alpha |,`$ (18)
where $`h_{\overline{\alpha }}`$ is the kinetic-energy operator of the third $`\alpha `$-particle. Owing to this relationship, we can replace $`V_\alpha ^{\mathrm{BFW}}`$ in Eq. (15b) by $`𝒱_\alpha (E)`$. Following the same procedure as developed in Ref. TRGM  for $`\stackrel{~}{P}\mathrm{\Psi }=\phi _\alpha +\phi _\beta +\phi _\gamma `$, we can derive

$`\phi =G_0\stackrel{~}{T}S\phi +{\displaystyle \underset{\tau 1}{}}|u\stackrel{~}{f}^\tau \mathrm{\Psi }_0|\stackrel{~}{P}\mathrm{\Psi },`$ (19)
where $`\stackrel{~}{f}^\tau =u|\mathrm{\Psi }_0=\sqrt{(1+\tau )/3}f^\tau `$. In the intermediate step, we also find

$`[EH_0{\displaystyle \underset{\alpha }{}}𝒱_\alpha (E)]\stackrel{~}{P}\mathrm{\Psi }`$
$`={\displaystyle \underset{\alpha }{}}|u_\alpha u_\alpha |EH_0|\phi _\beta +\phi _\gamma .`$ (20)
We multiply this equation by $`\mathrm{\Psi }_0|`$ from the left, and subtract the resultant equation from Eq. (15a). Then the symmetry of the matrix elements yields $`\mathrm{\Psi }_0|(𝒱_\alpha (E)V_\alpha ^{\mathrm{BFW}})|\stackrel{~}{P}\mathrm{\Psi }=u_\alpha \stackrel{~}{f}_\alpha ^\tau |EH_0|\phi _\beta +\phi _\gamma `$ for each $`\alpha `$ and $`\tau 1`$, or (from Eq. (18) and restoring $`V^{\mathrm{BFW}}`$)

$`\mathrm{\Psi }_0|\stackrel{~}{P}\mathrm{\Psi }`$ $`=`$ $`{\displaystyle \frac{u\stackrel{~}{f}^\tau |(EH_0)S|\phi }{u\stackrel{~}{f}^\tau |EH_0V^{\mathrm{BFW}}|u\stackrel{~}{f}^\tau }}.`$ (21)
Using this relationship in Eq. (19), we eventually obtain a new type of Faddeev equation

$`\phi `$ $`=`$ $`G_0\stackrel{~}{T}S\phi +{\displaystyle \underset{\tau 1}{}}|uf^\tau {\displaystyle \frac{1}{uf^\tau |EH_0V^{\mathrm{BFW}}|uf^\tau }}`$ (22)
$`\times uf^\tau |(EH_0)S|\phi .`$
The solutions of this equation are given in Ref. ocm03 , together with the results of the direct variational calculations of Eq. (14), using the translationally invariant h.o. basis.

In summary, we probably cannot obtain the (04)-dominant compact ground state without a small admixture of the redundant components, which is related with the model space character that no exact -symmetric solution exists in the model space $`|uf^\tau `$, when the BFW bound-state solution is used for $`|u`$. If one insists mathematical rigorousness that the forbidden components should be completely eliminated from the exact solution, we have to say that there is no compact bound state possible in the $`3\alpha `$ problem for the BFW $`\alpha \alpha `$ potential. We, however, keep in mind that the orthogonality condition model is just a model which takes into account the major roles of the Pauli principle among clusters. From the microscopic viewpoint based on the resonating-group method, a small admixture of the redundant components is easily swept away by the effect of antisymmetrization. It is our opinion that the description of the physical ground state of the $`3\alpha `$ system with the compact shell-model like structure is far more important than the strict demand to eliminate the redundant components.

###### Acknowledgements.

We wish to thank D. Baye, P. Descouvemont and E. M. Tursunov for many useful discussions. This work was supported by Grants-in-Aid for Scientific Research (C) from Japan Society for the Promotion of Science (JSPS) (Nos. 15540270, 15540284). One of the authors (Y. F.) also thanks FNRS foundation of Belgium for making his visit to Free University of Brussels possible during the summer, 2002.


text:
# Entropic PriorsPresented at MaxEnt 2003, the 23th International Workshop on Bayesian Inference and Maximum Entropy Methods (August 3-8, Jackson Hole, WY, USA).

## 1 Introduction

Among the methods used to update from a prior probability distribution to a posterior distribution when new information becomes available there are two that can claim the distinction of being systematic, objective, and of wide applicability: one is based on Bayes’ theorem (for applications to physics see ) and the other is based on the maximization of (relative) entropy . The choice between the two methods is dictated by the nature of the information being processed.

Bayes’ theorem should be used when we want to update our beliefs about the values of quantities $`\theta `$ on the basis of observed values of data $`y`$ and of the known relation between them – the likelihood $`p(y|\theta )`$. The posterior distribution is $`p(\theta |y)\pi (\theta )p(y|\theta )`$. The previous knowledge about $`\theta `$ is codified both in the prior distribution $`\pi (\theta )`$ and also in the likelihood $`p(y|\theta )`$.

The selection of the prior is a difficult problem  because it is not always clear how to translate our previous beliefs about $`\theta `$ into a distribution $`\pi (\theta )`$ in an objective way. One approach that seems to work, at least sometimes, is to rely on experience and physical intuition but this becomes unreliable in situations of increasing complexity. Attempts to achieve objectivity include arguments invoking symmetry – generalized forms of the principle of insufficient reason – and arguments that seek to identify that state of knowledge that reflects complete ignorance. The latter suggest connections with the notion of entropy  and have led to proposals for “entropic priors” . This brings us to the second method of processing information, the method of maximum entropy, which is designed for processing information given in the form of constraints on the family of posterior distributions .

In this paper we use entropic arguments to translate information into a prior distribution . Rather than seeking a totally non-informative prior, we translate information that we do in fact have: the knowledge of the likelihood function, $`p(y|\theta )`$, already constitutes valuable prior information. The prior thus obtained is an “entropic prior.” The *bare* entropic priors discussed here apply to a situation where all we know about the quantities $`\theta `$ is that they appear as parameters in the likelihood $`p(y|\theta )`$. It is straightforward, however, to extend the method and incorporate additional relevant information beyond that contained in the likelihood.

The first proposal of priors of this form is due to Skilling  for the case of discrete distributions. The second proposal, due to Rodríguez , provided the generalization to the continuous case and further elaborations . In section 2 we give a derivation that is closer in spirit to applications of ME to statistical mechanics. A difficulty with the case of experiments that can be indefinitely repeated, which had been identified in , is diagnosed and resolved with the introduction of a hyper-parameter $`\alpha `$ in section 3. The analogy to statistical mechanics is important: the interpretation of $`\alpha `$ as a Lagrange multiplier affects how $`\alpha `$ should be estimated and is an important difference between the entropic prior proposed here and those of Skilling and Rodríguez. The example of a Gaussian likelihood is given in section 4. In section 5 we collect our conclusions and some final comments.

## 2 The basic idea

We use the ME method  to derive a prior $`\pi (\theta )`$ for use in Bayes’ theorem $`p(\theta |y)p(y,\theta )=\pi (\theta )p(y|\theta )`$. As discussed in , since Bayes’ theorem follows from the product rule we must focus our attention on $`p(y,\theta )`$ rather than $`\pi (\theta )`$. Thus, the relevant universe of discourse is the product $`\mathrm{\Theta }\times Y`$ of $`\mathrm{\Theta }`$, the space of all $`\theta `$s, and the data space $`Y`$. This important point was first made by Rodríguez  but both our derivation and final results differ from his .

To rank distributions on the space $`\mathrm{\Theta }\times Y`$ we must first decide on a prior $`m(y,\theta )`$. When nothing is known about the variables $`\theta `$ – in particular, no relation between $`y`$ and $`\theta `$ is yet known – the prior must be a product $`m(y)\mu (\theta )`$ of the separate priors in the spaces $`Y`$ and $`\mathrm{\Theta }`$ because maximizing the relative entropy

$$\sigma [p]=𝑑y𝑑\theta p(y,\theta )\mathrm{log}\frac{p(y,\theta )}{m(y)\mu (\theta )},$$
(1)
yields $`p(y,\theta )m(y)\mu (\theta )`$.This distribution reflects our state of ignorance: the data about $`y`$ tells us absolutely nothing about $`\theta `$.

In what follows we assume that $`m(y)`$ is known because it is an important part of understanding what data it is that has been collected. Furthermore, if the $`\theta `$s are parameters labeling some distributions $`p(y|\theta )`$, then for each particular choice of the functional form of $`p(y|\theta )`$ there is a natural distance in the space $`\mathrm{\Theta }`$ given by the Fisher-Rao metric $`d\mathrm{}^2=g_{ij}d\theta ^id\theta ^j`$,

$$g_{ij}=𝑑yp(y|\theta )\frac{\mathrm{log}p(y|\theta )}{\theta ^i}\frac{\mathrm{log}p(y|\theta )}{\theta ^j}.$$
(2)
Therefore the prior on $`\theta `$ is $`\mu (\theta )=g^{1/2}(\theta )`$ where $`g(\theta )`$ is the determinant of $`g_{ij}`$.

Next we incorporate the crucial piece of information: of all joint distributions $`p(y,\theta )=\pi (\theta )p(y|\theta )`$ we consider the subset where the likelihood $`p(y|\theta )`$ has a fixed, known functional form. Notice that this is an unusual constraint; it is not an expectation value. Note also that the only information we are using about the quantities $`\theta `$ is that they appear as parameters in the known likelihood $`p(y|\theta )`$, *nothing else*. But, of course, should additional relevant information (*i.e.*, an additional constraint) be known it should also be taken into account.

The preferred distribution $`p(y,\theta )`$ is chosen by varying $`\pi (\theta )`$ to maximize

$$\sigma [\pi ]=𝑑y𝑑\theta \pi (\theta )p(y|\theta )\mathrm{log}\frac{\pi (\theta )p(y|\theta )}{g^{1/2}(\theta )m(y)}.$$
(3)
Assuming that both $`\pi (\theta )`$ and $`p(y|\theta )`$ are normalized the result is

$$\pi (\theta )d\theta =\frac{1}{\zeta }e^{S(\theta )}g^{1/2}(\theta )d\theta \text{where}\zeta =𝑑\theta g^{1/2}(\theta )e^{S(\theta )},$$
(4)
and $`S(\theta )`$ is the entropy of the likelihood,

$$S(\theta )=𝑑yp(y|\theta )\mathrm{log}\frac{p(y|\theta )}{m(y)}.$$
(5)
The entropic prior eq.(4) is our first important result: it gives the probability that the value of $`\theta `$ should lie within the small volume $`g^{1/2}(\theta )d\theta `$. The preferred value of $`\theta `$ is that which maximizes the entropy $`S(\theta )`$ because this maximizes the scalar probability density $`\mathrm{exp}S(\theta )`$. Note that eq.(4) manifestly invariant under changes of the coordinates $`\theta `$.

To summarize: for the special case of a fixed data space $`Y`$, that is, for experiments that cannot be repeated, we have succeeded in translating the information contained in the model – the space $`Y`$, its measure $`m(y)`$, and the conditional distribution $`p(y|\theta )`$ – into a prior $`\pi (\theta )`$.

But for experiments that can be repeated indefinitely the prior (4) yields nonsense and we have a problem. Indeed, let us assume that $`\theta `$ is not a “random” variable, its value is fixed but unknown. For $`N`$ independent repetitions of an experiment, the joint distribution in the space $`\mathrm{\Theta }\times Y^N`$ is

$$p(y^{(N)},\theta )=\pi ^{(N)}(\theta )p(y^{(N)}|\theta )=\pi ^{(N)}(\theta )p(y_1|\theta )\mathrm{}p(y_N|\theta ),$$
(6)
and maximization of the appropriate $`\sigma ^{(N)}`$ entropy gives

$$\pi ^{(N)}(\theta )=\frac{1}{Z^{(N)}}g^{1/2}(\theta )e^{NS(\theta )},$$
(7)
which is clearly wrong. The dependence of $`\pi ^{(N)}`$ on the amount $`N`$ of data would lead us to a perpetual revision of the prior as more data is collected. For large $`N`$ the data becomes irrelevant.

The problem, as we will see next, is not a failure of the ME method but a failure to include all the relevant information. Indeed, when an experiment can be repeated we actually know more than just $`p(y^{(N)}|\theta )=p(y_1|\theta )\mathrm{}p(y_N|\theta )`$. We also know that discarding the values of say $`y_2,\mathrm{}y_N`$, yields an experiment that is indistinguishable from the single, $`N=1`$, experiment. This *additional* information, which is expressed by $`𝑑y_2\mathrm{}𝑑y_Np(y^{(N)},\theta )=p(y_1,\theta )`$ leads to $`\pi ^{(N)}(\theta )=\pi ^{(1)}(\theta )`$ for all $`N`$. Next we identify a constraint that codifies this information within each space $`\mathrm{\Theta }\times Y^N`$.

## 3 More information: the Lagrange multiplier $`\alpha `$

For large $`N`$ the prior $`\pi ^{(N)}(\theta )`$ in eq.(7) reflects an overwhelming preference for the value of $`\theta `$ that maximizes the entropy $`S(\theta )`$. Indeed, as $`N\mathrm{}`$ we have

$$S=𝑑\theta \pi ^{(N)}(\theta )S(\theta )\stackrel{N\mathrm{}}{}S(\theta _{\mathrm{max}}),$$
(8)
which is manifestly incorrect. This suggests that information about the actual numerical value $`\overline{S}`$ of the expected entropy $`S`$ is very relevant (because if $`\overline{S}`$ were known the problem above would not arise) and that we should maximize $`\sigma ^{(N)}`$ subject to an additional constraint on $`\overline{S}`$. Naturally, additional steps will be needed to estimate the unknown $`\overline{S}`$. A similar argument justifying the introduction of constraints in statistical physics is explored in .

We maximize the entropy

$$\sigma ^{(N)}[\pi ]=𝑑\theta 𝑑y^{(N)}\pi (\theta )p(y^{(N)}|\theta )\mathrm{log}\frac{\pi (\theta )p(y^{(N)}|\theta )}{g^{1/2}(\theta )m(y^{(N)})}$$
(9)
subject to constraints on $`S`$ and that $`\pi `$ be normalized. (An unimportant factor of $`N^{d/2}`$ has been dropped from the Fisher-Rao measure $`g^{(N)}(\theta )`$.) The result is

$$\pi (\theta )=\frac{1}{\zeta }g^{1/2}(\theta )\mathrm{exp}\left[(N+\lambda _N)S(\theta )\right].$$
(10)
The undesired dependence on $`N`$ is eliminated if the Lagrange multipliers $`\lambda _N`$ in each space $`\mathrm{\Theta }\times Y^N`$ are chosen so that $`N+\lambda _N=\alpha `$ is a constant independent of $`N`$. The resulting entropic prior,

$$\pi (\theta |\alpha )=\frac{1}{\zeta (\alpha )}g^{1/2}(\theta )e^{\alpha S(\theta )}$$
(11)
is our second important result. The prior $`\pi (\theta |\alpha )`$ incorporates information contained in the likelihood plus information about

$$S=\overline{S}(\alpha )=\frac{d}{d\alpha }\mathrm{log}\zeta (\alpha )\text{where}\zeta (\alpha )=𝑑\theta g^{1/2}(\theta )e^{\alpha S(\theta )}.$$
(12)
The last step would be to estimate $`\alpha `$ and $`\theta `$ from Bayes’ theorem

$$p(\alpha ,\theta |y^{(N)})=\pi (\alpha )\pi (\theta |\alpha )\frac{p(y^{(N)}|\theta )}{p(y^{(N)})},$$
(13)
where $`\pi (\alpha )`$ is a prior for $`\alpha `$. However, if we are only interested in $`\theta `$, we can just marginalize over $`\alpha `$ to get

$$p(\theta |y^{(N)})=𝑑\alpha p(\alpha ,\theta |y^{(N)})=\overline{\pi }(\theta )\frac{p(y^{(N)}|\theta )}{p(y^{(N)})}$$
(14)
where

$$\overline{\pi }(\theta )=𝑑\alpha \pi (\alpha )\pi (\theta |\alpha ).$$
(15)
The averaged $`\overline{\pi }(\theta )`$ is our final expression for the entropic prior. It is independent of the actual data $`y^{(N)}`$ as it should.

Next we assign an entropic prior to $`\alpha `$. We start by pointing out that $`\alpha `$ is not on the same footing and should not be treated like the other parameters $`\theta `$ because the relation between $`\alpha `$ and the data $`y`$ is indirect: $`\alpha `$ is related to $`\theta `$ through $`\pi (\theta |\alpha )`$, and $`\theta `$ is related to $`y`$ through $`p(y|\theta ).`$ Once $`\theta `$ is given, the data $`y`$ contains no further information about $`\alpha `$. Since the whole significance of $`\alpha `$ is derived purely from $`\pi (\theta |\alpha )`$, eq.(11), the relevant universe of discourse is $`A\times \mathrm{\Theta }`$ with $`\alpha A`$ and not $`A\times \mathrm{\Theta }\times Y^N`$ as in  which requires the introduction of an endless chain of hyper-parameters.

We therefore consider the joint distribution $`\pi (\alpha ,\theta )=\pi (\alpha )\pi (\theta |\alpha )`$ and obtain $`\pi (\alpha )`$ by maximizing the entropy

$$\mathrm{\Sigma }[\pi ]=𝑑\alpha 𝑑\theta \pi (\alpha ,\theta )\mathrm{log}\frac{\pi (\alpha ,\theta )}{\gamma ^{1/2}(\alpha )g^{1/2}(\theta )}$$
(16)
where $`\gamma ^{1/2}(\alpha )`$ is determined below. Since no reference is made to repeatable experiments in $`Y^N`$ there is no need for any further constraints – and no further hyper-parameters – except for normalization. The result is

$$\pi (\alpha )=\frac{1}{z}\gamma ^{1/2}(\alpha )e^{s(\alpha )},$$
(17)
where using eqs.(11) and (12) the Fisher-Rao measure $`\gamma (\alpha )`$ is

$$\gamma (\alpha )=𝑑\theta \pi (\theta |\alpha )\left[\frac{d}{d\alpha }\mathrm{log}\pi (\theta |\alpha )\right]^2=\frac{d^2\mathrm{log}\zeta (\alpha )}{d\alpha ^2},$$
(18)
and where $`s(\alpha )`$ is given by

$$s(\alpha )=𝑑\theta \pi (\theta |\alpha )\mathrm{log}\frac{\pi (\theta |\alpha )}{g^{1/2}(\theta )}=\mathrm{log}\zeta (\alpha )\alpha \frac{d\mathrm{log}\zeta (\alpha )}{d\alpha }.$$
(19)
This completes our derivation of the actual prior for $`\theta `$: the averaged $`\overline{\pi }(\theta )`$ in eq.(15) codifies information contained in the likelihood function, plus the insight that for repeatable experiments, information about the expected likelihood entropy, even if unavailable, is relevant.

## 4 Example: a Gaussian model

Consider data $`y^{(N)}=\{y_1,\mathrm{},y_N\}`$ that are scattered around an unknown value $`\mu `$,

$$y=\mu +\nu $$
(20)
with $`\nu =0`$ and $`\nu ^2=\sigma ^2.`$ The goal is to estimate $`\theta =(\mu ,\sigma )`$ on the basis of $`y^{(N)}`$ and the information implicit in the data space $`Y`$, its measure $`m(y)`$ (discussed below), and the Gaussian likelihood,

$$p(y|\mu ,\sigma )=\frac{1}{\left(2\pi \sigma ^2\right)^{1/2}}\mathrm{exp}\left[\frac{(y\mu )^2}{2\sigma ^2}\right].$$
(21)
We asserted earlier that knowing the measure $`m(y)`$ is part of knowing what data has been collected. In many physical situations where the data happen to be distributed according to eq.(21) the underlying space $`Y`$ is invariant under translations and we can assume $`m(y)=m=\mathrm{constant}`$. Indeed, the Gaussian distribution can be obtained by maximizing an entropy with an underlying constant measure and constraints on the relevant information the mean $`\mu `$ and the variance $`\sigma ^2`$.

From eqs.(5) and (21) the entropy of the likelihood is

$$S(\mu ,\sigma )=\mathrm{log}\left[\frac{\sigma }{\sigma _0}\right]\text{where}\sigma _0\stackrel{def}{=}\left(\frac{e}{2\pi }\right)^{1/2}\frac{1}{m},$$
(22)
and the corresponding Fisher-Rao measure, from eq.(2) is

$$g(\mu ,\sigma )=det\left|\begin{array}{cc}1/\sigma ^2& 0\\ 0& 2/\sigma ^2\end{array}\right|=\frac{2}{\sigma ^4}.$$
(23)
Note that both $`S(\mu ,\sigma )`$ and $`g(\mu ,\sigma )`$ are independent of $`\mu `$. This means that if we were concerned with the simpler problem of estimating $`\mu `$ in a situation where $`\sigma `$ happens to be known, the Bayesian estimate of $`\mu `$ using entropic priors coincides with the maximum likelihood estimate.

When $`\sigma `$ is unknown the $`\alpha `$-dependent entropic prior, eq.(11), is

$$\pi (\mu ,\sigma |\alpha )=\frac{2^{1/2}}{\zeta (\alpha )}\frac{\sigma ^{\alpha 2}}{\sigma _0^\alpha }.$$
(24)
Since $`\pi (\mu ,\sigma |\alpha )`$ is improper in both $`\mu `$ and $`\sigma `$ we must introduce high and low cutoffs for both $`\mu `$ and $`\sigma `$. The fact that without cutoffs the model is not well defined is interpreted as a request for additional relevant information, namely, the values of the cutoffs.

We write the range of $`\mu `$ as $`\mathrm{\Delta }\mu =\mu _H\mu _L`$ and introduce dimensionless quantities $`\epsilon _L`$ and $`\epsilon _H`$; $`\sigma `$ extends from $`\sigma _L=\sigma _0\epsilon _L`$ to $`\sigma _H=\sigma _0/\epsilon _H`$. Then $`\zeta (\alpha )`$ and $`\pi (\mu ,\sigma |\alpha )`$ are given by

$$\zeta (\alpha )=\frac{2^{1/2}\mathrm{\Delta }\mu }{\sigma _0}\frac{\epsilon _H^{1\alpha }\epsilon _L^{\alpha 1}}{\alpha 1}.$$
(25)
and

$$\pi (\mu ,\sigma |\alpha )=\frac{1}{\mathrm{\Delta }\mu \sigma _0}\frac{\alpha 1}{\epsilon _H^{1\alpha }\epsilon _L^{\alpha 1}}\left(\frac{\sigma }{\sigma _0}\right)^{\alpha 2}.$$
(26)
Note that $`\pi (\mu ,\sigma |\alpha =1)`$ reduces to $`d\sigma /\sigma `$ which is the Jeffreys prior usually introduced by imposing invariance under scale transformations, $`\sigma \lambda \sigma `$.

Writing $`\epsilon \stackrel{def}{=}(\epsilon _L\epsilon _H)^{1/2}`$, the prior for $`\alpha `$, is obtained from eq.(18),

$$\gamma (\alpha )=\frac{1}{(\alpha 1)^2}\left(\frac{2\mathrm{log}\epsilon }{\epsilon ^{1\alpha }\epsilon ^{\alpha 1}}\right)^2$$
(27)
and from eqs.(15) and (19),

$$\pi (\alpha )=\frac{\gamma ^{1/2}(\alpha )}{z}\frac{\epsilon ^{1\alpha }\epsilon ^{\alpha 1}}{\alpha 1}\mathrm{exp}\left[\frac{1}{\alpha 1}+\alpha \frac{\epsilon ^{1\alpha }+\epsilon ^{\alpha 1}}{\epsilon ^{1\alpha }\epsilon ^{\alpha 1}}\mathrm{log}\epsilon \right],$$
(28)
where the normalization $`z`$ has been suitably redefined.

Eqs.(27) and (28) simplify in the limit $`\epsilon 0`$. Note that the same result is obtained irrespective of the order in which we let $`\epsilon _H0`$ and/or $`\epsilon _L0`$. The resulting $`\gamma (\alpha )`$ and $`\pi (\alpha )`$ are

$$\gamma (\alpha )=\frac{1}{(\alpha 1)^2},$$
(29)
and

$$\pi (\alpha )=\{\begin{array}{cc}\frac{1}{\left(1\alpha \right)^2}\mathrm{exp}\left[\frac{1}{\alpha 1}\right]& \text{for }\alpha <1\\ 0& \text{for }\alpha 1\end{array}$$
(30)
where $`\pi (\alpha )`$ is normalized and is shown in Fig. 1.

$`\pi (\alpha )`$ reaches its maximum value at $`\alpha =1/2`$. Since $`\pi (\alpha )\alpha ^2`$ for $`\alpha \mathrm{}`$ the expected value of $`\alpha `$ and all higher moments diverge. This suggests that replacing the unknown $`\alpha `$ in the prior $`\pi (\theta |\alpha )`$ by any given numerical value $`\widehat{\alpha }`$ is probably not a good approximation.

Since $`\alpha `$ is unknown the effective prior for $`\theta =(\mu ,\sigma )`$ is obtained marginalizing $`\pi (\mu ,\sigma ,\alpha )=\pi (\mu ,\sigma |\alpha )\pi (\alpha )`$ over $`\alpha `$, eq.(15). Since $`\pi (\alpha )=0`$ for $`\alpha 1`$ as $`\epsilon 0`$ we can safely take the limit $`\epsilon _H0`$ or $`\sigma _H\mathrm{}`$ while keeping $`\sigma _L`$ fixed,

$$\pi (\mu ,\sigma ,\alpha )=\{\begin{array}{cc}\frac{1}{\mathrm{\Delta }\mu \sigma _L}\frac{\mathrm{exp}\left[\frac{1}{\alpha 1}\right]}{1\alpha }\left(\frac{\sigma }{\sigma _L}\right)^{\alpha 2}& \text{for }\alpha <1\\ 0& \text{for }\alpha 1.\end{array}$$
(31)
(However we cannot take $`\sigma _L0`$). The averaged prior for $`\mu `$ and $`\sigma `$ is

$$\overline{\pi }(\mu ,\sigma )=\frac{(\sigma _L/\sigma )^2}{\mathrm{\Delta }\mu \sigma _L}_{\mathrm{}}^1\frac{\mathrm{exp}\left[\frac{1}{\alpha 1}\right]}{1\alpha }\left(\frac{\sigma }{\sigma _L}\right)^\alpha 𝑑\alpha =\frac{2}{\mathrm{\Delta }\mu \sigma }K_0\left(2\sqrt{\mathrm{log}\frac{\sigma }{\sigma _L}}\right),$$
(32)
where $`K_0`$ is a modified Bessel function of the second kind. This is the entropic prior for the Gaussian model. The function

$$P(x)=\frac{2}{x}K_0\left(2\sqrt{\mathrm{log}x}\right)$$
(33)
is shown in Fig. 2 as a function of $`x=\sigma /\sigma _L`$. The singularity as $`x1`$ is integrable.

## 5 Final remarks

Using the method of maximum relative entropy we have translated the information contained in the known form of the likelihood into a prior distribution. The argument follows closely the analogous application of the ME method to statistical mechanics. For experiments that cannot be repeated the resulting “entropic prior” is formally identical with the Einstein fluctuation formula. For repeatable experiments, however, additional relevant information – represented in terms of a Lagrange multiplier $`\alpha `$ – must be included in the analysis. The important case of a Gaussian likelihood was treated in detail.

We have dealt with the simplest case where all we know about the quantities $`\theta `$ is that they appear as parameters in the likelihood $`p(y|\theta )`$. Our argument can, however, be generalized to situations where we know of additional relevant information beyond what is contained in the likelihood. Such information can be taken into account through additional constraints in the maximization of the entropy $`\sigma `$.

To conclude we comment briefly on the entropic priors proposed by Skilling and by Rodríguez. Skilling’s prior, unlike ours, is not restricted to probability distributions but is intended for generic “positive additive distributions” . Our argument, which consists in maximizing the entropy $`\sigma `$ subject to a constraint $`p(y,\theta )=\pi (\theta )p(y|\theta )`$, makes no sense for generic positive additive distributions for which there is no available product rule. Another important difference arises from the fact that Skilling’s entropy is not, in general, dimensionless and his hyper-parameter $`\alpha `$ is interpreted some sort of cutoff carrying the appropriate corrective units. Difficulties with Skilling’s prior were identified in .

Rodríguez’s approach is, like ours, derived from a maximum entropy principle . One (minor) difference is his treatment of the underlying measure $`m(y)`$. For us knowing $`m(y)`$ is part of knowing what data has been collected; for him $`m(y)`$ is an initial guess and he suggests setting $`m(y)=p(y|\theta _0)`$ for some value $`\theta _0`$. The more important difference, however, is that the number of observed data $`N`$ is left unspecified. The space $`\mathrm{\Theta }\times Y^N`$ over which distributions are defined, and therefore the distributions themselves, also remain unspecified. It is not clear what the maximization of an entropy over such unspecified spaces could possibly mean but a hyper-parameter $`\alpha `$ is eventually introduced and it is interpreted as a “virtual number of observations supporting the initial guess $`\theta _0`$.” A different interpretation is given in . Since $`\alpha `$ is treated on the same footing as the other parameters $`\theta `$,- Rodríguez’s approach requires an endless chain of hyper-parameters.

Acknowledgments- Many of our comments and arguments have been inspired by Carlos C. Rodríguez, Volker Dose, and Rainer Fischer through insightful questions and discussions which we gratefully acknowledge. A. C. also acknowledges the hospitality of the Max-Planck-Institut für Plasmaphysik during the two extended visits when most of this work was carried out.