format: text

text:
# Maximum Entropy and Bayesian Data Analysis: Entropic Priors

## 1 Introduction

The inference of physical quantities from data generated either by experiment or by numerical simulation is a ubiquitous and often cumbersome task. Whether the data is corrupted by noise, hampered by finite resolution or tied up in correlations, in principle it should always be possible to improve the analysis by taking into account, in addition to the information contained in the data, whatever other knowledge one might have about the physical quantities to be inferred or about how the data was generated. The way to link this prior information with the new information in the data is found in Bayesian probability theory.

Bayesian methods are increasingly popular in physics . They are essential whenever repeating the experiment many times in order to reduce the measurement uncertainty is either too expensive or time consuming. This is a common situation in astronomy and astrophysics , and also in large laboratory experiments as in fusion  and in high energy physics . Other typical uses in physics arise in spectrum restoration, in ill-posed inversion problems  and when separating a signal from an unknown background . Applications include mass spectrometry , Rutherford backscattering  and nuclear magnetic resonance .

From a general point of view the problem of inductive inference is to update from a prior probability distribution to a posterior distribution when new information becomes available. The challenge is to develop updating methods that are systematic and objective. Two methods have been found which are of very broad applicability: one is based on Bayes’ theorem and the other is based on the maximization of entropy. The choice between these two updating methods is dictated by the nature of the information being processed.

When we want to update our beliefs about the values of certain quantities $`\theta `$ on the basis of the observed values of other quantities $`y`$ – the data – and of some known relation between $`\theta `$ and $`y`$ we must use Bayes’ theorem. The updated or posterior distribution is $`p(\theta |y)\pi (\theta )p(y|\theta )`$; the relation between $`y`$ and $`\theta `$ is supplied by a known model $`p(y|\theta )`$; the previous knowledge about $`\theta `$ is codified both into the “prior” probability $`\pi (\theta )`$ and also in the “likelihood” distribution $`p(y|\theta )`$.

The selection of the prior $`\pi (\theta )`$ is a controversial issue which has generated an enormous literature . The difficulty is that it is not clear how to carry out an objective translation of our previous beliefs about $`\theta `$ into a distribution $`\pi (\theta )`$. One reasonable attitude is to admit subjectivity and recognize that different individuals may start from the same information and legitimately end with different translations. In simple cases experience and physical intuition have led to a considerable measure of success, but we are often confronted with new complex situations involving perhaps parameter spaces of high dimensionality where we have neither a previous experience nor a reliable intuition.

On the other hand, there are special cases where some degree of objectivity can be attained. For example, requirements of invariance can go a long way towards the complete specification of a prior. Considerable effort has been spent seeking an objective characterization of that elusive state of knowledge that presumably reflects complete ignorance. Although there are convincing arguments against the existence of such non-informative priors , the search has had the merit of suggesting connections with the notion of entropy  including two proposals for “entropic priors” . This brings us to the second method of processing information.

Bayes’ theorem follows from the product rule for joint probabilities, $`p(y,\theta )=\pi (\theta )p(y|\theta )`$, and therefore its applicability is restricted to situations where assertions concerning the joint values of the data $`y`$ and the parameters $`\theta `$ are meaningful. But there are situations where the available information is of a different nature and involves assertions about the probabilities themselves. Such information, which includes but is not limited to assertions about expected values, cannot be processed using Bayes’ theorem.

The method of Maximum Entropy (ME) is designed for updating from a prior probability distribution to a posterior distribution when the information to be processed takes the form of a constraint on the family of acceptable posterior distributions . The early and less satisfactory justification of the ME method followed from interpreting entropy, through the Shannon axioms, as a measure of the amount of uncertainty in a probability distribution . Objections to this approach are that the Shannon axioms refer to probabilities of discrete variables, the entropy of continuous variables is not defined, and that the use of entropy as the unique measure of uncertainty remained questionable. Other so-called entropies could and, indeed, were introduced. Ultimately, the real problem is that Shannon was not concerned with inductive inference. He was not trying to update probability distributions but was instead analyzing the capacity of communication channels. Shannon’s entropy makes no reference to prior distributions.

Considerations such as these motivated several attempts to justify the ME method directly as a method of inductive inference without invoking questionable measures of uncertainty . The concept of relative entropy is then introduced as a tool for consistent reasoning which, in the special case of uniform priors, reduces to the usual entropy. There is no need for an interpretation in terms of heat, disorder, or uncertainty, or even in terms of an amount of information. Perhaps this is the explanation of why the search for the meaning of entropy has turned out to be so elusive: strictly, *entropy needs no interpretation*. In section 2, as background for the rest of the paper, we present a brief outline of one such ‘no-interpretation’ approach inspired by .

In this paper we use entropic arguments to translate prior information into a prior distribution. Rather than seeking a totally non-informative prior, we make use of information that we do in fact have. Remarkably, it turns out that the very conditions that allow us to contemplate using Bayes’ theorem – namely, knowledge of a likelihood function, $`p(y|\theta )`$ – already constitute valuable prior information. In this sense one can assert that the search for completely non-informative priors is misplaced: if we do not know the likelihood, then prior distributions are not needed anyway. The prior thus obtained is an “entropic prior.” The name and the first proposal of a prior of this kind is due to Skilling  for the case of discrete distributions. The generalization to the continuous case and further elaborations by Rodríguez  constitute a second proposal.

It is essential for the successful use of any prior, and of entropic priors in particular, to be aware of what information they contain and, crucially, what information they do not contain. No prior can be expected to succeed unless all the information relevant to the problem at hand has been taken into account. It is quite likely that most practical problems that were encountered with entropic priors in the past can be traced to a failure to identify and incorporate all the relevant information.

The information that has, in this paper, been translated into the entropic prior is that contained in the likelihood. The *bare* entropic priors discussed here apply to a situation where all we know about the quantities $`\theta `$ is that they appear as parameters in the likelihood $`p(y|\theta )`$, *and nothing else*. Generalizations are, of course, possible. Sometimes we are aware of additional relevant information beyond what is contained in the likelihood and it can easily be incorporated into a modified entropic prior. Other times we might be guilty of overlooking additional information we already have. Indeed, we would not be willing to spend valuable effort in the determination of a parameter $`\theta `$ unless we suspected that knowledge of $`\theta `$ has important implications elsewhere. Typically we know something about the physical significance and the physical meaning of $`\theta `$. It is clear that in these cases we know considerably more than just that $`\theta `$ is a parameter appearing in the likelihood. We might even conceive of several different experiments, $`e=1,2,\mathrm{}`$, each yielding different sets of data $`y_e`$ related to $`\theta `$ by different likelihood functions $`p_e(y_e|\theta )`$. It is sometimes objected that one’s prior knowledge about $`\theta `$ should not depend on which experiment one decides to use to measure it, but this objection is misplaced: the mere fact that $`\theta `$ is measurable through one or another experiment is additional information which, if relevant, should be taken into account.

Another family of problems that can be tackled as a rather straightforward extension of the ideas described here involve choosing which likelihood distribution from among several competing candidates is responsible for generating the data. Indeed, it is clear that any systematic approach to model selection requires as a prerequisite the capability to process in an objective way the information implicit in each of those likelihoods. Except for some brief remarks in the final section, all these further developments, valuable as they might be, will be addressed elsewhere.

Our contribution includes a derivation of an entropic prior (section 3) following the same principles of ME inference that have been successful in statistical mechanics. In fact, our whole approach is guided by intuition gained from applications of ME to statistical mechanics. Preliminary steps along this direction were taken in  where a problem with the important case of experiments that can be indefinitely repeated had already been identified but not fully resolved. This problem, re-examined in section 4, is interpreted as a symptom that important relevant information has been overlooked. The complete resolution, which hinges on identifying and incorporating this additional information, is given in sections 5 and 6. The actual way in which ME is used in the derivation, in analogy to standard applications in statistical mechanics, turns out to be important because it clarifies what it is that has been derived and how to use it: ours is, in effect, a third proposal for an entropic prior. In section 7 we discuss in detail the important example of a Gaussian likelihood and finally, in section 8, we summarize and comment on the differences among the three versions of entropic prior and on possible further developments.

## 2 The logic behind the ME method

The goal is to update beliefs about $`yY`$ which are codified in the prior probability distribution $`m(y)`$ to a posterior distribution $`p(y)`$ when new information in the form of a constraint becomes available. (The constraints can, but need not, be linear.) The selection is carried out by ranking the probability distributions according to increasing *preference*. One feature we impose on the ranking scheme is transitivity: if distribution $`p_1`$ is preferred over distribution $`p_2`$, and $`p_2`$ is preferred over $`p_3`$, then $`p_1`$ is preferred over $`p_3`$. Such transitive rankings are implemented by assigning to each $`p(x)`$ a real number $`S[p]`$ called the entropy of $`p`$ in such a way that if $`p_1`$ is preferred over $`p_2`$, then $`S[p_1]>S[p_2]`$. The selected $`p`$ will be that which maximizes $`S[p]`$. Thus the method involves entropies which are real numbers and entropies that should be maximized. These are features imposed by design; they are dictated by the function that the ME method is supposed to perform .

Next we determine the functional form of $`S[p]`$. This is the rule that defines the ranking scheme. *The purpose of the rule is to do induction.* We want to extrapolate, to generalize from those special cases where we know what the preferred distribution should be to the much larger number of cases where we do not. Thus, in order to be an inductive rule $`S[p]`$ must have wide applicability; we will assume that *the same rule applies to all cases*. There is no justification for this universality except for the usual pragmatic justification of induction: we must be inclined to generalize lest we become paralyzed into not generalizing at all. But then, we should remain cautious and keep in mind that in many instances induction just fails.

The argument goes as follows . If a general theory exists, then it must apply to special cases. Furthermore, if in a certain special case the preferred distribution is known, then this knowledge can be used to constrain the form of $`S[p]`$. Finally, if enough special cases are known, then $`S[p]`$ will be completely determined. The known special cases are called the “axioms” of ME. As we will see below the axioms reflect the conviction that one should not change one’s mind frivolously, that whatever was learned in the past is important. The chosen posterior distribution should coincide with the prior as closely as possible and one should only update those aspects of one’s beliefs for which corrective new evidence has been supplied. The three axioms are listed below.

Axiom 1: Locality. *Local information has local effects.* We do not revise the relative probabilities $`p(y^{})/p(y)`$ with $`y`$ and $`y^{}`$ within a certain domain $`DY`$ unless the newly provided information refers explicitly to the domain $`D`$. The power of this axiom stems from the arbitrariness in the choice of $`D`$. The consequence of the axiom is that non-overlapping domains of $`y`$ contribute additively to the entropy: $`S[p]=𝑑yF(p(y))`$ where $`F`$ is some unknown function.

Axiom 2: Coordinate invariance. *The ranking should not depend on the system of coordinates.* The coordinates that label the points $`y`$ are arbitrary; they carry no information. The consequence of this axiom is that $`S[p]=𝑑yp(y)f(p(y)/m(y))`$ involves coordinate invariants such as $`dyp(y)`$ and $`p(y)/m(y)`$, where the density $`m(y)`$ and the function $`f`$ are, at this point, unknown.

Next we make a second use of the locality axiom to enforce objectivity. We allow domain $`D`$ to extend over the whole space $`Y`$ and assert that *when there is no new information there is no reason to change one’s mind.* When there are no constraints the selected posterior distribution should coincide with the prior distribution. This eliminates the arbitrariness in the density $`m(y)`$: up to normalization $`m(y)`$ is the prior distribution.

Axiom 3: Consistency for independent subsystems. *When a system is composed of independent subsystems it should not matter whether the inference procedure treats them separately or jointly.* If $`y=(y_1,y_2)Y=Y_1\times Y_2`$, and the subsystem priors $`m_1(y_1)`$ and $`m_2(y_2)`$ are respectively upgraded to $`p_1(y_1)`$ and $`p_2(y_2)`$, then the prior for the whole system $`m_1(y_1)m_2(y_2)`$ should be upgraded to $`p_1(y_1)p_2(y_2)`$. This axiom restricts the function $`f`$ to be a logarithm. (The fact that the logarithm applies also when the subsystems are not independent follows from our inductive hypothesis that the ranking scheme has universal applicability.)

The overall consequence of these axioms  is that probability distributions $`p(y)`$ should be ranked relative to the prior $`m(y)`$ according to their (relative) entropy ,

$$S[p,m]=𝑑yp(y)\mathrm{log}\frac{p(y)}{m(y)}.$$
(1)
The derivation has singled out $`S[p,m]`$ as *the unique entropy to be used in inductive inference*. Other expressions, such as $`S[m,p]`$, or $`S[p,m]+S[m,p]`$, or even expressions that do not involve the logarithm, may be useful for other purposes, but they do not constitute an induction: they are not a generalization from the simple cases described in the axioms.

We end this section with two comments on the prior density $`m(y)`$. First, $`S[p,m]`$ may be infinitely negative when $`m(y)`$ vanishes within some region $`D`$. In other words, the ME method confers an overwhelming preference on those distributions $`p(y)`$ that vanish whenever $`m(y)`$ does. Is this a problem? Not really. A similar “problem” also arises in the context of Bayes’ theorem. A vanishing prior represents a tremendously serious prejudice because no amount of data to the contrary would allow us to revise it. The solution in both cases is to recognize that unless we are absolutely certain that $`y`$ could not possibly lie within $`D`$ then we should not have assigned $`m(y)=0`$ in the first place. Assigning a very low but non zero prior represents a safer and less prejudiced representation of one’s beliefs and/or doubts both in the context of Bayesian and of ME inference.

Second, choosing the prior density $`m(y)`$ can be tricky. When there is no information leading us to prefer one microstate of a physical system over another we might as well assign equal prior probability to each state. Thus it is reasonable to identify $`m(y)`$ with the density of states and the invariant $`m(y)dy`$ is the number of microstates in $`dy`$. This is the basis for statistical mechanics. Other examples of relevance to physics arise when there is no reason to prefer one region of the space $`Y`$ over another. Then we should assign the same prior probability to regions of the same “volume,” and we can choose $`_R𝑑ym(y)`$ to be the volume of a region $`R`$ in the space $`Y`$. Notice that because of the presence of the prior $`m(y)`$ not all subjectivity has been eliminated and Laplace’s principle of insufficient reason still plays an important role, albeit in a somewhat modified form. Just as with Bayes’ theorem, what is objective here is the manner in which information is processed, not the initial probability assignments.

## 3 Entropic priors: the basic idea

In this section we follow  closely. We use the ME method to derive a prior $`\pi (\theta )`$ for use in Bayes’ theorem,

$$p(\theta |y)p(y,\theta )=\pi (\theta )p(y|\theta ).$$
(2)
The selection of a preferred distribution using the ME method demands that the space in which the search will be conducted be specified. Being a consequence of the product rule for joint probabilities, Bayes’ theorem requires that assertions such as ‘$`y`$ and $`\theta `$’ be meaningful and that the ‘probability of $`y`$ and $`\theta `$’ be well defined. Therefore we must focus our attention on $`p(y,\theta )`$ rather than $`\pi (\theta )`$; the relevant universe of discourse is neither $`\mathrm{\Theta }`$, the space of all $`\theta `$s, nor the data space $`Y`$, but the product $`\mathrm{\Theta }\times Y`$. This point, first made by Rodríguez , is central to the argument. Our derivation and the final result, however, differ from his in several respects .

To rank distributions in the space $`\mathrm{\Theta }\times Y`$ we must decide on a prior $`m(y,\theta )`$. At this starting point absolutely nothing is known about the variables $`\theta `$, in particular, they have no physical meaning, and no relation between $`y`$ and $`\theta `$ is known. The $`\theta `$s are totally arbitrary. Therefore the prior must be a product $`m(y)\mu (\theta )`$ of the separate priors in the spaces $`Y`$ and $`\mathrm{\Theta }`$. Indeed, the distribution that maximizes the relative entropy

$$\sigma [p]=𝑑y𝑑\theta p(y,\theta )\mathrm{log}\frac{p(y,\theta )}{m(y)\mu (\theta )},$$
(3)
when no constraints are imposed is $`p(y,\theta )m(y)\mu (\theta )`$; it is such that data about $`y`$ tells us absolutely nothing about $`\theta `$.

In what follows we assume that $`m(y)`$ is known. We consider this an important part of understanding what data it is that has been collected. In section 7 we will suggest a reasonable $`m(y)`$ for the special case of a Gaussian likelihood. The prior $`\mu (\theta )`$ remains unspecified.

Next we incorporate the crucial piece of information from which the parameters $`\theta `$ derive their physical meaning and which establishes the relation between $`\theta `$ and $`y`$: the likelihood function $`p(y|\theta )`$ is known. This has two consequences: First, the joint distribution $`p(y,\theta )`$ is constrained to be of the form $`\pi (\theta )p(y|\theta )`$. Notice that this constraint is not in the form that is most usual for applications of the ME method: it is not an expectation value. Note also that the only information we are using about the quantities $`\theta `$ is that they appear as parameters in the likelihood $`p(y|\theta )`$, *nothing else*. In many situations of experimental interest there exists additional relevant information beyond what is contained in the likelihood; such information should be included as additional constraints in the maximization of the relative entropy $`\sigma `$.

Second, now that a bare minimum is known about $`\theta `$, namely that each $`\theta `$ represents a probability distribution, there is a natural but still subjective choice for $`\mu (\theta )`$. As discussed in , except for an overall multiplicative constant, there is a unique Riemannian metric that adequately reflects the fact that the points in a space of probability distributions are not ‘structureless’, but happen to be probability distributions; this is the Fisher-Rao metric. Within the finite-dimensional subspace defined by the constraint – the known $`p(y|\theta )`$ – the natural metric on $`\mathrm{\Theta }`$ is $`d\mathrm{}^2=g_{ij}d\theta ^id\theta ^j`$, where the unique $`g_{ij}`$ induced by the family of distributions $`p(y|\theta )`$ is

$$g_{ij}=𝑑yp(y|\theta )\frac{\mathrm{log}p(y|\theta )}{\theta ^i}\frac{\mathrm{log}p(y|\theta )}{\theta ^j}.$$
(4)
Accordingly we choose $`\mu (\theta )=g^{1/2}(\theta )`$, where $`g(\theta )`$ is the determinant of $`g_{ij}`$. Having identified the prior measure and the constraints, we allow the ME method to take over.

The preferred distribution $`p(y,\theta )`$ is chosen by varying $`\pi (\theta )`$ to maximize

$`\sigma [\pi ]`$ $`={\displaystyle 𝑑y𝑑\theta \pi (\theta )p(y|\theta )\mathrm{log}\frac{\pi (\theta )p(y|\theta )}{g^{1/2}(\theta )m(y)}}`$ (5)
$`={\displaystyle 𝑑\theta \pi (\theta )\mathrm{log}\frac{\pi (\theta )}{g^{1/2}(\theta )}}+{\displaystyle 𝑑\theta \pi (\theta )S(\theta )},`$
where $`S(\theta )`$ is the entropy of the likelihood,

$$S(\theta )=𝑑yp(y|\theta )\mathrm{log}\frac{p(y|\theta )}{m(y)}.$$
(6)
Writing the Lagrange multiplier that enforces $`𝑑\theta \pi (\theta )=1`$ as $`1\mathrm{log}\zeta `$, and assuming $`p(y|\theta )`$ is normalized yields

$$0=𝑑\theta \left(\mathrm{log}\frac{\pi (\theta )}{g^{1/2}(\theta )}+S(\theta )\mathrm{log}\zeta \right)\delta \pi (\theta ),$$
(7)
Therefore the probability that the value of $`\theta `$ should lie within the small volume $`g^{1/2}(\theta )d\theta `$ is

$$\pi (\theta )d\theta =\frac{1}{\zeta }e^{S(\theta )}g^{1/2}(\theta )d\theta \text{with }\zeta =𝑑\theta g^{1/2}(\theta )e^{S(\theta )}.$$
(8)
This entropic prior is our first main result. It tells us that the preferred value of $`\theta `$ is that which maximizes the entropy $`S(\theta )`$ because this maximizes the scalar probability density $`\mathrm{exp}S(\theta )`$. It also tells us the degree to which values of $`\theta `$ away from the maximum are ruled out; in many cases the preference for the ME distribution can be overwhelming. Note also that the density $`\mathrm{exp}S(\theta )`$ is a scalar function and the presence of the Jacobian factor $`g^{1/2}(\theta )`$ makes eq.(8) manifestly invariant under changes of the coordinates $`\theta `$ in the space $`\mathrm{\Theta }`$.

We can claim a partial success. The ingredients that have been used are precisely those that led us to consider using Bayes’ theorem in the first place. The information contained in the model – by which we mean that the data space $`Y`$, its measure $`m(y)`$, and the conditional distribution $`p(y|\theta )`$ – has been translated into a prior $`\pi (\theta )`$. The success is partial because it has been achieved for the special case of the fixed data space $`Y`$ of those experiments which cannot conceivably be repeated. A more complete treatment requires that we address the important case of experiments that can be repeated indefinitely.

## 4 Repeatable experiments

Experiments need not be repeatable but sometimes they are. Let us assume that successive repetitions are possible and that they happen to be independent. Suppose, to be specific, that the experiment is performed twice so that the space of data $`Y\times Y=Y^2`$ consists of the possible outcomes $`y_1`$ and $`y_2`$. Suppose further that $`\theta `$ is not a “random” variable; the value of $`\theta `$ is fixed but unknown. Then the joint distribution in the space $`\mathrm{\Theta }\times Y^2`$ is

$$p(y_1,y_2,\theta )=\pi ^{(2)}(\theta )p(y_1,y_2|\theta )=\pi ^{(2)}(\theta )p(y_1|\theta )p(y_2|\theta ),$$
(9)
and the appropriate $`\sigma `$ entropy is

$$\sigma ^{(2)}[\pi ]=𝑑y_1𝑑y_2𝑑\theta p(y_1,y_2,\theta )\mathrm{log}\frac{p(y_1,y_2,\theta )}{\left[g^{(2)}(\theta )\right]^{1/2}m(y_1)m(y_2)},$$
(10)
where $`g^{(2)}(\theta )`$ is the determinant of the Fisher-Rao metric for $`p(y_1,y_2|\theta )`$. From Eq.(4) it follows that $`g_{ij}^{(2)}=2g_{ij}`$ so that $`g^{(2)}(\theta )=2^dg(\theta )`$, $`d`$ being the dimension of $`\theta `$. Maximizing $`\sigma ^{(2)}[\pi ]`$ subject to $`𝑑\theta \pi ^{(2)}(\theta )=1`$ we get

$$\pi ^{(2)}(\theta )=\frac{1}{Z^{(2)}}g^{1/2}(\theta )e^{S^{(2)}(\theta )}=\frac{1}{Z^{(2)}}g^{1/2}(\theta )e^{2S(\theta )},$$
(11)
where $`S^{(2)}(\theta )=2S(\theta )`$ is the entropy of $`p(y_1,y_2|\theta )`$, and $`S(\theta )\stackrel{def}{=}S^{(1)}(\theta )`$. The generalization to $`N`$ repetitions of the experiment, with data space $`Y^N`$, is immediate,

$$\pi ^{(N)}(\theta )=\frac{1}{Z^{(N)}}g^{1/2}(\theta )e^{S^{(N)}(\theta )}=\frac{1}{Z^{(N)}}g^{1/2}(\theta )e^{NS(\theta )}.$$
(12)
This is clearly wrong: the dependence of $`\pi ^{(N)}`$ on the amount $`N`$ of data would lead us to a perpetual revision of the prior as more data is collected. The absurdity of this situation becomes manifest when we consider the case of large $`N`$. Then the exponential preference for the value of $`\theta `$ that maximizes $`S(\theta )`$ becomes so pronounced that no amount of data to the contrary can successfully overcome its effect. The data becomes irrelevant, and the more data we have, the more irrelevant it becomes.

Repeatable experiments present us with a problem. One possible attitude is to blame the ME method: it gives nonsense and cannot be trusted. As with all inductive methods this is, of course, a logical possibility. A second, more constructive approach, is to always be prepared to question the results of ME calculations on the basis that there is no guarantee that all the information relevant to the situation at hand has been taken into account. The problem is not a failure of the ME method but a failure to include all the relevant information.

That this is indeed the case can be seen as follows: When we say an experiment can be repeated twice, $`N=2`$, we actually know more than just $`p(y_1,y_2|\theta )=p(y_1|\theta )p(y_2|\theta )`$. We also know that forgetting or discarding the value of say $`y_2`$, yields an experiment that is totally indistinguishable from the single, $`N=1`$, experiment. This *additional* information is quantitatively expressed by $`𝑑y_2p(y_1,y_2,\theta )=p(y_1,\theta )`$, or equivalently

$$𝑑y_2\pi ^{(2)}(\theta )p(y_1|\theta )p(y_2|\theta )=\pi ^{(1)}(\theta )p(y_1|\theta ),$$
(13)
which leads to $`\pi ^{(2)}(\theta )=\pi ^{(1)}(\theta )`$. In the general case we get the manifestly reasonable result

$$\pi ^{(N)}(\theta )=\pi ^{(N1)}(\theta )=\mathrm{}=\pi ^{(1)}(\theta ).$$
(14)
The challenge then is to identify a constraint that codifies this information within each space $`\mathrm{\Theta }\times Y^N`$.

## 5 More information: the Lagrange multiplier $`\alpha `$

The problem with the prior $`\pi ^{(N)}(\theta )`$ in eq.(12) is that it expresses an overwhelming preference for the value $`\theta _{\mathrm{max}}`$ of $`\theta `$ that maximizes the entropy $`S(\theta )`$. Indeed, as $`N\mathrm{}`$ we have $`\pi ^{(N)}(\theta )\delta (\theta \theta _{\mathrm{max}})`$ leading to

$$S=𝑑\theta \pi ^{(N)}(\theta )S(\theta )\stackrel{N\mathrm{}}{}S(\theta _{\mathrm{max}}),$$
(15)
which is manifestly incorrect. This suggests that a better prior would be obtained by maximizing the entropy $`\sigma ^{(N)}`$ of distributions on the space space $`\mathrm{\Theta }\times Y^N`$ subject to an additional constraint on the numerical value $`\overline{S}`$ of the expected entropy $`S`$. It is not that we happen to know the numerical value $`\overline{S}`$ of $`S`$. In fact we do not. It is rather that we recognize that information about $`\overline{S}`$ is relevant in the sense that if $`\overline{S}`$ were known the problem above would not arise. Naturally, additional effort will be required to obtain the needed value of $`\overline{S}`$.

The logic of the previous paragraph may sound unfamiliar and further comments may be helpful. When justifying the use of the ME method to obtain, say, the canonical Boltzmann-Gibbs distribution ($`P_qe^{\beta E_q}`$) it has been common to say something like “we seek the minimally biased (*i.e.* maximum entropy) distribution that codifies the information we do possess (the expected energy) and nothing else”. Many authors find this justification objectionable. Indeed, they might argue, for example, that the spectrum of black body radiation is what it is independently of whatever information happens to be available to us. We prefer to phrase our objection differently: in most realistic situations the expected value of the energy is not a quantity we happen to know. Nevertheless, it is still true that maximizing entropy subject to a constraint on this (unknown) expected energy leads to correct predictions. Therefore, the justification behind imposing a constraint on the expected energy cannot be that this is a quantity that happens to be known – because it is not – but rather that theexpected energy is the quantity that *should* beknown. Even if unknown, we recognize it as the crucial relevant information without which no successful predictions can be made. Therefore we proceed as if this crucial information were available and produce a formalism that contains the temperature as a free parameter that will later have to be obtained from the experiment itself. In other words, the temperature (or expected energy) is one additional parameter to be inferred from the data.

The entropy on the space $`\mathrm{\Theta }\times Y^N`$ is

$`\sigma ^{(N)}[\pi ]`$ $`={\displaystyle 𝑑\theta 𝑑y^{(N)}\pi (\theta )p(y^{(N)}|\theta )\mathrm{log}\frac{\pi (\theta )p(y^{(N)}|\theta )}{g^{1/2}(\theta )m(y^{(N)})}}`$
$`={\displaystyle 𝑑\theta \pi (\theta )\mathrm{log}\frac{\pi (\theta )}{g^{1/2}(\theta )}}+N{\displaystyle 𝑑\theta \pi (\theta )S(\theta )}`$ (16)
where $`S(\theta )`$ given by eq.(6). (A constant factor of $`N^{d/2}`$ associated to the Fisher-Rao measure $`g^{(N)}(\theta )`$ has been omitted. It would eventually be absorbed into the normalization of $`\pi (\theta )`$.) To obtain the prior $`\pi (\theta )`$ we maximize $`\sigma ^{(N)}`$ subject to constraints on $`S`$ and that $`\pi `$ be normalized,

$$\delta \left[\sigma ^{(N)}+(1\mathrm{log}\zeta )\left(𝑑\theta \pi (\theta )1\right)+\lambda _N\left(𝑑\theta \pi (\theta )S(\theta )\overline{S}\right)\right]=0.$$
(17)
This gives,

$$𝑑\theta \left(\mathrm{log}\frac{\pi (\theta )}{g^{1/2}(\theta )}+(N+\lambda _N)S(\theta )\mathrm{log}\zeta \right)\delta \pi (\theta )=0.$$
(18)
Therefore,

$$\pi (\theta )=\frac{1}{\zeta }g^{1/2}(\theta )\mathrm{exp}\left[(N+\lambda _N)S(\theta )\right].$$
(19)
The undesired dependence on $`N`$ is eliminated if in each space $`\mathrm{\Theta }\times Y^N`$ the Lagrange multipliers $`\lambda _N`$ are chosen so that $`N+\lambda _N=\alpha `$ is a constant independent of $`N`$. The resulting entropic prior,

$$\pi (\theta |\alpha )=\frac{1}{\zeta (\alpha )}g^{1/2}(\theta )e^{\alpha S(\theta )},$$
(20)
satisfies eq.(14). This is our second main result. The prior $`\pi (\theta |\alpha )`$ codifies information contained in the likelihood function, plus information about the expected value of the entropy of the likelihood implicit in the hyper-parameter $`\alpha `$,

$$\overline{S}(\alpha )=\frac{d}{d\alpha }\mathrm{log}\zeta (\alpha ),$$
(21)
with $`\zeta (\alpha )`$ is given by

$$\zeta (\alpha )=𝑑\theta g^{1/2}(\theta )e^{\alpha S(\theta )}.$$
(22)
The next and final step is figure out which $`\alpha `$ applies to the particular experimental situation under consideration. The natural way to proceed is to invoke Bayes’ theorem

$$p(\alpha ,\theta |y^N)=\pi (\alpha )\pi (\theta |\alpha )\frac{p(y^N|\theta )}{p(y^N)}.$$
(23)
The choice of a prior $`\pi (\alpha )`$ for $`\alpha `$ itself is addressed in the next section. If we were truly interested in the actual $`\alpha `$, we could marginalize over $`\theta `$ to obtain

$$p(\alpha |y^N)=𝑑\theta p(\alpha ,\theta |y^N)=\frac{\pi (\alpha )}{p(y^N)}𝑑\theta \pi (\theta |\alpha )p(y^N|\theta ).$$
(24)
But our interest in the value of $`\alpha `$ is only indirect; $`\alpha `$ is a necessary but annoying technical complication along the way to the real goal which is inferring $`\theta `$. Marginalizing over $`\alpha `$, we get

$$p(\theta |y^N)=𝑑\alpha p(\alpha ,\theta |y^N)=\overline{\pi }(\theta )\frac{p(y^N|\theta )}{p(y^N)}$$
(25)
where

$$\overline{\pi }(\theta )=𝑑\alpha \pi (\alpha )\pi (\theta |\alpha ).$$
(26)
This is the answer we sought: the effective prior for $`\theta `$, the averaged $`\overline{\pi }(\theta )`$, is independent of the actual data $`y^N`$, as it should. The last step is the assignment of $`\pi (\alpha )`$.

## 6 An entropic prior for $`\alpha `$

To remain consistent with the spirit of this paper, namely using ME to obtain priors, the prior for $`\alpha `$ must itself be an entropic prior. The motivation behind discussing entropic priors is that we wish to consider information included in the likelihood function. Since $`p(y|\theta )`$ refers to $`\theta `$ but makes no reference to any hyper-parameters it is quite clear that $`\alpha `$ should not be treated like the other $`\theta `$s. The relation between $`\alpha `$ and the data $`y`$ is indirect: $`\alpha `$ is related to $`\theta `$, and $`\theta `$ is related to $`y`$. Once $`\theta `$ is given, the data $`y`$ becomes irrelevant, it contains no further information about $`\alpha `$. The whole significance of $`\alpha `$ is derived purely from its appearance in $`\pi (\theta |\alpha )`$, eq.(20). Therefore, the relevant universe of discourse is $`A\times \mathrm{\Theta }`$ with $`\alpha A`$. We focus our attention on the joint distribution

$$\pi (\alpha ,\theta )=\pi (\alpha )\pi (\theta |\alpha ).$$
(27)
and we obtain $`\pi (\alpha )`$ by maximizing the entropy

$$\mathrm{\Sigma }[\pi ]=𝑑\alpha 𝑑\theta \pi (\alpha ,\theta )\mathrm{log}\frac{\pi (\alpha ,\theta )}{\gamma ^{1/2}(\alpha )g^{1/2}(\theta )}$$
(28)
where $`\gamma ^{1/2}(\alpha )`$ is determined below. Since no reference is made to repeatable experiments in $`Y^N`$ there is no need for any further constraints except for normalization.

The Fisher-Rao measure $`\gamma ^{1/2}(\alpha )`$ in eq.(28) is

$$\gamma (\alpha )=𝑑\theta \pi (\theta |\alpha )\left[\frac{d}{d\alpha }\mathrm{log}\pi (\theta |\alpha )\right]^2.$$
(29)
Using eqs.(20),(21) and (22) we get

$$\gamma (\alpha )=𝑑\theta \pi (\theta |\alpha )\left[S(\theta )\frac{d\mathrm{log}\zeta (\alpha )}{d\alpha }\right]^2=(\mathrm{\Delta }S)^2,$$
(30)
but

$$\frac{d\overline{S}(\alpha )}{d\alpha }=\frac{d}{d\alpha }\frac{1}{\zeta (\alpha )}\frac{d\zeta (\alpha )}{d\alpha }=\frac{1}{\zeta (\alpha )}\frac{d^2\zeta (\alpha )}{d\alpha ^2}\left[\frac{1}{\zeta (\alpha )}\frac{d\zeta (\alpha )}{d\alpha }\right]^2=(\mathrm{\Delta }S)^2.$$
(31)
Therefore,

$$\gamma (\alpha )=\frac{d^2\mathrm{log}\zeta (\alpha )}{d\alpha ^2}.$$
(32)
The interpretation is straightforward: the distance between $`\pi (\theta |\alpha )`$ and $`\pi (\theta |\alpha +d\alpha )`$ is given by

$$\gamma ^{1/2}(\alpha )d\alpha =\mathrm{\Delta }S(\alpha )d\alpha ,$$
(33)
or, in words, the local entropy uncertainty $`\mathrm{\Delta }S`$ is the distance per unit change in $`\alpha `$.

To maximize $`\mathrm{\Sigma }`$ rewrite it as

$$\mathrm{\Sigma }[\pi ]=𝑑\alpha \pi (\alpha )\mathrm{log}\frac{\pi (\alpha )}{\gamma ^{1/2}}+𝑑\alpha \pi (\alpha )s(\alpha ),$$
(34)
where $`s(\alpha )`$ is given by

$`s(\alpha )`$ $`={\displaystyle 𝑑\theta \pi (\theta |\alpha )\mathrm{log}\frac{\pi (\theta |\alpha )}{g^{1/2}(\theta )}}`$ (35)
$`=\mathrm{log}\zeta (\alpha )\alpha {\displaystyle \frac{d\mathrm{log}\zeta (\alpha )}{d\alpha }}.`$
Then, varying with respect to $`\pi (\alpha )`$ gives

$$\pi (\alpha )=\frac{1}{z}\gamma ^{1/2}(\alpha )e^{s(\alpha )}.$$
(36)
This is our third main result. It completes our derivation of the actual prior for $`\theta `$: the averaged $`\overline{\pi }(\theta )`$ in eq.(26) codifies information contained in the likelihood function, plus the insight that for repeatable experiments, information about the expected likelihood entropy, even if unavailable, is relevant.

We argued above that the hyper-parameter $`\alpha `$ should not be treated in the same way as the other parameters $`\theta `$ because the likelihood $`\pi (y|\theta )`$ refers only to $`\theta `$s and not to $`\alpha `$. Nonetheless, it may still be worthwhile to discuss briefly what would happen if $`\alpha `$ were treated as one of the $`\theta `$s. In this case, the entropic prior $`\pi (\alpha )`$ would be determined by focusing our attention on the joint distribution

$$p(\alpha ,\theta ,y^N)=\pi (\alpha )\pi (\theta |\alpha )p(y^N|\theta ),$$
(37)
where the last two factors on the right are assumed known. The assumed universe of discourse would be $`A\times \mathrm{\Theta }\times Y^N`$. A straightforward application of the ME method would, as before, run into trouble with an unwanted $`N`$ dependence which would require the introduction of a new constraint on the appropriate expected entropy. Thus, the entropic prior for $`\alpha `$ would involve a second hyper-parameter $`\alpha _2`$. The unknown $`\alpha _2`$ would itself require its own entropic prior, involving yet a third hyper-parameter $`\alpha _3`$, and so on. There would be an endless chain of hyper-parameters . In any practical calculation, the chain would have to be truncated. Whether the predictions about $`\theta `$ depend on where and how the truncation is carried out remains to be studied. But, fortunately, this is not necessary: $`\alpha `$ is not like the other $`\theta `$s.

## 7 Example: a Gaussian model

Consider data $`y^N=\{y_1,\mathrm{},y_N\}`$ that are scattered around an unknown value $`\mu `$,

$$y=\mu +\nu $$
(38)
with $`\nu =0`$ and $`\nu ^2=\sigma ^2.`$ The goal is to estimate the parameters $`\theta =(\theta ^1,\theta ^2)=(\mu ,\sigma )`$ on the basis of the data $`y^N`$ and the information implicit in the model: the data space $`Y`$, the measure $`m(y)`$ (discussed below), and the Gaussian likelihood,

$$p(y|\mu ,\sigma )=\frac{1}{\left(2\pi \sigma ^2\right)^{1/2}}\mathrm{exp}\left[\frac{(y\mu )^2}{2\sigma ^2}\right].$$
(39)
In section 3 we asserted that knowing the measure $`m(y)`$ is part of knowing what data has been collected. Therefore, nothing can be said about $`m(y)`$ without further specification of the experimental situation. It turns out, however, that in many physical situations where the data happen to be distributed according to eq.(39) the underlying space $`Y`$ is sufficiently symmetric, *i.e.*, invariant under translations, that we can assume $`m(y)=m=\mathrm{constant}.`$ This is physically reasonable. Gaussian distributions arise when the measured value of $`y`$ is the sum of a large number of “microscopic” contributions and the details of how the individual contributions are themselves distributed are washed out in the “macroscopic” sum. The macroscopically relevant features are just those that distinguish one Gaussian from another, namely, the mean $`\mu `$ and the variance $`\sigma ^2`$. This is the physical basis behind the Central Limit Theorem. But if microscopic details are irrelevant it should be possible to understand the situation from a purely macroscopic point of view: it should be possible to obtain the Gaussian distribution as the preferred one among all those with the given $`\mu `$ and $`\sigma ^2`$, and this is, indeed, the case: setting $`m(y)=\mathrm{constant}`$ in $`S[p,m]`$, eq.(1), and maximizing subject to constraints on the mean and variance yields eq.(39).

From eqs. (6) and (39) the entropy of the likelihood is

$$S(\mu ,\sigma )=\mathrm{log}\left[\frac{\sigma }{\sigma _0}\right]\text{where}\sigma _0\stackrel{def}{=}\left(\frac{e}{2\pi }\right)^{1/2}\frac{1}{m},$$
(40)
and the corresponding Fisher-Rao measure, from eq.(4) is

$$g(\mu ,\sigma )=det\left|\begin{array}{cc}1/\sigma ^2& 0\\ 0& 2/\sigma ^2\end{array}\right|=\frac{2}{\sigma ^4}.$$
(41)
Note that both $`S(\mu ,\sigma )`$ and $`g(\mu ,\sigma )`$ are independent of $`\mu `$. This means that if we were concerned with the simpler problem of estimating $`\mu `$ in a situation where $`\sigma `$ happens to be known, then the entropic prior, in any of the versions eq.(8), (20), or (26), is a constant independent of $`\mu `$. In other words, when $`\sigma `$ is known, the Bayesian estimate of $`\mu `$ using entropic priors coincides with the maximum likelihood estimate, *i.e.*, by the popular procedure of minimizing

$$\chi ^2=\frac{1}{\mu }\underset{i=1}{\overset{N}{}}\left(y_i\mu \right)^2.$$
(42)
Returning to the more interesting case of unknown $`\sigma `$, the $`\alpha `$-dependent entropic prior, eq.(20) is

$$\pi (\mu ,\sigma |\alpha )=\frac{2^{1/2}}{\zeta (\alpha )}\frac{\sigma ^{\alpha 2}}{\sigma _0^\alpha }.$$
(43)
$`\pi (\mu ,\sigma |\alpha )`$ is improper in both $`\mu `$ and $`\sigma `$; normalization requires the introduction of high and low cutoffs for both $`\mu `$ and $`\sigma `$. The fact that without cutoffs the model is not well defined is an indication that more relevant information is being requested: the cutoffs constitute relevant information that must be taken into account. (The logic parallels that which led to the introduction of $`\alpha `$ in section 5.) The case of unknown cutoff values is important and we intend to explore it in detail in future work. The basic idea is that specifying cutoffs is an integral part of defining the model, and therefore the choice of cutoffs can be tackled as a problem of model selection. In the remainder of this section, however, we will assume that the information about cutoffs is already available.

It is convenient to write the range of $`\mu `$ as $`\mathrm{\Delta }\mu =\mu _H\mu _L`$ and to define the $`\sigma `$ cutoffs in terms of dimensionless quantities $`\epsilon _L`$ and $`\epsilon _H`$; $`\sigma `$ extends from $`\sigma _L=\sigma _0\epsilon _L`$ to $`\sigma _H=\sigma _0/\epsilon _H`$. Then $`\zeta (\alpha )`$ and $`\pi (\mu ,\sigma |\alpha )`$ are given by

$$\zeta (\alpha )=\frac{2^{1/2}\mathrm{\Delta }\mu }{\sigma _0}\frac{\epsilon _H^{1\alpha }\epsilon _L^{\alpha 1}}{\alpha 1}.$$
(44)
and

$$\pi (\mu ,\sigma |\alpha )=\frac{1}{\mathrm{\Delta }\mu \sigma _0}\frac{\alpha 1}{\epsilon _H^{1\alpha }\epsilon _L^{\alpha 1}}\left(\frac{\sigma }{\sigma _0}\right)^{\alpha 2}.$$
(45)
Notice that in the special case of $`\alpha =1`$, the prior over $`\sigma `$ reduces to $`d\sigma /\sigma `$ which is called the Jeffreys prior and is usually introduced by the requirement of invariance under scale transformations, $`\sigma \lambda \sigma `$.

Writing $`\epsilon \stackrel{def}{=}(\epsilon _L\epsilon _H)^{1/2}`$, the prior for $`\alpha `$ can be obtained from eq.(32),

$$\gamma (\alpha )=\frac{1}{(\alpha 1)^2}\left(\frac{2\mathrm{log}\epsilon }{\epsilon ^{1\alpha }\epsilon ^{\alpha 1}}\right)^2$$
(46)
and from eqs.(26) and (35),

$$\pi (\alpha )=\frac{\gamma ^{1/2}(\alpha )}{z}\frac{\epsilon ^{1\alpha }\epsilon ^{\alpha 1}}{\alpha 1}\mathrm{exp}\left[\frac{1}{\alpha 1}+\alpha \frac{\epsilon ^{1\alpha }+\epsilon ^{\alpha 1}}{\epsilon ^{1\alpha }\epsilon ^{\alpha 1}}\mathrm{log}\epsilon \right],$$
(47)
where the normalization $`z`$ has been suitably redefined.

Eqs.(46) and (47) simplify considerably when we take the limit $`\epsilon 0`$. Clearly the same result is obtained whether we let $`\epsilon _H0`$ while keeping $`\epsilon _L`$ fixed, or letting $`\epsilon _L0`$ while keeping $`\epsilon _H`$ fixed, or even allowing $`\epsilon _H0`$ and $`\epsilon _L0`$ simultaneously. The resulting $`\gamma (\alpha )`$ and $`\pi (\alpha )`$ are

$$\gamma (\alpha )=\frac{1}{(\alpha 1)^2},$$
(48)
and

$$\pi (\alpha )=\{\begin{array}{cc}\frac{1}{\left(1\alpha \right)^2}\mathrm{exp}\left[\frac{1}{\alpha 1}\right]& \text{for }\alpha <1\\ 0& \text{for }\alpha 1\end{array}$$
(49)
where $`\pi (\alpha )`$ is normalized. This is shown in Fig. 1.

$`\pi (\alpha )`$ reaches its maximum value at $`\alpha =1/2`$. Since $`\pi (\alpha )\alpha ^2`$ for $`\alpha \mathrm{}`$ the expected value of $`\alpha `$ and all higher moments diverge. This suggests that replacing the unknown $`\alpha `$ in the prior $`\pi (\theta |\alpha )`$ by any given numerical value $`\widehat{\alpha }`$ is probably not a good approximation.

As explained in section 5, since $`\alpha `$ is unknown, the effective prior for $`\theta =(\mu ,\sigma )`$ is obtained marginalizing $`\pi (\mu ,\sigma ,\alpha )=\pi (\mu ,\sigma |\alpha )\pi (\alpha )`$ over $`\alpha `$, eq.(26). Since $`\pi (\alpha )=0`$ for $`\alpha 1`$ as $`\epsilon 0`$ we can safely take the limit $`\epsilon _H0`$ or $`\sigma _H\mathrm{}`$. Conversely, since $`\pi (\alpha )0`$ for $`\alpha <1`$ we cannot take $`\epsilon _L0`$ or $`\sigma _L0`$. The limit $`\sigma _H\mathrm{}`$ while keeping $`\sigma _L`$ fixed gives,

$$\pi (\mu ,\sigma ,\alpha )=\{\begin{array}{cc}\frac{1}{\mathrm{\Delta }\mu \sigma _L}\frac{\mathrm{exp}\left[\frac{1}{\alpha 1}\right]}{1\alpha }\left(\frac{\sigma }{\sigma _L}\right)^{\alpha 2}& \text{for }\alpha <1\\ 0& \text{for }\alpha 1.\end{array}$$
(50)
The averaged prior for $`\mu `$ and $`\sigma `$ is

$$\overline{\pi }(\mu ,\sigma )=\frac{1}{\mathrm{\Delta }\mu \sigma _L}\left(\frac{\sigma _L}{\sigma }\right)^2_{\mathrm{}}^1\frac{\mathrm{exp}\left[\frac{1}{\alpha 1}\right]}{1\alpha }\left(\frac{\sigma }{\sigma _L}\right)^\alpha 𝑑\alpha ,$$
(51)
which integrates to

$$\overline{\pi }(\mu ,\sigma )=\frac{2}{\mathrm{\Delta }\mu \sigma }K_0\left(2\sqrt{\mathrm{log}\frac{\sigma }{\sigma _L}}\right),$$
(52)
where $`K_0`$ is a modified Bessel function of the second kind. This is the entropic prior for the Gaussian model. The function

$$P(x)=\frac{2}{x}K_0\left(2\sqrt{\mathrm{log}x}\right)$$
(53)
is shown in Fig. 2 as a function of $`x=\sigma /\sigma _L`$.

$`P(x)`$ has an integrable singularity as $`x1`$ where it behaves as

$$P(x)\frac{2}{x}\left(\mathrm{log}\sqrt{\mathrm{log}x}\gamma \right)\text{for}x1.$$
(54)
Since $`\sigma _L`$ is a lower cutoff the region of large $`x`$ is more relevant. The leading asymptotic behavior is given by

$$P(x)\frac{\sqrt{\pi }}{x\left(\mathrm{log}x\right)^{1/4}}\mathrm{exp}\left(2\sqrt{\mathrm{log}x}\right)\text{for}x1.$$
(55)
Finally, we turn to Bayes’ theorem, eq.(25), with the prior (52) to obtain estimators for $`\mu `$ and $`\sigma `$. For large $`N`$ the results are independent of the prior and the estimators coincide with the standard maximum likelihood results. The case when $`N`$ is not so large is the more interesting one. As estimators we can take the expected values $`\mu `$ and $`\sigma ^2`$ over the posterior (25). The integrations can be performed numerically and are not particularly illuminating. Alternatively, one can follow standard practice and marginalize eq.(25) over $`\sigma `$ to obtain the distribution $`p(\mu |y^N)`$ and calculate the estimator $`\widehat{\mu }`$ from

$$\frac{d}{d\mu }\mathrm{log}p(\mu |y^N)|_{\widehat{\mu }}=0,$$
(56)
and its error bar $`\widehat{\sigma }`$ from

$$\frac{d^2}{d\mu ^2}\mathrm{log}p(\mu |y^N)|_{\widehat{\mu }}=\frac{1}{\widehat{\sigma }^2}.$$
(57)
When $`p(\mu |y^N)`$ happens to be a Gaussian these estimators coincide with the expected values $`\mu `$ and $`\sigma ^2`$. The final result for $`\widehat{\mu }`$ is very simple. For any value of $`N`$ we have

$$\widehat{\mu }=\frac{1}{N}\underset{i}{}y_i=\overline{y},$$
(58)
the estimator $`\widehat{\mu }`$ is the sample average. The result for $`\widehat{\sigma }`$ is not as elegant but, of course, for large $`N`$ it asymptotically reduces to $`\widehat{\sigma }^2(\overline{y^2}\overline{y}^2)/N.`$

## 8 Final remarks

In this paper the method of maximum relative entropy has been used to translate the information contained in the known form of the likelihood into a prior distribution for Bayesian inference. The argument follows closely the analogous ME methods that have been so successful in statistical mechanics. For experiments that cannot be repeated the resulting “entropic prior” is formally identical with the Einstein fluctuation formula. For repeatable experiments, however, the expected value of the entropy of the likelihood – represented in terms of a Lagrange multiplier $`\alpha `$ – turns out to be relevant information that must be included in the analysis. As an illustration the important case of a Gaussian likelihood was treated in detail.

It may be useful to comment briefly on the differences between our entropic prior and the versions previously proposed by Skilling and by Rodríguez. Perhaps the main difference with Skilling’s prior is that, unlike ours, its use is not restricted to probability distributions but is intended for generic “positive additive distributions” including, for example, the distributions of intensities in images . One problem here is that of justifying the applicability of the ME method in such a general context. Our impulse to generalize is a dangerous one; we may get away with indulging it occasionally but overindulgence will certainly lead to error. In any case, our argument in section 3, which consists in maximizing the entropy $`\sigma `$ subject to a constraint $`p(y,\theta )=\pi (\theta )p(y|\theta )`$, makes no sense in the case of generic positive additive distributions for which there is no available product rule. A more specific problem arises from the fact that Skilling’s entropy is not, in general, dimensionless and the hyper-parameter $`\alpha `$ is vaguely interpreted some sort of cutoff carrying the appropriate corrective units. Some of the difficulties, which led Skilling to seek an alternative approach, were identified in .

Rodríguez’s approach is closer to ours. His prior applies to probability distributions and appears to be derived from a ME principle . One difference, perhaps a minor one, is his treatment of the underlying measure $`m(y)`$. For us $`m(y)`$ is not arbitrary; knowing $`m(y)`$ is part of knowing what data has been collected. For him $`m(y)`$ is just an initial guess and he suggests setting $`m(y)=p(y|\theta _0)`$ for some value $`\theta _0`$. The more important difference, however, is that the number of observed data $`n`$ is deliberately and explicitly left unspecified. The space $`\mathrm{\Theta }\times Y^n`$ over which distributions are defined, and therefore the distributions themselves, also remain unspecified. It is not clear what the maximization of an entropy over such unspecified spaces could possibly mean but a hyper-parameter $`\alpha `$ is eventually introduced and it is interpreted as a “virtual number of observations supporting the initial guess $`\theta _0`$.” He proposes that $`\alpha `$ be considered as one more among the parameters $`\theta `$ to be inferred. As mentioned earlier this leads to the introduction of an endless chain of additional hyper-parameters.

There are several directions in which the ideas of this paper can be further extended. First, we emphasize once again that the entropic priors discussed here apply to a situation where all we know about the quantities $`\theta `$ is that they appear as parameters in the likelihood $`p(y|\theta )`$, *and nothing else*. In many situations of experimental interest there exists additional relevant information beyond what is contained in the likelihood. Such information should be included as additional constraints in the maximization of the relative entropy $`\sigma `$ in eq.(17). The resulting modified entropic prior would provide a better representation of our state of knowledge prior to the acquisition of the data. Indeed, the advantage of the Bayesian approach over the usual method of maximum likelihood is the possibility of including additional relevant information by replacing a flat prior by an appropriately more informative prior. There is nothing to prevent us from performing a similar improvement and going beyond the “bare” entropic priors discussed in this paper. Two kinds of additional information that are easy to include are restrictions on the range of the parameters $`\theta `$ and information about the known expected values of some variables $`a(\theta )`$. Steps in this direction were taken in section 5, where $`a(\theta )`$ is the likelihood entropy, and in section 7 where high and low cutoffs on the range of the Gaussian parameters were introduced.

Second, in the introduction we mentioned the interesting possibility of analyzing data $`y_e`$ from different experiments, $`e=1,2,\mathrm{}`$, related to $`\theta `$ by different likelihood functions $`p_e(y_e|\theta )`$. Clearly this can be analyzed as a single combined experiment with likelihood $`p(y_1,y_2,\mathrm{}|\theta )=p_1(y_1|\theta )p_2(y_2|\theta )\mathrm{}`$ to which all our previous results apply. As we stated earlier, the mere fact that $`\theta `$ is measurable through one or another experiment is additional relevant information that can be taken into account.

Third, we also mentioned that problems of model selection can be tackled as an extension of the ideas described in this paper. On the basis of data $`y`$ we want to select one model among several competing candidates labeled by $`m=1,2,\mathrm{}`$ with likelihood distributions given by $`p(y|m,\theta _m)`$. The answer, *i.e.*, the probability of model $`m`$ given the data $`y`$, is given by Bayes’ theorem,

$`p(m|y)`$ $`={\displaystyle \frac{\pi (m)}{p(y)}}p(y|m)={\displaystyle \frac{\pi (m)}{p(y)}}{\displaystyle 𝑑\theta _mp(y,\theta _m|m)}`$
$`={\displaystyle \frac{1}{p(y)}}{\displaystyle 𝑑\theta _m\pi (m,\theta _m)p(y|m,\theta _m)}.`$ (59)
This is exact. The problem is solved, at least in principle, once an entropic prior for $`\pi (m,\theta _m)`$ is assigned. However, the remaining practical problems associated with carrying out the actual numerical calculations could, of course, still be quite formidable.

Finally, we end with a word of caution. As in all instances of inductive inference there is the possibility that predictions based on the ME method could be wrong because not all the information relevant to the problem at hand was taken into account. This potential problem is not peculiar to the ME method, it is a problem shared by all methods of induction. Nevertheless, we are confident that the rewards of extending the benefits of an inductive method singled out by requirements of objectivity, the ME method, beyond its traditional territory of statistical mechanics and into that of data analysis will be enormous.

Acknowledgments- Many of our comments and arguments have been inspired by Carlos C. Rodríguez, Volker Dose, and Rainer Fisher through insightful questions and discussions which we gratefully acknowledge. A. C. also acknowledges the hospitality of the Max-Planck-Institut für Plasmaphysik during the two extended visits when most of this work was carried out.


text:
# Cross-sectional STM of Mn-doped GaAs: theory and experiment

## I Introduction

GaAs can be doped with Mn to form a dilute magnetic semiconductor with a Curie temperature as high as 160 K. chiba03 ; ku03 ; edmonds02  It is generally accepted that holes created by the substitution of Mn for Ga mediate the ferromagnetic interaction between Mn dopants in this material.dietl00 ; dietl01  Naively, each substitutional Mn is expected to produce one hole; therefore the nominal hole concentration, p, should be equal to the Mn concentration. For typical Mn fractions of 5%, this implies a hole concentration $`p=1.1\times 10^{21}`$ cm-3. Measured hole concentrations of as-grown material are much less than this, typically by a factor $``$3. The source of this compensation remains somewhat controversial, having been variously attributed to either excess As in the form of antisites and interstitialssanvito02 ; bergqvist03 ; hayashi01 ; potashnik01  or Mn interstitials. masek01 ; yu02 ; erwin02 ; mahieu03  Recent experiments show that careful annealing near the growth temperature can significantly enhance the conductivity and hole concentration.yu02 ; edmonds02  This, as well as recent ion channelling experiments,yu02  suggests that interstitial Mn is the more likely source. A complete picture is lacking, however; for example, the distribution of Mn interstitials in as-grown material is still unclear.

Cross-sectional scanning tunneling microscopy (XSTM) is an effective tool for addressing this issue because it can image, with atomic resolution, the structural and electronic configuration of impurities and defects as they are present in the bulk. There are several recent XSTM studies of Mn-doped GaAs;grandidier00 ; mahieu03 ; tsuruoka01 ; tsuruoka02  these studies used the correlation between transport data, level of Mn doping, and observed defect density to infer the location (substitutional versus interstitial) of Mn dopants and As antisites. The relative abundance of As antisites and interstitial Mn deduced by these studies lacks a clear consensus: one study finds that antisites are not present at all,tsuruoka01  while another finds that both antisites and interstitial Mn are present, and compensate substitutional Mn.mahieu03

To investigate the nature of dopants and defects in this material, we performed high resolution XSTM measurements on a (110) cleavage plane of Mn-doped GaAs and used first-principles calculations to interpret the images. Specifically, we used density-functional theory to simulate STM images for a number of Mn configurations near the GaAs (110) surface. Configurations were chosen based on their calculated energetic stability, and included several metastable configurations that would probably be kinetically stabilized. A detailed comparison of the resulting simulated and experimental images leads us to the following conclusions: (1) isolated substitutional Mn as well as pairs of substitutional Mn occur with roughly comparable frequency in bulk Mn-doped GaAs, and (2) interstitial Mn is typically bound in complexes with substitutional Mn in several different configurations.

## II Experimental XSTM Images

The Mn-doped GaAs sample used in this study was grown by molecular-beam epitaxy (MBE) using well established methods and conditions.ohno96  A buffer layer of high quality undoped GaAs was first grown on an n-type GaAs substrate. This was followed by growth of a 100 nm thick GaAs buffer layer grown at 250 C and a 260 nm thick layer of Mn-doped GaAs at the same temperature. Growth quality was monitored by reflection high-energy electron diffraction. The Mn concentration was determined with x-ray diffraction to be 0.6%.

This sample was then characterized by XSTM measurements in a multichamber ultrahigh-vacuum facility. It order to obtain an atomically abrupt cleave across the epilayer, the samples were thinned ex situ to $``$200 $`\mu `$m. After being loaded into the XSTM chamber (base pressure $``$10-10 Torr), the samples were scribed and cleaved in situ to expose a (110) surface (perpendicular to the MBE growth direction), as previously described.nosho00  The constant-current (40 pA) images shown here are of filled electronic states measured with a sample bias of $``$2.5 V.

Figure 1 shows the results of filled-state XSTM measurements on this sample. The scan area of this image is 170 $`\times `$ 170 Å2 and displays $``$1500 surface atoms. Filled state STM images of GaAs (110) reveal the As surface sublattice; the surface Ga sublattice evident in empty-state images, on the other hand, is not directly revealed in this image. Since only the top As atoms contribute, STM of the GaAs (110) surface reveals only every other layer in the (001) direction.

Figure 1 reveals many surface features not normally seen in STM of the GaAs (110) surface. Several occur with enough regularity to warrant close attention. Feature A is an elliptical region of intensity in an As (001) plane with major and minor axes the size of two and one As surface atoms, respectively. Feature B is circular, and only slightly larger and more intense than the contributions from the surrounding As. Feature C is more diffuse and broad than both A and B, with intensity spread over two pairs of As atoms in neighboring (001) planes. As discussed in more detail below, this is what one would expect if there were two A features in neighboring As (001) planes. Feature D is of similar shape to that of C but is more asymmetric, with one side of the defect the size of a single surface As atom. There does not appear to be a preferred crystalline orientation of feature D as different orientations are visible in Fig. 1. Feature E is a dark region close to a surface As atom displaced in the (00$`\overline{1}`$) direction; there is also an overall apparent depression in the vicinity of this feature, characteristic of band bending that occurs near positively charged defects on a p-type surface.ebert99

On the basis of the experimental data alone it is extremely difficult, if not impossible, to determine what dopants or defects produce these features. As we show in this work, however, when interpreted with first-principles theory, these measurements resolve both the type and crystalline orientation of Mn dopants in GaAs.

## III Theoretical Approach

### III.1 Physical and Electronic Structure

We modeled the GaAs (110) surface in a supercell slab geometry consisting of 5 layers of GaAs in a 4$`\times `$4 surface unit cell. A vacuum region of 13.4 Å was used for all the calculations. In such a cell the separation between a dopant and its periodic image is $``$16 Å, sufficiently large to ensure that dopants interact only negligibly with their periodic images. The bottom layer of the slab was passivated; the atomic positions were fully relaxed in all but the bottom two layers.

The wavefunctions and charge density were expanded in a plane-wave basis and evaluated using an ultrasoft pseudopotential approachvanderbilt90 ; kresse94  as implemented in the VASP code.kresse96a ; kresse93 ; kresse96  Electron correlations were treated at the level of the local spin-density approximation (LSDA) with total energies converged to 10-4 eV.

A single wave vector was used to sample the Brilloun zone of the slab supercells and a plane-wave cutoff of 227.24 eV was used for all the calculations. For supercells containing Mn, atoms within $``$7.0 Å of the Mn site(s) were fully relaxed until the total energy between two structural configurations changed by less than 10-3 eV.

### III.2 Simulated STM Images

Theoretical STM images were simulated using the method of Tersoff and Hamann.tersoff83  In this approach, the central quantity is the the local density of states (LDOS) near the surface. The LDOS is integrated over an energy range determined by the experimental bias voltage; contours of constant energy-integrated LDOS simulate a constant-current image. For filled state imaging, the upper bound of this energy range is the Fermi level. The Mn-doped GaAs sample under consideration in this work is p-type with an estimated hole density of 1019cm-3, so that the Fermi level should be near the valence band maximum (VBM). Since we do not know the distribution of donors and acceptors present in the sample, this information alone is not sufficient to determine the exact location of the Fermi level for our sample. However, we have found that, as a consequence of the large bias, the simulated images are rather insensitive to the value of the Fermi level used in the LDOS integration, and so we choose arbitrarily a point 0.4 eV above the VBM as the upper bound for all the STM simulations. For this choice, the Fermi level is above the acceptor level of substitutional Mn and below the donor levels of interstitial Mn. Simulations performed with the position of the Fermi level shifted by $`\pm `$0.3 eV give similar results. The location of the VBM for each type of dopant was determined by inspection of the total DOS relative to that projected onto the dopant site.

As a measure of the agreement between simulated and measured STM images, we consider mainly the overall shape and spatial extent of features of interest. The defect-free (110) surface has a ($`\overline{1}`$10) mirror plane symmetry, so it is also useful to note whether a particular dopant configuration preserves, even approximately, this symmetry. Finally, we use qualitative arguments to judge whether the number of observed features are consistent with our total-energy calculations.

## IV Substitutional Mn

### IV.1 Simple Substitutional Dopants

Substitutional Mn dopants are of primary importance for the magnetic and transport properties of Mn-doped GaAs, and hence we discuss the simulated STM images for these types of dopants first. Since STM is a near-surface sensitive probe, we have only considered substitutional Mn dopants in the top three layers near the (110) surface, as shown in Fig. 2(a).

We will use the notation $`s(n)`$ to denote a substitutional Mn in layer $`n`$ (with the topmost surface layer defined as $`n=1`$). Figures 3(a), (b), and (c) show the simulated images from $`s(1)`$, $`s(2)`$, and $`s(3)`$, respectively. The $`s(1)`$ and $`s(3)`$ images are strikingly similar, for the following reasons. The $`s(1)`$ Mn atom forms bonds with two As atoms that are nominally in the same plane; likewise for the $`s(3)`$ Mn atom. Thus the overall shape of the dopant consists of three overlapping maxima, centered on the two As atoms and on the Mn atom, giving rise to an elongated feature with a ($`\overline{1}`$10) mirror-plane symmetry. Hence, both of these sites are consistent with feature A in Fig. 1.

The $`s(2)`$ dopant on the other hand, has a very different image. In the $`s(2)`$ location the Mn forms a bond with an As atom in the top layer, with the bond pointing toward the surface. The perturbation due to $`s(2)`$ is centered on the surface As site involved in the bond. This overlaps with the contribution from the Mn dopant itself, and thus the shape is more circular compared to the $`s(1)`$ and $`s(3)`$ dopants. It is also larger in size than surface As atom contributions and has a ($`\overline{1}`$10) mirror-plane symmetry. These agree qualitatively with the characteristics of feature B in Fig. 1, leading us to identify those features as arising from $`s(2)`$ dopants.

### IV.2 Substitutional Mn Pairs

Since there will always be some fraction of neighboring Mn that occur simply by chance, we consider pairs of substitutional Mn as well. Figures 3(d), (e) and (f) show simulated images of pairs of Mn substitutionals $`s(1)s(1)`$, $`s(3)s(3)`$ and $`s(3)s(1)`$, respectively. For the $`s(3)s(1)`$ pair, we note that even though the two Mn involved are $``$6.8 Å apart, and in different atomic planes, because of the projected view of the STM scan the appearance of this pair is quite similar to those of the $`s(1)s(1)`$ and $`s(3)s(3)`$ pairs. All three of these simulated images have a shape comparable to that of feature C in Fig. 1. Common features include a region of low intensity between the two dopants, a ($`\overline{1}`$10) mirror-plane symmetry (the experimental data appear to only approximately have this symmetry) and a similar spatial extent involving four surface As atoms. It is also interesting to note that these pairs have simulated images that are essentially a linear superposition of the images of the isolated substitutionals shown in Figs. 3(a) and (c). This suggests that, at the level of detail that STM reveals, two substitutional Mn do not interact very strongly when they are next-nearest neighbors.

Figures 3(g) and (h) show the simulated images for nearest-neighbor substitutional pairs $`s(1)s(2)`$ and $`s(3)s(2)`$, respectively. Overall, the simulated images of these pairs are similar to what one would obtain from a linear superposition of the images of the isolated defects. However, the simulated image for $`s(1)s(2)`$ clearly reveals that the contribution from the $`s(1)`$ dopant is now asymmetric, lacking a ($`\overline{1}`$10) mirror plane. This indicates that substitutional pairs interact more strongly when they are in near-neighbor configuration than in the next-nearest-neighbor configuration described in the paragraph above. The shape of these simulated images is generally consistent with that of feature D in Fig. 1, with a characteristic “L” shape involving three As atoms, two of which are in the same As (001) plane. We have not found any other dopants that yield such a favorable comparison to the experimental data, leading us to identify feature D as $`s(1)s(2)`$ and $`s(3)s(2)`$ pairs. In the experimental data one can observe three of the four possible crystal orientations of these pairs of substitutionals.

An estimate of the number of neighboring pairs of Mn substitutionals based on a random distribution of Mn on the Ga sublattice is much lower than the number of compound features, in particular those labeled “C”, seen in Fig. 1. Even if there is some positional correlation among the substitutional dopants, for example, the strong short-range attraction between Mn found in a recent theoretical study,vanschilfgaarde01  it is unlikely that a description based entirely on substitutionals and clusters of substitutionals can describe the frequency and shapes of all the features in Fig. 1. This suggests that other Mn complexes, in particular those involving interstitial Mn, may help explain the remaining features.

## V Dopant Complexes

### V.1 Total Energy Considerations

It has recently been shown that although the formation energy of interstitial Mn is much higher than that of substitutional Mn in bulk GaAs, the presence of the (001) growth surface significantly enhances the likelihood of Mn occupying interstitial sites.erwin02  Thus one expects that both substitutional and interstitial Mn will be present in typical Mn-doped GaAs samples, as shown by recent measurements of the Mn distribution in Mn-doped GaAs.yu02

In the bulk, the barrier for diffusion of a charged ($`+`$2e) interstitial Mn is $``$0.5 eV; therefore at room temperature we expect interstitial Mn to rapidly diffuse throughout the bulk material unless it becomes kinetically trapped. Since substitutional dopants are acceptors, they are a natural trap for interstitials, electrostatically binding to them in a number of different physical configurations. Clustering phenomena for compensating dopants such as these have recently been predicted using Monte Carlo techniques.timm02  On the other hand, since interstitial Mn in bulk is a double donor with charge of $`+`$2e, under conditions of filled-state imaging (for which the tip is positively biased relative to the sample) free interstitial Mn will be repelled away from the surface, leaving behind only those interstitials that are bound in complexes. To determine which such interstitial-substitutional complexes are most likely to be present, we turn to total-energy calculations.

To simplify the discussion, we extend the notation introduced earlier to now include Mn interstitials, and complexes of interstitial and substitutional Mn. Isolated interstitial Mn in layer $`n`$, coordinated to As and Ga, will be referred to as $`i^{\mathrm{As}}(n)`$ and $`i^{\mathrm{Ga}}(n)`$, respectively. For reference, the configuration of isolated interstitial dopants in the first two layers is shown in Fig. 2(b). Complexes composed of interstitial and substitutional Mn will be referred to with a composite notation that takes into account the crystalline orientation of the atoms involved. For example, since the GaAs (110) surface does not have a (001) mirror-plane symmetry, there are two distinct ways of forming a complex of interstitial and substitutional Mn. In Fig. 2 the surface Ga atoms are to the right of the surface As atoms to which they are bonded, so that we may consider complexes where the interstitial Mn is to the left or right of the neighboring substitutional dopant; such complexes will be denoted as $`i^{\mathrm{As}}(n)s(m)`$ and $`s(m)i^{\mathrm{As}}(n)`$, respectively. Figure 2(c), (d), and (e) shows the configuration of near-surface complexes we have considered in this work. Thus, for example, Fig. 2(c) shows the complexes $`s(2)i^{\mathrm{Ga}}(2)`$ and $`s(2)i^{\mathrm{As}}(2)`$, which we discuss below.

Figure 4 shows the calculated total energies of Mn complexes relative to that of the $`s(2)i^{\mathrm{As}}(2)`$ dopant, which we identify as the lowest energy configuration. There is a large variation in the energies, up to $``$1.3 eV above the energy of $`s(2)i^{\mathrm{As}}(2)`$. We also note that for complexes involving only Mn atoms on the surface (Figs. 4(a) and (c)) the lowest energy configuration occurs when the interstitial site is nearest the substitutional site. Relative to these nearest-neighbor complexes, when interstitial Mn is displaced by lateral translations in the ($`\overline{1}`$10) direction, the energy increases dramatically; the next lowest energy configurations are $``$0.3 eV higher in energy. Thus, diffusion of surface interstitial Mn from high-energy locations (denoted with  and ∗∗) to low-energy configurations is likely to form nearest-neighbor complexes on the surface.

It is difficult to estimate, on energetic grounds alone, the expected number of such complexes. The presence of both the surface and the STM tip create additional complications. For example, the energy of interstitial Mn depends strongly on its proximity to the GaAs (110) surface; an interstitial Mn in the interior of GaAs far from the surface can lower its energy as much as $``$1 eV by moving to a location just below the surface.sullivan03  This suggests that interstitial Mn will diffuse from the bulk-like region toward the surface; during the STM scans these “excess” interstitials will be observed only if they are bound in complexes. Such phenomena are difficult to quantify and beyond the scope of this work. Instead we choose simply to report simulated STM images of several low-energy complexes within the shaded regions of Fig. 4.

### V.2 Simulated STM Images

#### V.2.1 Substitutional-interstitial complexes

Figure 5 shows the simulated images for the low-energy configurations of substitutional-interstitial complexes. The simulated image of the lowest energy substitutional-interstitial surface complex, $`s(1)i^{\mathrm{Ga}}(1)`$, is shown in Fig. 5(a). There is a reduction in intensity of the As surface atom to the right of the $`s(1)`$ site and in the region between the As (001) planes to the left of the $`s(1)`$ site. The simulated images of $`s(1)i^{\mathrm{Ga}}(1)^{}`$ \[Fig. 5(c)\], $`s(2)i^{\mathrm{As}}(2)`$ \[Fig. 5(d)\], and $`s(1)i^{\mathrm{As}}(2)`$ \[Fig. 5(f)\] are all similar, but the reduction in the simulated intensity between the As (001) planes to the left of the $`s(1)`$ site is less apparent. Among these, only the $`s(2)i^{\mathrm{As}}(2)`$ complex has a ($`\overline{1}`$10) mirror-plane symmetry. All four of these simulated images are generally consistent with the characteristics of feature E in the experimental data of Fig. 1. However, since the $`s(2)i^{\mathrm{As}}(2)`$ complex is much lower in energy ($``$0.5 eV) than the other complexes just described, and since there appears to be only two such features in Fig. 1, we suggest that feature E is most likely a $`s(2)i^{\mathrm{As}}(2)`$ complex. If this is correct, then the observed apparent displacement of the surface As atom results mainly from an electronic effect rather than a real atomic displacement.

The simulated image of the $`s(1)i^{\mathrm{As}}(1)`$ complex shown in Fig. 5(b) is rather different than those for the complexes just described. This simulated image obviously has a ($`\overline{1}`$10) mirror-plane symmetry and a “butterfly” shape with a region of lower intensity between the Mn atoms. Since there is no (001) mirror plane, the left and right wings of the butterfly are clearly different, with the contribution from the $`s(1)`$ Mn more intense than that of the $`i^{\mathrm{As}}`$(1). Overall, these characteristics are similar to those of feature C, which appears to involve four As surface atoms in neighboring (001) planes. As we discuss below, there are other energetically competitive complexes with similar simulated images, so that we postpone any conclusions regarding feature C until all the relevant complexes have been discussed.

The simulated image of the $`s(2)i^{\mathrm{Ga}}(2)`$ complex shown in Fig. 5(e) does not have a ($`\overline{1}`$10) mirror plane symmetry. In this configuration there is an oblate region of intensity along the As (001) plane and a reduction in the surface As atom intensity nearest the interstitial site. On the basis of the available STM data, we cannot rule out the possibility that some of those features labeled B in Fig. 1 result from $`s(2)i^{\mathrm{Ga}}(2)`$ complexes.

#### V.2.2 Interstitial-substitutional complexes

Figure 6 shows the simulated images for the low-energy configurations of interstitial-substitutional complexes. The simulated images of the $`i^{\mathrm{As}}(1)s(1)`$, $`i^{\mathrm{As}}(1)s(1)^{}`$, and $`i^{\mathrm{Ga}}(2)s(1)`$ complexes are all very similar in shape, with an increase in intensity on the four nearest surface As atoms and between the As (001) planes to the left of the $`s(1)`$ site for the first two and to the right of the $`s(1)`$ site for the latter. Obviously, the $`i^{\mathrm{As}}(1)s(1)`$ and $`i^{\mathrm{Ga}}(2)s(1)`$ complexes have a ($`\overline{1}`$10) mirror-plane symmetry, whereas the $`i^{\mathrm{As}}(1)s(1)^{}`$ appears to approximately have this symmetry. The characteristics of these images are similar to that of the $`s(1)i^{\mathrm{As}}(1)`$ complex in Fig. 5(b), but with more intensity between the relevant As (001) planes as a result of the interstitial location.

The simulated image of the $`i^{\mathrm{Ga}}(1)s(1)`$ complex shown in Fig. 6(c) has no apparent symmetry. There is an increase in intensity between the two surface As atoms nearest the $`i^{\mathrm{Ga}}(1)`$ site, whereas the $`s(1)`$ Mn is only very weakly visible. Features such as this are not apparent in the experimental data, although XSTM scans of lower doped samples may help clarify whether they are present.

From our STM data and the theoretical simulations, we cannot unambiguously assign feature C to one particular complex among the plausible alternatives: $`s(1)i^{\mathrm{As}}(1)`$, $`i^{\mathrm{As}}(1)s(1)`$, $`i^{\mathrm{As}}(1)s(1)^{}`$, $`i^{\mathrm{Ga}}(2)s(1)`$, $`s(1)s(1)`$, $`s(3)s(3)`$ and $`s(1)s(3)`$. Considering the large number of such features in Fig. 1 and the comparable energies of the different complexes, it is likely that more than one of these dopants are present in the scan area of Fig. 1. Measurements on samples with much smaller Mn concentration are ongoing and should be able to address these issues in more detail.

## VI Summary

In summary, we have combined first-principles calculations and high-resolution XSTM measurements to characterize Mn-doped GaAs. XSTM on a (110) cleavage plane reveals five reproducible features not found in bulk GaAs. Total-energy calculations were used to screen the most likely configuration of Mn dopants near the GaAs (110) surface. The low-energy configurations were then further scrutinized by comparison of simulated and measured STM images. These comparisons reveal that there are predominantly two types of Mn-related dopants in the sample: (1) those involving only Mn substitutional(s), including both isolated and pairs of substitutionals, and (2) complexes composed of substitutional and interstitial Mn.


text:
# Decaying Λ cosmology with varying G

## I Introduction

The advent of precise observational cosmology and the recent discovery of a small cosmological constant $`\mathrm{\Lambda }`$ have reinforced two theoretical problems that have remained open throughout the years GRF . The first is the absolute value of $`\mathrm{\Lambda }`$, $`122`$ orders of magnitude smaller than the value expected for the Planck time. The second is why the vacuum energy density has, at the present time, the same order as the matter density. This second problem is usually known as the cosmic coincidence problem.

A possible explanation for the small value of $`\mathrm{\Lambda }`$ can be based on the idea that the vacuum energy density is not constant, but decays as the universe expands Wu -Narlikar . Nevertheless, this does not explain the cosmic coincidence, except by means of a fine tuning of initial conditions.

In the present contribution, we will consider a FLRW, spatially flat, decaying $`\mathrm{\Lambda }`$ cosmology, in the realm of a model where the gravitation “constant” G varies with time at a cosmological scale. We will show that, in the asymptotic limit of late times, our model admits, besides the usual de Sitter solution, three other solutions characterized by a constant ratio between the matter density and the total energy density. Two of them have a decelerating expansion. The third one presents a coasting expansion, with the universe age given by $`Ht=1`$, and a relative matter density given by $`\rho _m/\rho =1/3`$.

## II Decaying $`\mathrm{\Lambda }`$ solutions with constant $`G`$

In the flat case, the Einstein equations are given by

$`\rho =3H^2,`$ (1)
$`\dot{\rho }+3H(\rho +p)=0,`$ (2)
where $`H=\dot{a}/a`$ is the Hubble parameter.

We will consider a twofold energy content, formed by dust matter with energy density $`\rho _m`$, and by a vacuum term with equation of state $`p_\mathrm{\Lambda }=\rho _\mathrm{\Lambda }`$. Then we have, for the total energy and pressure,

$`\rho `$ $`=`$ $`\rho _m+\rho _\mathrm{\Lambda },`$ (3)
$`p`$ $`=`$ $`\rho _\mathrm{\Lambda }.`$ (4)
Substituting (1), (3) and (4) into (2), we obtain

$$2\dot{H}+\rho _m=0.$$
(5)
If we suppose that, for late times, $`H`$ and $`\rho _m`$ falls monotonically with the scale factor $`a`$, we can expand them in negative power series of $`a`$. Taking the dominant terms, we then have

$`H`$ $`=`$ $`\beta /a^k,`$ (6)
$`\rho _m`$ $`=`$ $`\gamma /a^n,`$ (7)
where $`n`$ and $`k`$ are positive integers.

Substituting into (5) leads to

$$\gamma a^n2k\beta ^2a^{2k}=0.$$
(8)
For the above equation to be valid for any (large) value of $`a`$, we must have

$`n`$ $`=`$ $`2k,`$ (9)
$`\gamma `$ $`=`$ $`2\beta ^2k.`$ (10)
Now, it is straightforward to show that

$$\mathrm{\Omega }_m\rho _m/\rho =2k/3.$$
(11)
Since $`\mathrm{\Omega }_m1`$, it follows that $`k=0,1`$. In the case $`k=0`$, we have

$`H`$ $`=`$ $`\beta ,`$ (12)
$`\rho _m`$ $`=`$ $`0.`$ (13)
This solution corresponds to a de Sitter universe, with a constant vacuum energy density.

In the case $`k=1`$, it follows that

$`a`$ $`=`$ $`\beta t,`$ (14)
$`Ht`$ $`=`$ $`1,`$ (15)
$`\rho _m/\rho `$ $`=`$ $`2/3.`$ (16)
As discussed in Alcaniz ; Alcaniz2 , among the solutions of the form $`at^n`$, the best fitting of supernova and radio sources observations is obtained in the coasting case, $`at`$. On the other hand, the relation $`Ht=1`$ gives the best estimation for the universe age. Nevertheless, we know that a relative matter density equal to $`2/3`$ is not consistent with the observed amount of visible and dark matter.

## III Decaying $`\mathrm{\Lambda }`$ solutions with varying $`G`$

In the realm of a varying gravitational “constant”, we will still consider the ansatz

$`\rho ={\displaystyle \frac{3H^2}{8\pi G}},`$ (17)
$`\dot{\rho }+3H(\rho +p)=0,`$ (18)
where we have re-introduced the factor $`8\pi G`$.

The continuity equation (18) does not depend on the varying or constant character of $`G`$, being just an expression of energy conservation. As far as relation (17) is concerned, it is valid today, and we will assume that it is valid for any late time.

As to the variation law for $`G`$, let us take the Eddington-Weinberg empirical relation GS

$$G\frac{H}{m^3}=\frac{H}{8\pi \lambda },$$
(19)
where $`m`$ has the order of the pion mass, and the constant $`\lambda `$ was introduced for convenience. Once again, this relation is valid today, and we will assume that it is valid for any late time.

Substituting (19) into (17), we have

$$\rho =3\lambda H.$$
(20)
For the total energy density and pressure, we will use again

$`\rho `$ $`=`$ $`\rho _m+\rho _\mathrm{\Lambda },`$ (21)
$`p`$ $`=`$ $`\rho _\mathrm{\Lambda }.`$ (22)
Carrying (20-22) into the continuity equation (18), we obtain

$$\lambda \dot{H}+\rho _mH=0.$$
(23)
Let us expand again $`H`$ and $`\rho _m`$ in negative powers of $`a`$, and take the leading terms

$`H`$ $`=`$ $`\beta /a^k,`$ (24)
$`\rho _m`$ $`=`$ $`\gamma /a^n.`$ (25)
Substituting into (23) leads to

$$\gamma a^n\lambda \beta ka^k=0.$$
(26)
Now, the possible solutions are given by

$`k`$ $`=`$ $`n,`$ (27)
$`\gamma `$ $`=`$ $`\lambda \beta n.`$ (28)
It is easy to show that this leads to

$$\rho _m/\rho =n/3,$$
(29)
which means that, now, $`n=0,1,2,3`$.

For $`n=0`$, we have

$`H`$ $`=`$ $`\beta ,`$ (30)
$`\rho _m`$ $`=`$ $`0,`$ (31)
$`G`$ $`=`$ $`{\displaystyle \frac{\beta }{8\pi \lambda }}.`$ (32)
That is, the usual de Sitter solution, with $`\mathrm{\Lambda }`$ and $`G`$ constant.

For the other values of $`n`$, we obtain

$`a`$ $`=`$ $`(n\beta t)^{1/n},`$ (33)
$`Ht`$ $`=`$ $`1/n,`$ (34)
$`\rho _m/\rho `$ $`=`$ $`n/3.`$ (35)
For $`n=2,3`$, we have decelerating solutions, leading to a too young universe and a too high matter density. For $`n=1`$, it follows that

$`a`$ $`=`$ $`\beta t,`$ (36)
$`Ht`$ $`=`$ $`1,`$ (37)
$`\rho _m/\rho `$ $`=`$ $`1/3.`$ (38)
Therefore, we re-obtain a coasting expansion, with an acceptable age for the universe. Moreover, this time the corresponding relative matter density matches surprisingly well the observed value.

## IV Concluding remarks

To conclude, let us obtain the variation rate of $`G`$, and the rate of matter production in this model.

For $`G`$ we have used the evolution law

$$G=\frac{H}{8\pi \lambda },$$
(39)
which leads to the relative variation rate

$$\dot{G}/G=(1+q)H,$$
(40)
where $`q=a\ddot{a}/\dot{a}^2`$ is the deceleration parameter.

For $`n=1`$, we have $`q=0`$, and so

$$\dot{G}/G=H.$$
(41)
For the rate of matter production (coming from the decaying vacuum energy), we have (for $`n=1`$)

$$\frac{1}{a^3}\frac{d}{dt}(\rho _ma^3)=2\rho _mH.$$
(42)
Finally, a note on the variation of the vacuum energy in the present context. It is possible to show that the vacuum energy density and the corresponding cosmological “constant” are given by

$`\rho _\mathrm{\Lambda }`$ $`=`$ $`(3n)\lambda H={\displaystyle \frac{(3n)}{8\pi }}m^3H,`$ (43)
$`\mathrm{\Lambda }`$ $`=`$ $`8\pi G\rho _\mathrm{\Lambda }=(3n)H^2,`$ (44)
leading to present values in agreement with observation.

The first equation is in accordance with a recent derivation by Schützhold Ralf , based on quantum field calculations in an expanding background. The second equation agrees with an ansatz originally proposed by Chen and Wu Wu , and generalized later by Carvalho, Lima and Waga Waga , and by Alcaniz and Maia Maia . Note that, in the realm of a constant $`G`$, those two equations would be incompatible to each other.

I would like to thank Ralf Schützhold for useful discussions.