text:
# 1 Introduction

## 1 Introduction

Our base field is the field $``$ of complex numbers. We study normal complete algebraic varieties $`X`$ endowed with an action of a connected reductive algebraic group $`G`$ admitting a dense orbit $`\mathrm{\Omega }`$. The stabilizer $`H`$ of a point of the dense orbit will be called *generic stabilizer*. Clearly, $`H`$ is parabolic if and only if $`X=\mathrm{\Omega }`$.

If $`H`$ is connected, Borel showed in  that the homogeneous space $`G/H`$ has at most two ends, *i.e.* the complement $`X\mathrm{\Omega }`$ has at most two connected components. When $`X\mathrm{\Omega }`$ is disconnected, the generic stabilizer contains a maximal unipotent subgroup $`U`$ of $`G`$. This property results from the connectedness (proved in ) of the $`U`$-fixed point subset of $`X`$. In this situation, the varieties $`X`$ share other peculiarities which were clarified by Ahiezer in  where he studied and classified these objects.

This article deals with the left situation – $`X\mathrm{\Omega }`$ is connected –, as we assume this complementary to be $`G`$-homogeneous. Actually, this case appears as the most natural and “simplest” one to consider after the case of projective homogeneous varieties (Grassmaniann, flag varieties, etc.). We call naturally such varieties $`X`$ *two-orbit varieties*. These varieties have been intensively studied before. Ahiezer in  gave a classification of the pairs $`(G,H)`$ such that $`\mathrm{\Omega }=G/H`$ admits a compactification $`X`$ by one homogeneous divisor. This list was independently obtained by Huckleberry and Snow  in the more general context of kählerian varieties. Later, Brion gave in  a purely algebraic approach to these results using the general theory of embeddings developed by Luna and Vust (*see* ). Finally, Feldmüller classified in  all pairs $`(G,H)`$ giving raise to two-orbit varieties whose closed orbit is of codimension $`2`$.

The aim of this paper is to give the complete classification of all two-orbit varieties and to prove Luna’s conjecture, namely that all two-orbit varieties are spherical, *i.e.* admit a dense orbit of a Borel subgroup $`B`$ of $`G`$. In particular, spherical varieties have only finitely many $`B`$-orbits. We refer the reader to  for an introduction to this subject.

To obtain the classification, we use two main steps. First of all, we show that defining a two-orbit variety as being normal is not a major restriction since the normalization map for a complete variety with two orbits is bijective (*see* section 2). Secondly, we give only an explicit list of all *cuspidal* two-orbit varieties from which we obtain all the two-orbit varieties by a parabolic induction procedure explicitly described in section 2.

Acknowledgements. I would like to thank my adviser Peter Littelmann for his constant support. I am also grateful to Michel Brion for his advice and remarks on my PhD’s work in which I obtained the results presented in this paper (*see* ).

Note added in proof. Alexander Smirnov found similar results while I was submitting this article.

## 2 Notation and main results

Let $`G`$ be a connected reductive algebraic group. We fix a Borel subgroup $`B`$ and a maximal torus $`T`$ in $`B`$. Let $`\mathrm{\Phi }`$ be the root system of $`G`$, we denote $`\mathrm{\Phi }^+\mathrm{\Phi }`$ the set of positive roots relative to $`B`$ and $`\mathrm{\Delta }\mathrm{\Phi }^+`$ the set of simple roots corresponding to $`B`$. In case $`G`$ is simple, we enumerate the simple roots as in  and when it is more convenient, we use the short notation of \[loc. cit.\] for the description of positive roots. For example, if $`G`$ is of type $`F_4`$, $`1111`$ stands for the root $`\alpha _1+\alpha _2+\alpha _3+\alpha _4`$.

The subset $`\gamma _1`$,…,$`\gamma _r`$ of $`\mathrm{\Phi }`$ will refer to the set of roots obtained as $``$-linear combinations of the given roots $`\gamma _1,\mathrm{},\gamma _r`$.

For $`\alpha `$ a root of $`G`$, we have the corresponding root space $`𝔤_\alpha 𝔤=LieG`$. We let $`Y_\alpha `$ be any element of $`𝔤_\alpha \{0\}`$, then $`Y_\alpha =𝔤_\alpha `$. The 0-th root space of $`𝔤`$ is $`LieT=𝔱`$.

Throughout this article, $`X`$ will denote a connected normal complete algebraic variety endowed with an action of $`G`$ such that $`X`$ has two $`G`$-orbits. Such a variety is naturally called a *two-orbit $`G`$-variety*. If $`Gx`$ corresponds to the dense orbit and $`Gy`$ to the closed one, we have $`X=GxGy=Cl(Gx)`$ (the closure of $`Gx`$). In other words, a two-orbit $`G`$-variety can be regarded as a complete two-orbit embedding of the homogeneous space $`G/H`$ where $`H`$ is a generic stabilizer.

Actually, a two-orbit variety is even projective. For, according to Sumihiro (*see* ), the closed orbit admits a $`G`$-stable quasi-projective neighbourhood – which has to be all of $`X`$. As a weak converse, we state: a connected projective $`G`$-variety with two $`G`$-orbits (say $`X^{}=Gx^{}Gy^{}`$) is bijective to its normalization, which is, in particular, a two-orbit $`G`$-variety. Let us sketch the proof. Consider the $`G`$-equivariant morphism $`\pi :\stackrel{~}{X}^{}X^{}`$ given by the normalization of $`X^{}`$. First of all, $`\pi `$ being birational, $`\stackrel{~}{X}^{}`$ has a dense $`G`$-orbit isomorphic to $`Gx^{}`$. Secondly, due to the finiteness of $`\pi `$ together with the connectedness of a parabolic subgroup, we get that $`\pi ^1(Gy^{})`$ consits of a finite union of $`G`$-orbits all bijective to $`Gy^{}`$. Therefore, all these orbits are projective. Finally, we prove, by contradiction, that actually $`\pi ^1(Gy^{})`$ is $`G`$-homogeneous. Suppose that $`\pi ^1(Gy^{})`$ consists of more than one (projective) $`G`$-orbit. By , $`\stackrel{~}{X}^{}Gx^{}`$ will have two connected components. Thus, (see the corresponding argument given in the introduction), the generic stabilizer of a point of $`\stackrel{~}{X}^{}`$, and hence of $`X^{}`$, will contain a maximal unipotent subgroup of $`G`$, say $`U`$ subgroup of $`B`$. Consequently, we will end up with two $`B`$-fixed points in $`X^{}`$ – a situation which can not occur.

Similarly, by considering the $`G`$-equivariant finite morphism given by the projection $`G/HG/H^{}`$, one can prove that: if $`G/H`$ has a complete two-orbit embedding, then so does $`G/H^{}`$. The converse is obviously true. More precisely, if $`Z`$ is a complete two-orbit embedding of $`G/H^{}`$, $`G/H`$ inherits a complete two-orbit embedding given as the quotient of $`Z`$ by $`H/H^{}`$. As a consequence, we will restrict our study to two-orbit varieties with connected generic stabilizers, without loss of generality.

Before stating the main results on the two-orbit varieties, we need to make one more remark. Blowing down a given two-orbit variety along its closed orbit may produce a new two-orbit variety. We will illustrate and clarify this natural geometrical construction via the next example.

Let $`G`$ be a simple group of rank $`2`$ and $`\lambda `$ a dominant weight such that $`(\lambda ,\alpha _1^{})>1`$ and $`(\lambda ,\alpha _2^{})=0`$, $`(,)`$ being the Killing form of $`G`$ and $`\alpha _i^{}`$ the coroot associated to $`\alpha _i`$. We consider the irreducible $`G`$-module $`V(\lambda )`$ associated to $`\lambda `$ and the weight vector $`v_{\lambda \alpha _1}V(\lambda )`$ of weight $`\lambda \alpha _1`$. Take $`PB`$ the parabolic subgroup associated to $`\alpha _1`$. Then $`Cl\left(P[v_{\lambda \alpha _1}]\right)\left(V(\lambda )\right)`$ is a $`P`$-variety with two $`P`$-orbits and $`Cl\left(G[v_{\lambda \alpha _1}]\right)=GCl\left(P[v_{\lambda \alpha _1}]\right)`$ is a two-orbit $`G`$-variety. We have an obvious $`G`$-equivariant morphism $`G\times _PCl\left(P[v_{\lambda \alpha _1}]\right)Cl\left(G[v_{\lambda \alpha _1}]\right)`$. This morphism is only isomorphic on the open $`G`$-orbits, the morphism on the closed $`G`$-orbits being $`G/BG/Q`$ where $`QB`$ is the parabolic subgroup associated to $`\alpha _2`$.

This example is very instructive in the sense that it gives the procedure to obtain all two-orbit varieties and motivates the following notion inspired by Luna’s work . In the statement of this definition (and in the rest of the text), the terminology *two-orbit $`P`$-variety* for $`P`$ a parabolic subgroup of $`G`$ will refer, by abuse of language, to a complete normal $`P`$-variety, with two $`P`$-orbits, on which the unipotent radical of $`P`$ acts trivially.

* Definition. A two-orbit $`G`$-variety $`X`$ is *obtained by parabolic induction from a pair* $`(P,Y)`$ if $`PG`$ is a parabolic subgroup and $`Y`$ a two-orbit $`P`$-variety such that

(i) the radical of $`P`$ acts trivially on $`Y`$;

(ii) there exists a $`P`$-equivariant injective morphism $`\phi :YX`$ inducing a birational morphism $`G\times _PYX`$.

A two-orbit variety is *cuspidal* if it can not be obtained by parabolic induction.

###### Proposition 2.1

(i) The geometry of a two-orbit variety obtained by parabolic induction from a pair $`(P,Y)`$ is completely determined by its closed orbit and the geometry of $`Y`$.

(ii) Each (non cuspidal) two-orbit variety is obtained by parabolic induction from a unique pair $`(P,Z)`$ such that $`Z`$ is cuspidal.

Once this result is proved (*see* section 4), we can concentrate only on cuspidal two-orbit varieties. We get their description as follows. We start to show which groups can occur as generic stabilizers (*see* sections 5 and 6). This is done mainly with combinatorial methods. Thus, afterwards, it is just a matter of computations, using Luna-Vust Theory (*see* ), to describe the corresponding complete two-orbit embeddings. We list the pairs in this section but the embeddings only in the appendix to avoid too much notation in this part of the text. Recall that we consider only the case of connected generic stabilizers $`H`$, which is not restricted at all as explained in the beginning of this section.

###### Theorem 2.2

The cuspidal two-orbit varieties are obtained as embeddings of the homogeneous spaces $`G/H`$ with $`(G,H)`$ in Table $`1`$ and Table $`2`$. In particular, $`G`$ is simple or equal to $`SL_2\times SL_2`$, in this situation. Moreover, each such $`G/H`$ has exactly one complete two-orbit embedding described in the appendix.

###### Corollary 2.3

Two-orbit varieties are spherical.

* Proof . As the property of being spherical is not destroyed by parabolic induction, it suffices to check that the homogeneous spaces $`G/H`$ obtained from Table 1 and 2 are indeed spherical. It appears that these embeddings either have been already classified by  or are wonderful (thus spherical by ) of rank $`2`$ (*see* ). $`\mathrm{}`$

## 3 General properties

First of all, note that we can assume the group $`G`$, acting on $`X`$, to be semisimple. Indeed, the radical $`R(G)`$ of $`G`$ acts trivially on $`X`$. For, if there exists a generic element of $`X`$ not fixed by $`R(G)`$, we will obtain at least two $`R(G)`$-fixed points in the closure of its $`R(G)`$-orbit. But it is a well-know fact, that these fixed points belong to different connected components of the $`R(G)`$-fixed point set in $`X`$ – which is connected in our situation since it is equal to the closed $`G`$-orbit of $`X`$, according to

###### Lemma 3.1

Let $`H`$ be a subgroup of $`G`$ and $`T^{}`$ be a torus of $`G`$. If $`G^{}`$ is the identity-component of the centralizer in $`G`$ of $`T^{}`$, then the connected components of $`(G/H)^T^{}`$, the $`T^{}`$-fixed points of $`G/H`$, are exactly given by its $`G^{}`$-orbits.

* Proof . We prove this standard statement on the corresponding tangent spaces. $`\mathrm{}`$

We will start with the case of the two-orbit $`SL_2`$-varieties. We work out this case separately not only because it is simple but also because it will be used quite often later on.

###### Proposition 3.2

The two-orbit $`SL_2`$-varieties are $`^1\times ^1`$ and $`^2`$ with the obvious $`SL_2`$-actions. The respective generic stabilizers are the maximal torus $`T`$ and its normalizer.

* Proof . The a priori possible dimensions for a two-orbit $`SL_2`$-variety (that is, the possible dimensions for a $`SL_2`$-orbit) are $`0,1,2`$ or $`3`$.

Using the fact that projective $`SL_2`$-orbits are 0- or 1-dimensional (and vice versa) and that 3-dimensional $`SL_2`$-orbits are affine, we prove that a two-orbit $`SL_2`$-variety can only be 2-dimensional. Therefore, it has at least three points fixed by $`T`$ a maximal torus of $`SL_2`$ (*see* \[13, Theorem 25.2\]) Since a projective $`SL_2`$-orbit has only two $`T`$-fixed points, a generic stabilizer $`H`$ must contain this torus $`T`$. Finally, we get that $`H/T`$ is finite because $`dimH=1=dimT`$ so $`H=T`$ or $`H=N_G(T)`$. The respective two-orbit $`SL_2`$-varities are clearly $`^1\times ^1`$ and $`^2`$. Note that the action of $`SL_2`$ on $`^2`$ is induced by $`SO_3`$. $`\mathrm{}`$

From now on, the group $`G`$ is of rank greater than $`2`$. Let us start by studying locally the two-orbit $`G`$-variety $`X`$. Consider $`y`$ the $`B`$-fixed point of $`X`$ and $`P`$ its stabilizer in $`G`$. The group $`P`$ is parabolic and $`P^uL`$ will be its Levi decomposition such that $`TL`$. By \[5, Theorem 1.4\], there exists an affine $`L`$-stable subvariety $`Z`$ of $`X`$ such that $`dimZ1`$ and $`ZGy=\{y\}`$. Using the fact that $`T`$-orbits are affine, it is easy to see that an affine $`T`$-variety containing fixed points also contains $`1`$-dimensional orbits if the action is not trivial. Therefore, there is a generic element $`x`$ of $`X`$ such that $`codim_TT_x1`$. In other words, we have just obtained:

###### Proposition 3.3

The ranks of the algebraic group $`G`$ and of the generic stabilizer differ at most by $`1`$.

* Definition. The two-orbit varieties such that the rank of $`G`$ is equal to (resp. different from) the rank of $`H`$ are said of *type I* (resp. *of type II*).

Suppose that the generic element $`x`$ is such that the torus $`T_x^{}`$ is a maximal torus of $`G_x`$. Let $`L^{}`$ be the centralizer in $`G`$ of $`T_x^{}`$. Then, by the above proposition, $`L^{}`$ is equal to $`T`$ or is a Levi subgroup of semisimple rank $`1`$. We are going to prove that the latter possibility can not occur *i.e.*

###### Proposition 3.4

The maximal tori of the generic stabilizer are regular tori of the group $`G`$.

* Proof . We proceed by contradiction; suppose that $`L^{}T`$ and consider the $`L^{}`$-variety $`Cl(L^{}x)`$. Since $`T_x^{}T`$, $`Cl(L^{}x)`$ is actually a $`SL_2`$-variety but not a two-orbit $`SL_2`$-variety (*see* Proposition 3.2). More precisely, $`Cl(L^{}x)L^{}x`$ is a finite union of closed $`L^{}`$-orbits of dimension $`1`$ all contained in $`Gy`$ because of Lemma 3.1. Therefore (*see* also the proof of Proposition 3.2), the $`L^{}`$-variety $`Cl(L^{}x)`$ is of dimension $`2`$.

Let $`B_L^{}`$ be the Borel subgroup of $`L^{}`$ equal to $`BL^{}`$ and $`U_L^{}`$ be its unipotent radical. There exists an element, say $`x^{}`$, in $`L^{}x`$ fixed by $`U_L^{}`$ (*see* the introduction). Moreover, $`T_x^{}`$ fixes obviously $`x^{}`$ so, by Proposition 3.3, $`T_x^{}=T_x^{}^{}`$. Thus, $`Cl(L^{}x^{})=LCl(B_L^{}x^{})=L^{}Cl(Tx^{})`$. If $`y_1`$ and $`y_2`$ denote the $`T`$-fixed points in $`Cl(Tx^{})`$ then $`Cl(L^{}x)=L^{}xL^{}y_1L^{}y_2`$. The variety $`Cl(L^{}x)`$, being 2-dimensional, has at least three $`T`$-fixed points, (*see* \[13, Theorem 25.2\]). Therefore, $`y_1`$ and $`y_2`$ can not be simultaneously fixed by $`L^{}`$. Suppose that $`L^{}y_1y_1`$ and consider the element, say $`y^{}`$, satisfying $`Cl(Ty^{})=L^{}y_1`$. Then we get:

$$T_y^{}^{}=T_x^{}^{},dim(Tx^{})=dim(Ty^{})=1\text{and}y_1Cl(Tx^{})Cl(Ty^{}).$$

It results from the lemma below that $`y_2Ly_1`$ which is incompatible with the fact that $`Cl(L^{}x)`$ is not a two-orbit variety. Thus the assumption $`L^{}T`$ was absurd – which proves Proposition 3.4. $`\mathrm{}`$

###### Lemma 3.5

Let $`x`$ and $`z`$ be two elements of a projective $`G`$-variety such that

$$dimTx=dimTz=1\text{and}T_x^{}=T_z^{}.$$

There are two or four $`T`$-fixed points in $`Cl(Tx)Cl(Tz)`$ assuming that they are all contained in one same closed $`G`$-orbit.

* Proof . By embedding the given variety $`G`$-equivariantly into the projective space of a representation, we can write the elements $`x`$ and $`z`$ as: $`x=[v_{\mu _1}+\mathrm{}+v_{\mu _r}]`$ and $`z=[v_{\nu _1}+\mathrm{}+v_{\nu _s}]`$. By assumption, the weights of the support of $`x`$ (resp. of the support of $`z`$) sit on a same affine line $`D_x`$ (resp. $`D_z`$) and moreover,

$$\mathrm{ker}(\mu _1\mu _r)^{}=T_x^{}=T_z^{}=\mathrm{ker}(\nu _1\nu _s)^{}.$$

From these equalities, we can deduce that the lines $`D_x`$ and $`D_z`$ are strictly parallel or equal. The first situation clearly yields four $`T`$-fixed points in $`Cl(Tx)Cl(Tz)`$ and the second one only two $`T`$-fixed points (use the fact that the Weyl group acts transitively on the $`T`$-fixed points of a projective orbit). $`\mathrm{}`$

As a consequence of Proposition 3.4, we have:

###### Corollary 3.6

Consider two elements $`x`$ and $`z`$ of the dense $`G`$-orbit of a two-orbit variety such that $`T_x^{}`$ (resp. $`T_z^{}`$) is a maximal torus of $`G_x`$ (resp. $`G_z`$). Then $`zN_G(T)x`$; in particular, if $`x`$ and $`z`$ are $`B`$-conjugated they are actually $`T`$-conjugated.

## 4 Parabolic induction

This section is the main step to the classification of two-orbit varieties. It consists essentially in proving that the problem of classifying the two-orbit varieties can be reduced to a subclass of two-orbit varieties called *cuspidal*, *i.e.* the two-orbit varieties which can not be obtained by parabolic induction (*see* section 2).

### 4.1 Statements

###### Proposition 4.1

If $`X`$ is obtained by parabolic induction from a pair $`(P,Y)`$, then it is completely determined by its closed orbit and the variety $`Y`$.

* Proof . This statement is a direct consequence of the fact that the morphism $`G\times _PYX`$ is a blowing down of the closed $`G`$-orbit of $`X`$ and that the variety $`X`$ is completely determined, according to a result of Luna and Vust (*see* theorem 8.3 of ), by its $`G`$-stable prime divisors and by the $`B^{}`$-stable prime divisors of its dense $`G`$-orbit, whose closure in $`X`$ contains the closed $`G`$-orbit of $`X`$. $`\mathrm{}`$

We can put an order $``$ on the set of induction pairs of $`X`$, defined naturally, for two pairs $`(P_1,Y_1)`$ and $`(P_2,Y_2)`$, by:

$$(P_1,Y_1)(P_2,Y_2)\text{if }P_1P_2\text{ and }\phi _1(Y_1)\phi _2(Y_2).$$

###### Theorem 4.2

The set of induction pairs of a two-orbit variety (endowed with the order $``$) has an unique minimal element.

### 4.2 Proof of Theorem 4.2

In order to prove Theorem 4.2, we are constructing explicitly, in this section, the minimal induction pair.

Let us consider an induction pair $`(P,Y)`$ and two elements $`x`$ and $`z`$ in the dense $`G`$-orbit of $`X`$ such that $`Bx`$ and $`Bz`$ are closed in $`Gx`$. These elements can be chosen such that $`T_x^{}`$ (resp. $`T_z^{}`$) is a maximal torus of $`G_x`$ (resp. $`G_z`$). Then, by Corollary 3.6, they are conjugated by an element $`nN_G(T)`$. Because of the choice made on $`Bx`$ and on $`Bz`$, one can show by a short computation that $`n`$ is in fact an element of $`P`$. We have got:

###### Lemma 4.3

If $`X`$ is obtained from a pair $`(P,Y)`$ by parabolic induction then the $`B`$-orbits which are closed in the dense $`G`$-orbit are in a same $`P`$-orbit of $`X`$. In particular, they are contained in $`\phi (Y)`$.

Let $`x`$ be as above, denote $`G_x`$ by $`H`$. Let $`P_1`$ be the parabolic subgroup of $`G`$ generated by $`B`$, $`H`$ and the elements $`n`$ of $`N_G(T)`$ such that $`Bnx`$ is closed in $`Gx`$. By construction, for all pairs $`(P,Y)`$, we have: $`P_1P`$ and $`X_1=Cl(P_1x)\phi (Y)`$. However, $`X_1`$ may not be a two-orbit $`P_1`$-variety. More precisely, we have: $`X_1Gx=P_1x`$ but an analogous equality may not hold for $`X_1Gy`$ ($`y`$ a $`T`$-fixed element of the closed orbit in $`X_1`$). So, instead of $`P_1`$, we have to consider $`P_H`$ the parabolic subgroup of $`G`$ generated by $`P_1`$ and the elements $`wW`$ such that $`wyX_1`$. Thus, $`X_H=Cl(P_Hx)=Cl(P_HX_1)`$ is a two-orbit $`P_H`$-variety with $`P_HP`$ and $`X_H\phi (Y)`$.

Finally, let us consider $`\stackrel{~}{X}_H`$ the normalized variety of $`X_H`$. The pair $`(P_H,\stackrel{~}{X}_H)`$ is the required element.

###### Proposition 4.4

The pair $`(P_H,\stackrel{~}{X}_H)`$ is an induction pair; this is the minimal element for the set of induction pairs of $`X`$.

We already know that the variety $`\stackrel{~}{X}_H`$ is a two-orbit $`P_H`$-variety. So we are left only to prove that the radical of $`P_H`$ acts trivially on $`\stackrel{~}{X}_H`$. For this, we use the following

###### Proposition 4.5

Let $`Y`$ be a two-orbit $`Q`$-variety with $`Q`$ a parabolic subgroup of $`G`$ containing $`B`$. Consider an element $`z`$ of its dense $`Q`$-orbit such that the torus $`T_z^{}`$ is a maximal torus of $`Q_z`$. Suppose that the radical of $`Q`$ doesn’t act trivially on $`Y`$. Then the orbit $`Lz`$ is complete for $`L`$ a Levi subgroup of $`Q`$ containing the torus $`T`$.

* Proof of Proposition 4.4. According to Proposition 4.5 applied to $`\stackrel{~}{X}_H`$ and $`P_H`$, if we prove that $`Lz`$ is not complete, Propostion 4.4 will follow. If $`T_z^{}T`$, that $`Lz`$ is not complete is obvious. Consider then the other case: $`T_z^{}=T`$. If $`\alpha `$ is a simple root of $`P_H`$ such that $`U_\alpha G_z`$ then (because of the assumption made on $`Bz`$) $`s_\alpha Bs_\alpha G_z`$ contains a Borel subgroup of $`G_z`$. In other words, $`s_\alpha Bs_\alpha `$ is closed in $`P_Hz`$ and by definition of $`P_H`$, it means that $`SL_2(\alpha )L`$. To conclude that $`Lz`$ is not complete, we need only to observe that $`Cl(Lz)`$ contains a $`T`$-fixed point of the closed $`G`$-orbit (because $`Cl(U_\alpha z)`$ does by Corollary 3.6). $`\mathrm{}`$

* Proof of Proposition 4.5. First of all, note that the radical of $`Q`$ acts trivially on the closed $`Q`$-orbit of $`Y`$. So the proposition relies only on the dense $`Q`$-orbit.

We choose an element $`z`$ verifying the hypotheses of the proposition and such that $`Bz`$ is closed in $`Qz`$.

Let us start with the case: $`T_z^{}T`$. Thus $`Lz`$ is not complete. Let $`\beta \mathrm{\Phi }^+`$ be such that $`U_\beta Q_z`$. Such a root exists otherwise $`Y`$ will have two $`B`$-fixed points given by the $`T`$-fixed points of $`Cl(Tz)`$. Consider the $`2`$-dimensional variety $`X_\beta =Cl(U_\beta Tz)`$ and denote by $`T_{X_\beta }`$ ($`\mathrm{ker}\beta `$) its generic stabilizer in $`T`$. To get our result, we are going to prove that $`T_{X_\beta }^{}`$ contains the identity-component $`Z(L)^{}`$ of the center of $`L`$.

Let $`y_1`$ and $`y_2`$ be the $`T`$-fixed points of $`Cl(Tz)`$. If $`y_i`$ ($`i=1,2`$) is not fixed by $`U_\beta `$, we denote by $`y_i^{}`$ the other $`T`$-fixed point in $`Cl(U_\beta y_i`$); otherwise, we set $`y_i^{}=y_i`$. The points $`y_1^{}`$ and $`y_2^{}`$ are distinct; this follows easily from Lemma 3.5 and from the fact that $`T_z^{}`$ is regular.

The variety $`X_\beta ^{U_\beta }`$, being connected and containing two distinct elements, $`y_1`$ and $`y_2`$, is $`1`$-dimensional.

+ Claim. There exist $`z_1,\mathrm{},z_r`$ and $`\rho _1,\mathrm{},\rho _{r+1}`$ ($`r1`$) in $`X_\beta ^{U_\beta }`$ such that $`dimTz_i=1`$ and such that $`\rho _i,\rho _{i+1}`$ are the $`T`$-fixed points in $`Cl(Tz_i)`$. Moreover, if $`y_i=y_i^{}`$ ($`i=1,2`$) then $`r2`$.

To construct the elements $`z_i`$ and $`\rho _i`$, it suffices to consider the convex hull of the support of $`X_\beta `$; the elements $`z_i`$ (resp. $`\rho _i`$) have as support the edges (resp. vertices) of this polytope.

Becasuse of the claim, we have now at hand at least two elements $`u`$ and $`v`$ simultaneously in the closed $`Q`$-orbit and in $`X_\beta `$ such that $`dim(Tu)=1=dim(Tv)`$ and $`Cl(Tu)Cl(Tv)=1`$. Therefore: $`\left(T_uT_v\right)^{}`$ (of codimension $`2`$) is equal to $`T_{X_\beta }^{}`$. But since $`Z(L)`$ acts trivially on the closed $`Q`$-orbit, we end up with the required inclusion: $`Z_LT_{X_\beta }^{}`$.

Assume now that $`T_z^{}=T`$ and that $`Lz`$ is not closed. Therefore, there exists at least one positive root, say $`\alpha `$, in the root system of $`(L,L)`$ such that $`U_\alpha Q_z`$. Recall that we want to prove that the radical of $`Q`$ acts trivially on $`Y`$. To do so, we proceed by contradiction: suppose there exists $`\beta \mathrm{\Phi }^+`$, $`\beta \alpha `$ such that $`U_\beta Q^u`$ and $`U_\beta Q_z`$. Then, the variety $`X_{\beta ,\alpha }=Cl(U_\beta U_\alpha z)`$ is 2-dimensional and contains a dense $`T`$-orbit; denote by $`T_{X_\beta }`$ its generic stabilizer in $`T`$.

If $`y_1`$ and $`y_2`$ are the $`T`$-fixed points (distinct from $`z`$) of $`Cl(U_\alpha z)`$ and of $`Cl(U_\beta z)`$ respectively, then necessarily $`y_1`$ and $`y_2`$ are distinct (as they do not have the same support). If $`U_\beta y_1y_1`$, we denote by $`y_1^{}`$ the other $`T`$-fixed point of $`Cl(U_\beta y_1)`$; otherwise, we set $`y_1^{}=y_1`$. Then the variety $`X_{\beta ,\alpha }^{U_\beta }`$ is 1-dimensional, since it contains the distinct points $`y_1^{}`$ and $`y_2`$. But, $`y_1`$ and $`y_2`$ must be the only $`T`$-fixed points of the closed $`Q`$-orbit of $`Y`$ in $`X_{\beta ,\alpha }`$ otherwise with the same arguments used in the first case ($`T_z^{}T`$), we will have: $`Z(L)^{}T_{X_\beta }`$. So, we can conclude that $`U_\beta `$ must fix $`y_1`$ and that there exists an element $`u`$ in the closed $`Q`$-orbit of $`Y`$ such that: $`X_{\beta ,\alpha }^{U_\beta }=Cl(Tu)`$.

By considering the variety $`Cl(U_\alpha y_1)`$, we get $`y_3`$, the other $`T`$-fixed point in it. For the same reasons as before, the points $`y_1`$ and $`y_3`$ are distinct.

To conclude, we have constructed two 1-dimensional subvarieties of the closed $`Q`$-orbit, $`X_{\beta ,\alpha }`$ and $`Cl(U_\alpha y_1)`$ such that $`y_1`$ is their only common $`T`$-fixed point. According to Lemma 3.5, this yields the contradiction: $`(T_u\mathrm{ker}\alpha )^{}=T_{X_\beta }^{}`$. $`\mathrm{}`$

## 5 Two-orbit varieties of type I

In this section, the two-orbit $`G`$-variety $`X=Cl(Gx)=GxGy`$ is of type I, *i.e.* $`T_x^{}=T_x`$. Recall (*see* section 2) that a two-orbit variety is projective. So we can embed $`X`$ in $`(V)`$ with $`V`$ a finite $`G`$-module. The elements $`x`$ and $`y`$ of $`X`$ can be written as: $`x=[v_\mu ]`$ and $`y=[v_\lambda ]`$ with $`v_\mu `$ and $`v_\lambda `$ weight vectors of $`V`$. We can choose $`\lambda `$ to be dominant.

Let $`\beta \mathrm{\Phi }^+`$ be such that $`U_\beta G_x`$ (such a root exists otherwise $`BG_x`$) and consider $`z=[v_\mu +v_{\mu +\beta }\mathrm{}+v_{\mu +k\beta }]U_\beta x`$. With a judicious choice of $`x`$, (for instance $`\mu `$ dominant), one shows easily that $`[v_{\mu +k\beta }]Gy`$. So finally, we can choose $`x`$ such that $`\lambda =\mu +k\beta `$ (take a $`W`$-conjugate of the previous $`x`$). Define the support of a root (denoted $`supp`$) as the set of simple roots really involved in its writting.

###### Proposition 5.1

If $`X`$ is cuspidal then $`G`$ is simple and $`supp\beta =\mathrm{\Delta }`$.

* Proof . The first assertion comes from the construction of the minimal induction pair (*see* section 4.2). More precisely, as $`X`$ is cuspidal, $`G`$ must be generated by the parabolic subgroup $`P_1`$ and the elements $`wW`$ such that $`w[v_\lambda ]X_1`$. Recall that $`P_1`$ is spanned by $`H`$, $`B`$ and the elements $`nN_G(T)`$ such that $`Bnx`$ is closed in $`Gx`$ and $`X_1=Cl(P_1x)`$.

If $`G=G_1\times \mathrm{}\times G_r`$ with $`G_i`$ simple and $`\mathrm{\Delta }=\mathrm{\Delta }_1\times \mathrm{}\times \mathrm{\Delta }_r`$ with $`\mathrm{\Delta }_i`$ associated to $`G_i`$, we are going to show that $`H=H_1\times \mathrm{}\times H_r`$ with $`H_i=G_i`$ for all $`ii_0`$ and $`supp\beta \mathrm{\Delta }_{i_0}`$.

First of all, note that if $`\alpha supp\beta `$, then $`U_\alpha G_x`$. Indeed, $`\mu +\mathrm{}\alpha `$ ($`\mathrm{}>0`$) is not of shape $`\lambda _{\gamma \mathrm{\Delta }}n_\gamma \gamma `$, (weights of $`V`$) since $`\lambda =\mu +k\beta `$ (*see* the choice of $`x`$ made above). Moreover, if $`\alpha \mathrm{\Delta }\mathrm{\Delta }_{i_0}`$ then with the above description of $`G`$, we must have $`U_\alpha G_x`$. Thus: $`U_{\pm \alpha }G_x`$, for all $`\alpha \mathrm{\Delta }\mathrm{\Delta }_{i_0}`$. The acting group $`G`$ can then be assumed to be simple.

To obtain the second assertion, consider the parabolic subgroup $`P`$ associated to $`supp\beta `$ and the $`P`$-variety $`Z=X\left(_\nu V_\nu \right)`$ for $`\nu =\lambda n_\alpha \alpha `$ with $`\alpha supp\beta `$ and $`n_\alpha 0`$. Then, if $`supp\beta \mathrm{\Delta }`$, $`(P,Z)`$ is an induction pair of $`X`$. $`\mathrm{}`$

From now on, $`G`$ will be simple and $`x`$ will satisfy the above conditions as well as the two following ones. If $`\lambda =\mu +k\beta `$ then $`k`$ is minimal and moreover if $`[v_\mu ^{}]`$ is another $`T`$-fixed generic element satisfying the same assumptions as $`x`$ then $`\mu >\mu ^{}`$ for $`\mu `$ and $`\mu ^{}`$ comparable.

Consider a second positive root, say $`\gamma `$, with $`\gamma \beta `$ and $`U_\gamma G_x`$ (there exists at least one such a root which is simple). Denote by $`L`$ the Levi subgroup associated to $`\beta `$ and $`\gamma `$, *i.e.* $`L`$ is the centralizer in $`G`$ of $`(\mathrm{ker}\beta \mathrm{ker}\gamma )^{}`$. Then we have:

###### Lemma 5.2

The variety $`Cl(Lx)`$ is a cuspidal two-orbit $`L`$-variety.

* Proof . From Lemma 3.1, we have $`Lx=Cl(Lx)Gx`$ and $`Cl(Lx)Lx`$ consists of a finite union of complete $`L`$-orbits. Thus, if $`Cl(Lx)`$ is not a two-orbit $`L`$-variety, we will have $`(Lx)^{U_L}\mathrm{}`$ for $`U_L`$ an unipotent maximal subgroup of $`L`$ (again the same argument as in the introduction). It will follow that $`Cl(Lx)=Lx`$. But this equality can not hold since $`[v_\lambda ]Cl(Lx)Lx`$.

As we can not find any proper parabolic subgroup $`P`$ such that $`P^uL_xP`$ (because $`U_{\pm \beta }`$, $`U_{\pm \gamma }G_x`$), $`Lx`$ must be cuspidal. $`\mathrm{}`$

By this procedure, we have constructed two-orbit varieties for some subgroups $`L`$ of $`G`$ of semisimple rank $`2`$. Thus, once we know what the two-orbit varieties are, for the simple groups of rank $`2`$, we will know $`L_x`$ (for all $`L`$’s) and then $`G_x`$.

Let us start with determining the cuspidal two-orbit varieties in the rank $`2`$ case. For this, we need two technical lemmas. Let $`\mathrm{\Lambda }`$ be the convex hull of the support of the variety $`Cl(U_\gamma U_\beta x)`$. We have the following picture and notation in the weight lattice $`𝒳`$ of $`G`$.

###### Lemma 5.3

The extremal points $`\nu _i`$ of $`\mathrm{\Lambda }`$ are $`W`$-conjugated to $`\lambda `$ and the $`\beta _i`$’s are some roots of $`\mathrm{\Phi }`$.

* Proof . First of all, note that the points $`[v_{\nu _i}]`$ and the elements $`[v_i]`$ whose support is $`[\nu _i,\nu _{i+1}]𝒳`$ sit in $`Cl(U_\gamma U_\beta x)`$. If $`\mathrm{\Lambda }`$ has more than three extremal points then the cardinality of $`supp[v_i]`$ is smaller than $`suppx`$’s one. So by the minimality assumption made on $`x`$, we must have $`[v_i]Gy`$ and thus $`[v_{\nu _i}]Gy`$. If $`\mathrm{\Lambda }`$ has three extremal points, the assumed maximality of the weight $`\mu `$ ($`x=[v_\mu ]`$) forces $`\nu _1`$ to be in $`W\lambda `$.

The second assertion of the lemma follows from the fact that $`T_{[v_i]}^{}=\mathrm{ker}(\nu _i\nu _{i+1})^{}`$ is singular. $`\mathrm{}`$

###### Lemma 5.4

Let $`\delta `$ be a positive root such that $`[v_{s_\gamma ^{}(\lambda )}]Cl(U_\delta x)`$, for $`\gamma ^{}\mathrm{\Delta }`$. Suppose that: if there exists $`r0`$ such that $`\mu +r\gamma ^{}`$ is extremal as weight of $`V`$, we must have $`\mu +r\gamma ^{}=s_\alpha (\lambda )`$ for $`\alpha \mathrm{\Delta }`$. Then, $`U_\gamma ^{}G_x`$ if $`(\mu ,\gamma ^{})0`$ and $`U_\gamma ^{}G_x`$ if $`(\mu ,\gamma ^{})0`$.

* Proof . If there is no $`r>0`$ such that $`\mu +r\gamma ^{}`$ is extremal then by Lemma 5.3, we must have $`U_\gamma ^{}G_x`$ if $`(\mu ,\gamma ^{})0`$. If $`(\mu ,\gamma ^{})0`$ and $`U_\gamma ^{}G_x`$ then $`Cl(U_\gamma ^{}x)`$ contains a $`T`$-fixed point of $`Gy`$ and so does $`Cl(U_\gamma ^{}x)`$ – which is absurd.

Suppose now that $`\mu +r\gamma ^{}`$ is extremal and that $`(\mu ,\gamma ^{})0`$. Let $`x^{}=\epsilon _\delta (1)xU_\delta x`$, $`\epsilon _\delta `$ being the natural map associated to $`\delta `$ from $``$ to $`U_\delta `$. Then by assumption $`x^{}=[v_\mu +\mathrm{}+v_{s_\gamma ^{}(\lambda )}]`$. The study of $`supp(\epsilon _\beta (1)x^{})`$ gives raise to a $`j>0`$ such that

$$\frac{Y_\beta ^{n_j}}{n_j!}v_{\mu +j\delta }=sY_\gamma ^{}v_{s_\gamma ^{}(\lambda )}\text{for }s\{0\}\text{ and }n_j0\text{.}$$
(1)
Recall (*see* section 2) that $`Y_\beta `$ denotes an element of $`𝔤_\beta \{0\}`$.

Furthermore, consider the support of the variety $`Cl\left(T\mathrm{exp}(Y_\beta +sY_\gamma ^{})x^{}\right)`$. The points $`s_\alpha (\lambda )`$, $`s_\gamma ^{}(\lambda )`$ are extremal points of this support but $`\lambda `$ is not; otherwise, we will have a gap in this support since the weight $`\gamma ^{}+s_\gamma ^{}(\lambda )`$ is missing by equality (1). It implies that $`[s_\alpha (\lambda ),s_\gamma ^{}(\lambda )]`$ must be an edge of this support – which contradicts Lemma 5.3 since $`s_\alpha (\lambda )s_\gamma ^{}(\lambda )`$ is not a root, $`\alpha `$ and $`\gamma ^{}`$ being simple.

If $`(\mu ,\gamma ^{})0`$, we can go back to the positive case with the element $`s_\gamma ^{}x`$. $`\mathrm{}`$

Let us show how we can apply these two lemmas to get the two-orbit varieties in the rank $`2`$ case, through the following

* Example. Suppose $`G`$ of type $`B_2`$. Then there are two possibilities for $`\beta `$: $`\beta =\alpha _1+\alpha _2`$ or $`\beta =\alpha _1+2\alpha _2`$. Let us compute $`𝔤_x`$ in case $`\beta =\alpha _1+\alpha _2`$.

If $`U_{\alpha _1}G_x`$ then $`\alpha _2`$ verifies the conditions of Lemma 5.4 with $`(\mu ,\alpha _2^{})0`$. Therefore $`U_{\alpha _2}G_x`$. Let us show that $`U_{\alpha _1+2\alpha _2}G_x`$. If there does not exist $`r>0`$ such that $`\mu +r(\alpha _1+2\alpha _2)`$ is extremal then this inclusion is given by Lemma 5.3. Otherwise, we will have: $`\mu +r\alpha =s_1(\lambda )`$. A simple computation leads to: $`2(\lambda ,\alpha _1^{})=(\lambda ,\alpha _2^{})`$. But in this latter case, $`\mu `$ does not satisfy the good conditions; in particular, “$`k`$” is not minimal. We have obtained the pair: $`(SO_5,𝔤_x=𝔱𝔤_{\alpha _2}𝔤_{\alpha _1+2\alpha _2})`$.

If $`U_{\alpha _1}G_x`$ then $`U_{\alpha _2}G_x`$ and we have $`(\mu ,\pm \alpha _1)=0`$. Thus by Lemma 5.4, we get $`U_{\pm \alpha _1}G_x`$. It follows from Lemma 5.3 that $`U_{\alpha _1+2\alpha _2}G_x`$. Therefore, we have obtained the pairs: $`(SO_5,𝔰𝔬_4)`$ and $`(SO_5,𝔤_x=𝔱𝔤_{\pm \alpha _1}𝔤_{\alpha _1+2\alpha _2})`$.

By the procedure given in this example, we get the two-orbit varieties for the rank $`2`$ case. In other words, we have a description of $`\mathrm{\Phi }(G_x)`$ the root system of $`G_x`$.

###### Lemma 5.5

Let $`\gamma \mathrm{\Phi }^+`$, we have the following:

(i) if $`\gamma ,\beta `$ is of type $`A_1\times A_1`$ then $`\pm \gamma \mathrm{\Phi }(G_x)`$;

(ii) if $`\gamma ,\beta `$ is of type $`A_2`$ then $`\pm \gamma \mathrm{\Phi }(G_x)`$ or $`\pm s_\gamma (\beta )\mathrm{\Phi }(G_x)`$;

(iii) if $`\gamma ,\beta `$ is of type $`B_2`$ with $`\beta =\epsilon _1+\epsilon _j`$ then $`\mathrm{\Phi }(G_x)\gamma ,\beta =\{\epsilon _1\epsilon _j,\epsilon _1\}`$;

(iv) if $`\gamma ,\beta `$ is of type $`B_2`$ with $`\beta =\epsilon _1`$ then $`\mathrm{\Phi }(G_x)\gamma ,\beta \{\epsilon _1+\epsilon _j,\pm (\epsilon _1\epsilon _j)\}`$ or $`\mathrm{\Phi }(G_x)\gamma ,\beta =\{\epsilon _1+\epsilon _j,\epsilon _j\}`$.

By applying this statement to $`\beta `$, a positive root of maximal support and to any other positive root, we get Table 1 just by computations. Let us work out two examples to understand how it works.

* Example. Suppose $`G`$ of type $`A_n`$. Then $`\beta =\alpha _1+\mathrm{}+\alpha _n`$. Since $`\alpha _i,\beta `$ is of type $`A_1\times A_1`$, by Lemma 5.5-(i), we have $`\pm \alpha _i\mathrm{\Phi }(G_x)`$ for $`2in1`$. Moreover, Lemma 5.5-(ii) yields: $`\pm \alpha _1\mathrm{\Phi }(G_x)`$ or $`\pm \alpha _n\mathrm{\Phi }(G_x)`$. This gives the pair $`(SL_{n+1},𝔤𝔩_n)`$.

Suppose $`G`$ of type $`B_n`$ and $`\beta =\epsilon _1`$. Applying Lemma 5.5-(i) and Lemma 5.5-(ii) respectively, we get $`\{\pm \alpha _2,\mathrm{},\pm \alpha _n\}\mathrm{\Phi }(G_x)`$ and respectively $`\{\epsilon _1+\epsilon _i:i2\}\mathrm{\Phi }(G_x)`$ with $`\pm (\epsilon _1\epsilon _i)\mathrm{\Phi }(G_x)`$ or $`\epsilon _i\mathrm{\Phi }(G_x)`$ for all $`i2`$. If $`\pm \alpha _1\mathrm{\Phi }(G_x)`$ then $`\{\pm \alpha _1,\mathrm{},\pm \alpha _{n1},\alpha _{n1}+2\alpha _n\}\mathrm{\Phi }(G_x)`$. Therefore, we obtain the two pairs $`(SO_{2n+1},𝔰𝔬_{2n})`$ and $`\left(SO_{2n+1},𝔤_x=_{\alpha \mathrm{\Psi }}𝔤_\alpha 𝔱\right)`$ where $`\mathrm{\Psi }=\pm \alpha _1,\mathrm{},\pm \alpha _{n1},\alpha _{n1}+2\alpha _n`$. If $`\pm \alpha _1\mathrm{\Phi }(G_x)`$, we get the pair $`(SO_{2n+1},𝔤_x=_{\alpha \mathrm{\Psi }}𝔤_\alpha 𝔱)`$, with $`\mathrm{\Psi }=\pm \alpha _2,\mathrm{},\pm \alpha _{n1},\alpha _n,\epsilon _1+\epsilon _n`$.

## 6 Two-orbit varieties of type II

In this section, the two-orbit $`G`$-variety $`X=Cl(Gx)`$ is of type II, that is, by definition, $`T_x^{}T`$. We embed $`X`$ in $`(V)`$ with $`V`$ a finite $`G`$-module as in the previous section. Then the generic element $`x`$ can be written as $`[v_{\lambda _0}+\mathrm{}+v_{\lambda _{r+1}}]`$ with the $`\lambda _i`$’s sitting on a same affine line, say $`D_x`$ (because of Proposition 3.3). we order the weights $`\lambda _i`$ of $`suppx`$ in such way that: $`Cl(Tx)=Tx\{[v_{\lambda _0}],[v_{\lambda _{r+1}}]\}`$. As elements of $`Gy`$, $`[v_{\lambda _0}]`$ and $`[v_{\lambda _{r+1}}]`$ are $`W`$-conjugate, *i.e.* there exists $`wW`$ such that $`\lambda _0=w(\lambda _{r+1})`$. We choose $`x`$ such that $`\lambda _0=\lambda `$ and satisfying the following condition of minimality: if $`x^{}=[v_\lambda +\mathrm{}+v_{w^{}(\lambda )}]`$ is another generic element then $`w<w^{}`$ for $`w`$ and $`w^{}`$ comparable.

Let $`\alpha `$ be a simple root such that $`s_\alpha w<w`$ and $`U_\alpha G_x`$. Take for instance $`\alpha `$ such that $`(w(\lambda ),\alpha )<0`$. Consider the variety $`X_\alpha =Cl(TU_\alpha x)`$ and in particular, the convex hull $`\mathrm{\Lambda }`$ of its support pictured below in the weight lattice $`𝒳`$ of $`G`$.

Similarly as for Lemma 5.3, we get:

###### Lemma 6.1

The elements $`y_iX_\alpha `$ whose support sits on the line $`D_i`$ belong to the closed $`G`$-orbit of $`X`$. The directions of the affine lines $`D_i`$ are given by roots $`\beta _i`$. In particular, the extremal points $`\nu _i`$ are $`W`$-conjugated and $`suppy_i=[\nu _i,\nu _{i+1}]𝒳`$.

The roots $`\beta _i`$ span a root system of rank $`2`$; let $`\{\alpha ,\beta \}`$ be a basis of this root system.

###### Corollary 6.2

$`w=s_\alpha s_\beta `$.

* Proof . We know that $`s_\alpha w<w`$ and that $`\lambda w(\lambda )`$ can not be, up to a scalar, a root because $`T_x^{}=\mathrm{ker}(\lambda w(\lambda ))^{}`$ is regular (*see* Proposition 3.4). So we can assume $`\alpha ,\beta `$ to be of type $`G_2`$ with $`w=(s_\alpha s_\beta )^2`$ – the case $`w=w_0`$ being easily ruled out.

Consider the convex hull $`\mathrm{\Lambda }`$ (*see* the corresponding picture). Since we have: $`suppy_i=[\nu _i,\nu _{i+1}]𝒳`$, there exists a weight $`\nu D_x`$ such that $`\lambda \beta _t=\nu +k\alpha `$, $`k0`$. In other words, if $`D_{\alpha ,\lambda \beta _t}`$ denotes the line of direction $`\alpha `$ passing through $`\lambda \beta _t`$, we must have:

$$D_xD_{\alpha ,\lambda \beta _t}𝒳\mathrm{}.$$
(2)
If $`w=(s_\alpha s_\beta )^2`$ with $`\alpha ,\beta `$ of type $`G_2`$ then $`\beta _t=\beta `$ or $`\beta =\beta +\alpha `$. But (2) forces $`(\lambda ,\alpha )/\left((\lambda ,\alpha )+(\lambda ,\beta )\right)`$ to be an integer – which can not occur because $`(\lambda ,\alpha )(\lambda ,\beta )0`$ since $`T_x^{}`$ is regular. $`\mathrm{}`$

Using this corollary and the same arguments given in the proof of Proposition 5.1, we get:

###### Proposition 6.3

If $`X`$ is cuspidal of type II then $`G`$ is simple or of type $`A_1\times A_1`$. Furthermore, $`supp\beta \{\alpha \}=\mathrm{\Delta }`$.

From now on $`G`$ will be assumed to be simple or of type $`A_1\times A_1`$. Let $`L`$ be the Levi subgroup associated to $`\alpha `$ and $`\beta `$ and $`𝔩`$ be its Lie algebra.

###### Proposition 6.4

(i) If $`\alpha ,\beta `$ is of type $`A_1\times A_1`$ then $`𝔩_x=𝔱^{}(Y_\alpha +Y_\beta )(Y_\alpha +Y_\beta )`$.

(ii) Otherwise, $`𝔩_x=𝔱^{}(Y_\alpha +Y_{s_\alpha (\beta )})_{\begin{array}{c}\gamma \mathrm{\Phi }^+\alpha ,\beta \\ \gamma \alpha ,s_\alpha (\beta )\end{array}}𝔤_\gamma `$.

Here, $`𝔱^{}`$ is the kernel of $`\alpha +s_\alpha (\beta )`$ considered as element of the dual $`𝔱^{}`$.

* Proof . Set $`\delta =s_\alpha (\beta )`$, $`(s_\alpha s_\beta (\lambda ),\delta ^{})<0`$. Thus $`Y_\delta 𝔤_{[v_{w(\lambda )}]}`$ and $`Y_\delta 𝔤_x`$.

In order to get: $`Y_\delta +Y_\alpha 𝔤_x`$, we are going to prove

$$(\lambda ,\alpha ^{})=(\lambda ,\beta ^{}).$$
(3)
Set $`\lambda =m\omega _\alpha +n\omega _\beta `$ ($`m,n>0`$) and consider the variety $`Cl(TU_\alpha x)`$. The arguments used in the proof of the previous corollary give a weight $`\lambda _isuppx`$ such that $`Y_\alpha ^rv_{\lambda _i}`$ is of weight $`\lambda \beta `$ (for $`r>0`$). Translating the latter in terms of equations, we get $`\lambda _i=\lambda \alpha _n/m\delta `$ and $`n/m𝐍`$. Similarly, considering the variety $`Cl(U_\alpha x)`$, we get $`m/n𝐍`$ thus $`m=n`$ and also

$$Y_\alpha v_\lambda =qY_\delta v_{\lambda _i}q𝐂^{}.$$
(4)
Finally, let $`Z_t`$ be the variety $`Cl\left(T\mathrm{exp}t(Y_\delta +qY_\alpha )x\right)`$, $`t𝐂`$. Its support is entirely contained in the triangle of vertices $`\lambda `$, $`s_\alpha (\lambda )`$ and $`w(\lambda )`$. But there is a gap in this support: the weight $`\lambda \alpha `$ is missing because of (4). This implies that $`\mathrm{exp}t(Y_\delta +qY_\alpha )G_x`$.

To show that $`U_\gamma G_x`$ for all $`\gamma \mathrm{\Phi }\{\alpha ,\delta \}`$, we proceed by contradiction and as before, we will find a gap in the support of $`Cl(U_\gamma x)`$.

To conclude, we have to note that if $`\alpha ,\beta `$ is not of type $`A_1\times A_1`$, then $`Y_\alpha +Y_{s_\alpha (\beta )}𝔤_x`$. The first assertion is obtained just by symmetry. $`\mathrm{}`$

As a consequence of the previous proof, we have

###### Corollary 6.5

(i)$`(\lambda ,\alpha ^{})=(\lambda ,\beta ^{})`$;

(ii)$`T_x^{}=\mathrm{ker}(\alpha +s_\alpha (\beta ))^{}`$;

(iii)$`N_G(H)/H`$ is finite;

(iv)$`Cl(Lx)`$ is a cuspidal two-orbit $`L`$-variety.

The main thing to do, in order to get the cuspidal two-orbit varieties of type II, is to give the list of the possible $`(G,\alpha ,\beta )`$ where $`\alpha `$ and $`\beta `$ are the positive roots considered previously. If $`G`$ is of rank $`2`$, it is done already by Proposition 6.3. So the acting group $`G`$ will be definitely of rank greater than $`2`$. Sum up the properties of $`\alpha `$ and $`\beta `$:

1) $`\alpha \mathrm{\Delta }`$, $`(w(\lambda ),\alpha )0`$ and $`U_\alpha G_x`$;

2) $`\lambda w(\lambda )\alpha ,\beta _{}`$, the $``$-vector space spanned by $`\alpha `$ and $`\beta `$;

3) $`\{\alpha ,\beta \}`$ basis of $`\alpha ,\beta `$;

4) $`supp\beta \{\alpha \}=\mathrm{\Delta }`$;

5) $`w=s_\alpha s_\beta `$;

6) $`(\lambda ,\alpha ^{})=(\lambda ,\beta ^{})`$.

It will appear quickly that there are very few roots satisfying all these conditions mainly because of the two following statements.

* Remark. Let $`\gamma _1`$, $`\gamma _2`$ and $`\gamma _3`$ be three roots of $`\mathrm{\Phi }`$ spanning a root system $`\mathrm{\Psi }`$ of rank $`3`$. Suppose $`\mathrm{\Psi }`$ verifies the property:

$$\eta =n_1\gamma _1+n_2\gamma _2+n_3\gamma _3\mathrm{\Psi }\eta n_3\gamma _3\mathrm{\Phi }\text{(up to a scalar).}$$

Then, $`\gamma _1,\gamma _2_{}\gamma _3,\eta _{}`$ is generated by a root or is equal to $`\{0\}`$, if $`\eta `$ is a root of $`\mathrm{\Phi }`$.

Let us consider a simple root $`\delta `$ such that $`(\beta ,\delta )>0`$ (then $`\delta supp\beta `$). Set $`\gamma =s_\alpha (\delta )`$. Then

$$(w(\lambda ),\gamma )0.$$
(5)
###### Lemma 6.6

If $`\alpha `$, $`\beta `$ and $`\gamma `$ satisfy the property given in the remark with $`\gamma =\gamma _3`$ then $`(w(\lambda ),\gamma )=0`$.

* Proof . If $`(w(\lambda ),\gamma )0`$ then according to (5), $`(w(\lambda ),\gamma )<0`$. And therefore, there exists $`\eta \mathrm{\Phi }`$ such that $`\lambda w(\lambda )\gamma ,\eta _{}`$ (argue similarly as we did to get the root $`\beta `$). It implies: $`\lambda w(\lambda )\alpha ,\beta _{}\gamma ,\eta _{}`$. But this is impossible because of the remark and the fact that $`T_x^{}`$ is regular (*see* Proposition 3.4). $`\mathrm{}`$

Start with $`G`$ classical and suppose: $`(\alpha ,\delta )=0`$. If $`supp\beta =\mathrm{\Delta }`$ then $`\alpha `$, $`\beta `$ and $`\delta `$ satisfy the conditions of Lemma 6.6. Therefore we get: $`(\lambda ,\delta )=(\lambda ,\beta )`$. But this equality is incompatible with $`(\lambda ,\alpha ^{})=(\lambda ,\beta ^{})`$. So assume that $`supp\beta =\mathrm{\Delta }\{\alpha \}`$. Then we have the following possibilities for $`(G,\alpha ,\beta )`$:

$``$$`(A_n,\alpha _i(i=1,n),\stackrel{~}{\alpha }\alpha _i)`$;

$``$$`(G,\alpha _n,\epsilon _1\epsilon _n)`$ for $`G=B_n,C_n`$;

$``$$`(G,\alpha _1,\epsilon _2+\epsilon _n)`$ for $`G=B_n,C_n`$;

$``$$`(G,\alpha _1,\epsilon _2+\epsilon _j(2<j<n))`$ for $`G=B_n`$, $`C_n`$, $`D_n`$;

$``$$`(D_n,\alpha _{n1},\epsilon _1+\epsilon _n)`$;

$``$$`(D_n,\alpha _n,\epsilon _1\epsilon _n)`$.

Applying Lemma 6.6, we end up again with a contradiction. Thus necessarily, $`(\alpha ,\delta )<0`$. The possible triples $`(G,\alpha ,\beta )`$ are now:

$``$$`(A_3,\alpha _2,\stackrel{~}{\alpha })`$;

$``$$`(G,\alpha _2,\epsilon _1+\epsilon _3)`$ for $`G=B_n`$, $`C_n`$ or $`D_n`$;

$``$$`(B_3,\alpha _3,\stackrel{~}{\alpha })`$;

$``$$`(B_n,\alpha _1,\epsilon _2)`$;

$``$$`(C_n,\alpha _2,\stackrel{~}{\alpha })`$;

$``$$`(C_n,\alpha _1,2\epsilon _2)`$;

$``$$`(D_n,\alpha _1,\stackrel{~}{\alpha })`$.

* Claim. In all these cases, $`(w(\lambda ),\gamma )0`$.

But because of Lemma 6.6, we may also have: $`(w(\lambda ),\gamma )=0`$ if $`\alpha `$, $`\beta `$ and $`\gamma `$ satisfy the good conditions. So we have to reduce the list to

$``$$`(A_3,\alpha _2,\stackrel{~}{\alpha })`$;

$``$$`(C_3,\alpha _2,\epsilon _1+\epsilon _3)`$;

$``$$`(B_3,\alpha _3,\stackrel{~}{\alpha })`$;

$``$$`(C_n,\alpha _2,\stackrel{~}{\alpha })`$;

$``$$`(D_n,\alpha _1,\stackrel{~}{\alpha })`$.

The fourth triple is ruled out just by considering $`(w(\lambda ),\alpha _1)`$. The other ones give raise to some pairs of Table 2.

For the exceptional case, we proceed similarly and we obtain the left pairs of Table 2. This ends the proof of the main theorem: the classification of two-orbit varieties.

Appendix

To make Table $`3`$ and Table $`4`$ readable to the reader, we will need the following notation.

We shall recall, at first, that, in these tables, the two-orbit varieties are *cuspidal* with *connected* generic stabilizers. All other two-orbit varieties are obtained (*see* section 2) either by parabolic induction or as the quotient of a cuspidal two-orbit variety by $`H/H^{}`$ for $`H^{}`$ the corresponding connected generic stabilizer given in Tables $`1`$ and $`2`$.

In the first column, we have listed the type of the group $`G`$ acting on the two-orbit variety designed in the second column. When the action is obvious, we shall not state it precisely.

Once we have the acting group, we fix a Borel subgroup $`B`$ and a maximal torus in it and use the standard notation (*see* ) to denote $`\alpha _i`$ the simple roots, $`\omega _i`$ the fundamental weight corresponding to $`\alpha _i`$, $`s_i`$ the simple reflection of the Weyl group associated to $`\alpha _i`$ and $`P_i`$ the maximal parabolic subgroup attached to $`\alpha _i`$.

The Grassmaniann of $`m`$-planes in $`^n`$ is denoted by $`Gr(n;m)`$. For instance, $`Gr(n;1)`$ is just $`^n`$ and $`Gr(n;n1)`$ is $`^n^{}`$.

We have a natural action of $`SO_n`$ on the quadric $`Q(n)=\{[z_0:z_1:\mathrm{}:z_n]^n:z_0^2=z_i^2\}`$ given by $`g[z_0:z^{}]=[z_0:gz^{}]`$, $`gSO_n`$ and $`z^{}^n`$.

As usual, $`\mathrm{\Sigma }_k`$ denotes the Hirzebruch surface: the $`2`$-dimensional smooth complete torus embedding corresponding to the integer $`k`$. In particular, when $`k=1`$, the normalization of $`\mathrm{\Sigma }_k`$ is just $`^2`$.


format: text

text:
# The Free Electron Maser in Pulsar Magnetospheres

## 1 Introduction

A strong source for coherent radiation in laboratory is the free electron laser (FEL) \[(O’Shea and Freund 2001)\]. Its principle is the interaction of a relativistic electron beam (REB) and a magnetic wiggler system. The beam of high energy electrons with energy $`\gamma m_\mathrm{e}c^2`$ passes between permanent magnets of alternating polarity and periodicity $`\lambda _\mathrm{W}`$. The energy of the REB is efficiently converted by electron bunching. In the observer’s frame the bunches emit the relativistically Doppler-shifted wavelength $`\lambda =\lambda _\mathrm{W}/\gamma ^2`$. A comparable effect can be driven by a REB penetrating a strongly magnetized plasma. In phase space such a configuration corresponds to a two-stream-instability. In the plasma version the free energy of the emitted radiation is supplied by the REB, as it is in the case of the FEL, but the positive gradient of the momentum distribution function $`df/dp>0`$ is the source of the instability which excites strong density fluctuations. In a strongly magnetized plasma such density oscillations are predominately longitudinal Langmuir waves which grow to nonlinear amplitudes \[(Benford and Weatherall 1991), (Kato et al. 1983), (Weatherall 1988)\]. The interaction of the REB with the electrostatic fluctuations of the Langmuir waves results in very strong coherent electromagnetic waves \[(Kato et al. 1983), (Levron et al. 1987)\]. In the rest frame of the beam electrons, they experience the Doppler shifted electrostatic wave moving towards them. The nonlinear waves force the REB to wiggle and to emit dipole radiation of the same frequency as the waves. In the laboratory frame this Hertz’ dipole radiation is again Doppler boosted and strongly beamed in forward direction due to the relativistic lighthouse effect. This interaction can be viewed as coherent inverse Compton scattering (CICS) off Langmuir waves \[(Benford and Weatherall 1991)\]. Coherence occurs due to the bunch structure of the beam which means a phase coupled electrostatic field and density modulation.

## 2 Computational Details

Since CICS has been discussed to be promising candidate for the origin of the extremely coherent radio emission from pulsars \[(Kunzl et al. 1998), (Melrose and Gedalin 1999), (Melrose 2000)\], we use the physical parameters of a pulsar magnetosphere as an instructive application. Radio observations indicate that at about 500 km from the strongly magnetized neutron star the coherent pulsed radio emission is produced. The coherence is proved by pulse substructures on time scales down to a few tens of nanoseconds with increasing flux densities\[(Rickett 1995)\]. In these regions of a pulsar magnetosphere typical values are: background electron density $`n_\mathrm{e}=10^{12}\mathrm{m}^{\text{-}3}`$, beam electron density equal to the plasma density $`n_\mathrm{b}=10^{12}\mathrm{m}^{\text{-}3}`$, beam electron Lorenz factor $`\gamma =\sqrt{5}`$ ($`\gamma \beta =2`$) and an extremely strong background magnetic field $`B`$ of about $`1000\mathrm{T}`$. Especially the strong guiding field is important for the efficiency of the CICS-process. It reduces the electron dynamics to the spatial dimension along the magnetic field lines which greatly improves the stability of the excited Langmuir waves and thus leads to strong and stable electrostatic wiggler fields \[(Pelletier et al. 1988)\]. For the temperature of both the beam– and background electrons values we take the value of $`T_\mathrm{e}=T_\mathrm{b}=100\mathrm{eV}`$, typical polar cap temperature of neutron stars derived from their thermal X-ray-emission \[(Pavlov and Petekhin 1995)\]. The PIC–code used for this numerical simulation is fully three dimensional and it conserves mass and energy. In the simulation the direction of electron beam propagation $`\gamma 𝜷`$ and of the magnetic field $`𝑩`$ is the positive $`z`$–direction, which is also called the longitudinal direction. The simulated box has a numerical extension of $`80\times 80\times 200`$ grid points and a physical extension of $`100\mathrm{m}\times 100\mathrm{m}\times 250\mathrm{m}`$, which corresponds to the size of the expected features. The resolution of $`\delta x=\delta y=\delta z=1.25\mathrm{m}`$ is more than sufficient to resolve the expected maximum wavelengths of approximately $`\lambda _{\mathrm{Rad}}=2\pi /k_{\mathrm{Rad}}=2\pi c/\gamma ^2\omega _{\mathrm{pe}}6.7\mathrm{m}`$. A particle density of $`10^{12}\mathrm{m}^{\text{-}3}`$ is approximated by 10 quasi particles per cell, leading to 16 million background quasi electrons that are accompanied by a varying number of beam quasi electrons of not more than 2.5 million, which eventually enter the computational box. Those numbers are doubled, since for quasi neutrality an equally large number of positive particles are required.

The initial condition is given by a homogeneous background plasma of density $`n_\mathrm{e}`$, temperature $`T_\mathrm{e}`$ and a longitudinal magnetic field $`B_z=0.1\mathrm{T}`$. This value is found to be sufficiently high to represent a much stronger field of 1000 $`\mathrm{T}`$. Initially there is no beam in the computational box. The beam is injected later at the $`z=0`$ plane of the box with a density profile of $`n\left(r\right)=n_\mathrm{b}/\left(1+\mathrm{exp}\left[\frac{rR}{\mathrm{\Delta }R}\right]\right)`$. Here $`r`$ denotes the distance from the $`x`$$`y`$ center of the box and $`R=20\mathrm{m}`$ and $`\mathrm{\Delta }R=3\mathrm{m}`$ give the radius and the sharpness of the electron beam. The homogeneous background plasma is disturbed in phase space (see Fig. 1)

in order to accelerate the excitation of Langmuir waves by giving the system a hint of the relevant length scales. We used the wave length of the fastest growing Langmuir mode, which follows from an analytic calculation of the linear growth rate. The disturbance is chosen not to induce any finite charge or current densities, thus leaving the background plasma initially homogeneous, neutral and current free.

The boundary conditions during the simulation are quite simple. In the transversal directions $`x`$ and $`y`$ we chose periodic boundaries and in the longitudinal direction $`z`$ the box is open, which means that plasma particles and fields can leave the box unaffected. In addition at the lower $`z`$ plane ($`z=0`$) an electron component is continuously injected with the density profile mentioned above, a mean $`\gamma \beta `$ of 2 and the temperature $`T_\mathrm{b}`$.

The simulation runs for 600 time steps with $`\delta t=2.41\mathrm{ns}`$. The total time simulated was $`1.44\mu \mathrm{s}`$, after which the growing nonlinear Langmuir waves saturate and a quasi stationary situation had developed as it is shown in the next section.

## 3 Results

The beam particles entering the computational box interact with the faint traces of fluctuations we introduced in our initial setup and excite the appropriate modes to grow. In Fig. 7 on page 7 the temporal evolution of the electrons phase space in $`z`$–direction is presented by showing phase diagrams for the time steps 150, 300, 325 and 400. In the first plot (upper left) the penetrating electron beam can be seen together with the background plasma, both are still nearly undisturbed. In the following plots the development of strong Langmuir waves can be observed. After 781 $`\mathrm{ns}`$, the disturbance in the beam and the background has grown so strong, that both components start to mix, i.e. there is no distinction between beam and background electrons anymore. Beam electrons are strongly decelerated even until $`v0`$, whereas background electrons are accelerated up to a Lorenz factor of 5. This can be seen best in Fig. 7 on page‘7, where each of the both components are plotted in separate diagrams for the time step 400. In Fig. 7 left the phase space of the beam electrons is shown. Clearly visible are the ring structures of fully developed Langmuir waves. Some of beam electrons are accelerated up to a $`\gamma \beta `$ of 4–5 as it is the case for the background electrons, which is shown in Fig. 7 right. On the other hand the beam electrons are also decelerated to zero momentum during their interaction with the Langmuir waves. The extremely strong accelerations are due to the strong electrostatic fields of the excited Langmuir waves. Since the Langmuir turbulence structure in Fig. 7 moves with a much lower speed (which is even continuously decreasing throughout the simulation) than the yet unaffected beam electrons, it acts as the wiggler field that stimulates the CI CS-emission. The electrostatic fluctuations directly correspond to electron density fluctuations (solid curve in Fig. 7), which reach a value of $`+100\%/50\%`$ oriented at $`n=n_\mathrm{e}+n_\mathrm{b}=210^{12}\mathrm{m}^{\text{-}3}`$. The structure of the electron density is better seen in Fig. 8 (page 8) and Fig. 9 (page 9), where the temporal evolution of contour surfaces at $`n=1.510^{12}\mathrm{m}^{\text{-}3}`$ and $`n_\mathrm{e}=1.910^{12}\mathrm{m}^{\text{-}3}`$ and of a cut through the electron beam is shown. The electron density varies between $`4`$ and $`1`$ times $`10^{12}\mathrm{m}^{\text{-}3}`$, which implies the strong electrostatic fields. It can be clearly seen, that the beam decays in a series of ”pancakes”. Such pancakes are absolutely necessary to explain the coherent nature of the pulsed radio emission of neutron stars. Why? Since the emission from relativistic particles is confined to a forward cone with half angle $`1/\gamma `$ and the emission is nearly along the magnetic field. Thus, in Fourier space bunching emission corresponds to the component $`k_{}`$ to the fields being smaller than the component $`k_{}`$ along the field line by a factor $`1/\gamma `$. This corresponds to flat pancake shaped bunch with the normal within an angle $`1/\gamma `$ of $`B`$ \[(Melrose 2000)\]. This is exactly what we observe in our simulations! We briefly note that our simulations are the first ones which show this kind of pancake bunches! The number of particles in such a pancake corresponds to the the maximum coherence achievable for CICS and it should be noted, that in our simulation the thickness of such a pancake is significantly larger ($`1\mathrm{m}`$) than the Debye length which is usually used in analytical models. A typical volume of one bunch is about $`10m^3`$ giving a total number of particles per bunch of about $`10^{13}`$. Our simulations prove, that a much higher coherence is achievable than expected from analytical models.

The growth of Langmuir waves can be deduced by the Fourier transform of the electrostatic field $`\stackrel{~}{E}_z`$. In Fig. 2

the energy density of the electrostatic modes is plotted versus the corresponding wavenumber $`k`$ and time $`t`$. The $`k0.25\mathrm{m}^{\text{-}1}`$ mode is growing by four orders of magnitudes within some ten nanoseconds. It should be noted however, that the fastest growing mode is slightly off compared to the initially given wave number can be seen in Fig. 2 as the small peak at early times. This means, that for our simulation the analytical value for the fastest growing mode is not precisely correct and it proves, that although the initial disturbance has given a hint to the system it has not forced it towards an unphysical solution. We note that the wave energy density is distributed to higher $`k`$ when it exceeds a value of about $`10^{\text{-}7}\mathrm{J}\mathrm{m}^3`$, driven by nonlinear wave–wave interactions.

It is important to recognize that the generated electrostatic fluctuations react on the beam electrons similar to the wiggler fields in the laboratory FEL, with one major difference. In our case the wave wiggling self-consistently excited is along the propagation direction of the beam, whereas in the FEL-case the external magnetic wiggling is perpendicular to the beam propagation. The strongest emission originates in regions with the largest density gradients Fig. 3.

Every de– and acceleration region forces the beam electrons to emit Hertz’ dipole radiation, which is beamed in forward direction, as can be seen in Fig. 4 (right).

$`\theta _\mathrm{M}`$ represents the angle of maximum intensity, expected from theoretical consideration ($`\mathrm{tan}\theta _\mathrm{M}=1/2\gamma \beta `$) with a $`\gamma \beta =2`$. The striking resemblance with a relativistically beamed Hertz’ dipole is obvious, but the emission characteristic in our simulation also gives precisely the expected opening angle of the emission cone, which is proposed for a CICS-process. In Fig. 4 (left) the ”spectrum” of the emitted radiation is shown, which has a strong peak at $`\omega =5\omega _{\mathrm{pe}}`$ and a couple of minor peaks at $`4\omega _{\mathrm{pe}}`$ and $`3\omega _{\mathrm{pe}}`$ due to nonlinear wave coupling. Again we find rather precisely the theoretically expected value of $`w=\gamma ^2\omega _{\mathrm{pe}}`$. The total power emitted by the simulated computational box is shown in Fig. 5.

It is the escaping poynting flux at the top of the computational box ($`z=250\mathrm{m}`$) integrated over the top surface. After about $`850\mathrm{ns}`$ the emitted power rises within some ten nanoseconds by more than $`6`$ orders of magnitude to a value of $`1\mathrm{GW}`$. Afterwards the total output power increases more slowly by one more order of magnitude and reaches a maximum of $`10\mathrm{GW}`$ continuous radiative power at the end of the simulation. This is a significant fraction of the power injected by kinetic energy of the beam particles, shown in Fig. 5 as $`+`$ symbols at a constant value of $`50\mathrm{GW}`$. The rest of the energy leaves the box as kinetic energy of beam electrons, which would probably be also emitted if the computational would have a greater size in z-direction.

## 4 Discussion

We have shown that in a strongly magnetized plasma a relativistic electron beam can be forced to emit highly coherent radio emission by self-induced nonlinear density fluctuations. Such slowly moving nonlinear structures oscillate with the local plasma frequency at which the relativistic electrons are scattered. Beam electrons dissipate a significant amount of their kinetic energy by inverse Compton radiation at a frequency of about $`\gamma ^2\omega _{\mathrm{pe}}`$. Since the beam is sliced into pancake structures which experience the same electric field the inverse Compton scattering is coherent. Such a process is a very promising candidate for the coherent radio emission of pulsars.

## Acknowledgments

###### Acknowledgements.

The authors would like to thank the John von Neumann Institute for Computing (NIC) and the Leibniz Rechenzentrum München for the granted computational time and the Deutsche Forschungsgemeinschaft which supported our work through the grant LE 1039/3-1.


text:
# Nonequilibrium Green function modelling of transport in mesoscopic systems1footnote 11footnote 1Based on a talk presented at the conference “Progress in Nonequilibrium Green Functions, Dresden, Germany, 19.-22. August 2002”

## I Introduction

### I.1 Some background, and a few historical remarks

A mesoscopic transport measurement is often concerned with a situation where the ”device”, whose properties are the subject of the investigation, is connected to structureless ”contacts” via ”ideal leads”, i.e., in a situation which is accessible via the Landauer formula (or, more generally, the scattering approach to transport). An important feature is the fact that the size of the device is finite, and comparable to other important length-scales of the system, such as the phase-braking length or impurity mean free path. Thus the wave-properties of the charge carriers are important, leading to a number of interesting interference effects, such as weak localization, or universal conductance fluctuations. The conductance $`g`$ of the device, for example, is then given by the celebrated Landauer formulaLand57 ; Land70

$$g=\frac{2e^2}{\mathrm{}}|t|^2,$$
(1)
where $`t`$ is the quantum mechanical transmission amplitude through the device.

The expression given above holds for the one-channel case, for low applied voltages, and in a situation when the ”device” can be modelled by a noninteracting system. It is then natural to ask: Can this equation be generalized to the case, when interactions are important? Or: Can this equation be extended to time-dependent situations, or situations where superconductivity or magnetism are essential for the physics? How about strong driving fields?

Many authors have addressed these questions, with a number of different approaches. In the spirit of this conference, this review is concerned with the subset of these theories which use the nonequilibrium Green function technique. The earliest applications to mesoscopic transport, to my knowledge, are due to French researchersCaro71a ; Caro71b ; Caro72 ; Comb71 : these researchers were mainly interested in inelastic effects in tunneling through oxide barriers. It is a curious side-note to observe that these early, and pioneering, papers were essentially forgotten during the 80’s, but have obtained a substantial revival since mid-90’s, and are presently cited more often than ever earlier. The explanation lies perhaps in the fact that the whole idea of mesoscopics is newer than these early papers, and it took the mesoscopic community a few years to realize the applicability of these ideas.

For the purposes of the present review, the next important development was the paper by Meir and WingreenMeir92 , which gave a very useful formal expression for the current in terms of the exact Green function of the device (or ”central region”). This formula, and similar expressions obtained by other groups, were then applied to the Kondo problem out of equilibrium, a notoriously difficult problem, which remains a topic of active research even today. My previous reviewJauh00  in the first meeting of the present series focused in some of these issues, paying particular attention to the time-dependent generalization of the Meir-Wingreen expressionJauh94 . I shall not repeat any of that material, but rather focus on other topics: the selection criterion has been that either they were not discussed during the first meeting, or that they are strictly post-1999 vintage. I have chosen to discuss three examples: (i) Transport in a semiconductor superlattice; (ii) Calculation of the noise in a spintronic system; and (iii) Tunneling transport in a nanoelectromechanical (NEMS) device. These three topics have a common feature: they all have practical applications, and, dare I say in the present meeting, even commercial potential.

It should be noted that there are many other recent applications of the NGF to mesoscopic transport which, due to space and time limitations, the present review does not address. One such example is transport in nanowires, fabricated either with scanning tunnelling microscope, or with break junctions, which is a large research field where many theoretical calculations emply the NGF techniques. An exhaustive review has recently become availableAgra02 . Molecular electronics holds enormous potential, and here a combination of ab initio electronic structure calculations (within the density-functional scheme) and NGF appears to the most promising theoretical techniquePaul02 ; Bran02 . Yet another example is the ”circuit theory” developed by NazarovNaza94 ; Naza99 , which has successfully been applied to a number of hybrid structures, consisting of superconductors, ferromagnets, or semiconductors. The more abstract field-theoretic formulations, based on path-integrals and/or Grassmann variables fall also outside our present purposes, even though they play an important role in the study of disordered systems, or dephasing due to the environment.

### I.2 The basic equations, and their limitations

For completeness, we sketch here a derivation of the basic expressions used in the theory. Several more complete accounts are available elsewhereJauh00 ; Jauh94 ; Haug96 . A brief reminder of how the nonequilibrium formalism works in the context of mesoscopic transport measurements is also in place. One reasons as follows. In the remote past the contacts and the central region (i.e., the ”device”) are assumed to be decoupled, and each region is in thermal equilibrium. The equilibrium distribution functions for the three regions are characterized by their respective chemical potentials; these do not have to coincide nor are the differences between the chemical potentials necessarily small. The couplings between the different regions are then established and treated as perturbations via the standard techniques of perturbation theory. The nonequilibrium nature of the problem manifests itself in that symmetry of remote past and remote future has been broken, and thus one must do the calculations on the two-branch time contour. It is important to notice that the couplings do not have to be small, e.g., with respect level to spacings or $`k_\mathrm{B}T`$, and typically must be treated to all orders.

Let us next consider some generic Hamiltonians: $`H=H_L+H_R+H_T+H_{\mathrm{cen}}`$, or, explicitly:

$`H`$ $`=`$ $`{\displaystyle \underset{k,\alpha L/R}{}}ϵ_{k,\alpha }c_{k,\alpha }^{}c_{k,\alpha }`$ (2)
$`+{\displaystyle \underset{k,\alpha L/R;n}{}}[V_{k\alpha ;n}c_{k,\alpha }^{}d_n+\mathrm{h}.\mathrm{c}.]+H_{\mathrm{cen}}[\{d_n\},\{d_n^{}\}],`$
where the central part Hamiltonian must be chosen according to the system under consideration. The operators $`\{d_n\},\{d_n^{}\}`$ refer to a complete set of single-particle states of the central region. Occasionally we specify explicitly the orbital and spin quantum numbers: $`n=m,\sigma `$, and analogously for the states in the leads. The derivation of the basic formula for the current does not require an explicit form for $`H_{\mathrm{cen}}`$; the actual evaluation of the formula of course requires this information. We write $`H_{\mathrm{cen}}=_nϵ_nd_n^{}d_n+H_{\mathrm{int}}`$, where $`H_{\mathrm{int}}`$ could be electron-phonon interaction,

$$H_{\mathrm{int}}^{\mathrm{el}\mathrm{ph}}=\underset{m\sigma }{}d_{m,\sigma }^{}d_{m,\sigma }\underset{𝐪}{}M_{m,𝐪}\left[a_𝐪^{}+a_𝐪\right],$$
(3)
or an Anderson impurity:

$$H_{\mathrm{int}}^A=U\underset{m}{}d_{m,}^{}d_{m,}d_{m,}^{}d_{m,}.$$
(4)
The current operator for the (say) left lead is

$`I_L`$ $`=`$ $`e\dot{N}_L={\displaystyle \frac{ie}{\mathrm{}}}[H,N_L]`$ (5)
$`=`$ $`{\displaystyle \frac{ie}{\mathrm{}}}{\displaystyle \underset{k,n}{}}\left[V_{kL;n}c_{kL}^{}d_n+V_{kL;n}^{}d_n^{}c_{kL}\right].`$
The physically relevant observables can be expressed in terms of expectation values of the current operator, or its higher powers. For example, one can showJauh94 ; Haug96  that the current leaving the left contact is

$`I_L=J_L(t)`$ $`=`$ $`{\displaystyle \frac{2e}{\mathrm{}}}{\displaystyle _{\mathrm{}}^t}dt_1{\displaystyle }{\displaystyle \frac{dϵ}{2\pi }}\mathrm{ImTr}\{e^{iϵ(t_1t)}𝚪^L(ϵ,t_1,t)`$ (6)
$`\times [𝐆^<(t,t_1)+f_L^0(ϵ)𝐆^r(t,t_1)]\}.`$
Here the Green functions are defined by

$`G_{nm}^<(t,t_1)`$ $`=`$ $`id_m^{}(t_1)d_n(t)`$ (7)
$`G_{nm}^r(t,t_1)`$ $`=`$ $`i\theta (tt_1)[d_n(t),d_m^{}(t_1)],`$ (8)
$`\mathrm{\Gamma }_{mn}`$ describes the coupling between the central region and the contacts, and $`f_L^0(ϵ)`$ is the equilibrium distribution function of the left contact. In the dc-limit, (6) reduces to the result of Meir and WingreenMeir92 :

$`J`$ $`=`$ $`{\displaystyle \frac{ie}{2\mathrm{}}}{\displaystyle }{\displaystyle \frac{dϵ}{2\pi }}\mathrm{Tr}\{[𝚪^L(ϵ)𝚪^R(ϵ)]𝐆^<(ϵ)`$ (9)
$`+[f_L^0(ϵ)𝚪^L(ϵ)f_R^0(ϵ)𝚪^R(ϵ)][𝐆^r(ϵ)𝐆^a(ϵ)]\}`$
$`=`$ $`{\displaystyle \frac{ie}{\mathrm{}}}{\displaystyle \frac{d\epsilon }{2\pi }\left[f_L(\epsilon )f_R(\epsilon )\right]T(\epsilon )},`$ (10)
where

$$T(\epsilon )=\mathrm{Tr}\left\{\frac{𝚪^L(\epsilon )𝚪^R(\epsilon )}{𝚪^L(\epsilon )+𝚪^R(\epsilon )}\left[𝐆^r(\epsilon )𝐆^a(\epsilon )\right]\right\}.$$
(11)
The expressions (6) and (10) are the central formal results whose consequences we explore in this review.222In the Section 3 we also give an analogous expression for the noise spectrum. They are formally exact, and give the tunneling current for an interacting system coupled to noninteracting contacts (or, more precisely, for contacts which can be described by an effective single-body Hamiltonian). Thus, in a time-dependent situation the displacement current must be considered separately. It should also be noted that these equations only define the starting point of any calculation: to get into physical results one must evaluate the correlation function and the retarded/advanced Green function, Eqs.(7) and (8), respectively. These functions obey the Keldysh equation, and the (nonequilibrium) Dyson equation:

$`G^<`$ $`=`$ $`G^r\mathrm{\Sigma }^<G^a,`$ (12)
$`G^r`$ $`=`$ $`G_0^r+G_0^r\mathrm{\Sigma }^rG^r.`$ (13)
The success of the theory depends on whether one can construct a self-energy functional that captures the essential physics, and that a good solution can be found for these coupled equations. Both of these steps may be hard indeed.

## II Transport in a Semiconductor Superlattice

In 1970 Esaki and TsuEsak70  suggested that semiconductor superlattices, man-made structures which consist of alternating layers of different semiconductor materials, would have physical properties which could be used for a number of device applications. Very shortly, the spatial variations in the band-gaps will lead to a spatially varying conduction band edge, which supports minibands, which in turn diplay very interesting transport properties, such as Bloch oscillations, or negative differential resistance. A well known-result is the Esaki-Tsu IV-characteristic,

$$I(V)=2I_{\mathrm{max}}V_0\frac{V}{V^2+V_0^2},$$
(14)
where $`I_{\mathrm{max}}`$ and $`V_0`$ depend on the physical paratmeres of the system, such as the superlattice period, scattering rate, and temperature. To derive expressions like this, three main approaches have been used in the literature: (i) Miniband transportEsak70 , (ii) Wannier-Stark hoppingTsu75 , and (iii) sequential tunnelingMill94 . The three different approaches have different domains of validity, and are all likely to fail if the three basic energy scales, i.e. scattering induced broadening, miniband width, and potential drop per period all have comparable values. The basic features of these three approaches are summarized in Figure 1.

In order to map out the boundaries of the various domains of validity, and to access the region where the approaches (i)–(iii) fail, a higher level theory is required. To achieve this, Andreas Wacker and myself, together with several colleagues, launched a program whose task was to develop a nonequilibrium Green function theory for superlattice transport.333There are other theoretical methods capable of including (some) quantum effects, such as the density-matrix methodKuhn98 ; Bryk97 , or the balance equation approach.Lei91  Certain aspects of this program are now completedWack98 ; Wack99 , and in what follows I will review some of the highlights. It should be noted that the literature on superlattice transport is vast and here I can give only a very superficial discussion; the reader is referred to two recent review articles where a much fuller account can be foundWack02 ; Boni02 .

Let me start with a few disclaimers. The quantum theory has not yet been fully developed to the case when the electric field is inhomogeneous (domain formation), nor is it available for the time-dependent case (photo-assisted transport; progress is however being made see, e.g., Appendix C in the review by WackerWack02 ). For these important situations one has to apply one of the simpler approaches discussed above. As far scattering is concerned, impurity scattering and phonon scattering have been discussed, but carrier-carrier interaction is still a future task.

The task is now to solve the coupled Keldysh and Dyson equations, Eq.(1213). We adopt the tight-binding representation of the single-particle Hamiltonian:

$$H_{n,m}=(\delta _{nm1}+\delta _{n,m+1})T_1+\delta _{n,m}(E_kneFd),$$
(15)
where $`T_1`$ is the nearest neighbor coupling, $`E_k=\mathrm{}^2k^2/(2m)`$ the kinetic energy perpendicular to the growth direction, $`F`$ the applied field, and $`d`$ the superlattice period. In this basis the Keldysh and Dyson equations (1213) read

$`G_{mn}^<(E)`$ $`=`$ $`{\displaystyle \underset{m_1}{}}G_{mm_1}^r\left(E+eFd{\displaystyle \frac{m_1n}{2}}\right)`$ (16)
$`\times `$ $`\mathrm{\Sigma }_{m_1}^<\left[E+eFd\left(m_1{\displaystyle \frac{m+n}{2}}\right)\right]`$
$`\times `$ $`G_{m_1n}^a\left(E+eFd{\displaystyle \frac{m_1m}{2}}\right)`$
$`G_{mn}^r(E)`$ $`=`$ $`g_m^r\left(E+eFd{\displaystyle \frac{mn}{2}}\right)`$ (17)
$`\times `$ $`[\delta _{mn}+{\displaystyle \underset{l}{}}\mathrm{\Sigma }_{ml}^r(E+eFd{\displaystyle \frac{ln}{2}})`$
$`\times G_{ln}^r(E+eFd{\displaystyle \frac{lm}{2}})].`$
Next one needs to specify the self-energies. We have consideredWack98 ; Wack99  impurity scattering, optical phonon scattering, and mimicked acoustic phonon scattering by a very low-energy optical phonon, all in the self-consistent Born approximation. By numerically solving these coupled equations, computing the current, and comparing to the corresponding IV-curves found by the simpler approaches (i)–(iii), we can construct a ”phase-diagram” (see Figure 2), which indicates where the simpler approaches hold, and where a quantum approach is necessary.

We have also compared the quantum mechanical drift–velocity vs. field relation to the results obtained with a semiclassical Monte Carlo simulation. This is quite interesting because the two methods are totally different, and both require computationally rather intensive calculations. Typical results are shown in Figure 3. For the parameters considered here, the Monte Carlo simulation gives very good results, except that it misses the weak phonon replica seen in the quantum calculation.

The approach sketched here is ideally suited to transport phenomena where quantum phenomena, such as resonant tunneling, or phonon-assisted tunneling, play an important role. Another application concerns quantum cascade lasersFais94 , where the current injection occurs through a ”funnel”: the superlattice is designed so that the miniband width varies with distance. Another recent calculation concerns the evaluation of gain in such structuresWack02b .

## III Noise in Spintronics

The emerging field of spintronicsdda02 ; saw01 ; gap98 , where, in addition to the charge, also the electron spin is used to design new devices, has led to fascinating and novel ideas such as spin filtersrf99 ; jce98 ; tk02 , spin field-effect transistorssd90 , and proposals for solid state quantum computingdpv95 . For example, quantum dot systems can in principle be used to control the electron spin and are thus suitable for creating quantum bits relevant for quantum gate operationshae01 .

A detailed theoretical study of nonequilibrium transport properties of spintronic devices is necessary in order to understand the basic physical phenomena and to predict new functionalities. Calculation of the current, for example, can give the conductance/resistance of a system and its dependence on magnetic field, Coulomb interaction, spin-flip and so on. On the other hand, current fluctuations, due to the granularity of the charge (shot noiseymb00 ), are also relevant because their measurements can provide additional information not contained in the average currentclk97 .

Here we illustrate how the nonequilibrium Green function technique can be used to calculate current and its fluctuations (noise)444NGF calculations of noise have been performed by several groupsChen91 ; Hers92 ; Ding97 ; Dong02 . in a quantum dot coupled to two ferromagnetic leads as a function of the applied voltage for parallel (P) and antiparallel (AP) lead-polarization alignments. We include Coulomb interaction in the Hartree-Fock approximation as well as spin-flip in the dot. We show that spin-flip makes the alignment of the lead polarizations less important; both P and AP results coincide for large enough spin-flip rates. This fact gives rise to a reduction of both Fano factor555The Fano factor, defined as $`\gamma =S(0)/2eI`$, characterizes deviations from the Poissonian noise. and tunnelling magnetoresistance (TMR) as we show below.

We model the central region with the Hamiltonian

$$H_D=\underset{\sigma }{}ϵ_0d_\sigma ^{}d_\sigma +Un_{}n_{}+R(d_{}^{}d_{}+d_{}^{}d_{}),$$
(18)
where $`d_\sigma `$ ($`d_\sigma ^{}`$) destroys (creates) an electron in the dot with spin $`\sigma `$ and the energy $`ϵ_0`$ is spin independentwr01 ; pz02 . In addition, we assume that the dot is a small enough in order to have only one active level $`ϵ_0`$. In the presence of a voltage the level shifts by $`ϵ_0=ϵ_d\frac{eV}{2}`$, where $`ϵ_d`$ is the dot level for zero bias. In a more realistic calculation one should determine the bias dependence self-consistently. The spin-flip scattering amplitude $`R`$ is viewed here as a phenomenological parameter. The spin-flip process lifts the degeneracy, splitting the quantum dot level to two states, let us call them $`ϵ_{1,2}`$, with corresponding operators. The current is readily evaluated with the formulas given in Section 1.2 with the result

$$J_L=\frac{2e}{\mathrm{}}\mathrm{Re}𝑑t_2\mathrm{Tr}\{𝐆^r(t,t_2)𝚺^{L<}(t_2,t)+𝐆^<(t,t_2)𝚺^{La}(t_2,t)\},$$
(19)
where $`𝐆^r`$ and $`𝐆^<`$ are the nonequilibrium dot Green functions, with elements $`G_{ij}^<(t,t_2)=id_j^{}(t_2)d_i(t)`$ and $`G_{ij}^r(t,t_2)=i\theta (tt_2)\{d_i(t),d_j^{}(t_2)\}`$. The lesser (retarded, advanced) tunnelling self-energy is given by

$`𝚺^{L<(r,a)}`$ $`(t_2,t)={\displaystyle \frac{1}{2}}{\displaystyle \underset{k}{}}|t_{kL}^2|`$
$`\times `$ $`\left(\begin{array}{cc}g_{kL}^{<(r,a)}(t_2,t)+g_{kL}^{<(r,a)}(t_2,t)& g_{kL}^{<(r,a)}(t_2,t)g_{kL}^{<(r,a)}(t_2,t)\\ g_{kL}^{<(r,a)}(t_2,t)g_{kL}^{<(r,a)}(t_2,t)& g_{kL}^{<(r,a)}(t_2,t)+g_{kL}^{<(r,a)}(t_2,t)\end{array}\right),`$
where $`g_{kL\sigma }^{<(r,a)}`$ is the lesser (retarded, advanced) uncoupled Green function for lead $`L`$. Equation (III) leads to a generalization of the coupling $`\mathrm{\Gamma }`$ found in Section 1.2 above; the coupling matrix now becomes

$$𝚪^L=\frac{1}{2}\left(\begin{array}{cc}\mathrm{\Gamma }_{}^L+\mathrm{\Gamma }_{}^L& \mathrm{\Gamma }_{}^L\mathrm{\Gamma }_{}^L\\ \mathrm{\Gamma }_{}^L\mathrm{\Gamma }_{}^L& \mathrm{\Gamma }_{}^L+\mathrm{\Gamma }_{}^L\end{array}\right).$$
(21)
Accounting for Coulomb interaction in the Hartree-Fock approximation, we can write down a matrix Dyson equation for the retarded Green function, $`𝐆^r=𝐆^{0r}+𝐆^{0r}𝚺^r𝐆^r`$, and a Keldysh equation for the lesser Green function $`𝐆^<=𝐆^r𝚺^<𝐆^a`$, where $`𝐆^{0r}`$ is the uncoupled dot Green function. In these equations the self energies are the sum of the left and right self energies, i.e., $`\mathrm{\Sigma }^{(r,<)}=\mathrm{\Sigma }^{L(r,<)}+\mathrm{\Sigma }^{R(r,<)}`$. A self-consistent calculation is required to calculate $`n_{\overline{i}}`$ and $`d_{\overline{i}}^{}d_i`$, which are given by the lesser Green function, $`d_j^{}d_i=\frac{d\omega }{2\pi }\mathrm{Im}G_{ij}^<(\omega )`$.

The current operator can be written as its average value plus some fluctuation, i.e., $`I_\eta =J_\eta +\delta I_\eta `$ (here $`\eta =L/R`$ labels the contacts). In our system there are two sources of noise, namely, thermal noise and shot noise. The first one is due to thermal fluctuations in the occupations of the leads. It vanishes for zero temperature, but can be finite for $`T0`$ and $`eV=0`$. On the other hand, shot noise is due to the granularity of the electron charge; it is a nonequilibrium property of the system in the sense that it is nonzero only when there is a finite current ($`eV0`$). To calculate the noise (thermal+shot noise) we use the definition $`S_{\eta \eta ^{}}(tt^{})=\{\delta I_\eta (t),\delta I_\eta ^{}(t^{})\}`$, which can also be written as $`S_{\eta \eta ^{}}(tt^{})=\{I_\eta (t),I_\eta ^{}(t^{})\}2J_\eta ^2`$. After a lengthy but straightforward calculationfms02 , we find for the noise power spectrum (dc limit; a scalar version of this equation has been found earlierDong02 )

$$\begin{array}{cc}\hfill S_{\eta \eta ^{}}(0)=\frac{e^2}{\mathrm{}}\frac{d\omega }{2\pi }& \{\delta _{\eta \eta ^{}}in_\eta 𝚪^\eta 𝐆^>\delta _{\eta \eta ^{}}i(1n_\eta )𝚪^\eta 𝐆^<+𝐆^<𝚪^\eta 𝐆^>𝚪^\eta ^{}\hfill \\ & n_\eta (1n_\eta ^{})𝐆^r𝚪^\eta 𝐆^r𝚪^\eta ^{}n_\eta ^{}(1n_\eta )𝐆^a𝚪^\eta 𝐆^a𝚪^\eta ^{}\hfill \\ & 𝐆^<𝚪^\eta [(1n_\eta ^{})𝐆^r(1n_\eta )𝐆^a]𝚪^\eta ^{}\hfill \\ & +(n_\eta 𝐆^rn_\eta ^{}𝐆^a)𝚪^\eta 𝐆^>𝚪^\eta ^{}\}.\hfill \end{array}$$
(22)
The $`dc`$ noise (zero frequency) is position independent, and it is possible to show that $`S_{LL}(0)=S_{RR}(0)=S_{LR}(0)=S_{RL}(0)`$ymb00 . In our numerics we make a few simplifying assumptions. We assume that the couplings $`𝚪^\eta `$ are energy independent, but allow a polarization dependence. For the physical parameters we use accepted values from the current literaturewr01 . Our Hartree-Fock approximation for the electron-electron interaction does not include correlations of the Kondo type, however we do not expect these to change our results in the present range of parameters.

Figure 4 shows current (a) and noise (b) as a function of the bias with $`R=0`$ (solid line) and $`R=0.1U`$ (dotted line) for both P and AP configurations. Because P and AP curves for $`R=0.1U`$ coincide, we plotted only the AP case. The first enhancement of the current and noise at $`eV=U`$ happens when $`ϵ_0`$ crosses the left chemical potential, allowing electrons to tunnel from the emitter (left lead) to the dot and then to the collector (right lead). The current and noise remain constant until the second level $`ϵ_0+U`$ reaches $`\mu _L`$ at $`eV=3U`$, when another enhancement is observed. Each enhancement corresponds to a peak in the differential conductance $`\sigma _{\mathrm{diff}}`$. When the system changes from parallel (P) to antiparallel (AP) configurations the current is reduced. This is a typical behavior of tunnelling magnetoresistance (TMR). The noise is also affected by this resistance variation, showing a similar reduction.

Looking at the effects of spin-flip on current and noise we see that the AP curves with $`R=0.1U`$ (dotted lines) tend to lie on the P curves with $`R=0`$, thus showing that lead alignments are less important when spin-flip plays a part. This AP current enhancement due to spin-flip gives rise to a reduction of the TMR; since $`\mathrm{TMR}=(I_PI_{AP})/I_{AP}`$, when $`I_{AP}I_P`$ we have $`\mathrm{TMR}0`$. For a somewhat simpler model W. Rudziński et al.wr01  found a similar behavior.

## IV Nanoelectromechanical Systems

Microelectromechanical systems (MEMS) are today an important part of our technology. Their functionality is based on combining mechanical and electronic degrees of freedom, the great advantage being that the whole device can be fabricated with standard Si-processing technology. Typical applications include hearing aids, sensors, or actuators. As the fabrication technology gets refined, we soon expect to find systems where the mechanical parts are in the nanometer range, see the review by CraigheadCrai00 ; hence the acronym NEMS. An example is the nanomechanical electron shuttle constructed by Erbe et al.Erbe01  (see Figure 5), based on the theoretical ideas of Gorelik et al..Gore98

Transport through the device was modelled with rate-equationsErbe01 , however these are not expected to work when one enters the coherent transport regime, at lower temperatures and smaller devices, and a quantum theory of transport must be invoked. Such an approach has recently been formulated by Fedorets et al.Fedo02 , and I’ll give a brief introduction to this topic.

The archetypal NEMS device consists of a moving part, and electrodes. We shall model the moving part by a quantum well, whose coupling to the electrodes are position dependent: the tunnelling amplitude is written as $`T_{L/R}(x(t))=\tau _{L/R}\mathrm{exp}[x(t)/\lambda ]`$, where $`\lambda `$ is some characteristic tunnelling length, and $`x(t)`$ is the center-of-mass coordinate of the moving quantum dot. The electronic degrees of freedom are governed by exactly the same Hamiltonians as discussed in Section 2; now the dot-level depends on the dot’s location via $`ϵ_d=ϵ_0Ex(t)`$, where $`E`$ is the electric field, which depends on the applied bias.666In principle a rather complicated electrostatic calculation, including the screening due to all neighboring metallic bodies, is required. The center-of-mass of the dot obeys Newton’s equation of motion,

$$\ddot{x}+\omega _0^2x=F(x)/M,$$
(23)
where $`M`$ is the mass of the grain, $`\omega _0^2=k/M`$ is the characteristic frequency, and $`F`$ is the force, which includes both the electric force acting on the charge(s) on the dot, and an exchange force, which arises due to the $`x`$-dependence of the tunnelling matrix elements. One can evaluate the force via $`F=H/x`$, and the result is

$$F(t)=iEG^<(t,t)+2/\lambda \underset{\alpha ,k}{}(1)^\alpha T_\alpha (t)\mathrm{Im}[G_{\alpha ,k}^<(t,t)],$$
(24)
using the notation of Section 2. The problem is thus (again) reduced to the determination of the two lesser Green function: $`G^<`$ for the dot, and the non-diagonal $`G_{\alpha ,k}^<=ic_{k,\alpha }^{}d(t)`$. Fedorets et al.Fedo02  calculate these for a noninteracting system, and proceed to present an analysis of the mechanical stability of the system: does the dot execute regular oscillations (whose frequencies are determined by an appropriate linearization of (23) and (24), or does it perhaps become unstable, as the bias is increased? The details of the analysis are not our concern here; the upshot is that above a certain threshold value an instability results. The analysis has also bearing on a recent experimentPark00 , where vibronic anomalies were observed in a single-$`\mathrm{C}_{60}`$-transistor when current was passed through it.

The analysis of Fedorets et al.Fedo02  is very interesting and suggests for several further refinements. For example, what are the results for an interacting system (Coulomb blockade)? How about the environmental degrees of freedom? This issue was addressed recently Armour and MacKinnonArmo02 , in a slightly different context. Finally, is it possible to combine the spintronic effects with charge shuttles? Can one envisage a spin shuttle? Will there be a new technology called NEMSS (nanoelectromechanical spin systems)?

## V Conclusions

I have reviewed some of the post-1999 developments in applying NGF to modelling of transport in mesoscopic systems. The common theme in my review has been to focus on ”real devices”, which may have ”real applications”. I find it very pleasing that the NGF technique, often regarded as an academic exercise most suited for theoretical games, is now becoming a strong tool in the analysis of practical devices. This trend is also confirmed by several other talks at this conference. At the same time there is still much room for theoretical refinements, and I’m convinced that in the coming years we will witness significant progress in this field, both abstract and practical.

## Acknowledgments

The author is grateful to acknowledge fruitful collaborations with A. Wacker, and F. Macedo and J. C. Egues, which resulted in the material presented in Sections 2 and 3, respectively.