format: text

text:
# Representations of compact Lie groups and the osculating spaces of their orbits

## 1 Introduction

Several classes of irreducible orthogonal representations of compact Lie groups that are of importance in Differential Geometry have the property that the second osculating space of all of their nontrivial orbits coincide with the representation space. We say that representations with this property are of class $`𝒪^2`$. Our approach in the present paper will be to find restrictions on the class $`𝒪^2`$ and then apply them to classify variationally complete and taut representations. The known classifications of cohomogeneity one and two orthogonal representations and more generally of polar representations will also follow easily. We refer to Section 2 for the definitions of concepts used in this introduction.

Variationally complete actions were introduced by Bott in . Bott and Samelson proved in  that isotropy representations of symmetric spaces are variationally complete. We will prove the following converse of their theorem in Section 6.

###### 1.1 Theorem

A variationally complete representation of a compact connected Lie group is orbit equivalent to the isotropy representation of a symmetric space.

We will in fact prove more than we have stated here and give a complete classification of variationally complete representations (up to image equivalence; see the beginning of Section 5 for this concept). As a corollary we get a new proof of the classification of polar representations due to Dadok () since these are variationally complete (see ) and isotropy representations of symmetric spaces are easily seen to be polar. It follows that an orthogonal representation is variationally complete if and only if it is polar, and it is polar if and only if it is orbit equivalent to the isotropy representation of a symmetric space.

Another important result of Bott and Samelson in  is that the distance functions to orbits of variationally complete representations are perfect Morse functions. We say that submanifolds with this property are taut and call representations taut if all of their orbits are taut. In Section 7 we will prove the following classification theorem which provides new examples of taut submanifolds and taut representations.

###### 1.2 Theorem

A taut irreducible representation $`\rho `$ of a compact connected Lie group $`G`$ is either orbit equivalent to the isotropy representation of a symmetric space or it is one of the following orthogonal representations ($`n2`$):

$$\begin{array}{cc}& \\ G& \rho \\ & \\ \mathrm{𝐒𝐎}(2)\times \mathrm{𝐒𝐩𝐢𝐧}(9)& \text{(standard)}_𝐑\text{(spin)}\\ 𝐔(2)\times \mathrm{𝐒𝐩}(n)& \text{(standard)}_𝐂\text{(standard)}\\ \mathrm{𝐒𝐔}(2)\times \mathrm{𝐒𝐩}(n)& \text{(standard)}^3_𝐇\text{(standard)}\end{array}$$

As a consequence of this theorem, the classes of polar, variationally complete and taut irreducible representations of compact simple groups coincide. Notice that the three exceptional representations in Theorem 1.2 are precisely the irreducible representations of cohomogeneity three that are not polar (see ). Hence, in the case of a general compact Lie group, the taut irreducible representations are precisely those that are either polar or of cohomogeneity three. A further related result is the classification of homogeneous taut submanifolds with flat normal bundle due to Olmos (). He shows that such a submanifold is a principal orbit of the isotropy representation of a symmetric space. It follows that the principal orbits of the three nonpolar representations in Theorem 1.2 do not have flat normal bundle which is also easy to see directly.

Bott and Samelson proved the tautness of variationally complete representations as an application of their $`K`$-cycles which are concrete representatives of a basis for the homology of the orbits. (In  the group acting is denoted by $`K`$ which explains the terminology ‘$`K`$-cycle’). The three nonpolar representations in Theorem 1.2 admit generalizations of the $`K`$-cycles of Bott and Samelson. We will not verify this claim in the present paper since it requires methods quite different from those employed here and would make the paper too long; we refer instead to the forthcoming paper . Instead we will prove their tautness with shorter, although maybe less illuminating arguments based on a theorem of Floyd (). This approach is adapted from the paper  of Duistermaat.

Kuiper observed in  that the second osculating space of a taut submanifold in a Euclidean space $`V`$ coincides with $`V`$ if the submanifold is not contained in a proper affine subspace. In fact, he proved this more generally for tight submanifolds, but this is unimportant for us since an orbit is tight if and only if it is taut. Since the classes of representations we are dealing with are all taut, it follows from this observation of Kuiper that they belong to class $`𝒪^2`$ if they are irreducible. The class $`𝒪^2`$ is much more tractable than the other classes of representations we are dealing with since it involves an infinitesimal condition. The technique of Dadok (), notably his invariant $`k(\lambda )`$, turns out to be an extremely powerful tool to reduce the class $`𝒪^2`$ in size so that the remaining cases are accessible to the geometric methods developed in Section 3.

We have already pointed out that Dadok’s classification of polar representations follows as a consequence of Theorem 1.1. Some other classification results can also easily be proved with our methods as shown in Section 8. These include the classification of cohomogeneity one representations as well as the classification of cohomogeneity two representations due to Hsiang-Lawson (). The cohomogeneity two representations are polar and therefore included in Dadok’s classification, but the point here is that it is very easy to see directly that they belong to class $`𝒪^2`$ without referring to tautness.

One could also use our methods to classify symmetric spaces and their isotropy representations. This would not lead to a simpler proof than those existing in the literature. Insisting on the details needed to classify symmetric spaces and their isotropy representations would make the paper much longer and more involved than it is. We will therefore use this classification in the paper222Except for the classification of cohomogeneity one representations. and assume knowledge of the lists in , Tables 8.11.2 and 8.11.5. (See also , but notice that the isotropy representations are not listed in this reference). Otherwise, our guiding principle in writing the paper has been to make it as self-contained as possible.

Previous to this paper taut representations were studied in  and . It is decided in , with some exceptions, which representations of $`\mathrm{𝐒𝐔}(n)`$ and $`𝐔(n)`$ can be taut. In  it is proved among other things that a compact group admitting an almost faithful taut representation can have at most four simple factors. Eschenburg and Heintze use the theory of isoparametric submanifolds in  to reprove Dadok’s characterization up to orbit equivalence of polar representations under the assumption of cohomogeneity at least three. In  they use the same theory to decide which irreducible representations are orbit equivalent to isotropy representations of symmetric spaces. This follows immediately from our work in Section 6. Bergmann in  decides which reducible representations are polar. Her result is equivalent to the last step in our classification of variationally complete representations. Di Scala and Olmos give in  a direct proof of the fact that a variationally complete representation is polar.

The first author wishes to thank the *Alexander von Humboldt Foundation* for its generous support and constant assistance during the completion of this work.

## 2 A review of basic definitions and results

Let $`G`$ be a compact Lie group acting on a Riemannian manifold $`M`$ by isometries. A geodesic $`\gamma `$ in $`M`$ is called $`G`$-transversal if it is orthogonal to the $`G`$-orbit through $`\gamma (t)`$ for every $`t`$. One can show that a geodesic $`\gamma `$ is $`G`$-transversal if there is a point $`t_0`$ such that $`\dot{\gamma }(t_0)`$ is orthogonal to $`G\gamma (t_0)`$. A Jacobi field along a geodesic in $`M`$ is called $`G`$-transversal if it is the variational vector field of a variation through $`G`$-transversal geodesics. The action of $`G`$ on $`M`$ is called variationally complete if every $`G`$-transversal Jacobi field $`J`$ in $`M`$ that is tangent to the $`G`$-orbits at two different parameter values is the restriction of a Killing field on $`M`$ induced by the $`G`$-action (see  and ). It is proved in  on p. 974 that instead of requiring tangency at two different points in the definition of variational completeness it is equivalent to require tangency at one point and vanishing at another point.

Let $`\rho `$ be a variationally complete reducible representation and $`\stackrel{~}{\rho }`$ a summand of $`\rho `$. Then it is easy to see that $`\stackrel{~}{\rho }`$ is also variationally complete, see Proposition 3.4. Notice though that it is not true that the direct sum of two variationally complete representations of a compact Lie group $`G`$ is variationally complete as can be seen by taking the direct sum of of two copies of $`\mathrm{𝐒𝐎}(2)`$ acting on $`𝐑^2`$.

Let $`N`$ be a properly embedded submanifold of a Euclidean space $`V`$ and let $`x`$ be some point in $`V`$. Then we define the distance function $`L_x:N𝐑`$ from $`x`$ to $`N`$ by setting $`L_x(p)=px^2`$. It follows that $`L_x`$ is a non-negative proper function since $`N`$ is properly embedded. Hence it is possible to apply Morse theory to $`L_x`$. We say that $`L_x`$ is perfect with respect to a field $`F`$ if it is a Morse function and the Morse inequalities for $`L_x`$ with respect to $`F`$ are equalities. Furthermore we say that $`N`$ is $`F`$-taut or simply taut if $`L_x`$ is perfect with respect to $`F`$ whenever $`L_x`$ is a Morse function, see . The concept of tautness can be extended to submanifolds of complete Riemannian manifolds, see , but we will not need that here. We will say that an orthogonal representation $`\rho :G𝐎(V)`$ of a compact Lie group $`G`$ is $`F`$-taut or simply taut if the the orbits of $`G`$ are $`F`$-taut submanifolds of $`V`$.

If $`\rho `$ is a reducible taut representation and $`\stackrel{~}{\rho }`$ one of its factors, then $`\stackrel{~}{\rho }`$ is clearly taut. Notice that it is not true that direct sums of taut representations of a compact Lie group $`G`$ are also taut. An example of this is the direct sum of $`n`$ copies of $`\mathrm{𝐒𝐔}(n)`$ acting on $`𝐂^n`$. This representation is not taut since one can compute that $`\mathrm{𝐒𝐔}(n)`$ is not taut in the matrix space $`M(n;𝐂)`$.

The following theorem mentioned in the introduction was proved by Bott and Samelson more generally for variationally complete actions on complete Riemannian manifolds. In fact it is an immediate corollary of Theorem I in . Notice that Bott and Samelson do not use the concept of a taut submanifold which was introduced later.

###### 2.1 Theorem (Bott-Samelson)

A variationally complete representation of a compact connected Lie group on an Euclidean space is $`𝐙_2`$-taut.

Bott proved in , Propositions 7.1 and 11.6, that the action of a compact Lie group $`G`$ with a bi-invariant metric on itself by conjugations and the adjoint representation of $`G`$ on its Lie algebra $`𝔤`$ are variationally complete. This was generalized by Bott and Samelson in , Theorem II as follows. Let $`(L,G)`$ be a symmetric pair and $`𝔩=𝔤𝔭`$ the corresponding Cartan decomposition. Then the action of $`L\times L`$ on $`L`$, the action of $`G`$ on $`L/G`$ and the action of $`\mathrm{Ad}_L(G)`$ restricted to $`𝔭`$ are variationally complete. The action of $`\mathrm{Ad}_L(G)`$ on $`𝔭`$ is equivalent to the isotropy representation of $`G`$ on the tangent space $`T_p(L/G)`$ where $`p`$ denotes the coset $`G`$. Therefore we have the following theorem mentioned in the introduction.

###### 2.2 Theorem (Bott-Samelson)

The isotropy representation of a symmetric space is variationally complete.

Conlon considered in  actions of a Lie group $`G`$ on a complete Riemannian manifold $`M`$ with the property that there is a connected submanifold $`\mathrm{\Sigma }`$ of $`M`$ that meets all orbits of $`G`$ in such a way that the intersections between $`\mathrm{\Sigma }`$ and the orbits of $`G`$ are all orthogonal. Such a submanifold is called a section and an action admitting a section is now usually called polar if $`\mathrm{\Sigma }`$ is properly embedded. Notice that Conlon does not assume in  that $`\mathrm{\Sigma }`$ is properly embedded, but it is usually required in the recent literature on the subject. It is easy to see that a section $`\mathrm{\Sigma }`$ is totally geodesic in $`M`$. An action admitting a section that is flat in the induced metric is called hyperpolar. There is clearly no difference between polar and hyperpolar representations since totally geodesic submanifolds of a Euclidean space are affine subspaces. Moreover, the question whether $`\mathrm{\Sigma }`$ should be required to be properly embedded or not becomes redundant. Conlon proved the following theorem in .

###### 2.3 Theorem (Conlon)

A hyperpolar action of a compact Lie group on a complete Riemannian manifold is variationally complete.

Polar representations were classified by Dadok in . We recall that two representations $`\rho _1:G_1O(V_1)`$ and $`\rho _2:G_2O(V_2)`$ are said to be orbit equivalent if there is an isometry $`A:V_1V_2`$ under which the orbits of $`G_1`$ and $`G_2`$ correspond. As a consequence of his classification he obtained the following result.

###### 2.4 Theorem (Dadok)

A polar representation of a compact connected Lie group is orbit equivalent to the isotropy representation of a symmetric space.

We can summarize this discussion as follows. Let $`G`$ be a compact connected Lie group. Denote by $``$ the representations of $`G`$ that are isotropy representations of symmetric spaces, by $`𝒫`$ those that are polar, by $`𝒱`$ those that are variationally complete, and by $`𝒯`$ those that are taut. Then

$$𝒫𝒱𝒯.$$

The starting point in the present paper will be to consider a class of representations that is a priori larger than $`𝒯`$, but easier to deal with, see . We recall that the second osculating space $`O_p^2(N)`$ at a point $`p`$ of a submanifold $`N`$ in a Euclidean space $`V`$ is the vector space spanned by the first and second derivatives at $`p`$ of the inclusion of $`N`$ into $`V`$. It is easy to see that

$$O_p^2(N)=T_pN\{\alpha (X,Y)|X,YT_pN\}.$$

where $`\alpha `$ denotes the second fundamental form of $`N`$ and $`S`$ stands for the linear hull of the set $`S`$. The following is a corollary of the discussion of Kuiper in .

###### 2.5 Theorem (Kuiper)

Let $`N`$ be a taut submanifold of a Euclidean space. Then the affine hull of $`N`$ coincides with $`p+O_p^2(N)`$ for every point $`p`$ in $`N`$.

We let $`𝒪^2`$ denote the class of representations of a compact connected Lie group $`G`$ such that the representation space coincides with $`O_p^2(Gp)`$ for all nonzero $`p`$. The representations in $`𝒪^2`$ are clearly irreducible. If a taut representation is irreducible, then Theorem 2.5 implies that it belongs to class $`𝒪^2`$. Thus we have the inclusions

$$_i𝒫_i𝒱_i𝒯_i𝒪^2,$$

where $`_i`$, $`𝒫_i`$, $`𝒱_i`$, and $`𝒯_i`$ are the the subclasses of $``$, $`𝒫`$, $`𝒱`$, and $`𝒯`$ consisting of irreducible representations.

## 3 A collection of geometric propositions

In this section we collect technical results of geometric nature that will be needed to prove our main theorems.

### 3.1 Variationally complete representations

We first study focal points of orbits of variationally complete representations with the aim to prove that a representation that is orbit equivalent to a variationally complete one is also variationally complete. We then show how reducible variationally complete representations relate to irreducible ones. Finally we prove an important criterion which can be used to decide in some cases whether a variationally complete representation is orbit equivalent to the isotropy representation of a symmetric space.

Let $`q`$ be a focal point of an orbit $`Gp`$ of an orthogonal representation of a compact connected Lie group $`G`$. It is clear that the orbit $`Gq`$ consists of focal points of $`Gp`$. We therefore call $`Gq`$ a focal orbit of $`Gp`$. It is also clear that the singular orbits of a representation are always focal orbits of the principal orbits. Slightly more generally, $`Gq`$ is a focal orbit of $`Gp`$ if the segment $`\overline{qp}`$ is perpendicular to $`Gp`$ in $`p`$ and there is a one-parameter group in $`G_q`$ that does not fix $`p`$. We will use the following lemma to prove Proposition 3.3.

###### 3.1 Lemma

A representation $`\rho :G𝐎(V)`$ is variationally complete if and only if every orbit $`Gp`$ has the following property: if $`q`$ is a focal point of $`Gp`$ relative to $`p`$ with multiplicity $`k>0`$, then $`k=dimG_qp`$.

###### 3.2 Remark (Compare .)

Let $`\rho :G𝐎(V)`$ be a variationally complete representation of a compact connected Lie group $`G`$. Then Lemma 3.1 implies that the focal orbits of any orbit of $`G`$ are singular orbits. Furthermore, the multiplicity of the focal points of a principal orbit $`Gp`$ in a singular orbit $`Gq`$ is equal to the difference between the dimension of $`Gp`$ and the dimension of $`Gq`$. In particular, it follows that any principal orbit is a *totally focal submanifold*; see  for a definition of this concept. Terng claims in  that this fact can be used to show that variationally complete representations are polar. However, as Terng herself pointed out in , p. 197, there is a gap in that argument.

Proof of Lemma 3.1. We first assume that $`\rho `$ is variationally complete. Let $`q`$ be a focal point of $`Gp`$ along the line segment $`\overline{pq}`$ with multiplicity $`k`$. Let $`𝒥`$ be the space of Jacobi fields along $`\overline{pq}`$ that are the variational vector fields of variations $`\gamma _s`$ of $`\overline{pq}`$ through line segments such that all $`\gamma _s`$ meet the orbit $`Gp`$ orthogonally for $`t=0`$ and the Jacobi field $`J`$ of the variation of $`\gamma `$ vanishes for $`t=1`$. The multiplicity $`k`$ of the focal point $`q`$ is equal to the dimension of the space $`𝒥`$. The action of $`G_q`$ on $`\overline{pq}`$ induces variations of $`\overline{pq}`$ whose variational vector fields are contained in $`𝒥`$. Let us denote by $`\widehat{𝒥}`$ the subspace of $`𝒥`$ that consists of Jacobi fields induced by the action of $`G_q`$ on $`\overline{pq}`$. It follows that $`dim\widehat{𝒥}=dimG_qvk`$ where $`v`$ is the tangent vector of $`\overline{pq}`$ at $`q`$. We now prove that $`dimG_qv=k`$. Let $`\gamma _s`$ be a variation of $`\overline{pq}`$ through line segments such that all $`\gamma _s`$ meet the orbit $`Gp`$ orthogonally for $`t=0`$ and the Jacobi field $`J`$ of the variation along $`\gamma `$ vanishes for $`t=1`$. The segments $`\gamma _s`$ are orthogonal to the orbit $`Gp`$. Hence they are $`G`$-transversal and $`J`$ is a $`G`$-transversal Jacobi field. It is clear that $`J`$ is tangent to the orbits through $`p=\gamma (0)`$ and $`q=\gamma (1)`$. By variational completeness, there is a Killing field $`X`$ on $`M`$ induced by the $`G`$-action such that $`J=X\gamma `$. Hence $`J\widehat{𝒥}`$ and it follows that $`\widehat{𝒥}=𝒥`$. Thus $`dimG_qv=k`$.

To prove the converse, we have to show that $`dimG_qv=k`$ for all focal points $`q`$ implies that $`\rho `$ is variationally complete. This is equivalent to show that a Jacobi field $`J`$ that vanishes in a point $`q`$ and is tangent to an orbit $`Gp`$ in $`p`$ is induced by the group action. We let the symbols $`𝒥`$ and $`\widehat{𝒥}`$ have the same meaning as in the first part of the proof. We have to show that $`\widehat{𝒥}=𝒥`$. This is clear since $`dim𝒥=k`$ and $`dim\widehat{𝒥}=dimG_qv=k`$. $`\mathrm{}`$

The following proposition that we will use to classify variationally complete representations is now easy.

###### 3.3 Proposition

Let $`\rho `$ be a representation that is orbit equivalent to a variationally complete representation. Then $`\rho `$ is itself variationally complete.

Proof. We can assume that we have representations $`\rho `$ and $`\widehat{\rho }`$ of the groups $`G`$ and $`\widehat{G}`$ with the same representation space and the same orbits and that $`\widehat{\rho }`$ is variationally complete. Clearly the focal orbits of the two representations coincide since their orbits are the same. Let $`v`$ be a normal vector in $`q`$ of the orbit $`Gq=\widehat{G}q`$. It follows from the slice theorem and the fact that orbits of $`\rho `$ and $`\widehat{\rho }`$ are the same that $`G_qv=\widehat{G}_qv`$. Since $`\widehat{\rho }`$ is variationally complete, Lemma 3.1 now implies that $`\rho `$ is also variationally complete. $`\mathrm{}`$

The next proposition will be important in the proof of Theorem 1.1 in the introduction. It is analogous to a result of Dadok on polar representations, see Theorem 4 in .

###### 3.4 Proposition

Let $`G`$ be a compact connected Lie group. Let $`\rho :G𝐎(V)`$ be a nontrivial reducible variationally complete representation and let $`\rho =\rho _1\rho _2`$ be a nontrivial decomposition. Let $`V_1`$ and $`V_2`$ be the representation spaces of $`\rho _1`$ and $`\rho _2`$ respectively. Then we have the following:

1. The representations $`\rho _1`$ and $`\rho _2`$ are variationally complete.
2. The restriction of $`\rho _2`$ to the connected component of the isotropy subgroup of any point in $`V_1`$ is orbit equivalent to $`\rho _2`$.
3. The representations $`\rho _1`$ and $`\rho _2`$ are not equivalent as representations.
4. The representation $`\rho `$ is orbit equivalent to the representation $`\widehat{\rho }:G\times G𝐎(V)`$ defined by $`\widehat{\rho }(g_1,g_2)=\rho _1(g_1)+\rho _2(g_2)`$.

Proof. The claim in part (a) is clear, since the orbits of $`\rho _1`$ and $`\rho _2`$ are also orbits of $`\rho `$.

To prove part (b), note that it is true in general, without assuming $`\rho `$ is variationally complete, that any point $`p_1V_1`$ is a focal point of any orbit $`Gp_2V_2`$ with multiplicity equal to $`dimGp_2`$. Now if $`\rho `$ is variationally complete then the claim follows from Lemma 3.1 and the fact that the $`G`$-orbits are connected.

Next we prove (c). Assume $`A:V_1V_2`$ is an isomorphism such that $`\rho _2=A\rho _1A^1`$. Notice that $`G_p=G_{Ap}`$ for all $`pV_1`$. Part (b) implies that $`G_p`$ is transitive on the orbit $`G(Ap)`$, which is impossible if we choose $`pV_1`$ such that $`dimGp>0`$. Therefore such an isomorphism $`A`$ does not exist.

Finally we prove part (d). Let $`v=v_1+v_2`$ be a point in $`V`$ and $`v_1`$ and $`v_2`$ its components in $`V_1`$ and $`V_2`$ respectively. Let $`H_1`$ (resp. $`H_2`$) be the isotropy group of $`v_1`$ (resp. $`v_2`$). Notice that $`v_2`$ is a focal point of $`Gv_1`$ with multiplicity $`dimGv_1`$. It follows from part (b) that $`H_2v_1=Gv_1`$ and similarly that $`H_1v_2=Gv_2`$. We next prove the factorization $`G=H_1H_2`$. Let $`gG`$. Then there is an element $`h_1H_1`$ such that $`h_1v_2=gv_2`$. Then $`h_1^1gH_2`$. Set $`h_2=h_1^1g`$. Then $`g=h_1h_2`$. It follows that

$$𝔤=𝔥_1+𝔥_2$$

where $`𝔤`$, $`𝔥_1`$ and $`𝔥_2`$ are the Lie algebras of $`G`$, $`H_1`$ and $`H_2`$ respectively. It is clear that the orbit of $`v`$ under $`\rho (G)`$ is contained in its orbit under $`\widehat{\rho }(G\times G)`$. The tangent space of $`\widehat{\rho }(G\times G)v`$ at $`v`$ is

$$(𝔤\times 𝔤)v=𝔤v_1+𝔤v_2=𝔥_2v_1+𝔥_1v_2.$$

Let $`X_1𝔥_1`$ and $`X_2𝔥_2`$. Then $`X_1v_1=0`$ and $`X_2v_2=0`$ and hence

$$X_1v_2+X_2v_1=(X_1+X_2)v𝔤v.$$

Hence the dimension of the tangent spaces of the orbits of $`v`$ under $`\rho `$ and $`\widehat{\rho }`$ coincide. Since $`G`$ is connected the orbits coincide. Since $`v`$ was chosen arbitrarily, all orbits $`\rho `$ and $`\widehat{\rho }`$ coincide.
$`\mathrm{}`$

###### 3.5 Remark

Part (c) in Proposition 3.4 generalizes Proposition 2.10 in .

The next lemma is a preparation for Proposition 3.7. We follow the terminology of Bredon in , pp. 180-181, and distinguish between three types of orbits, namely principal (regular), exceptional and singular. Recall that orbits are called *exceptional* if they have maximal dimension without being principal, and *singular* if their dimension is not maximal. We also call points *regular* when they belong to principal orbits.

###### 3.6 Lemma

Let $`G`$ be a closed subgroup of the compact connected Lie group $`K`$ and let $`\rho :K𝐎(V)`$ be a representation. If $`\rho `$ has an orbit of maximal dimension in common with $`\rho |G`$, then $`\rho `$ and $`\rho |G`$ are orbit equivalent.

Proof. We denote the maximal dimension of an orbit of $`\rho `$ by $`n`$. Let us denote by $`𝒪_G`$ the union of the $`G`$-orbits of maximal dimension and by $`𝒪_K`$ the corresponding set relative to $`K`$. It is well-known that both $`𝒪_G`$ and $`𝒪_K`$ contain an open and dense subset of $`V`$. Hence the set $`𝒪_G𝒪_K`$ also contains an open and dense set. The dimensions of the orbits of $`G`$ and $`K`$ through points in $`𝒪_G𝒪_K`$ are all equal to $`n`$. It follows that the orbits of $`G`$ and $`K`$ through points in $`𝒪_G𝒪_K`$ coincide since $`K`$ is connected. Let $`pV𝒪_G𝒪_K`$. We would like to show that $`Gp=Kp`$. Let $`k(p)`$ be some other point in $`Kp`$. Let $`(p_n)`$ be a sequence in $`𝒪_G𝒪_K`$ that converges to $`p`$. Then $`(k(p_n))`$ converges to $`k(p)`$. There is for every $`n`$ an element $`g_n`$ in $`G`$ such that $`g_n(p_n)=k(p_n)`$. Now $`(g_n(p_n))`$ converges to $`k(p)`$. By going to a subsequence if necessary we can assume that $`g_n`$ converges to an element $`g`$ in $`G`$, so that the continuity of $`G\times VV`$ implies that $`(g_n(p_n))`$ converges to $`g(p)`$. Hence $`g(p)=k(p)`$ and we have seen that all orbits of $`G`$ and $`K`$ coincide. $`\mathrm{}`$

###### 3.7 Proposition

Let $`G`$ be a closed subgroup of the compact connected Lie group $`K`$ and let $`\rho :K𝐎(V)`$ be the isotropy representation of an irreducible symmetric space. If the restriction $`\rho |G`$ is variationally complete and $`\rho |G`$ has a nontrivial orbit in common with $`\rho `$, then $`\rho `$ and $`\rho |G`$ are orbit equivalent.

Proof. Let $`pV`$ be a point such that the orbits of $`G`$ and $`K`$ through $`p`$ coincide and have the largest possible dimension with this property. Assume that $`Kp`$ is not an orbit of maximal dimension for the action of $`K`$. Let $`𝔞`$ be a Cartan subspace in $`V`$ with respect to the symmetric space representation of $`K`$ such that $`p𝔞`$. Let $`T`$ be a tubular neighborhood of the common orbit $`Kp=Gp`$. Let $`q𝔞`$ be a point in $`Gp=Kp`$ different from $`p`$ and such that the segment $`\overline{pq}`$ from $`p`$ to $`q`$ passes through a point $`r`$ in $`T`$ with $`dimK_pr1`$. (Here it is important that the symmetric space is irreducible.) It follows that $`dimKp<dimKr`$ by the slice theorem. Hence $`dimGr<dimKr`$ since the dimension of $`Gp=Kp`$ is maximal among the common orbits of $`G`$ and $`K`$. It follows that $`dimG_pr<dimK_pr`$.

Let $`X`$ be an element of the Lie algebra $`𝔨_p`$ of $`K_p`$ such that $`\dot{\gamma }_{X,r}(0)`$ is not in $`T_r(G_pr)`$ where $`\gamma _{X,r}(s)=\mathrm{exp}(sX)r`$. Now let us consider the variation $`(\overline{pq})_s=\mathrm{exp}(sX)(\overline{pq})`$ of $`\overline{pq}`$. Let $`J`$ be the corresponding Jacobi field along $`\overline{pq}`$. This variation is $`K`$-transversal and hence also $`G`$-transversal. The value of $`J`$ over $`p`$ and $`q`$ are tangent vectors of the common orbit $`Gp=Kp`$. By the variational completeness of the action of $`G`$, the Jacobi field $`J`$ is the restriction of a Killing field on $`V`$ to $`\overline{pq}`$ that is induced by the action of $`G`$ on $`V`$. Hence all values of $`J`$ are tangent vectors to orbits of $`G`$. The value of $`J`$ over $`r`$ is $`\dot{\gamma }_{X,r}(0)`$ which is not a tangent vector of the orbit of $`G`$ through $`r`$. This is a contradiction. Hence $`G`$ and $`K`$ have an orbit in common which is of maximal dimension among the $`K`$-orbits. It now follows from Lemma 3.6 that all orbits of $`G`$ and $`K`$ coincide. $`\mathrm{}`$

### 3.2 Taut representations

We will first prove that the slice representation of a taut representation is also taut and end the section with an important proposition about taut reducible representations. It also follows immediately from our methods that a taut representation does not have exceptional orbits.

###### 3.8 Proposition

Let $`G`$ be a compact connected Lie group with a taut representation $`\rho :G𝐎(V)`$. Let $`pV`$ and let $`\nu _p:G_p𝐎(N_p(Gp))`$ be the slice representation of $`\rho `$ at $`p`$. Then $`\nu _p`$ is taut.

Proof. Let $`N^ϵ(Gp)`$ denote the bundle of normal vectors of length less than $`ϵ`$ over the orbit $`Gp`$. Let $`ϵ>0`$ be so small that $`T=\mathrm{exp}(N^ϵ(Gp))`$ is a tubular neighborhood around the orbit $`Gp`$. Let $`vN_p(Gp)`$. We want to show that $`G_pv`$ is taut in the normal space $`N_p(Gp)`$. Let $`\alpha >0`$ be a number that is so small that $`w=\alpha v`$ has length less than $`ϵ`$. It is clear that $`G_pv`$ is taut in $`N_p(Gp)`$ if and only if $`G_pw`$ is taut in $`N_p(Gp)`$. It is also clear that $`G_pw`$ is taut in $`N_p(Gp)`$ if and only if $`\mathrm{exp}_p(G_pw)=G_p\mathrm{exp}_p(w)`$ is taut in $`\mathrm{exp}_p(N_p(Gp))`$ since $`\mathrm{exp}_p`$ is an isometry. We set $`q=\mathrm{exp}_p(w)`$. A submanifold in an affine subspace $`A`$ of $`V`$ is taut in $`A`$ if and only if it is taut in $`V`$. Hence $`G_pv`$ is taut in $`N_p(Gp)`$ if and only if $`G_pq`$ is taut in $`V`$.

Now let $`L_p:Gq𝐑`$ be the distance function. The segment $`\overline{pq}`$ is orthogonal to $`Gp`$ and hence also to $`Gq`$. It follows that $`q`$ is a critical point of $`L_p`$. Since $`q`$ is in the tubular neighborhood $`T`$ and the length of the segment $`\overline{pq}`$ is smaller than $`ϵ`$ it follows that $`L_p`$ takes on its minimum in $`q`$. Let $`C`$ be the subset of $`Gq`$ on which $`L_p`$ takes on its minimum value. Clearly $`C=Gq\mathrm{exp}_p(N_p^ϵ(Gp))`$ since a geodesic segment between $`Gp`$ and $`Gq`$ that is orthogonal to $`Gp`$ is orthogonal to $`Gq`$ and vice versa. By the slice theorem $`G_pq=Gq\mathrm{exp}_p(N_p^ϵ(Gp))`$. Hence $`G_pq=C`$. The set of critical points of a distance function to a taut submanifold is a union over taut submanifolds by a theorem of Ozawa, see . It follows that $`G_pq`$ is taut and hence that the slice representation $`\nu _p`$ is taut. $`\mathrm{}`$

###### 3.9 Remark

We are assuming in Proposition 3.8 that $`G`$ is connected. Hence its orbits are connected and it follows from their tautness that the set of points where a distance function takes on its minimal value is connected. The proof of Proposition 3.8 now implies that the orbits of $`G_p`$ are connected even if $`G_p`$ is not connected.

It is well-known that the isotropy representation of a symmetric space does not have an exceptional orbit. The methods used in the proof of Proposition 3.8 immediately give that this is also the case for taut representations, as we show in the next proposition.

###### 3.10 Proposition

A taut representation $`\rho :G𝐎(V)`$ of a compact connected Lie group $`G`$ does not have exceptional orbits.

Proof. Assume the orbit through $`p`$ in $`V`$ is exceptional. Then the slice representation of $`G_p`$ at $`p`$ has a disconnected orbit. Arguing exactly as in the proof of Proposition 3.8, we see that the distance function from $`p`$ to a principal orbit through some regular point close to $`p`$ assumes its minimum value on a disconnected set, contradicting tautness. Thus $`\rho `$ cannot have an exceptional orbit. $`\mathrm{}`$

We close this subsection with a discussion of taut reducible representations.

###### 3.11 Proposition

Let $`\rho _1`$ and $`\rho _2`$ be representations of a compact connected Lie group $`G`$ with representation spaces $`V_1`$ and $`V_2`$ respectively. Assume that $`\rho _1\rho _2`$ is $`F`$-taut. Then the restriction of $`\rho _2`$ to the isotropy group $`G_{v_1}`$ is taut for every $`v_1V_1`$.

Furthermore, we have $`p(G(v_1,v_2);F)=p(Gv_1;F)p(G_{v_1}v_2;F)`$ where $`p(M;F)`$ denotes the Poincaré polynomial of $`M`$ with respect to the field $`F`$. In particular, $`G_{v_1}v_2`$ is connected and

$$b_1(G(v_1,v_2);F)=b_1(Gv_1;F)+b_1(G_{v_1}v_2;F)$$

where $`b_1(M;F)`$ denotes the first Betti number of $`M`$ with respect to $`F`$.

Proof. We can work with height functions instead of distance functions in this proof since a submanifold contained in a round sphere in a Euclidean space is taut if and only if all height functions are perfect Morse functions, see . Furthermore, in this situation the set of critical points of a distance function will also occur as the set of critical points of a height function, and vice versa. Fix $`(v_1,v_2)V_1V_2`$. Let $`aV_1`$ be such that the height function $`h_a:Gv_1𝐑`$ defined by $`h_a(v)=a,v`$ is a Morse function. We define the height function $`h_{(a,0)}:G(v_1,v_2)𝐑`$ similarly. The point $`(u_1,u_2)`$ is a critical point of $`h_{(a,0)}`$ with index $`i`$ if and only if $`u_1`$ is a critical point of $`h_a`$ with index $`i`$. Hence the critical set $`C`$ on the critical level $`h_{(a,0)}(u_1,u_2)=h_a(u_1)`$ is

$$C=\{(w_1,w_2)G(v_1,v_2)|w_1=u_1\}=\{(u_1,w_2)|w_2G_{u_1}v_2\}.$$

Ozawa proves in  that the set of critical points of a distance function on a taut submanifold is a union over nondegenerate critical submanifolds that are again taut submanifolds. It follows that $`C`$ is taut. The projection of $`C`$ into $`V_2`$, which coincides with $`G_{u_1}v_2`$, is then also taut. We can choose $`a`$ such that $`h_a`$ is a Morse function of which $`v_1`$ is a critical point. It follows that $`G_{v_1}v_2`$ is taut for every $`v_1V_1`$ and $`v_2V_2`$ and hence that the restriction of $`\rho _2`$ to $`G_{v_1}`$ is taut.

Now fix again a point $`(v_1,v_2)`$. If $`CG(v_1,v_2)`$ is an arbitrary critical submanifold of $`h_{(a,0)}`$, then it is diffeomorphic to $`G_{v_1}v_2`$. To see this let $`\pi _1:G(v_1,v_2)Gv_1`$ be the projection onto the first factor. Then one easily sees that

$$\pi _1^1(gv_1)=G_{gv_1}gv_2=gG_{v_1}g^1(gv_2)=gG_{v_1}v_2.$$

Hence we have that $`\pi _1`$ is a $`G`$-equivariant fibration. The critical submanifolds of $`h_{(a,0)}`$ are fibers of $`\pi _1`$ and hence diffeomorphic to each other.

Since the orbit $`G(v_1,v_2)`$ is taut we have by  that the Morse-Bott inequalities are equalities. Let $`C_1,\mathrm{},C_k`$ be the critical manifolds of $`h_{(a,0)}`$. Let $`i(C_j)`$ be the index of $`C_j`$ for $`j=1,\mathrm{},k`$. Then

$$p(G(v_1,v_2))=\underset{j=1}{\overset{k}{}}p(C_j)t^{i(C_j)}=p(G_{v_1}v_2)\underset{j=1}{\overset{k}{}}t^{i(C_j)}.$$

since the critical submanifolds are all diffeomorphic to $`G_{v_1}v_2`$. We have

$$p(Gv_1)=\underset{j=1}{\overset{k}{}}t^{i(C_j)}$$

since $`h_a`$ is a perfect Morse functions with critical points $`p_1=\pi _1(C),\mathrm{},p_k=\pi _1(C_k)`$ and the index of $`p_j`$ is equal to $`i(C_j)`$. It follows that

$$p(G(v_1,v_2))=p(Gv_1)p(G_{v_1}v_2).$$

Multiplying out the Poincaré polynomials gives $`b_0(G_{v_1}v_2)=1`$ and

$$b_1(G(v_1,v_2))=b_1(Gv_1)+b_1(G_{v_1}v_2).$$

This finishes the proof of the proposition. $`\mathrm{}`$

###### 3.12 Remark

The property that $`\rho _2|G_{v_1}`$ is taut in Proposition 3.11 is a weakening of the property that $`\rho _2|G_{v_1}`$ is orbit equivalent to $`\rho _2`$ for a variationally complete reducible representation $`\rho _1\rho _2`$ (cf. Proposition 3.4, part (b)).

We give examples of how Proposition 3.11 can be used.

###### 3.13 Examples

(i) Let $`G=\mathrm{𝐒𝐎}(n)`$ and let $`\rho _1`$ be the $`\mathrm{𝐒𝐎}(n)`$-conjugation on the space $`V_1`$ of real traceless symmetric $`n\times n`$ matrices. Then $`\rho _1`$ is taut since it is the isotropy representation of the symmetric space $`\mathrm{𝐒𝐋}(n,𝐑)/\mathrm{𝐒𝐎}(n)`$. Let $`\rho _2`$ be any other nontrivial representation of $`\mathrm{𝐒𝐎}(n)`$ with representation space $`V_2`$. Then $`\rho _1\rho _2`$ cannot be taut if $`n3`$. To see this, let $`v_1V_1`$ be a regular point. Then $`G_{v_1}`$ is the discrete group consisting of all diagonal matrices with determinant one and entries $`\pm 1`$ on the diagonal. The kernel of $`\rho _2`$ is contained in the center of $`\mathrm{𝐒𝐎}(n)`$. Since $`n3`$, we see that $`G_{v_1}`$ cannot be contained in the center of $`\rho _2`$. Hence there is an element $`v_2V_2`$ that is not fixed by $`G_{v_1}`$. It follows that $`G_{v_1}v_2`$ is disconnected. Now Proposition 3.11 implies that $`\rho _1\rho _2`$ is not taut.

(ii) Now let $`G`$ be a compact connected simple Lie group of rank at least two and let $`\rho _1`$ denote the adjoint representation of $`G`$. We assume that $`G`$ is simply connected. Let $`\rho _2`$ be any other nontrivial representation of $`G`$. Then $`\rho _1\rho _2`$ is not taut. To see this let $`T`$ be a maximal torus in $`G`$. We denote the representation spaces of $`\rho _1`$ and $`\rho _2`$ by $`V_1`$ and $`V_2`$ respectively. There is a regular element $`v_1V_1`$ with $`G_{v_1}=T`$. The restriction of $`\rho _2`$ to $`T`$ has a discrete kernel that is contained in the center of $`G`$. If $`v_2V_2`$ is a $`T`$-regular point then the isotropy subgroup $`T_{v_2}`$ coincides with the kernel of $`\rho _2|T`$. Hence $`G_{v_1}v_2`$ is diffeomorphic to $`T`$ and it follows that $`b_1(G_{v_1}v_2;F)`$ is equal to the rank of $`G`$. In particular $`b_1(G_{v_1}v_2;F)2`$. Now notice that the isotropy group of $`(v_1,v_2)`$ is also $`T_{v_2}`$. Hence $`\pi _1(G(v_1,v_2))=T_{v_2}`$ which implies $`H_1(G(v_1,v_2);𝐙)=T_{v_2}`$ since $`T_{v_2}`$ is Abelian. If $`G\mathrm{𝐒𝐩𝐢𝐧}(4k)`$ then the center of $`G`$ is a cyclic group and it follows that $`b_1(G(v_1,v_2);F)1`$. If $`G=\mathrm{𝐒𝐩𝐢𝐧}(4k)`$, then $`k2`$ and we get $`b_1(G_{v_1}v_2;F)=2k4`$; since the center of $`\mathrm{𝐒𝐩𝐢𝐧}(4k)`$ is $`𝐙_2\times 𝐙_2`$, we have $`b_1(G(v_1,v_2);F)2`$. In either case, $`b_1(G_{v_1}v_2;F)>b_1(G(v_1,v_2);F)`$ which implies by Proposition 3.11 that $`\rho _1\rho _2`$ is not taut.

### 3.3 A reduction principle

We now discuss a reduction principle which will be used later to prove that certain representations are or are not taut. Let $`\rho :G𝐎(V)`$ be an orthogonal linear action of a compact Lie group $`G`$ which is not assumed to be connected. Denote by $`H`$ a fixed principal isotropy subgroup of the $`G`$-action on $`V`$ and let $`V^H`$ be the subspace of $`V`$ that is left pointwise fixed by the action of $`H`$. Let $`N`$ be the normalizer of $`H`$ in $`G`$. Then the group $`N/H`$ acts on $`V^H`$ with trivial principal isotropy subgroup. Moreover, the following result is known ():

###### 3.14 Theorem (Luna-Richardson)

The inclusion $`V^HV`$ induces a stratification preserving homeomorphism between orbit spaces

(3.15)
$$V^H/NV/G.$$

The injectivity of the map (3.15) means that $`Np=GpV^H`$ for $`pV^H`$. In particular, the $`H`$-fixed point set of a $`G`$-orbit is a smooth manifold. Observe also that for a regular point $`pV^H`$ the normal space to the principal orbit $`M=Gp`$ at $`p`$ is contained in $`V^H`$, because the slice representation at $`p`$ is trivial.

###### 3.16 Lemma

Let $`pV^H`$ be a regular point and consider a normal vector $`\xi `$ to $`M=Gp`$ at $`p`$. Then the Weingarten operator of $`M`$ at $`\xi `$ restricts to the Weingarten operator of $`M^H=MV^H`$ at $`\xi `$, in symbols, $`A_\xi ^M|_{T_pM^H}=A_\xi ^{M^H}`$.

Proof. Let $`vT_pM^H`$. Consider an extension $`\stackrel{~}{\xi }(t)`$ of $`\xi `$ to a normal vector field along a curve $`\gamma (t)`$ in $`M^H`$ with $`\dot{\gamma }(0)=v`$. Then $`A_\xi ^Mv=\stackrel{~}{\xi }^{}(0)^T`$, where “T” denotes the orthogonal projection onto $`T_pM`$. Now $`\stackrel{~}{\xi }(t)V^H`$, so the derivative $`\stackrel{~}{\xi }^{}(0)V^H`$. The normal component of $`\stackrel{~}{\xi }^{}(0)`$ is already in $`V^H`$, so its component in $`T_pM`$ is also in $`V^H`$. Hence $`A_\xi ^{M^H}v=\stackrel{~}{\xi }^{}(0)^T`$ which proves the claim. $`\mathrm{}`$

###### 3.17 Lemma

Let $`p`$, $`qV^H`$ and suppose that $`q`$ is a regular point for $`G`$. Consider $`M=Gp`$ and let $`L_q:M𝐑`$ be the distance function. Then the critical set of $`L_q`$ is contained in $`M^H=MV^H`$, namely, $`\text{Crit(}L_q\text{)}=\text{Crit(}L_q|_{M^H}\text{)}`$.

Proof. Let $`xM`$ be a critical point of $`L_q:M𝐑`$. Then $`x=gp`$ for some $`gG`$ and $`v=qx`$ is a normal vector to $`M`$ at $`x`$. Consider first the case where $`p`$ is a regular point for $`G`$. Then the isotropy subgroup at $`x`$ is principal, so $`G_x=gHg^1`$ acts trivially on the normal space. In particular $`gHg^1v=v`$, which implies that $`gHg^1q=q`$. Now $`gHg^1G_q=H`$ and therefore $`gN`$. Thus $`x=gpV^H`$ and $`x`$ is a critical point for the restriction $`L_q:M^H𝐑`$. Consider now the case where $`p`$ is not a regular point for $`G`$. Then there is $`ϵ>0`$ sufficiently small such that $`y=x+ϵv`$ is a regular point for $`G`$. Now $`y`$ is a critical point for $`L_q:Gy𝐑`$, since a line segment meeting one orbit orthogonally is orthogonal to all orbits it meets. By the previous case $`yV^H`$. Hence $`xV^H`$ and again $`x`$ is a critical point for the restriction $`L_q:M^H𝐑`$.

On the other hand, let $`xM^H`$ be a critical point of $`L_q:M^H𝐑`$. Consider first the case where $`p`$ is a regular point for $`G`$. Then $`x`$ is also a regular point for $`G`$ so that the normal spaces $`N_x(M^H)`$ and $`N_x(M)`$ coincide. It follows that $`x`$ is a critical point of $`L_q:M𝐑`$. Consider now the case where $`p`$ is not a regular point for $`G`$. Then there is $`ϵ>0`$ sufficiently small such that $`y=x+ϵvV^H`$ is a regular point for $`G`$, where $`v=qx`$. Since $`GyV^H=Ny`$, we get that $`y`$ is a critical point for $`L_q:Ny𝐑`$. By the previous case $`y`$ is a critical point for $`L_q:Gy𝐑`$. Hence again $`x`$ is a critical point of $`L_q:M𝐑`$.
$`\mathrm{}`$

For the next lemma and proposition we found inspiration in .

###### 3.18 Lemma

Suppose there is a subgroup $`LH`$ which is a finitely iterated $`𝐙_2`$-extension of the identity and such that the fixed point sets $`V^L=V^H`$. Let $`M=Gp`$ be a principal orbit. Then the sum of the Betti numbers (with respect to $`𝐙_2`$ coefficients) of $`M^H`$ is less than or equal the sum of the Betti numbers of $`M`$, in symbols $`\beta (M^H;𝐙_2)\beta (M;𝐙_2)`$.

Proof. We can write $`L=\{h_1,\mathrm{},h_n\}H`$ such that $`h_{i+1}`$ normalizes $`H_i`$ and $`h_{i+1}^2H_i`$ for $`i:0,\mathrm{},n1`$, where $`H_0`$ is the identity group and $`H_i`$ is the subgroup of $`H`$ generated by $`h_1,\mathrm{},h_i`$. Now each transformation $`h_{i+1}`$ has order $`2`$ in the fixed point set $`M^{H_i}=MV^{H_i}`$, so by a theorem of Floyd ()

$$\beta (M^L;𝐙_2)\beta (M^{H_{n1}};𝐙_2)\mathrm{}\beta (M^{H_0};𝐙_2).$$

But $`M^L=M^H`$ and $`M^{H_0}=M`$ and this completes the proof. $`\mathrm{}`$

###### 3.19 Proposition

Suppose there is a subgroup $`LH`$ which is a finitely iterated $`𝐙_2`$-extension of the identity and such that the fixed point sets $`V^L=V^H`$. Suppose also that the reduced representation $`\overline{\rho }:(N/H)^0𝐎(V^H)`$ is $`𝐙_2`$-taut where $`(N/H)^0`$ denotes the connected component of the identity of $`N/H`$. Then $`\rho :G𝐎(V)`$ is $`𝐙_2`$-taut.

Proof. Let $`M=Gp`$ be an arbitrary orbit. We shall prove that $`M`$ is $`𝐙_2`$-taut. Since $`V^H`$ intersects all orbits, we may assume $`pV^H`$. Suppose $`L_q:M𝐑`$ is a Morse distance function. We must show that $`L_q`$ is $`𝐙_2`$-perfect, and in fact it is enough to do that for $`q`$ in a dense subset of $`V`$, so we may take $`q`$ to be a regular point. Since $`G`$ acts on $`V`$ by isometries, we may moreover assume that $`qV^H`$. If $`\overline{\rho }`$ is $`𝐙_2`$-taut, then each connected component of $`M^H`$ is $`𝐙_2`$-taut, hence $`M^H`$ itself is $`𝐙_2`$-taut. Note that $`L_q|_{M^H}`$ is a Morse function. Hence $`L_q|_{M^H}`$ is $`𝐙_2`$-perfect. Now Lemmas 3.17 and 3.18 and the Morse inequalities imply that

$$\beta (M^H;𝐙_2)\beta (M;𝐙_2)\mathrm{\#}\text{Crit(}L_q\text{)}=\mathrm{\#}\text{Crit(}L_q|_{M^H}\text{)},$$

where $`\mathrm{\#}A`$ denotes the cardinality of the finite set $`A`$. It follows that $`L_q`$ is $`𝐙_2`$-perfect. $`\mathrm{}`$

## 4 A necessary condition for a representation to be of class $`𝒪^2`$

Let $`\pi `$ be a complex representation of the compact connected Lie group $`G`$ on a finite-dimensional vector space. We say that $`\pi `$ is of *real type* if it comes from a representation of $`G`$ on a real vector space by extension of scalars, and we say that $`\pi `$ is of *quaternionic type* if it comes from a representation of $`G`$ on a quaternionic vector space by restriction of scalars. If $`\pi `$ is neither of real type nor of quaternionic type, we say that $`\pi `$ is of *complex type*333It is useful to remark that $`\pi `$ is of real type (resp., quaternionic type) if and only if there exists an invariant real (resp., quaternionic) structure on the representation space, i. e. a $`𝐂`$-conjugate linear, $`G`$-invariant endomorphism $`ϵ`$ of the representation space such that $`ϵ^2=1`$ (resp., $`ϵ^2=1`$)..

Now it is known that the finite-dimensional real irreducible representations $`\rho `$ of $`G`$ fall into one of the following disjoint classes:

1. the complexification $`\rho ^c`$ is irreducible and $`\rho ^c=\pi `$ is a complex representation of real type;
2. the complexification $`\rho ^c`$ is reducible and $`\rho ^c=\pi \pi `$ where $`\pi `$ is a complex irreducible representation of quaternionic type;
3. the complexification $`\rho ^c`$ is reducible and $`\rho ^c=\pi \pi ^{}`$ where $`\pi `$ is a complex irreducible representation of complex type and $`\pi ^{}`$ is not equivalent to $`\pi `$ (where $`\pi ^{}`$ denotes the dual representation of $`\pi `$).

The relation between $`\rho `$ and $`\pi `$ is that $`\rho `$ is a real form of $`\pi `$ in the first case ($`\rho ^c=\pi `$), but $`\rho `$ is $`\pi `$ viewed as a real representation in the other two cases ($`\rho =\pi ^r`$). We shall call $`\rho `$ of *real*, *quaternionic* or *complex type* according to whether the associated $`\pi `$ is of real, quaternionic or complex type. Note also that $`\pi `$ is self-dual precisely in the first two cases.

Suppose now that $`G`$ is semisimple, let $`𝔤`$ denote its Lie algebra and $`𝔤^c`$ its complexification. Write $`\mathrm{\Delta }`$ for the root system of $`𝔤^c`$ with respect to a chosen Cartan subalgebra, $`\mathrm{\Delta }^+`$ for the positive root system with respect to an ordering of the roots, and $`𝒮=\{\alpha _1,\mathrm{},\alpha _r\}`$ for the corresponding simple root system. Let $`\lambda _1,\mathrm{},\lambda _r`$ be the fundamental highest weights defined by the relations $`2(\lambda _i,\alpha _j)/(\alpha _j,\alpha _j)=\delta _{ij}`$ where $`(,)`$ is the Cartan-Killing form. The Theorem of the Highest Weight of É. Cartan states that the complex irreducible representations of $`𝔤^c`$ are parametrized by their highest weights, and these are exactly the linear combinations $`\lambda =_{i=1}^rm_i\lambda _i`$ for $`m_i\{0,1,2,\mathrm{}\}`$. The following proposition is a useful criterium of Dadok to decide for which $`\lambda `$ the corresponding representation $`\pi _\lambda `$ is of real, quaternionic or complex type. Recall that the roots $`\alpha `$, $`\beta \mathrm{\Delta }`$ are called *strongly orthogonal* if $`\alpha \pm \beta `$ is not a root.

###### 4.1 Proposition ()

There is a maximal subset $`=\{\beta _1,\mathrm{},\beta _s\}\mathrm{\Delta }^+`$ of strongly orthogonal roots such that:

1. we have $`s_0=s_{\beta _1}\mathrm{}s_{\beta _s}`$ is the Weyl group element that maps the positive Weyl chamber into its negative;
2. the representation $`\pi _\lambda `$ is of complex type if and only if $`\lambda `$ does not belong to the real span of $``$;
3. the representation $`\pi _\lambda `$ is of real type (resp. quaternionic type) if and only if $`\lambda `$ belongs to the real span of $``$ and

$$k(\lambda )=\underset{i=1}{\overset{s}{}}\frac{(\lambda ,\beta _i)(s_0\lambda ,\beta _i)}{(\beta _i,\beta _i)}$$

is an even (resp. odd) integer.

###### 4.2 Remark

For a simple Lie algebra the set $``$ can be constructed as follows, as is explained in  and will be assumed throughout our paper (cf. Appendix A). Let $`\beta _1`$ be the highest root. The root system $`\{\alpha \mathrm{\Delta }:(\alpha ,\beta _1)=0\}`$ is either irreducible or equals $`\{\pm \zeta _1\}\mathrm{\Delta }_1`$, with $`\mathrm{\Delta }_1`$ being irreducible and $`\zeta _1\mathrm{\Delta }^+`$. In the former case set $`\beta _2`$ equal to the highest root of $`\mathrm{\Delta }_1`$ (with the inherited order from $`\mathrm{\Delta }`$), and proceed by induction. In the latter case set $`\beta _2=\zeta _1`$ and $`\beta _3`$ equal to the highest root of $`\mathrm{\Delta }_1`$, and proceed by induction.

We continue to assume that $`G`$ is semisimple. Let $`H_\alpha `$, $`\alpha \mathrm{\Delta }`$, be the coroots of $`𝔤^c`$. It is possible to choose root vectors $`X_\alpha `$ for $`𝔤^c`$, $`\alpha \mathrm{\Delta }`$, such that the compact real form $`𝔤`$ is spanned by

(4.3)
$$iH_\alpha ,X_\alpha X_\alpha ,i(X_\alpha +X_\alpha ),\text{where }\alpha \mathrm{\Delta }^+.$$

Now if $`\pi `$ is a complex representation of $`G`$, we have that the adjoint map $`\pi (X_\alpha )^{}=\pi (X_\alpha )`$ with respect to any $`G`$-invariant Hermitian product on the representation space. Moreover, if $`\pi `$ is of real (resp. quaternionic) type and $`ϵ`$ is an invariant real (resp. quaternionic) structure on the representation space, then $`\pi (X_\alpha )ϵ=ϵ\pi (X_\alpha )`$.

By compactness of $`G`$, any real representation is equivalent to an orthogonal one. The following proposition, which is stated as a remark in , p. 128, and the ensuing lemma, which is a refinement of a result in , are precisely the ingredients we need to establish a necessary condition for an orthogonal representation of $`G`$ to be of class $`𝒪^2`$.

###### 4.4 Proposition ()

Let $`𝒰^k(𝔤^c)`$ be the $`k`$th level in the natural filtration of the universal enveloping algebra of $`𝔤^c`$. Fix $`\pi _\lambda `$, the irreducible representation of $`𝔤^c`$ with highest weight $`\lambda `$, and fix $`v_\lambda `$, a highest weight vector. Let

$$n_i=\frac{(\lambda ,\beta _i)(s_0\lambda ,\beta _i)}{(\beta _i,\beta _i)},$$

for $`i=1,\mathrm{},s`$. Then the element $`s_0v_\lambda `$ is in $`𝒰^{k(\lambda )}(𝔤^c)v_\lambda `$, but not in $`𝒰^l(𝔤^c)v_\lambda `$ for any $`l<k(\lambda )`$, where $`s_0v_\lambda `$ is given as

$$s_0v_\lambda =\pi (X_{\beta _1})^{n_1}\mathrm{}\pi (X_{\beta _s})^{n_s}v_\lambda .$$

There is no proof of the above proposition in . In , p. 270, there is an attempt to prove the proposition which in our view contains serious gaps.

Proof. It is enough to consider the case where $`𝔤^c`$ is a complex simple Lie algebra. We first claim that the element $`s_0v_\lambda `$ is not zero. In fact, consider the complex subalgebra $`𝔨𝔤^c`$ generated by the root spaces of $`𝔤^c`$ corresponding to $`\pm \beta _1,\mathrm{},\pm \beta _s`$. Then $`𝔨`$ is isomorphic to the direct product of $`s`$ copies of $`𝔰𝔩(2,𝐂)`$. We restrict $`\pi _\lambda `$ to $`𝔨`$ and let $`U_\lambda `$ be the unique irreducible $`𝔨`$-module generated by $`v_\lambda `$. Since $`\lambda s_0\lambda =_{i=1}^sn_i\beta _i`$, it is now clear that $`U_\lambda `$ is the representation space of $`𝔨:\text{}\mathrm{}\text{}`$ and that $`s_0v_\lambda U_\lambda `$. Therefore, our claim is reduced to the case of the $`(n+1)`$-dimensional complex irreducible representation of $`𝔰𝔩(2,𝐂)`$, namely , which is immediate to verify.

It is obvious that $`s_0v_\lambda 𝒰^{k(\lambda )}(𝔤^c)v_\lambda `$. We next prove that $`s_0v_\lambda 𝒰^l(𝔤^c)v_\lambda `$ for $`l<k(\lambda )`$ by contradiction. In fact, enumerate the positive roots $`\mathrm{\Delta }^+=\{\alpha _1,\mathrm{},\alpha _t\}`$ and suppose $`s_0v_\lambda 𝒰^l(𝔤^c)v_\lambda `$ for $`l<k(\lambda )`$. It follows from the Poincaré-Birkhoff-Witt theorem that we can write

$$s_0v_\lambda =\underset{p_1,\mathrm{},p_t0}{}c_{p_1\mathrm{}p_t}X_{\alpha _1}^{p_1}\mathrm{}X_{\alpha _t}^{p_t}v_\lambda ,$$

for some complex constants $`c_{p_1\mathrm{}p_t}`$. Now, using that $`X_\alpha ^{}=X_\alpha `$,

$`0`$ $``$ $`(s_0v_\lambda ,s_0v_\lambda )`$
$`=`$ $`{\displaystyle \underset{p_1,\mathrm{},p_t0}{}}c_{p_1\mathrm{}p_t}(X_{\alpha _1}^{p_1}\mathrm{}X_{\alpha _t}^{p_t}v_\lambda ,X_{\beta _1}^{n_1}\mathrm{}X_{\beta _s}^{n_s}v_\lambda )`$
$`=`$ $`{\displaystyle \underset{p_1,\mathrm{},p_t0}{}}c_{p_1\mathrm{}p_t}(X_{\beta _s}^{n_s}\mathrm{}X_{\beta _1}^{n_1}X_{\alpha _1}^{p_1}\mathrm{}X_{\alpha _t}^{p_t}v_\lambda ,v_\lambda ).`$
Recall that the highest weight space is one-dimensional; hence we can write

$$X_{\beta _s}^{n_s}\mathrm{}X_{\beta _1}^{n_1}X_{\alpha _1}^{p_1}\mathrm{}X_{\alpha _t}^{p_t}v_\lambda =cv_\lambda ,$$

for some nonnegative integers $`p_1,\mathrm{},p_t`$ such that $`k(\lambda )=n_1+\mathrm{}+n_s>p_1+\mathrm{}+p_t=l`$ and for some nonzero complex constant $`c`$. Note that $`n_1\beta _1+\mathrm{}+n_s\beta _s=p_1\alpha _1+\mathrm{}+p_t\alpha _t`$. The contradiction we are aiming at now follows from the following claim.

###### Claim 1

If $`N>M`$ then

$$X_{\delta _N}\mathrm{}X_{\delta _1}X_{\gamma _1}\mathrm{}X_{\gamma _M}v_\lambda =0,$$

where $`\delta _1,\mathrm{},\delta _N`$, and $`\gamma _1,\mathrm{},\gamma _M\mathrm{\Delta }^+`$.

We proceed by induction on the integer $`N`$. The cases $`N=1`$ and $`N=2`$ are trivial (use that $`v_\lambda `$ is a highest weight vector for $`N=1`$ and that the sum of two roots in $``$ is not a root for $`N=2`$). Assume the assertion is true for some $`N12`$ and let us prove it for $`N`$. Since $`X_{\beta _1},\mathrm{},X_{\beta _s}`$ pairwise commute, we may assume that $`\delta _1=\beta _i`$ where $`ij`$ for every $`j=1,\mathrm{},s`$ which satisfies $`\beta _j=\delta _k`$ for some $`k=1,\mathrm{},N`$.

###### Claim 2

For each $`j=1,\mathrm{},M`$, if $`\delta _1\gamma _j`$ is a root then it is a positive root.

Indeed we have $`\delta _1=\beta _i`$ where $`i`$ is as above. If $`i=1`$, then we are done because $`\beta _1`$ is the highest root. If not, $`0=(\beta _1,\delta _1+\mathrm{}+\delta _N)=(\beta _1,\gamma _1+\mathrm{}+\gamma _M)`$. Now $`(\beta _1,\gamma _1)=(\beta _1,\gamma _2+\mathrm{}+\gamma _M)`$, and since $`\beta _1`$ is the highest root we have $`(\beta _1,\gamma _1)0,\mathrm{},(\beta _1,\gamma _M)0`$. It follows that $`(\beta _1,\gamma _1)=0`$ and $`(\beta _1,\gamma _2+\mathrm{}+\gamma _M)=0`$. An easy induction argument shows that $`(\beta _1,\gamma _j)=0`$ for $`j=1,\mathrm{},M`$. Consider now the root system $`\{\alpha \mathrm{\Delta }:(\alpha ,\beta _1)=0\}`$. The first case occurs when it is irreducible. Then $`\beta _2`$ is by definition its highest root so that $`\beta _2\gamma _j`$ for $`j=1,\mathrm{},M`$. If $`i=2`$ we are done. If not, we see that $`(\beta _2,\gamma _j)=0`$ for $`j=1,\mathrm{},M`$ and we proceed by induction. The second case occurs when the above root system is of the form $`\{\pm \zeta _1\}\mathrm{\Delta }_1`$, with $`\mathrm{\Delta }_1`$ being irreducible and $`\zeta _1\mathrm{\Delta }^+`$. Here $`\beta _2=\zeta _1`$ and $`\beta _3`$ is the highest root of $`\mathrm{\Delta }_1`$. If $`\gamma _j\mathrm{\Delta }_1`$ for $`j=1,\mathrm{},M`$ then $`(\beta _2,\gamma _j)=0`$ for $`j=1,\mathrm{},M`$ so that $`i3`$ and we proceed by induction as above. On the other hand, if $`\zeta _1=\gamma _{j_0}`$ for some $`j_0`$, then $`i=2`$. In this case, for each $`j=1,\mathrm{},M`$ either $`\beta _2=\gamma _j`$ or $`\gamma _j\mathrm{\Delta }_1`$. So $`\beta _2\gamma _j`$ is never a root. This completes the proof of Claim 2.

###### Claim 3

We have

$$X_{\delta _N}\mathrm{}X_{\delta _2}X_{\gamma _1}\mathrm{}X_{\gamma _{j1}}H_\zeta X_{\gamma _{j+1}}\mathrm{}X_{\gamma _M}v_\lambda =0,$$

where $`H_\zeta `$ is the coroot vector corresponding to $`\zeta \mathrm{\Delta }^+`$ and $`j=1,\mathrm{},M`$.

In order to prove Claim 3, proceed by induction on $`j`$. The initial case is $`j=M`$, which follows from the induction hypothesis on $`N`$, since $`H_\zeta v_\lambda =(\zeta ,\lambda )v_\lambda `$. Next write

$`X_{\delta _N}\mathrm{}X_{\delta _2}X_{\gamma _1}\mathrm{}X_{\gamma _{j1}}H_\zeta X_{\gamma _{j+1}}\mathrm{}X_{\gamma _M}v_\lambda =`$
$`X_{\delta _N}\mathrm{}X_{\delta _2}X_{\gamma _1}\mathrm{}X_{\gamma _{j1}}X_{\gamma _{j+1}}H_\zeta \mathrm{}X_{\gamma _M}v_\lambda `$
$`+`$ $`X_{\delta _N}\mathrm{}X_{\delta _2}X_{\gamma _1}\mathrm{}X_{\gamma _{j1}}[H_\zeta ,X_{\gamma _{j+1}}]\mathrm{}X_{\gamma _M}v_\lambda .`$
The first summand on the right hand side of (4) is zero by the induction hypothesis on $`j`$, and the second summand is zero because $`[H_\zeta ,X_{\gamma _{j+1}}]=(\zeta ,\gamma _{j+1})X_{\gamma _{j+1}}`$ so that we can use the induction hypothesis on $`N`$. This proves Claim 3.

###### Claim 4

We have

$$X_{\delta _N}\mathrm{}X_{\delta _2}X_{\gamma _1}\mathrm{}X_{\gamma _{j1}}X_\zeta X_{\gamma _{j+1}}\mathrm{}X_{\gamma _M}v_\lambda =0,$$

where $`X_\zeta `$ is the root vector corresponding to $`\zeta \mathrm{\Delta }^+`$ and $`j=1,\mathrm{},M`$.

In order to prove Claim 4, proceed by induction on $`j`$. The initial case is $`j=M`$, which is trivial because $`v_\lambda `$ is a highest weight vector. Next write

$`X_{\delta _N}\mathrm{}X_{\delta _2}X_{\gamma _1}\mathrm{}X_{\gamma _{j1}}X_\zeta X_{\gamma _{j+1}}\mathrm{}X_{\gamma _M}v_\lambda =`$
$`X_{\delta _N}\mathrm{}X_{\delta _2}X_{\gamma _1}\mathrm{}X_{\gamma _{j1}}X_{\gamma _{j+1}}X_\zeta \mathrm{}X_{\gamma _M}v_\lambda `$
$`+`$ $`X_{\delta _N}\mathrm{}X_{\delta _2}X_{\gamma _1}\mathrm{}X_{\gamma _{j1}}[X_\zeta ,X_{\gamma _{j+1}}]\mathrm{}X_{\gamma _M}v_\lambda .`$
The first summand on the right hand side of (4) is zero by the induction hypothesis on $`j`$. The second summand is also zero for the following reasons. If $`\zeta =\gamma _{j+1}`$ we use Claim 3. If not, then either $`\zeta \gamma _{j+1}`$ is not a root and then $`[X_\zeta ,X_{\gamma _{j+1}}]=0`$, or else $`\zeta \gamma _{j+1}`$ is a root. In the latter case, if it is a positive root we can use the induction hypothesis on $`j`$, and if it is a negative root then we can use the induction hypothesis on $`N`$. This completes the proof of Claim 4. We finally turn to the proof of Claim 1. We can write

$`X_{\delta _N}\mathrm{}X_{\delta _1}X_{\gamma _1}\mathrm{}X_{\gamma _M}v_\lambda =`$
$`{\displaystyle \underset{j=1}{\overset{M}{}}}X_{\delta _N}\mathrm{}X_{\delta _2}X_{\gamma _1}\mathrm{}X_{\gamma _{j1}}[X_{\delta _1},X_{\gamma _j}]X_{\gamma _{j+1}}\mathrm{}X_{\gamma _M}v_\lambda `$
$`+`$ $`X_{\delta _N}\mathrm{}X_{\delta _2}X_{\gamma _1}\mathrm{}X_{\gamma _M}X_{\delta _1}v_\lambda .`$
The second summand in the right hand side of (4) is zero because $`v_\lambda `$ is a highest weight vector. The first summand is also zero because if $`\delta _1=\gamma _j`$ we can apply Claim 3. Otherwise, either $`\delta _1\gamma _j`$ is not a root and then $`[X_{\delta _1},X_{\gamma _j}]=0`$ or else $`\delta _1\gamma _j`$ is a root and then it is a positive root by Claim 2 so that we can apply Claim 4. This completes the proof of Claim 1 and the proof of the proposition. $`\mathrm{}`$

###### 4.8 Lemma

Let $`\pi _\lambda `$ be the complex irreducible representation of $`G`$ with highest weight $`\lambda `$ and representation space $`V_\lambda `$. Let $`\mu `$ be a weight of $`\pi _\lambda `$ and fix a weight vector $`v_\mu `$. Let $`\rho `$ denote a real form of $`\pi _\lambda `$ in case $`\pi _\lambda `$ is of real type, or the realification of $`\pi _\lambda `$ in case it is either of quaternionic or of complex type. Suppose that $`\rho `$ is of class $`𝒪^2`$.

1. If $`\pi _\lambda `$ is of real type, then

$$𝒰^2(𝔤^c)v_\mu +𝒰^2(𝔤^c)ϵ(v_\mu )=V_\lambda ,$$

where $`ϵ`$ is the real structure on $`V_\lambda `$ defined by $`\rho `$. In particular, by taking $`\mu =\lambda `$, we find that

$$𝒰^2(𝔤^c)v_\lambda +𝒰^2(𝔤^c)ϵ(v_\lambda )=V_\lambda .(C_2)$$
2. If $`\pi _\lambda `$ is of complex or quaternionic type, then $`𝒰^2(𝔤^c)v_\mu =V_\lambda `$.

Proof. (a) Consider the real vector $`p=v_\mu +ϵv_\mu `$. The second osculating space $`𝒪_p^2(Gp)`$ is spanned over $`𝐑`$ by $`Xp`$, $`XYp`$, where $`X`$, $`Y𝔤`$. Taking linear combinations with complex coefficients of these vectors, we can write that the complexification

$$𝒪_p^2(Gp)^c𝒰^2(𝔤^c)v_\mu +𝒰^2(𝔤^c)ϵ(v_\mu )V_\lambda .$$

But $`\rho `$ of class $`(𝒪^2)`$ implies that $`𝒪_p^2(Gp)^c=V_\lambda `$.

(b) Let $`p=v_\mu `$. We have that $`𝒪_p^2(Gp)𝒰^2(𝔤^c)v_\mu V_\lambda `$ as real vector spaces and the proof is similar as in (a). $`\mathrm{}`$

We now state the main result of this section. Notice that the same result is claimed in , p. 271, with an attempt of a proof which in our opinion is not satisfactory.

###### 4.9 Proposition

Let $`\rho `$ be a real (orthogonal) irreducible representation of a compact connected semisimple Lie group $`G`$ with complexified Lie algebra $`𝔤^c`$, and let $`\pi _\lambda `$ be the associated complex irreducible representation. Suppose that $`\rho `$ is of class $`𝒪^2`$.

1. If $`\rho `$ is of quaternionic type, then $`k(\lambda )=1`$.
2. If $`\rho `$ is of complex type, then $`k(\lambda )=1`$, $`2`$.
3. If $`\rho `$ is of real type, then $`k(\lambda )=2`$, $`4`$.

Proof. First consider $`\rho `$ to be of quaternionic or complex type, i. e. $`\rho ^c=\pi _\lambda \pi _\lambda ^{}`$. Then $`\rho `$ of class $`𝒪^2`$ forces $`𝒰^2(𝔤^c)v_\lambda =V_\lambda `$ (Lemma 4.8) and then $`s_0v_\lambda 𝒰^2(𝔤^c)v_\lambda `$ implies that $`k(\lambda )2`$ (Proposition 4.4). From this follow (a) and (b).

Now take $`\rho `$ to be of real type. Let $`v_\lambda `$ be a highest weight vector of $`\rho ^c=\pi _\lambda `$. Since $`\pi _\lambda `$ is self-dual,

$$\text{lowest weight of }\pi _\lambda =\text{highest weight of }\pi _\lambda ^{}=\lambda =s_0\lambda .$$

Also, there exists a $`𝐂`$-conjugate linear, $`G`$-invariant involution $`ϵ`$ of $`V_\lambda `$ and $`ϵ(v_\lambda )`$ is a lowest weight vector. We shall assume $`k(\lambda )6`$ and derive a contradiction. Since $`k(\lambda )`$ is even, it is possible to write

$$\frac{k(\lambda )}{2}=n_1+\mathrm{}+n_{i_0}+m$$

for some integers $`0i_0<s`$, $`0m<n_{i_0+1}`$, where $`n_i`$ is as in Proposition 4.4. Set

$$\mu =\lambda n_1\beta _1\mathrm{}n_{i_0}\beta _{i_0}m\beta _{i_0+1}.$$

Note that $`\mu `$ is a weight: a $`\mu `$-weight vector is

$$u=X_{\beta _1}^{n_1}\mathrm{}X_{\beta _{i_0}}^{n_{i_0}}X_{\beta _{i_0+1}}^mv_\lambda .$$

Use Lemma 4.8 to decompose $`u=u_1+u_2`$ where $`u_1𝒰^2(𝔤^c)v_\lambda `$ and $`u_2𝒰^2(𝔤^c)ϵ(v_\lambda )`$. First, assume that $`u_20`$. Since $`\frac{k(\lambda )}{2}3`$, we have

(4.10)
$$u_2=uu_1𝒰^{\frac{k(\lambda )}{2}}(𝔤^c)v_\lambda +𝒰^2(𝔤^c)v_\lambda 𝒰^{\frac{k(\lambda )}{2}}(𝔤^c)v_\lambda .$$

Moreover, it is clear that $`u_2`$ may be assumed to be a $`\mu `$-weight vector (because a component of $`u_2`$ in a different weight space has to cancel with the corresponding component of $`u_1`$ in the same weight space) and that

$$u_2=\underset{\gamma ,\delta \mathrm{\Delta }^+}{}c_{\gamma ,\delta }X_\gamma X_\delta ϵ(v_\lambda )$$

for some complex constants $`c_{\gamma ,\delta }`$ (we do not need to consider terms of first order in the sum because $`\lambda \mu `$ cannot be a root). Therefore,

$$0(u_2,u_2)=(u_2,\underset{\gamma ,\delta \mathrm{\Delta }^+}{}c_{\gamma ,\delta }X_\gamma X_\delta ϵ(v_\lambda ))=(\underset{\gamma ,\delta \mathrm{\Delta }^+}{}\overline{c}_{\gamma ,\delta }X_\delta X_\gamma u_2,ϵ(v_\lambda )).$$

This shows that

$$\underset{\gamma ,\delta \mathrm{\Delta }^+}{}\overline{c}_{\gamma ,\delta }X_\delta X_\gamma u_2$$

is a nonzero multiple of the lowest weight vector $`ϵ(v_\lambda )`$. This, combined with (4.10), gives that $`ϵ(v_\lambda )`$ is in $`𝒰^{\frac{k(\lambda )}{2}+2}(𝔤^c)v_\lambda `$. But $`\frac{k(\lambda )}{2}+2<k(\lambda )`$, contradicting Proposition 4.4.

In case $`u_2=0`$ we have that $`u=u_1𝒰^2(𝔤^c)v_\lambda `$. Since

$$X_{\beta _{i_0+1}}^{n_{i_0+1}m}X_{\beta _{i_0+2}}^{n_{i_0+2}}\mathrm{}X_{\beta _s}^{n_s}u=s_0v_\lambda $$

is a nonzero multiple of the lowest weight vector $`ϵ(v_\lambda )`$ and $`n_{i_0+1}m+n_{i_0+2}+\mathrm{}+n_s=\frac{k(\lambda )}{2}`$, again $`ϵ(v_\lambda )𝒰^{\frac{k(\lambda )}{2}+2}(𝔤^c)v_\lambda `$, contradicting the same proposition. $`\mathrm{}`$

## 5 The candidates to a position in class $`𝒪^2`$

In this section we want to elaborate a list of possibilities for real irreducible representations of class $`𝒪^2`$ which we shall use later in Sections 6 and 7 to classify variationally complete and taut representations. Our point of view, as is usually the case in other papers on the subject, is to classify orthogonal representations up to the following equivalence relation: we call two representations $`\rho :G𝐎(V)`$, $`\rho ^{}:G^{}𝐎(V^{})`$ *image equivalent* if there exists an isometry $`\psi :VV^{}`$ such that $`𝐎(\psi )(\rho (G))=\rho ^{}(G^{})`$, where $`𝐎(\psi ):𝐎(V)𝐎(V^{})`$ is the induced conjugation map444This is finer than the notion of orbit equivalence which is explained in Section 2, but of course not as fine as the usual notion of equivalence for representations.. In particular, the image equivalence class of $`\rho `$ always contains its dual $`\rho ^{}`$ (in fact any representation obtained from $`\rho `$ by an automorphism of the Dynkin diagram), as well as the pull back representation $`\stackrel{~}{\rho }=\rho p`$ of a covering group $`p:\stackrel{~}{G}G`$.

Let $`G`$ be a compact connected Lie group. We have that $`G`$ is finitely covered by $`T^n\times G_s`$, where $`T^n`$ is an $`n`$-dimensional torus and $`G_s`$ is a compact connected semisimple Lie group (which may also be assumed to be simply-connected, whenever convenient), and any representation of $`G`$ pulls back to a representation of $`T^n\times G_s`$. In view of image equivalence, in order to study representations of $`G`$, we may assume that $`G=T^n\times G_s`$ and restrict to almost faithful representations. Next we observe that any complex irreducible representation of $`T^n`$ is a character (hence of complex type). Therefore a complex irreducible representation of $`G`$ can be of real type or of quaternionic type only if it is trivial on $`T^n`$, and in case it is of complex type, then it has an $`(n1)`$-dimensional kernel on $`T^n`$. Now we have come to our working hypothesis (for a circle group $`S^1`$, let $`x^n`$ denote the $`n`$th power representation of that circle which is a complex representation of complex type):

> Let $`\rho `$ be a real irreducible representation of a compact connected Lie group $`G`$ on a finite-dimensional real vector space $`V`$.
>
> * If $`\rho `$ is of real type, we assume that $`G=G_s`$ is a compact connected semisimple Lie group, and $`\rho `$ is a real form of a complex irreducible representation $`\pi =\pi _\lambda `$ of $`G`$ of real type on the complex vector space $`V_\lambda `$ for some highest weight $`\lambda `$.
> * If $`\rho `$ is of quaternionic type, we assume that $`G=G_s`$, where $`G_s`$ is as above, and $`\rho `$ is the realification of a complex irreducible representation $`\pi =\pi _\lambda `$ of $`G`$ of quaternionic type on the complex vector space $`V_\lambda `$ for some highest weight $`\lambda `$.
> * If $`\rho `$ is of complex type, we assume that $`G=G_s`$ or $`G=S^1\times G_s`$, where $`G_s`$ is as above, and $`\rho `$ is the realification of a complex irreducible representation $`\pi `$ of $`G`$ of complex type. According to the form of $`G`$, we write $`\pi =\pi _\lambda `$ or $`\pi =x\pi _\lambda `$ as a representation on the complex vector space $`V_\lambda =𝐂_𝐂V_\lambda `$ for some highest weight $`\lambda `$.

Isotropy representations of symmetric spaces and cohomogeneity one representations of compact connected Lie groups are of course two classes of representations that belong to class $`𝒪^2`$. In order to keep the exposition as simple as possible, we assume knowledge of the classification of symmetric spaces. So in the course of this section we will be disregarding all the isotropy representations of symmetric spaces that we encounter, according to tables 8.11.2 and 8.11.5 in . Moreover, we classify in Section 8 the cohomogeneity one representations of compact connected Lie groups. (As a side remark, note that our proof of Theorem 8.2 is completely independent from the classification of the symmetric spaces.) Therefore, in addition to our previous hypotheses we assume throughout this section:

> The real irreducible representation $`\rho `$ is of class $`𝒪^2`$, cohomogeneity greater than one and it is not the isotropy representation of a symmetric space.

The results of our investigation in this section (namely, Propositions 5.1, 5.2, 5.17, 5.18, 5.19, 5.20 and 5.21) finally imply: the possibilities for $`\rho `$ are those collected in the table of Proposition 6.3 and in Tables C.1, C.2, C.3 and C.4 in Appendix C.

We need some amount of notation (cf. , p. 237). Let $`𝐀_n=\mathrm{𝐒𝐔}(n+1)`$, $`𝐁_n=\mathrm{𝐒𝐩𝐢𝐧}(2n+1)`$, $`𝐂_n=\mathrm{𝐒𝐩}(n)`$, $`𝐃_n=\mathrm{𝐒𝐩𝐢𝐧}(2n)`$, $`𝐆_2`$, $`𝐅_4`$, $`𝐄_6`$, $`𝐄_7`$ and $`𝐄_8`$ refer to É. Cartan’s classification types of simple Lie groups and Lie algebras. For the complex semisimple Lie algebra $`𝔤^c`$, we use the following notation for its complex irreducible representation $`\pi _\lambda `$: if the integer $`2(\lambda ,\alpha _i)/(\alpha _i,\alpha _i)0`$, then we write it next to the vertex of the Dynkin diagram of $`𝔤^c`$ which corresponds to $`\alpha _i`$ (recall that $`𝒮=\{\alpha _1,\mathrm{},\alpha _r\}`$ is the simple root system). For example,

$$\text{},\text{},\text{},\text{},$$

denote respectively the vector representations of $`𝐀_n`$, $`𝐁_n`$, $`𝐂_n`$, $`𝐃_n`$, and the adjoint representations are given by:

$$\begin{array}{cccc}& & & \\ 𝐀_1\hfill & \text{}& 𝐆_2\hfill & \text{}\\ 𝐀_n,n2\hfill & \text{}& 𝐅_4\hfill & \text{}\\ 𝐁_n,n3\hfill & \text{}& 𝐄_6\hfill & \text{}\\ 𝐂_n,n2\hfill & \text{}& 𝐄_7\hfill & \text{}\\ 𝐃_n,n4\hfill & \text{}& 𝐄_8\hfill & \text{}\end{array}$$

Note that we are using the dot convention: if there are two lengths of roots, then the short roots are black in the Dynkin diagram. And a real irreducible representation is denoted by the diagram of its complexification.

We add that we shall refer repeatedly to Table B.1 in Appendix B, which contains the values of the invariant $`k(\lambda )`$ for the fundamental representations of the complex simple Lie algebras.

### 5.1 The case where $`\rho `$ is of quaternionic type

Here there is nothing:

###### 5.1 Proposition

The representation $`\rho `$ cannot be of quaternionic type.

Proof. If $`\rho `$ is of class $`𝒪^2`$ and quaternionic type, Proposition 4.9 says that $`k(\lambda )=1`$. A glance at Table B.1 now shows that $`G=\mathrm{𝐒𝐩}(n)`$ and $`\pi _\lambda `$ is the vector representation. But then $`\rho `$ is of cohomogeneity one. $`\mathrm{}`$

### 5.2 The case where $`\rho `$ is of complex type

Here the result is:

###### 5.2 Proposition

Let $`\rho `$ be of complex type. Then $`\rho `$ is one of the following:

$$\begin{array}{ccc}& & \\ G& \rho & Conditions\\ & & \\ \mathrm{𝐒𝐔}(n)\times \mathrm{𝐒𝐔}(m)& (\text{}\text{})(\text{}\text{})& nm\\ \mathrm{𝐒𝐔}(n)\times \mathrm{𝐒𝐩}(m)& (\text{}\text{})\text{}& n3,m2\\ S^1\times \mathrm{𝐒𝐔}(n)\times \mathrm{𝐒𝐩}(m)& (x\text{}x^1\text{})\text{}& m2\\ \mathrm{𝐒𝐔}(n)& \text{}\text{}& n\text{ odd}\\ \mathrm{𝐒𝐩𝐢𝐧}(10)& \text{}\text{}& \\ \mathrm{𝐒𝐎}(2)\times \mathrm{𝐒𝐩𝐢𝐧}(7)& (xx^1)\text{}& \\ \mathrm{𝐒𝐎}(2)\times \mathrm{𝐒𝐩𝐢𝐧}(9)& (xx^1)\text{}& \\ \mathrm{𝐒𝐎}(2)\times 𝐆_2& (xx^1)\text{}& \end{array}$$

The proof of Proposition 5.2 will be given after the proof of the following two lemmas.

###### 5.3 Lemma

Let $`\pi _\lambda `$ be a complex irreducible representation of a compact connected semisimple Lie group $`G_s`$ and suppose that the realification of the tensor product representation $`\pi =x\pi _\lambda `$ of $`G=S^1\times G_s`$ is of class $`𝒪^2`$. We have:

1. if $`\pi _\lambda `$ is of real type and $`G_s`$ is simple, then $`G_s=\mathrm{𝐒𝐩𝐢𝐧}(7)`$ or $`G_s=\mathrm{𝐒𝐩𝐢𝐧}(9)`$ and $`\pi _\lambda `$ is the respective spin representation, or $`G_s=𝐆_2`$ and $`\pi _\lambda `$ is the $`7`$-dimensional representation, or $`G_s=\mathrm{𝐒𝐎}(m)`$ for $`m2`$, $`4`$ and $`\pi _\lambda `$ is the vector representation;
2. if $`\pi _\lambda `$ is of real type and $`G_s`$ is not simple, then $`G_s=\mathrm{𝐒𝐩}(1)\times \mathrm{𝐒𝐩}(m)`$ and $`\pi _\lambda `$ is the tensor product of the vector representations of each of the factors;
3. if $`\pi _\lambda `$ is of quaternionic type or complex type, then $`k(\lambda )=1`$, $`2`$.

Proof. (a) and (b) Let $`V_\lambda `$ be the representation space of $`\pi _\lambda `$ and $`V`$ be the real subspace where a real form acts. Note that the representation space for $`\pi `$ is still $`V_\lambda `$, but $`\pi `$ is of complex type. Let $`p=1a𝐂_𝐑V=V^c=V_\lambda `$, $`n`$ the dimension of $`V`$ and $`q`$ the codimension of $`G_sa`$ in $`V`$. Since the realification of $`\pi `$ is of class $`𝒪^2`$, we have that $`𝒪_p^2(Gp)=V^{cr}ViV`$. But

$$𝒪_p^2(Gp)1[𝐑a+𝒪_a^2(G_sa)]+i[𝐑a+T_a(G_sa)],$$

so by comparing real and imaginary parts we get that $`V=𝐑a+T_a(G_sa)`$. Therefore $`q=1`$ and $`G`$ acts with cohomogeneity one on $`V`$. Now (a) follows from Lemmas 5.11 and 5.14 below, and (b) follows from Lemmas 5.11 and 5.15 below.

(c) Consider the point $`p=v_\lambda V_\lambda `$. We have that $`𝒪_p^2(Gp)𝒰^2(𝔤^c)v_\lambda V_\lambda `$ as real vector spaces. Since the realification of $`\pi `$ is of class $`𝒪^2`$, we have that $`𝒰^2(𝔤^c)v_\lambda =V_\lambda `$. But $`𝒰^2(𝔤_s^c)v_\lambda =𝒰^2(𝔤^c)v_\lambda `$, since the circle subgroup $`S^1`$ preserves the weight spaces of $`\pi _\lambda `$. Therefore $`𝒰^2(𝔤_s^c)v_\lambda =V_\lambda `$ and we can apply Proposition 4.4 as in the first paragraph of the proof of Proposition 4.9. $`\mathrm{}`$

Part (a) of the following lemma also follows from .

###### 5.4 Lemma

1. Let $`\pi _\lambda `$ be one of the following complex irreducible representations of complex type of the compact connected semisimple Lie group $`G_s`$:

$$\begin{array}{ccc}& & \\ G_s& \pi _\lambda & Conditions\\ & & \\ \mathrm{𝐒𝐔}(n)\times \mathrm{𝐒𝐔}(m)& \text{}\text{}& nm\\ \mathrm{𝐒𝐔}(n)& \text{}& n=2p+1\\ \mathrm{𝐒𝐩𝐢𝐧}(10)& \text{}& \end{array}$$

Consider the tensor product representation $`\pi =x\pi _\lambda `$ of $`S^1\times G_s`$. Then the realifications of $`\pi _\lambda `$ and $`\pi `$ are orbit equivalent.
2. The realifications of the following complex irreducible representations $`\pi _\lambda `$ of complex type of the compact connected semisimple Lie group $`G_s`$ are *not* of class $`𝒪^2`$:

$$\begin{array}{ccc}& & \\ G_s& \pi _\lambda & Conditions\\ & & \\ \mathrm{𝐒𝐔}(n)& \text{}& n3\\ \mathrm{𝐒𝐔}(n)& \text{}& n=2p6\\ \mathrm{𝐒𝐔}(n)\times \mathrm{𝐒𝐔}(n)& \text{}\text{}& n3\\ 𝐄_6& \text{}& \end{array}$$

Proof. Preliminary considerations: Each one of these representations $`\pi _\lambda `$ has the property that the realification of $`\pi =x\pi _\lambda `$ is the isotropy representation of a compact irreducible Hermitian symmetric space $`X=L_0/K_0`$, where $`K_0`$ is locally isomorphic to $`S^1\times G_s`$. We refer to , Chapter VIII, $`\mathrm{\S }7`$, for results about Hermitian symmetric spaces that we will be using in the following. Let $`𝔩_0=𝔨_0+𝔭_0`$ be the decomposition of the Lie algebra $`𝔩_0`$ of $`L_0`$ into the $`\pm 1`$-eigenspaces of the symmetry. Let $`𝔠_0`$ be the Lie algebra of $`S^1`$, and let $`𝔱_0`$ be some Cartan subalgebra of the Lie algebra $`𝔤_s`$ of $`G_s`$. Then $`𝔥_0=𝔠_0+𝔱_0`$ is a Cartan subalgebra of $`𝔨_0`$ and of $`𝔩_0`$. Let $`𝔩`$ be the complexification of $`𝔩_0`$ and let $`𝔠`$, $`𝔱`$, $`𝔥`$, $`𝔨`$, $`𝔭`$ be the complex subspaces of $`𝔩`$ spanned by $`𝔠_0`$, $`𝔱_0`$, $`𝔥_0`$, $`𝔨_0`$, $`𝔭_0`$. Let $`\mathrm{\Delta }`$ denote the root system of $`(𝔩,𝔥)`$ and consider the root space decomposition $`𝔩=𝔥+_{\alpha \mathrm{\Delta }}𝔩_\alpha `$. For each $`\alpha \mathrm{\Delta }`$, we have that either $`𝔩_\alpha 𝔨`$ or $`𝔩_\alpha 𝔭`$, in which cases the root $`\alpha `$ is called respectively compact or noncompact. A root is compact if and only if it vanishes on $`𝔠`$. Let $`\mathrm{\Delta }_c`$ and $`\mathrm{\Delta }_n`$ denote respectively the subsets of $`\mathrm{\Delta }`$ of compact and noncompact roots. Then we have decompositions $`𝔨=𝔥+_{\alpha \mathrm{\Delta }_c}𝔩_\alpha `$, $`𝔭=_{\gamma \mathrm{\Delta }_n}𝔩_\gamma `$. Each root is real-valued on $`i𝔥_0`$. We introduce a lexicographic ordering on the dual of $`i𝔥_0`$ that takes $`i𝔠_0`$ before $`i𝔱_0`$. Let $`\mathrm{\Delta }^+`$, $`\mathrm{\Delta }_c^+`$ and $`\mathrm{\Delta }_n^+`$ be the set of positive roots in $`\mathrm{\Delta }`$, $`\mathrm{\Delta }_c`$ and $`\mathrm{\Delta }_n`$, and define

$$𝔭^+=\underset{\gamma \mathrm{\Delta }_n^+}{}𝔩_\gamma ,𝔭^{}=\underset{\gamma \mathrm{\Delta }_n^+}{}𝔩_\gamma .$$

Then $`𝔭=𝔭^++𝔭^{}`$ is an $`\text{ad}_𝔨`$-invariant decomposition. The $`\text{ad}_{𝔨_0}`$-invariant complex structure $`J_0`$ on $`𝔭_0`$ is given by $`\text{ad}_{H_0}`$, where $`H_0𝔠_0`$ is determined by $`\gamma (H_0)=i`$ for $`\gamma \mathrm{\Delta }_n^+`$. The $`𝐂`$-linear extension $`J`$ of $`J_0`$ to $`𝔭`$ has $`𝔭^\pm `$ as $`\pm i`$-eigenspaces. Let $`\mathrm{\Gamma }=\{\gamma _1,\mathrm{},\gamma _s\}\mathrm{\Delta }_n^+`$ be a maximal subset of strongly orthogonal roots. Then $`𝔞_0=_{j=1}^s𝐑(X_{\gamma _j}X_{\gamma _j})`$ is a maximal Abelian subspace of $`𝔭_0`$. We view $`\pi ^r`$ as the adjoint action of $`K_0`$ on $`𝔭_0`$, and then $`\pi ^r`$ is polar and $`𝔞_0`$ is a section for $`\pi ^r`$. We also view $`\pi `$ as the the adjoint action of $`K_0`$ on $`𝔭^+`$, and then the weight system of $`\pi `$ is $`\mathrm{\Delta }_n^+`$. Now the root system of $`(𝔤_s^c,𝔱)`$ is

$$\mathrm{\Delta }_0=\{\alpha |_𝔱:\alpha \mathrm{\Delta }_c\}(\text{also }\mathrm{\Delta }_0^+=\{\alpha |_𝔱:\alpha \mathrm{\Delta }_c^+\}),$$

and the weight system of $`\pi _\lambda `$ is

$$\mathrm{\Phi }=\{\gamma |_𝔱:\gamma \mathrm{\Delta }_n^+\}.$$

It is also useful to remark that any $`\text{ad}_{𝔨_0}`$-invariant inner product $`<,>`$ on $`𝔭_0`$ is given by a negative multiple of the restriction of the Killing form of $`𝔩_0`$; fix one and extend it to a Hermitian product $`<<,>>`$ on $`𝔭`$. Then $`<<,>>`$ is $`\text{ad}_{𝔨_0}`$-invariant and $`J`$ is a skew-Hermitian operator on $`𝔭`$ with respect to $`<<,>>`$.

Proof of (a): Since $`𝔞_0`$ is a section for $`\pi ^r`$, it is clearly enough to check that given

$$p=\underset{j=1}{\overset{s}{}}a_j(X_{\gamma _j}X_{\gamma _j})𝔞_0,$$

where $`a_j𝐑`$, there exists $`Hi𝔱_0`$ such that

(5.5)
$$i\text{ad}_Hp=J_0p.$$

Write $`H=_{\alpha \mathrm{\Delta }_0^+}c_\alpha H_\alpha `$, where $`H_\alpha `$ are the coroots. Then (5.5) is equivalent to the following system of linear equations:

$$\underset{\alpha \mathrm{\Delta }_0^+}{}c_\alpha (\alpha ,\gamma _j)=1,j:1,\mathrm{},s,$$

which we can solve in each one of the three cases with the help of the tables in Appendix A.

Let $`G_s=\mathrm{𝐒𝐔}(n)\times \mathrm{𝐒𝐔}(m)`$, $`n<m`$, $`\pi _\lambda :\text{}\text{}`$. Then $`L_0`$ is $`\mathrm{𝐒𝐔}(n+m)`$,

$$\mathrm{\Delta }_0^+=\{\theta _i\theta _j:1i<jn\text{ or }n+1i<jn+m\},$$

$$\mathrm{\Phi }=\{\theta _i\theta _j:1in\text{ and }n+1jn+m\}$$

and $`\mathrm{\Gamma }=\{\theta _i\theta _{n+m+1i}:1in\}`$. Here we can take $`H`$ to be a suitable multiple of $`H_{\theta _m\theta _{m+1}}+H_{\theta _m\theta _{m+2}}+\mathrm{}+H_{\theta _m\theta _{m+n}}`$.

Let $`G_s=\mathrm{𝐒𝐔}(n)`$, $`n`$ odd, $`\pi _\lambda :\text{}`$. Then $`L_0`$ is $`\mathrm{𝐒𝐎}(2n)`$, $`\mathrm{\Delta }_0^+=\{\theta _i\theta _j:1i<jn\}`$, $`\mathrm{\Phi }=\{\theta _i+\theta _j:1i<jn\}`$ and $`\mathrm{\Gamma }=\{\theta _{2i1}+\theta _{2i}:1i\frac{n1}{2}\}`$. Here we can take $`H`$ to be a suitable multiple of $`H_{\theta _1\theta _n}+H_{\theta _3\theta _n}+\mathrm{}+H_{\theta _{n2}\theta _n}`$.

Let $`G_s=\mathrm{𝐒𝐩𝐢𝐧}(10)`$, $`\pi _\lambda :`$     . Then $`L_0`$ is $`E_6`$, $`\mathrm{\Delta }_0^+=\{\pm \theta _i+\theta _j:1i<j5\}`$, $`\mathrm{\Phi }=\{\frac{1}{2}(\theta _8\theta _7\theta _6+_{i=1}^5ϵ_i\theta _i):\mathrm{\Pi }_{i=1}^5ϵ_i=+1\}`$ and $`\mathrm{\Gamma }=\{\frac{1}{2}(\theta _8\theta _7\theta _6+\theta _5+\theta _4+\theta _3+\theta _2+\theta _1),\frac{1}{2}(\theta _8\theta _7\theta _6+\theta _5\theta _4\theta _3\theta _2\theta _1)\}`$. Here we can take $`H`$ to be a suitable multiple of $`H_{\theta _4+\theta _5}H_{\theta _4\theta _5}`$. This completes the proof of (a).

Proof of (b): We will verify directly that the second osculating space at the point

$$p=\underset{j=1}{\overset{s}{}}X_{\gamma _j}X_{\gamma _j}𝔞_0𝔭_0$$

is not of maximal dimension. More precisely, we shall see that

(5.6)
$$<\text{ad}_Xp,J_0p>=<\text{ad}_Y\text{ad}_Xp,J_0p>=0,$$

for all $`X`$, $`Y𝔤_s`$. Notice that $`J_0p=_{j=1}^si(X_{\gamma _j}+X_{\gamma _j})𝔭_0`$.

It suffices to prove the identities (5.6) for the basis elements (4.3) of $`𝔤_s`$. We have that $`<\text{ad}_Xp,J_0p>=\mathrm{}<<\text{ad}_Xp,Jp>>`$ and $`<\text{ad}_Y\text{ad}_Xp,J_0p>=\mathrm{}<<\text{ad}_Xp,\text{ad}_YJp>>`$. The only cases where the real parts of the Hermitian products are not obviously zero are the following ($`\alpha `$, $`\beta \mathrm{\Delta }_0^+`$):

(5.7) $`<<\text{ad}_{iH_\alpha }p,Jp>>`$ $`=`$ $`{\displaystyle \underset{j,k=1}{\overset{s}{}}}<<\text{ad}_{H_\alpha }(X_{\gamma _j}X_{\gamma _j}),X_{\gamma _k}+X_{\gamma _k}>>`$
$`=`$ $`{\displaystyle \underset{j,k=1}{\overset{s}{}}}\gamma _j(H_\alpha )<<X_{\gamma _j}+X_{\gamma _j},X_{\gamma _k}+X_{\gamma _k})>>`$
$`=`$ $`2{\displaystyle \underset{j=1}{\overset{s}{}}}\gamma _j(H_\alpha )=2({\displaystyle \underset{j=1}{\overset{s}{}}}\gamma _j,\alpha ),`$
assuming the root vectors to have length one. We will show below that the last expression in (5.7) vanishes for the representations we are interested.

(5.8) $`<<\text{ad}_{i(X_\alpha +X_\alpha )}p,Jp>>`$ $`=`$ $`<<\text{ad}_{X_\alpha +X_\alpha }p,_{k=1}^s(X_{\gamma _k}+X_{\gamma _k})>>`$
$`=`$ $`{\displaystyle \underset{j,k=1}{\overset{s}{}}}<<\text{ad}_{X_\alpha +X_\alpha }(X_{\gamma _j}X_{\gamma _j}),X_{\gamma _k}+X_{\gamma _k}>>`$
$`=`$ $`0,`$
because $`\text{ad}_{X_\alpha }X_{\gamma _j}`$ is either a $`(\gamma _j+\alpha )`$-root vector or $`0`$; $`\pm \gamma _j\pm \gamma _k\mathrm{\Delta }`$; and root spaces corresponding to distinct roots are orthogonal.

$`<<\text{ad}_{iH_\alpha }p,\text{ad}_{X_\beta X_\beta }Jp)>>`$ $`=`$ $`{\displaystyle \underset{j,k=1}{\overset{s}{}}}\gamma _j(H_\alpha )<<(X_{\gamma _j}+X_{\gamma _j}),\text{ad}_{X_\beta X_\beta }(X_{\gamma _k}+X_{\gamma _k})>>`$
$`=`$ $`0,`$
for the same reason why (5.8) is zero.

(5.9) $`\mathrm{}<<\text{ad}_{X_\alpha X_\alpha }Jp,\text{ad}_{i(X_\beta +X_\beta )}p>>`$ $`=`$ $`\mathrm{}<<\text{ad}_{X_\alpha }Jp,i\text{ad}_{X_\beta }p>>`$
$`\mathrm{}<<\text{ad}_{X_\alpha }Jp,i\text{ad}_{X_\beta }p>>`$
$`+\mathrm{}<<\text{ad}_{X_\alpha }Jp,i\text{ad}_{X_\beta }p>>`$
$`\mathrm{}<<\text{ad}_{X_\alpha }Jp,i\text{ad}_{X_\beta }p>>`$
$`=`$ $`\mathrm{}<<Jp,i\text{ad}_{[X_\beta ,X_\alpha ]}p>>`$
$`+\mathrm{}<<Jp,i\text{ad}_{[X_\beta ,X_\alpha ]}p>>,`$
because, for instance,

$`<<\text{ad}_{X_\alpha }Jp,i\text{ad}_{X_\beta }p>>`$ $`=`$ $`<<\text{ad}_{X_\beta }\text{ad}_{X_\alpha }Jp,ip>>`$
$`=`$ $`<<J\text{ad}_{X_\beta }\text{ad}_{X_\alpha }p,ip>>`$
$`=`$ $`<<\text{ad}_{X_\beta }\text{ad}_{X_\alpha }p,Jip>>`$
$`=`$ $`<<\text{ad}_{X_\beta }\text{ad}_{X_\alpha }p,iJp>>`$
$`=`$ $`<<i\text{ad}_{X_\beta }\text{ad}_{X_\alpha }p,Jp>>`$
$`=`$ $`\overline{<<Jp,i\text{ad}_{X_\beta }\text{ad}_{X_\alpha }p>>}.`$
Now (5.9) shows that everything amounts to checking that (5.7) vanishes. Next we do that in each one of the four cases. Let $`\gamma =_{j=1}^s\gamma _j`$. Then the vanishing of (5.7) is equivalent to

(5.10)
$$(\gamma ,\mathrm{\Delta }_0^+)=0.$$

Let $`G_s=\mathrm{𝐒𝐔}(n)`$, $`\pi _\lambda :\text{}`$. Then $`L_0`$ is $`\mathrm{𝐒𝐩}(n)`$, $`\mathrm{\Delta }_0^+=\{\theta _i\theta _j:1i<jn\}`$, $`\mathrm{\Phi }=\{\theta _i+\theta _j:1i<jn\}\{2\theta _i:1in\}`$, $`\mathrm{\Gamma }=\{2\theta _i:1in\}`$ and $`\gamma =2\theta _1+\mathrm{}+2\theta _n`$.

Let $`G_s=\mathrm{𝐒𝐔}(n)`$, $`n`$ even, $`\pi _\lambda :\text{}`$. Then $`L_0`$ is $`\mathrm{𝐒𝐎}(2n)`$, $`\mathrm{\Delta }_0^+=\{\theta _i\theta _j:1i<jn\}`$, $`\mathrm{\Phi }=\{\theta _i+\theta _j:1i<jn\}`$, $`\mathrm{\Gamma }=\{\theta _{2i1}+\theta _{2i}:1i\frac{n}{2}\}`$ and $`\gamma =\theta _1+\theta _2+\mathrm{}+\theta _n`$.

Let $`G_s=\mathrm{𝐒𝐔}(n)\times \mathrm{𝐒𝐔}(n)`$, $`\pi _\lambda :\text{}\text{}`$. Then $`L_0`$ is $`\mathrm{𝐒𝐔}(2n)`$, $`\mathrm{\Delta }_0^+=\{\theta _i\theta _j:1i<jn\text{ or }n+1i<j2n\}`$, $`\mathrm{\Phi }=\{\theta _i\theta _j:1in\text{ and }n+1j2n\}`$, $`\mathrm{\Gamma }=\{\theta _i\theta _{2n+1i}:1in\}`$ and $`\gamma =\theta _1+\theta _2+\mathrm{}+\theta _n\theta _{n+1}\theta _{n+2}\mathrm{}\theta _{2n}`$.

Let $`G_s=𝐄_6`$, $`\pi _\lambda :\text{}`$. Then $`L_0=𝐄_7`$,

$$\mathrm{\Delta }_0^+=\{\pm \theta _i+\theta _j:1i<j5\}\{\frac{1}{2}(\theta _8\theta _7\theta _6+\underset{i=1}{\overset{5}{}}ϵ_i\theta _i):\mathrm{\Pi }_{i=1}^5ϵ_i=+1\},$$

$$\mathrm{\Phi }=\{\theta _8\theta _7\}\{\pm \theta _i+\theta _6:1i5\}\{\frac{1}{2}(\theta _8\theta _7+\theta _6+\underset{i=1}{\overset{5}{}}ϵ_i\theta _i):\mathrm{\Pi }_{i=1}^5ϵ_i=1\},$$

$`\mathrm{\Gamma }=\{\theta _8\theta _7,\theta _6+\theta _5,\theta _6\theta _5\}`$ and $`\gamma =\theta _8\theta _7+2\theta _6`$.

It is immediate to see that (5.10) holds in each case, and this completes the proof of (b).
$`\mathrm{}`$

Proof of Proposition 5.2. We know from Proposition 4.9 and Lemma 5.3 that $`k(\lambda )=1`$, $`2`$. In particular, $`G_s`$ can have at most two simple factors.

Suppose $`k(\lambda )=1`$. This implies $`G_s`$ is simple. In the case $`G=G_s`$, we have $`\pi `$ is of complex type, so $`G=\mathrm{𝐒𝐔}(n)`$ and $`\pi `$ is the vector representation. In the case $`G=S^1\times G_s`$, we have $`\pi _\lambda `$ is of quaternionic or of complex type, so $`G_s=\mathrm{𝐒𝐩}(n)`$ or $`G_s=\mathrm{𝐒𝐔}(n)`$ and $`\pi _\lambda `$ is the respective vector representation. This only gives representations with cohomogeneity one.

Next suppose $`k(\lambda )=2`$. The first case is $`G_s=G_1\times G_2`$, where $`G_i`$ is a simple group. Here $`\pi _\lambda `$ decomposes as an outer tensor product $`\pi _{\mu _1}\pi _{\mu _2}`$, where $`\pi _{\mu _i}`$ is the representation of $`G_i`$ of highest weight $`\mu _i`$ with $`k(\mu _i)=1`$. This forces each $`G_i`$ to be either $`\mathrm{𝐒𝐩}(n)`$ or $`\mathrm{𝐒𝐔}(n)`$ for some $`n`$, and $`\pi _{\mu _i}`$ to be the corresponding vector representation. Using Lemma 5.3, item (b), and Lemma 5.4, item (b), we get the first three representations in the table.

The second case is $`G_s`$ is a simple group. For $`G=G_s`$, an inspection of Table B.1 reveals four possibilities for $`\pi _\lambda `$, namely:

$$\text{},\text{},\text{}\text{and}\text{}.$$

Now Lemma 5.4, item (b), restricts these to the next two representations in the table.

For $`G=S^1\times G_s`$, if $`\pi _\lambda `$ is of complex type, we only get representations associated to symmetric spaces, and if $`\pi _\lambda `$ is of real type, Lemma 5.3, item (a), gives the last three representations in our table. $`\mathrm{}`$

### 5.3 The case where $`\rho `$ is of real type

The real case is much more involved than the previous two. We know from Proposition 4.9 that $`k(\lambda )=2`$, $`4`$. In particular, $`G`$ can have at most four simple factors.

As a first step towards the classification in the case $`\rho `$ is of real type, we introduce the following variations of condition $`(C_2)`$ of Lemma 4.8 for a representation $`\pi _\lambda `$ with representation space $`V_\lambda `$ and highest weight vector $`v_\lambda `$:

$`𝒰^2(𝔤^c)v_\lambda +𝒰^1(𝔤^c)ϵ(v_\lambda )`$ $`=`$ $`V_\lambda (C_{1\frac{1}{2}})`$
$`𝒰^1(𝔤^c)v_\lambda +𝒰^2(𝔤^c)ϵ(v_\lambda )`$ $`=`$ $`V_\lambda (C_{1\frac{1}{2}}^{})`$
$`𝒰^1(𝔤^c)v_\lambda +𝒰^1(𝔤^c)ϵ(v_\lambda )`$ $`=`$ $`V_\lambda (C_1)`$
$`𝒰^1(𝔤^c)v_\lambda +𝐂ϵ(v_\lambda )`$ $`=`$ $`V_\lambda (C_{\frac{1}{2}})`$
$`𝐂v_\lambda +𝒰^1(𝔤^c)ϵ(v_\lambda )`$ $`=`$ $`V_\lambda (C_{\frac{1}{2}}^{})`$
It follows from the identity $`ϵX_\alpha =X_\alpha ϵ`$ that conditions $`(C_{1\frac{1}{2}})`$ and $`(C_{1\frac{1}{2}}^{})`$ are equivalent, and that conditions $`(C_{\frac{1}{2}})`$ and $`(C_{\frac{1}{2}}^{})`$ are equivalent.

###### 5.11 Lemma

The following conditions are equivalent for a complex irreducible representation $`\pi _\lambda `$ of *real type* of a compact connected semisimple Lie group $`G`$, with invariant real form $`V`$:

1. condition $`(C_1)`$;
2. $`G`$ is transitive on the unit sphere of $`V`$;
3. $`G`$ acts with cohomogeneity $`1`$ on $`V`$;
4. $`\{Xp:X𝔤\}=(𝐑p)^{}`$ for all $`pV`$;
5. $`\{Xp:X𝔤\}=(𝐑p)^{}`$ for some $`pV`$.

Proof. (b), (c), (d), and (e) are clearly equivalent. We prove that (a) is equivalent to (e). In fact, let $`p=v_\lambda +ϵ(v_\lambda )`$. Clearly $`\{Xp:X𝔤\}(𝐑p)^{}`$. Using the basis (4.3), we find that $`\{Xp:X𝔤\}^c`$ is spanned over $`𝐂`$ by

$$v_\lambda ϵ(v_\lambda ),X_\alpha v_\lambda ,X_\alpha ϵ(v_\lambda ),\alpha \mathrm{\Delta }^+.$$

But condition $`(C_1)`$ holds precisely when this is also a system of generators for $`(𝐑p)^c`$ over $`𝐂`$.
$`\mathrm{}`$

###### 5.12 Lemma

The following conditions are equivalent for a complex representation $`\pi _\lambda `$ of *quaternionic or complex type* of a compact connected semisimple Lie group $`G`$ on $`V_\lambda `$, with highest weight $`v_\lambda `$:

1. $`𝒰^1(𝔤^c)v_\lambda =V_\lambda `$;
2. $`G`$ is transitive on the unit sphere of $`V_\lambda ^r`$;
3. $`G`$ acts with cohomogeneity $`1`$ on $`V_\lambda ^r`$;
4. $`\{Xp:X𝔤\}=(𝐑p)^{}`$ for all $`pV_\lambda ^r`$;
5. $`\{Xp:X𝔤\}=(𝐑p)^{}`$ for some $`pV_\lambda ^r`$.

Proof. The proof is analogous to the proof of Lemma 5.11. $`\mathrm{}`$

###### 5.13 Lemma

The self-dual complex irreducible representations of compact connected *simple* Lie groups satisfying condition $`(C_{\frac{1}{2}})`$ are precisely 555The half-spin representations of $`\mathrm{𝐒𝐩𝐢𝐧}(8)`$ are image equivalent to the vector representation.: the vector representation of $`\mathrm{𝐒𝐎}(m)`$ for $`m2`$, $`4`$, the vector representation of $`\mathrm{𝐒𝐩}(m)`$, the $`7`$-dimensional representation of $`𝐆_2`$, and the spin representation of $`\mathrm{𝐒𝐩𝐢𝐧}(7)`$.

We will prove Lemma 5.13 together with the next lemma.

###### 5.14 Lemma

The self-dual complex irreducible representations of compact connected *simple* Lie groups satisfying condition $`(C_1)`$ are, *besides those listed in Lemma 5.13*, precisely the following:

$$\begin{array}{ccc}& & \\ G& \pi _\lambda & k(\lambda )\\ & & \\ \mathrm{𝐒𝐔}(2)& \text{}& 3\\ \mathrm{𝐒𝐔}(6)& \text{}& 3\\ \mathrm{𝐒𝐩𝐢𝐧}(9)& \text{}& 2\\ \mathrm{𝐒𝐩𝐢𝐧}(11)& \text{}& 3\\ \mathrm{𝐒𝐩𝐢𝐧}(12)& \text{}& 3\\ \mathrm{𝐒𝐩}(3)& \text{}& 3\\ 𝐄_7& \text{}& 3\end{array}$$

Proof of Lemmas 5.13 and 5.14. It is obvious that condition $`(C_{\frac{1}{2}})`$ implies condition $`(C_1)`$. So we consider $`(C_1)`$. It implies the following:

1. $`k(\lambda )3`$.
2. The dimension of $`\pi _\lambda `$ is bounded by $`dim𝔤^c\text{rank}𝔤^c+2`$.
3. If $`0`$ is a weight, then its multiplicity is one. In particular, if $`\pi _\lambda `$ is the adjoint representation, then the rank is at most one.

In fact, (a) follows from the fact that $`k(\lambda )4`$ contradicts Proposition 4.4. For (b) notice that the dimension of the left hand side of $`(C_1)`$ is bounded by $`2(\text{number of positive roots})+2`$. Finally, if $`0`$ is a weight, then the $`0`$-weight space is contained in $`𝒰^1(𝔤^c)v_\lambda `$ (because $`ϵX_\alpha =X_\alpha ϵ`$), and from this we see that $`\lambda `$ is a root and the $`0`$-weight space is in fact spanned by $`X_\lambda v_\lambda `$, which implies (c).

We will use the above observations to complete the proof of the lemmas.

Let $`\pi _\lambda `$ be of real type. Then $`k(\lambda )=2`$. The adjoint representations of $`𝐀_n`$, $`𝐁_n`$, $`𝐂_n`$, $`𝐃_n`$ with $`n2`$, and of $`𝐆_2`$, $`𝐅_4`$, $`𝐄_6`$, $`𝐄_7`$, $`𝐄_8`$ are immediately excluded. We run through all the remaining cases of representations of real type with $`k(\lambda )=2`$.

Let $`G`$ be of $`𝐀_n`$-type. Then $`\pi _\lambda `$ is or which are cited in the lemmas.

Let $`G`$ be of $`𝐁_n`$-type, $`n2`$. Then $`\pi _\lambda `$ is or or . All of them are cited in the lemmas.

Let $`G`$ be of $`𝐂_n`$-type, $`n3`$. Then $`\pi _\lambda `$ is , which cannot occur because the multiplicity of the $`0`$-weight is $`n1`$.

Let $`G`$ be of $`𝐃_n`$-type, $`n4`$. Then $`\pi _\lambda `$ is    , which is cited in the lemmas.

Let $`G`$ be of exceptional type. Then $`\pi _\lambda `$ is or . Only the former is cited in the lemmas; the latter has $`0`$ as a weight of multiplicity $`2`$.

Now let $`\pi _\lambda `$ be of quaternionic type. Then $`k(\lambda )=1`$ or $`k(\lambda )=3`$. The case $`k(\lambda )=1`$ gives the vector representation of $`\mathrm{𝐒𝐩}(m)`$, which is cited in the lemmas. In the case $`k(\lambda )=3`$, the only possibilities not cited in the lemmas are the following four representations, which we next eliminate:

* $`G=𝐁_6`$, $`\pi _\lambda :\text{}`$: here $`\lambda =\frac{1}{2}(\theta _1+\theta _2+\theta _3+\theta _4+\theta _5+\theta _6)`$, $`\mu =\frac{1}{2}(\theta _1+\theta _2+\theta _3\theta _4\theta _5\theta _6)`$ is a weight and $`\lambda \pm \mu `$ are not roots which violates condition $`(C_1)`$.

In the remaining cases we make use of the bound on the dimension of the representation (see e. g. Table 5 in the Reference Chapter of ):

* $`G=𝐂_n`$, $`\pi _\lambda :\text{}`$: $`dim\pi _\lambda =\frac{2}{3}n(n+1)(2n+1)>2n^2+2=dim𝐂_nn+2`$, for $`n2`$;
* $`G=𝐂_n`$, $`\pi _\lambda :\text{}`$: $`dim\pi _\lambda =\frac{8}{3}n(n1)(n+1)>2n^2+2=dim𝐂_nn+2`$, for $`n2`$;
* $`G=𝐂_n`$, $`\pi _\lambda :\text{}`$: $`dim\pi _\lambda =\frac{2}{3}n(n2)(2n+1)>2n^2+2=dim𝐂_nn+2`$, for $`n4`$;

On the other hand, the representations listed in the statements of the Lemmas 5.13 and 5.14 do satisfy $`(C_1)`$, as can be easily checked using the fact that each of them has all weights with multiplicities one. This concludes the classification of self-dual representations that satisfy $`(C_1)`$. As for condition $`(C_{\frac{1}{2}})`$, we observe that $`(C_{\frac{1}{2}})`$ implies that $`k(\lambda )2`$. This eliminates all representations in the table of Lemma 5.14 except ; that one can be eliminated by a direct check. Finally it is easy to verify that the representations cited in Lemma 5.13 do satisfy $`(C_{\frac{1}{2}})`$. $`\mathrm{}`$

###### 5.15 Lemma

The only complex irreducible representation of real type of a compact connected semisimple *nonsimple* Lie group $`G`$ satisfying condition $`(C_1)`$ is the tensor product of the vector representations of $`\mathrm{𝐒𝐩}(1)`$ and $`\mathrm{𝐒𝐩}(m)`$.

Proof. We already know that $`k(\lambda )=2`$ (cf. proof of Lemmas 5.13 and 5.14), so $`G=G_1\times G_2`$, where $`G_i`$ is a simple group, and $`\pi _\lambda =\pi _{\mu _1}\pi _{\mu _2}`$ with $`k(\mu _i)=1`$. Since each $`\pi _{u_i}`$ is self-dual, it must be of quaternionic type. Therefore, each $`G_i=\mathrm{𝐒𝐩}(n_i)`$ and the corresponding $`\pi _{\mu _i}`$ is just the vector representation. Now one of the $`n_i`$ must be $`1`$, for otherwise condition $`(C_1)`$ for $`\pi _\lambda `$ is violated by consideration of a weight vector of type $`v_1v_2`$, where $`v_i`$ is an *intermediate* (that is, neither highest nor lowest) weight vector for $`\pi _{\mu _i}`$. $`\mathrm{}`$

###### 5.16 Lemma

The complex irreducible representations $`\pi _\lambda `$ of quaternionic type of compact connected simple Lie groups $`G`$ satisfying $`k(\lambda )=3`$ and condition $`(C_{1\frac{1}{2}})`$, *but not condition $`(C_1)`$*, are the following:

$$\begin{array}{cc}& \\ G& \pi _\lambda \\ & \\ \mathrm{𝐒𝐩𝐢𝐧}(13)& \text{}\\ \mathrm{𝐒𝐩}(2)& \text{}\end{array}$$

We postpone the proof of Lemma 5.16 to the end of Subsection 5.3.5, since the methods used to prove it better belong there.

#### 5.3.1 The case where $`G`$ has four simple factors

In this case we have:

###### 5.17 Proposition

Let $`\rho `$ be of real type and $`G=G_1\times G_2\times G_3\times G_4`$, where $`G_i`$ is a simple group. Then $`G=\mathrm{𝐒𝐔}(2)\times \mathrm{𝐒𝐔}(2)\times \mathrm{𝐒𝐔}(2)\times \mathrm{𝐒𝐩}(n)`$, $`n2`$, and $`\pi `$ is the tensor product of the vector representations of each of the factors.

Proof. Set $`\pi _\lambda =\pi _{\mu _1}\pi _{\mu _2}\pi _{\mu _3}\pi _{\mu _4}`$, where $`\pi _{\mu _i}`$ is the representation of $`G_i`$ of highest weight $`\mu _i`$. We have $`k(\mu _i)=1`$ for all $`i`$, and since each $`\pi _{\mu _i}`$ is self-dual, it must be of quaternionic type. Therefore, each $`G_i=\mathrm{𝐒𝐩}(n_i)`$ and the corresponding $`\pi _{\mu _i}`$ is just the vector representation. Notice that $`n_i=1`$ for all $`i`$ gives the isotropy representation of a symmetric space. Suppose now that for two indices, say $`3`$ and $`4`$, we have $`n_3>1`$ and $`n_4>1`$. Take weight vectors $`v_i`$ of $`\pi _{\mu _i}`$ such that: $`v_1`$ is the highest weight vector of $`\pi _{\mu _1}`$, $`v_2`$ is the lowest weight vector of $`\pi _{\mu _2}`$, and $`v_3`$ and $`v_4`$ are intermediate weight vectors of $`\pi _{\mu _3}`$ and $`\pi _{\mu _4}`$, respectively. Then $`v_1v_2v_3v_4`$ is a weight vector of $`\pi _\lambda `$ which cannot be contained in $`𝒰^2(𝔤^c)v_\lambda +𝒰^2(𝔤^c)ϵ(v_\lambda )`$, violating the condition $`(C_2)`$ from Lemma 4.8. $`\mathrm{}`$

#### 5.3.2 The case where $`G`$ has three simple factors

The classification is:

###### 5.18 Proposition

Let $`\rho `$ be of real type and $`G=G_1\times G_2\times G_3`$, where $`G_i`$ is a simple group. Then $`\rho `$ is one of the following:

$$\begin{array}{ccc}& & \\ G& \rho & \text{Conditions}\\ & & \\ \mathrm{𝐒𝐔}(2)\times \mathrm{𝐒𝐩}(n)\times \mathrm{𝐒𝐎}(2m)& \text{}\text{}\text{}& m3,n2\\ \mathrm{𝐒𝐔}(2)\times \mathrm{𝐒𝐩}(n)\times \mathrm{𝐒𝐎}(2m+1)& \text{}\text{}\text{}& n2\\ \mathrm{𝐒𝐔}(2)\times \mathrm{𝐒𝐩}(n)\times 𝐆_2& \text{}\text{}\text{}& \\ \mathrm{𝐒𝐔}(2)\times \mathrm{𝐒𝐩}(n)\times \mathrm{𝐒𝐩𝐢𝐧}(7)& \text{}\text{}\text{}& \\ \mathrm{𝐒𝐔}(2)\times \mathrm{𝐒𝐔}(2)\times \mathrm{𝐒𝐩𝐢𝐧}(9)& \text{}\text{}\text{}& \end{array}$$

Proof. Let $`\pi _\lambda =\pi _{\mu _1}\pi _{\mu _2}\pi _{\mu _3}`$, where $`\pi _{\mu _i}`$ is the representation of $`G_i`$ of highest weight $`\mu _i`$, say with $`k(\mu _1)=k(\mu _2)=1`$, $`k(\mu _3)=2`$. Then $`\pi _{\mu _1}`$, $`\pi _{\mu _2}`$ are each of quaternionic type and $`\pi _{\mu _3}`$ is of real type. So each of $`\pi _{\mu _1}`$, $`\pi _{\mu _2}`$ is the vector representation of $`\mathrm{𝐒𝐩}(m)`$ for some $`m`$. Moreover, one of them must have $`m=1`$, say $`\pi _{\mu _1}`$, for otherwise condition $`(C_2)`$ for $`\pi _\lambda `$ is violated by consideration of a weight vector of type $`v_1v_2v_3`$, where $`v_i`$ is an intermediate weight vector for $`\pi _{\mu _i}`$. Now we have $`G_1=\mathrm{𝐒𝐔}(2)`$, $`G_2=\mathrm{𝐒𝐩}(n)`$, and $`\pi _{\mu _1}`$, $`\pi _{\mu _2}`$ the respective vector representations. Next take a weight vector $`v_1v_2v_3`$ for $`\pi _\lambda `$ such that $`v_1`$ is highest, $`v_2`$ is lowest and $`v_3`$ is intermediate. Condition $`(C_2)`$ for $`\pi _\lambda `$ forces $`\pi _{\mu _3}`$ to satisfy condition $`(C_1)`$. If $`n2`$, we could also take $`v_2`$ to be intermediate, and then condition $`(C_2)`$ for $`\pi _\lambda `$ implies something stronger, namely that $`\pi _{\mu _3}`$ must satisfy condition $`(C_{\frac{1}{2}})`$. Our table now follows from Lemmas 5.13 and 5.14. $`\mathrm{}`$

#### 5.3.3 The case where $`G`$ has two simple factors and $`\rho `$ has two factors of real type

Here the result is:

###### 5.19 Proposition

Let $`\rho `$ be of real type and $`G=G_1\times G_2`$, where $`G_i`$ is a simple group. Write $`\rho ^c=\pi _\lambda `$, decompose $`\pi _\lambda =\pi _{\mu _1}\pi _{\mu _2}`$ and suppose that each $`\pi _{\mu _i}`$ is of real type. Then each $`\pi _{\mu _i}`$ is one of the following representations: the vector representation of $`\mathrm{𝐒𝐎}(m)`$ for $`m2`$, $`4`$, or the $`7`$-dimensional representation of $`𝐆_2`$, or the spin representation of $`\mathrm{𝐒𝐩𝐢𝐧}(7)`$, or the spin representation of $`\mathrm{𝐒𝐩𝐢𝐧}(9)`$; but the case where $`G_1=G_2=\mathrm{𝐒𝐩𝐢𝐧}(9)`$ is excluded.

Proof. Here $`k(\mu _1)=k(\mu _2)=2`$. Let $`v_1v_2`$ be a weight vector for $`\pi _\lambda `$ such that each $`v_i`$ is an intermediate weight vector for $`\pi _{\mu _i}`$. Condition $`(C_2)`$ for $`\pi _\lambda `$ applied to $`v_1v_2`$ forces each $`\pi _{\mu _i}`$ to satisfy $`(C_1)`$. In fact, one of the $`\pi _{\mu _i}`$ must even satisfy $`(C_{\frac{1}{2}})`$. The claim follows from Lemmas 5.13 and 5.14. $`\mathrm{}`$

#### 5.3.4 The case where $`G`$ has two simple factors and $`\rho `$ has two factors of quaternionic type

The classification is:

###### 5.20 Proposition

Let $`\rho `$ be of real type and $`G=G_1\times G_2`$, where $`G_i`$ is a simple group. Write $`\rho ^c=\pi _\lambda `$, decompose $`\pi _\lambda =\pi _{\mu _1}\pi _{\mu _2}`$ and suppose that each $`\pi _{\mu _i}`$ is of quaternionic type. Then $`\rho `$ is one of the following:

$$\begin{array}{ccc}& & \\ G& \rho & Conditions\\ & & \\ \mathrm{𝐒𝐩}(n)\times \mathrm{𝐒𝐔}(2)& \text{}\text{}& n2\\ \mathrm{𝐒𝐩}(n)\times \mathrm{𝐒𝐔}(6)& \text{}\text{}& n2\\ \mathrm{𝐒𝐩}(n)\times \mathrm{𝐒𝐩𝐢𝐧}(11)& \text{}\text{}& \\ \mathrm{𝐒𝐩}(n)\times \mathrm{𝐒𝐩𝐢𝐧}(12)& \text{}\text{}& n2\\ \mathrm{𝐒𝐩}(1)\times \mathrm{𝐒𝐩𝐢𝐧}(13)& \text{}\text{}& \\ \mathrm{𝐒𝐩}(1)\times \mathrm{𝐒𝐩}(2)& \text{}\text{}& \\ \mathrm{𝐒𝐩}(n)\times \mathrm{𝐒𝐩}(3)& \text{}\text{}& n2\\ \mathrm{𝐒𝐩}(n)\times 𝐄_7& \text{}\text{}& n2\end{array}$$

Proof. If $`k(\lambda )=2`$, then $`k(\mu _1)=k(\mu _2)=1`$ and each $`\pi _{\mu _i}`$ must be the vector representation of $`\mathrm{𝐒𝐩}(n_i)`$ for some $`n_i`$, so we get a representation associated to a symmetric space. In the case $`k(\lambda )=4`$ we can write $`k(\mu _1)=1`$ and $`k(\mu _2)=3`$. Now $`\pi _{\mu _1}`$ must be the vector representation of $`\mathrm{𝐒𝐩}(n)`$ and there are two cases to consider:

* $`n=1`$: here $`\pi _{\mu _2}`$ must satisfy condition $`(C_{1\frac{1}{2}})`$;
* $`n2`$: here $`\pi _{\mu _2}`$ must satisfy the stronger condition $`(C_1)`$.

The claim now follows from Lemmas 5.14 and 5.16. $`\mathrm{}`$

#### 5.3.5 The case where $`G`$ is a simple group

The representations of real type of simple groups that are candidates to a position in class $`𝒪^2`$ are:

###### 5.21 Proposition

Let $`\rho `$ be of real type and suppose $`G`$ is a simple group. Then $`\rho `$ is one of the following:

$$\begin{array}{cc}& \\ G& \rho \\ & \\ \mathrm{𝐒𝐩𝐢𝐧}(7)& \text{}\\ \mathrm{𝐒𝐩𝐢𝐧}(9)& \text{}\\ \mathrm{𝐒𝐩𝐢𝐧}(15)& \text{}\\ \mathrm{𝐒𝐩𝐢𝐧}(17)& \text{}\end{array}$$

We will accomplish the proof of the above proposition by running through all the cases of representations of real type of a simple group with $`k(\lambda )4`$ given in Table B.1. In fact, according to that table, all representations with $`k(\lambda )=2`$ are either associated to symmetric spaces or of cohomogeneity one. So we only need to worry about the case $`k(\lambda )=4`$. A careful inspection of Table B.1 shows:

###### 5.22 Lemma

Let $`\pi _\lambda `$ be a representation of real type of a compact connected simple Lie group $`G`$ with $`k(\lambda )=4`$. Then $`(G,\pi _\lambda )`$ is one of the following:

$$\begin{array}{cccc}& & & \\ & G& \pi _\lambda & Conditions\\ & & & \\ \hfill 1& 𝐀_1& \text{}& \\ \hfill 2& 𝐀_3& \text{}& \\ \hfill 3& 𝐀_3& \text{}& \\ \hfill 4& 𝐀_7& \text{}& \\ \hfill 5& 𝐀_n& \text{}& n2\\ \hfill 6& 𝐀_n& \text{}& n4\end{array}$$

$$\begin{array}{cccc}& & & \\ & G& \pi _\lambda & Conditions\\ & & & \\ \hfill 7& 𝐁_3& \text{}& \\ \hfill 8& 𝐁_3& \text{}& \\ \hfill 9& 𝐁_3& \text{}& \\ \hfill 10& 𝐁_4& \text{}& \\ \hfill 11& 𝐁_4& \text{}& \\ \hfill 12& 𝐁_4& \text{}& \\ \hfill 13& 𝐁_7& \text{}& \\ \hfill 14& 𝐁_8& \text{}& \\ \hfill 15& 𝐁_n& \text{}& n2\\ \hfill 16& 𝐁_n& \text{}& n3\\ \hfill 17& 𝐁_n& \text{}& n3\\ \hfill 18& 𝐁_n& \text{}& n4\\ \hfill 19& 𝐁_n& \text{}& n5\end{array}$$

$$\begin{array}{cccc}& & & \\ & G& \pi _\lambda & Conditions\\ & & & \\ \hfill 20& 𝐂_n& \text{}& n2\\ \hfill 21& 𝐂_n& \text{}& n2\\ \hfill 22& 𝐂_n& \text{}& n3\\ \hfill 23& 𝐂_n& \text{}& n3\\ \hfill 24& 𝐂_n& \text{}& n4\end{array}$$

$$\begin{array}{cccc}& & & \\ & G& \pi _\lambda & Conditions\\ & & & \\ \hfill 25& 𝐃_4& \text{}& \\ \hfill 26& 𝐃_5& \text{}& \\ \hfill 27& 𝐃_8& \text{}& \\ \hfill 28& 𝐃_n& \text{}& n4\\ \hfill 29& 𝐃_n& \text{}& n4\\ \hfill 30& 𝐃_n& \text{}& n4\\ \hfill 31& 𝐃_n& \text{}& n5\\ \hfill 32& 𝐃_n& \text{}& n6\end{array}$$

$$\begin{array}{ccc}& & \\ & G& \pi _\lambda \\ & & \\ \hfill 33& 𝐆_2& \text{}\\ \hfill 34& 𝐆_2& \text{}\\ \hfill 35& 𝐆_2& \text{}\\ \hfill 36& 𝐅_4& \text{}\\ \hfill 37& 𝐅_4& \text{}\\ \hfill 38& 𝐅_4& \text{}\\ \hfill 39& 𝐅_4& \text{}\\ \hfill 40& 𝐄_6& \text{}\\ \hfill 41& 𝐄_6& \text{}\\ \hfill 42& 𝐄_7& \text{}\\ \hfill 43& 𝐄_7& \text{}\\ \hfill 44& 𝐄_8& \text{}\\ \hfill 45& 𝐄_8& \text{}\end{array}$$

To begin with, we note the representations listed in the tables of Lemma 5.22 associated to a symmetric space, namely, numbers $`1`$, $`2`$, $`4`$, $`15`$, $`24`$ ($`n=4)`$, $`27`$ and $`28`$.

###### 5.23 Lemma

Let $`\pi _\lambda `$ be a representation of real type with $`k(\lambda )=4`$. Assume that $`\lambda =\frac{1}{2}(3\beta _i+\beta _j)`$ where $`\beta _i`$, $`\beta _j`$ are distinct long roots and that there exists a Weyl group involution $`s`$ such that $`s\beta _i=\beta _j`$. Then a real form of $`\pi _\lambda `$ is not of class $`𝒪^2`$.

Proof. We have that $`\mu =ss_{\beta _i}(\lambda )=\frac{1}{2}(\beta _i3\beta _j)`$ is a weight and $`\lambda \mu =\beta _i+2\beta _j`$, $`\lambda +\mu =2\beta _i\beta _j`$. Since $`\beta _i`$ and $`\beta _j`$ are orthogonal, we get that $`\lambda \pm \mu ^2=5`$ (normalizing the length of a long root to be $`1`$). On the other hand, for $`\alpha ^{}`$, $`\alpha ^{\prime \prime }\mathrm{\Delta }`$ we have that $`\alpha ^{}+\alpha ^{\prime \prime }^2(\alpha ^{}+\alpha ^{\prime \prime })^24`$, so $`\lambda \pm \mu `$ is neither a root nor a sum of two roots. Thus a $`\mu `$-weight vector is not in $`𝒰^2(𝔤^c)v_\lambda +𝒰^2(𝔤^c)ϵ(v_\lambda )`$, violating condition $`(C_2)`$. $`\mathrm{}`$

One can use the last lemma together with the tables in Appendix A to get rid of the representations listed in Lemma 5.22 under the numbers $`3`$, $`12`$, $`17`$, $`21`$, $`30`$ and $`37`$.

We next eliminate the representations $`\pi _\lambda `$ with $`\lambda =2(\text{highest root})=2\beta _1=\frac{1}{2}(4\beta _1)`$, using the following observations.

###### 5.24 Lemma

Let $`\pi _\lambda `$ be a representation of real type with $`\lambda =2\beta _1`$ and assume there is a nonzero weight $`\mu `$ such that $`(\lambda ,\mu )=0`$. Then a real form of $`\pi _\lambda `$ is not of class $`𝒪^2`$.

Proof. We check that condition $`(C_2)`$ is violated. In fact, $`\lambda \pm \mu ^2=\lambda +\mu ^2>4\beta _1^2=4`$ (normalizing the length of a long root to be $`1`$), but $`\alpha ^{}`$, $`\alpha ^{\prime \prime }\mathrm{\Delta }`$ implies that $`\alpha ^{}+\alpha ^{\prime \prime }^2(\alpha ^{}+\alpha ^{\prime \prime })^24`$, and we are done as in the proof of Lemma 5.23. $`\mathrm{}`$

###### 5.25 Corollary

Let $`\pi _\lambda `$ be a representation of real type with $`\lambda =2\beta _1`$. If there is a long root $`\beta _j`$ with $`\beta _j\beta _1`$, then a real form of $`\pi _\lambda `$ is not of class $`𝒪^2`$.

Proof. For in this case $`2\beta _1`$, $`\beta _1`$, $`0`$, $`\beta _1`$, $`2\beta _1`$ is the maximal $`\beta _1`$-string through $`2\beta _1`$, so $`\beta _1`$ is a weight, $`\beta _j`$ is a weight, too ($`\beta _1`$ and $`\beta _j`$ are long roots, hence in the same Weyl orbit, see e. g. , Ch. VI, § 1, no. 1.3, Proposition 11) and $`(\beta _1,\beta _j)=0`$. $`\mathrm{}`$

We use Corollary 5.25 to discard the representations listed in the tables of Lemma 5.22 under the numbers $`5`$ ($`n3`$), $`16`$, $`20`$, $`29`$, $`36`$, $`41`$, $`42`$ and $`45`$. And we use Lemma 5.24 with $`\mu =\alpha _1+\alpha _2`$ (resp. $`\mu =\beta _2=\alpha _1`$) to eliminate representation number $`5`$ for $`n=2`$ (resp. number $`34`$). There still remain $`20`$ representations to be eliminated in the tables, namely numbers $`6`$, $`7`$, $`9`$, $`10`$, $`18`$, $`19`$, $`22`$, $`23`$, $`24`$ ($`n5`$), $`25`$, $`26`$, $`31`$, $`32`$, $`33`$, $`35`$, $`38`$, $`39`$, $`40`$, $`43`$ and $`44`$, which in the following we go on to analyze case by case, but before that we want to reformulate an argument that has already been used and that will frequently be used in the sequel.

###### 5.26 Lemma

Let $`\pi _\lambda `$ be a representation of real type and suppose that its highest weight $`\lambda `$ can be written as a root or as a sum of two roots in $`N`$ different ways (where the order of the summands is *not* important). If $`0`$ is a weight and its multiplicity is bigger than $`N`$, then a real form of $`\pi _\lambda `$ is not of class $`𝒪^2`$.

Proof. Write $`v_\lambda `$ for a highest weight vector of $`\pi _\lambda `$. If condition $`(C_2)`$ is satisfied, then the $`0`$-weight space must be contained in $`𝒰^2(𝔤^c)v_\lambda +𝒰^2(𝔤^c)ϵ(v_\lambda )`$. The identity $`ϵX_\alpha =X_\alpha ϵ`$ shows that the $`0`$-weight space must in fact be contained in $`𝒰^2(𝔤^c)v_\lambda `$. But the complex dimension of the intersection of the $`0`$-weight space with $`𝒰^2(𝔤^c)v_\lambda `$ is at most $`N`$. $`\mathrm{}`$

In the remainder of this section several decompositions of tensor products of representations will be used, all of which can be found in Table 5 in the Reference Chapter of .

###### 5.27 Lemma

Representation number $`6`$ is not of class $`𝒪^2`$.

Proof. We label the simple roots of $`𝐀_n`$ as , where $`\alpha _i=\theta _i\theta _{i+1}`$. Then the root system is $`\mathrm{\Delta }=\{\pm (\theta _i\theta _j):1i<jn+1\}`$ and the highest weight of is $`\lambda =\lambda _2+\lambda _{n1}=\theta _1+\theta _2\theta _n\theta _{n+1}`$, so it can decomposed as a sum of two roots in exactly two ways:

$`\lambda `$ $`=`$ $`(\theta _1\theta _{n+1})+(\theta _2\theta _n)`$
$`=`$ $`(\theta _1\theta _n)+(\theta _2\theta _{n+1}).`$
On the other hand, we find that:

$`\text{}\text{}=`$
$`\text{}\text{}\text{(trivial)},`$
from where we deduce that the multiplicity of the $`0`$-weight in $`\pi _\lambda `$ is $`\frac{n(n+1)}{2}n1=\frac{n^2n2}{2}>2`$, so we can apply Lemma 5.26. $`\mathrm{}`$

###### 5.28 Lemma

Representations numbers $`18`$, $`10`$ and $`19`$ are not of class $`𝒪^2`$.

Proof. We have that

$`=`$ $`\mathrm{\Lambda }^3(\text{}),n4,`$
$`=`$ $`\mathrm{\Lambda }^4(\text{}),`$
$`=`$ $`\mathrm{\Lambda }^4(\text{}),n5.`$
This shows that the multiplicity of the $`0`$-weight is, respectively, $`n`$, $`6`$ and $`\left(\genfrac{}{}{0pt}{}{n}{2}\right)`$ (because the vector representation has weights $`\theta _1`$, $`\theta _2,\mathrm{},\theta _n`$, $`0`$, $`\theta _n`$, $`\theta _{n1},\mathrm{},\theta _1`$). But the integer $`N=3`$ in all cases. $`\mathrm{}`$

###### 5.29 Lemma

Representation number $`9`$ is not of class $`𝒪^2`$.

Proof. The highest weight is $`\lambda =\frac{1}{2}(3\theta _1+3\theta _2+\theta _3)`$. Now $`\mu =\frac{1}{2}(3\theta _13\theta _2+\theta _3)`$ is a weight and $`\lambda +\mu =3\theta _1+\theta _3`$, $`\lambda \mu =3\theta _2`$ are neither a root nor a sum of two roots, so we are done as in the proof of Lemma 5.23. $`\mathrm{}`$

###### 5.30 Lemma

Representations numbers $`22`$ and $`23`$ are not of class $`𝒪^2`$. Representation number $`24`$ is not of class $`𝒪^2`$ if $`n5`$.

Proof. We first observe that:

* the vector representation $`C_n:\text{}`$ has weights $`\pm \theta _i`$ with multiplicity $`1`$;
* the representation is a subrepresentation of $`\mathrm{\Lambda }^2(\text{})`$ and has weights $`\pm \theta _i\pm \theta _j`$ with multiplicity $`1`$, and $`0`$ with multiplicity $`n1`$;
* the representation is a subrepresentation of $`\mathrm{\Lambda }^3(\text{})`$ and has weights $`\pm \theta _i\pm \theta _j\pm \theta _k`$ with multiplicity $`1`$, and $`\pm \theta _i`$ with multiplicity $`n2`$;
* the representation is a subrepresentation of

$$\mathrm{\Lambda }^4(\text{})$$

and has weights $`\pm \theta _i\pm \theta _j\pm \theta _k\pm \theta _l`$ with multiplicity $`1`$, and $`\pm \theta _i\pm \theta _j`$ with multiplicity $`n1`$, and $`0`$ with multiplicity $`\left(\genfrac{}{}{0pt}{}{n}{2}\right)n`$.

If $`n5`$ then the multiplicity of the $`0`$-weight in is $`\frac{1}{2}n(n3)5`$. Since $`\lambda =\theta _1+\theta _2+\theta _3+\theta _4`$ can be written as a root or as a sum of two roots in $`N=3`$ ways only, namely,

$`\lambda `$ $`=`$ $`(\theta _1+\theta _2)+(\theta _3+\theta _4)`$
$`=`$ $`(\theta _1+\theta _3)+(\theta _2+\theta _4)`$
$`=`$ $`(\theta _1+\theta _4)+(\theta _2+\theta _3),`$
Lemma 5.26 says that , $`n5`$, cannot be of class $`𝒪^2`$.

Next, it follows from

$`\text{}\text{}=\text{}\text{},`$
$`\text{}\text{}=`$
$`\text{}\text{}\text{},n4,`$
that the multiplicity of the $`0`$-weight in , $`n3`$, is given by $`\frac{1}{2}(n2)(3n1)4`$. But $`\lambda =2\theta _1+\theta _2+\theta _3`$ can be written as a root or as sum of two roots in $`N=2`$ ways only:

$`\lambda `$ $`=`$ $`(\theta _1+\theta _2)+(\theta _1+\theta _3)`$
$`=`$ $`2\theta _1+(\theta _2+\theta _3).`$
Lemma 5.26 now implies that , $`n3`$, is not of class $`𝒪^2`$.

One can also see that $`S^2(\text{})`$, $`n3`$, has $`0`$ as a weight of multiplicity $`\frac{3}{2}n(n1)`$. Now

$`S^2(\text{})=\text{}\text{}\text{(trivial)},`$
$`S^2(\text{})=`$
$`\text{}\text{}\text{}\text{(trivial)},n4,`$
so the multiplicity of the $`0`$-weight in , $`n3`$, is $`n(n1)6`$. But $`\lambda =2\theta _1+2\theta _2`$ can be written as a root or as sum of two roots in $`N=2`$ ways only:

$`\lambda `$ $`=`$ $`(\theta _1+\theta _2)+(\theta _1+\theta _2)`$
$`=`$ $`2\theta _1+2\theta _2.`$
Lemma 5.26 implies again that , $`n3`$, is not of class $`𝒪^2`$. $`\mathrm{}`$

###### 5.31 Lemma

Representations numbers $`26`$ and $`32`$ are not of class $`𝒪^2`$.

Proof. These representations are just

$$𝐃_n:\mathrm{\Lambda }^4(\text{}),n5,$$

where    is the vector representation. Therefore the multiplicity of the $`0`$-weight is $`\left(\genfrac{}{}{0pt}{}{n}{2}\right)10`$. But the highest weight $`\lambda =\theta _1+\theta _2+\theta _3+\theta _4`$ decomposes as a sum of two roots in $`N=3`$ ways, so we can use Lemma 5.26. $`\mathrm{}`$

###### 5.32 Lemma

Representation number $`35`$ is not of class $`𝒪^2`$.

Proof. Here $`\lambda =\lambda _1+\lambda _2=5\alpha _1+3\alpha _2`$ decomposes only as $`(2\alpha _1+\alpha _2)+(3\alpha _1+2\alpha _2)`$, so that $`N=1`$. Now we find that:

$$\text{}\text{}=\text{}\text{}\text{},$$

where is the vector representation and $`S^2(\text{})=\text{}\text{(trivial)}`$. Therefore the multiplicity of the $`0`$-weight in is $`4`$ and we can use Lemma 5.26. $`\mathrm{}`$

###### 5.33 Lemma

Representations numbers $`38`$ and $`39`$ are not of class $`𝒪^2`$.

Proof. First we observe that the $`26`$-dimensional representation $`𝐅_4:\text{}`$ has weights $`\frac{1}{2}(\pm \theta _1\pm \theta _2\pm \theta _3\pm \theta _4)`$ and $`\pm \theta _i`$ with multiplicity $`1`$, and $`0`$ with multiplicity $`2`$. Therefore $`\mathrm{\Lambda }^2(\text{})`$ has $`0`$ as a weight of multiplicity $`13`$. Now

$$\mathrm{\Lambda }^2(\text{})=\text{}\text{},$$

where is the adjoint representation. It follows that $`\pi _\lambda :\text{}`$ has $`0`$ as a weight of multiplicity $`9`$. But its highest weight $`\lambda =\frac{1}{2}(3\theta _1+\theta _2+\theta _3+\theta _4)`$ is not a root and decomposes as a sum of two roots as follows:

$`\lambda `$ $`=`$ $`{\displaystyle \frac{1}{2}}(\theta _1+\theta _2+\theta _3+\theta _4)+\theta _1`$
$`=`$ $`{\displaystyle \frac{1}{2}}(\theta _1\theta _2+\theta _3+\theta _4)+(\theta _1+\theta _2)`$
$`=`$ $`{\displaystyle \frac{1}{2}}(\theta _1+\theta _2\theta _3+\theta _4)+(\theta _1+\theta _3)`$
$`=`$ $`{\displaystyle \frac{1}{2}}(\theta _1+\theta _2+\theta _3\theta _4)+(\theta _1+\theta _4).`$
Thus the integer of Lemma 5.26 is $`N=4`$ and is out.

Similarly, the multiplicity of the $`0`$ weight in $`S^2(\text{})`$ is 15, and

$$S^2(\text{})=\text{}\text{}\text{(trivial)}.$$

It follows that $`\pi _\lambda :\text{}`$ has $`0`$ as a weight of multiplicity $`12`$. But its highest weight $`\lambda =2\theta _1`$ is not a root and decomposes as a sum of two roots as follows:

$`\lambda `$ $`=`$ $`(\theta _1)+(\theta _1)`$
$`=`$ $`(\theta _1+\theta _2)+(\theta _1\theta _2)`$
$`=`$ $`(\theta _1+\theta _3)+(\theta _1\theta _3)`$
$`=`$ $`(\theta _1+\theta _4)+(\theta _1\theta _4).`$
Thus the integer of Lemma 5.26 is $`N=4`$ and is out, too. $`\mathrm{}`$

###### 5.34 Lemma

Representation number $`40`$ is not of class $`𝒪^2`$.

Proof. We have that

and
are respectively the $`27`$-dimensional representation and its dual. Therefore the multiplicity of the $`0`$-weight in the tensor product of these representaions is at least $`27`$. But

$$\text{}\text{}=\text{}\text{}\text{(trivial)}$$

where the second summand on the right hand side is the adjoint representation. It follows that the multiplicity of the $`0`$-weight in $`\pi _\lambda :\text{}`$ is at least $`20`$. But its highest weight $`\lambda =\theta _8\theta _7\theta _6+\theta _5`$ is not a root and decomposes as a sum of two roots as follows:

$`\lambda `$ $`=`$ $`{\displaystyle \frac{1}{2}}(\theta _8\theta _7\theta _6+\theta _5+\theta _4+\theta _3+\theta _2+\theta _1)+{\displaystyle \frac{1}{2}}(\theta _8\theta _7\theta _6+\theta _5\theta _4\theta _3\theta _2\theta _1)`$
$`=`$ $`{\displaystyle \frac{1}{2}}(\theta _8\theta _7\theta _6+\theta _5+\theta _4+\theta _3\theta _2\theta _1)+{\displaystyle \frac{1}{2}}(\theta _8\theta _7\theta _6+\theta _5\theta _4\theta _3+\theta _2+\theta _1)`$
$`=`$ $`{\displaystyle \frac{1}{2}}(\theta _8\theta _7\theta _6+\theta _5+\theta _4\theta _3+\theta _2\theta _1)+{\displaystyle \frac{1}{2}}(\theta _8\theta _7\theta _6+\theta _5\theta _4+\theta _3\theta _2+\theta _1)`$
$`=`$ $`{\displaystyle \frac{1}{2}}(\theta _8\theta _7\theta _6+\theta _5\theta _4+\theta _3+\theta _2\theta _1)+{\displaystyle \frac{1}{2}}(\theta _8\theta _7\theta _6+\theta _5+\theta _4\theta _3\theta _2+\theta _1).`$
Thus the integer of Lemma 5.26 is $`N=4`$ and we are done. $`\mathrm{}`$

###### 5.35 Lemma

Representation number $`43`$ is not of class $`𝒪^2`$.

Proof. We have that     is the $`56`$-dimensional representation. It is self-dual and does not have $`0`$ as a weight. Therefore the multiplicity of the $`0`$-weight in the exterior square of this representation is $`28`$. But

$$\mathrm{\Lambda }^2(\text{})=\text{}\text{(trivial)}.$$

Therefore the multiplicity of the $`0`$-weight in $`\pi _\lambda `$ is $`27`$. But its highest weight $`\lambda =\theta _8\theta _7+\theta _6+\theta _5`$ is not a root and one can easily check that it decomposes as a sum of two roots in exactly $`N=17`$ different ways. $`\mathrm{}`$

###### 5.36 Lemma

Representation number $`44`$ is not of class $`𝒪^2`$.

Proof. Here we slightly change the argument used in the previous lemmas. The highest weight $`\lambda =2\theta _8`$ and $`(\lambda ,\theta _8\theta _7)0`$, so $`\nu =\lambda (\theta _8\theta _7)=\theta _8+\theta _7`$ is a weight. Now $`\theta _8+\theta _7`$ and $`\theta _6+\theta _5`$ are long roots of $`𝐄_8`$, hence in the same Weyl orbit. It follows that $`\mu =\theta _6+\theta _5`$ is a weight, too. Observe that each of $`\lambda \pm \mu `$ is not a root and can be decomposed as a sum of two roots in only one way:

$`\lambda +\mu `$ $`=`$ $`(\theta _8+\theta _6)+(\theta _8+\theta _5),`$
$`\lambda \mu `$ $`=`$ $`(\theta _8\theta _6)+(\theta _8\theta _5).`$
The proof will be complete if we show that the multiplicity of $`\mu `$ as a weight is greater than $`2`$, since then we will have that condition $`(C_2)`$ is violated.

In fact, the multiplicity of $`\mu `$ is the same as the multiplicity of $`\nu `$. We use Freudenthal’s formula as it is stated in , namely, the multiplicity of $`\nu `$ as a weight of $`\pi _\lambda `$ is given by the following formula:

$$m_\nu =\frac{2}{c(\nu )}\underset{\alpha \mathrm{\Delta }^+}{}\underset{k1}{}(\nu +k\alpha ,\alpha )m_{\nu +k\alpha },$$

where $`c(\nu )=\lambda +\rho ^2\nu +\rho ^2`$, $`\rho =\frac{1}{2}_{\alpha \mathrm{\Delta }^+}\alpha `$ and $`m_{\nu +k\alpha }`$ is the multiplicity of $`\nu +k\alpha `$ as a weight of $`\pi _\lambda `$.

We content ourselves with an estimate. Using that $`\rho =\theta _2+2\theta _3+3\theta _4+4\theta _5+5\theta _6+6\theta _7+23\theta _8`$, we compute that $`c(\nu )=36\theta _1^2`$. Since $`\nu >0`$, all terms in Freudenthal’s formula are nonnegative. Consider the roots $`\alpha =\frac{1}{2}(\theta _8\theta _7+_{i=1}^6ϵ_i\theta _i)`$, $`\beta =\frac{1}{2}(\theta _8\theta _7_{i=1}^6ϵ_i\theta _i)`$, where $`\mathrm{\Pi }_{i=1}^6ϵ_i=1`$. Note that $`\alpha +\beta =\theta _8\theta _7`$. Since $`(\lambda ,\alpha )0`$, we have that $`\lambda \alpha `$ is a weight. Now we have two strings of weights starting at $`\nu `$, namely

$$\begin{array}{c}\nu ,\nu +(\theta _8\theta _7)=\lambda \text{and}\hfill \\ \nu ,\nu +\beta =\lambda \alpha .\hfill \end{array}$$

Note that in fact there are $`32`$ different possible choices of the signs $`ϵ_i=\pm 1`$, each of which gives rise to a different string of weights of the type $`\nu `$, $`\nu +\beta `$. Freudenthal’s formula gives

$`m_\nu `$ $``$ $`{\displaystyle \frac{2}{36\theta _1^2}}((\lambda ,\theta _8\theta _7)m_\lambda +32(\lambda \alpha ,\beta )m_{\lambda \alpha })`$
$`=`$ $`{\displaystyle \frac{1}{18}}(21+3221)={\displaystyle \frac{33}{9}}>3,`$
so $`m_\nu 4`$ and we are done. $`\mathrm{}`$

###### 5.37 Lemma

Representations numbers $`25`$ and $`31`$ are not of class $`𝒪^2`$.

Proof. Here we will use the sharper version of condition $`(C_2)`$ given by Lemma 4.8, item (a). These representations are just

$$\pi _\lambda :\mathrm{\Lambda }^3(\text{}),n4,$$

where    is the vector representation. The vector representation has weights $`\pm \theta _i`$, $`1in`$; let $`\{e_i,e_{n+i}:1in\}`$ be a basis of weight vectors such that $`e_i`$ (resp., $`e_{n+i}`$) corresponds to the $`\theta _i`$-weight (resp., $`\theta _i`$-weight) weight and $`ϵ(e_i)=e_{n+i}`$, where $`ϵ`$ is a invariant real structure on the representation space.

We have that $`v_\mu =e_1e_3e_{n+3}`$ is a $`(\mu =\theta _1)`$-weight vector for $`\pi _\lambda `$ and $`ϵ(v_\mu )=e_{n+1}e_{n+3}e_3`$. We want to show that the condition (a) of Lemma 4.8 is violated by proving that the $`\theta _2`$-weight vector $`e_2e_4e_{n+4}`$ is not in the complex span of

$$v_\mu ,ϵ(v_\mu ),X_\alpha v_\mu ,X_\alpha ϵ(v_\mu ),X_\beta X_\alpha (v_\mu ),X_\beta X_\alpha (ϵ(v_\mu )),$$

for $`\alpha `$, $`\beta \mathrm{\Delta }`$. But this follows immediately from the description of the action of a root vector $`X_\alpha `$, $`\alpha \mathrm{\Delta }=\{\pm (\theta _i\pm \theta _j):1i<jn\}`$, on the weight vectors $`\{e_i,e_{n+i}:1in\}`$ under the vector representation, given by the following matrices:

$`X_{\theta _i+\theta _j}`$ $`=`$ $`E_{i,n+j}E_{j,n+i},`$
$`X_{\theta _i\theta _j}`$ $`=`$ $`E_{n+i,j}E_{n+j,i},`$
$`X_{\theta _i\theta _j}`$ $`=`$ $`E_{i,j}E_{n+j,n+i},`$
where $`E_{ij}`$ denotes the $`2n\times 2n`$-matrix with $`1`$ at the $`(i,j)`$-position and $`0`$ elsewhere. $`\mathrm{}`$

###### 5.38 Lemma

Representation number $`7`$ is not of class $`𝒪^2`$.

Proof. We use an argument somewhat similar to the one used in the previous lemma. The representation is $`\pi _\lambda :\mathrm{\Lambda }^3(\text{})`$, where is the vector representation. Let $`e_1`$, $`e_2`$, $`e_3`$, $`e_4=ϵ(e_1)`$, $`e_5=ϵ(e_2)`$, $`e_6=ϵ(e_3)`$, $`e_7=ϵ(e_7)`$ be weight vectors of the vector representation corresponding to the weights $`\theta _1`$, $`\theta _2`$, $`\theta _3`$, $`\theta _1`$, $`\theta _2`$, $`\theta _3`$, $`0`$, respectively, where $`ϵ`$ is the invariant real structure on the representation space.

We have that $`v_\mu =e_1e_4e_7`$ is a ($`\mu =0`$)-weight vector for $`\pi _\lambda `$ and $`ϵ(v_\mu )=v_\mu `$. Now condition (a) of Lemma 4.8 is violated because the $`\theta _2`$-weight vector $`e_2e_3e_6`$ is not in the complex span of

$$v_\mu ,X_\alpha v_\mu ,X_\beta X_\alpha (v_\mu ),$$

for $`\alpha `$, $`\beta \mathrm{\Delta }`$. $`\mathrm{}`$

###### 5.39 Lemma

Representation number $`33`$ is not of class $`𝒪^2`$.

Proof. We start with a description of the Lie algebra of $`𝐆_2`$ and of its $`7`$-dimensional representation, as it is done in . Let $`\alpha _1`$, $`\alpha _2`$ be the simple roots. We label the other positive roots as $`\alpha _3=\alpha _1+\alpha _2`$, $`\alpha _4=2\alpha _1+\alpha _2`$, $`\alpha _5=3\alpha _1+\alpha _2`$, $`\alpha _6=3\alpha _1+2\alpha _2`$. Choose root vectors $`X_i`$, $`Y_i`$ for $`\alpha _i`$, $`\alpha _i`$, respectively, $`i=1`$, $`2`$, such that $`[H_i,X_i]=2Y_i`$, $`[H_i,Y_i]=2X_i`$, where $`H_i=[X_i,Y_i]`$, $`i=1`$, $`2`$. Next define

$$\begin{array}{cc}X_3=[X_1,X_2],\hfill & Y_3=[Y_1,Y_2],\hfill \\ X_4=\frac{1}{2}[X_1,X_3],\hfill & Y_4=\frac{1}{2}[Y_1,Y_3],\hfill \\ X_5=\frac{1}{3}[X_1,X_4],\hfill & Y_5=\frac{1}{3}[Y_1,Y_4],\hfill \\ X_6=[X_2,X_5],\hfill & Y_6=[Y_2,Y_5],\hfill \end{array}$$

and $`H_i=[X_i,Y_i]`$ for $`i=1,\mathrm{},6`$. Then $`[H_i,X_i]=2Y_i`$, $`[H_i,Y_i]=2X_i`$ for $`i=3,\mathrm{},6`$, and $`X_i`$, $`Y_i`$ are root vectors for $`\alpha _i`$, $`\alpha _i`$, respectively, $`i=1,\mathrm{},6`$.

The $`7`$-dimensional representation of $`𝐆_2`$ has weights $`\alpha _1`$, $`\alpha _1`$, $`\alpha _3`$, $`\alpha _3`$, $`\alpha _4`$, $`\alpha _4`$, $`0`$ with respective weight vectors $`v_1`$, $`w_1`$, $`v_3`$, $`w_3`$, $`v_4`$, $`w_4`$, $`u`$, such that the action of the basis vectors of the Lie algebra of $`𝐆_2`$ is described in the following table:

$$\begin{array}{ccccccccccccccc}& & & & & & & & & & & & & & \\ & H_1& H_2& X_1& X_2& X_3& X_4& X_5& X_6& Y_1& Y_2& Y_3& Y_4& Y_5& Y_6\\ & & & & & & & & & & & & & & \\ \hfill u& 0& 0& 2v_1& 0& 2v_3& 2v_4& 0& 0& 2w_1& 0& 2w_3& 2w_4& 0& 0\\ \hfill v_4& v_4& 0& 0& 0& 0& 0& 0& 0& v_3& 0& v_1& u& w_1& w_3\\ \hfill v_1& 2v_1& v_1& 0& v_3& v_4& 0& 0& 0& u& 0& 0& w_3& w_4& 0\\ \hfill v_3& v_3& v_3& v_4& 0& 0& 0& 0& 0& 0& v_1& u& w_1& 0& w_4\\ \hfill w_4& w_4& 0& w_3& 0& w_1& u& v_1& v_3& 0& 0& 0& 0& 0& 0\\ \hfill w_1& 2w_1& w_1& u& 0& 0& v_3& v_4& 0& 0& w_3& w_4& 0& 0& 0\\ \hfill w_3& w_3& w_3& 0& w_1& u& v_1& 0& v_4& w_4& 0& 0& 0& 0& 0\end{array}$$

Now we have the equation

$$S^2(\text{})=\text{}\text{(trivial)},$$

from which we learn three things. First, $`\pi _\lambda :\text{}`$ is a subrepresentation of the symmetric square of the $`7`$-dimensional representation, so we can read it off the table above. Second, the multiplicity of the weights of $`\pi _\lambda `$; in particular, $`\alpha _4`$ is a weight of multiplicity $`2`$. And third, the trivial summand above is spanned by $`\frac{1}{2}u^2+2v_1w_1+2v_3w_3+2v_4w_4`$, so that an invariant real structure on the representation space of the $`7`$-dimensional representation is given by $`ϵ(v_i)=w_i`$, $`i=1`$, $`2`$, $`3`$ and $`ϵ(u)=u`$.

Let $`v_\mu =2u^28v_1w_1+6v_3w_3+6v_4w_4`$, which is a real, ($`\mu =0`$)-weight vector for $`\pi _\lambda `$. We next show that condition (a) of Lemma 4.8 is violated because the $`\alpha _4`$-weight space is not contained in the complex span of

(5.40)
$$v_\mu ,X_\alpha v_\mu ,X_\beta X_\alpha (v_\mu ),$$

for $`\alpha `$, $`\beta \mathrm{\Delta }`$. In fact, $`\alpha _40`$ can be written as a root or as a sum of two roots as follows:

$`\alpha _40`$ $`=`$ $`2\alpha _1+\alpha _2`$
$`=`$ $`\alpha _1+(\alpha _1+\alpha _2)`$
$`=`$ $`\alpha _1+(3\alpha _1+\alpha _2)`$
$`=`$ $`(3\alpha _1+2\alpha _2)(\alpha _1+\alpha _2)`$
Now applying the corresponding root vectors to $`v_\mu `$ we get:

$`X_4(v_\mu )`$ $`=`$ $`14(uv_4+v_1v_3),`$
$`X_3X_1(v_\mu )`$ $`=`$ $`X_3(0)=0,`$
$`X_5Y_1(v_\mu )`$ $`=`$ $`X_5(0)=0,`$
$`Y_3X_6(v_\mu )`$ $`=`$ $`Y_3(0)=0.`$
This shows that the intersection of the complex span of (5.40) with the $`\alpha _4`$-weight space has dimension $`1`$; but the weight space itself has dimension $`2`$. $`\mathrm{}`$

Proof of Lemma 5.16. We eliminate the possibilities given by Table B.1 and not listed in the table.

Let $`G=𝐂_n`$, $`\pi _\lambda :\text{}`$, $`n2`$. Then $`\lambda =3\theta _1`$, $`\mu =3\theta _2`$ is a weight and $`\lambda \pm \mu `$ are neither roots nor a sum of two roots, so $`\pi _\lambda `$ does not even satisfy $`(C_2)`$.

Let $`G=𝐂_n`$, $`\pi _\lambda :\text{}`$, $`n3`$. We have

$`\text{}\text{}=`$
$`\text{}\text{}\text{},`$
from where we deduce that the multiplicity of $`\mu =\theta _3`$ as a weight of is $`2n24`$. Now $`\lambda \mu `$ is not a root and can be written as a sum of two roots in two ways,

$`\lambda \mu `$ $`=`$ $`(2\theta _1)+(\theta _2\theta _3)`$
$`=`$ $`(\theta _1+\theta _2)+(\theta _1\theta _3),`$
and $`\lambda +\mu =2\theta _1+\theta _2+\theta _3`$ is not a root. It follows that the dimension of the intersection of the $`\mu `$-weight space with $`𝒰^2(𝔤^c)v_\lambda +𝒰^1(𝔤^c)ϵ(v_\lambda )`$ is at most $`2`$, so that condition $`(C_{1\frac{1}{2}})`$ is not satisfied.

Let $`G=𝐂_n`$, $`\pi _\lambda :\text{}`$, $`n4`$. Then $`\lambda =\theta _1+\theta _2+\theta _3`$ and this representation is a subrepresentation of the cubic exterior power of the vector representation. Also, $`\mu =\theta _1`$ is a weight of multiplicity $`n22`$. But one can check by direct computation that the dimension of the intersection of the $`\mu `$-weight space with $`𝒰^2(𝔤^c)v_\lambda +𝒰^1(𝔤^c)ϵ(v_\lambda )`$ is $`1`$, so condition $`(C_{1\frac{1}{2}})`$ is not satisfied. $`\mathrm{}`$

## 6 The proof of the converse to a Theorem of Bott and Samelson

In this section we classify variationally complete representations. The strategy is as follows. We first exclude many irreducible representations in class $`𝒪^2`$ from being variationally complete by making use of Propositions 3.7 and 3.8. We then observe that the remaining irreducible representations are all orbit equivalent to isotropy representations of symmetric spaces, hence variationally complete by Proposition 3.3. Using Proposition 3.4, item (d), we see that arbitrary variationally complete representations are precisely those that are orbit equivalent to isotropy representations of symmetric spaces. This result can be viewed as a converse to Theorem 2.2 of Bott and Samelson. Finally we classify arbitrary variationally complete representations, mainly as a repeated application of Proposition 3.4, item (b).

Recall that the irreducible representations which according to the results in Section 5 can belong to class $`𝒪^2`$ without being either the isotropy representation of a symmetric space or of cohomogeneity one are listed in the tables of Propositions 5.1, 5.2, 5.17, 5.18, 5.19, 5.20 and 5.21. For the sake of our forthcoming arguments, these representations are rearranged in a more systematic way in Tables C.1, C.2, C.3 and C.4 in Appendix C and in the table of Proposition 6.3.

###### 6.1 Proposition

The representations listed in Tables C.1, C.2 and C.3 are not variationally complete.

Proof. For each representation $`\rho `$ of the compact connected Lie group $`G`$ on the real vector space $`V`$ listed in these tables, we need to exhibit a compact symmetric space $`X=L/K`$ such that:

1. $`K`$ contains $`G`$ as a closed subgroup;
2. the isotropy representation $`\stackrel{~}{\rho }`$ of $`X`$ restricts to $`\rho `$ on $`G`$;
3. there exists $`pV`$ such that the orbits of $`\stackrel{~}{\rho }`$ and $`\rho `$ through $`p`$ are the equal;
4. there exists $`qV`$ such that the orbits of $`\stackrel{~}{\rho }`$ and $`\rho `$ through $`q`$ are different.

It will then follow from Proposition 3.7 that $`\rho `$ is not variationally complete.

Let $`G=\mathrm{𝐒𝐎}(2)\times \mathrm{𝐒𝐩𝐢𝐧}(9)`$. Then $`X=\mathrm{𝐒𝐎}(18)/\mathrm{𝐒𝐎}(2)\times \mathrm{𝐒𝐎}(16)`$, $`\stackrel{~}{\rho }`$ is the tensor product of the vector representations of $`\mathrm{𝐒𝐎}(2)`$, $`\mathrm{𝐒𝐎}(16)`$ on $`𝐑^2`$, $`𝐑^{16}`$, respectively, and we take $`p=v_1v_2𝐑^2𝐑^{16}`$. Note that the orbits of $`\stackrel{~}{\rho }`$ and $`\rho `$ through $`p`$ are equal because $`\mathrm{𝐒𝐎}(2)`$, $`\mathrm{𝐒𝐩𝐢𝐧}(9)`$ and $`\mathrm{𝐒𝐎}(16)`$ are all transitive on unit spheres, but the orbits through a common regular point $`q`$ are different because $`\stackrel{~}{\rho }`$ is of cohomogeneity two and $`\rho `$ is of cohomogeneity three.

Let $`G=𝐔(2)\times \mathrm{𝐒𝐩}(n)`$, $`n2`$. Then $`X=\mathrm{𝐒𝐔}(2+2n)/𝐒(𝐔(2)\times 𝐔(2n))`$, $`\stackrel{~}{\rho }`$ can be viewed as the realification of the complex tensor product of the vector representations of $`𝐔(2)`$ and $`\mathrm{𝐒𝐔}(2n)`$ on $`𝐂^2`$, $`𝐂^{2n}`$, respectively, and we take $`p=v_1v_2𝐂^2𝐂^{2n}`$. The orbits of $`\stackrel{~}{\rho }`$ and $`\rho `$ through $`p`$ are equal because $`𝐔(2)`$, $`\mathrm{𝐒𝐩}(n)`$ and $`\mathrm{𝐒𝐔}(2n)`$ are all transitive on unit spheres, but the orbits through a common regular point $`q`$ are different because $`\stackrel{~}{\rho }`$ is of cohomogeneity two and $`\rho `$ is of cohomogeneity three.

Let $`G=\mathrm{𝐒𝐔}(2)\times \mathrm{𝐒𝐩}(n)`$, $`n2`$. Then $`X=\mathrm{𝐒𝐩}(2+n)/\mathrm{𝐒𝐩}(2)\times \mathrm{𝐒𝐩}(n)`$, $`\stackrel{~}{\rho }`$ is the quaternionic tensor product of the vector representations of $`\mathrm{𝐒𝐩}(2)`$, $`\mathrm{𝐒𝐩}(n)`$ on $`𝐇^2`$, $`𝐇^n`$, respectively, and we take $`p=e_1f_1+e_2f_2𝐇^2_𝐇𝐇^n`$, where $`\{e_1,e_2\}𝐇^2`$ and $`\{f_1,f_2\}𝐇^n`$ are orthonormal. The orbits of $`\stackrel{~}{\rho }`$ and $`\rho `$ through $`p`$ are equal because both of them are $`8n6`$-dimensional, but the orbits through a common regular point $`q`$ are different because $`\stackrel{~}{\rho }`$ is of cohomogeneity two and $`\rho `$ is of cohomogeneity three.

We omit the details for the representations of Tables C.2 and C.3 because they are analogous to the above (For instance, the reason for the existence of the common orbit (item (c)) is that as in the first two cases above we are dealing with tensor products of representations transitive on unit spheres.) Moreover, we will prove something stronger later in Propositions 7.5 and 7.6, namely that these representations are not taut. $`\mathrm{}`$

###### 6.2 Proposition

The representations listed in Table C.4 are not taut.

Proof. Let $`\pi _\lambda `$ be a representation of the compact connected Lie group $`G`$ on $`V_\lambda `$ which is in the table. Notice that $`\pi _\lambda `$ is of real type. Let $`\rho `$ be a real form acting on $`VV_\lambda `$ and let $`ϵ`$ be the invariant real structure on $`V_\lambda `$. We shall go case by case and show that the slice representation of $`\rho `$ at a specific point $`pV`$ is not taut, and then use Proposition 3.8 to conclude that $`\rho `$ is not taut. In fact, we use $`p=v_\lambda +ϵ(v_\lambda )V`$ where $`v_\lambda `$ is a highest weight vector for $`\pi _\lambda `$. In the following, we omit the tedious and sometimes lengthy calculations involved and simply state the final results.

Let $`G=\mathrm{𝐒𝐩}(n)\times \mathrm{𝐒𝐔}(6)`$. Then the isotropy subalgebra at $`p`$ is

$$𝔲(1)+𝔰𝔲(3)+𝔰𝔲(3)+𝔰𝔭(n1),n2$$

The dimension of the orbit through $`p`$ is $`dimG(p)=4n+17`$, the dimension of the normal space at $`p`$ is $`36n17`$, and the complexified slice representation at $`p`$ is given by:

(trivial)
$`(x^4\text{}\text{}x^4\text{}\text{})\text{(trivial)}`$
$`(x\text{}\text{}x^1\text{}\text{})\text{}.`$
Therefore, the slice representation contains as a summand the realification of

$$x\text{}\text{}\text{}.$$

But this is a representation of complex type with Dadok invariant $`3`$. By Proposition 4.9, it cannot be of class $`𝒪^2`$. In particular, it is not taut.

Let $`G=\mathrm{𝐒𝐩}(n)\times \mathrm{𝐒𝐩𝐢𝐧}(12)`$. Then the isotropy subalgebra at $`p`$ is

$$𝔲(1)+𝔰𝔲(6)+𝔰𝔭(n1),n2$$

The dimension of the orbit through $`p`$ is $`dimG(p)=4n+29`$, the dimension of the normal space at $`p`$ is $`60n29`$, and the complexified slice representation at $`p`$ is given by:

(trivial)
$`(x^4\text{}x^4\text{})\text{(trivial)}`$
$`(x\text{}x^1\text{})\text{}.`$
Therefore, the slice representation contains as a summand the realification of

$$x\text{}\text{}.$$

But this is a representation of complex type with Dadok invariant $`3`$. By Proposition 4.9, it cannot be of class $`𝒪^2`$. In particular, it is not taut.

Let $`G=\mathrm{𝐒𝐩}(n)\times \mathrm{𝐒𝐩}(3)`$. Then the isotropy subalgebra at $`p`$ is

$$𝔲(1)+𝔰𝔲(3)+𝔰𝔭(n1),n2$$

The dimension of the orbit through $`p`$ is $`dimG(p)=4n+11`$, the dimension of the normal space at $`p`$ is $`24n11`$, and the complexified slice representation at $`p`$ is given by:

(trivial)
$`(x^4\text{}x^4\text{})\text{(trivial)}`$
$`(x\text{}x^1\text{})\text{}.`$
Therefore, the slice representation contains as a summand the realification of

$$x\text{}\text{}.$$

But this is a representation of complex type with Dadok invariant $`3`$. By Proposition 4.9, it cannot be of class $`𝒪^2`$. In particular, it is not taut.

Let $`G=\mathrm{𝐒𝐩}(n)\times 𝐄_7`$. Then the isotropy subalgebra at $`p`$ is

$$𝔲(1)+𝔢_6+𝔰𝔭(n1),n2$$

The dimension of the orbit through $`p`$ is $`dimG(p)=4n+53`$, the dimension of the normal space at $`p`$ is $`108n53`$, and the complexified slice representation at $`p`$ is given by:

(trivial)
$`(x^4\text{}x^4\text{})\text{(trivial)}`$
$`(x\text{}x^1\text{})\text{}.`$
Therefore, the slice representation contains as a summand the realification of

$$x\text{}\text{}.$$

But this is a representation of complex type with Dadok invariant $`3`$. By Proposition 4.9, it cannot be of class $`𝒪^2`$. In particular, it is not taut.

Let $`G=\mathrm{𝐒𝐩}(1)\times \mathrm{𝐒𝐩𝐢𝐧}(13)`$. Then the isotropy subalgebra at $`p`$ is

$$𝔲(1)+𝔰𝔲(6).$$

The dimension of the orbit through $`p`$ is $`dimG(p)=45`$, the dimension of the normal space at $`p`$ is $`83`$, and the complexified slice representation at $`p`$ is given by:

(trivial)
$`(x^5\text{}x^5\text{})`$
$`(x^4\text{}x^4\text{})`$
$`(x^3x^3)\text{}.`$
Therefore, the slice representation contains as a summand the realification of

$$x^3\text{}.$$

But this is a representation of complex type with Dadok invariant $`3`$. By Proposition 4.9, it cannot be of class $`𝒪^2`$. In particular, it is not taut.

Let $`G=\mathrm{𝐒𝐩}(1)\times \mathrm{𝐒𝐩}(2)`$. Then the isotropy subalgebra at $`p`$ is

$$𝔲(1)+𝔲(1).$$

The dimension of the orbit through $`p`$ is $`dimG(p)=11`$, the dimension of the normal space at $`p`$ is $`21`$, and the slice representation at $`p`$ is given by the realification of:

(trivial)
$`(x^5x^2x^1)x^2`$
$`(x^5x^42x^12x^2)x^4`$
$`x^3x^6.`$
Therefore, the slice representation contains as a summand the realification of $`x^2x^2x^4x^4`$ which is orbit equivalent to the circle action

$$x^2x^4$$

on $`𝐑^2𝐑^2`$. But the orbit of the circle action through $`((1,0),(0,1))`$ is a torus-knot which is not taut.

Let $`G=\mathrm{𝐒𝐩𝐢𝐧}(15)`$. Then the isotropy subalgebra at $`p`$ is $`𝔰𝔲(7)`$. The dimension of the orbit through $`p`$ is $`dimG(p)=57`$, the dimension of the normal space at $`p`$ is $`71`$, and the complexified slice representation at $`p`$ is given by:

$$\text{(trivial)}\text{}\text{}$$

Therefore, the slice representation contains as a summand the realification of

But this is a representation of complex type with Dadok invariant $`3`$. By Proposition 4.9, it cannot be of class $`𝒪^2`$. In particular, it is not taut.

Let $`G=\mathrm{𝐒𝐩𝐢𝐧}(17)`$. Then the isotropy subalgebra at $`p`$ is $`𝔰𝔲(8)`$. The dimension of the orbit through $`p`$ is $`dimG(p)=73`$, the dimension of the normal space at $`p`$ is $`183`$, and the complexified slice representation at $`p`$ is given by:

$`\text{(trivial)}\text{}`$
$`\text{}\text{}`$
Therefore, the slice representation contains as a summand the realification of

But this is a representation of complex type with Dadok invariant $`3`$. By Proposition 4.9, it cannot be of class $`𝒪^2`$. In particular, it is not taut.

Let $`G=\mathrm{𝐒𝐩𝐢𝐧}(7)`$. Then the isotropy subalgebra at $`p`$ is

$$𝔲(1)+𝔰𝔲(2)$$

The dimension of the orbit through $`p`$ is $`dimG(p)=17`$, the dimension of the normal space at $`p`$ is $`31`$, and the slice representation at $`p`$ is given by:

(trivial)
$`(2(x^4x^4)(x^2x^2))\text{(trivial)}`$
$`(2(x^1x^1)(x^7x^7))\text{}`$
$`((x^2x^2)(x^4x^4)))\text{}`$
Therefore, the slice representation contains as a summand the realification of

$$x^4x^2$$

acting on $`𝐑^2𝐑^2`$. But the orbit through $`((1,0),(0,1))`$ is a torus-knot which is not taut.

Let $`G=\mathrm{𝐒𝐩𝐢𝐧}(9)`$. Then the isotropy subalgebra at $`p`$ is

$$𝔲(1)+𝔰𝔲(3)$$

The dimension of the orbit through $`p`$ is $`dimG(p)=27`$, the dimension of the normal space at $`p`$ is $`101`$, and the complexified slice representation at $`p`$ is given by:

(trivial)
$`((x^3x^3)2(x^2x^2)(xx^1))\text{(trivial)}`$
$`((xx^1)(x^2x^2))\text{}`$
$`((3x2x^0x^3)\text{}(3x^12x^0x^3)\text{})`$
$`((x^0x^1)\text{}(x^0x^1)\text{})`$
Therefore, the slice representation contains as a summand the realification of

$$x\text{}$$

But this representation cannot be of class $`𝒪^2`$ (because is of real type and we use Lemma 5.3, item (a)). In particular, it is not taut.

Let $`G=\mathrm{𝐒𝐩}(n)\times \mathrm{𝐒𝐩𝐢𝐧}(11)`$. Then the isotropy subalgebra at $`p`$ is

$$𝔲(1)+𝔰𝔲(5)+𝔰𝔭(n1),n2$$

The dimension of the orbit through $`p`$ is $`dimG(p)=4n+29`$ and the dimension of the normal space at $`p`$ is $`60n29`$. For $`n2`$, the complexified slice representation at $`p`$ is given by:

(trivial)
$`(x^8\text{}x^8\text{})\text{(trivial)}`$
$`(x^6\text{}x^6\text{})\text{(trivial)}`$
$`(x^3\text{}x^3\text{})\text{}`$
$`(x\text{}x^1\text{})\text{}`$
Therefore, the slice representation contains as a summand the realification of

$$x\text{}\text{}.$$

But this is a representation of complex type with Dadok invariant $`3`$. By Proposition 4.9, it cannot be of class $`𝒪^2`$. In particular, it is not taut.

For $`n=1`$ we need a different argument. In this case the connected component $`K`$ of the isotropy group at $`p`$ is locally isomorphic to $`𝐔(5)`$, and the slice representation at $`p`$ contains as a summand the realification of

$$x^8\text{}x^6\text{}.$$

Denote by $`V_1`$, $`V_2`$ the respective representation spaces of the above summands. Then $`V_1=𝐂^5`$ and $`V_2=\mathrm{\Lambda }^2𝐂^5`$. Choose an orthonormal basis $`\{e_1,\mathrm{},e_5\}`$ for $`𝐂^5`$. Let $`q_2=ae_1e_2+be_3e_4V_2`$ be a regular point, where $`a`$, $`b`$ are distinct positive real numbers. We have that the isotropy $`K_{q_2}`$ is locally isomorphic to $`\mathrm{𝐒𝐔}(2)\times \mathrm{𝐒𝐔}(2)\times 𝐔(1)`$ sitting diagonally in $`K`$. Let $`q_1=e_1+e_3V_1`$. Now the orbit $`K_{q_2}(q_1)`$ is diffeomorphic to $`S^3\times S^3`$, whereas the orbit $`K(q_1,q_2)`$ is diffeomorphic to $`\mathrm{𝐒𝐔}(5)`$. It is known that the third Betti number of a compact connected simple Lie group is one. It follows from Proposition 3.11 that the slice representation at $`p`$ is not taut. $`\mathrm{}`$

###### 6.3 Proposition

The following is a list of pairs of irreducible representations $`\rho `$, $`\rho ^{}`$ of compact connected Lie groups $`G`$, $`G^{}`$ such that $`\rho ^{}`$ is the isotropy representation of a symmetric space, $`G`$ is a subgroup of $`G^{}`$ and $`\rho =\rho ^{}|G`$ is orbit equivalent to $`\rho ^{}`$.

$$\begin{array}{cc}& \\ G\text{and}G^{}& \rho \text{and}\rho ^{}\\ & \\ & \\ \begin{array}{c}\mathrm{𝐒𝐔}(n)\times \mathrm{𝐒𝐔}(m),nm\\ \text{and}\end{array}& \begin{array}{c}(\text{}\text{})\\ (\text{}\text{})\text{and}\end{array}\\ S^1\times \mathrm{𝐒𝐔}(n)\times \mathrm{𝐒𝐔}(m)& \begin{array}{c}(\text{}x\text{})\\ (\text{}x^1\text{})\end{array}\\ & \\ \mathrm{𝐒𝐔}(n),n\text{ odd, and}& \text{}\text{}\text{and}\\ 𝐔(n)& (x\text{})(x^1\text{})\\ & \\ \mathrm{𝐒𝐩𝐢𝐧}(10)\text{and}& \text{}\text{and}\\ \mathrm{𝐒𝐎}(2)\times \mathrm{𝐒𝐩𝐢𝐧}(10)& (xx^1)\text{}\\ & \\ \mathrm{𝐒𝐎}(2)\times \mathrm{𝐒𝐩𝐢𝐧}(7)\text{and}& (xx^1)\text{}\text{and}\\ \mathrm{𝐒𝐎}(2)\times \mathrm{𝐒𝐎}(8)& (xx^1)\text{}\\ & \\ \mathrm{𝐒𝐎}(2)\times 𝐆_2\text{and}& (xx^1)\text{}\text{and}\\ \mathrm{𝐒𝐎}(2)\times \mathrm{𝐒𝐎}(7)& (xx^1)\text{}\\ & \\ \mathrm{𝐒𝐎}(3)\times \mathrm{𝐒𝐩𝐢𝐧}(7)\text{and}& \text{}\text{}\text{and}\\ \mathrm{𝐒𝐎}(3)\times \mathrm{𝐒𝐎}(8)& \text{}\text{}\end{array}$$

Proof. The first three cases in the table were treated in Lemma 5.4, item (a). It remains to analyze the other three cases. It suffices to show the existence of a common principal orbit by Lemma 3.6.

Let $`G=\mathrm{𝐒𝐎}(2)\times \mathrm{𝐒𝐩𝐢𝐧}(7)`$. Then $`\rho `$ is the tensor product of the vector representation of $`\mathrm{𝐒𝐎}(2)`$ on $`𝐑^2`$ and the spin representation of $`\mathrm{𝐒𝐩𝐢𝐧}(7)`$ on $`𝐑^8`$. Let $`p=a_1e_1f_1+a_2e_2f_2𝐑^2𝐑^8`$, where $`e_1`$, $`e_2𝐑^2`$ are orthonormal, $`f_1`$, $`f_2𝐑^8`$ are orthonormal, and $`a_1`$, $`a_2`$ are distinct positive numbers. Then the orbit $`G^{}p`$ is principal and of dimension $`14`$. Since the isotropy subalgebra of $`G`$ at $`p`$ is easily seen to be $`𝔰𝔲(3)`$, it follows that $`Gp`$ is also an orbit of dimension $`14`$. Hence, it coincides with $`G^{}p`$. It follows from Lemma 3.6 that $`\rho `$ and $`\rho ^{}`$ are orbit equivalent.

Similarly, one sees that in the last two cases in the table $`\rho `$ has respectively an orbit of dimension $`12`$, $`21`$, and therefore is orbit equivalent to $`\rho ^{}`$. $`\mathrm{}`$

###### 6.4 Theorem (Converse to a Theorem of Bott and Samelson)

A variationally complete representation of a compact connected Lie group is orbit equivalent to the isotropy representation of a symmetric space.

Proof. We first assume that the given representation is irreducible. A variationally complete, irreducible representation $`\rho `$ of a compact connected Lie group $`G`$ is of class $`𝒪^2`$. If $`\rho `$ is not the isotropy representation of one of the irreducible symmetric spaces and has cohomogeneity greater than one, we have obtained all possibilities for it in Propositions 5.1, 5.2, 5.17, 5.18, 5.19, 5.20 and 5.21, and these are listed in the table of Proposition 6.3 and in Tables C.1, C.2, C.3 and C.4 in Appendix C. Now Propositions 6.1 and 6.2 eliminate all possibilities for $`\rho `$, except those listed in the table of Proposition 6.3. But the same proposition asserts that $`\rho `$ is orbit equivalent to another representation $`\rho ^{}`$ which is the isotropy representation of a symmetric space.

Now assume that the representation is reducible. Each of its irreducible factors is variationally complete by part (a) of Proposition 3.4 and hence orbit equivalent to the isotropy representation of an irreducible symmetric space by what we have just proved. Part (d) of the same proposition now implies that the reducible representation is orbit equivalent to the isotropy representation of the symmetric space that is the product of the irreducible symmetric spaces corresponding to the irreducible summands of the representation. $`\mathrm{}`$

###### 6.5 Corollary

A variationally complete representation of a compact connected Lie group is polar.

The first paragraph in the proof of Theorem 6.4 combined with Theorem 8.2 to be proved in Section 8 also implies:

###### 6.6 Theorem (Compare )

Let $`\rho :G𝐎(V)`$ be a variationally complete irreducible representation of a compact connected Lie group $`G`$. Then there are two cases:

1. The image $`\rho (G)`$ is the maximal compact connected Lie group with its orbits. In this case $`\rho `$ is the isotropy representation of an irreducible symmetric space.
2. The image $`\rho (G)`$ is not the maximal compact connected Lie group with its orbits. In this case we have:

1. If $`\rho `$ has cohomogeneity greater than one, then there exists a larger group $`G^{}G`$ and an isotropy representation of an irreducible symmetric space $`\rho ^{}:G^{}𝐎(V)`$ which is an extension of $`\rho `$. Moreover, the possibilities for the pair $`(\rho ,\rho ^{})`$ are listed in the table of Proposition 6.3.
    2. If $`\rho `$ has cohomogeneity one, then it is listed in the tables of Theorem 8.2.

In order to complete the classification of variationally complete representations, we finally discuss the reducible case and state a result in Theorem 6.8 which, in view of Theorem 6.4, is equivalent to a result of Bergmann in . First note that there are some trivial ways to produce variationally complete reducible representations from known variationally complete representations.

Let $`\rho :G𝐎(V)`$ be a representation of a compact connected Lie group $`G`$.

1. Assume $`G=G^{}\times K`$ where $`G^{}`$, $`K`$ are compact connected Lie groups. We say that $`K`$ is *nonessential* for $`\rho `$ if the restriction $`\rho |G^{}`$ is orbit equivalent to the original representation $`\rho `$. Otherwise, we say that $`K`$ is *essential* for $`\rho `$.
2. Assume $`\rho `$ reducible, namely $`\rho =\rho _1\rho _2`$ where $`\rho _i:G𝐎(V_i)`$ and $`dimV_i>0`$ ($`i=1`$$`2`$). We say that $`\rho `$ is *splitting* if we can write $`G=G_1\times G_2`$ for compact connected Lie groups $`G_i`$ ($`i=1`$, $`2`$) such that $`\rho `$ is equivalent to the outer direct sum representation $`\rho _1|G_1\widehat{}\rho _2|G_2:G_1\times G_2𝐎(V_1V_2)`$. We say that $`\rho `$ is *almost splitting* if we can write $`G=G_1\times G_2\times K`$ for compact connected Lie groups $`K`$, $`G_i`$ ($`i=1`$, $`2`$) such that $`K`$ is nonessential at least for one of $`\rho _1|G_1\times K`$, $`\rho _2|G_2\times K`$ and $`\rho |G_1\times G_2`$ is equivalent to the outer direct sum representation $`\rho _1|G_1\widehat{}\rho _2|G_2`$.

It is immediate from these definitions that a representation with a trivial summand is automatically splitting, and that splitting implies almost splitting. We also have:

###### 6.7 Proposition

An almost splitting representation $`\rho =\rho _1\rho _2`$ is variationally complete if and only if each one of its summands $`\rho _i`$ is variationally complete666Note that the isotropy representations of reducible symmetric spaces are splitting..

Proof. One direction follows from Proposition 3.4, item (a); we prove that the summands being variationally complete implies the same for the sum. We may assume that $`K`$ is nonessential for $`\rho _1|G_1\times K`$. Then $`\rho _1`$ is orbit equivalent to $`\rho _1|G_1`$ and $`\rho _2`$ is orbit equivalent to $`\rho _2|G_2\times K`$, so $`\rho _1|G_1`$ and $`\rho _2|G_2\times K`$ are variationally complete. Now $`\rho `$ is orbit equivalent to $`\rho _1|G_1\widehat{}\rho _2|G_2\times K`$ and hence variationally complete, because the outer direct sum of two variationally complete representations is also variationally complete, as is easy to see. $`\mathrm{}`$

We are left with the classification of variationally complete reducible representations which are not almost splitting. This is given in the following theorem.

###### 6.8 Theorem (Compare )

Let $`\rho `$ be a variationally complete reducible representation of a compact connected Lie group $`G`$ which is not almost splitting. Then $`\rho `$ is one of the following orthogonal representations:

$$\begin{array}{cc}& \\ G& \rho \\ & \\ \mathrm{𝐒𝐔}(4)& (\text{}\text{})\text{}\\ 𝐔(4)& (x\text{}x^1\text{})\text{}\\ \mathrm{𝐒𝐩𝐢𝐧}(7)& \text{}\text{}\\ \mathrm{𝐒𝐎}(2)\times \mathrm{𝐒𝐩𝐢𝐧}(7)& (xx^1)\text{}\text{}\\ \mathrm{𝐒𝐩𝐢𝐧}(8)& \text{}\text{}\\ \mathrm{𝐒𝐩𝐢𝐧}(8)& \text{}\text{}\\ \mathrm{𝐒𝐎}(2)\times \mathrm{𝐒𝐩𝐢𝐧}(8)& (xx^1)\text{}\text{}\\ \mathrm{𝐒𝐎}(2)\times \mathrm{𝐒𝐩𝐢𝐧}(8)& (xx^1)\text{}\text{}\\ \mathrm{𝐒𝐎}(3)\times \mathrm{𝐒𝐩𝐢𝐧}(8)& \text{}\text{}\text{}\\ \mathrm{𝐒𝐎}(3)\times \mathrm{𝐒𝐩𝐢𝐧}(8)& \text{}\text{}\text{}\end{array}$$

Sketch of a proof. Bergmann proves her theorem in  using Theorem 6.6 as it is stated in . We sketch here an argument in a similar vein, but divided into cases in a different way. Since summands of almost faithful reducible representations need not be almost faithful, this is the only place in this paper where we have to exercise some care in distinguishing between a group $`G`$ and its image $`\rho (G)`$ under a representation $`\rho :G𝐎(V)`$.

$`G`$ is a simple group. Let $`\rho =\rho _1\rho _2`$ be a decomposition where $`\rho _1`$ is irreducible. Then $`\rho _1`$, $`\rho _2`$ are variationally complete. Denote by $`V_1`$, $`V_2`$ the representation spaces and by $`H_1`$, $`H_2`$ the connected components of the principal isotropy subgroups. We have that $`\rho _1`$ is an almost faithful variationally complete irreducible representation of a simple group, which is orbit equivalent to the restriction $`\rho _1|H_2`$, where $`H_2`$ is a proper subgroup of $`G`$; see Proposition 3.4, item (b). Since $`\rho _1(H_2)𝐎(V_1)`$ is not the maximal compact connected group with its orbits, $`\rho _1:H_2𝐎(V_1)`$ is not the isotropy representation of a symmetric space. It follows from Theorem 6.6 that $`\rho _1`$ has cohomogeneity one, because the larger of the two groups for each pair in the table of Proposition 6.3 is never a simple group. Since $`\rho _1`$ is an arbitrary irreducible summand of $`\rho `$, we conclude that every irreducible summand of $`\rho `$ must have cohomogeneity one. Moreover, we know from Proposition 3.4, item (c), that there cannot be two equivalent summands. The only simple groups admitting at least two inequivalent cohomogeneity one representations are $`\mathrm{𝐒𝐩𝐢𝐧}(n)`$ for $`n\{3,5,6,7,8,9\}`$. Using the fact that $`\rho _2`$ must be orbit equivalent to $`\rho _2|H_1`$ we exclude the groups $`\mathrm{𝐒𝐩𝐢𝐧}(3)`$, $`\mathrm{𝐒𝐩𝐢𝐧}(5)`$, $`\mathrm{𝐒𝐩𝐢𝐧}(9)`$ and get the four examples corresponding to simple groups in the table.

$`G`$ is not semisimple. Let $`\rho _1`$ be an irreducible summand of $`\rho `$ such that $`\rho _1(G)`$ is locally isomorphic to $`S^1\times G^{}`$, where $`G^{}`$ is a nontrivial semisimple factor of $`G`$. Let $`\rho _2`$ be another irreducible summand of $`\rho `$ with the connected component of the principal isotropy group being $`H_2`$. If $`\rho _1(H_2)=\rho _1(G)`$ for all such $`\rho _2`$, then $`\rho `$ is splitting. So we assume there exists such a $`\rho _2`$ with $`\rho _1(H_2)\rho _1(G)`$. Now we can view $`\rho _1`$ as an almost faithful variationally complete irreducible representation of $`S^1\times G^{}`$, which is orbit equivalent to the restriction $`\rho _1|H_2^{}`$, where $`H_2^{}`$ is a compact connected proper subgroup of $`S^1\times G^{}`$ and $`\rho _1(H_2^{})=\rho _1(H_2)`$. It follows from Theorem 6.6, item (b), that the possible local isomorphism classes of $`H_2^{}`$, $`S^1\times G^{}`$ and the representations $`\rho _1|H_2^{}`$, $`\rho _1|S^1\times G^{}`$ are given as follows:

$$\begin{array}{ccc}& & \\ & H_2^{}\text{and}S^1\times G^{}& \rho _1|H_2^{}\text{and}\rho _1|S^1\times G^{}\\ & & \\ 1& \mathrm{𝐒𝐎}(2)\times 𝐆_2\text{and}& (xx^1)\text{}\text{and}\\ & \mathrm{𝐒𝐎}(2)\times \mathrm{𝐒𝐎}(7)& (xx^1)\text{}\\ & & \\ 2& \mathrm{𝐒𝐎}(2)\times \mathrm{𝐒𝐩𝐢𝐧}(7)\text{and}& (xx^1)\text{}\text{and}\\ & \mathrm{𝐒𝐎}(2)\times \mathrm{𝐒𝐎}(8)& (xx^1)\text{}\\ & & \\ 3& \mathrm{𝐒𝐩𝐢𝐧}(10)\text{and}& \text{}\text{and}\\ & \mathrm{𝐒𝐎}(2)\times \mathrm{𝐒𝐩𝐢𝐧}(10)& (xx^1)\text{}\\ & & \\ 4& \mathrm{𝐒𝐔}(n),n\text{ odd, and}& \text{}\text{}\text{and}\\ & S^1\times \mathrm{𝐒𝐔}(n)& (x\text{})(x^1\text{})\\ & & \\ 5& \begin{array}{c}\mathrm{𝐒𝐔}(n)\times \mathrm{𝐒𝐔}(m),nm\\ \text{and}\end{array}& \begin{array}{c}(\text{}\text{})\\ (\text{}\text{})\text{and}\end{array}\\ & S^1\times \mathrm{𝐒𝐔}(n)\times \mathrm{𝐒𝐔}(m)& \begin{array}{c}(\text{}x\text{})\\ (\text{}x^1\text{})\end{array}\\ & & \\ 6& \mathrm{𝐒𝐩}(n)\text{and}& \text{}\text{and}\\ & S^1\times \mathrm{𝐒𝐔}(2n)& x\text{}x^1\text{}\\ & & \\ 7& \mathrm{𝐒𝐔}(n)\text{and}& \text{}\text{}\text{and}\\ & S^1\times \mathrm{𝐒𝐔}(n)& x\text{}x^1\text{}\\ & & \\ 8& S^1\times \mathrm{𝐒𝐩}(n)\text{and}& (xx^1)\text{}\text{and}\\ & S^1\times \mathrm{𝐒𝐔}(2n)& x\text{}x^1\text{}\\ & & \\ 9& \mathrm{𝐒𝐩}(n)\text{and}& \text{}\text{and}\\ & S^1\times \mathrm{𝐒𝐩}(n)& (xx^1)\text{}\end{array}$$

In the following we examine different possibilities for $`\rho _2`$. Recall that a variationally complete representation does not admit two equivalent nontrivial irreducible summands.

1. $`\rho _2`$ is as in case $`1`$. Here $`\rho _2`$ is trivial on $`\mathrm{𝐒𝐎}(2)`$, so we can view $`\rho _2`$ as an almost faithful representation $`\mathrm{𝐒𝐩𝐢𝐧}(7)\times K𝐎(V_2)`$ ($`K`$ is a compact connected Lie group) which is variationally complete, irreducible and such that the projection of the connected component of its principal isotropy subgroup in $`\mathrm{𝐒𝐩𝐢𝐧}(7)`$ is $`𝐆_2`$. The only possibility for $`\rho _2`$ is , $`K=\{1\}`$. Any other irreducible summand $`\rho _3`$ of $`\rho `$ either would satisfy $`\rho _1(H_3)=\rho _1(G)`$ (where $`H_3`$ is the connected component of the principal isotropy subgroup of $`\rho _3`$) or would have to be in the same case as $`\rho _2`$. The first possibility would make $`\rho `$ splitting and the second possibility is excluded because $`\rho `$ cannot have two equivalent irreducible summands. It follows that $`\rho `$ has exactly two irreducible summands and it is $`(xx^1)\text{}\text{}`$.
2. $`\rho _2`$ is as in case $`2`$. Here $`\rho _2`$ is trivial on $`\mathrm{𝐒𝐎}(2)`$, so we can view $`\rho _2`$ as an almost faithful representation $`\mathrm{𝐒𝐩𝐢𝐧}(8)\times K𝐎(V_2)`$ ($`K`$ is a compact connected Lie group) which is variationally complete, irreducible and such that the projection of the connected component of its principal isotropy subgroup in $`\mathrm{𝐒𝐩𝐢𝐧}(8)`$ is $`\mathrm{𝐒𝐩𝐢𝐧}(7)`$. Here $`\mathrm{𝐒𝐩𝐢𝐧}(7)`$ is in the conjugacy class such that the vector representation of $`\mathrm{𝐒𝐩𝐢𝐧}(8)`$ restricts to the spin representation of $`\mathrm{𝐒𝐩𝐢𝐧}(7)`$. The only possibilities for $`\rho _2`$ are the half-spin representations of $`\mathrm{𝐒𝐩𝐢𝐧}(8)`$, $`K=\{1\}`$. Similar considerations to those in (i) show that $`\rho `$ must have exactly two irreducible summands and that it is either one of

$$(xx^1)\text{}\text{},(xx^1)\text{}\text{}.$$
3. $`\rho _2`$ is as in one of the cases $`3`$, $`4`$, $`5`$ or $`9`$. Here $`S^1`$ is not essential for $`\rho _1`$ and $`G^{}\mathrm{ker}\rho _2`$. Also, for any other irreducible summand $`\rho _3`$ either $`\rho _1(H_3)=\rho _1(G)`$ (where $`H_3`$ is the connected component of the principal isotropy subgroup of $`\rho _3`$) or $`G^{}\mathrm{ker}\rho _3`$. It follows that $`\rho `$ is almost splitting.
4. $`\rho _2`$ is as in case $`6`$. This case cannot occur, for otherwise $`\rho _2:𝐔(2n)\times K𝐎(V_2)`$ ($`K`$ a compact connected Lie group) is an almost faithful variationally complete irreducible representation such that the projection of the connected component of its principal isotropy subgroup in $`𝐔(2n)`$ is $`\mathrm{𝐒𝐩}(n)`$. But such a $`\rho _2`$ does not exist.
5. $`\rho _2`$ is as in case $`7`$. Here $`S^1`$ is not essential for $`\rho _1`$ and $`G^{}\mathrm{ker}\rho _2`$. Also, for any other irreducible summand $`\rho _3`$ we have either one of the following: $`\rho _1(H_3)=\rho _1(G)`$ (where $`H_3`$ is the connected component of the principal isotropy subgroup of $`\rho _3`$); or $`G^{}\mathrm{ker}\rho _3`$; or $`\rho _3`$ is as in case $`8`$. In the first two possibilities $`\rho `$ is almost splitting and in the third one we go to the next case.
6. $`\rho _2`$ is as in case $`8`$. Here $`\rho _2`$ is trivial on $`S^1`$, so we can view $`\rho _2`$ as an almost faithful representation $`\mathrm{𝐒𝐔}(2n)\times K𝐎(V_2)`$ ($`K`$ is a compact connected Lie group) which is variationally complete, irreducible and such that the projection of the connected component of its principal isotropy subgroup in $`\mathrm{𝐒𝐔}(2n)`$ is $`\mathrm{𝐒𝐩}(n)`$. The only possibility for $`\rho _2`$ is , $`K=\{1\}`$, $`n=2`$. If $`\rho _3`$ is a third irreducible summand of $`\rho `$ then either $`\rho _1(H_3)=\rho _1(G)`$ (where $`H_3`$ is the connected component of the principal isotropy subgroup of $`\rho _3`$) or $`\rho _1(H_3)`$ must be as in case $`7`$, therefore $`\rho _3`$ is trivial on $`\mathrm{𝐒𝐔}(4)`$. So if there is any irreducible summand besides $`\rho _1`$ and $`\rho _2`$ then $`\rho `$ is almost splitting. Therefore there are no other irreducible summands and $`\rho `$ is

$$(x\text{}x^1\text{})\text{}.$$

$`G`$ is semisimple but not simple. Let $`\rho _1`$ be an irreducible summand of $`\rho `$ such that $`\rho _1(G)`$ is locally isomorphic to $`G^{}`$, a semisimple but not simple factor of $`G`$. Let $`\rho _2`$ be another irreducible summand of $`\rho `$ with connected component of the principal isotropy group $`H_2`$. If $`\rho _1(H_2)=\rho _1(G)`$ for all such $`\rho _2`$, then $`\rho `$ is splitting. So we assume there exists such a $`\rho _2`$ with $`\rho _1(H_2)\rho _1(G)`$. Now we can view $`\rho _1`$ as an almost faithful variationally complete irreducible representation of $`G^{}`$, which is orbit equivalent to the restriction $`\rho _1|H_2^{}`$, where $`H_2^{}`$ is a compact connected proper subgroup of $`G^{}`$ and $`\rho _1(H_2^{})=\rho _1(H_2)`$. It follows from Theorem 6.6, item (b), that the possible local isomorphism classes of $`H_2^{}`$, $`G^{}`$ and the representations $`\rho _1|H_2^{}`$, $`\rho _1|G^{}`$ are given as follows:

$$\begin{array}{ccc}& & \\ & H_2^{}\text{and}G^{}& \rho _1|H_2^{}\text{and}\rho _1|G^{}\\ & & \\ 10& \mathrm{𝐒𝐩}(n)\text{and}& \text{}\text{and}\\ & \mathrm{𝐒𝐩}(1)\times \mathrm{𝐒𝐩}(n)& \text{}\text{}\\ & & \\ 11& \mathrm{𝐒𝐎}(3)\times \mathrm{𝐒𝐩𝐢𝐧}(7)\text{and}& \text{}\text{}\text{and}\\ & \mathrm{𝐒𝐎}(3)\times \mathrm{𝐒𝐎}(8)& \text{}\text{}\end{array}$$

If $`\rho _2`$ is an in case $`10`$, then $`\rho `$ is almost splitting by the same argument as in item (iii). If $`\rho _2`$ is as in case $`11`$, then by an argument similar to that in item (ii) we show that $`\rho `$ must be either one of

$$\text{}\text{}\text{},\text{}\text{}\text{}.$$

$`\mathrm{}`$

## 7 Taut representations

The main result of this section is the following classification theorem.

###### 7.1 Theorem

A taut irreducible representation $`\rho `$ of a compact connected Lie group $`G`$ is either orbit equivalent to the isotropy representation of a symmetric space or it is one of the following orthogonal representations ($`n2`$):

$$\begin{array}{cc}& \\ G& \rho \\ & \\ \mathrm{𝐒𝐎}(2)\times \mathrm{𝐒𝐩𝐢𝐧}(9)& (xx^1)\text{}\\ 𝐔(2)\times \mathrm{𝐒𝐩}(n)& (xx^1)\text{}\text{}\\ \mathrm{𝐒𝐔}(2)\times \mathrm{𝐒𝐩}(n)& \text{}\text{}\end{array}$$

###### 7.2 Corollary

A taut irreducible representation of a compact connected *simple* Lie group is orbit equivalent to the isotropy representation of a symmetric space.

###### 7.3 Remark

According to , see also , the representations given in the table of Theorem 7.1 are exactly the irreducible representations of compact connected Lie groups which have cohomogeneity three and are not orbit equivalent to the isotropy representation of a symmetric space.

It follows from the main results of Section 6 that Theorem 7.1 is equivalent to the following:

###### 7.4 Theorem

A taut irreducible representation $`\rho `$ of a compact connected Lie group $`G`$ is one of the following:

1. The isotropy representation of an irreducible symmetric space (listed in Tables 8.11.2 and 8.11.5 of ).
2. A cohomogeneity one representation not listed in (a) (cf. Theorem 8.2).
3. A representation orbit equivalent to the isotropy representation of an irreducible symmetric space and not listed in (a) or (b) (cf. Proposition 6.3).
4. One of the three exceptional cases given in Theorem 7.1.

Proof of Theorems 7.1 and 7.4. A taut irreducible representation $`\rho `$ of a compact connected Lie group $`G`$ is of class $`𝒪^2`$. If $`\rho `$ is not orbit equivalent to the isotropy representation of one of the irreducible symmetric spaces, we have obtained all possibilities for it, and these are listed in Tables C.1, C.2, C.3 and C.4 in Appendix C. Now we already know from Proposition 6.2 that the representations in Table C.4 are not taut. Therefore the assertion follows from Propositions 7.5, 7.6 and 7.12 below. $`\mathrm{}`$

We start by considering Table C.3.

###### 7.5 Proposition

The representations listed in Table C.3 are not taut.

Proof. Let $`G=𝐆_2\times \mathrm{𝐒𝐩𝐢𝐧}(7)`$. Then $`\rho `$ is the tensor product of the $`7`$-dimensional representation of $`𝐆_2`$ on $`𝐑^7`$ and the spin representation of $`\mathrm{𝐒𝐩𝐢𝐧}(7)`$ on $`𝐑^8`$. Let $`p=v_1v_2𝐑^7𝐑^8`$. The isotropy subalgebra at $`p`$ is $`𝔰𝔲(3)+𝔤_2`$. The dimension of the orbit through $`p`$ is $`dimG(p)=13`$, the dimension of the normal space at $`p`$ is $`43`$, and the slice representation at $`p`$ minus the trivial component $`𝐑p`$ is given by the tensor product of the vector representation of $`𝔰𝔲(3)`$ on $`𝐑^6`$ and the $`7`$-dimensional representation of $`𝔤_2`$ on $`𝐑^7`$. Since this representation is of complex type and Dadok invariant $`3`$, it follows from Proposition 4.9 that it cannot be of class $`𝒪^2`$. In particular, it is not taut. Now Proposition 3.8 implies that $`\rho `$ is not taut.

Let $`G=\mathrm{𝐒𝐩𝐢𝐧}(7)\times \mathrm{𝐒𝐩𝐢𝐧}(9)`$. Then $`\rho `$ is the tensor product of the spin representation of $`\mathrm{𝐒𝐩𝐢𝐧}(7)`$ on $`𝐑^8`$ and the spin representation of $`\mathrm{𝐒𝐩𝐢𝐧}(9)`$ on $`𝐑^{16}`$. Let $`p=v_1v_2𝐑^8𝐑^{16}`$. The isotropy subalgebra at $`p`$ is $`𝔤_2+𝔰𝔭𝔦𝔫(7)`$. The dimension of the orbit through $`p`$ is $`dimG(p)=22`$, the dimension of the normal space at $`p`$ is $`106`$, and the slice representation at $`p`$ minus the trivial component $`𝐑p`$ is given by the real tensor product of the $`7`$-dimensional representation of $`𝔤_2`$ and the vector representation of $`𝔰𝔭𝔦𝔫(7)`$ plus the real tensor product of the $`7`$-dimensional representation of $`𝔤_2`$ and the spin representation of $`𝔰𝔭𝔦𝔫(7)`$. Since we already know that the tensor product of the $`7`$-dimensional representation of $`𝔤_2`$ and the spin representation of $`𝔰𝔭𝔦𝔫(7)`$ is not taut (see the last paragraph), it follows from Proposition 3.8 that $`\rho `$ is not taut either.

Let $`G=𝐆_2\times \mathrm{𝐒𝐩𝐢𝐧}(9)`$. Then $`\rho `$ is the tensor product of the $`7`$-dimensional representation of $`𝐆_2`$ on $`𝐑^7`$ and the spin representation of $`\mathrm{𝐒𝐩𝐢𝐧}(9)`$ on $`𝐑^{16}`$. Let $`p=v_1v_2𝐑^7𝐑^{16}`$. The isotropy subalgebra at $`p`$ is $`𝔰𝔲(3)+𝔰𝔭𝔦𝔫(7)`$. The dimension of the orbit through $`p`$ is $`dimG(p)=21`$, the dimension of the normal space at $`p`$ is $`91`$, and the slice representation at $`p`$ minus the trivial component $`𝐑p`$ is given by the real tensor product of the vector representations of $`𝔰𝔲(3)`$ and $`𝔰𝔭𝔦𝔫(7)`$ plus the real tensor product of the vector representation of $`𝔰𝔲(3)`$ and the spin representation of $`𝔰𝔭𝔦𝔫(7)`$. Since the second summand is a representation of complex type and Dadok invariant $`3`$, it follows from Proposition 4.9 that it cannot be of class $`𝒪^2`$. In particular, it is not taut. Now Proposition 3.8 implies that $`\rho `$ is not taut.

Consider777We use the convention that $`GH`$ refers to a quotient of $`G\times H`$ by a finite central subgroup. $`G=\mathrm{𝐒𝐩}(1)\mathrm{𝐒𝐩}(n)\times 𝐆_2`$, $`n2`$. Let $`\tau _n`$ be a real form of $`\text{}\text{}`$. Then $`\tau _n`$ can be realized as the representation of $`\mathrm{𝐒𝐩}(1)\mathrm{𝐒𝐩}(n)`$ on $`𝐑^{4n}`$ given by $`\tau _n(q,A)x=Axq^1`$, where $`q\mathrm{𝐒𝐩}(1)`$, $`A\mathrm{𝐒𝐩}(n)`$, $`x𝐇^n`$ and we identify $`𝐇^n𝐑^{4n}`$. Now $`\rho `$ is the real tensor product of $`\tau _n`$ and the $`7`$-dimensional representation of $`𝐆_2`$. Let $`p=v_1v_2𝐑^{4n}𝐑^7`$. The isotropy subalgebra at $`p`$ is isomorphic to $`𝔰𝔭(1)+𝔰𝔭(n1)+𝔰𝔲(3)`$. The dimension of the orbit through $`p`$ is $`dimG(p)=4n+5`$, the dimension of the normal space at $`p`$ is $`24n5`$, and the slice representation at $`p`$ minus the trivial component $`𝐑p`$ is given by the real tensor product of $`\tau _{n1}`$ and the vector representation of $`𝔰𝔲(3)`$ plus the real tensor product of the adjoint representation of $`𝔰𝔭(1)`$ and the the vector representation of $`𝔰𝔲(3)`$. Since the second summand is a representation of complex type and Dadok invariant $`3`$, it follows from Proposition 4.9 that it cannot be of class $`𝒪^2`$. In particular, it is not taut. Now Proposition 3.8 implies that $`\rho `$ is not taut.

Consider $`G=\mathrm{𝐒𝐩}(1)\mathrm{𝐒𝐩}(n)\times \mathrm{𝐒𝐩𝐢𝐧}(7)`$, $`n2`$. Let $`\tau _n`$ be as above. Now $`\rho `$ is the tensor product of $`\tau _n`$ and the spin representation of $`\mathrm{𝐒𝐩𝐢𝐧}(7)`$. Let $`p=v_1v_2𝐑^{4n}𝐑^8`$. The isotropy subalgebra at $`p`$ is isomorphic to $`𝔰𝔭(1)+𝔰𝔭(n1)+𝔤_2`$. The dimension of the orbit through $`p`$ is $`dimG(p)=4n+6`$, the dimension of the normal space at $`p`$ is $`28n6`$, and the slice representation at $`p`$ minus the trivial component $`𝐑p`$ is given by the real tensor product of $`\tau _{n1}`$ and the $`7`$-dimensional representation of $`𝔤_2`$ plus the real tensor product of the adjoint representation of $`𝔰𝔭(1)`$ and the the $`7`$-dimensional representation of $`𝔤_2`$. Since the first summand is not a taut representation by the above (for $`n=2`$ we refer to the case of $`G=\mathrm{𝐒𝐎}(4)\times 𝐆_2`$ which will be dealt with in Proposition 7.6), it follows from Proposition 3.8 that $`\rho `$ is not taut either.

Let $`G=\mathrm{𝐒𝐩𝐢𝐧}(7)\times \mathrm{𝐒𝐩𝐢𝐧}(7)`$. Then $`\rho `$ is the tensor product of the spin representations of each of the factors on $`𝐑^8`$. Let $`p=v_1v_2𝐑^8𝐑^8`$. The isotropy subalgebra at $`p`$ is $`𝔤_2+𝔤_2`$. The dimension of the orbit through $`p`$ is $`dimG(p)=14`$, the dimension of the normal space at $`p`$ is $`50`$, and the slice representation at $`p`$ minus the trivial component $`𝐑p`$ is given by the real tensor product of the $`7`$-dimensional representations of each of the factors. Since this representation is not taut (see next case), it follows from Proposition 3.8 that $`\rho `$ is not taut either.

Consider $`G=𝐆_2\times 𝐆_2`$. Then $`\rho `$ is the tensor product of the $`7`$-dimensional representations of each of the factors on $`𝐑^7`$. Let $`\{e_1,\mathrm{},e_7\}`$ be an orthonormal basis of $`𝐑^7`$ and take $`p=e_1e_1+\mathrm{}+e_7e_7`$. Then the isotropy subalgebra at $`p`$ is

$$𝔤_p=\{(X,X):X𝔤_2\}$$

which is isomorphic to $`𝔤_2`$. Now the restriction of $`\rho `$ to $`𝔤_p`$ is a real form of

$$𝔤_2:\text{}(\text{})^{}=\text{}\text{}$$

acting on $`𝐑^{49}`$, and $`𝐑^{49}=T_pG(p)N_pG(p)`$ is a $`𝔤_p`$-invariant decomposition. We have the following decomposition into irreducible components:

$`𝔤_2:^2(\text{})`$ $`=`$ $`S^2(\text{})\mathrm{\Lambda }^2(\text{})`$
$`=`$ $`(\text{}\text{(trivial)})(\text{}\text{}).`$
Since $`T_pG(p)`$ is $`14`$-dimensional, it follows that the $`27`$-dimensional real representation must be a component of the slice representation at $`p`$. But we saw in Lemma 5.39 that it is not of class $`𝒪^2`$. Now Proposition 3.8 implies that $`\rho `$ is not taut. $`\mathrm{}`$

###### 7.6 Proposition

The representations listed in Table C.2 are not taut.

Proof. Let $`G=\mathrm{𝐒𝐎}(m)\times 𝐆_2`$, $`m4`$. Then $`\rho `$ is the tensor product of the vector representation of $`\mathrm{𝐒𝐎}(m)`$ on $`𝐑^m`$ and the $`7`$-dimensional representation of $`𝐆_2`$ on $`𝐑^7`$. Let $`p=v_1v_2𝐑^m𝐑^7`$. The isotropy subalgebra at $`p`$ is $`𝔰𝔬(m1)+𝔰𝔲(3)`$. The dimension of the orbit through $`p`$ is $`dimG(p)=m+5`$, the dimension of the normal space at $`p`$ is $`6m5`$, and the slice representation at $`p`$ minus the trivial component $`𝐑p`$ is given by the tensor product of the vector representations of $`𝔰𝔬(m1)`$ and $`𝔰𝔲(3)`$ on $`𝐑^{m1}`$ and $`𝐑^6`$, respectively. Since this is a representation of complex type and Dadok invariant $`3`$, it follows from Proposition 4.9 that it cannot be of class $`𝒪^2`$. In particular, it is not taut. Now Proposition 3.8 implies that $`\rho `$ is not taut.

The proof for $`G=\mathrm{𝐒𝐎}(3)\times 𝐆_2`$ will be done in Lemma 7.8 below, since this case is more involved.

Let $`G=\mathrm{𝐒𝐎}(m)\times \mathrm{𝐒𝐩𝐢𝐧}(7)`$, $`m4`$. Then $`\rho `$ is the tensor product of the vector representation of $`\mathrm{𝐒𝐎}(m)`$ on $`𝐑^m`$ and the spin representation of $`\mathrm{𝐒𝐩𝐢𝐧}(7)`$ on $`𝐑^8`$. Let $`p=v_1v_2𝐑^m𝐑^8`$. The isotropy subalgebra at $`p`$ is $`𝔰𝔬(m1)+𝔤_2`$. The dimension of the orbit through $`p`$ is $`dimG(p)=m+6`$, the dimension of the normal space at $`p`$ is $`7m6`$, and the slice representation at $`p`$ minus the trivial component $`𝐑p`$ is given by the real tensor product of the vector representation of $`𝔰𝔬(m1)`$ and the $`7`$-dimensional representation of $`𝔤_2`$. Since this representation is not taut by the above, it follows from Proposition 3.8 that $`\rho `$ is not taut either.

Let $`G=\mathrm{𝐒𝐎}(m)\times \mathrm{𝐒𝐩𝐢𝐧}(9)`$, $`m4`$. Then $`\rho `$ is the tensor product of the vector representation of $`\mathrm{𝐒𝐎}(m)`$ on $`𝐑^m`$ and the spin representation of $`\mathrm{𝐒𝐩𝐢𝐧}(9)`$ on $`𝐑^{16}`$. Let $`p=v_1v_2𝐑^m𝐑^{16}`$. The isotropy subalgebra at $`p`$ is $`𝔰𝔬(m1)+𝔰𝔭𝔦𝔫(7)`$. The dimension of the orbit through $`p`$ is $`dimG(p)=m+14`$, the dimension of the normal space at $`p`$ is $`15m14`$, and the slice representation at $`p`$ minus the trivial component $`𝐑p`$ is given by the real tensor product of the vector representations of $`𝔰𝔬(m1)`$ and $`𝔰𝔭𝔦𝔫(7)`$ plus the real tensor product of the vector representation of $`𝔰𝔬(m1)`$ and the spin representation of $`𝔰𝔭𝔦𝔫(7)`$. If $`m5`$ then the second summand is not a taut representation by the above and it follows from Proposition 3.8 that $`\rho `$ is not taut either. In the case $`m=4`$ we give a special argument. Here we have that the connected component of the isotropy subgroup at $`p`$ is $`K=\mathrm{𝐒𝐎}(3)\times \mathrm{𝐒𝐩𝐢𝐧}(7)`$. Let $`\nu _1`$, $`\nu _2`$ be respectively the irreducible summands of dimensions $`21`$, $`24`$ of the slice representation at $`p`$, and denote with $`V_1`$, $`V_2`$ the representation spaces. Note that both $`\nu _1`$ and $`\nu _2`$ are taut representations of $`K`$, since $`\nu _1`$ is the isotropy representation of a real Grassmann manifold and $`\nu _2`$ is orbit equivalent to the isotropy representation of a real Grassmann manifold. Let $`q_1V_1`$ be a regular point. Then the isotropy group $`K_{q_1}`$ is isomorphic to $`𝐙_2^2\times \mathrm{𝐒𝐩𝐢𝐧}(4)`$. We can choose $`q_2V_2`$ so that the isotropy $`(K_{q_1})_{q_2}`$ is contained in $`\{1\}\times \mathrm{𝐒𝐩𝐢𝐧}(4)`$. Hence the orbit $`K_{q_1}(q_2)`$ is disconnected. Since the orbit $`K(q_1,q_2)`$ is connected, Proposition 3.11 implies that the slice representation at $`p`$ is not taut. Now Proposition 3.8 yields that $`\rho `$ is not taut either.

We postpone the proof for the case $`\mathrm{𝐒𝐎}(3)\times \mathrm{𝐒𝐩𝐢𝐧}(9)`$ to Lemma 7.7 below.

Let $`G=\mathrm{𝐒𝐔}(n)\times \mathrm{𝐒𝐩}(m)`$, $`n3`$ and $`m2`$. Then $`\rho `$ is the realification of the tensor product of the vector representations of $`\mathrm{𝐒𝐔}(n)`$ on $`𝐂^n`$ and of $`\mathrm{𝐒𝐩}(m)`$ on $`𝐂^{2m}`$. Consider complex bases $`\{e_1,\mathrm{},e_n\}`$ for $`𝐂^n`$ and $`\{f_1,\mathrm{},f_m,f_{m+1},\mathrm{},f_{2m}\}`$ for $`𝐂^{2m}`$ where $`f_{m+j}=ϵf_j`$, $`ϵ`$ the quaternionic structure on $`𝐂^{2m}`$. Let $`p=e_1f_1`$. Then the isotropy $`K=G_p`$ is isomorphic to $`T_0\mathrm{𝐒𝐔}(n1)\times \mathrm{𝐒𝐩}(m1)`$, where $`T_0`$ is the circle group generated by

$`\phi _0(t)=(\text{diag}(e^{(n1)it},e^{it},\mathrm{},e^{it}),`$
$`\text{diag}(e^{(n1)it},1,\mathrm{},1;e^{(n1)it},1,\mathrm{},1))\mathrm{𝐒𝐔}(n)\times \mathrm{𝐒𝐩}(m),`$
and the slice representation $`\nu _p`$ minus the trivial component corresponding to the radial direction $`𝐑p`$ decomposes into irreducible components as $`\nu _1\nu _2`$, where $`\nu _1`$ restricted to $`\mathrm{𝐒𝐔}(n1)\times \mathrm{𝐒𝐩}(m1)`$ is the realification of the complex tensor product of the vector representations of the factors and $`\nu _2`$ is the realification of the vector representation on $`\mathrm{𝐒𝐔}(n1)`$ and it is trivial on $`\mathrm{𝐒𝐩}(m1)`$. Denote the representation spaces by $`V_1`$, $`V_2`$. Then $`V_1`$ is spanned by $`e_\alpha f_\beta `$ and $`i(e_\alpha f_\beta )`$, for $`2\alpha n`$, $`2\beta m`$ or $`m+2\beta 2m`$, and $`V_2`$ is spanned by $`e_\alpha f_{m+1}`$ and $`i(e_\alpha f_{m+1})`$, for $`2\alpha n`$. Let $`q_1=e_2f_2V_1`$. Then the isotropy $`K_1=K_{q_1}`$ is isomorphic to $`T_1T_2\mathrm{𝐒𝐔}(n2)\times \mathrm{𝐒𝐩}(m2)`$, where $`T_1`$, $`T_2`$ are circle groups respectively generated by

$`\phi _1(t)=(\text{diag}(1,e^{(n2)it},e^{it},\mathrm{},e^{it}),`$
$`\text{diag}(1,e^{(n2)it},1,\mathrm{},1;1,e^{(n2)it},1,\mathrm{},1))\mathrm{𝐒𝐔}(n)\times \mathrm{𝐒𝐩}(m),`$
and

$`\phi _2(t)=(\text{diag}(e^{(n1)it},e^{it},e^{it},\mathrm{},e^{it}),`$
$`\text{diag}(e^{(n1)it},e^{it},1,\mathrm{},1;e^{(n1)it},e^{it},1,\mathrm{},1))\mathrm{𝐒𝐔}(n)\times \mathrm{𝐒𝐩}(m).`$
(Here we have used the hypothesis $`n3`$ and $`m2`$). Finally let $`q_2=e_2f_{m+1}V_2`$. Then the isotropy $`K_2=(K_{q_1})_{q_2}`$ is isomorphic to $`T_3\mathrm{𝐒𝐔}(n2)\times \mathrm{𝐒𝐩}(m2)`$, where $`T_3`$ is the circle subgroup of $`T_1T_2`$ generated by $`\phi _3(t)=\phi _1(t)\phi _2(t)`$. It is easy to see that $`K_1/K_2`$ is diffeomorphic to $`S^1`$, whereas $`K/K_2`$ is simply-connected. Therefore Proposition 3.11 implies that $`\nu _p`$ is not taut. Now Proposition 3.8 yields that $`\rho `$ is not taut either.

The case where $`G=𝐔(n)\times \mathrm{𝐒𝐩}(m)`$, $`n3`$ and $`m2`$, is completely analogous to the previous case.

Let $`G=\mathrm{𝐒𝐎}(m)\times \mathrm{𝐒𝐩}(1)\mathrm{𝐒𝐩}(n)`$, $`m3`$ and $`n2`$. Then $`\rho `$ is the tensor product of the vector representation of $`\mathrm{𝐒𝐎}(m)`$ on $`𝐑^m`$ and $`\tau _n`$, where $`\tau _n`$ is a representation on $`𝐑^{4n}`$ as in Proposition 7.5. We choose an orthonormal basis $`\{e_1,\mathrm{},e_m\}`$ for $`𝐑^m`$ and an orthonormal basis

$$\{f_1,\mathrm{},f_n,f_1i,\mathrm{},f_ni,f_1j,\mathrm{},f_nj,f_1k,\mathrm{},f_nk\}$$

for $`𝐑^{4n}`$. Let $`p=e_1f_1`$. The connected component of the isotropy subgroup $`K=G_p^0`$ is isomorphic to $`\mathrm{𝐒𝐎}(m1)\times \mathrm{𝐒𝐩}(1)^{}\mathrm{𝐒𝐩}(n1)`$, where

$$\mathrm{𝐒𝐩}(1)^{}=\{(q,\left(\begin{array}{cccc}q& & & \\ & 1& & \\ & & \mathrm{}& \\ & & & 1\end{array}\right)):q\mathrm{𝐒𝐩}(1)\}\mathrm{𝐒𝐩}(1)\mathrm{𝐒𝐩}(n).$$

The slice representation at $`p`$ minus the trivial component $`𝐑p`$ decomposes as $`\nu _1\nu _2`$, where $`\nu _1`$ is the real tensor product of the vector representation of $`\mathrm{𝐒𝐎}(m1)`$ and $`\tau _{n1}`$, and $`\nu _2`$ is the real tensor product of the vector representation of $`\mathrm{𝐒𝐎}(m1)`$ and the adjoint representation of $`\mathrm{𝐒𝐩}(1)`$ ($`\nu _2`$ is trivial on $`\mathrm{𝐒𝐩}(n1)`$). Note that $`\nu _2`$ is the isotropy representation of a real Grassmann manifold. Let $`V_1`$, $`V_2`$ denote the representation spaces of $`\nu _1`$, $`\nu _2`$. Then $`V_1`$ is spanned by $`e_\alpha f_\beta `$, $`e_\alpha f_\beta i`$, $`e_\alpha f_\beta j`$, $`e_\alpha f_\beta k`$, for $`2\alpha m`$, $`2\beta n`$, and $`V_2`$ is spanned by $`e_\alpha f_1i`$, $`e_\alpha f_1j`$, $`e_\alpha f_1k`$, for $`2\alpha m`$. First consider the case $`m4`$. Let $`q_2V_2`$ be a regular point, say $`q_2=ae_2f_1i+be_3f_1j+ce_4f_1k`$, where $`a`$, $`b`$, $`c`$ are pairwise distinct positive real numbers. Then the isotropy group $`K_{q_2}`$ is isomorphic to $`\mathrm{𝐒𝐎}(m4)\times Q\mathrm{𝐒𝐩}(n1)`$, where $`Q`$ is the quaternion subgroup $`\{\pm 1,\pm i,\pm j,\pm k\}`$ of $`\mathrm{𝐒𝐩}(1)`$. Consider the restriction of $`\nu _1`$ to $`K_{q_2}`$. We can choose $`q_1V_1`$ so that the orbit $`K_{q_2}(q_1)`$ is disconnected, say $`q_1=ae_2f_2+be_3f_2i`$, where $`a`$, $`b`$ are distinct positive real numbers. In fact, if $`(A,q,B)\mathrm{𝐒𝐎}(m4)\times Q\mathrm{𝐒𝐩}(n1)`$ is in the isotropy of $`q_1`$, then we must have $`Bf_2q^1=f_2`$ and $`Bf_2iq^1=f_2i`$, which implies that $`Bf_2q^1i=Bf_2iq^1`$ and then $`q^1i=iq^1`$, so that $`q=\pm 1`$ or $`q=\pm i`$. In particular, no element of the form $`(A,j,B)\mathrm{𝐒𝐎}(m4)\times Q\mathrm{𝐒𝐩}(n1)`$ is in the isotropy of $`q_1`$, which implies that the orbit $`K_{q_2}(q_1)`$ is disconnected. Since the orbit $`K(q_1,q_2)`$ is connected, Proposition 3.11 implies that the slice representation at $`p`$ is not taut. Now Proposition 3.8 yields that $`\rho `$ is not taut either. Finally we analyse the case $`m=3`$. Then $`K=\mathrm{𝐒𝐎}(2)\times \mathrm{𝐒𝐩}(1)^{}\mathrm{𝐒𝐩}(n1)`$. Consider a regular point $`q_2V_2`$, say $`q_2=ae_2f_1i+be_3f_1j`$, where $`a`$, $`b`$ are distinct positive real numbers. Then we have that the isotropy $`K_{q_2}`$ is isomorphic to $`𝐙_4\times \mathrm{𝐒𝐩}(n1)`$. Here $`𝐙_4`$ is generated by $`(1,k)`$, where $`1`$ is minus the identity matrix in $`\mathrm{𝐒𝐎}(2)`$ and $`k\mathrm{𝐒𝐩}(1)^{}`$. Consider the restriction of $`\nu _1`$ to $`K_{q_2}`$. We can choose $`q_1V_1`$ as above such that its isotropy does not contain elements of the form $`(1,k,B)`$, for $`B\mathrm{𝐒𝐩}(n1)`$. It follows that the orbit $`K_{q_2}(q_1)`$ is disconnected and the proof follows as above. $`\mathrm{}`$

###### 7.7 Lemma

The tensor product $`\rho `$ of the vector representation of $`\mathrm{𝐒𝐎}(3)`$ on $`𝐑^3`$ and the spin representation of $`\mathrm{𝐒𝐩𝐢𝐧}(9)`$ on $`𝐑^{16}`$ is not taut.

Proof. Let $`p=v_1v_2𝐑^3𝐑^{16}`$. Then the connected component of the isotropy group $`K=G_p^0`$ is isomorphic to $`\mathrm{𝐒𝐎}(2)\times \mathrm{𝐒𝐩𝐢𝐧}(7)`$. The normal space at $`p`$ to the orbit $`G(p)`$ decomposes as $`𝐑pV_1V_2`$, where $`V_1=𝐑^2𝐑^7`$, $`V_2=𝐑^2𝐑^8`$ are the representation spaces for the irreducible components $`\nu _1`$, $`\nu _2`$ of the slice representation at $`p`$. Here $`\nu _1`$ is the tensor product of the vector representations and $`\nu _2`$ is the tensor product of the vector representation of $`\mathrm{𝐒𝐎}(2)`$ and the spin representation of $`\mathrm{𝐒𝐩𝐢𝐧}(7)`$. Now let $`q=v_3v_4V_2`$. Then the connected component of the isotropy group $`K_q^0`$ is isomorphic to $`𝐆_2`$ and the slice representation at $`q`$ of the representation of $`K`$ on $`V_1V_2`$ is a representation of $`𝐆_2`$ on $`𝐑^{22}`$ which decomposes as a trivial component $`𝐑q`$ plus three copies of the $`7`$-dimensional representation of $`𝐆_2`$. Next we shall show that the sum $`\mu =\mu _1\mu _2\mu _3`$ of three copies of $`7`$-dimensional representation of $`H=𝐆_2`$ is not taut. It then follows from Proposition 3.8 that $`\rho `$ is not taut either.

In fact, let $`W_i`$ be the representation space of $`\mu _i`$. We can select $`w_iW_i`$ such that the isotropy groups $`H_{w_1}`$, $`(H_{w_1})_{w_2}`$, $`((H_{w_1})_{w_2})_{w_3}`$ are respectively $`\mathrm{𝐒𝐔}(3)`$, $`\mathrm{𝐒𝐔}(2)`$, $`\{1\}`$. Since $`H_{(w_1,w_2,w_3)}=((H_{w_1})_{w_2})_{w_3}`$, we get that the orbit $`H(w_1,w_2,w_3)`$ is diffeomorphic to $`𝐆_2`$. Now if this orbit is taut, Proposition 3.11 implies that it must have the product homology of the orbits through $`w_1`$, $`w_2`$ and $`w_3`$, namely, $`S^6`$, $`S^5`$ and $`S^3`$. But it is well known that $`𝐆_2`$ has the product homology of $`S^{11}`$ and $`S^3`$ (this also follows from Satz 1 in ). Thus, $`\mu `$ is not taut.
$`\mathrm{}`$

###### 7.8 Lemma

The tensor product $`\rho `$ of the vector representation of $`\mathrm{𝐒𝐎}(3)`$ on $`𝐑^3`$ and the $`7`$-dimensional representation of $`𝐆_2`$ is not taut.

Proof. We shall use the reduction technique described in Subsection 3.3. Note that $`G=\mathrm{𝐒𝐎}(3)\times 𝐆_2`$ is a closed subgroup of $`\widehat{G}=\mathrm{𝐒𝐎}(3)\times \mathrm{𝐒𝐎}(7)`$ and $`\rho `$ is the restriction of a representation $`\widehat{\rho }`$ of $`\widehat{G}`$ which is the isotropy representation of a real Grassmann manifold.

We view the Cayley numbers $`𝐎=𝐇e𝐇`$, so that $`\{1,e,i,j,k,ei,ej,ek\}`$ is a real orthonormal basis for $`𝐎`$. The Lie group $`𝐆_2`$ is the automorphism group of $`𝐎`$ with respect to its nonassociative algebra structure. We next compute the space of orbits of $`G`$ acting by $`\rho `$ on $`V=𝐑^3𝐑^7`$. Take $`\{e,i,j,k,ei,ej,ek\}`$ as a basis for $`𝐑^7`$ and take an orthonormal basis $`\{f_1,f_2,f_3\}`$ for $`𝐑^3`$. Recall that the subspace spanned by $`\{i,j,k,ei,ej,ek\}`$ has a complex structure given by left multiplication by $`e`$. Since all these representations are self-adjoint, it is equivalent to consider $`\rho `$ acting on $`𝐑^7𝐑^3`$ which we may identify with the space of $`3\times 7`$ real matrices.

Let

$$p_1=\left(\begin{array}{ccccccc}1& 0& 0& 0& 0& 0& 0\\ 0& 0& 0& 0& 0& 0& 0\\ 0& 0& 0& 0& 0& 0& 0\end{array}\right),$$

Then the orbit $`G(p_1)`$ is $`8`$-dimensional and its normal space at $`p_1`$ intersects all orbits. So, given an arbitrary orbit $`M`$, it contains a point of the form

$$\left(\begin{array}{ccccccc}a& 0& 0& 0& 0& 0& 0\\ 0& & & & & & \\ 0& & & & & & \end{array}\right),$$

for some $`a0`$, for we can multiply on the left by

$$\left(\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right)$$

to change the sign of $`a`$, if necessary, and remain in the orbit $`M`$. Now the connected component of the isotropy subgroup at $`p_1`$, $`K_1=\mathrm{𝐒𝐎}(2)\times \mathrm{𝐒𝐔}(3)`$, acts on the normal space to $`G(p_1)`$ by fixing the radial direction $`𝐑p_1`$ and by the tensor product of the vector representations of $`\mathrm{𝐒𝐎}(2)`$ and $`\mathrm{𝐒𝐔}(3)`$ on the orthogonal complement $`𝐑^{12}`$. The point

$$p_2=\left(\begin{array}{ccccccc}0& 0& 0& 0& 0& 0& 0\\ 0& 1& 0& 0& 0& 0& 0\\ 0& 0& 0& 0& 0& 0& 0\end{array}\right),$$

is in $`𝐑^{12}`$ and has a $`6`$-dimensional $`K_1`$-orbit. So $`M`$ has a representative of the form

$$\left(\begin{array}{ccccccc}a& 0& 0& 0& 0& 0& 0\\ 0& b& 0& 0& 0& 0& 0\\ 0& 0& & & & & \end{array}\right),$$

for some $`b0`$, since we can also multiply on the left by

$$\left(\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right),$$

if necessary, and remain in the same orbit. Finally, the connected component of the isotropy subgroup of $`K_1`$ at $`p_2`$ is $`\mathrm{𝐒𝐔}(2)`$ and it acts on the normal space to $`K_1(p_2)`$ with respect to $`𝐑^{12}`$ by fixing a $`2`$-plane and by the vector representation on the orthogonal complement. So $`M`$ has an element of the form

(7.9)
$$\left(\begin{array}{ccccccc}a& 0& 0& 0& 0& 0& 0\\ 0& b& 0& 0& 0& 0& 0\\ 0& 0& c& 0& d& 0& 0\end{array}\right),$$

for some $`c0`$. We can assume $`ab`$ by multiplying on the left by

$$\left(\begin{array}{ccc}0& 1& 0\\ 1& 0& 0\\ 0& 0& 1\end{array}\right),$$

and on the right by

$$\left(\begin{array}{ccccccc}0& 1& 0& 0& 0& 0& 0\\ 1& 0& 0& 0& 0& 0& 0\\ 0& 0& 1& 0& 0& 0& 0\\ 0& 0& 0& 0& 0& 1& 0\\ 0& 0& 0& 0& 1& 0& 0\\ 0& 0& 0& 1& 0& 0& 0\\ 0& 0& 0& 0& 0& 0& 1\end{array}\right),$$

if necessary. Multiplying (7.9) on the left by

$$\left(\begin{array}{ccc}1& 0& 0\\ 0& 0& 1\\ 0& 1& 0\end{array}\right)$$

and on the right by

$$\left(\begin{array}{ccccccc}1& 0& 0& 0& 0& 0& 0\\ 0& 0& \frac{c}{\sqrt{c^2+d^2}}& 0& \frac{d}{\sqrt{c^2+d^2}}& 0& 0\\ 0& \frac{c}{\sqrt{c^2+d^2}}& 0& 0& 0& \frac{d}{\sqrt{c^2+d^2}}& 0\\ 0& 0& 0& 1& 0& 0& 0\\ 0& \frac{d}{\sqrt{c^2+d^2}}& 0& 0& 0& \frac{c}{\sqrt{c^2+d^2}}& 0\\ 0& 0& \frac{d}{\sqrt{c^2+d^2}}& 0& \frac{c}{\sqrt{c^2+d^2}}& 0& 0\\ 0& 0& 0& 0& 0& 0& 1\end{array}\right)$$

we get

$$\left(\begin{array}{ccccccc}a& 0& 0& 0& 0& 0& 0\\ 0& \sqrt{c^2+d^2}& 0& 0& 0& 0& 0\\ 0& 0& \frac{bc}{\sqrt{c^2+d^2}}& 0& \frac{bd}{\sqrt{c^2+d^2}}& 0& 0\end{array}\right).$$

We can therefore assume $`b\sqrt{c^2+d^2}`$. It is straightforward to check that no further restrictions can be put on $`a`$, $`b`$, $`c`$, $`d`$. Hence we see that the orbit space is described by the matrices of the form (7.9) subject to the conditions $`ab\sqrt{c^2+d^2}`$ and $`c0`$.

Next we want to compute a principal isotropy subgroup of $`G`$. Let $`p`$ be an element of the form (7.9) which is regular for both $`G`$ and $`\widehat{G}`$. The principal isotropy group $`\widehat{G}_p`$ is isomorphic to $`𝐙_2^2\times \mathrm{𝐒𝐎}(4)`$, and $`G_p=\widehat{G}_pG`$. Now the $`𝐙_2^2`$-factor is contained in $`G_p`$, because

$$(\text{diag(}1,1,1\text{)},\text{diag(}1,1,1,1,1,1,1\text{)})$$

and

$$(\text{diag(}1,1,1\text{)},\text{diag(}1,1,1,1,1,1,1\text{)})$$

are elements of $`G`$ that fix $`p`$. On the other hand, an element in the $`\mathrm{𝐒𝐎}(4)`$-factor that fixes $`p`$ and is in $`G=\mathrm{𝐒𝐎}(3)\times 𝐆_2`$ is in fact in $`\{1\}\times 𝐆_2`$ and has to fix $`e`$, $`i`$, $`cj+d(ei)𝐑^7`$. Therefore it also fixes $`j=\frac{1}{c}[(cj+d(ei))d(ei)]`$ ($`c0`$ because $`p`$ is regular for $`G`$) and hence it is the identity. We conclude that the principal isotropy group $`H=G_p`$ is exactly $`𝐙_2^2`$.

It is immediate to see that the fixed subspace $`V^H`$ is

$$\left(\begin{array}{ccccccc}& 0& 0& & 0& 0& 0\\ 0& & 0& 0& 0& & 0\\ 0& 0& & 0& & 0& 0\end{array}\right).$$

Let $`N`$ be the normalizer of $`H`$ in $`G`$ and $`\overline{G}=N/H`$ the reduced group as in Subsection 3.3. The connected component $`\overline{G}^0`$ must be contained in $`\{1\}\times 𝐆_2`$. One can use the fact that $`N`$ equals the normalizer of $`V^H`$ in $`G`$ to verify that $`\overline{G}^0`$ is the $`2`$-torus group consisting of the matrices

(7.10)
$$\left(\begin{array}{ccccccc}\mathrm{cos}t& 0& 0& \mathrm{sin}t& 0& 0& 0\\ 0& \mathrm{cos}s& 0& 0& 0& \mathrm{sin}s& 0\\ 0& 0& \mathrm{cos}(st)& 0& \mathrm{sin}(st)& 0& 0\\ \mathrm{sin}t& 0& 0& \mathrm{cos}t& 0& 0& 0\\ 0& 0& \mathrm{sin}(st)& 0& \mathrm{cos}(st)& 0& 0\\ 0& \mathrm{sin}s& 0& 0& 0& \mathrm{cos}s& 0\\ 0& 0& 0& 0& 0& 0& 1\end{array}\right)$$

for $`s`$, $`t𝐑`$.

Let $`pV^H`$ be a regular point and let $`M=Gp`$ be the corresponding principal orbit in $`V`$. We next prove that $`M`$ is not taut by contradiction. In fact, assume $`M`$ is taut. The orbit $`\overline{G}^0p`$ is a connected component of $`M^H`$, and it is a substantial $`2`$-torus embedded in the $`6`$-dimensional Euclidean space $`V^H`$. Since its second osculating space is at most $`5`$-dimensional, it cannot be taut by Theorem 2.5. It is known that a surface is taut if and only if its lines of curvature are circles. Therefore there is a parallel normal vector field $`\xi (t)`$ along a curve $`\gamma (t)`$ in $`\overline{G}^0pM^H`$ such that $`A_{\xi (t)}^{M^H}\dot{\gamma }(t)=\lambda (t)\dot{\gamma }(t)`$ and $`\lambda (t)`$ is not constant. (*A posteriori* it is clear that one can find $`\gamma (t)`$, $`\xi (t)`$, $`\lambda (t)`$ directly by computation using (7.10).) Since $`M`$ is a principal $`G`$-orbit, the normal spaces of $`M`$ and $`M^H`$ in $`V`$ coincide along $`\gamma (t)`$ implying that $`\xi (t)`$ is also parallel with respect to $`M`$. By Lemma 3.16 we have

(7.11)
$$A_{\xi (t)}^M\dot{\gamma }(t)=\lambda (t)\dot{\gamma }(t).$$

We do not claim that $`\gamma (t)`$ is a curvature line of $`M`$ since the eigenspace corresponding to $`\lambda (t)`$ might not be one-dimensional. Still the argument in , Lemmas 1 and 2 (see also ) carries through and shows that (7.11) together with the tautness of $`M`$ implies that $`\lambda (t)`$ is constant, which is a contradiction. Hence $`M`$ is not taut. Thus the representation $`\rho `$ is not taut either. $`\mathrm{}`$

###### 7.12 Proposition

The representations listed in Table C.1 are taut.

Proof. We use the reduction technique discussed in Subsection 3.3. Consider first the case of $`G=𝐔(2)\times \mathrm{𝐒𝐩}(n)`$, $`n2`$, acting on the realification of $`𝐂^2𝐂^{2n}`$ by the tensor product of the vector representations. It is equivalent to consider $`\rho `$ acting on the space of $`2\times 2n`$ complex matrices, i. e. $`\rho (A,B)X=AXB^1`$ for $`A𝐔(2)`$, $`B\mathrm{𝐒𝐩}(n)\mathrm{𝐒𝐔}(2n)`$, $`XM(2\times 2n;𝐂)`$. It follows rather easily that the orbit space consists of matrices of the form

$$\left(\begin{array}{cccccccccc}a& 0& 0& \mathrm{}& 0& 0& 0& 0& \mathrm{}& 0\\ 0& b& 0& \mathrm{}& 0& c& 0& 0& \mathrm{}& 0\end{array}\right),$$

where $`a\sqrt{b^2+c^2}`$ and $`b`$, $`c0`$. Since these matrices are all real, it follows that the group obtained by adjoining to $`G`$ the complex conjugation of matrices $`\sigma `$ still has the same orbits. We replace $`G`$ by the enlarged group and continue to denote it with the same letter.

Now the principal isotropy group $`H`$ is generated by $`\sigma `$ and the subgroup of $`G`$ isomorphic to $`𝐔(1)\times \mathrm{𝐒𝐩}(n2)`$ consisting of elements of the form

$$(\left(\begin{array}{cc}e^{i\theta }& 0\\ 0& e^{i\theta }\end{array}\right),\left(\begin{array}{cccccc}e^{i\theta }& & & 0& & \\ & e^{i\theta }& & & 0& \\ & & C& & & D\\ 0& & & e^{i\theta }& & \\ & 0& & & e^{i\theta }& \\ & & \overline{D}& & & \overline{C}\end{array}\right)),$$

where $`\theta 𝐑`$ and $`\left(\begin{array}{cc}C& D\\ \overline{D}& \overline{C}\end{array}\right)\mathrm{𝐒𝐩}(n2)`$. Now $`V^H`$ is the space of matrices of the form

$$\left(\begin{array}{cccccccccc}a& 0& 0& \mathrm{}& 0& 0& d& 0& \mathrm{}& 0\\ 0& b& 0& \mathrm{}& 0& c& 0& 0& \mathrm{}& 0\end{array}\right),$$

whose entries are real, and $`(N/H)^0`$ is the circle group consisting of the coclasses defined by the matrices

$$\left(\begin{array}{cccccc}\mathrm{cos}\theta & 0& & 0& \mathrm{sin}\theta & \\ 0& \mathrm{cos}\theta & & \mathrm{sin}\theta & 0& \\ & & \mathrm{𝟏}& & & \mathrm{𝟎}\\ 0& \mathrm{sin}\theta & & \mathrm{cos}\theta & 0& \\ \mathrm{sin}\theta & 0& & 0& \mathrm{cos}\theta & \\ & & \mathrm{𝟎}& & & \mathrm{𝟏}\end{array}\right)\{1\}\times \mathrm{𝐒𝐩}(n)$$

for $`\theta 𝐑`$, where $`\mathrm{𝟏}`$ (resp. $`\mathrm{𝟎}`$) denotes the $`(n2)\times (n2)`$ identity (resp. zero) block. We see that the reduced representation $`\overline{\rho }:(N/H)^0𝐎(V^H)`$ is just the direct sum of two copies of the vector representation of $`\mathrm{𝐒𝐎}(2)`$ on $`𝐑^2`$, hence taut. Finally, we conclude that $`\rho `$ is taut by Proposition 3.19, where we choose $`L`$ to be generated by the following elements: $`\sigma `$, the diagonal elements in $`\mathrm{𝐒𝐩}(n2)H`$ with $`\pm 1`$ entries, and the element

$$(\left(\begin{array}{cc}i& 0\\ 0& i\end{array}\right),\left(\begin{array}{cccccc}i& & & & & \\ & i& & & & \\ & & \mathrm{𝟏}& & & \\ & & & i& & \\ & & & & i& \\ & & & & & \mathrm{𝟏}\end{array}\right)).$$

Next consider the case of $`G=\mathrm{𝐒𝐎}(2)\times \mathrm{𝐒𝐩𝐢𝐧}(9)`$ acting on $`𝐑^2𝐑^{16}`$ by the tensor product of the vector and spin representations. It is equivalent to consider $`\rho `$ acting on the space of $`2\times 16`$ real matrices, namely, $`\rho (A,B)X=AXB^1`$ for $`A\mathrm{𝐒𝐎}(2)`$, $`B\mathrm{𝐒𝐩𝐢𝐧}(9)\mathrm{𝐒𝐎}(16)`$, $`XM(2\times 16;𝐑)`$. We identify $`𝐑^{16}𝐎𝐎`$ and use $`\{1,e,i,j,k,ei,ej,ek\}`$ as a basis of $`𝐎`$ over $`𝐑`$. For the computations involved in this case, Lemmas 14.61 and 14.77 from  are useful, which we state as follows:

###### 7.13 Lemma

A matrix $`g𝐎(8)𝐎(𝐎)`$ belongs to $`\mathrm{𝐒𝐩𝐢𝐧}(7)`$ if and only if $`g(uv)=g(u)\chi (g)(v)`$ for all $`u`$, $`v𝐎`$, where $`\chi (g)(v)=g(g^1(1)v)`$ for $`v𝐎`$ defines the vector representation of $`\mathrm{𝐒𝐩𝐢𝐧}(7)`$ on $`\mathrm{}𝐎𝐑^7`$.

###### 7.14 Lemma

The subgroup $`\mathrm{𝐒𝐩𝐢𝐧}(9)\mathrm{𝐒𝐎}(16)\mathrm{𝐒𝐎}(𝐎𝐎)`$ is generated by the $`8`$-sphere

$$\{\left(\begin{array}{cc}r& R_u\\ R_{\overline{u}}& r\end{array}\right):r𝐑,u𝐎,r^2+|u|^2=1\}$$

in $`\text{End}(𝐎𝐎)`$, where $`R_u:𝐎𝐎`$ denotes right translation by the element $`u𝐎`$.

Mainly as a consequence of

(7.15)
$$\text{}|_{B_3}=\text{(trivial)}\text{}\text{}$$

one can see that the $`G`$-orbit space consists of matrices of the form

(7.16)
$$\left(\begin{array}{cccccccccccccccc}a& 0& 0& 0& 0& 0& 0& 0& 0& 0& 0& 0& 0& 0& 0& 0\\ 0& b& 0& 0& 0& 0& 0& 0& c& 0& 0& 0& 0& 0& 0& 0\end{array}\right),$$

and, with some more effort (using Lemma 7.14), that the restrictions $`a\sqrt{b^2+c^2}`$ and $`b`$, $`c0`$ apply. Also, Lemma 7.13 shows that

$$g_1=\text{diag}(1,1,1,1,111,1)\mathrm{𝐒𝐩𝐢𝐧}(7),$$

which combined with (7.15) yields that

$$g_2=\left(\begin{array}{ccc}1& & \\ & \chi (g_1)& \\ & & g_1\end{array}\right)\mathrm{𝐒𝐩𝐢𝐧}(9),$$

where $`\chi (g_1)=\text{diag}(1,1,1,1,1,1,1)`$. In particular, the equality $`\left(\begin{array}{cc}1& 0\\ 0& 1\end{array}\right)p=pg_2`$ where $`p`$ is as in (7.16) shows that $`𝐎(2)\times \mathrm{𝐒𝐩𝐢𝐧}(9)`$ has the same orbits as $`\mathrm{𝐒𝐎}(2)\times \mathrm{𝐒𝐩𝐢𝐧}(9)`$. Hereafter we consider $`G=𝐎(2)\times \mathrm{𝐒𝐩𝐢𝐧}(9)`$.

The principal isotropy group $`H`$ is isomorphic to $`𝐙_2\times 𝐙_2\times \mathrm{𝐒𝐔}(3)`$, where the first $`𝐙_2`$ corresponds to the kernel of $`\rho `$ and it is generated by the element $`h_1=(1,1)𝐎(2)\times \mathrm{𝐒𝐩𝐢𝐧}(9)`$, the second $`𝐙_2`$ is generated by the element

$$h_2=(\text{diag}(1,1),g_2),$$

and the $`\mathrm{𝐒𝐔}(3)`$-factor consists of matrices of the form

$$\left(\begin{array}{cccccc}1& & & & & \\ & 1& & & & \\ & & \begin{array}{cc}A& B\\ B& A\end{array}& & & \\ & & & 1& & \\ & & & & 1& \\ & & & & & \begin{array}{cc}A& B\\ B& A\end{array}\end{array}\right)\{1\}\times \mathrm{𝐒𝐩𝐢𝐧}(9),$$

where $`A+iB\mathrm{𝐒𝐔}(3)`$. Now $`V^H`$ is the space of matrices of the form

$$\left(\begin{array}{cccccccccccccccc}a& 0& 0& 0& 0& 0& 0& 0& 0& d& 0& 0& 0& 0& 0& 0\\ 0& b& 0& 0& 0& 0& 0& 0& c& 0& 0& 0& 0& 0& 0& 0\end{array}\right),$$

whose entries are real, and $`(N/H)^0`$ is the circle group consisting of the coclasses defined by the following matrices of $`\{1\}\times \mathrm{𝐒𝐩𝐢𝐧}(9)`$:

(7.17)
$$\left(\begin{array}{cccccc}\mathrm{cos}\theta & & & & \mathrm{sin}\theta & \\ & \mathrm{cos}\theta & & \mathrm{sin}\theta & & \\ & & \begin{array}{cc}\mathrm{cos}\theta \mathrm{𝟏}& \\ & \mathrm{cos}\theta \mathrm{𝟏}\end{array}& & & \begin{array}{cc}& \mathrm{sin}\theta \mathrm{𝟏}\\ \mathrm{sin}\theta \mathrm{𝟏}& \end{array}\\ & \mathrm{sin}\theta & & \mathrm{cos}\theta & & \\ \mathrm{sin}\theta & & & & \mathrm{cos}\theta & \\ & & \begin{array}{cc}& \mathrm{sin}\theta \mathrm{𝟏}\\ \mathrm{sin}\theta \mathrm{𝟏}& \end{array}& & & \begin{array}{cc}\mathrm{cos}\theta \mathrm{𝟏}& \\ & \mathrm{cos}\theta \mathrm{𝟏}\end{array}\end{array}\right)$$

for $`\theta 𝐑`$, where $`\mathrm{𝟏}`$ denotes the $`3\times 3`$ identity block (note that the matrix (7.17) belongs to $`\mathrm{𝐒𝐩𝐢𝐧}(9)`$ because it is the product of the two matrices in $`\mathrm{𝐒𝐩𝐢𝐧}(9)`$ which are obtained by setting $`r=\mathrm{cos}\theta `$, $`u=\mathrm{sin}\theta e`$ and $`r=1`$, $`u=0`$ in Lemma 7.14). We see that the reduced representation $`\overline{\rho }`$ is just the direct sum of two copies of the vector representation of $`\mathrm{𝐒𝐎}(2)`$ on $`𝐑^2`$, hence taut. We conclude that $`\rho `$ is taut by Proposition 3.19, where we choose $`L`$ to be generated by $`h_1`$, $`h_2`$ and the elements in the $`\mathrm{𝐒𝐔}(3)`$-factor of $`H`$ with $`A`$ diagonal with $`\pm 1`$ entries and $`B=0`$.

Finally we consider the case of $`G=\mathrm{𝐒𝐔}(2)\times \mathrm{𝐒𝐩}(n)`$, $`n2`$, acting on a real form $`V`$ of $`𝐂^4𝐂^{2n}`$. It is equivalent to consider $`\rho `$ acting on a subspace of $`4\times 2n`$ complex matrices, namely, $`\rho (A,B)Z=\eta (A)ZB^1`$ for $`A\mathrm{𝐒𝐔}(2)`$, $`B\mathrm{𝐒𝐩}(n)\mathrm{𝐒𝐔}(2n)`$, $`ZV=\left\{\left(\begin{array}{cc}X& Y\\ \overline{Y}& \overline{X}\end{array}\right)M(4\times 2n;𝐂)\right\}`$, where $`\eta :\mathrm{𝐒𝐔}(2)\mathrm{𝐒𝐩}(2)\mathrm{𝐒𝐔}(4)`$ is the $`4`$-dimensional complex representation of $`\mathrm{𝐒𝐔}(2)`$ given by

$$\eta \left(\begin{array}{cc}\alpha & \beta \\ \overline{\beta }& \overline{\alpha }\end{array}\right)=\left(\begin{array}{cccc}\alpha ^3& \sqrt{3}\alpha \beta ^2& \beta ^3& \sqrt{3}\alpha ^2\beta \\ \sqrt{3}\alpha \overline{\beta }^2& \alpha \overline{\alpha }^22\overline{\alpha }\beta \overline{\beta }& \sqrt{3}\overline{\alpha }^2\beta & \beta \overline{\beta }^22\alpha \overline{\alpha }\overline{\beta }\\ \overline{\beta }^3& \sqrt{3}\overline{\alpha }^2\overline{\beta }& \overline{\alpha }^3& \sqrt{3}\overline{\alpha }\overline{\beta }^2\\ \sqrt{3}\alpha ^2\overline{\beta }& 2\alpha \overline{\alpha }\beta \beta ^2\overline{\beta }& \sqrt{3}\overline{\alpha }\beta ^2& \alpha ^2\overline{\alpha }2\alpha \beta \overline{\beta }\end{array}\right).$$

Let

$$p_1=\left(\begin{array}{cccccccccc}1& 0& 0& \mathrm{}& 0& 0& 0& 0& \mathrm{}& 0\\ 0& 0& 0& \mathrm{}& 0& 0& 0& 0& \mathrm{}& 0\\ 0& 0& 0& \mathrm{}& 0& 1& 0& 0& \mathrm{}& 0\\ 0& 0& 0& \mathrm{}& 0& 0& 0& 0& \mathrm{}& 0\end{array}\right)V.$$

Then the orbit $`G(p_1)`$ is $`(4n+1)`$-dimensional and its normal space intersects all orbits at $`p_1`$. So, given an arbitrary orbit $`M`$, it contains a point of the form

(7.18)
$$\left(\begin{array}{cccccccccc}a& 0& 0& \mathrm{}& 0& 0& 0& 0& \mathrm{}& 0\\ x_1& x_2& x_3& \mathrm{}& x_n& 0& y_2& y_3& \mathrm{}& y_n\\ 0& 0& 0& \mathrm{}& 0& a& 0& 0& \mathrm{}& 0\\ 0& \overline{y}_2& \overline{y}_3& \mathrm{}& \overline{y}_n& \overline{x}_1& \overline{x}_2& \overline{x}_3& \mathrm{}& \overline{x}_n\end{array}\right)$$

where $`x_i`$, $`y_i𝐂`$ and $`a0`$, for we can multiply on the left by the minus identity matrix to change the sign of $`a`$, if necessary, and remain in the orbit $`M`$. The isotropy subgroup $`K_1=G_{p_1}`$ is isomorphic to $`S^1\times \mathrm{𝐒𝐩}(n1)`$, where the circle factor consists of the elements

$$(\left(\begin{array}{cc}e^{i\theta }& \\ & e^{i\theta }\end{array}\right),\left(\begin{array}{cccc}e^{3i\theta }& & & \\ & \mathrm{𝟏}& & \\ & & e^{3i\theta }& \\ & & & \mathrm{𝟏}\end{array}\right))$$

for $`\theta 𝐑`$, where $`\mathrm{𝟏}`$ denotes the $`(n1)\times (n1)`$ identity block. By letting $`K_1`$ act on the matrix (7.18) we now see that the $`G`$-orbit space consists of real matrices of the form

$$\left(\begin{array}{cccccccccc}a& 0& 0& \mathrm{}& 0& 0& 0& 0& \mathrm{}& 0\\ c& b& 0& \mathrm{}& 0& 0& 0& 0& \mathrm{}& 0\\ 0& 0& 0& \mathrm{}& 0& a& 0& 0& \mathrm{}& 0\\ 0& 0& 0& \mathrm{}& 0& c& b& 0& \mathrm{}& 0\end{array}\right).$$

(One can also check that some restrictions on $`a`$, $`b`$ and $`c`$ apply, but this is unimportant to us.)

The principal isotropy group

$$H=\{(q,\left(\begin{array}{cc}\eta (q)& \\ & C\end{array}\right)):C\mathrm{𝐒𝐩}(n2),q=\left(\begin{array}{cc}\alpha & \beta \\ \overline{\beta }& \overline{\alpha }\end{array}\right),(\alpha ,\beta )=\pm (1,0),\pm (i,0),\pm (0,1),\pm (0,i)\}$$

is isomorphic to $`Q\times \mathrm{𝐒𝐩}(n2)`$, where $`Q`$ is quaternion group $`\{\pm 1,\pm i,\pm j,\pm k\}`$. Now $`V^H`$ is the space of matrices of the form

$$\left(\begin{array}{cccccccccc}a& d& 0& \mathrm{}& 0& 0& 0& 0& \mathrm{}& 0\\ c& b& 0& \mathrm{}& 0& 0& 0& 0& \mathrm{}& 0\\ 0& 0& 0& \mathrm{}& 0& a& d& 0& \mathrm{}& 0\\ 0& 0& 0& \mathrm{}& 0& c& b& 0& \mathrm{}& 0\end{array}\right),$$

where the entries are real, and $`(N/H)^0`$ is the circle group consisting of the coclasses defined by the matrices

$$\left(\begin{array}{cccccc}\mathrm{cos}\theta & \mathrm{sin}\theta & & & & \\ \mathrm{sin}\theta & \mathrm{cos}\theta & & & & \\ & & \mathrm{𝟏}& & & \\ & & & \mathrm{cos}\theta & \mathrm{sin}\theta & \\ & & & \mathrm{sin}\theta & \mathrm{cos}\theta & \\ & & & & & \mathrm{𝟏}\end{array}\right)\{1\}\times \mathrm{𝐒𝐩}(n)$$

We see that the reduced representation $`\overline{\rho }`$ is just the direct sum of two copies of the vector representation of $`\mathrm{𝐒𝐎}(2)`$ on $`𝐑^2`$, hence taut. We conclude that $`\rho `$ is taut by Proposition 3.19, where we choose $`L`$ to be generated by $`Q`$ and the diagonal elements in $`\mathrm{𝐒𝐩}(n2)H`$ with $`\pm 1`$ entries. $`\mathrm{}`$

## 8 Other applications

In this last section we collect some further applications of our methods.

###### 8.1 Theorem (Dadok)

A polar representation of a compact connected Lie group is orbit equivalent to the isotropy representation of a symmetric space.

Proof. Conlon’s theorem (Theorem 2.3) says that a polar representation is variationally complete. Now use Theorem 6.4. $`\mathrm{}`$

The next theorem, which is a special case of a result of Montgomery and Samelson on transitive homeomorphic actions on spheres (), see also  by Borel, was used in Sections 5 and 6.

###### 8.2 Theorem

The transitive effective linear actions of compact connected Lie groups $`G`$ on the spheres $`S^d`$ are given by the following tables (in each case, the action is the obvious one):

$$\begin{array}{cccccc}& & & & & \\ G& \mathrm{𝐒𝐎}(n)& 𝐆_2& \mathrm{𝐒𝐩𝐢𝐧}(7)& \mathrm{𝐒𝐩𝐢𝐧}(9)& \mathrm{𝐒𝐩}(1)\mathrm{𝐒𝐩}(n)\\ & & & & & \\ S^d& S^{n1}& S^6& S^7& S^{15}& S^{4n1}\end{array}$$

$$\begin{array}{ccccc}& & & & \\ G& \mathrm{𝐒𝐔}(n)& S^1\mathrm{𝐒𝐔}(n)& \mathrm{𝐒𝐩}(n)& S^1\mathrm{𝐒𝐩}(n)\\ & & & & \\ S^d& \multicolumn{2}{c}{S^{2n1}}& \multicolumn{2}{c}{S^{4n1}}\end{array}$$

Proof. We may assume that the action is given by a real irreducible representation $`\rho `$ of $`G`$. It is enough to classify $`\rho `$ up to image equivalence and then factor the kernel out. If $`\rho `$ is of quaternionic or complex type, then Lemma 5.12 implies that $`k(\lambda )=1`$ and we get the standard action of $`\mathrm{𝐒𝐎}(2)`$ on $`S^1`$ and the actions in the second table. If $`\rho `$ is of real type, then Lemmas 5.11, 5.14 and 5.15 together give the remaining actions in the first table. $`\mathrm{}`$

Representations $`\rho :G𝐎(V)`$ with cohomogeneity equal to two were classified up to orbit equivalence in  by Hsiang and Lawson888The classification of cohomogeneity two and three representations in  contain gaps which were to our knowledge first corrected in the papers .. It is not difficult to prove that a representation with cohomogeneity two is polar. Hence it is also taut and if it is irreducible it is therefore in class $`𝒪^2`$.

We next show that it is quite easy to see directly that an irreducible representation with cohomogeneity two is in class $`𝒪^2`$. We formulate this in the next proposition.

###### 8.3 Proposition

An irreducible representation $`\rho :G𝐎(V)`$ with cohomogeneity two is in class $`𝒪^2`$.

Proof. We can clearly restrict our attention to orbits contained in the unit sphere $`S`$ of $`V`$. Let $`p`$ in $`S`$ be a point whose orbit has codimension two. The second osculating space $`𝒪_p^2(Gp)`$ is the direct sum of $`T_p(Gp)`$ and the linear span of $`\{\alpha (X,Y)|X,YT_p(Gp)\}`$ where $`\alpha `$ is the second fundamental form of the orbit $`Gp`$ as a submanifold of $`V`$. It is a general property of submanifolds contained in the sphere $`S`$ that $`\alpha (X,X),\xi =1`$ for all unit vectors $`XT_p(Gp)`$, where $`\xi `$ is the inward pointing unit normal vector of $`S`$ in $`p`$. We therefore need to prove that the linear span of $`\{\alpha (X,Y)|X,YT_p(Gp)\}`$ is not one dimensional. Let $`\eta `$ be a normal vector of $`Gp`$ at $`p`$ that is tangent to the sphere and let $`A_\eta ^S`$ be the Weingarten map of $`Gp`$ at $`p`$ as a submanifold of $`S`$. The orbit $`Gp`$ cannot be umbilical in $`S`$ since then it would be a round sphere contradicting the assumption that $`\rho `$ is irreducible. Hence $`A_\eta ^S`$ has at least two different eigenvalues $`\lambda _1`$ and $`\lambda _2`$ with corresponding eigenvectors $`X_1`$ and $`X_2`$ that we assume to have length one. Then

$`\alpha (X_i,X_i)`$ $`=`$ $`\alpha (X_i,X_i),\xi \xi +\alpha (X_i,X_i),\eta \eta `$
$`=`$ $`\xi +\lambda _i\eta `$
for $`i=1,2`$. It follows that $`\alpha (X_1,X_1)`$ and $`\alpha (X_2,X_2)`$ are not linearly dependent and hence that $`V=𝒪_p^2(Gp)`$.

Now assume that the orbit through $`p`$ in $`S`$ is singular. If we restrict the action of $`\rho `$ to $`S`$, then we have a cohomogeneity one action. It is well known that the slice representations of cohomogeneity one actions are irreducible and transitive on spheres in the normal space. If we now think of the orbit $`Gp`$ as a submanifold of $`V`$, then we see that the slice representation of $`G_p`$ on the normal space $`N_p(Gp)`$ decomposes into the trivial representation on $`𝐑\xi `$ and an irreducible representation on $`N_p^S(Gp)`$ where $`\xi `$ is a normal vector of the sphere $`S`$ as above and $`N_p^S(Gp)`$ is the normal space of $`Gp`$ in $`S`$. The linear span of $`\{\alpha (X,Y)|X,YT_p(Gp)\}`$ is clearly invariant under the isotropy representation of $`G_p`$. It follows that it contains $`N_p^S(Gp)`$ since it must have a component in $`N_p^S(Gp)`$. (If it had no component in $`N_p^S(Gp)`$, then $`Gp`$ would be totally geodesic in $`S`$, hence a round sphere, contradicting the assumption that $`\rho `$ is irreducible.) The linear span of $`\{\alpha (X,Y)|X,YT_p(Gp)\}`$ cannot coincide with $`N_p^S(Gp)`$, since $`\alpha (X,X)N_p^S(Gp)`$ for a nonvanishing $`X`$ in $`T_p(Gp)`$. It follows that $`V=𝒪_p^2(Gp)`$. $`\mathrm{}`$

###### 8.4 Theorem (Hsiang-Lawson)

The cohomogeneity two linear actions of compact connected Lie groups on the Euclidean spaces are given, up to orbit equivalence, by the isotropy representations of the rank two symmetric spaces.

Proof. A cohomogeneity two reducible representation is the direct sum of two cohomogeneity one representations and therefore orbit equivalent to the isotropy representation of the product of two spheres.

A cohomogeneity two irreducible representation of a compact connected Lie group is of class $`𝒪^2`$ (Proposition 8.3). Therefore the result follows from the fact that no representation listed in Tables C.1, C.2, C.3 and C.4 is of cohomogeneity two, which is easy to check. $`\mathrm{}`$

## Appendix A Description of the maximal subsets of strongly orthogonal roots of the complex simple Lie algebras

In this appendix, for each one of the complex simple Lie algebras, we give a description of the maximal subset of strongly orthogonal roots $`=\{\beta _1,\mathrm{},\beta _s\}`$ which was introduced in Proposition 4.1. In each case, $`\mathrm{\Delta }^+`$ is the positive root system, $`𝒮=\{\alpha _1,\mathrm{},\alpha _n\}`$ is the simple root system, and $`\lambda _1,\mathrm{},\lambda _r`$ are the fundamental highest weights ($`\theta _1`$, $`\theta _2,\mathrm{}`$ refer to specific linear forms on the Cartan subalgebra; see e. g. ).

$$\begin{array}{ccc}& & \\ \multicolumn{3}{c}{𝐀_n:\text{}}\\ & & \\ \multicolumn{3}{c}{𝒮=\{\alpha _1=\theta _1\theta _2,\mathrm{},\alpha _n=\theta _n\theta _{n+1},\}}\\ & & \\ \multicolumn{3}{c}{\mathrm{\Delta }^+=\{\theta _i\theta _j:1i<jn+1\}}\\ & & \\ \lambda _1=\theta _1\hfill & \beta _1=\theta _1\theta _{n+1}\hfill & \beta _1=\theta _1\theta _{n+1}\hfill \\ \lambda _2=\theta _1+\theta _2\hfill & \beta _2=\theta _2\theta _n\hfill & \beta _2=\theta _2\theta _n\hfill \\ \mathrm{}\hfill & \mathrm{}\hfill & \mathrm{}\hfill \\ \lambda _n=\theta _1+\theta _2+\mathrm{}+\theta _n\hfill & \beta _n=\theta _{\frac{n}{2}}\theta _{\frac{n}{2}+2}\hfill & \beta _n=\theta _{\frac{n+1}{2}}\theta _{\frac{n+1}{2}+1}\hfill \\ & \text{(}n\text{ even)}\hfill & \text{(}n\text{ odd)}\hfill \end{array}$$

$$\begin{array}{ccc}& & \\ \multicolumn{3}{c}{𝐁_n:\text{}}\\ & & \\ \multicolumn{3}{c}{𝒮=\{\alpha _1=\theta _1\theta _2,\mathrm{},\alpha _{n1}=\theta _{n1}\theta _n,\alpha _n=\theta _n\}}\\ & & \\ \multicolumn{3}{c}{\mathrm{\Delta }^+=\{\theta _i\pm \theta _j:1i<jn\}\{\theta _i:1n\}}\\ & & \\ \lambda _1=\theta _1\hfill & \beta _1=\theta _1+\theta _2\hfill & \beta _1=\theta _1+\theta _2\hfill \\ \lambda _2=\theta _1+\theta _2\hfill & \beta _2=\theta _1\theta _2\hfill & \beta _2=\theta _1\theta _2\hfill \\ \mathrm{}\hfill & \mathrm{}\hfill & \mathrm{}\hfill \\ \lambda _{n2}=\theta _1+\theta _2+\mathrm{}+\theta _{n2}\hfill & \beta _{n2}=\theta _{n2}+\theta _{n1}\hfill & \beta _{n1}=\theta _{n1}+\theta _n\hfill \\ \lambda _{n1}=\theta _1+\theta _2+\mathrm{}+\theta _{n2}+\theta _{n1}\hfill & \beta _{n1}=\theta _{n2}\theta _{n1}\hfill & \beta _n=\theta _{n1}\theta _n\hfill \\ \lambda _n=\frac{1}{2}(\theta _1+\theta _2+\mathrm{}+\theta _{n1}+\theta _n)\hfill & \beta _n=\theta _n\hfill & \\ & \text{(}n\text{ odd)}\hfill & \text{(}n\text{ even)}\hfill \end{array}$$

$$\begin{array}{cc}& \\ \multicolumn{2}{c}{𝐂_n:\text{}}\\ & \\ \multicolumn{2}{c}{𝒮=\{\alpha _1=\theta _1\theta _2,\mathrm{},\alpha _{n1}=\theta _{n1}\theta _n,\alpha _n=2\theta _n\}}\\ & \\ \multicolumn{2}{c}{\mathrm{\Delta }^+=\{\theta _i\pm \theta _j:1i<jn\}\{2\theta _i:1n\}}\\ & \\ \lambda _1=\theta _1\hfill & \beta _1=2\theta _1\hfill \\ \lambda _2=\theta _1+\theta _2\hfill & \beta _2=2\theta _2\hfill \\ \mathrm{}\hfill & \mathrm{}\hfill \\ \lambda _n=\theta _1+\theta _2+\mathrm{}+\theta _n\hfill & \beta _n=2\theta _n\hfill \end{array}$$

$$\begin{array}{ccc}& & \\ \multicolumn{3}{c}{𝐃_n:\text{}}\\ & & \\ \multicolumn{3}{c}{𝒮=\{\alpha _1=\theta _1\theta _2,\mathrm{},\alpha _{n2}=\theta _{n2}\theta _{n1},\alpha _{n1}=\theta _{n1}\theta _n,\alpha _n=\theta _{n1}+\theta _n\}}\\ & & \\ \multicolumn{3}{c}{\mathrm{\Delta }^+=\{\theta _i\pm \theta _j:1i<jn\}}\\ & & \\ \lambda _1=\theta _1\hfill & \beta _1=\theta _1+\theta _2\hfill & \beta _1=\theta _1+\theta _2\hfill \\ \lambda _2=\theta _1+\theta _2\hfill & \beta _2=\theta _1\theta _2\hfill & \beta _2=\theta _1\theta _2\hfill \\ \mathrm{}\hfill & \mathrm{}\hfill & \mathrm{}\hfill \\ \lambda _{n2}=\theta _1+\theta _2+\mathrm{}+\theta _{n2}\hfill & \beta _{n2}=\theta _{n2}+\theta _{n1}\hfill & \beta _{n1}=\theta _{n1}+\theta _n\hfill \\ \lambda _{n1}=\frac{1}{2}(\theta _1+\theta _2+\mathrm{}+\theta _{n1}\theta _n)\hfill & \beta _{n1}=\theta _{n2}\theta _{n1}\hfill & \beta _n=\theta _{n1}\theta _n\hfill \\ \lambda _n=\frac{1}{2}(\theta _1+\theta _2+\mathrm{}+\theta _{n1}+\theta _n)\hfill & \text{(}n\text{ odd)}\hfill & \text{(}n\text{ even)}\hfill \end{array}$$

$$\begin{array}{cc}& \\ \multicolumn{2}{c}{𝐆_2:\text{}}\\ & \\ \multicolumn{2}{c}{\mathrm{\Delta }^+=\{\alpha _1,\alpha _2,\alpha _1+\alpha _2,2\alpha _1+\alpha _2,3\alpha _1+\alpha _2,3\alpha _1+2\alpha _2\}}\\ & \\ \lambda _1=2\alpha _1+\alpha _2\hfill & \beta _1=3\alpha _1+2\alpha _2\hfill \\ \lambda _2=3\alpha _1+2\alpha _2\hfill & \beta _2=\alpha _1\hfill \end{array}$$

$$\begin{array}{cc}& \\ \multicolumn{2}{c}{𝐅_4:\text{}}\\ & \\ \multicolumn{2}{c}{𝒮=\{\alpha _1=\theta _2\theta _3,\alpha _2=\theta _3\theta _4,\alpha _3=\theta _4,\alpha _4=\frac{1}{2}(\theta _1\theta _2\theta _3\theta _4)\}}\\ & \\ \multicolumn{2}{c}{\mathrm{\Delta }^+=\{\theta _i:1i4\}\{\theta _i\pm \theta _j:1i<j4\}\{\frac{1}{2}(\theta _1\pm \theta _2\pm \theta _3\pm \theta _4)\}}\\ & \\ \lambda _1=\theta _1+\theta _2\hfill & \beta _1=\theta _1+\theta _2\hfill \\ \lambda _2=2\theta _1+\theta _2+\theta _3\hfill & \beta _2=\theta _1\theta _2\hfill \\ \lambda _3=\frac{1}{2}(3\theta _1+\theta _2+\theta _3+\theta _4)\hfill & \beta _3=\theta _3+\theta _4\hfill \\ \lambda _4=\theta _1\hfill & \beta _4=\theta _3\theta _4\hfill \end{array}$$

$$\begin{array}{cc}& \\ \multicolumn{2}{c}{𝐄_6:\text{}}\\ & \\ \multicolumn{2}{c}{𝒮=\{\alpha _1=\frac{1}{2}(\theta _1\theta _2\theta _3\theta _4\theta _5\theta _6\theta _7+\theta _8),}\\ \multicolumn{2}{c}{\alpha _2=\theta _1+\theta _2,\alpha _3=\theta _2\theta _1,\alpha _4=\theta _3\theta _2,\alpha _5=\theta _4\theta _3,\alpha _6=\theta _5\theta _4\}}\\ & \\ \multicolumn{2}{c}{\mathrm{\Delta }^+=\{\pm \theta _i+\theta _j:1i<j5\}\{\frac{1}{2}(\theta _8\theta _7\theta _6+_{i=1}^5ϵ_i\theta _i):\mathrm{\Pi }_{i=1}^5ϵ_i=+1\}}\\ & \\ \lambda _1=\frac{2}{3}(\theta _8\theta _7\theta _6)\hfill & \beta _1=\frac{1}{2}(\theta _1+\theta _2+\theta _3+\theta _4\hfill \\ \lambda _2=\frac{1}{2}(\theta _1+\theta _2+\theta _3+\theta _4+\theta _5\theta _6\theta _7+\theta _8)\hfill & +\theta _5\theta _6\theta _7+\theta _8)\hfill \\ \lambda _3=\frac{5}{6}(\theta _8\theta _7\theta _6)+\frac{1}{2}(\theta _1+\theta _2+\theta _3+\theta _4+\theta _5)\hfill & \beta _2=\frac{1}{2}(\theta _1\theta _2\theta _3\theta _4\hfill \\ \lambda _4=\theta _3+\theta _4+\theta _5\theta _6\theta _7+\theta _8\hfill & +\theta _5\theta _6\theta _7+\theta _8)\hfill \\ \lambda _5=\frac{2}{3}(\theta _8\theta _7\theta _6)+\theta _4+\theta _5\hfill & \beta _3=\theta _4\theta _1\hfill \\ \lambda _6=\frac{1}{3}(\theta _8\theta _7\theta _6)+\theta _5\hfill & \beta _4=\theta _3\theta _2\hfill \end{array}$$

$$\begin{array}{cc}& \\ \multicolumn{2}{c}{𝐄_7:\text{}}\\ & \\ \multicolumn{2}{c}{𝒮=\{\alpha _1=\frac{1}{2}(\theta _1\theta _2\theta _3\theta _4\theta _5\theta _6\theta _7+\theta _8),}\\ \multicolumn{2}{c}{\alpha _2=\theta _1+\theta _2,\alpha _3=\theta _2\theta _1,\alpha _4=\theta _3\theta _2,\alpha _5=\theta _4\theta _3,\alpha _6=\theta _5\theta _4,\alpha _7=\theta _6\theta _5\}}\\ & \\ \multicolumn{2}{c}{\mathrm{\Delta }^+=\{\pm \theta _i+\theta _j:1i<j6\}\{\theta _8\theta _7\}\{\frac{1}{2}(\theta _8\theta _7+_{i=1}^6ϵ_i\theta _i):\mathrm{\Pi }_{i=1}^6ϵ_i=1\}}\\ & \\ \lambda _1=\theta _8\theta _7\hfill & \beta _1=\theta _8\theta _7\hfill \\ \lambda _2=\frac{1}{2}(\theta _1+\theta _2+\theta _3+\theta _4+\theta _5+\theta _62\theta _7+2\theta _8)\hfill & \beta _2=\theta _6+\theta _5\hfill \\ \lambda _3=\frac{1}{2}(\theta _1+\theta _2+\theta _3+\theta _4+\theta _5+\theta _63\theta _7+3\theta _8)\hfill & \beta _3=\theta _6\theta _5\hfill \\ \lambda _4=\theta _3+\theta _4+\theta _5+\theta _6+2(\theta _8\theta _7)\hfill & \beta _4=\theta _4+\theta _3\hfill \\ \lambda _5=\frac{1}{2}(2\theta _4+2\theta _5+2\theta _6+3(\theta _8\theta _7))\hfill & \beta _5=\theta _4\theta _3\hfill \\ \lambda _6=\theta _5+\theta _6\theta _7+\theta _8\hfill & \beta _6=\theta _2+\theta _1\hfill \\ \lambda _7=\theta _6+\frac{1}{2}(\theta _8\theta _7)\hfill & \beta _7=\theta _2\theta _1\hfill \end{array}$$

$$\begin{array}{cc}& \\ \multicolumn{2}{c}{𝐄_8:\text{}}\\ & \\ \multicolumn{2}{c}{𝒮=\{\alpha _1=\frac{1}{2}(\theta _1\theta _2\theta _3\theta _4\theta _5\theta _6\theta _7+\theta _8),\alpha _2=\theta _1+\theta _2,}\\ \multicolumn{2}{c}{\alpha _3=\theta _2\theta _1,\alpha _4=\theta _3\theta _2,\alpha _5=\theta _4\theta _3,\alpha _6=\theta _5\theta _4,\alpha _7=\theta _6\theta _5,\alpha _8=\theta _7\theta _6\}}\\ & \\ \multicolumn{2}{c}{\mathrm{\Delta }^+=\{\pm \theta _i+\theta _j:1i<j8\}\{\frac{1}{2}(\theta _8+_{i=1}^7ϵ_i\theta _i):\mathrm{\Pi }_{i=1}^7ϵ_i=+1\}}\\ & \\ \lambda _1=2\theta _8\hfill & \beta _1=\theta _7+\theta _8\hfill \\ \lambda _2=\frac{1}{2}(\theta _1+\theta _2+\theta _3+\theta _4+\theta _5+\theta _6+\theta _7+5\theta _8)\hfill & \beta _2=\theta _8\theta _7\hfill \\ \lambda _3=\frac{1}{2}(\theta _1+\theta _2+\theta _3+\theta _4+\theta _5+\theta _6+\theta _7+7\theta _8)\hfill & \beta _3=\theta _6+\theta _5\hfill \\ \lambda _4=\theta _3+\theta _4+\theta _5+\theta _6+\theta _7+5\theta _8\hfill & \beta _4=\theta _6\theta _5\hfill \\ \lambda _5=\theta _4+\theta _5+\theta _6+\theta _7+4\theta _8\hfill & \beta _5=\theta _4+\theta _3\hfill \\ \lambda _6=\theta _5+\theta _6+\theta _7+3\theta _8\hfill & \beta _6=\theta _4\theta _3\hfill \\ \lambda _7=\theta _6+\theta _7+2\theta _8\hfill & \beta _7=\theta _2+\theta _1\hfill \\ \lambda _8=\theta _7+\theta _8\hfill & \beta _8=\theta _2\theta _1\hfill \end{array}$$

## Appendix B Table for Dadok’s invariant $`k(\lambda )`$

We next tabulate the value of the Dadok invariant $`k(\lambda )`$ for the the complex irreducible representations $`\pi _\lambda `$ of the complex simple Lie algebras, where $`\lambda `$ is a fundamental highest weight (since $`k(\lambda )`$ is linear on $`\lambda `$, this already determines the value of $`k(\lambda )`$ for all $`\lambda `$). The number in parenthesis next to the vertex of the Dynkin diagram which specifies $`\alpha _i`$ is $`k(\lambda _i)`$. The last column tells, for each complex simple Lie algebra, whether all representations are self-dual, or else, the dual of $`\pi _{\lambda _i}`$ is given by $`\pi _{\lambda _j}`$, where the vertices that specify $`\alpha _i`$ and $`\alpha _j`$ are images one of the other under the symmetry of the Dynkin diagram.

$$\begin{array}{ccc}& & \\ G& k(\lambda )& Conditions\\ & & \\ 𝐀_{2n1}& \text{}& \text{duality given by symmetry of diagram}\\ 𝐀_{2n}& \text{}& \text{duality given by symmetry of diagram}\\ 𝐁_{2n1}& \text{}& \text{all self-dual}\\ 𝐁_{2n}& \text{}& \text{all self-dual}\\ 𝐂_n& \text{}& \text{all self-dual}\\ 𝐃_{2n}& \text{}& \text{all self-dual}\\ 𝐃_{2n+1}& \text{}& \text{duality given by symmetry of diagram}\\ 𝐆_2& \text{}& \text{all self-dual}\\ 𝐅_4& \text{}& \text{all self-dual}\\ 𝐄_6& \text{}& \text{duality given by symmetry of diagram}\\ 𝐄_7& \text{}& \text{all self-dual}\\ 𝐄_8& \text{}& \text{all self-dual}\end{array}$$

Table B.1: The invariant of Dadok.

## Appendix C Table of candidates to class $`𝒪^2`$

In the tables below are compiled all the real irreducible representations of compact connected Lie groups that are not orbit equivalent to the isotropy representation of a symmetric space but that have a chance of being a representation of class $`𝒪^2`$. These are just the representations mentioned in Propositions 5.1, 5.2, 5.17, 5.18, 5.19, 5.20 and 5.21, except those excluded by the discussion in Section 6; note however that the presentation here slightly differs from the presentation in the above mentioned propositions for the sake of convenience.

$$\begin{array}{ccc}& & \\ G& \rho & Conditions\\ & & \\ \mathrm{𝐒𝐎}(2)\times \mathrm{𝐒𝐩𝐢𝐧}(9)& (xx^1)\text{}& \\ 𝐔(2)\times \mathrm{𝐒𝐩}(n)& (xx^1)\text{}\text{}& n2\\ \mathrm{𝐒𝐔}(2)\times \mathrm{𝐒𝐩}(n)& \text{}\text{}& n2\end{array}$$

Table C.1: Taut representations.

$$\begin{array}{ccc}& & \\ G& \rho & Conditions\\ & & \\ \mathrm{𝐒𝐔}(n)\times \mathrm{𝐒𝐩}(m)& \begin{array}{c}(\text{}\text{})\\ \text{}\end{array}& n3,m2\\ 𝐔(n)\times \mathrm{𝐒𝐩}(m)& \begin{array}{c}(x\text{}x^1\text{})\\ \text{}\end{array}& n3,m2\\ & & \\ \begin{array}{c}\mathrm{𝐒𝐎}(m)\times \mathrm{𝐒𝐩}(1)\mathrm{𝐒𝐩}(n)\\ (n2)\end{array}& \begin{array}{c}\text{}\text{}\text{}\\ \text{}\text{}\text{}\\ \text{}\text{}\text{}\text{}\\ \text{}\text{}\text{}\end{array}& \begin{array}{c}m=2p>4\\ m=2p+1>3\\ m=4\\ m=3\end{array}\\ & & \\ \mathrm{𝐒𝐎}(m)\times 𝐆_2& \begin{array}{c}\text{}\text{}\\ \text{}\text{}\\ \text{}\text{}\text{}\\ \text{}\text{}\end{array}& \begin{array}{c}m=2p>4\\ m=2p+1>3\\ m=4\\ m=3\end{array}\\ & & \\ \mathrm{𝐒𝐎}(m)\times \mathrm{𝐒𝐩𝐢𝐧}(7)& \begin{array}{c}\text{}\text{}\\ \text{}\text{}\\ \text{}\text{}\text{}\end{array}& \begin{array}{c}m=2p>4\\ m=2p+15\\ m=4\end{array}\\ & & \\ \mathrm{𝐒𝐎}(m)\times \mathrm{𝐒𝐩𝐢𝐧}(9)& \begin{array}{c}\text{}\text{}\\ \text{}\text{}\\ \text{}\text{}\text{}\\ \text{}\text{}\end{array}& \begin{array}{c}m=2p>4\\ m=2p+1>3\\ m=4\\ m=3\end{array}\end{array}$$

Table C.2: Nontaut representations (first group).

$$\begin{array}{ccc}& & \\ G& \rho & Conditions\\ & & \\ 𝐆_2\times 𝐆_2& \text{}\text{}& \\ 𝐆_2\times \mathrm{𝐒𝐩𝐢𝐧}(7)& \text{}\text{}& \\ 𝐆_2\times \mathrm{𝐒𝐩𝐢𝐧}(9)& \text{}\text{}& \\ \mathrm{𝐒𝐩𝐢𝐧}(7)\times \mathrm{𝐒𝐩𝐢𝐧}(7)& \text{}\text{}& \\ \mathrm{𝐒𝐩𝐢𝐧}(7)\times \mathrm{𝐒𝐩𝐢𝐧}(9)& \text{}\text{}& \\ \mathrm{𝐒𝐩}(1)\mathrm{𝐒𝐩}(n)\times 𝐆_2& \text{}\text{}\text{}& n2\\ \mathrm{𝐒𝐩}(1)\mathrm{𝐒𝐩}(n)\times \mathrm{𝐒𝐩𝐢𝐧}(7)& \text{}\text{}\text{}& n2\end{array}$$

Table C.3: Nontaut representations (second group).

$$\begin{array}{ccc}& & \\ G& \rho & Conditions\\ & & \\ \mathrm{𝐒𝐩}(n)\times \mathrm{𝐒𝐔}(6)& \text{}\text{}& n2\\ \mathrm{𝐒𝐩}(n)\times \mathrm{𝐒𝐩𝐢𝐧}(11)& \text{}\text{}& \\ \mathrm{𝐒𝐩}(n)\times \mathrm{𝐒𝐩𝐢𝐧}(12)& \text{}\text{}& n2\\ \mathrm{𝐒𝐩}(1)\times \mathrm{𝐒𝐩𝐢𝐧}(13)& \text{}\text{}& \\ \mathrm{𝐒𝐩}(1)\times \mathrm{𝐒𝐩}(2)& \text{}\text{}& \\ \mathrm{𝐒𝐩}(n)\times \mathrm{𝐒𝐩}(3)& \text{}\text{}& n2\\ \mathrm{𝐒𝐩}(n)\times 𝐄_7& \text{}\text{}& n2\\ \mathrm{𝐒𝐩𝐢𝐧}(7)& \text{}& \\ \mathrm{𝐒𝐩𝐢𝐧}(9)& \text{}& \\ \mathrm{𝐒𝐩𝐢𝐧}(15)& \text{}& \\ \mathrm{𝐒𝐩𝐢𝐧}(17)& \text{}& \end{array}$$

Table C.4: Nontaut representations (third group).