text:
# Infrared conductivity of metallic (III,Mn)V ferromagnets

\[

## Abstract

We present a theory of the infrared conductivity and absorption coefficients of metallic (III,Mn)V ferromagnetic semiconductors. We find that the conductivity is dominated by inter-valence-band transitions that produce peaks at $`\mathrm{}\omega 220\mathrm{m}\mathrm{e}\mathrm{V}`$ and obscure the broadened Drude peak. We demonstrate that transverse f-sum rule measurements can be used to extract accurate values for the free carrier density, bypassing the severe characterization difficulties that have till now been created by the large anomalous Hall effect in these materials.

\]

Following the discovery of carrier-induced ferromagnetism in (In,Mn)As  and (Ga,Mn)As  in the early nineties, studies of ferromagnetism in (III,Mn)V diluted magnetic semiconductors (DMS’s) have yielded many surprises. The transport and optical properties of these heavily-doped semiconductors are richer than those of conventional itinerant electron ferromagnets because of strong valence-band spin-orbit coupling, and because of the sensitivity of their magnetic state to growth conditions, doping, and external fields. These novel ferromagnets are likely to have a major technological impact  if systems with Curie temperatures above room temperature can be created and better control of disorder effects can be achieved. We present a theory of the infrared conductivity of (III,Mn)V ferromagnets which we believe will increase the value of this powerful probe in characterizing present and future DMS ferromagnets. In particular, we demonstrate that infrared conductivity measurements in metallic samples can be used to obtain accurate measurements of the total free carrier density, likely the key to understanding the now firmly established dependence of magnetic properties on growth and post-growth annealing conditions. This experimental possibility is especially important in these semiconductors because they are ferromagnetic and have a large anomalous Hall conductivity  which severely complicates Hall-effect carrier-density measurements,  even when they are performed in very strong magnetic fields.

Our conclusions are based on a semi-phenomenological model  in which host semiconductor valence band electrons, described by a $`𝐤.𝐩`$ model, interact with randomly distributed aligned ($`T=0`$) Mn local- moment acceptors via Coulomb and short-range exchange interactions, and via Coulomb interactions only with the compensating charged defects known to be present. Some of our considerations are based on standard linear-response theory expressions for the ac conductivity of weakly disordered metals, in which disorder is included only through finite quasiparticle lifetimes and localization effects are ignored. In the dc limit, with quasiparticle scattering rates evaluated using Born approximation expressions for scattering off the spatial fluctuations of the screened-Coulomb and exchange potentials, these expressions imply mobilities  that are consistent with values measured in optimally annealed, high-$`T_c`$, DMS ferromagnets. Although the predictions discussed below are intended to be most reliable for the most metallic systems, they appear to explain much currently available infrared optical data, most of which has been obtained in studies of systems with relatively low dc conductivities.

The main features observed in experiments  are the absence of a clear low-frequency Drude peak, and a broad absorption peak near $`220`$ meV that becomes stronger as the samples are cooled. The $`220`$ meV peak has been attributed either to inter-valence-band transitions, to transitions between the semiconductor valence band states and a Mn induced impurity band, or to a combination of these contributions. Dynamical mean-field-theory studies , for a single-band model that neglects the spin-orbit coupling and the heavy-light degeneracy of a III-V semiconductor valence band, demonstrate that non-Drude impurity-band related features can occur in DMS ferromagnets. The conductivities predicted by this model are, however, inconsistent with experiment in magnitude and in temperature trend. As demonstrated in the following paragraphs, realistic features of the valence band electronic structure are key to understanding the conductivity data. We find that the absorption peak near 220 meV is associated with heavy-hole to light-hole inter-valence-band transitions, and that transitions to the bands split-off by spin-orbit interactions are responsible for the gradual increase in absorption observed in experiments  at frequencies above $`500`$ meV.

The physical picture we have developed for the infrared conductivity starts from an examination of the limit in which disorder is ignored. In building a picture starting from this limit we are motivated in part by the observation that many magnetic and transport property trends (critical temperatures, magnetic anisotropy and its strain dependence, magnetic stiffness, anomalous Hall effect, etc. ) in DMS ferromagnets appear to be adequately described without accounting for disorder, even though it is relatively strong by some measures. The linear response of the system to a light probe is characterized by the optical conductivity tensor $`\sigma _{\alpha ,\beta }(\omega )`$. In thin film absorption measurements, the real part of the conductivity is related to the absorption coefficient by

$$\stackrel{~}{\alpha }(\omega )=2\frac{\mathrm{Re}[\sigma (\omega )]}{Y+Y_0},$$
(1)
where $`Y`$ and $`Y_0`$ are the admittances of the substrate and free space, respectively . We have evaluated $`\sigma _{\alpha ,\beta }(\omega )`$ at $`T=0`$ using the standard Kubo formula for non-interacting quasi-particles. The clean-limit ac conductivity of our model is plotted in Fig. 1 for a typical set of (Ga,Mn)As parameter values, carrier density $`p=0.3`$ nm-3, 5% Mn substitution, implying a Mn concentration $`N_{Mn}=1.1`$ nm-3, and band parameters that incorporate the strain produced by epitaxial growth on a GaAs substrate . When disorder is neglected, Coulomb interactions have no effect, while the exchange interactions produce a uniform exchange mean Zeeman field, $`\stackrel{}{h}=J_{pd}N_{Mn}S\widehat{\mathrm{\Omega }}`$, where $`\widehat{\mathrm{\Omega }}`$ is the magnetization direction. The finite-frequency conductivity results from transitions between occupied and unoccupied valence bands states; the band Kramers degeneracies are split by an amount comparable to the Fermi energy. ($`J_{pd}=55`$ meV nm3 is the exchange-coupling constant so that $`h150\mathrm{m}\mathrm{e}\mathrm{V}`$.) The results illustrated in Fig. 1 are for $`\widehat{\mathrm{\Omega }}`$ along the growth ($`001`$) direction, the magnetic easy-axis for a tensile strained (Ga,Mn)As layers ; clean limit results for samples under compressive strain (GaAs substrates) which have in plane easy axes differ in detail . The two split-peaks present in the clean limit at $`250`$ and $`325`$ meV correspond to heavy-hole to light-hole transitions and to light-hole to split off (s-o) band transitions, respectively. Beyond $`500`$ meV there is a slight rise in the absorption which corresponds to the heavy-hole to s-o split off band transition. In the clean limit, the conductivity has a $`\delta `$-function contribution at $`\omega =0`$ due to intraband transitions. Its relative contribution to the f-sum rule of our model, discussed further below, is 35%, nearly independent of carrier density over the relevant range.

We account for disorder by including the lifetime broadening of quasiparticle spectral functions in evaluating the Kubo formula. The effective lifetime for transitions between bands $`n`$ and $`n^{}`$, $`\tau _{n,n^{}}`$, is calculated by averaging quasiparticle scattering rates calculated from Fermi’s golden rule including both screened Coulomb and exchange interactions . The solid curve in Fig. 1 illustrates the influence of disorder on the conductivity. The intraband conductivity evolves into a broadened Drude peak that overlaps with broadened interband absorption features. In Fig. 2, we plot ac conductivities calculated in this way for $`\mathrm{Ga}_{.95}\mathrm{Mn}_{.05}`$As at a series of carrier densities. In our model, an increase in carrier density is accompanied by a decrease in the density $`N_c`$ of compensating charged defects; we assume that all have charge $`Z=2`$ so that $`p=N_{Mn}2N_c`$ . As $`p`$ increases, $`N_c`$ and quasiparticle scattering rates decrease in tandem. The evaluation of quasiparticle scattering rates allows us to check the consistency of our weak-scattering approach, which requires $`\mathrm{}/(\tau _{nn^{}}E_F)>1`$, where $`E_F`$ is the hole Fermi energy. For 5% Mn doping, the condition is safely satisfied only for $`p0.3\mathrm{nm}^3`$. As the carrier density increases above this value we see in Fig. 2 that the peak near 220 meV sharpens and the low frequency Drude peak emerges more clearly.

Since all model parameters are accurately known from other experiments, the ac conductivity values in Fig. 2 are parameter free. Although direct comparisons with infrared conductivity experiments is difficult because the carrier density is not usually known, our dc conductivities are generally larger than those of optimally annealed samples . Our quasiparticle scattering rates are likely smaller than those in any current experimental sample, due to some combination of inaccuracy in our approximate scattering amplitudes and unknown sources of additional unintended disorder. Further progress in sorting out variable transport and magnetic properties requires accurate experimentally determined carrier density values, which we believe can be obtained directly from infrared conductivity data as we explain below.

In the following paragraphs we discuss the f-sum rule of our model:

$$F_0^{\mathrm{}}𝑑\omega \mathrm{Re}[\sigma _{xx}(\omega )]=\frac{\pi e^2}{2V}\underset{\alpha }{}f_\alpha \alpha |\frac{^2H_{band}}{\mathrm{}^2k_x^2}|\alpha .$$
(2)
In this equation $`f_\alpha `$ is a quasiparticle Fermi factor and $`^2H_{band}/\mathrm{}^2k_x^2`$ is the $`xx`$ component of the $`𝐤.𝐩`$ model inverse effective mass operator, which is diagonal in envelope function wavevector but off diagonal in envelope function spinor index. We emphasize that this sum rule is completely independent of the weak-scattering approximations on which the ac conductivity calculations discussed above were based. In practical applications, it is necessary to choose an upper cut-off for the frequency integral . We define a weakly cut-off dependent optical effective mass for DMS ferromagnets by defining $`F=\pi e^2p/2m_{opt}`$. As illustrated in the inset of Fig 2, for the case of cut-off frequency $`\mathrm{}\omega _{max}=800\mathrm{m}\mathrm{e}\mathrm{V}`$, the f-sum rule values of $`F`$ evaluated from our weak scattering theory are accurately linear in $`p`$ over the entire range of relevant carrier densities. The optical masses evaluated here, which provide the f-sum rule with practical utility, represent a complicated average over heavy and light hole masses in the spin-split ferromagnetic valence bands. Disorder does have a small but measurable effect on $`m_{opt}`$ as illustrated in Fig. 2. By solving self-consistent mean-field calculations for finite-size realizations of our DMS ferromagnet model, we have established that a fully non-perturbative treatment of disorder alters $`m_{opt}`$, by less than 10% for metallic DMS ferromagnets. We have also verified that the optical mass is not perceptibly dependent on the magnetization orientation and that it is altered by less than 1% by the exchange mean-field. Using the six-band Luttinger model, the optical masses for GaAs, InAs, and GaSb DMS ferromagnets with a 800, 400, and 700 meV cutoffs are 0.25-0.29 me, 0.40-0.43 me, and 0.21-0.23 me respectively, the extremes of these ranges corresponding to the clean (lower) and disordered (upper) limits of our model . $`m_{opt}`$ depends to a good approximation only on the frequency cutoff and on the Luttinger parameters that characterize the host semiconductor valence bands , permitting accurate carrier-density measurements based on the f-sum rule.

We have applied our procedure for determining the DMS carrier density to a sample with 4% Mn doping, measured by Katsumoto et al , obtaining a value of 0.17 $`\pm .03`$ nm-3 (Their data is shown as the dot-dashed line in Fig. 2). Although their sample is strongly disordered, the f-sum rule based density estimate still appears reasonable. It is consistent, within 30%, with the  carrier density that would produce a ferromagnetic state with the measured  critical temperature of 90 K. Note that the hole density estimate  based on the measured room temperature Hall coefficient for the same sample is $`5\times `$ smaller than that expected on the basis of its critical temperature, emphasizing once again the difficulty of extracting carrier densities from Hall data.

The strong spin-orbit coupling in the (Ga,Mn)As valence band ($`\mathrm{\Delta }_{so}=341`$ meV) compared to the typical spectral broadening of the hole quasiparticles ($`\mathrm{}/\tau 100200`$ meV) leads to a dependence of the absorption spectrum on the direction of the magnetization. This dependence should lead to measurable infrared conductivity changes that can be induced by an external magnetic field larger than the sample’s magnetocrystalline anisotropy field $`100`$ mT , applied along the hard $`100`$ magnetic axis. The inset of Fig. 3 shows the orientation dependence of the optical conductivity and absorption coefficients for the same sample parameters as in Fig. 1, calculated neglecting disorder and with the incident light polarization along the $`100`$ direction. In the main plot of Fig. 3 we compare absorption spectra evaluated using our weak disorder approximation. The sharp spectral features near 325 meV are smeared out by disorder, however, the $``$15 meV relative shift of the absorption peak for the two magnetization orientations is nearly disorder independent and should be observable experimentally. This effect is stronger when the density of compensating defects is reduced.

In summary, we find that the infrared conductivity of metallic DMS ferromagnets has Drude and interband contributions that have approximately equal importance. In current samples these contributions overlap in frequency, but we expect that they will begin to separate if recent improvements in sample conductivity can be extended. We predict that the infrared conductivity will be very sensitive to changes in compensating defect and carrier densities, believed to be responsible for the sensitivity of DMS ferromagnets to growth and annealing conditions. In particular, we have evaluated optical effective masses that will allow carrier densities to be extracted from f-sum rule measurements, circumventing difficulties posed by the large anomalous Hall effect of (III,Mn)V ferromagnetic semiconductors. Simultaneous measurement of optical, magnetic, and dc transport properties should prove to be potent in advancing the materials science of these interesting systems, when our optical masses are used to extract carrier densities. We have also demonstrated the sensitivity of the infrared optical conductivity to magnetization orientation that is due to the strong spin-orbit coupling in these systems.

We acknowledge helpful discussions with Jacek Furdyna, Bruce McCombe, Andrea Markelz, Joaquin Fernandez-Rossier, Peter Schiffer, and Jason Singley. The work was supported by the Welch Foundation, DARPA, the Grant Agency of the Czech Republic under grant 202/02/0912, the Ministry of Education of the Czech Republic under grant OC P5.10, and KOSEF Quantum-functional Semiconductor Research Center at Dongguk University.


format: text

text:
# 1 Introduction

## 1 Introduction

Dynamical Lefschetz trace formulas for flows on compact manifolds were first conjectured by Guillemin \[G\] and later but independently by Patterson \[P\]. These formulas express the alternating sum of the traces of the induced flow on cohomology by a sum over contributions from the compact orbits.

Here the cohomology theory depends on the choice of a foliation which is respected by the flow.

This note makes a contribution to Patterson’s setup. He looks at Anosov flows and considers the unstable foliation. An interesting example of this situation is provided by the geodesic flow on the sphere bundle of a cocompact quotient of $`\mathrm{PSL}_2()`$ by a lattice. Using representation theory, Patterson was able to define a trace on the infinite dimensional foliation cohomologies. The dynamical Lefschetz trace formula was then shown to be a consequence of the Selberg trace formula. Incidentally this example also fits into Guillemin’s framework who treated it in \[G\].

Much more work in the context of more general symmetric spaces was done by Bunke, Deitmar, Juhl, Olbrich and Schubert. We refer the reader to the book of Juhl \[J\] for a comprehensive overview.

Among all Anosov flows there are the ones with an integrable complement or equivalently, those where the sum of the stable and the unstable bundle is integrable. One gets examples by suspending Anosov diffeomorphisms and conjecturally all Anosov flows with an integrable complement should arise in this way up to a time change by a constant factor, \[Pl\] § 3.

Granting this conjecture the dynamical Lefschetz trace formula for such Anosov flows can be reduced to the following problem:

Let $`f`$ be an Anosov diffeomorphism of a compact manifold $`X`$ whose unstable foliation $`^u`$ is smooth. Establish a dynamical Lefschetz trace formula of the following form:

(1)
$$\underset{p}{}(1)^p\mathrm{Tr}(f^{}|H_^u^p(X))=\underset{\genfrac{}{}{0pt}{}{x}{f(x)=x}}{}\epsilon _x|\mathrm{det}(1T_xf|T_x^s)|^1,$$

where

$$\epsilon _x=\mathrm{sgn}det(1T_xf|T_x^u).$$

Here for a fixed point $`x,T_x^s`$ resp. $`T_x^u`$ is the subspace of $`T_xX`$ corresponding to the eigenvalues of $`T_xf`$ of absolute value smaller resp. greater than one.

Note that since $`f`$ is supposed to be Anosov, in particular the eigenvalue $`\lambda =1`$ does not occur on $`T_xX`$. Hence all determinants are non-zero. The definition of $`H_^u^p(X)`$ can be found in § 2 below.

Formula (1) is a dynamical Lefschetz trace formula for the discrete dynamical system determined by $`f`$.

The analytic difficulties regarding dynamical Lefschetz trace formulas for flows persist in formula (1) for diffeomorphisms. The main problem being that the complex defining $`H_^u^{}(X)`$ is not elliptic but only elliptic along the unstable leaves. In particular we cannot appeal to the work of Atiyah and Bott on Lefschetz trace formulas for endomorphisms of elliptic complexes.

According to \[M\] § 4 the following classification theorem for Anosov diffeomorphisms is expected. After lifting to a finite covering any Anosov diffeomorphism should become topologically conjugate to an algebraic Anosov diffeomorphism $`f`$. The latter come about as follows. Let $`G`$ be a simply connected real nilpotent Lie group with a cocompact lattice $`\mathrm{\Gamma }`$ and consider an automorphism $`f`$ of $`G`$ with $`f(\mathrm{\Gamma })=\mathrm{\Gamma }`$. Assume that the induced Lie algebra automorphism $`f_{}`$ of $`𝔤`$ does not have any eigenvalue of absolute value one. Then the induced diffeomorphism $`f`$ of the compact nilmanifold $`X=\mathrm{\Gamma }G`$ is Anosov, and the Anosov diffeomorphisms obtained in this way are called “algebraic”.

If we only demand that $`f`$ is an endomorphism of $`G`$ with $`f(\mathrm{\Gamma })\mathrm{\Gamma }`$ and such that $`f_{}`$ does not have $`1`$ as an eigenvalue then the induced map $`f`$ of $`X`$ will be called a generalized algebraic Anosov map.

In the present note we first prove formula (1) for algebraic Anosov diffeomorphisms. In this case by a subtle argument we can calculate the reduced leafwise cohomology i.e. the maximal Hausdorff quotient of $`H_^u^{}(X)`$. It turns out to be finite dimensional so that the definition of the trace of $`f^{}`$ does not present a problem.

Next, using basic aspects of the representation theory of $`𝔤`$, we prove formula (1) also for generalized algebraic Anosov maps. In this case, even the reduced leafwise cohomology is infinite dimensional in general. Our definition of the trace of $`f^{}`$ then depends in an essential way on representation theory. This seems to be the only method to define these traces. Namely the simplest non-trivial case of a generalized Anosov map $`f`$ which is not Anosov is multiplication by $`1`$ on $`/`$. We are not aware of any purely functional-analytic definition of a trace which applies to the map $`f^{}`$ on $`C^{\mathrm{}}(/)`$.

We would like to thank the referees whose comments have led to some improvements of the paper.

## 2 Background on leafwise cohomology

For a smooth foliation $``$ on a closed smooth manifold $`X`$ consider the de Rham complex of smooth real valued forms along the leaves:

(2)
$$(\mathrm{\Gamma }(X,\mathrm{\Lambda }^{}T^{}),d_{}).$$

It is a complex of nuclear Fréchet spaces.

Here the exterior differential $`d_{}`$ is defined by the same formula as the ordinary differential $`d`$. Integrability of $`T`$ translates into the condition that $`d_{}^2=0`$.

The cohomology groups

(3)
$$H_{}^{}(X):=H^{}(\mathrm{\Gamma }(X,\mathrm{\Lambda }^{}T^{}),d_{})$$

are called the leafwise cohomology groups of the foliated manifold $`X`$. If $``$ is the sheaf of smooth real valued functions on $`X`$ which are locally constant on the leaves, we have

(4)
$$H_{}^{}(X)=H^{}(X,).$$

The reduced leafwise cohomology is the maximal Hausdorff-quotient $`\overline{H}_{}^{}(X)`$ of $`H_{}^{}(X)`$. It is the nuclear Fréchet space obtained by dividing $`\mathrm{Ker}d_{}`$ by the closure of $`\mathrm{Im}d_{}`$. In general even the reduced cohomology is infinite dimensional, compare example 5.5 for example.

Given another foliated manifold $`X^{},^{}`$, any map $`f:XX^{}`$ which maps leaves of $``$ into leaves of $`^{}`$ induces a continuous pullback map on cohomology

$$f^{}:H_{^{}}^{}(X^{})H_{}^{}(X).$$

These maps pass to the reduced versions of leafwise cohomology.

Consider now a smooth compact manifold $`X=\mathrm{\Gamma }G`$ obtained as the quotient of a connected real Lie group $`G`$ by a cocompact lattice $`\mathrm{\Gamma }G`$. For an immersed subgroup $`PG`$ with Lie algebra $`𝔭𝔤`$ consider the foliation $``$ of $`X`$ with leaves $`\mathrm{\Gamma }gP`$ for $`gG`$. Then we have a topological isomorphism with Lie algebra cohomology

(5)
$$H_{}^{}(X)\stackrel{}{}H^{}(𝔭,C^{\mathrm{}}(X)).$$

Indeed let $`L=𝔭C^{\mathrm{}}(X)`$ be the Lie algebra of vector fields tangent to the leaves. Then we have:

$$\mathrm{\Gamma }(X,\mathrm{\Lambda }^{}T^{})=\mathrm{Hom}_{C^{\mathrm{}}(X)}(\mathrm{\Lambda }_{C^{\mathrm{}}(X)}^{}(L),C^{\mathrm{}}(X)).$$

This is an isomorphisms of complexes if the right hand side is equipped with the differential (3A.5) in \[MS\] Ch. III, Appendix. Formula (5) follows.

We now decompose the reduced leafwise cohomology using representation theory. The existence of the cocompact lattice $`\mathrm{\Gamma }`$ implies that Haar measure on $`G`$ is unimodular. Let $`\mu `$ denote the unique $`G`$-invariant measure on $`X`$ of volume one. Let $`R`$ be the “regular” unitary representation of $`G`$ on the complex Hilbert space $`L^2(X)=L^2(X,\mu )`$ by right translation:

$$(R_gf)(\mathrm{\Gamma }g^{})=f(\mathrm{\Gamma }g^{}g).$$

Since $`X`$ is compact, the representation $`R`$ decomposes into isotypic components $`H(\pi )`$ of finite multiplicity:

(6)
$$L^2(X)=\widehat{\underset{\pi \widehat{G}}{}}H(\pi ).$$

Here $`\widehat{G}`$ is the unitary dual of $`G`$, i.e. the set of equivalence classes of irreducible unitary representations of $`G`$. Each isotypic component is isomorphic to $`M_\pi V_\pi `$ where $`(\rho _\pi ,V_\pi )`$ is a fixed representation in the class $`\pi `$ and where $`M_\pi `$ is the finite dimensional multiplicity space, i.e.

$$M_\pi =\mathrm{Hom}_G(V_\pi ,L^2(X)).$$

We can now analyse the reduced leafwise cohomology using representation theory. Denote by $`\overline{H}^p(𝔭,H(\pi )^{\mathrm{}})`$ the maximal Hausdorff quotient of
$`H^p(𝔭,H(\pi )^{\mathrm{}})`$, where $`H(\pi )^{\mathrm{}}`$ denotes the space of smooth vectors in $`H(\pi )`$.

###### Lemma 2.1

The natural maps

$$\overline{H}^p(𝔭,H(\pi )^{\mathrm{}})\overline{H}_{}^p(X,)=\overline{H}_{}^p(X)$$

are split injections and hence have closed image. In this way the algebraic direct sum

$$\underset{\pi }{}\overline{H}^p(𝔭,H(\pi )^{\mathrm{}})$$

becomes a dense subspace of

$$\overline{H}_{}^p(X,)=\overline{H}_{}^p(X).$$

Every element of $`\overline{H}_{}^p(X,)`$ can be written as an unconditionally convergent series $`_\pi h_\pi `$ with $`h_\pi \overline{H}^p(𝔭,H(\pi )^{\mathrm{}})`$.

The proof is given below. It is based on an auxiliary result which requires the following notions:

###### Definition 2.2

A sequence $`(E_i)_{i1}`$ of non-trivial closed linear subspaces of a Fréchet space $`E`$ is called a Schauder basis of subspaces of $`E`$ if the following conditions hold:
a) Every element $`vE`$ can be written uniquely as a convergent series $`v=_{i=1}^{\mathrm{}}v_i`$ with $`v_iE_i`$.
b) The projections $`EE_i,vv_i`$ resulting from a) are continuous.
If the series in a) converge unconditionally the basis $`(E_i)`$ is called unconditional.

###### Sublemma 2.3

Let $`C^{}`$ be a complex of Fréchet spaces with continuous differentials. Consider closed subcomplexes $`C_i^{}C^{}`$ for $`i1`$ such that for every $`p`$ the sequence $`(C_i^p)_{i1}`$ is an unconditional Schauder basis of subspaces of $`C^p`$. Then the natural maps

$$\overline{H}^p(C_i^{})\overline{H}^p(C^{})$$

are split injective and hence have closed image. They induce an inclusion with dense image

$$\underset{i}{}\overline{H}^p(C_i^{})\overline{H}^p(C^{}).$$

Every element of $`\overline{H}^p(C^{})`$ can be written as an unconditionally convergent series $`_ih_i`$ with $`h_i\overline{H}^p(C_i^{})`$.

Proof For any $`p`$ the assumptions give continuous projections

$$\beta _i:C^{p1}C_i^{p1}$$

such that for all $`c`$ in $`C^{p1}`$ we have

$$c=\underset{i}{}\beta _i(c).$$

Since the differential is continuous we get

$$dc=\underset{i}{}d\beta _i(c).$$

On the other hand we have

$$dc=\underset{i}{}\beta _i(dc).$$

The representation of an element in $`C^p`$ as a convergent series of elements in the $`C_i^p`$ is unique. Hence we find

$$d\beta _i(c)=\beta _i(dc)$$

for all $`c`$ i.e. $`d\beta _i=\beta _id`$. From this and the continuity of $`\beta _i`$ it follows that

(7)
$$\beta _i(\overline{dC^{p1}})\overline{\beta _i(dC^{p1})}=\overline{d\beta _i(C^{p1})}=\overline{dC_i^{p1}}.$$

The inclusion $`\kappa _i:C_i^{}C^{}`$ of complexes induces a continuous map on the $`p`$-th cohomology and hence a continuous map

$$\overline{\kappa }_i:\overline{H}^p(C_i^{})\overline{H}^p(C^{}).$$

Similarly the continuous surjective map of complexes

$$\beta _i:C^{}C_i^{}$$

leads to a continuous map

$$\overline{\beta }_i:\overline{H}^p(C^{})\overline{H}^p(C_i^{}).$$

We have $`\beta _i\kappa _i=\mathrm{id}`$ and hence $`\overline{\beta }_i\overline{\kappa }_i=\mathrm{id}`$. Therefore $`\overline{\kappa }_i`$ is a split injection. Its image is closed since it is also the kernel of the continuous projector $`\pi _i=\mathrm{id}\overline{\kappa }_i\overline{\beta }_i`$ on $`\overline{H}^p(C^{})`$.

We now show that the induced map

$$\underset{i}{}\overline{H}^p(C_i^{})\overline{H}^p(C^{})$$

is injective. So assume that the element $`_i[c_i]`$ goes to zero in $`\overline{H}^p(C^{})`$. This means that $`_ic_i`$ lies in $`\overline{dC^{p1}}`$. Here $`c_iC_i^p,dc_i=0`$ and almost all $`c_i`$ are zero. Applying $`\beta _i`$ we see that $`c_i`$ lies in $`\beta _i(\overline{dC^{p1}})`$ and by (7) therefore in $`\overline{dC_i^{p1}}`$. Thus $`[c_i]=0`$ for all $`i`$.

Next consider any element $`[c]`$ in $`\overline{H}^p(C^{})`$. We may write $`c`$ as an unconditionally convergent series

$$c=\underset{i}{}c_i\text{with}c_i=\beta _i(c)C_i^p.$$

We have $`0=\beta _i(dc)=d\beta _i(c)=dc_i`$. Hence $`[c_i]\overline{H}^p(C^{})`$ lies in the image of $`\overline{H}^p(C_i^{})`$ and we have

$$[c]=\underset{i}{}[c_i]$$

in $`\overline{H}^p(C^{})`$ the series being unconditionally convergent. In particular the algebraic direct sum of the spaces $`\overline{H}^p(C_i^{})`$ is dense in $`\overline{H}^p(C^{})`$.  $`\mathrm{}`$

Proof of 2.1 Using the Sobolev embedding theorem one sees that the non-zero subspaces $`H(\pi )^{\mathrm{}}C^{\mathrm{}}(X)=L^2(X)^{\mathrm{}}`$ form an unconditional Schauder basis of subspaces of $`C^{\mathrm{}}(X)`$. It follows that the subcomplexes $`C^{}(𝔤,H(\pi )^{\mathrm{}})`$ of $`C^{}(𝔤,C^{\mathrm{}}(X))`$ satisfy the assumptions of the sublemma. This implies lemma 2.1.  $`\mathrm{}`$

## 3 $`\mathrm{\Gamma }`$-acceptable subgroups and leafwise cohomology

In this section we first prove that the reduced leafwise cohomology of certain homogenous foliations on nilmanifolds is isomorphic to finite dimensional Lie algebra cohomology. This result applies to the stable and unstable foliations of an algebraic Anosov diffeomorphism. It also applies to the foliation defined by a normal subgroup where it gives a result of Álvarez López and Hector.

Let $`G`$ be a simply connected nilpotent Lie group with a cocompact lattice $`\mathrm{\Gamma }G`$. Let

(8)
$$G=C_0C_1\mathrm{}C_kC_{k+1}=1$$

be the descending central series of $`G`$ defined by $`C_{j+1}=[G,C_j]`$ and $`C_0=G`$. From the definition we see:

###### 3.1

The group $`C_j/C_{j+1}`$ is central in $`G/C_{j+1}`$ for every $`0jk`$. In particular $`C_k`$ is central in $`G`$.

We will need the following basic results of Malcev \[Ma\]:

###### 3.2

The group $`\mathrm{\Gamma }_j=\mathrm{\Gamma }C_j`$ is a cocompact lattice in $`C_j`$ for every $`j`$.

###### 3.3

The image $`\mathrm{\Gamma }(j)`$ of $`\mathrm{\Gamma }`$ in $`G/C_j`$ is a cocompact lattice for every $`j`$.

It follows from 3.2 and 3.3 that the image of $`\mathrm{\Gamma }_j`$ in the abelian Lie algebra $`C_j/C_{j+1}`$ is a cocompact lattice. The quotient is a real torus

$$\mathrm{\Delta }_j=C_j/C_{j+1}\mathrm{\Gamma }_j=\mathrm{\Gamma }_jC_j/C_{j+1}.$$

###### Definition 3.4

A closed connected subgroup $`P`$ of $`G`$ is called $`\mathrm{\Gamma }`$-acceptable if the image of $`P_j=PC_j`$ in $`\mathrm{\Delta }_j`$ is dense for every $`0jk`$.

In 3.7 below we will see that the stable and unstable groups of an algebraic Anosov diffeomorphism of $`X=\mathrm{\Gamma }G`$ are $`\mathrm{\Gamma }`$-acceptable. Moreover, normal subgroups of $`G`$ whose orbits in $`X`$ are dense are $`\mathrm{\Gamma }`$-acceptable by proposition 3.11.

Let $``$ be the foliation of $`X`$ by the orbits of the right $`P`$-operation on $`X`$. The embedding of $``$ into $`C^{\mathrm{}}(X)`$ induces a map

(9)
$$H^{}(𝔭,)\overline{H}^{}(𝔭,C^{\mathrm{}}(X))\stackrel{(\text{5})}{}\overline{H}_{}^{}(X).$$

###### Theorem 3.5

If $`P`$ is $`\mathrm{\Gamma }`$-acceptable this is an isomorphism:

$$H^{}(𝔭,)\overline{H}_{}^{}(X).$$

For the proof we require the following lemma.

###### Lemma 3.6

Let $`𝔭`$ be a finite dimensional real Lie algebra and let

$$0=𝔞_0\mathrm{}𝔞_n𝔭$$

be a chain of normal subalgebras of $`𝔭`$ such that $`𝔞_j/𝔞_{j1}`$ is central in $`𝔭/𝔞_{j1}`$ for every $`1jn`$. Let $`M`$ be a complex vector space on which $`𝔭`$ acts $``$-linearly. We assume that $`𝔞_{n1}`$ acts trivially and that $`𝔞_n`$ acts by a nontrivial character $`\alpha `$. Then $`H^{}(𝔭,M)=0`$.

Proof The Hochschild–Serre spectral sequence

$$H^p(𝔭/𝔞_n,H^q(𝔞_n,M))H^{p+q}(𝔭,M)$$

shows that we may assume that $`𝔭=𝔞_n`$. Clearly we may also assume that the inclusions in the chain

$$0=𝔞_0\mathrm{}𝔞_n=𝔭$$

are strict. Since $`𝔞_n=𝔭`$ acts non-trivially we have $`𝔭0`$ and hence $`n1`$. The assertion is proved by induction on $`n`$. For $`n=1`$ the Lie algebra $`𝔭`$ is abelian. Hence it operates on $`M`$-valued cocycles and therefore on $`H^{}(𝔭,M)`$ by multiplication with $`\alpha `$ c.f. \[HS\] p. 591 below. On the other hand the operation of $`𝔤`$ on $`H^{}(𝔤,N)`$ is trivial for any Lie algebra $`𝔤`$ and any $`𝔤`$-module $`N`$. Since $`\alpha `$ was non-trivial, it follows that $`H^{}(𝔭,M)=0`$.

Now fix some $`n2`$ and assume the assertion proven for $`n1`$. Consider the spectral sequence

(10)
$$H^p(𝔭/𝔞_1,H^q(𝔞_1,M))H^{p+q}(𝔭,M).$$

Since $`𝔞_1`$ is central in $`𝔭=𝔞_n`$ it follows that $`𝔭`$ acts by multiplication with $`\alpha `$ on $`M`$-valued cocycles of $`𝔞_1`$ and hence on $`H^q(𝔞_1,M)`$ c.f. \[HS\] p. 591. Setting $`M^{}=H^q(𝔞_1,M)`$ and $`𝔞_i^{}=𝔞_{i+1}/𝔞_0,𝔭^{}=𝔭/𝔞_1`$ we get a chain

$$0=𝔞_0^{}\mathrm{}𝔞_{n1}^{}=𝔭^{}.$$

Applying the induction hypotheses to the $`𝔭^{}`$-action on $`M^{}`$ gives

$$H^{}(𝔭/𝔞_1,H^q(𝔞_1,M))=H^{}(𝔭^{},M^{})=0.$$

Using the spectral sequence (10) we conclude that $`H^{}(𝔭,M)=0`$.  $`\mathrm{}`$

Proof of 3.5 It is enough to show that the natural inclusion $`^{\mathrm{}}(X)=C^{\mathrm{}}(X,)`$ induces an isomorphism

(11)
$$H^{}(𝔭,)\stackrel{}{}\overline{H}^{}(𝔭,^{\mathrm{}}(X)).$$

According to 3.3 the image $`\mathrm{\Gamma }(j)`$ of $`\mathrm{\Gamma }`$ in $`G(j)=G/C_j`$ is a lattice for every $`0jk+1`$. The natural projection of right $`P`$-spaces

$$XX_j=\mathrm{\Gamma }(j)G(j)=\mathrm{\Gamma }C_jG$$

induces a $`𝔭`$-equivariant injection

(12)
$$^{\mathrm{}}(X_j)^{\mathrm{}}(X).$$

We will show by descending induction on $`j`$ that (12) induces isomorphisms of Fréchet spaces for all $`0jk+1`$

(13)
$$\overline{H}^{}(𝔭,^{\mathrm{}}(X_j))\stackrel{}{}\overline{H}^{}(𝔭,^{\mathrm{}}(X)).$$

For $`j=k+1`$ this is clear. For the induction step fix $`0mk`$ and assume that (13) is an isomorphism for $`j=m+1`$. Now consider the torus

$$\mathrm{\Delta }=\mathrm{\Delta }_m=C_m/C_{m+1}\mathrm{\Gamma }_m=\mathrm{\Gamma }_mC_m/C_{m+1}.$$

The right $`G`$-action on $`X_{m+1}`$ induces a $`\mathrm{\Delta }`$-action on $`X_{m+1}`$ since the elements of $`C_m`$ are central $`\mathrm{mod}C_{m+1}`$. Fourier theory implies that the $`\chi `$-isotypical components

$$^{\mathrm{}}(X_{m+1})(\chi )$$

of the induced $`\mathrm{\Delta }`$-action on $`^{\mathrm{}}(X_{m+1})`$ form an unconditional Schauder basis of subspaces in the sense of definition 2.2. Here $`\chi `$ runs over the characters of $`\mathrm{\Delta }`$. It follows that the subcomplexes

$$C^{}(𝔭,^{\mathrm{}}(X_{m+1})(\chi ))$$

of $`C^{\mathrm{}}(𝔭,^{\mathrm{}}(X_{m+1}))`$ satisfy the assumptions of sublemma 2.3. Hence there is a natural injection

(14)
$$\underset{\chi }{}\overline{H}^{}(𝔭,^{\mathrm{}}(X_{m+1})(\chi ))\overline{H}^{}(𝔭,^{\mathrm{}}(X_{m+1}))$$

with dense image. Moreover the spaces $`\overline{H}^{}(𝔭,^{\mathrm{}}(X_{m+1})(\chi ))`$ are closed in $`\overline{H}^{}(𝔭,^{\mathrm{}}(X_{m+1}))`$. We now apply lemma 3.6 to $`𝔭=\mathrm{Lie}P`$ and the chain

$$0=𝔞_0\mathrm{}𝔞_n𝔭$$

defined by $`𝔞_j=\mathrm{Lie}(PC_{k+1j})`$. Here $`n=k+1m`$. For $`M`$ we take $`M=^{\mathrm{}}(X_{m+1})(\chi )`$ where $`\chi `$ is any character of $`\mathrm{\Delta }`$ which is non-trivial on $`P_m=PC_m`$. Then $`𝔞_n=\mathrm{Lie}P_m`$ acts by the non-trivial character $`\alpha =\chi _{}`$ on $`M`$ and $`𝔞_{n1}=\mathrm{Lie}P_{m+1}`$ acts trivially. It follows from the lemma that only $`\chi `$’s with $`\chi (P_m)=1`$ can contribute to the direct sum in (14).

Now, by assumption $`P`$ is $`\mathrm{\Gamma }`$-acceptable. Hence the image of $`P_m`$ in $`\mathrm{\Delta }`$ is dense. The condition $`\chi (P_m)=1`$ therefore implies that $`\chi `$ is the trivial character $`\chi =1\mathrm{l}`$. Thus $`\overline{H}^{}(𝔭,^{\mathrm{}}(X_{m+1})(1\mathrm{l}))`$ is a dense subspace of $`\overline{H}^{}(𝔭,^{\mathrm{}}(X_{m+1}))`$. Since it is also closed we get:

(15)
$$\overline{H}^{}(𝔭,^{\mathrm{}}(X_m))=\overline{H}^{}(𝔭,^{\mathrm{}}(X_{m+1})).$$

Here we took into account that

$$^{\mathrm{}}(X_{m+1})(1\mathrm{l})=^{\mathrm{}}(X_{m+1}/\mathrm{\Delta })=^{\mathrm{}}(X_m).$$

By the induction hypotheses we have an isomorphism

$$\overline{H}^{}(𝔭,^{\mathrm{}}(X_{m+1}))\stackrel{}{}\overline{H}^{}(𝔭,^{\mathrm{}}(X)).$$

Combining it with (15) we get the isomorphism (13) for $`j=m`$. Note that a continuous algebraic isomorphism of Fréchet spaces is necessarily a topological isomorphism by the open mapping theorem. Thus we have completed the induction step. The case $`j=0`$ of (13) now give the assertion of the theorem.  $`\mathrm{}`$

###### 3.7

Let $`f`$ be an algebraic Anosov diffeomorphism of a nilmanifold $`X=\mathrm{\Gamma }G`$ as defined in the introduction. It is induced by an automorphism of $`G`$ which we also denote by $`f`$. Let $`𝔤^s𝔤`$ be the subspace of all $`v𝔤`$ for which $`f_{}^n(v)`$ tends to zero as $`n\mathrm{}`$. This is the stable subspace of $`𝔤`$. The unstable subspace $`𝔤^u`$ of $`𝔤`$ is the space of all $`v`$ of $`𝔤`$ such that $`f_{}^n(v)0`$ for $`n\mathrm{}`$. Since $`f`$ is a group automorphism it follows that $`f_{}`$ is a Lie algebra automorphism and so $`𝔤^s`$ and $`𝔤^u`$ are Lie subalgebras of $`𝔤`$. Let $`G^s`$ and $`G^u`$ denote the corresponding connected subgroups of $`G`$.

###### Proposition 3.8

In the situation of 3.7 the groups $`P=G^s`$ and $`P=G^u`$ are $`\mathrm{\Gamma }`$-acceptable.

Proof The stable subgroup of $`f^1`$ is the unstable subgroup of $`f`$. Hence it is enough to treat the case $`P=G^s`$. We have to show that for $`0jk`$ the image of $`P_j`$ is dense in $`\mathrm{\Delta }_j=\mathrm{\Gamma }_jG_j`$ where $`G_j=C_j/C_{j+1}`$ is abelian. We have $`P_j=G_j^s`$ with respect to the induced automorphism $`f_j`$ of $`G_j`$. Hence it suffices to show that for abelian $`G`$ the image of $`P=G^s`$ is dense in $`\mathrm{\Gamma }G`$ or equivalently that the image of $`\mathrm{\Gamma }`$ is dense in $`G/G^s`$.

So suppose $`G`$ is abelian. Then the exponential map is an isomorphism of the Lie algebra of $`G`$ with $`G`$, so $`G`$ is a finite dimensional $``$-vector space. We therefore rephrase the claim as follows. Let $`V`$ be a non-zero finite dimensional real vector space and let $`f`$ be an automorphism of $`V`$ with no complex eigenvalue of absolute value $`1`$. Let $`V^s`$ be the stable subspace, i.e.,

$$V^s=\{vVf^n(v)0\mathrm{as}n\mathrm{}\}.$$

Let $`\mathrm{\Gamma }V`$ be a lattice with $`f(\mathrm{\Gamma })=\mathrm{\Gamma }`$. Then we have to show that the image $`\mathrm{\Gamma }_W`$ of $`\mathrm{\Gamma }`$ is dense in $`W=V/V^s`$. This will follow if for every $`\epsilon `$-neighborhood $`U`$ of zero in $`W`$ the set $`U\mathrm{\Gamma }_W`$ contains a basis of $`W`$. Namely the lattice generated by such a basis is contained in $`\mathrm{\Gamma }_W`$ and has a fundamental domain in $`U`$. Thus every given point in $`W`$ is $`\epsilon `$-close to some point of this lattice and hence of $`\mathrm{\Gamma }_W`$.
So, let $`U`$ be a neighborhood of zero in $`W`$. On $`W`$ all eigenvalues of $`f`$ are of absolute value $`>1`$, therefore $`f^1`$ is contracting there. Let $`v_1,\mathrm{},v_r`$ be a basis of $`W`$ contained in $`\mathrm{\Gamma }_W`$. Then the vectors $`f^n(v_1),\mathrm{},f^n(v_r)`$ form a basis for every $`n`$ and this basis shrinks to zero as $`n\mathrm{}`$, so the claim follows.  $`\mathrm{}`$

Remark Indirectly the proof shows that $`V^s0`$. This is clear a priori since on the one hand $`|detf|=1`$ because $`f`$ respects a lattice and on the other hand $`f`$ does not have eigenvalues of absolute value $`1`$.

###### Corollary 3.9

In the situation of 3.7 we have natural isomorphisms

$$\overline{H}_^s^{}(X)H^{}(𝔤^s,)\text{and}\overline{H}_^u^{}(X)H^{}(𝔤^u,).$$

In particular these reduced leafwise cohomologies are finite dimensional.

Proof This follows from proposition 3.8 and theorem 3.5 using that $`^s`$ and $`^u`$ are the foliations by the orbits of the $`G^s`$-resp. $`G^u`$-action on $`X`$.  $`\mathrm{}`$

Remark We conjecture that for any Anosov diffeomorphism of a compact manifold with smooth stable and unstable foliations the reduced leafwise cohomologies with respect to these foliations are finite dimensional.

###### 3.10

In this final subsection we consider normal subgroups in simply connected real Lie groups. The following result holds:

###### Proposition 3.11

A normal subgroup $`P`$ with $`\mathrm{\Gamma }P`$ dense in $`G`$ is $`\mathrm{\Gamma }`$-acceptable.

Remark The proposition and theorem 3.5 imply the isomorphism

$$\overline{H}_{}^{}(X)H^{}(𝔭,).$$

This result is due originally to Álvarez López and Hector \[AH\] Theorem 2.12 with a different proof.

Proof It suffices to show that $`\mathrm{\Gamma }_jP_j`$ is dense in $`C_j`$. We use induction on $`j`$. For $`j=0`$ this is the assumption. Assume the claim proven for $`j`$. Since $`C_{j+1}=[G,C_j]`$ we pick $`xG`$ and $`yC_j`$. Let $`\gamma _np_n`$ be a sequence converging to $`x`$, where $`\gamma _n\mathrm{\Gamma }`$ and $`p_nP`$. Likewise let $`\beta _nl_n`$ be a sequence with $`\beta _n\mathrm{\Gamma }_j`$ and $`l_nP_j`$ converging to $`y`$. Then the commutator $`[\gamma _np_n,\beta _nl_n]`$ converges to $`[x,y]`$. If we can show that the former commutator lies in $`\mathrm{\Gamma }_{j+1}P_{j+1}`$ then we are done since $`P_{j+1}`$ is normal in $`G`$ so that $`\mathrm{\Gamma }_{j+1}P_{j+1}`$ is a subgroup. Now the claim will follow from the following group theoretical lemma.  $`\mathrm{}`$

###### Lemma 3.12

Let $`G`$ be a group and let $`A,B`$ be subgroups. Suppose that $`B`$ is normal. Let $`C_j`$ for $`j=0,1,2,`$ denote the lower central series. Let $`A_j=AC_j`$ and $`B_j=BC_j`$. Then

$$[AB,A_jB_j]A_{j+1}B_{j+1}.$$

Proof Let $`aA,bB,\alpha A_j,\beta B_j`$. For two group elements $`x,y`$ we write $`x^y`$ for $`y^1xy`$. Then

$`[ab,\alpha \beta ]`$ $`=`$ $`ab\alpha \beta b^1a^1\beta ^1\alpha ^1`$
$`=`$ $`a\alpha b^\alpha \beta b^1a^1\beta ^1\alpha ^1`$
$`=`$ $`[a,\alpha ]b^{\alpha a^1\alpha ^1}\beta ^{a^1\alpha ^1}(b^{a^1\alpha ^1})^1(\beta ^{\alpha ^1})^1`$
$`=`$ $`[a,\alpha ][b^{\alpha a^1\alpha ^1},\beta ^{a^1\alpha ^1}]\beta ^{a^1\alpha ^1}b^{\alpha a^1\alpha ^1}(b^{a^1\alpha ^1})^1(\beta ^{\alpha ^1})^1.`$
The two commutators are in $`A_{j+1}`$ resp. $`B_{j+1}`$ alright so we only have to care about the rest,

$$\beta ^{a^1\alpha ^1}b^{\alpha a^1\alpha ^1}(b^1)^{a^1\alpha ^1}(\beta ^1)^{\alpha ^1}.$$

In the same fashion as before we can move the first $`\beta `$-term past the two $`b`$-terms producing commutators in $`B_{j+1}`$. It remains to consider

$$b^{\alpha a^1\alpha ^1}(b^1)^{a^1\alpha ^1}\beta ^{a^1\alpha ^1}(\beta ^1)^{\alpha ^1}=\left(\alpha ^1b\alpha b^1\right)^{a^1\alpha ^1}\left(a\beta a^1\beta ^1\right)^{\alpha ^1}.$$

The commutator $`\alpha ^1b\alpha b^1=[\alpha ^1,b]`$ lies in $`[A_j,B]C_{j+1}`$. But on the other hand, since $`B`$ is normal, this commutator lies in $`B`$, therefore in $`B_{j+1}`$. The latter also is normal, hence the first factor lies in $`B_{j+1}`$. The second commutator $`a\beta a^1\beta ^1=[a,\beta ]`$ lies in $`[A,B_j]`$ and likewise turns out to lie in $`B_{j+1}`$.  $`\mathrm{}`$

## 4 The dynamical Lefschetz trace formula for algebraic Anosov diffeomorphisms

Consider an algebraic Anosov diffeomorphism of a nilmanifold $`X=\mathrm{\Gamma }G`$ induced by an automorphism $`f`$ of $`G`$. According to corollary 3.9 the reduced leafwise cohomology $`\overline{H}_^u^p(X)`$ is finite dimensional. Thus the trace of the induced automorphism $`f^{}`$ on this cohomology is well defined.

###### Theorem 4.1

The following formula holds:

$$\underset{p}{}(1)^p\mathrm{Tr}(f^{}|\overline{H}_^u^p(X))=\underset{\genfrac{}{}{0pt}{}{x}{f(x)=x}}{}\epsilon _x|det(1T_xf|T_x^s)|^1.$$

Here

$$\epsilon _x=\mathrm{sgn}det(1T_xf|T_x^u).$$

Proof Set $`𝔭=𝔤^u`$ and $`=^u`$. Then we have:

(16) $`{\displaystyle \underset{p}{}}(1)^p\mathrm{Tr}(f^{}|\overline{H}_{}^p(X))`$ $`=`$ $`{\displaystyle \underset{p}{}}(1)^p\mathrm{Tr}(f^{}|H^p(𝔭,))`$
$`=`$ $`{\displaystyle \underset{p}{}}(1)^p\mathrm{Tr}(\mathrm{\Lambda }^p\stackrel{ˇ}{f}_{}|\mathrm{\Lambda }^p\stackrel{ˇ}{𝔭})`$
$`=`$ $`det(1f_{}|𝔭)`$
$`=`$ $`det(1f_{}|𝔤)det(1f_{}|𝔤/𝔭)^1.`$
Now by a theorem of Nomizu \[N\] it is known that

$$H^p(𝔤,)=H^p(X,)$$

for all $`p`$. Therefore by the same calculation as before:

$`det(1f_{}|𝔤)`$ $`=`$ $`{\displaystyle \underset{p}{}}(1)^p\mathrm{Tr}(f^{}|H^p(𝔤,))`$
$`=`$ $`{\displaystyle \underset{p}{}}(1)^p\mathrm{Tr}(f^{}|H^p(X,)).`$
As $`f`$ has only non-degenerate fixed points (see below), the ordinary Lefschetz trace formula now implies:

(17)
$$det(1f_{}|𝔤)=\underset{\genfrac{}{}{0pt}{}{x}{f(x)=x}}{}\mathrm{sgn}det(1T_xf|T_xX).$$

For a fixed point $`x=\mathrm{\Gamma }g`$ of $`f`$ we have $`\mathrm{\Gamma }=\mathrm{\Gamma }gf(g)^1`$. The commutative diagram:

$$\begin{array}{ccc}X& \stackrel{R_{g^1}}{}& X\\ f& & f& & \\ X& \underset{R_{f(g)^1}}{}& X\end{array}$$

induces commutative diagrams where $`\overline{e}=\mathrm{\Gamma }X`$:

(18)
$$\begin{array}{ccccc}T_xX& \stackrel{}{}& T_{\overline{e}}X& =& 𝔤\\ T_xf& & T_{\overline{e}}f& & f_{}& & \\ T_xX& \stackrel{}{}& T_{\overline{e}}X& =& 𝔤\end{array}\text{and}\begin{array}{ccc}T_x& \stackrel{}{}& 𝔭\\ T_xf& & f_{}& & \\ T_x& \stackrel{}{}& 𝔭.\end{array}$$

Hence:

$$\underset{\genfrac{}{}{0pt}{}{x}{f(x)=x}}{}\epsilon _x|\mathrm{det}(1T_xf|T_xX/T_x)|^1=\frac{\mathrm{sgn}det(1f_{}|𝔭)}{|\mathrm{det}(1f_{}|𝔤/𝔭)|}\mathrm{}\mathrm{Fix}f$$

and (17) asserts that:

$$det(1f_{}|𝔤)=\mathrm{sgn}det(1f_{}|𝔤)\mathrm{}\mathrm{Fix}f.$$

Together with equation (16) this gives the formula if we remark that $`T_xX/T_x=T_x^s=T_x^s`$.  $`\mathrm{}`$

## 5 Dynamical Lefschetz trace formulas for generalized algebraic Anosov maps

In this section we will prove dynamical Lefschetz trace formulas for generalized algebraic Anosov maps with respect to homogenous invariant foliations.

Thus let $`f`$ be a generalized algebraic Anosov map for the pair $`(G,\mathrm{\Gamma })`$ as in the introduction and let $`𝔭𝔤`$ be an $`f_{}`$-invariant sub Lie algebra of $`𝔤`$. To $`𝔭`$ corresponds an $`f`$-invariant closed subgroup $`P`$ of $`G`$. Denote by $``$ the corresponding foliation on $`X=\mathrm{\Gamma }G`$.

By assumption $`f`$ maps leaves of $``$ into leaves of $``$ and hence it induces an endomorphism $`f^{}`$ of $`\overline{H}_{}^p(X)`$. Under the isomorphism

(19)
$$\overline{H}_{}^p(X)=\overline{H}^p(𝔭,C^{\mathrm{}}(X))$$

the operation $`f^{}`$ corresponds to the map on Lie algebra cohomology induced by $`f_{}`$ on $`𝔭`$ and by pullback $`f^{}`$ on $`C^{\mathrm{}}(X)`$.

We want to define the trace of $`f^{}`$ on the generally infinite dimensional cohomology (19). For this we use representation theory.

Let $`(\rho ,V)`$ be any representation of $`G`$. The pullback representation along an endomorphism $`f:GG`$ is defined by

$$f^{}(\rho ,V)=(\rho f,V).$$

It is unitary if $`(\rho ,V)`$ is. For $`\pi \widehat{G}`$ we get an equivalence class $`f^{}\pi `$ of unitary representations. If $`f:GG`$ is an isomorphism, then $`f^{}\pi `$ is irreducible. In this case we get a bijection $`f^{}:\widehat{G}\widehat{G}`$.

The following lemma will be crucial

###### Lemma 5.1

Let $`f`$ be an endomorphism of a simply connected nilpotent Lie group $`G`$ such that $`f_{}`$ does not have $`1`$ as an eigenvalue. For $`\pi \widehat{G}`$ assume that there is a non-zero $`G`$-intertwining map $`T`$ from $`f^{}(V_\pi ^{\mathrm{}})`$ to $`V_\pi ^{\mathrm{}}`$. Then $`\pi `$ is the trivial representation.

Proof Let us write $`(\rho ,V)`$ for $`(\rho _\pi ^{\mathrm{}},V_\pi ^{\mathrm{}})`$. Let $`C_j`$ be the descending central series. The integer $`k0`$ such that $`C_{k+1}=1`$ but $`C_k1`$ is called the length of the central series. Since $`f`$ is an endomorphism it preserves all the $`C_j`$ and in particular the subgroup $`C_k`$. This connected subgroup lies in the center of $`G`$. Since $`\pi `$ was supposed to be irreducible there is hence a character

$$\chi :C_k^{}$$

such that

$$\rho (g)=\chi (g)\mathrm{id}_V\text{for all}gC_k.$$

Since $`f`$ preserves $`C_k`$ it follows that

$$(\rho f)(g)=(\chi f)(g)\mathrm{id}_V\text{for all}gC_k.$$

The intertwining relation $`\rho (g)T=T(\rho f)(g)`$ for all $`gG`$ now implies that

$$\chi (g)T=(\chi f)(g)T\text{for all}gG.$$

Since $`T`$ was supposed to be non-trivial this gives the relation $`\chi =\chi f`$ on $`C_k`$. For the derivative $`\chi _{}:𝔠_k=\mathrm{Lie}C_k`$ it follows that

$$\chi _{}=\chi _{}f_{}\text{on}𝔠_k.$$

By assumption $`f_{}`$ does not have eigenvalue $`1`$ on $`𝔤`$ and all the less on $`𝔠_k`$. Hence the transposed map $`\stackrel{ˇ}{f}_{}`$ does not have eigenvalue $`1`$ on $`\stackrel{ˇ}{𝔠}_k`$. It follows that $`\chi _{}=0`$, so $`\chi `$ is trivial.

If $`k=0`$ then $`C_k=G`$ is abelian and it follows that $`\rho =\chi =1`$. Thus $`\pi `$ is trivial.

If $`k>0`$ then $`\rho `$ factors over $`G/C_k`$ which is nilpotent of length $`k1`$. The map $`f`$ induces an endomorphism of this smaller group whose derivative does not have the eigenvalue $`1`$. Also, $`T`$ remains an intertwining map for $`G/C_k`$. Thus we can iterate the argument down to the abelian case $`k=0`$ to reach the same conclusion.  $`\mathrm{}`$

Recall the decomposition (6). According to lemma 2.1 the algebraic direct sum

$$=\underset{\pi }{}\overline{H}^p(𝔭,H(\pi )^{\mathrm{}})$$

is a dense subspace of $`\overline{H}_{}^p(X,)`$.

###### Definition–Theorem 5.2

Let $`f`$ be a generalized algebraic Anosov map for $`(G,\mathrm{\Gamma })`$. Then we have the following assertions:
1) If $`f:GG`$ is surjective, hence an automorphism, then the map $`f^{}`$ on $`\overline{H}_{}^p(X,)`$ permutes the closed subspaces $`\overline{H}^p(𝔭,H(\pi )^{\mathrm{}})`$:

$$f^{}(\overline{H}^p(𝔭,H(\pi )^{\mathrm{}}))=\overline{H}^p(𝔭,H(f^{}\pi )^{\mathrm{}}).$$

In particular $`f^{}`$ leaves $``$ invariant. The condition $`f^{}\pi =\pi `$ for the subspace $`\overline{H}^p(𝔭,H(\pi )^{\mathrm{}})`$ to be left invariant is equivalent to $`\pi `$ being the trivial representation. Hence it is reasonable to define the trace of $`f^{}`$ as follows:

$`\mathrm{Tr}(f^{}|\overline{H}_{}^p(X))`$ $`:=`$ $`\mathrm{Tr}(f^{}|)`$
$`:=`$ $`{\displaystyle \underset{f^{}\pi =\pi }{}}\mathrm{Tr}(f^{}|\overline{H}^p(𝔭,H(\pi )^{\mathrm{}}))`$
$`=`$ $`\mathrm{Tr}(f^{}|H^p(𝔭,)).`$
If $`\overline{H}_{}^p(X)`$ is finite dimensional then our definition agrees with the usual trace of $`f^{}`$ as an endomorphism of a finite dimensional space.

2) In general, if $`f`$ is only an endomorphism of $`G`$, then unless $`\pi `$ is the trivial representation,

$$f^{}(H(\pi )^{\mathrm{}})\text{is orthogonal to}H(\pi )^{\mathrm{}}$$

in the Hilbert space $`L^2(X)`$. Here

$$f^{}:C^{\mathrm{}}(X)C^{\mathrm{}}(X)$$

is the pullback map induced by $`f:XX`$. It follows that for non-trivial $`\pi `$ we have:

$$f^{}\overline{H}^p(𝔭,H(\pi )^{\mathrm{}})\overline{\underset{\pi ^{}\pi }{}}\overline{H}^p(𝔭,H(\pi ^{})^{\mathrm{}}),$$

where $`\overline{}`$ denotes the closure of the direct sum in the topology of $`\overline{H}_{}^p(X)`$. Again this suggests to define the trace of $`f^{}`$ by the formula

$$\mathrm{Tr}(f^{}|\overline{H}_{}^p(X)):=\mathrm{Tr}(f^{}|\overline{H}^p(𝔭,)).$$

If $`\overline{H}_{}^p(X)`$ is finite dimensional this agrees with the usual trace.

To motivate the definition of the trace in 5.2 further, assume that the cohomology spaces $`\overline{H}^p(𝔭,H(\pi )^{\mathrm{}})`$ are finite dimensional and choose bases in all of them. The union of these bases defines a basis of $``$. In the situation 1) above writing $`f^{}:`$ as a matrix in terms of these bases, we see that the sum of the diagonal entries is given by:

$$\underset{f^{}\pi =\pi }{}\mathrm{Tr}(f^{}|\overline{H}^p(𝔭,H(\pi )^{\mathrm{}})).$$

The more general case 2) can be similarly motivated.

Remark In example 5.5 below we will see that $`\overline{H}_{}^p(X)`$ can be infinite dimensional even if $``$ has dense leaves.

Proof of 5.2 1) Since $`f:XX`$ is a finite covering map we have

$$f_{}\mu =\mu .$$

Note here that $`f_{}\mu `$ is again $`G`$-invariant of volume one. Thus $`f`$ induces an isometric embedding

$$f^{}:L^2(X)L^2(X).$$

In general this map is not $`G`$-equivariant. However because of the relation

(20)
$$R_gf^{}=f^{}R_{f(g)}\text{on}L^2(X)\text{for all}gG,$$

it maps irreducible subrepresentations of $`L^2(X)`$ into irreducible subrepresentations. Here isomorphic subrepresentations get mapped to isomorphic ones. Using (20) it follows that for every isotypical component $`H(\pi )`$ we have

$$f^{}H(\pi )=H(f^{}\pi ).$$

Equivalently:

$$f^{}(H(\pi )^{\mathrm{}})=H(f^{}\pi )^{\mathrm{}}$$

under the pullback map $`f^{}:C^{\mathrm{}}(X)C^{\mathrm{}}(X)`$. Thus the map $`f^{}=(f_{},f^{})`$ on

$$\overline{H}_{}^p(X)=\overline{H}_{}^p(𝔭,C^{\mathrm{}}(X))$$

permutes the subspaces $`\overline{H}^p(𝔭,H(\pi )^{\mathrm{}})`$ as stated. That $`f^{}\pi =\pi `$ is equivalent to $`\pi `$ being the trivial representation is a special case of lemma 5.1.

2) For endomorphisms $`f:GG`$ we do not have an induced map on $`L^2(X)`$ unless $`f`$ is surjective. However the pullback map on $`C^{\mathrm{}}(X)`$ is all that is needed. Assume that $`f^{}(H(\pi )^{\mathrm{}})`$ is not orthogonal to $`H(\pi )^{\mathrm{}}`$. Then

$$P_\pi (f^{}(H(\pi )^{\mathrm{}}))0,$$

where $`P_\pi `$ denotes the orthogonal projection from $`L^2(X)`$ onto $`H(\pi )`$. Since $`P_\pi `$ maps smooth vectors to smooth vectors, it follows that

$$S=P_\pi |_{f^{}(H(\pi )^{\mathrm{}})}:f^{}(H(\pi )^{\mathrm{}})H(\pi )^{\mathrm{}}$$

is a non-zero $`G`$-intertwining map. So the map

$$\stackrel{~}{S}=Sf^{}:M_\pi V_\pi ^{\mathrm{}}H(\pi )^{\mathrm{}}H(\pi )^{\mathrm{}}M_\pi V_\pi ^{\mathrm{}}$$

is non-zero and satisfies

$$\rho (g)\stackrel{~}{S}=\stackrel{~}{S}\rho (f(g))$$

where $`\rho =\rho _\pi ^{\mathrm{}}`$ is the representation on $`V_\pi ^{\mathrm{}}`$. Note here that:

$$R_gf^{}=f^{}R_{f(g)}\text{on}C^{\mathrm{}}(X)\text{for all}G.$$

Since $`G`$ acts trivially on the finite dimensional space $`M_\pi `$, it follows that there is a non-zero $`G`$-intertwining map $`T`$ from $`V_\pi ^{\mathrm{}}`$ to itself satisfying

$$\rho (g)T=T\rho (f(g))\text{for all}gG.$$

In other words, $`T`$ is a non-zero $`G`$-intertwining map from $`f^{}(V_\pi ^{\mathrm{}})`$ to $`V_\pi ^{\mathrm{}}`$. Applying lemma 5.1 it follows that $`\pi `$ is the trivial representation. The remaining assertions are clear.  $`\mathrm{}`$

###### Theorem 5.3

For a generalized algebraic Anosov map $`f`$ for $`(G,\mathrm{\Gamma })`$ and a homogenous foliation $``$ corresponding to $`𝔭`$ as above, we have

$$\underset{p=0}{\overset{dim}{}}(1)^p\mathrm{Tr}(f^{}|\overline{H}_{}^p(X))=\underset{\genfrac{}{}{0pt}{}{x}{f(x)=x}}{}\epsilon _x|\mathrm{det}(1T_xf|T_xX/T_x)|^1$$

where

$$\epsilon _x=\mathrm{sgn}det(1T_xf|T_x).$$

Proof The proof of theorem 4.1 applies verbally in this more general situation.  $`\mathrm{}`$

Formula (1) of the introduction can now be extended. Let $`f`$ be a generalized algebraic Anosov diffeomorphism. Denote by $`𝔤_\lambda `$ the generalized $`\lambda `$-eigenspace of $`f_{}`$ on $`𝔤_{}`$ and set

$$𝔤_{}^u=\underset{|\lambda |>1}{}𝔤_\lambda ,𝔤_{}^s=\underset{|\lambda |<1}{}𝔤_\lambda ,𝔤_{}^e=\underset{|\lambda |=1}{}𝔤_\lambda .$$

Because of the relation

$$[𝔤_\lambda ,𝔤_\mu ]𝔤_{\lambda \mu }$$

these are sub Lie algebras of $`𝔤_{}`$. For $`𝔤^{}=𝔤_{}^{}𝔤`$ we have $`𝔤_{}^{}=𝔤^{}_{}`$. Hence there is an $`f_{}`$-invariant decomposition

$$𝔤=𝔤^u𝔤^s𝔤^e.$$

Here $`𝔤^s`$ and $`𝔤^u`$ can also be characterized as the set of $`v𝔤`$ such that $`f_{}^N(v)`$ tends to zero as $`N+\mathrm{}`$ resp. $`N\mathrm{}`$.

The subalgebras $`𝔤^s,𝔤^u,𝔤^e`$ of $`𝔤`$ are called the stable resp. unstable resp. neutral algebra of $`f`$. Let $`^{}`$ be the foliation corresponding to $`𝔤^{}`$. Then we have

$$TX=T^uT^sT^e$$

and hence

$$T_xX/T_x^u=T_x^sT_x^e.$$

Using (18) it follows that for a fixed point $`x`$ of $`f`$ on $`X`$ we have

$$T_x^sT_x^e=T_x^{se},T_x^u=T_x^u$$

where $`T_x^{}=T_x^{}T_xX`$ and $`T_x^{se}`$ resp. $`T_x^u`$ is the sum of the eigenspaces of $`T_xf`$ on $`T_xX`$ corresponding to eigenvalues $`\lambda `$ of absolute value $`|\lambda |1`$ resp. $`|\lambda |>1`$.

###### Corollary 5.4

For every generalized algebraic Anosov diffeomorphism $`f`$ of $`(G,\mathrm{\Gamma })`$ the following formula holds:

$$\underset{p}{}(1)^p\mathrm{Tr}(f^{}|\overline{H}_^u^p(X))=\underset{\genfrac{}{}{0pt}{}{x}{f(x)=x}}{}\epsilon _x|\mathrm{det}(1T_xf|T_x^{se})|^1$$

where

$$\epsilon _x=\mathrm{sgn}det(1T_xf|T_x^u).$$

The following example shows that even if the unstable foliation of a generalized algebraic Anosov diffeomorphism has dense leaves, its reduced leafwise cohomology can be infinite dimensional.

###### Example 5.5

Let $`G`$ be the $`3`$-dimensional simply connected real Heisenberg group and let $`\mathrm{\Gamma }`$ be its standard lattice. We can think of $`G`$ as the subgroup of matrices

$$[x,y,z]:=\left(\begin{array}{ccc}1& x& z\\ 0& 1& y\\ 0& 0& 1\end{array}\right)\text{in}\mathrm{GL}_3().$$

Then $`\mathrm{\Gamma }`$ consists of those matrices $`[n,m,k]`$ with integer entries. The quotient $`X=\mathrm{\Gamma }G`$ is compact. We will identify the Lie algebra $`𝔤`$ of $`G`$ with the Lie algebra of matrices

$$[|a,b,c|]:=\left(\begin{array}{ccc}0& a& c\\ 0& 0& b\\ 0& 0& 0\end{array}\right)\text{in}M_3().$$

One checks that $`f:GG`$ defined by

$$f[x,y,z]=[x+y,2x+y,z+(x+y)^2\frac{1}{2}y(y1)]$$

is a Lie group automorphism with inverse:

$$f^1[x,y,z]=[yx,2xy,x^2z\frac{1}{2}(2xy)(2xy1)].$$

The formulas show that $`f(\mathrm{\Gamma })\mathrm{\Gamma }`$ and $`f^1(\mathrm{\Gamma })\mathrm{\Gamma }`$ hence $`f(\mathrm{\Gamma })=\mathrm{\Gamma }`$. The differential $`f_{}:𝔤𝔤`$ of $`f`$ is given by

$$f_{}[|a,b,c|]=[|a+b,2a+b,\frac{b}{2}c|].$$

Its eigenvalues are $`1+\sqrt{2},1\sqrt{2},1`$. Hence $`f`$ is a generalized algebraic Anosov diffeomorphism. An eigenvector for $`\lambda =1+\sqrt{2}`$ is given by

$$v=[|1,\sqrt{2},0|].$$

Hence $`𝔤^u=v`$. As $`1,\sqrt{2}`$ are $``$-linear independent the one-dimensional foliation $`^u`$ has dense leaves \[AGH\] Ch. IV and according to \[DS\] Proposition 1.2 the reduced cohomology $`\overline{H}_^u^1(X)`$ is infinite dimensional. Incidentally because of the relation

$$det(1f_{}|𝔤)=4$$

the automorphism $`f`$ of $`X`$ has $`4`$ different fixed points.

A. Deitmar

School of mathematical sciences

University of Exeter

Laver building

North park road

Exeter EX4 4QE

Devon, UK

a.h.j.deitmar@exeter.ac.uk

C. Deninger

Mathematisches Institut

Einsteinstr. 62

48149 Münster

Germany

deninge@math.uni-muenster.de