text:
# Bound States in Mildly Curved Layers

## 1 Introduction

The investigation of quantum particles constrained to a spatial region $`\mathrm{\Omega }`$ of a prescribed shape, in particular, relations between its spectral properties and the geometry of $`\mathrm{\Omega }`$ became an attractive problem when the technology progress made it possible to fabricate various mesoscopic systems for which this is a reasonable model – see, *e.g.*, \[LCM\]. Curvature induced bound states in hard-wall strips and tubes have been demonstrated more than a decade ago \[ES\] and studied subsequently in numerous papers – *cf.* \[DE, LCM\] and references therein.

Much less attention was paid to quantum mechanics in layers, apart of the trivial planar case. While for an experimenter it is easier to prepare a semiconductor film on a curved substrate than to fabricate a quantum wire, from the mathematical point of view the opposite is true, and solving the Schrödinger equation in a nontrivial layer is more complicated than the corresponding problem in a tube. It was noticed long time ago that in the formal limit of zero width the layer curvatures give rise to an effective attractive potential \[KJ\] but the first results for curved finite-width layers, with the particle Hamiltonian being a multiple of the Dirichlet Laplacian $`\mathrm{\Delta }_D^\mathrm{\Omega }`$, appeared only recently.

We restrict ourselves to the case when $`\mathrm{\Omega }`$ is non-compact and nontrivially curved. The first proof of existence of geometrically induced bound states was given in \[DEK1\] under the assumption that the planar layer is curved only locally. A more general case of curved layers which are *asymptotically* planar in the sense that the curvatures of the generating surface vanish at large distances has been discussed in \[DEK2\]. We have derived there several sufficient conditions for the existence of bound states expressed in terms of geometric quantities characterizing the reference surface. While these conditions cover a wide class of layers, they do not represent an ultimate result: it is not clear, *e.g.*, whether bound states exist in layers built over surfaces of positive total Gauss curvature unless the latter are thin enough or endowed with a cylindrical symmetry.

After demonstrating their existence one is naturally interested in properties of the curvature-induced bound states. A particular question which we are going to address in the present paper concerns the weak-coupling regime. Since in our case the binding comes from the curvature alone, the described situation is expected to occur in *mildly curved* layers. We will show that the layer has then a unique eigenvalue and derive an asymptotic expansion for the gap between this eigenvalue and the threshold of the essential spectrum. The leading term in this formula will depend on the mean curvature of the reference surface.

Let us describe briefly the contents of the paper. In the next section we will give a precise formulation of the problem and describe the main result summarized in Theorem 2.1. The rest is devoted to the proof. The strategy is adopted from the “lower-dimensional” case of curved strips and tubes analyzed in \[DE, Sec. 4\]. First, in Section 3, we consider Schrödinger operators acting in a planar layer built over $`^2`$. We derive a necessary and sufficient condition under which such an operator has a bound state in the weak coupling limit and find the asymptotic expansion of the eigenvalue. The result contained in Theorem 3.4 is of an independent interest; we prove it in a greater generality for a layer in $`^{2+m},m1`$. Next, in Section 4, we apply it to the case of mildly curved quantum layers. We express the operator $`\mathrm{\Delta }_D^\mathrm{\Omega }`$ in the coordinates $`(x,u)`$, where $`x(x^1,x^2)^2`$ and $`u𝐼:=(a,a)`$ with $`a>0`$ parametrize the surface and its normal space, respectively, and pass to a unitarily equivalent operator with an effective potential, which can be consecutively estimated by operators to which Theorem 3.4 can be applied . We will also show how the leading term in the expansion looks like in the case of a thin layer.

## 2 The results

To give a precise statement of the main result we need first to specify what we mean by mildly curved quantum layers. Let a family of surfaces $`\mathrm{\Sigma }_\epsilon :=p(^2)`$ be given by a Monge patch

$$p:^2^3,p(x^1,x^2;\epsilon ):=(x^1,x^2,\epsilon f(x^1,x^2)),$$
(2.1)
where $`f`$ is supposed to be a $`C^4`$-smooth function. Here $`\epsilon >0`$ is the parameter which controls the deformation; it is supposed to be small so that $`\mathrm{\Sigma }_\epsilon `$ is a mildly curved plane. The cross-product of the tangent vectors $`p_{,\mu }:=p/x^\mu `$, $`\mu =1,2`$, defines a unit normal field $`n`$ on $`\mathrm{\Sigma }_\epsilon `$. We put $`\mathrm{\Omega }_0:=^2\times 𝐼`$ and define a layer $`\mathrm{\Omega }_\epsilon :=(\mathrm{\Omega }_0)`$ of width $`d=2a`$ over the surface $`\mathrm{\Sigma }_\epsilon `$ by virtue of the mapping $`:\mathrm{\Omega }_0^3`$ which acts as

$$(x,u;\epsilon ):=p(x;\epsilon )+un(x;\epsilon ).$$
(2.2)
The layer in question is thus the spatial region closed between two parallel surfaces – *cf.* \[Sp3, Prob. 12 of Chap. 3\] – represented by $`p\pm an`$. In this sense, we will hereafter refer to the surface coordinates $`(x^1,x^2)`$ as to longitudinal variables, while $`u`$ will be the transverse variable.

We consider a nonrelativistic spinless particle confined to $`\mathrm{\Omega }_\epsilon `$ which is free within it, and suppose that the boundary of the layer is a hard wall, *i.e.*, the wavefunctions satisfy the Dirichlet boundary condition there. For the sake of simplicity we set Planck’s constant $`\mathrm{}=1`$ and the mass of the particle $`m=\frac{1}{2}`$. Then the Hamiltonian of the system can be identified with the Dirichlet Laplacian $`\mathrm{\Delta }_D^{\mathrm{\Omega }_\epsilon }`$ on $`L^2(\mathrm{\Omega }_\epsilon )`$, which is defined for an open set $`\mathrm{\Omega }_\epsilon ^3`$ as the Friedrichs extension of the Laplacian acting on $`C_0^{\mathrm{}}(\mathrm{\Omega }_\epsilon )`$*cf.* \[RS4, Sec. XIII.15\] or \[Dav, Chap. 6\]. The domain of the closure of the corresponding quadratic form is the Sobolev space $`W_0^{1,2}(\mathrm{\Omega }_\epsilon )`$.

Next we must introduce some notation and formulate assumptions about the function $`f`$, in addition to the smoothness requirement mentioned above. We consider asymptotically planar surfaces – *cf.* \[DEK2\] – which means that the Gauss $`K`$ and mean $`M`$ curvatures of $`\mathrm{\Sigma }_\epsilon `$ vanish at large distances from a fixed point. This will hold if we require

$`\text{d1}`$ $`f_{,\mu }L^{\mathrm{}}(^2)`$ together with $`\text{d2}`$ $`f_{,\mu \nu }0`$ as $`|x|\mathrm{}`$

for $`\mu ,\nu \{1,2\}`$. This allows us to localize the essential spectrum. Adapting from \[DEK2, Thm. 4.1\] a simple argument based on a Neumann bracketing one finds that $`inf\sigma _{\mathrm{ess}}(\mathrm{\Delta }_D^{\mathrm{\Omega }_\epsilon })\kappa _1^2`$. Here $`\{\kappa _j^2:=(\kappa _1j)^2\}_{j=1}^{\mathrm{}}`$, with $`\kappa _1:=\pi /d`$, are the eigenvalues of $`\mathrm{\Delta }_D^𝐼`$; the corresponding eigenfunctions will be denoted by $`\chi _j`$. Assuming in addition

$`\text{d3}`$ $`f_{,\mu \nu \rho }0`$ as $`|x|\mathrm{}`$ and $`\text{d4}`$ $`f_{,\mu \nu \rho \sigma }L^{\mathrm{}}(^2)`$

for $`\mu ,\nu ,\rho ,\sigma \{1,2\}`$ we will be able to prove in Section 4.3 that the bound is sharp,

$$inf\sigma _{\mathrm{ess}}(\mathrm{\Delta }_D^{\mathrm{\Omega }_\epsilon })=\kappa _1^2.$$
(2.3)
We have also to require that the derivatives of $`f`$ satisfy the following integrability hypotheses:

$`\text{r1,2}`$ $`f_{,\mu \nu }`$, $`f_{,\mu \nu \rho }L^2(^2,(1+|x|^\delta )dx)`$, respectively

and

$`\text{r3}`$ $`f_{,\mu \nu \rho \sigma }`$ $`L^1(^2,(1+|x|^\delta )dx)`$  for some $`\delta >0`$

and $`\mu ,\nu ,\rho ,\sigma \{1,2\}`$. The main result of the present work reads as follows:

###### Theorem 2.1.

Let $`\mathrm{\Omega }_\epsilon `$ be a family of layers generated by the surfaces $`\mathrm{\Sigma }_\epsilon `$ given by *(2.1)*. Suppose that the function $`fC^4(^2)`$ satisfies the hypotheses *$`\text{d1–4}`$* and *$`\text{r1–3}`$*. If $`\mathrm{\Sigma }_1`$ is not planar, then for all $`\epsilon `$ small enough $`\mathrm{\Delta }_D^{\mathrm{\Omega }_\epsilon }`$ has exactly one isolated eigenvalue $`E(\epsilon )`$ below the essential spectrum. Moreover, it can be expressed as

$$E(\epsilon )=\kappa _1^2e^{2w(\epsilon )^1},$$

where $`w(\epsilon )`$ has the following asymptotic expansion

$$w(\epsilon )=\epsilon ^2\underset{j=2}{\overset{\mathrm{}}{}}(\chi _1,u\chi _j)_𝐼^2\left(\kappa _j^2\kappa _1^2\right)^2_^2\frac{|\widehat{m_0}(\omega )|^2}{|\omega |^2+\kappa _j^2\kappa _1^2}𝑑\omega +𝒪(\epsilon ^{2+\gamma })$$

with $`\gamma :=\mathrm{min}\{1,\delta /2\}`$. Here $`m_0`$ is the lowest-order term in the expansion of the mean curvature of $`\mathrm{\Sigma }_\epsilon `$ w.r.t. $`\epsilon `$*cf.* *(4.3)*.

###### Remarks.

(a) The subscript “$`𝐼`$” indicates the norm and the inner product in the space $`L^2(𝐼)`$. The sum runs in fact over even $`n`$ only because one integrates over the interval $`I=(a,a)`$ on which the function $`u\chi _1(u)u\chi _j(u)`$ is odd for odd $`j`$.

(b) The expression for the leading-term coefficient $`w_1`$ in the expansion $`w(\epsilon )=:\epsilon ^2w_1+𝒪(\epsilon ^{2+\gamma })`$ does not have a very transparent structure, however, for thin layers it can be rewritten as

$$w_1=\frac{1}{2\pi }m_0^2+\frac{\pi ^26}{24\pi ^3}m_0^2d^2+𝒪(d^4).$$
(2.4)
This formula is instructive because the first term comes from the surface attractive potential $`KM^2`$ which dominates the picture in this case, while the $`𝒪(d^4)`$ error term expresses the contribution of higher transverse modes. We refer to Section 4.4 for more details.

## 3 Weakly Coupled Schrödinger Operators in a Planar Layer

Let $`M`$ be an open connected precompact set in $`^m`$, $`m1`$. The object of our interest in this section will be the operator

$$H_\lambda =\mathrm{\Delta }_D+\lambda V\text{with}\lambda >0\text{on}:=L^2(^2)L^2(M),$$
(3.5)
where $`\mathrm{\Delta }_D`$ is the Dirichlet Laplacian on $`^2\times M`$ defined as the closure of $`\mathrm{\Delta }I_m+I_2\mathrm{\Delta }_D^M`$. In the last expression the unindexed $`\mathrm{\Delta }`$ stays for the Laplace operator in $`^2`$. The potential $`V`$ is supposed to be $`H_0^{\frac{1}{2}}`$-bounded, in other words

$`\text{a1}`$ $`a,b0\psi DomQ_0:V\psi a\psi +bH_0^{\frac{1}{2}}\psi `$

where $`DomQ_0=W_0^{1,2}(^2\times M)`$ is the form domain of $`H_\lambda `$.

The Dirichlet Laplacian $`\mathrm{\Delta }_D^M`$ on $`L^2(M)`$ has a purely discrete spectrum consisting of eigenvalues $`\kappa _1^2<\kappa _2^2\mathrm{}\kappa _j^2<\mathrm{}`$; the corresponding normalized eigenfunctions will be denoted as $`\chi _j`$, where $`j=1,2,\mathrm{}`$. The lowest eigenvalue is, of course, simple and the eigenfunction $`\chi _1`$ can be chosen positive – *cf.* \[RS4, Sec. XIII. 12\]. Important for our purpose are the transverse projections of the potential,

$$V_{jj^{}}:=_M\overline{\chi }_j(y)V(,y)\chi _j^{}(y)𝑑y$$

and the analogous quantities for other functions on $`^2\times M`$, in particular for $`|V|`$. We shall adopt the assumptions

$`\text{a2}`$ $`|V|_{11}𝐿^{1+\delta }\left(^2\right)`$ and $`\text{a3}`$ $`|V|_{11}L^1(^2,(1+|x|^\delta )dx)`$

for some $`\delta >0`$. We also suppose that the essential spectrum of $`H_\lambda `$ does not start below the lowest transverse-mode threshold, *i.e.*,

$`\text{a0}`$ $`inf\sigma _{\mathrm{ess}}(H_\lambda )\kappa _1^2`$

This is true, in particular, if $`V`$ vanishes at large distances from a fixed point.

The goal of this section is to show that for $`\lambda `$ small enough the discrete spectrum of $`H_\lambda `$ below $`\kappa _1^2`$ is not empty provided the projection of $`V`$ onto $`\chi _1`$ is not repulsive in the mean. This part of spectrum then contains only one eigenvalue $`E(\lambda )`$ for such a small $`\lambda `$ and it approaches $`\kappa _1^2`$ as $`\lambda `$ tends to zero. The last claim follows from the following elementary fact.

###### Proposition 3.1.

Assume *$`\text{a1}`$*. Then there are positive constants $`\lambda _0`$ and $`c`$ such that for all $`\lambda (0,\lambda _0)`$ we have $`H_\lambda \kappa _1^2c\lambda `$.

###### Proof:.

Using the Schwarz inequality and the assumption $`\text{a1}`$ together with the inequality between the geometric and arithmetic means, we have for all $`\psi DomQ_0`$ the bound

$`Q_\lambda [\psi ]:=H_\lambda ^{\frac{1}{2}}\psi ^2`$ $``$ $`H_0^{\frac{1}{2}}\psi ^2\lambda \psi V\psi `$
$``$ $`(1\lambda b/2)H_0^{\frac{1}{2}}\psi ^2\lambda (a+b/2)\psi ^2.`$
Since $`H_0^{\frac{1}{2}}\psi \kappa _1\psi `$, we obtain the assertion by putting $`\lambda _0:=2/b`$ and $`c:=a+(1+\kappa _1^2)b/2`$. ∎

### 3.1 Birman-Schwinger Analysis

Using the orthonormal basis of $`L^2(M)`$ given by $`\{\chi _j\}`$, the free resolvent operator $`R_0(\alpha ):=(H_0\alpha ^2)^1`$ is decoupled in the following way

$$R_0(\alpha )=\underset{j=1}{\overset{\mathrm{}}{}}\chi _j\left(\mathrm{\Delta }+k_j(\alpha )^2\right)^1\overline{\chi }_j,k_j(\alpha ):=\sqrt{\kappa _j^2\alpha ^2}.$$

If we are interested in eigenvalues of $`H_\lambda `$ below the lowest transverse mode, we have to consider $`\alpha [0,\kappa _1)`$ and the two-dimensional resolvent in the middle of the expansion is well defined for any $`j=1,2,\mathrm{}`$. It can be expressed in terms of Hankel’s function – *cf.* \[AGHH, Chap. I.5\]. Passing to Macdonald’s functions $`K_0`$ by \[AS, 9.6.4\], we arrive at the following integral kernel formula

$$R_0(x,y,x^{},y^{};\alpha )=\frac{1}{2\pi }\underset{j=1}{\overset{\mathrm{}}{}}\chi _j(y)K_0\left(k_j(\alpha )|xx^{}|\right)\overline{\chi }_j(y^{}).$$
(3.6)
Define $`K(\alpha ):=|V|^{\frac{1}{2}}R_0(\alpha )V^{\frac{1}{2}}`$, where we have employed the usual sign convention, $`V^{\frac{1}{2}}:=|V|^{\frac{1}{2}}sgnV`$. According to Birman-Schwinger principle – *cf.* \[Sim\] – the function $`\alpha (\lambda )^2E(\lambda )`$ is an eigenvalue of $`H_\lambda `$ if and only if the operator $`\lambda K(\alpha )`$ has the eigenvalue $`1`$, *i.e.*,

$$\alpha ^2\sigma _{\mathrm{disc}}(H_\lambda )1\sigma _{\mathrm{disc}}(\lambda K(\alpha )).$$
(3.7)
The first term in the expansion (3.6) referring to $`j=1`$ has a singularity at $`\alpha =\kappa _1`$, and as usual in such problems we have to single it out. For this purpose we first introduce the following decomposition.

###### Lemma 3.2.

There are real-analytic functions $`f`$ and $`g`$ such that

$$u(0,\mathrm{}):K_0(u)=f(u)\mathrm{ln}u+g(u)$$

and

1. $`f(u)=1+𝒪(u^2)`$, $`g(u)=(\mathrm{ln}2\gamma _\mathrm{E})+𝒪(u^2)`$ as $`u0+`$,
2. $`C_1>0u(0,\mathrm{}):\mathrm{max}\{f(u),g(u)\}C_1e^u`$,

where $`\gamma _\mathrm{E}`$ denotes Euler’s constant.

###### Proof:.

Around the origin a good choice to approximate the Macdonald function $`K_0`$ would be $`f:=I_0`$, where $`I_0`$ is the other modified Bessel function \[AS, 9.6\] but it has a bad behaviour at large distances. Hence we use an interpolation, for instance,

$`f(u)`$ $`:=`$ $`e^{u^2}I_0(u)(1e^{u^2})K_0(u)`$
$`g(u)`$ $`:=`$ $`e^{u^2}I_0(u)\mathrm{ln}u+\left[1+(1e^{u^2})\mathrm{ln}u\right]K_0(u).`$
Using the relation \[AS, 9.6.13\], we obtain the behaviour at the origin (i). On the other hand, it follows by \[AS, 9.7.2\] that $`f`$ and $`g`$ have a faster-than-exponential decay at infinity, which gives together with (i) that there is a positive $`C_1`$ such that (ii) holds. ∎

We will thus use the decomposition $`K(\alpha )=L_\alpha +M_\alpha `$, where

$$L_\alpha (x,y,x^{},y^{}):=\frac{1}{2\pi }|V(x,y)|^{\frac{1}{2}}\chi _1(y)\mathrm{ln}k_1(\alpha )\chi _1(y^{})V(x^{},y^{})^{\frac{1}{2}}$$

contains the singularity and the regular $`M_\alpha `$ splits into two parts again, $`M_\alpha =A_\alpha +B_\alpha `$. The operator $`B_\alpha `$ is given by the projection of the resolvent on higher transverse modes, *i.e.*, $`B_\alpha :=|V|^{\frac{1}{2}}R_0^{}(\alpha )V^{\frac{1}{2}}`$ with

$$R_0^{}(\alpha ):=\underset{j=2}{\overset{\mathrm{}}{}}\chi _j\left(\mathrm{\Delta }+k_j(\alpha )^2\right)^1\overline{\chi }_j,$$
(3.8)
and the kernel of the remaining term is therefore

$`A_\alpha (x,y,x^{},y^{})`$
$`:={\displaystyle \frac{1}{2\pi }}|V(x,y)|^{\frac{1}{2}}\chi _1(y)\left(K_0(k_1(\alpha )|xx^{}|)+\mathrm{ln}k_1(\alpha )\right)\chi _1(y^{})V(x^{},y^{})^{\frac{1}{2}}.`$
We note that $`M_\alpha `$ is by definition well defined for $`\alpha =\kappa _1`$. In particular,

$$A_{\kappa _1}(x,y,x^{},y^{})=\frac{1}{2\pi }|V(x,y)|^{\frac{1}{2}}\chi _1(y)\left(\gamma _\mathrm{E}+\mathrm{ln}\frac{|xx^{}|}{2}\right)\chi _1(y^{})V(x^{},y^{})^{\frac{1}{2}}.$$

Furthermore, we have the following lemma. Since its proof is purely technical, we postpone it to Section 3.2 below.

###### Lemma 3.3.

Assume *$`\text{a1–3}`$*. Then there are positive $`C_2,C_3`$ and $`C_4`$ such that

1. $`\alpha [0,\kappa _1]:M_\alpha <C_2`$,
2. $`M_\alpha M_{\kappa _1}C_3\lambda ^\gamma `$, with $`\gamma :=\mathrm{min}\{1,\delta /2\}`$,
3. $`\frac{dM_{\alpha (w)}}{dw}<C_4|w|^1`$ for $`\lambda `$ sufficiently small, where $`w:=(\mathrm{ln}k_1(\alpha ))^1`$.

We recall that Proposition 3.1 yields $`\alpha ^2\kappa _1^2`$ as $`\lambda 0+`$, and consequently, $`k_1(\alpha )0+`$. Hence the auxiliary variable $`w`$ is well defined and negative for $`\lambda `$ small enough, and $`w0`$ as $`\lambda 0+`$.

By the Birman-Schwinger principle (3.7) eigenvalues of $`H_\lambda `$ correspond to singularities of the operator $`(I+\lambda K(\alpha ))^1`$ which we can equivalently express as

$$\left(I+\lambda K(\alpha )\right)^1=\left[I+\lambda (I+\lambda M_\alpha )^1L_\alpha \right]^1(I+\lambda M_\alpha )^1.$$

Since $`M_\alpha `$ is bounded independently of $`\alpha `$ due to Lemma 3.3(i), we have $`\lambda M_\alpha <1`$ for all sufficiently small $`\lambda `$, and therefore the second term on the right hand side of the last relation is a bounded operator. On the other hand, $`\lambda (I+\lambda M_\alpha )^1L_\alpha `$ is a rank-one operator of the form $`(\psi ,)\phi `$, where

$`\psi (x,y)`$ $`:=`$ $`{\displaystyle \frac{\lambda }{2\pi }}\mathrm{ln}k_1(\alpha )V(x,y)^{\frac{1}{2}}\chi _1(y),`$
$`\phi (x,y)`$ $`:=`$ $`[(I+\lambda M_\alpha )^1|V|^{\frac{1}{2}}\chi _1](x,y),`$
so it has just one eigenvalue which is $`(\psi ,\phi )`$. The requirement that the latter equals $`1`$ yields the implicit equation

$$w=F(\lambda ,w),F(\lambda ,w):=\frac{\lambda }{2\pi }(V^{\frac{1}{2}}\chi _1,\left(I+\lambda M_{\alpha (w)}\right)^1|V|^{\frac{1}{2}}\chi _1),$$
(3.9)
where we use the auxiliary variable $`w`$ defined in Lemma 3.3 which determines the energy via $`\alpha ^2=\kappa _1^2e^{2w^1}`$. Solving (3.9), we arrive at the main result of this section.

###### Theorem 3.4.

Assume *$`\text{a0–3}`$* and exclude the trivial case, $`V0`$. Then $`H_\lambda `$ has for sufficiently small $`\lambda >0`$ exactly one eigenvalue $`E(\lambda )`$ if and only if

$$_^2V_{11}(x)𝑑x0,$$
(3.10)
and in this case we can express it as $`E(\lambda )=\kappa _1^2e^{2w(\lambda )^1},`$ where $`w(\lambda )`$ has the following asymptotic expansion:

$`w(\lambda )`$ $`=`$ $`{\displaystyle \frac{\lambda }{2\pi }}{\displaystyle _^2}V_{11}(x)𝑑x`$ (3.11)
$`+`$ $`\left({\displaystyle \frac{\lambda }{2\pi }}\right)^2\{{\displaystyle _{^2\times ^2}}V_{11}(x)(\gamma _\mathrm{E}+\mathrm{ln}{\displaystyle \frac{|xx^{}|}{2}})V_{11}(x^{})dxdx^{}`$
$`{\displaystyle \underset{j=2}{\overset{\mathrm{}}{}}}{\displaystyle _{^2\times ^2}}V_{1j}(x)K_0(k_j(\kappa _1)|xx^{}|)V_{j1}(x^{})dxdx^{}\}`$
$`+`$ $`𝒪(\lambda ^{2+\gamma })`$
with $`\gamma :=\mathrm{min}\{1,\delta /2\}`$.

###### Proof:.

Inserting the identity

$$(I+\lambda M_\alpha )^1=I\lambda M_{\kappa _1}\lambda (M_\alpha M_{\kappa _1})+\lambda ^2M_\alpha ^2(I+\lambda M_\alpha )^1,$$

into (3.9) and employing Lemma 3.3(i-ii), we get the asymptotic expansion (3.11). Note that its coefficients are well defined owing to $`\text{a3}`$; in particular, for the second-order terms it is true since $`(V^{\frac{1}{2}}\chi _1,M_{\kappa _1}|V|^{\frac{1}{2}}\chi _1)`$ estimated by the Schwarz inequality is finite because of $`\text{a3}`$ and Lemma 3.3(i). The sufficient and necessary condition (3.10) follows from the fact that $`E(\lambda )`$ converges to $`\kappa _1^2`$ as $`\lambda 0+`$ because of Proposition 3.1, which corresponds to the situation that $`w`$ goes to zero assuming *negative* values. In view of (3.11) it is evident that this is the case if $`V_{11}`$ is strictly negative. We want to show that $`w`$ is negative for small $`\lambda `$ also if the first term in the expansion vanishes. Suppose first that the potential projections $`V_{jj^{}}`$ belong to $`L^2(^2)`$. Then we have

$`{\displaystyle _{^2\times ^2}}V_{1j}(x)K_0(k_j(\kappa _1)|xx^{}|)V_{j1}(x^{})𝑑x𝑑x^{}`$ (3.12)
$`=(V_{1j},K_0V_{j1})=(\widehat{V_{1j}},\widehat{K_0V_{j1}})`$
$`=2\pi (\widehat{V_{1j}},\widehat{K_0}\widehat{V_{j1}})=(2\pi )^2{\displaystyle _^2}{\displaystyle \frac{|\widehat{V_{1j}}(\omega )|^2}{|\omega |^2+\kappa _j^2\kappa _1^2}}𝑑\omega >0,`$
because $`(2\pi )^1K_0(k_j(\kappa _1)||)`$ is the Green function of $`\mathrm{\Delta }+k_j(\kappa _1)^2`$. At the same time, by Lemma 3.2(i) and the fact that we deal with the case $`V_{00}=0`$ now,

$`{\displaystyle _{^2\times ^2}}V_{11}(x)\left(\gamma _\mathrm{E}+\mathrm{ln}{\displaystyle \frac{|xx^{}|}{2}}\right)V_{11}(x^{})𝑑x𝑑x^{}`$ (3.13)
$`=\underset{\epsilon 0+}{lim}{\displaystyle _{^2\times ^2}}V_{11}(x)\left(K_0(\epsilon |xx^{}|)+\mathrm{ln}\epsilon \right)V_{11}(x^{})𝑑x𝑑x^{}`$
$`=\underset{\epsilon 0+}{lim}{\displaystyle _{^2\times ^2}}V_{11}(x)K_0(\epsilon |xx^{}|)V_{11}(x^{})𝑑x𝑑x^{}`$
is also positive, which follows by the consecutive use of the Fourier transform trick (3.12). For a general $`V_{jj^{}}L^2(^2)`$ we can approximate the potential by the cut-off functions $`V^N:=\chi _{[N,N]^2}sgnV\mathrm{min}\{|V|,N\},`$ where $`\chi _𝒜`$ denotes the characteristic function of a set $`𝒜`$. Then the expressions in the first line of (3.12) and the last line of (3.13) are approximated by sequences whose elements are positive by the above argument. Using the dominated convergence and the absolute continuity of the Lebesgue integral we find that the limits exist, and of course they are positive again. Together we get that the second-order term in the expansion (3.11) is negative.

It remains to check that (3.11) is the only solution of (3.9) for $`\lambda `$ small. This will be true if we prove it for a non-positive $`V`$ since it represents a more attractive interaction; we thus replace $`V`$ by $`|V|`$ in the expression for $`F`$. Using the Schwarz and triangle inequalities together with Lemma 3.3(i), (iii), we arrive at the estimate

$`\left|{\displaystyle \frac{F}{w}}(\lambda ,w)\right|`$
$`={\displaystyle \frac{\lambda }{2\pi }}\left|(|V|^{\frac{1}{2}}\chi _1,(I+\lambda M_{\alpha (w)})^1\lambda {\displaystyle \frac{dM_{\alpha (w)}}{dw}}(I+\lambda M_{\alpha (w)})^1|V|^{\frac{1}{2}}\chi _1)\right|`$
$`{\displaystyle \frac{\lambda ^2}{2\pi }}|V|^{\frac{1}{2}}\chi _1_2^2\left(I+\lambda M_{\alpha (w)}\right)^1^2{\displaystyle \frac{dM_{\alpha (w)}}{dw}}`$
$`{\displaystyle \frac{\lambda ^2}{2\pi }}(1\lambda C_2)^2{\displaystyle \frac{C_4}{|w|}}|V|_{11}_1^2,`$
which holds for $`\lambda `$ sufficiently small and strictly less than $`C_2^1`$. The norm of the potential is finite by assumption $`\text{a3}`$. Excluding the trivial case $`V0`$ we note that there exists a $`c^{}>0`$ such that the inequality $`|w|^1c^{}\lambda ^1`$ is valid for any solution $`w`$ of the implicit equation (3.9) and $`\lambda `$ small enough. So there is a $`C_5>0`$ such that the partial derivative of $`F`$ w.r.t. $`w`$ is bounded by $`C_5\lambda `$ for all sufficiently small $`\lambda `$. Since any two solutions $`w_1,w_2`$ of the equation $`w=F(\lambda ,w)`$ have to fulfill

$$|w_2w_1|=\left|_{w_1}^{w_2}\frac{F}{w}𝑑w\right|\left|_{w_1}^{w_2}\right|\frac{F}{w}\left|dw\right|C_5\lambda |w_2w_1|,$$

the uniqueness is ensured for $`\lambda <C_5^1`$. ∎

### 3.2 Proof of Lemma 3.3

A reader not interested in the following technical analysis of the operator $`M_\alpha `$ may skip this subsection. We recall that $`M_\alpha `$ is given by the sum of $`A_\alpha `$ and $`B_\alpha `$, which are of a different nature. To prove the assertions of Lemma 3.3 we consider each of the operators separately.

#### 3.2.1 Analysis of $`A_\alpha `$

We show first that this operator is of the Hilbert-Schmidt class for $`\alpha =\kappa _1`$.

###### Lemma 3.5.

Asuume *$`\text{a2}`$*, *$`\text{a3}`$*, then $`A_{\kappa _1}`$ is a Hilbert-Schmidt operator.

###### Proof:.

Let us compute the HS-norm

$$A_{\kappa _1}_{HS}^2=\frac{1}{4\pi ^2}_^2_^2|V|_{11}(x)\left|\gamma _\mathrm{E}+\mathrm{ln}\frac{|xx^{}|}{2}\right|^2|V|_{11}(x^{})𝑑x𝑑x^{}.$$

It can be estimated by a sum of two integrals. The first one will be finite if $`|V|_{11}L^1(^2)`$ which is true by $`\text{a3}`$, so it remains to check that the integral

$$J:=_{^2\times ^2}|V|_{11}(x)\mathrm{ln}^2|xx^{}||V|_{11}(x^{})𝑑x𝑑x^{}$$

is finite. To estimate it, we divide the region of integration into two parts: if $`|xx^{}|1`$, then for any $`\delta >0`$ there is a $`C_\delta >0`$ such that

$$\mathrm{ln}^2|xx^{}|\mathrm{ln}^2(|x|+|x^{}|)\mathrm{ln}^2(1+|x|)(1+|x^{}|)C_\delta (1+|x|^\delta )(1+|x^{}|^\delta )$$

and the contribution to $`J`$ is thus finite because of $`\text{a3}`$. On the other hand, for $`|xx^{}|<1`$ we use the Hölder and Young inequalities

$`J`$ $`=`$ $`{\displaystyle _^2}𝑑x|V|_{11}(x){\displaystyle _^2}𝑑x^{}\chi _{[0,1]}(|xx^{}|)\mathrm{ln}^2|xx^{}||V|_{11}(x^{})`$
$``$ $`|V|_{11}_{1+\delta }\chi _{[0,1]}\mathrm{ln}^2|V|_{11}_{1+\delta ^1}`$
$``$ $`|V|_{11}_{1+\delta }^2\chi _{[0,1]}\mathrm{ln}_r=\mathrm{\Gamma }(r+1)^{\frac{1}{r}}|V|_{11}_{1+\delta }^2,`$
where $`r:=(1+\delta ^1)/2`$, which yields a finite value owing to $`\text{a2}`$. ∎

In a similar way we can estimate the HS-norm of $`A_\alpha A_{\kappa _1}`$.

###### Lemma 3.6.

Assume *$`\text{a3}`$*, then $`A_\alpha A_{\kappa _1}_{HS}C_6k_1(\alpha )^\delta `$ holds with a positive constant $`C_6`$ independent of $`\alpha `$.

###### Proof:.

Using the decomposition of Lemma 3.2,

$`(A_\alpha A_{\kappa _1})(x,y,x^{},y^{})`$
$`=|V(x,y)|^{\frac{1}{2}}\chi _1(y)\{[f(k_1(\alpha )|xx^{}|)f(0)]\mathrm{ln}k_1(\alpha )|xx^{}|`$
$`+g(k_1(\alpha )|xx^{}|)g(0)\}\chi _1(y^{})V(x^{},y^{})^{\frac{1}{2}},`$
which yields the estimate $`A_\alpha A_{\kappa _1}_{HS}^22(J_1^2+J_2^2)`$, where

$$J_{\mathrm{}}:=_{^2\times ^2}|V|_{11}(x)h_{\mathrm{}}(k_1(\alpha )|xx^{}|)|V|_{11}(x^{})𝑑x𝑑x^{},\mathrm{}=1,2,$$

with $`h_1(u):=[(f(u)f(0))\mathrm{ln}u]^2`$ and $`h_2(u):=[g(u)g(0)]^2`$. Since these functions are bounded, both integrals are finite under the assumption $`|V|_{11}L^1(^2)`$. Consequently, $`A_\alpha A_{\kappa _1}_{HS}`$ has a bound independent of $`\alpha `$. However, we want to prove in addition that $`A_\alpha `$ converges in HS-norm to $`A_{\kappa _1}`$, *i.e.*, the stated assertion. By virtue of Lemma 3.2, we have for any $`\delta (0,2)`$ the rough bounds $`h_{\mathrm{}}(u)c_{\mathrm{}}u^\delta `$ for some constants $`c_{\mathrm{}}>0`$. This bound together with $`|xx^{}|^\delta \mathrm{max}\{1,2^{\delta 1}\}(|x|^\delta +|x^{}|^\delta )`$ and $`|x|^\delta +|x^{}|^\delta (1+|x|^\delta )(1+|x^{}|^\delta )`$, yields the estimate

$$J_{\mathrm{}}\stackrel{~}{C}_6k_1(\alpha )^\delta \left(_^2|V|_{11}(x)(1+|x|^\delta )𝑑x\right)^2,$$

where $`\stackrel{~}{C}_6:=\mathrm{max}\{1,2^{\delta 1}\}\mathrm{max}\{c_1,c_2\}`$. Since the integral is finite by $`\text{a3}`$, we arrive at the sought result. ∎

It is now easy to check that the two preceding lemmas imply the claims (i) and (ii) of Lemma 3.3 for the operator $`A_\alpha `$, since $`_{HS}`$ and $`k_1(\alpha )=𝒪(\lambda ^{\frac{1}{2}})`$ as it follows from Proposition 3.1.

It is also easy to see that $`\alpha A_\alpha `$ considered as the operator-valued function is real-analytic in $`[0,\kappa _1)`$. It cannot be analytically continued to an open interval containing $`\kappa _1`$, however, if we consider instead the function $`wA_{\alpha (w)}`$ with the auxiliary variable $`w=(\mathrm{ln}k_1(\alpha ))^1`$, this one can be continued to a complex region that includes $`w=0`$, which is the point of our interest because it is obtained in the limit $`\lambda 0`$. We use this fact to estimate the norm of the derivative w.r.t. $`w`$, which establishes the claim (iii) of Lemma 3.3 for the operator $`A_{\alpha (w)}`$.

###### Lemma 3.7.

Assume *$`\text{a2}`$* and *$`\text{a3}`$*, then for sufficiently small $`w`$,

$$\frac{dA_{\alpha (w)}}{dw}<\frac{C_7}{|w|}\text{with some}C_7>0.$$

###### Proof:.

Let us choose the contour $`\mathrm{\Gamma }:=\{z=w+we^{it}:t[0,2\pi )\}`$ in the complex plane, where $`w`$ is supposed to be negative and small enough so that $`A_{\alpha (w)}`$ is analytic in the interior of the curve. We can use the Cauchy integral formula to estimate the complex derivative

$$\frac{dA_{\alpha (w)}}{dw}=\frac{1}{2\pi i}_\mathrm{\Gamma }\frac{A_{\alpha (z)}}{(zw)^2}𝑑z\frac{sup_{z\mathrm{\Gamma }}A_{\alpha (z)}}{|w|}.$$

It is straightforward to modify the proof of Lemma 3.6 (including the technical Lemma 3.2) in order to check the continuity of $`A_\alpha `$ in $`\kappa _1`$ w.r.t. to the HS-norm also for complex values of $`\alpha `$. This yields the desired result. ∎

#### 3.2.2 Analysis of $`B_\alpha `$

Mimicking \[BGRS, Lemma 2.2\] and \[BEGK, Lemma 2.3\] we denote as $`_1`$ the subspace $`L^2(^2)\{\chi _1\}`$. Let $`𝒫_1`$ be the corresponding projection and $`𝒫_1^{}:=I𝒫_1`$, then we can write $`R_0^{}(\alpha )`$ defined in (3.8) as $`𝒫_1^{}R_0(\alpha )𝒫_1^{}`$.

###### Lemma 3.8.

Assume *$`\text{a1}`$*, then $`\alpha B_\alpha `$ is uniformly bounded in the operator-norm topology on the interval $`[0,\kappa _2\epsilon ]`$ for any $`\epsilon (0,\kappa _2]`$.

###### Proof:.

Since the lowest point in the spectrum of $`H_0𝒫_1^{}𝒫_1^{}`$ is $`\kappa _2^2`$, the operator-valued function $`R_0^{}()`$ has an analytic continuation into the region $`\{\alpha |\alpha ^2[\kappa _2^2,\mathrm{})\}`$. In particular, this region includes the interval $`[0,\kappa _1]`$ actually considered, where one has the following estimate on the norm

$$R_0^{}(\alpha )=\underset{j=2,3,\mathrm{}}{sup}(\kappa _j^2\alpha ^2)^1=(\kappa _2^2\alpha ^2)^1<(\kappa _2^2\kappa _1^2)^1.$$

If $`V`$ was essentially bounded then this result would be sufficient for the boundedness of $`B_\alpha `$ because $`B_\alpha V^{\frac{1}{2}}_{\mathrm{}}R_0^{}(\alpha )V^{\frac{1}{2}}_{\mathrm{}}`$. In order to accommodate the extra factors $`|V|^{\frac{1}{2}},V^{\frac{1}{2}}`$ under our weaker assumption $`\text{a1}`$, we introduce the quadratic form

$$b_\alpha (\varphi ,\psi ):=(\varphi ,B_\alpha \psi )=(R_0^{}(\alpha )^{\frac{1}{2}}𝒫_1^{}|V|^{\frac{1}{2}}\varphi ,R_0^{}(\alpha )^{\frac{1}{2}}𝒫_1^{}V^{\frac{1}{2}}\psi )$$

with $`\alpha `$ supposed to be a real number from an interval $`[0,\kappa _2\epsilon ]`$. To check boundedness of this form, it is sufficient to verify that $`R_0^{}(\alpha )^{\frac{1}{2}}𝒫_1^{}|V|^{\frac{1}{2}}`$ is a bounded operator, *i.e.*, that $`|V|^{\frac{1}{2}}`$ is $`(R_0^{}(\alpha )^{\frac{1}{2}}𝒫_1^{})`$-bounded, which is equivalent to the statement that there exist $`c_1,c_20`$ such that

$$\psi DomQ_0:V^{\frac{1}{2}}𝒫_1^{}\psi ^2c_1\psi ^2+c_2R_0^{}(\alpha )^{\frac{1}{2}}𝒫_1^{}\psi ^2.$$

However,

$`V^{\frac{1}{2}}𝒫_1^{}\psi ^2`$ $`=`$ $`(𝒫_1^{}\psi ,|V|𝒫_1^{}\psi )𝒫_1^{}\psi V𝒫_1^{}\psi `$
$``$ $`𝒫_1^{}\psi \left(a𝒫_1^{}\psi +bH_0^{\frac{1}{2}}𝒫_1^{}\psi \right)`$
$``$ $`\left(a+{\displaystyle \frac{b}{2}}\right)\psi ^2+{\displaystyle \frac{b}{2}}H_0^{\frac{1}{2}}𝒫_1^{}\psi ^2.`$
In the first step we have used the Schwarz inequality, in the second our hypothesis $`\text{a1}`$, and finally the inequality between the geometric and arithmetic means together with $`𝒫_1^{}\psi \psi `$. At the same time,

$$R_0^{}(\alpha )^{\frac{1}{2}}𝒫_1^{}\psi ^2=(𝒫_1^{}\psi ,(H_0\alpha ^2)𝒫_1^{}\psi )H_0^{\frac{1}{2}}𝒫_1^{}\psi ^2,$$

so we can identify $`c_1:=a+b/2`$ and $`c_2:=b/2`$ which completes our proof. ∎

This establishes Lemma 3.3(i) for the operator $`B_\alpha `$. Since $`k_1(\alpha )^2c\lambda `$ owing to Proposition 3.1, the property (ii) is included in the following result.

###### Lemma 3.9.

Assume *$`\text{a1}`$*, then there exists $`C_7>0`$ such that

$$B_\alpha B_{\kappa _1}C_7k_1(\alpha )^2.$$

###### Proof:.

Using the first resolvent identity \[Wei, Thm. 5.13\], we infer

$$\begin{array}{c}B_\alpha B_{\kappa _1}=|\alpha \kappa _1||V|^{\frac{1}{2}}R_0^{}(\alpha )R_0^{}(\kappa _1)V^{\frac{1}{2}}\hfill \\ \hfill |\alpha \kappa _1||V|^{\frac{1}{2}}R_0^{}(\alpha )^{\frac{1}{2}}R_0^{}(\alpha )^{\frac{1}{2}}R_0^{}(\kappa _1)^{\frac{1}{2}}R_0^{}(\kappa _1)^{\frac{1}{2}}V^{\frac{1}{2}}.\end{array}$$

However, $`|\kappa _1\alpha |\kappa _1^1k_1(\alpha )^2`$ and it is clear from the proof of Lemma 3.8 that the remaining factors at the r.h.s. of the above estimate are finite. ∎

Similarly to the operator $`A_\alpha `$, we need also a bound on the derivative of $`B_{\alpha (w)}`$ w.r.t. $`w`$. However, the situation in the present case is simpler because $`R_0^{}(\alpha )`$ itself is analytic in an open complex set containing $`\alpha =\kappa _1`$.

###### Lemma 3.10.

Assume *$`\text{a1}`$*, then for sufficiently small $`w`$,

$$\frac{dB_{\alpha (w)}}{dw}C_8\text{with some}C_8>0.$$

###### Proof:.

Since

$$\frac{dB_{\alpha (w)}}{dw}=\frac{dB_\alpha }{d\alpha }\frac{d\alpha }{dw}=2\frac{e^{2w^1}}{w^2}|V|^{\frac{1}{2}}R_0^{}(\alpha (w))^2V^{\frac{1}{2}}$$

and the prefactor function $`w^2e^{2w^1}`$ is uniformly bounded in $`(\mathrm{},0)`$, we can employ the Schwarz inequality to get the estimate

$$|V|^{\frac{1}{2}}R_0^{}(\alpha )^2V^{\frac{1}{2}}|V|^{\frac{1}{2}}R_0^{}(\alpha )^{\frac{1}{2}}R_0^{}(\alpha )R_0^{}(\alpha )V^{\frac{1}{2}}.$$

It is clear from the proof of Lemma 3.8 that the right hand side of this inequality is uniformly bounded in $`[0,\kappa _2\epsilon ]`$ for any $`\epsilon (0,\kappa _2]`$. The former interval contains a neighbourhood of $`\kappa _1`$ in which $`w`$ is well defined. ∎

This yields the remaining assertion (iii) of Lemma 3.3 for $`|w|<1`$.

## 4 Mildly Curved Layers

Our strategy in proving the ground-state asymptotic expansion in mildly curved layers $`\mathrm{\Omega }_\epsilon `$ will be to estimate the corresponding Hamiltonian $`\mathrm{\Delta }_D^{\mathrm{\Omega }_\epsilon }`$ by an operator of the form $`\mathrm{\Delta }\mathrm{\Delta }_D^I+\epsilon V`$ and to apply Theorem 3.4 to the latter. Here $`\mathrm{\Delta }`$ is the Laplacian in the plane, $`\mathrm{\Delta }_D^I`$ is the transverse operator which is the particular case of $`\mathrm{\Delta }_D^M`$ discussed in the preceding section, and $`V`$ is an effective potential given by curvatures of $`\mathrm{\Sigma }_\epsilon `$.

### 4.1 The Geometry

The family of metric tensors for the surfaces given by (2.1) has the form

$$g_{\mu \nu }(\epsilon )=\delta _{\mu \nu }+\epsilon ^2\eta _{\mu \nu }\text{with}(\eta _{\mu \nu }):=\left(\begin{array}{cc}f_{,1}^{}{}_{}{}^{2}& f_{,1}f_{,2}\\ f_{,1}f_{,2}& f_{,2}^{}{}_{}{}^{2}\end{array}\right),$$

where the symbol $`\delta _{\mu \nu }`$ (as well as $`\delta ^{\mu \nu }`$) has to be understood as the identity matrix. Since $`det(\eta _{\mu \nu })=0`$ we get immediately

$$g(\epsilon ):=det(g_{\mu \nu })=1+\epsilon ^2tr(\eta _{\mu \nu })=1+\epsilon ^2(f_{,1}^{}{}_{}{}^{2}+f_{,2}^{}{}_{}{}^{2})$$
(4.1)
together with the expression for the inverse matrix

$$g^{\mu \nu }(\epsilon )=g(\epsilon )^1(\delta ^{\mu \nu }+\epsilon ^2\stackrel{~}{\eta }^{\mu \nu })\text{with}(\stackrel{~}{\eta }^{\mu \nu }):=\left(\begin{array}{cc}f_{,2}^{}{}_{}{}^{2}& f_{,1}f_{,2}\\ f_{,1}f_{,2}& f_{,1}^{}{}_{}{}^{2}\end{array}\right).$$

In particular, the Jacobian $`g^{\frac{1}{2}}`$ defines through $`d\mathrm{\Sigma }_\epsilon :=g^{\frac{1}{2}}dx`$ the invariant surface element. We will suppose that the matrix function $`\eta _{\mu \nu }`$ is bounded. Since its eigenvalues are $`0`$ and $`f_{,1}^{}{}_{}{}^{2}+f_{,2}^{}{}_{}{}^{2}`$, this will be true provided the second eigenvalue is a bounded function in $`^2`$, *i.e.*, if we adopt the assumption $`\text{d1}`$. Denote the bound of $`\eta _{\mu \nu }`$ by $`\eta _{\mathrm{}}`$. Then it follows that $`g_{\mu \nu }(\epsilon )`$ is uniformly elliptic for sufficiently small $`\epsilon `$ because

$$c_{}\delta _{\mu \nu }g_{\mu \nu }(\epsilon )c_+\delta _{\mu \nu }\text{with}c_\pm :=1\pm \epsilon ^2\eta _{\mathrm{}}.$$
(4.2)
At the same time, the vector $`n(\epsilon )=g(\epsilon )^{\frac{1}{2}}(\epsilon f_{,1},\epsilon f_{,2},1)`$ represents the surface normal and therefore the second fundamental form is given by

$$h_{\mu \nu }(\epsilon )=\epsilon g(\epsilon )^{\frac{1}{2}}\theta _{\mu \nu }\text{with}(\theta _{\mu \nu }):=\left(\begin{array}{cc}f_{,11}& f_{,12}\\ f_{,21}& f_{,22}\end{array}\right).$$

We can construct now the Weingarten tensor \[Kli, Def. 3.3.4 & Prop. 3.5.5\]

$$h_\mu ^\nu (\epsilon ):=h_{\mu \rho }g^{\rho \nu }=\epsilon g(\epsilon )^{\frac{3}{2}}\left(\theta _{\mu \rho }\delta ^{\rho \nu }+\epsilon ^2\theta _{\mu \rho }\stackrel{~}{\eta }^{\rho \nu }\right),$$

which determines respectively the Gauss curvature $`K`$ and the mean curvature $`M`$ of the surface $`\mathrm{\Sigma }_\epsilon `$:

$$\begin{array}{c}\begin{array}{cccc}\hfill K(\epsilon )& =\epsilon ^2g(\epsilon )^2k_0\hfill & \hfill \text{with}& k_0:=det(\theta _{\mu \nu })=f_{,11}f_{,22}f_{,12}^{}{}_{}{}^{2},\hfill \\ \hfill M(\epsilon )& =\epsilon g(\epsilon )^{\frac{3}{2}}\left(m_0+\epsilon ^2m_1\right)\hfill & \hfill \text{with}& m_0:=\frac{1}{2}tr(\theta _{\mu \nu })=\frac{1}{2}\left(f_{,11}+f_{,22}\right)\hfill \end{array}\hfill \\ \hfill \text{and}m_1:=\frac{1}{2}tr(\theta _{\mu \rho }\stackrel{~}{\eta }^{\rho \nu })=\frac{1}{2}\left(f_{,1}^{}{}_{}{}^{2}f_{,22}+f_{,2}^{}{}_{}{}^{2}f_{,11}2f_{,1}f_{,2}f_{,12}\right).\end{array}$$
(4.3)
Since we are interested in the case when $`\mathrm{\Sigma }_\epsilon `$ is asymptotically planar \[DEK2\], *i.e.*, when $`K,M0`$ as $`|x|\mathrm{}`$, we assume $`\text{d2}`$.

It is clear from the definition (2.2) that the metric tensor of the layer $`\mathrm{\Omega }_\epsilon `$ (as a manifold with a boundary in $`^3`$) has the block form

$$(G_{ij})=\left(\begin{array}{cc}(G_{\mu \nu })& 0\\ 0& 1\end{array}\right)\text{with}G_{\nu \mu }=(\delta _\nu ^\sigma uh_\nu ^\sigma )(\delta _\sigma ^\rho uh_\sigma ^\rho )g_{\rho \mu },$$
(4.4)
and thus $`G:=det(G_{ij})=g(12Mu+Ku^2)^2`$. Following \[DEK2\], one should make sure that the layer mapping $``$ is a diffeomorphism. However, this is automatically fulfilled for an arbitrary $`a`$ provided $`\epsilon `$ is small enough and the layer is built over surfaces of the special form (2.1). At the same time, $`G_{\mu \nu }`$ can be estimated by the surface metric,

$$C_{}g_{\mu \nu }G_{\mu \nu }C_+g_{\mu \nu }\text{with}C_\pm :=\left(1\pm a\rho _m^1\right)^2,$$
(4.5)
where $`\rho _m^1:=\mathrm{max}\{k_1_{\mathrm{}},k_2_{\mathrm{}}\}=𝒪(\epsilon )`$ and $`k_1,k_2`$ are the principal curvatures of $`\mathrm{\Sigma }_\epsilon `$, *i.e.*, the eigenvalues of $`h_\mu ^\nu `$.

### 4.2 The Hamiltonian

In the beginning we have identified the particle Hamiltonian with $`\mathrm{\Delta }_D^{\mathrm{\Omega }_\epsilon }`$. We want to replace it by a Schrödinger-type operator which would allow us to employ the result of the previous section. This is achieved by means of the unitary transformation

$$U:L^2(\mathrm{\Omega }_\epsilon )L^2(\mathrm{\Omega }_0,g^{\frac{1}{2}}dxdu):\{\psi U\psi :=(12Mu+Ku^2)^{\frac{1}{2}}\psi \},$$

which leads to the unitarily equivalent operator

$`H`$ $`:=U(\mathrm{\Delta }_D^{\mathrm{\Omega }_\epsilon })U^1=g^{\frac{1}{2}}_ig^{\frac{1}{2}}G^{ij}_j+V`$
$`V`$ $`=g^{\frac{1}{2}}(g^{\frac{1}{2}}G^{ij}J_{,j})_{,i}+J_{,i}G^{ij}J_{,j}\text{with}J:=\mathrm{ln}\sqrt{12Mu+Ku^2}.`$
It makes sense since we suppose that the surface is $`C^4`$-smooth. Using the block form (4.4) of $`G_{ij}`$, we can split $`H`$ into a sum, $`H=H_1+H_2`$. For the first operator which is given by the part of $`H`$ where one sums over the Greek indices (referring to the longitudinal coordinates) we have owing to (4.5) the estimate

$$(C_{}/C_+^2)\left(\mathrm{\Delta }_g+v_1\right)H_1(C_+/C_{}^2)\left(\mathrm{\Delta }_g+v_1\right)$$
(4.6)
which holds in $`L^2(^2\times I,g^{\frac{1}{2}}dxdu)`$ in the form sense. Here $`\mathrm{\Delta }_g`$ denotes the surface Laplace-Beltrami operator which can be written in the component form as $`g^{\frac{1}{2}}_\mu g^{\frac{1}{2}}g^{\mu \nu }_\nu `$, and

$$v_1:=\frac{|u^2_gK2u_gM|_g^2}{4(12Mu+Ku^2)^2}+\frac{u^2\mathrm{\Delta }_gK2u\mathrm{\Delta }_gM}{2(12Mu+Ku^2)},$$
(4.7)
where $`||_g`$ and $`_g`$ mean respectively the norm and the gradient operator induced by the metric $`g_{\mu \nu }`$. On the other hand,

$$H_2=_3^2+V_2,V_2=\frac{KM^2}{(12Mu+Ku^2)^2},$$
(4.8)
where the potential $`V_2`$ is attractive because $`KM^2`$ can be rewritten by means of the principal curvatures as $`\frac{1}{4}(k_1k_2)^2`$.

In the second step one uses the inequalities (4.2), which are a consequence of $`\text{d1}`$, in order to get the bounds

$$(c_{}^2/c_+^3)\left(\mathrm{\Delta }\right)\mathrm{\Delta }_g(c_+^2/c_{}^3)\left(\mathrm{\Delta }\right)\text{in}L^2(^2,g^{\frac{1}{2}}dx).$$

It remains to realize that – as another consequence of the uniform ellipticity of the metric – one can identify $`L^2(^2,g^{\frac{1}{2}}dx)=L^2(^2)`$, and also $`L^2(^2\times I,g^{\frac{1}{2}}dxdu)=L^2(^2\times I)`$ as sets. If we rescale the longitudinal variable by means of $`x\sigma _\pm x`$ with $`\sigma _\pm ^2:=c_{}^3C_{}^2/(c_\pm ^2C_\pm )`$, we obtain finally the above indicated bounds

$$H_{}HH_+\text{with}H_\pm :=\mathrm{\Delta }_3^2+\epsilon V_\pm ,$$
(4.9)
where

$$V_\pm (x,u):=\frac{1}{\epsilon }\left(\frac{C_\pm }{C_{}^2}v_1+V_2\right)(x/\sigma _\pm ,u).$$

We notice that since $`v_1`$ and $`V_2`$ are $`\epsilon `$-dependent – *cf.* (4.3) and the explicit formulae for the potentials (4.7) and (4.8) – the potentials $`V_\pm `$ are well defined for $`\epsilon =0`$.

### 4.3 The Ground State Asymptotics

Suppose that each of the operators $`H_\pm `$ has for all sufficiently small $`\epsilon `$ just one eigenvalue $`E_\pm (\epsilon )`$. Since $`H_{}`$ is below bounded, the minimax principle tells us that the same is true for $`H`$ and

$$E_{}(\epsilon )E(\epsilon )E_+(\epsilon ).$$
(4.10)
If we identify $`\epsilon `$ with $`\lambda `$ of the previous section, we see that under our assumptions the operators $`H_\pm `$ are well suited for application of Theorem 3.4.

Indeed, the decay requirements $`\text{d1–4}`$ yield that $`K,M,|_gK|_g,|_gM|_g`$ and $`\mathrm{\Delta }_gK`$ tend to zero at infinity, and consequently, $`(V_\pm )_{11}0`$ as $`|x|\mathrm{}`$. Notice that we do not assume a decay of $`\mathrm{\Delta }_gM`$ at infinity which would require $`f_{,\mu \nu \rho \sigma }0`$ as $`|x|\mathrm{}`$. The reason is that the term $`u\mathrm{\Delta }_gM`$ itself in the potential (4.7) vanishes when projected onto the first transverse mode $`\chi _1`$. One can thus take $`\psi _n(x,u):=\phi _n(x)\chi _1(u)`$, where $`\phi _n`$ with $`\phi _n=1`$ is a sequence of functions with increasing supports which move towards infinity as $`n\mathrm{}`$. If $`\phi _n`$ is properly chosen such $`\psi _n`$ form a Weyl sequence for $`H_\pm `$ and any $`\kappa _1^2+\alpha `$ with $`\alpha ^+`$. Using then a Neumann bracketing argument, as mentioned in Section 2, in order to show that no $`\kappa _1^2+\alpha `$ with $`\alpha <0`$ can belong to the essential spectrum, we conclude that $`\sigma _{\mathrm{ess}}(H_\pm )=[\kappa _1^2,\mathrm{})`$. This verifies $`\text{a0}`$ of Theorem 3.4. Furthermore, using this result together with (4.9) and the minimax principle, we infer that $`inf\sigma _{\mathrm{ess}}(H)=\kappa _1^2`$.

The requirement $`\text{a1}`$ holds trivially true because $`V_\pm `$ are essentially bounded due to the surface regularity assumption and $`\text{d1–4}`$. Under $`\text{r1–3}`$, explicit expressions for the covariant derivatives of $`K`$ and $`M`$ in terms of the partial derivatives of the function $`f`$ show that even $`sup\{V_\pm (,u)|uI\}`$ belongs to $`L^1(^2,(1+|x|^\delta )dx)`$ which gives $`\text{a3}`$. Let us remark that we require the $`L^1`$-integrability in $`\text{r3}`$, which is a stronger condition than the $`L^2`$-integrability required for the second and third derivatives of $`f`$. The assumption $`\text{a2}`$ is a consequence of $`\text{a3}`$ and the boundedness of $`V_\pm `$.

On the other hand, there is a difference between the Hamiltonian (3.5) and our operators $`H_\pm `$ because the potentials $`V_\pm `$ depend themselves on $`\epsilon `$. Inspecting the proof of Theorem 3.4 we see that one can modify it to get the expansion (3.11) using $`\epsilon V_\pm (\epsilon )`$ as well. However, expanding the latter in terms of $`\epsilon `$, the final expansion resulting from (3.11) changes and to ensure that it represents an eigenvalue it is necessary that the lowest-order term in this expansion is not only non-positive but rather strictly negative. Recall that one can find (physically interesting) examples of situations when the potential depends on the coupling parameter in a nonlinear way and a bound state in the critical zero-mean case may or may not exist \[BCEZ\].

Hence one has to examine carefully the behaviour with respect to powers of $`\epsilon `$ in the expansion obtained from (3.11) by an appropriate change of the integration variables,

$`w_\pm (\epsilon )`$ $`=`$ $`{\displaystyle \frac{\epsilon }{2\pi }}\sigma _\pm ^2{\displaystyle _^2}(V_\pm )_{11}(\sigma _\pm x;\epsilon )𝑑x`$
$`+`$ $`\left({\displaystyle \frac{\epsilon }{2\pi }}\right)^2\sigma _\pm ^4\{{\displaystyle _{^2\times ^2}}(V_\pm )_{11}(\sigma _\pm x;\epsilon )(\gamma _\mathrm{E}+\mathrm{ln}{\displaystyle \frac{\sigma _\pm |xx^{}|}{2}})`$
$`\times (V_\pm )_{11}(\sigma _\pm x^{};\epsilon )dxdx^{}`$
$``$ $`{\displaystyle \underset{j=2}{\overset{\mathrm{}}{}}}{\displaystyle _{^2\times ^2}}(V_\pm )_{1j}(\sigma _\pm x;\epsilon )K_0\left(k_j(\kappa _1)\sigma _\pm |xx^{}|\right)`$
$`\times (V_\pm )_{j1}(\sigma _\pm x^{};\epsilon )dxdx^{}\}+𝒪(\epsilon ^{2+\gamma }),`$
which determines the possible eigenvalues of $`H_\pm `$ via $`E_\pm =\kappa _1^2\mathrm{exp}(2w_\pm ^1)`$. By virtue of (4.3) we can write

$$\epsilon ^1V_2(x,u;\epsilon )=\epsilon \left(k_0(x)m_0(x)^2\right)+r_2(x,u;\epsilon ),$$

where $`r_2`$ is an integrable function collecting the terms of order $`\epsilon ^2`$ and higher. Hence

$$j:\epsilon ^1(V_2)_{1j}(x;\epsilon )=[\epsilon (k_0(x)m_0(x)^2)+𝒪(\epsilon ^2)]\delta _{1j},$$

where we abuse the notation a little because the error term depends on $`x`$ as well. The expansion of $`\epsilon ^1v_1`$ requires more attention because $`\epsilon ^1\mathrm{\Delta }_gM`$ in the second term of (4.7) is of order one. First of all, we can write

$`v_1(x,u;\epsilon )=u(\mathrm{\Delta }_gM)(x;\epsilon )`$
$`+u^2({\displaystyle \frac{1}{2}}\mathrm{\Delta }_gK|_gM|_g^22M\mathrm{\Delta }_gM)(x;\epsilon )+r_1(x,u;\epsilon ),`$
where $`r_1`$ is an integrable function of order $`𝒪(\epsilon ^3)`$. This does not mean, of course, that the first two terms constitute a quadratic polynomial in $`\epsilon `$, because $`||_g`$, $`_g`$ and $`\mathrm{\Delta }_g`$ expand as well. Since the first term in the above expansion of $`v_1`$ is an odd function of $`u`$, it does not contribute to $`(V_\pm )_{11}`$. On the other hand, it plays an important role in the higher modes:

$`(v_1)_{11}(x;\epsilon )`$ $`=`$ $`u\chi _1_𝐼^2({\displaystyle \frac{1}{2}}\mathrm{\Delta }_gK|_gM|_g^22M\mathrm{\Delta }_gM)(x;\epsilon )+𝒪(\epsilon ^3),`$
$`(v_1)_{1j}(x;\epsilon )`$ $`=`$ $`(\chi _1,u\chi _j)_𝐼(\mathrm{\Delta }_gM)(x;\epsilon )+𝒪(\epsilon ^2)\text{if}j\{1\}.`$
Notice that the inner product in the second line is non-zero for even $`j`$ only. Now we use the relations

$$|_gM|_g^2=|M|^2+\epsilon ^2M_{,\mu }\stackrel{~}{\eta }^{\mu \nu }M_{,\nu }\text{and}\mathrm{\Delta }_g=\mathrm{\Delta }+\epsilon ^2L(\epsilon ),$$

where $`L(\epsilon )`$ is a second-order differential operator with coefficients which expand as $`𝒪(1)`$. This together with (4.3) yields

$`\epsilon ^1(v_1)_{11}(x;\epsilon )`$ $`=`$ $`\epsilon u\chi _1_𝐼^2({\displaystyle \frac{1}{2}}\mathrm{\Delta }k_0|m_0|^22m_0\mathrm{\Delta }m_0)(x)+𝒪(\epsilon ^2)`$
$`\epsilon ^1(v_1)_{1j}(x;\epsilon )`$ $`=`$ $`(\chi _1,u\chi _j)_𝐼(\mathrm{\Delta }m_0)(x)+𝒪(\epsilon ).`$
Since $`C_\pm /C_{}^2`$ and $`\sigma _\pm `$ expand like $`1+𝒪(\epsilon )`$, we arrive at

$`w_\pm (\epsilon )`$ $`=`$ $`{\displaystyle \frac{\epsilon ^2}{2\pi }}\{{\displaystyle _^2}(k_0m_0^2)(x)dx`$ (4.11)
$`+`$ $`u\chi _1_𝐼^2{\displaystyle _^2}({\displaystyle \frac{1}{2}}\mathrm{\Delta }k_0|m_0|2m_0\mathrm{\Delta }m_0)(x)𝑑x`$
$``$ $`{\displaystyle \frac{1}{2\pi }}{\displaystyle \underset{j=2}{\overset{\mathrm{}}{}}}(\chi _1,u\chi _j)_𝐼^2{\displaystyle _{^2\times ^2}}(\mathrm{\Delta }m_0)(x)K_0\left(k_j(\kappa _1)\sigma _\pm |xx^{}|\right)`$
$`\times (\mathrm{\Delta }m_0)(x^{})dxdx^{}\}+𝒪(\epsilon ^{2+\gamma }),`$
where the sum runs in fact over even $`j`$ only.

This expression can be further simplified. By a double integration by parts where one uses the fact that $`f_{,\mu }f_{,\nu \rho }0`$ as $`|x|\mathrm{}`$ due to $`\text{d1}`$ and $`\text{d2}`$, we get that the integral of $`k_0`$ is equal to zero. We note in this connection that it follows then from (4.1) and (4.3) that the total Gauss curvature, $`𝒦:=_{\mathrm{\Sigma }_\epsilon }K𝑑\mathrm{\Sigma }_\epsilon `$, behaves as $`𝒪(\epsilon ^4)`$. Using the divergence theorem and the assumptions $`\text{d1}`$, $`\text{d2}`$ and $`\text{d3}`$ by which $`K0`$ as $`|x|\mathrm{}`$, we find that the integral from $`\mathrm{\Delta }k_0`$ does not contribute as well. A similar argument employing the Green formula gives

$$_^2(m_0\mathrm{\Delta }m_0)(x)𝑑x=_^2|m_0|^2(x)𝑑x.$$
(4.12)
It is convenient to rewrite the integral part of the last term in (4.11) by means of a convolution

$$\begin{array}{c}\frac{1}{2\pi }_{^2\times ^2}(\mathrm{\Delta }m_0)(x)K_0\left(k_j(\kappa _1)\sigma _\pm |xx^{}|\right)(\mathrm{\Delta }m_0)(x^{})𝑑x𝑑x^{}\hfill \\ \hfill =(\mathrm{\Delta }m_0,G_k\mathrm{\Delta }m_0),\end{array}$$

where $`G_k():=(2\pi )^1K_0(k||)`$ and $`k`$ abbreviates $`k_j(\kappa _1)\sigma _\pm `$. Since $`G_k`$ is the fundamental solution of the distributional equation $`(\mathrm{\Delta }+k^2)G_k=\delta `$, we get

$`(\mathrm{\Delta }m_0,G_k\mathrm{\Delta }m_0)=(\mathrm{\Delta }m_0,\mathrm{\Delta }G_km_0)=(\mathrm{\Delta }m_0,(k^2G_k\delta )m_0)`$ (4.13)
$`=k^2(\mathrm{\Delta }m_0,G_km_0)(\mathrm{\Delta }m_0,m_0)=k^2(G_k\mathrm{\Delta }m_0,m_0)+m_0^2`$
$`=k^2(\mathrm{\Delta }G_km_0,m_0)+m_0^2=k^2((k^2G_k\delta )m_0,m_0)+m_0^2`$
$`=k^4(m_0,G_km_0)k^2m_0^2+m_0^2.`$
In the second line we have employed the identity (4.12). If we insert the obtained expression into the expansion (4.11) and use, in addition, the Parseval identity,

$$u\chi _1_𝐼^2=\underset{j=1}{\overset{\mathrm{}}{}}(\chi _1,u\chi _j)_𝐼^2,$$
(4.14)
we find that the terms containing $`m_0`$ cancel, hence we arrive at

$`w_\pm (\epsilon )`$ $`=`$ $`{\displaystyle \frac{\epsilon ^2}{2\pi }}\left\{m_0^2+{\displaystyle \underset{j=2}{\overset{\mathrm{}}{}}}(\chi _1,u\chi _j)_𝐼^2k^2\left[k^2(m_0,G_km_0)m_0^2\right]\right\}`$
$`+𝒪(\epsilon ^{2+\gamma }),`$
where, of course, $`k`$ expands as $`k_j(\kappa _1)+𝒪(\epsilon )`$. Since

$$\underset{j=2}{\overset{\mathrm{}}{}}(\chi _1,u\chi _j)_𝐼^2k_j(\kappa _1)^2=u_3\chi _1_𝐼^2\kappa _1^2u\chi _1_𝐼^2=1,$$

we see that also the terms containing $`m_0`$ cancel. Using finally the Fourier transformation as in (3.12), we obtain

$$w_\pm (\epsilon )=\epsilon ^2\underset{j=2}{\overset{\mathrm{}}{}}(\chi _1,u\chi _j)_𝐼^2k_j(\kappa _1)^4_^2\frac{|\widehat{m_0}(\omega )|^2}{|\omega |^2+\kappa _j^2\kappa _1^2}𝑑\omega +𝒪(\epsilon ^{2+\gamma }),$$

where we have also expanded the remaining $`k`$ from $`G_k`$ w.r.t. $`\epsilon `$ and included the higher orders into the error term .

We note that $`m_0`$ (and therefore $`\widehat{m_0}`$) cannot be identically zero once $`\mathrm{\Sigma }_1`$ is not a plane. It can be seen from the formulae (4.3): suppose that $`m_00`$. Then $`f_{,11}f_{,22}`$, which yields $`k_0=(f_{,11}^{}{}_{}{}^{2}+f_{,12}^{}{}_{}{}^{2})`$ and consequently $`k_00`$ because we have shown that $`_^2k_0(x)𝑑x=0`$. Hence $`K0`$. At the same time, $`f_{,\mu \nu }0`$, which gives $`m_10`$ and therefore $`M0`$. If we thus exclude the trivial planar case, the lowest-order term in the expansion is strictly negative. Since it is identical for both $`w_+`$ and $`w_{}`$, it follows by (4.10) that it is the same also for the expansion of $`w(\epsilon )`$ in the ground-state energy $`E=\kappa _1^2e^{2w^1}`$ of the Hamiltonian $`H`$. This concludes the proof of Theorem 2.1.

###### Remark.

The surface $`\mathrm{\Sigma }_1`$ is not planar if the matrix $`\theta _{\mu \nu }`$ is not identically zero, *i.e.*, if $`f_{,\mu \nu }0`$. This can be seen as follows. Suppose that $`f_{,\mu \nu }=0`$ identically. Solving this as a system of differential equations, we get $`f(x)=c_\mu x^\mu `$, where $`c_\mu `$ is a constant vector, which is exactly a plane parametrization.

### 4.4 The Thin Layer Limit

We say that the layer $`\mathrm{\Omega }_\epsilon `$ is thin if $`a\rho _m`$. Since the minimum curvature radius introduced in (4.5) explodes as $`\epsilon 0`$ this condition is eventually always satisfied. It is useful, however, to consider also a “true” thin-layer limit in which we start with a mildly curved layer with a fixed $`\epsilon `$ which is already in the asymptotic regime, and make its width $`d`$ small. To this aim we rewrite the formula for $`w_1`$ in the expansion $`w(\epsilon )=:\epsilon ^2w_1+𝒪(\epsilon ^{2+\gamma })`$ of Theorem 2.1 in a different way. For this we go back to the intermediate expansion (4.11). We simplify it as above, however, do not use (4.13) and (4.14). Instead we apply the transformation (3.12) directly to $`(\mathrm{\Delta }m_0,G_k\mathrm{\Delta }m_0)`$, expand $`k`$ in terms of $`\epsilon `$ obtaining thus

$$w_1=\frac{1}{2\pi }m_0^2+\frac{u\chi _1_𝐼^2}{2\pi }m_0^2\underset{j=2}{\overset{\mathrm{}}{}}(\chi _1,u\chi _j)_𝐼^2_^2\frac{|\widehat{\mathrm{\Delta }m_0}(\omega )|^2}{|\omega |^2+k_j(\kappa _1)^2}𝑑\omega .$$

An explicit calculation yields $`u\chi _1_𝐼^2=(\pi ^26)/(12\kappa _1^2)`$, so the second term in the expression for $`w_1`$ is proportional to $`d^2`$ by the definition of $`\kappa _1`$. On the other hand, using the estimate $`|\omega |^2+k_j(\kappa _1)^2k_j(\kappa _1)^2\kappa _2^2\kappa _1^2=3\kappa _1^2`$ together with (4.14) and the above explicit result, we see that the third term is $`𝒪(d^4)`$. Note that this makes sense because $`\widehat{\mathrm{\Delta }m_0}=\mathrm{\Delta }m_0`$ is finite due to $`\text{r3}`$. Summing up the argument, we obtain the expansion (2.4).

This formula has a transparent structure. By virtue of (4.3), (4.1), and the fact that $`_^2k_0(x)𝑑x=0`$, we get the behaviour

$$_{\mathrm{\Sigma }_\epsilon }\left(KM^2\right)𝑑\mathrm{\Sigma }_\epsilon =\epsilon \left(m_0^2+𝒪(\epsilon ^2)\right).$$

Hence the first term of (2.4) comes from the surface attractive potential $`KM^2`$ which dominates the picture for thin layers. The last claim follows from (4.7) and (4.8), because $`v_10`$ and the leading term of $`V_2`$ is $`KM^2`$ in the limit $`u0`$, which one can do as soon as $`d0`$. At the same time, $`C_\pm 1`$, which in view of (4.6) leads to the “limiting” Hamiltonian $`\mathrm{\Delta }_g_3^2+KM^2`$. Such a limit is, of course, only formal since the transverse spectrum explodes, which classically corresponds to increasingly rapid oscillations in the transverse direction. Nevertheless, a similar limit with the confinement realized by a harmonic potential transverse to a compact curved surface was discussed recently in a rigorous way in \[FH\] and led to the same tangential operator $`\mathrm{\Delta }_g+KM^2`$.

### Acknowledgments

The authors wish to thank to Pierre Duclos for helpful discussions and for the hospitality in Centre de Physique Théorique, Marseille-Luminy, where a part of the work was performed. The research has been partially supported by the Grant GAAS A1048101.


format: text

text:
# Analysis of Keck HIRES spectra of early L-type dwarfs

## 1 Introduction

In recent years many new very low mass objects have been discovered, classified and the techniques to detect and observe them have been refined. With the increased sample size has come the discovery of bright sources, whose characteristics can be assumed to be representative of the underlying population. Those bright sources are accessible to study at high resolution and high signal-to-noise. Such investigations can now enable us to measure their parameters which is necessary if we want to understand the physics and nature of very low mass objects in general.

The models for such objects have experienced an equally rapid evolution in the last few years. The latest model atmospheres incorporate the presence of dust within the atmosphere. Although it has become clear that dust does form in cool atmospheres a realistic reproduction of the observed effects has not been achieved yet. The problems associated with implementing an accurate treatment of dust in cool dwarf atmospheres include devising consistent formation and destruction mechanisms, including condensation and diffusion, besides incorporating a realistic treatment of opacities.

Earlier work on the interpretation of L dwarf spectra (e.g. Burrows et al., 2000; Pavlenko et al., 2000; Leinert et al., 2000) focused either on the reproduction and explanation of individual features or used medium to low resolution spectra. Basri et al. (2000) analyzed high resolution spectra of a sample of L dwarfs using a first generation of the dusty and rained out models by Allard et al. (1997). The dusty models used in this earlier work seemed to overestimate the effects of the dust on the optical line formation. The rained out models appeared to provide an adequate fit to the lines. However, it was clear from other work by Ruiz et al. (1997), Leggett et al. (1998), and Kirkpatrick et al. (1999a) that these L type dwarfs share the near-infrared properties of dusty models and not those of rained out models.

Recently, Allard et al. (2001) computed an improved set of dusty and rained out models which include revised opacity and dust treatment. These models were used by Leggett et al. (2001) to analyse low and medium resolution spectra of the overall spectral distribution of M and L dwarfs. Their results confirmed that the new models could still reproduce the near-infrared properties of L dwarfs within uncertainties of the water vapor opacity profile, but also showed that they fitted adequately their optical spectral distributions.

In this paper we will use the improved models of Allard et al. (2001) which were also used in Leggett et al. (2001) to perform analyses of optical high resolution spectra similarly to Basri et al. (2000). In the next section we will describe the available data, in Section 3 we will describe the models and the important improvements, Section 4 contains the methods we used to analyze the data and the results.

## 2 Observations and data

Echelle spectroscopy of the L dwarfs discussed in this paper was obtained using the HIRES spectrograph (Vogt et al., 1994) on the Keck I 10-meter telescope. The observations were made on 23 and 24 August 1998, 13 November 1998 and 6 and 7 June 1999. In each case we used the red cross-disperser, covering selected regions of the spectrum in the 6000 to 9800Å wavelength region. The August observations were made using the $`1\mathrm{"}.15\times 7\mathrm{"}.0`$ slit, giving a resolution of $`34,000`$ (8.8 kms-1 or 4 pixels); seeing was sufficiently better than 1 arcsecond on the other three nights to allow use of the $`0\mathrm{"}.86\times 7\mathrm{"}.0`$ slit, giving an increased resolution of $`45,000`$ (6.4 kms-1 or 3 pixels). Typical exposure times were 3000 seconds.

The echelle data were extracted using the mckee suite of programs, written by T. Barlow. Most of the L dwarfs are barely detected at wavelengths shortward of $`7000`$Å (sometimes $`\lambda <8000`$Å), so we used observations of brighter, near-featureless stars (usually white dwarfs) as templates to outline the shape of each order. Similar techniques were used to extract the thorium-argon wavelength calibration spectra, obtained at the start and end of each night, while flat-field images are used to correct gross non-uniformities. The data were wavelength calibrated using standard IRAF routines and the thorium-argon spectra. We have not attempted to set the spectra on an absolute flux scale.

The low resolution spectra are data published in Kirkpatrick et al. (1999b) and Kirkpatrick et al. (2000).

## 3 The models

The models used in this work are the ones described in Allard et al. (2001). A detailed description can be found therein. They are the same as in the earlier work of Leggett et al. (2001). The models utilize the Ames H2O and TiO line lists by Partridge & Schwenke (1997) and Schwenke (1998). We stress that large uncertainties persist in these line data for the temperature range of this work (see Allard et al., 2000). The opacities in the models have been upgraded with (i) the replacement of the JOLA (Just Overlapping Line Approximation) opacities for FeH, VO and CrH by theoretical line data for which we do dynamical opacity sampling as described in Allard et al. (2001), (ii) the extension of our database of dust grain opacities from 10 to 40 species which are based on laboratory measurements and (iii) the revision of pressure induced H2 opacities.

The dust for all the species of Sharp & Huebner (1990) forms in chemical equilibrium using the Gibbs free energy from the JANAF database (Chase et al., 1985). The models treat the dust in the limiting case where the dust is in equilibrium with the gas phase. When dust forms in a layer it does not diffuse out of the layer but contributes in full to the opacity (Ames-Dusty-2000 models). As comparison we also used the other extreme where the dust forms and completely settles below the atmosphere (Ames-Cond models) and, therefore, does no longer contribute to the opacity. A detailed description of the treatment of dust and of the effects of dust on the temperature structure and the spectrum can be found in Allard et al. (2001).

The temperature structures of these models were used to calculate synthetic high resolution spectra at a resolution of 0.1Å between 6400 and 9100Å for all the models used in this analysis. The atomic lines in the spectra are dominated by van der Waals pressure broadening which has been taken into account as in the calculations as described in Schweitzer et al. (1996). Although not part of the targeted wavelength range, our calculations naturally include all strong lines whose wings end up in the targeted wavelength range (see e.g. Hauschildt et al., 1999). This includes most prominently the wings of the very strong Na D doublet. For the fitting process, the spectra were rotationally broadened if necessary. They were then convolved with a Gaussian to match the resolution of the observed HIRES spectra.

We include van der Waals (vdW) broadening since the lines are pressure broadened due to collisions with neutral species. We use a modification of the Unsöld treatment of vdW broadening which considers not only H but also H2, He and the most abundant trace species (see Schweitzer et al., 1996). In a different study, Burrows et al. (2000) deploy an approximation for pressure broadening due to H2 which is an extension to the impact approximation used by the Unsöld approach. For our model structures we calculated the Weisskopf radius which distinguishes between the validity of the impact theory and the statistical theory of vdW broadening (see Burrows et al., 2000) but found that the impact theory is adequate. This was confirmed by other tests we have made using different line broadening profiles like Margenau profiles (Margenau, 1932), which is a general approach to the statistical theory and proved worse (see section 4.2 for the high resolution fits). However, the models presented by Burrows et al. (2000) (which do not use a plain Unsöld treatment) do not include the molecular line data present in our models and, as a result, predict substantially different spectral energy distributions. Both sets of models therefore have their limitations. We aim to integrate more sophisticated broadening calculations, especially for the treatment of the far wings of resonance lines, in a future generation of stellar models.

However, the most important impact on line profiles is caused by the presence of dust. This has already been demonstrated and investigated by Leinert et al. (2000) and Basri et al. (2000) where Dusty and Cond models have been compared in reproducing individual lines. The line width is very sensitive to the amount of dust as a function of layer. Errors on line broadening theories become negligible compared to the effect of dust and that justifies why the former should only be addressed once the dust issue is settled.

## 4 Analysis and results

We used a $`\chi ^2`$ fitting technique to determine the best fitting model spectra. We used a grid of model spectra and for each spectrum we calculated a $`\chi ^2`$ in order to determine the best fitting model which had the smallest $`\chi ^2`$. All spectra were visually inspected. Some fits had to be corrected when the best fitting model was a feature poor spectrum and the fit was simply an overall canceling out of the observed features in the $`\chi ^2`$ determination. Excluding the problematic model from the grid of theoretical models usually corrected this problem. We did this for fitting both the low resolution models and the high resolution models. Our grid of theoretical spectra consisted of solar metallicity models with effective temperatures between 1600 K and 2400 K in steps of 100 K and surface gravities with $`\mathrm{log}(g)`$ between 3.5 and 5.5 in steps of 0.5.

### 4.1 Results for the low resolution spectra

We used the low resolution spectra to determine a first fit of the effective temperature. We compared the effect of $`\mathrm{log}(g)`$ for constant effective temperature and found that the gravity does not significantly change the shape of the optical low-resolution spectra. This is also supported by the results of our $`\chi ^2`$ fitting techniques which finds similar effective temperatures rather than similar gravities as the three best fitting models. We restricted the wavelength range to 7700Å to 9900Å to avoid the problematic VO band on the blue side of the wavelength range and the problematic FeH band on the red side. For both of these bands we use the latest not fully tested available data.

The results of the fitting process is summarized in Tab. 1 The derived parameters have a determination uncertainty of 100 K in effective temperature and 0.5 dex in $`\mathrm{log}(g)`$. These errors reflect the spacing of the grid. The $`\chi ^2`$ fitting process also determines the second and third best fits within 100 K and 0.5 dex. These errors do not include any systematic problems in the models and the values have to be considered as determined within the used models. The table lists the fits for both the AMES-Dusty and the AMES-Cond models. Except for the L3.5 dwarf, the Dusty models give better fits when inspected visually. With the AMES-Cond models the K I is too strong and the CrH and FeH bands at 8611Å and 8692Å are too strong. This is still the case for the L3.5 dwarf, however, for all parameters, the K I doublet is too weak. This gives a hint that at these temperatures models between the limiting cases of AMES-Dusty and AMES-Cond start being more appropriate. Such models that calculate the amount of dust which diffuses are being developed (Allard et al., in preparation). They will be needed for effective temperatures below approximately 1800 K. In this paper the dwarfs are reasonably hot enough that such models are not required. The fact that we can use the limiting AMES-Dusty models is also supported by the results of Leggett et al. (2001); and Chabrier et al. (2000) suggests that such atmosphere conditions exist.

The fits to the low resolution spectra usually return high gravity values. This is because the high gravity spectra have very broad atomic lines which smooth the whole spectrum. These spectra are therefore preferred by the $`\chi ^2`$ test.

The fits of AMES-Dusty models to the low resolution spectra are shown in Figs. 1 through 5. As can be seen the molecular bands are well reproduced. This is true for VO, CrH and FeH which produce the various features and also for TiO which determines the overall optical spectrum. No attempt has been made to fit individual strong atomic lines like the K I resonance doublet or the Na I subordinate doublet. The purpose of these fits was to determine the effective temperature by fitting the overall shape which is dominated by molecular absorption. The region around the Na I doublet usually can be fit by scaling the spectra but this would worsen the fit elsewhere. Similarly for the K I doublet, which could be fit better by slightly different parameters. This problem will be addressed below when discussing the differences between the results from high resolution and low resolution spectra.

As expected and within the error bars our fits confirm a correlation of spectral type with effective temperature. The overall temperature range of only 300 K for 4 spectral classes is not that small when translating the effective temperature back into luminosity (which it originally defines). However, the small temperature range is also not very surprising since dust condensation (and the associated molecule depletion) takes place very rapidly in parameter space. Therefore, molecular features are very sensitive to changes in $`T_{\mathrm{eff}}`$.

### 4.2 Results for the high resolution spectra

We fitted model spectra to the Cs I line at 8521.36 Å, the Rb I lines at 7800.26 Å and 7947.60 Å, the Na I doublet at 8190Å and the K I at 7685Å doublet when these lines were in the observed wavelength range. For two spectra we also used the Ca I line at 6572.77 Å and for one we used the Li I line at 6707.76 Å. The results are tabulated in Tab. 2. The table also demonstrates which lines could be observed in which object. The fits to the lines are shown in Figs. 614.

As can be seen the derived temperatures do not correlate with spectral type and the temperature derived from the low resolution spectra. The error bars are again 100 K and 0.5 dex within the previously established limits. The derived temperatures from low resolution and high resolution spectra don not agree for the later dwarfs in the sample. Only the earlier type objects show a weak correlation between effective temperatures derived from low resolution and high resolution spectra. To prevent a discrepancy between low resolution and high resolution spectra we could have fixed the effective temperature to the one derived from the low resolution spectra during the fitting process. We attempted this but the resulting fits were very poor so that we left the effective temperature as a free parameter.

The reason for the discrepancy between effective temperatures derived from low resolution spectra and high resolution spectra is most likely a dust opacity and dust settling problem (see also Leinert et al., 2000). A combination of temperature structure and opacity structure produces a line that “looks” too hot for the later dwarfs. This is another indicator that the cooler the models get, the less satisfied the conditions are that allow the use of the AMES-Dusty models. Usage of more consistent models will first change only parts of the atmosphere before the whole atmosphere and spectrum changes. The molecular bands form in different regions and will be affected differently by partial incorrectness than atomic lines which usually form in the deepest layers.

For each object, the values of $`\mathrm{log}(g)`$ are usually within $`\pm `$0.5 dex. However, all objects have rather large values for $`\mathrm{log}(g)`$ although the used parameter range started as low as $`\mathrm{log}(g)`$=3.5. This means that all objects have already contracted considerably. This is also supported by the fact that only two objects show the Li I line. We discuss this in detail in section 4.4.

All the models had been rotationally broadened if necessary. We did not include the rotationally velocity in the automatic fitting process. We rather adjusted this parameter by visual inspection. We want to note, however, that changing the rotational velocity by 5 or 10 km s-1 did usually change the quality of the fit, but it did not change the parameters of the best fitting models. Therefore, the accuracy of the rotational broadening is not important for the determination of $`T_{\mathrm{eff}}`$ and $`\mathrm{log}(g)`$. Keeping these caveats in mind we give the best fitting values for the rotational velocity in Tab. 2.

### 4.3 2MASS0345+2540

2MASS0345+2540 was also in the sample of Leggett et al. (2001) who found $`T_{\mathrm{eff}}`$=2000 K and $`\mathrm{log}(g)`$=6.0, whereas we find $`T_{\mathrm{eff}}`$=1900 K and $`\mathrm{log}(g)`$=5.5. This is well consistent within the error bars. It is however to note that Leggett et al. (2001) used the full spectral range from the optical to the NIR. The current data for water still has uncertainties (Allard et al., 2000) as it does not reproduce the H-band correctly. This affects the overall flux distribution and the optical will have a different pseudo-continuum level. An unrealistically shaped overall spectrum will also affect the pure technical fitting process which will find the best compromise. Both these effects will lead to systematic differences between this work and the work of Leggett et al. (2001).

This example shows that there is an unknown systematic error in addition to the the error bars from our measurement. Using a larger wavelength range and using better line opacities (which is not possible at the moment) may change the results.

### 4.4 Li presence and substellar nature

We only derived high ($`4.5`$) values for $`\mathrm{log}(g)`$. This is consistent with the fact that only the two objects 2MASS1146+2230 and 2MASS1726+1538 show a Li I line. The Li I line in 2MASS1726+1538 is very weak (see Kirkpatrick et al., 2000) and is not detected in the HIRES observation. To demonstrate this consistency we plot $`T_{\mathrm{eff}}`$ versus $`\mathrm{log}(g)`$ from the data by Chabrier et al. (2000) in Fig. 15. The dotted lines indicate 95% Li abundance remaining (left line) and 50% Li abundance remaining (right line). As can be seen, objects without Li should all have $`\mathrm{log}(g)`$ above 5.0, which is what we derived. 2MASS1146+2230 will then have to have a $`\mathrm{log}(g)`$ in the lower part of its error bars and is almost certainly a brown dwarf. 2MASS1726+1538 has a very weak Li I line and is, therefore, in the strip where Lithium depletion occurs. In the context of these models, 2MASS1726+1538 then has $`\mathrm{log}(g)`$=5.25 and m$``$0.06m and would be a brown dwarf as well. All other objects are hot enough to be either brown dwarfs or stars. The tracks below $`\mathrm{log}(g)`$=4.5 correspond to masses below 0.03 M and ages significantly younger than 100 Myrs. Such objects are unlikely to be found in a field sample such as 2MASS.

We want to emphasize that the derived error bars are much too large to use the spectroscopic parameters to determine mass and age from such a diagram. We use this diagram for illustrative purposes and to show that we are consistent with what is expected from evolutionary models.

## 5 Discussion and summary

We have presented fits to high and medium resolution spectra of L dwarfs using our latest model atmospheres and synthetic spectra.

The observed spectra including the high resolution line profiles can be reproduced very well. The derived parameters from the best fitting models are physically reasonable. They match the expected parameters for old ($``$ 1Gyr) VLM stars and massive brown dwarfs.

The important physics in the models is the inclusion of dust condensation and dust opacity as described above. The models used in this analysis use the limiting case in which the dust forms and does not diffuse. Since most objects can be well reproduced with these models, this assumption seems to be well justified for early L dwarfs. However, we found hints that for objects later than L3 (or with our models cooler than 1800 K) the assumption for the limiting case is no longer fully met (see also Leggett et al., 2001).

Most importantly, however, within the valid range our models reproduce accurately the observed spectra. Our new models reproduce realistic line profiles in the presence of dust. The applied Unsöld treatment for the vdW broadening is only an approximation. But for the objects in this paper it reproduces the atomic lines very well within the limits of the dust distribution uncertainties. Tests using other profiles did not reproduce better profiles. We are actively improving the models and considering more realistic line broadening mechanism similar to Burrows et al. (2000) but find no reason to change the current implementation yet.

We encountered the problem that the determined effective temperatures from high resolution spectra are different than those from low resolution spectra. We think this could have two reasons. The first one could be that the line formation of the atomic lines is not reproduced correctly due to an incorrect temperature or opacity structure. This can easily happen if the dust treatment does not produce the right dust properties for all layers (see also Leinert et al., 2000). The second reason is simply the fact that the pseudo continuum level in the optical is probably not correct due to incorrect or incomplete line data. If the molecular line data is incorrect then the overall flux is not distributed correctly which will lead to different pseudo continuum around the atomic lines and thus different best fitting parameters for atomic lines.

Both of these effects can be interpreted as seeing hotter (or cooler) temperatures in the atomic lines. Therefore, we think that the distortions on the $`\mathrm{log}(g)`$ determinations are not very large and that our derived $`\mathrm{log}(g)`$ values are not affected very much. This is also supported by the fact that we find high values for $`\mathrm{log}(g)`$ for objects without Li (see above).

Future models, which are being developed, will include a more consistent dust treatment which will also be valid for lower temperatures and later L dwarfs. The improved dust treatment plus more complete and improved line data which becomes available over time will make the models applicable to early and late type L dwarfs at the same time and it will give consistent results for low resolution and high resolution spectra.

We are grateful to Richard Freedman (NASA-Ames) who generously provided VO and CrH line lists for use in the current models. AS acknowledges support from NASA ATP grant NAG 5-8425 to the University of Georgia. JEG acknowledges the support of the Jet Propulsion Laboratory, California Institute of Technology, which is operated under contract with NASA. PHH acknowledges support in part from NASA ATP grant NAG 5-3018 and LTSA grant NAG 5-3619 to the University of Georgia and partial support from the Pôle Scientifique de Modélisation Numérique at ENS-Lyon. FA acknowledges support from CNRS. INR acknowledges partial support from a NASA/JPL grant to 2MASS Core Project Science. This work was also supported in part by NSF grants AST-9417242, AST-9731450, and NASA grant NAG5-3505 and an IBM SUR grant to the University of Oklahoma. Some of the calculations presented in this paper were performed on the IBM SP and the SGI Origin 2000 of the UGA UCNS, on the IBM SP of the San Diego Supercomputer Center (SDSC, with support from the National Science Foundation), on the Cray T3E and the IBM SP of the NERSC with support from the DoE, on the IBM SP2 of the French Centre National Universitaire Sud de Calcul (CNUSC). We thank all these institutions for a generous allocation of computer time. This paper is partially based on observations obtained at the W. M. Keck Observatory, which is operated jointly by the University of California, the California Institute of Technology and NASA.


text:
# Hyperelliptic jacobians and 𝐔₃(2^𝑚)

## 1. Introduction

In  the author proved that in characteristic $`0`$ the jacobian $`J(C)=J(C_f)`$ of a hyperelliptic curve

$$C=C_f:y^2=f(x)$$

has only trivial endomorphisms over an algebraic closure $`K_a`$ of the ground field $`K`$ if the Galois group $`\mathrm{Gal}(f)`$ of the irreducible polynomial $`fK[x]`$ is “very big”. Namely, if $`n=\mathrm{deg}(f)5`$ and $`\mathrm{Gal}(f)`$ is either the symmetric group $`𝐒_n`$ or the alternating group $`𝐀_n`$ then the ring $`\mathrm{End}(J(C_f))`$ of $`K_a`$-endomorphisms of $`J(C_f)`$ coincides with $`𝐙`$. Later the author  proved that $`\mathrm{End}(J(C_f))=𝐙`$ for an infinite series of $`\mathrm{Gal}(f)=\mathrm{PSL}_2(𝐅_{2^r})`$ and $`n=2^r+1`$ (with $`\mathrm{dim}(J(C_f))=2^{r1}`$) or when $`\mathrm{Gal}(f)`$ is the Suzuki group $`\mathrm{𝐒𝐳}(2^{2r+1})`$ and $`n=2^{2(2r+1)}+1`$ (with $`\mathrm{dim}(J(C_f))=2^{4r+1}`$). We refer the reader to , , , , , , ,  for a discussion of known results about, and examples of, hyperelliptic jacobians without complex multiplication.

We write $`=_f`$ for the set of roots of $`f`$ and consider $`\mathrm{Gal}(f)`$ as the corresponding permutation group of $``$. Suppose $`q=2^m>2`$ is an integral power of $`2`$ and $`𝐅_{q^2}`$ is a finite field consisting of $`q^2`$ elements. Let us consider a non-degenerate Hermitian (wrt $`xx^q`$) sesquilinear form on $`𝐅_{q^2}^3`$. In the present paper we prove that $`\mathrm{End}(J(C_f))=𝐙`$ when $`_f`$ can be identified with the corresponding “Hermitian curve” of isotropic lines in the projective plane $`𝐏^2(𝐅_{q^2})`$ in such a way that $`\mathrm{Gal}(f)`$, becomes either the projective unitary group $`\mathrm{PGU}_3(𝐅_q)`$ or the projective special unitary group $`𝐔_3(q):=\mathrm{PSU}_3(𝐅_q)`$. In this case $`n=\mathrm{deg}(f)=q^3+1=2^{3m}+1`$ and $`\mathrm{dim}(J(C_f))=q^3/2=2^{3m1}`$.

Our proof is based on an observation that the Steinberg representation is the only absolutely irreducible nontrivial representation (up to an isomorphism) over $`𝐅_2`$ of $`𝐔_3(2^m)`$, whose dimension is a power of $`2`$.

## 2. Main results

Throughout this paper we assume that $`K`$ is a field with $`\mathrm{char}(K)2`$. We fix its algebraic closure $`K_a`$ and write $`\mathrm{Gal}(K)`$ for the absolute Galois group $`\mathrm{Aut}(K_a/K)`$. If $`X`$ is an abelian variety defined over $`K`$ then we write $`\mathrm{End}(X)`$ for the ring of $`K_a`$-endomorphisms of $`X`$.

Suppose $`f(x)K[x]`$ is a separable polynomial of degree $`n5`$. Let $`=_fK_a`$ be the set of roots of $`f`$, let $`K(_f)=K()`$ be the splitting field of $`f`$ and $`\mathrm{Gal}(f):=\mathrm{Gal}(K()/K)`$ the Galois group of $`f`$, viewed as a subgroup of $`\mathrm{Perm}()`$. Let $`C_f`$ be the hyperelliptic curve $`y^2=f(x)`$. Let $`J(C_f)`$ be its jacobian, $`\mathrm{End}(J(C_f))`$ the ring of $`K_a`$-endomorphisms of $`J(C_f)`$.

###### Theorem 2.1.

Assume that there exist a positive integer $`m>1`$ such that $`n=2^{3m+1}+1`$ and $`\mathrm{Gal}(f)`$ contains a subgroup isomorphic to $`𝐔_3(2^m)`$. Then either $`\mathrm{End}(J(C_f))=𝐙`$ or $`\mathrm{char}(K)>0`$ and $`J(C_f)`$ is a supersingular abelian variety.

We will prove Theorems 2.1 in §5.

## 3. Permutation groups, permutation modules and very simplicity

Let $`B`$ be a finite set consisting of $`n5`$ elements. We write $`\mathrm{Perm}(B)`$ for the group of permutations of $`B`$. A choice of ordering on $`B`$ gives rise to an isomorphism

$$\mathrm{Perm}(B)𝐒_n.$$

Let $`G`$ be a subgroup of $`\mathrm{Perm}(B)`$. For each $`bB`$ we write $`G_b`$ for the stabilizer of $`b`$ in $`G`$; it is a subgroup of $`G`$. Further we always assume that $`n`$ is odd.

###### Remark 3.1.

Assume that the action of $`G`$ on $`B`$ is transitive. It is well-known that each $`G_b`$ is of index $`n`$ in $`G`$ and all the $`G_b`$’s are conjugate in $`G`$. Each conjugate of $`G_b`$ in $`G`$ is the stabilizer of a point of $`B`$. In addition, one may identify the $`G`$-set $`B`$ with the set of cosets $`G/G_b`$ with the standard action by $`G`$.

We write $`𝐅_2^B`$ for the $`n`$-dimensional $`𝐅`$-vector space of maps $`h:B𝐅_2`$. The space $`𝐅_2^B`$ is provided with a natural action of $`\mathrm{Perm}(B)`$ defined as follows. Each $`s\mathrm{Perm}(B)`$ sends a map $`h:B𝐅_2`$ into $`sh:bh(s^1(b))`$. The permutation module $`𝐅_2^B`$ contains the $`\mathrm{Perm}(B)`$-stable hyperplane

$$Q_B:=\{h:B𝐅_2\underset{bB}{}h(b)=0\}$$

and the $`\mathrm{Perm}(B)`$-invariant line $`𝐅1_B`$ where $`1_B`$ is the constant function $`1`$. Since $`n`$ is odd, there is a $`\mathrm{Perm}(B)`$-invariant splitting

$$𝐅_2^B=Q_B𝐅_21_B.$$

Clearly,

$$\mathrm{dim}_{𝐅_2}(Q_B)=n1$$

and $`𝐅_2^B`$ and $`Q_B`$ carry natural structures of $`G`$-modules. Clearly, $`Q_B`$ is a faithful $`G`$-module. It is also clear that the $`G`$-module $`Q_B`$ can be viewed as the reduction modulo $`2`$ of the $`𝐐[G]`$-module

$$(𝐐^B)^0:=\{h:B𝐐\underset{bB}{}h(b)=0\}.$$

It is well-known that the $`𝐐[G]`$-module $`(𝐐^B)^0`$ is absolutely simple if and only if the action of $`G`$ on $`B`$ is doubly transitive (, Sect. 2.3, Ex. 2).

###### Remark 3.2.

Assume that $`G`$ acts on $`B`$ doubly transitively and $`\mathrm{\#}(B)1=\mathrm{dim}_𝐐((𝐐_B)^0)`$ coincides with the largest power of $`2`$ dividing $`\mathrm{\#}(G)`$. Then it follows from a theorem of Brauer-Nesbitt (, Sect. 16.4, pp. 136–137 ; , p. 249) that $`Q_B`$ is an absolutely simple $`𝐅_2[G]`$-module. In particular, $`Q_B`$ is (the reduction of) the Steinberg representation , .

We refer to  for a discussion of the following definition.

###### Definition 3.3.

Let $`V`$ be a vector space over a field $`𝐅`$, let $`G`$ be a group and $`\rho :G\mathrm{Aut}_𝐅(V)`$ a linear representation of $`G`$ in $`V`$. We say that the $`G`$-module $`V`$ is very simple if it enjoys the following property:

If $`R\mathrm{End}_𝐅(V)`$ is an $`𝐅`$-subalgebra containing the identity operator $`\mathrm{Id}`$ such that

$$\rho (\sigma )R\rho (\sigma )^1R\sigma G$$

then either $`R=𝐅\mathrm{Id}`$ or $`R=\mathrm{End}_𝐅(V)`$.

###### Remarks 3.4.

1. If $`G^{}`$ is a subgroup of $`G`$ and the $`G^{}`$-module $`V`$ is very simple then obviously the $`G`$-module $`V`$ is also very simple.
2. A very simple module is absolutely simple (see , Remark 2.2(ii)).
3. If $`\mathrm{dim}_𝐅(V)=1`$ then obviously the $`G`$-module $`V`$ is very simple.
4. Assume that the $`G`$-module $`V`$ is very simple and $`\mathrm{dim}_𝐅(V)>1`$. Then $`V`$ is not induced from a subgroup $`G`$ (except $`G`$ itself) and is not isomorphic to a tensor product of two $`G`$-modules, whose $`𝐅`$-dimension is strictly less than $`\mathrm{dim}_𝐅(V)`$ (see , Examples 7.1).
5. If $`𝐅=𝐅_2`$ and $`G`$ is perfect then properties (ii)-(iv) characterize the very simple $`G`$-modules (see , Th. 7.7).

The following statement provides a criterion of very simplicity over $`𝐅_2`$.

###### Theorem 3.5.

Suppose a positive integer $`N>1`$ and a group $`H`$ enjoy the following properties:

* $`H`$ does not contain a subgroup of index dividing $`N`$ except $`H`$ itself.
* Let $`N=ab`$ be a factorization of $`N`$ into a product of two positive integers $`a>1`$ and $`b>1`$. Then either there does not exist an absolutely simple $`𝐅_2[H]`$-module of $`𝐅_2`$-dimension $`a`$ or there does not exist an absolutely simple $`𝐅_2[H]`$-module of $`𝐅_2`$-dimension $`b`$.

Then each absolutely simple $`𝐅_2[H]`$-module of $`𝐅_2`$-dimension $`N`$ is very simple.

###### Proof.

This is Corollary 4.12 of . ∎

## 4. Steinberg representation

We refer to  and  for a definition and basic properties of Steinberg representations.

Let us fix an algebraic closure of $`𝐅_2`$ and denote it by $``$. We write $`\varphi :`$ for the Frobenius automorphism $`xx^2`$. Let $`q=2^m`$ be a positive integral power of two. Then the subfield of invariants of $`\varphi ^m:`$ is a finite field $`𝐅_q`$ consisting of $`q`$ elements. Let $`q^{}`$ be an integral positive power of $`q`$. If $`d`$ is a positive integer and $`i`$ is a non-negative integer then for each matrix $`u\mathrm{GL}_d()`$ we write $`u^{(i)}`$ for the matrix obtained by raising each entry of $`u`$ to the $`2^i`$th power.

###### Remark 4.1.

Recall that an element $`\alpha 𝐅_q`$ is called primitive if $`\alpha 0`$ and has multiplicative order $`q1`$ in the cyclic multiplicative group $`𝐅_q^{}`$.

Let $`M<q1`$ be a positive integer. Clearly, the set

$$\mu _M(𝐅_q)=\{\alpha 𝐅_q\alpha ^M=1\}$$

is a cyclic multiplicative subgroup of $`𝐅_q^{}`$ and its order $`M^{}`$ divides both $`M`$ and $`q1`$. Since $`M<q1`$ and $`q1`$ is odd, the ratio $`(q1)/M^{}`$ is an odd integer $`>1`$. This implies that $`3(q1)/M^{}`$ and therefore

$$M^{}=\mathrm{\#}(\mu _M(𝐅_q))(q1)/3.$$

###### Lemma 4.2.

Let $`q>2`$, let $`d`$ be a positive integer and let $`G`$ be a subgroup of $`\mathrm{GL}_d(𝐅_q^{})`$. Assume that one of the following two conditions holds:

1. There exists an element $`uG\mathrm{GL}_d(𝐅_q^{})`$, whose trace $`\alpha `$ lies in $`𝐅_q^{}`$ and has multiplicative order $`q1`$;
2. There are exist a positive integer $`r>\frac{q1}{3}`$, distinct $`\alpha _1,\mathrm{},\alpha _r𝐅_q^{}`$ and elements

$$u_1,\mathrm{},u_rG\mathrm{GL}_d(𝐅_q^{})$$

such that the trace of $`u_i`$ is $`\alpha _i`$ for all $`i=1,\mathrm{},r`$.

Let $`V_0=^d`$ and $`\rho _0:G\mathrm{GL}_d(𝐅_q^{})\mathrm{GL}_d()=\mathrm{Aut}_{}(V_0)`$ be the natural $`d`$-dimensional representation of $`G`$ over $``$. For each positive integer $`i<m`$ we define a $`d`$-dimensional $``$-representation

$$\rho _i:G\mathrm{Aut}(V_i)$$

as the composition of

$$G\mathrm{GL}_d(𝐅_q^{}),xx^{(i)}$$

and the inclusion map

$$\mathrm{GL}_d(𝐅_q^{})\mathrm{GL}_d()\mathrm{Aut}_{}(V_i).$$

Let $`S`$ be a subset of $`\{0,1,\mathrm{}m1\}`$. Let us define a $`d^{\mathrm{\#}(S)}`$-dimensional $``$-representation $`\rho _S`$ of $`G`$ as the tensor product of representations $`\rho _i`$ for all $`iS`$. If $`S`$ is a proper subset of $`\{0,1,\mathrm{}m1\}`$ then there exists an element $`uG`$ such that the trace of $`\rho _S(u)`$ does not belong to $`𝐅_2`$. In particular, $`\rho _S`$ could not be obtained by extension of scalars to $``$ from a representation of $`G`$ over $`𝐅_2`$.

###### Proof.

Clearly,

$$\mathrm{tr}(\rho _i(u))=(\mathrm{tr}(\rho _0(u))^{2^i}uG.$$

This implies easily that

$$\mathrm{tr}(\rho _S(u))=\underset{iS}{}\mathrm{tr}(\rho _i(u))=(\mathrm{tr}(\rho _0(u))^M$$

where $`M=_{iS}2^i`$. Since $`S`$ is a proper subset of $`\{0,1,\mathrm{}m1\}`$, we have

$$0<M<\underset{i=0}{\overset{m1}{}}2^i=2^m1=\mathrm{\#}(𝐅_q^{}).$$

Assume that the condition (i) holds. Then there exists $`uG`$ such that $`\alpha =\mathrm{tr}(\rho _0(u))`$ lies in $`𝐅_q^{}`$ and the exact multiplicative order of $`\alpha `$ is $`q1=2^m1`$.

This implies that $`0\alpha ^M1`$. Since $`𝐅_2=\{0,1\}`$, we conclude that $`\alpha ^M𝐅_2`$. Therefore

$$\mathrm{tr}(\rho _S(u))=(\mathrm{tr}(\rho _0(u))^M=\alpha ^M𝐅_2.$$

Now assume that the condition (ii) holds. It follows from Remark 4.1 that there exists $`\alpha =\alpha _i0`$ such that $`\alpha ^M1`$ for some $`i`$ with $`1ir`$. This implies that if we put $`u=u_i`$ then

$$\mathrm{tr}(\rho _S(u))=(\mathrm{tr}(\rho _0(u))^M=\alpha ^M𝐅_2.$$

Now, let us put $`q^{}=q^2=p^{2m}`$. We write $`x\overline{x}`$ for the involution $`aa^q`$ of $`𝐅_q`$. Let us consider the special unitary group $`\mathrm{SU}_3(𝐅_q)`$ consisting of all matrices $`A\mathrm{SL}_3(𝐅_{q^2})`$ which preserve a nondegenerate Hermitian sesquilinear form on $`𝐅_{q^2}^3`$ say,

$$x,yx_1\overline{y_3}+x_2\overline{y_2}+x_3\overline{y_1}x=(x_1,x_2,x_3),y=(y_1,y_2,y_3).$$

It is well-known that the conjugacy class of the special unitary group in $`\mathrm{GL}_3(𝐅_{q^2})`$ does not depend on the choice of Hermitian form and $`\mathrm{\#}(\mathrm{SU}_3(𝐅_q))=(q^3+1)q^3(q^21)`$. Clearly, for each $`\beta 𝐅_q^{}`$ the group $`\mathrm{SU}_3(𝐅_q)`$ contains the diagonal matrix $`u=\mathrm{diag}(\beta ,1,\beta ^1)`$ with eigenvalues $`\beta ,1,\beta ^1`$; clearly, the trace of $`u`$ is $`\beta +\beta ^1+1`$.

###### Theorem 4.3.

Suppose $`G=\mathrm{SU}_3(𝐅_q)`$. Suppose $`V`$ is an absolutely simple nontrivial $`𝐅_2[G]`$-module. Assume that $`m>1`$. If $`\mathrm{dim}_{𝐅_2}(V)`$ is a power of $`2`$ then it is equal to $`q^3`$. In particular, $`V`$ is the Steinberg representation of $`\mathrm{SU}_3(𝐅_q)`$.

###### Proof.

Recall ( , p. 77, 2.8.10c), that the adjoint representation of $`G`$ in $`\mathrm{End}_{𝐅_{q^2}}(𝐅_{q^2}^3)`$ splits into a direct sum of the trivial one-dimensional representation (scalars) and an absolutely simple $`𝐅_{q^2}[G]`$-module $`\mathrm{St}_2`$ of dimension 8 (traceless operators). The kernel of the natural homomorphism

$$G=\mathrm{SU}_3(𝐅_q)\mathrm{Aut}_{𝐅_{q^2}}(\mathrm{St}_2)\mathrm{GL}_8(𝐅_{q^2})$$

coincides with the center $`Z(G)`$ which is either trivial or a cyclic group of order $`3`$ depending on whether $`(3,q+1)=1`$ or $`3`$. In both cases we get an embedding

$$G^{}:=G/Z(G)=𝐔_3(q)=\mathrm{PSU}_3(𝐅_q)\mathrm{GL}_8(𝐅_{q^2}).$$

If $`m=2`$ (i.e., $`q=4`$) then $`G=\mathrm{SU}_3(𝐅_4)=𝐔_3(4)`$ and one may use Brauer character tables  in order to study absolutely irreducible representations of $`G`$ in characteristic $`2`$. Notice (, p. 284) that the reduction modulo $`2`$ of the irrational constant b5 does not lie in $`𝐅_2`$. Using the Table on p. 70 of , we conclude that there is only one (up to an isomorphism) absolutely irreducible representation of $`G`$ defined over $`𝐅_2`$ and its dimension is $`64=q^3`$. This proves the assertion of the theorem in the case of $`m=2,q=4`$. So further we assume that

$$m3,q=2^m8.$$

Clearly, for each $`uG\mathrm{GL}_3(𝐅_{q^2})`$ with trace $`\delta 𝐅_{q^2}`$ the image $`u^{}`$ of $`u`$ in $`G^{}`$ has trace $`\overline{\delta }\delta 1𝐅_q`$. In particular, if $`u=\mathrm{diag}(\beta ,1,\beta ^1)`$ with $`\beta 𝐅_q^{}`$ then the trace of $`u^{}`$ is

$$t_\beta :=\mathrm{tr}(u^{})=(1+\beta +\beta ^1)(1+\beta +\beta ^1)1=(\beta +\beta ^1)^2.$$

Now let us start to vary $`\beta `$ in the $`q2`$-element set

$$𝐅_q𝐅_2=𝐅_q^{}\{1\}.$$

One may easily check that the set of all $`t_\beta `$’s consists of $`\frac{q2}{2}`$ elements of $`𝐅_q^{}`$. Since $`q8`$,

$$r:=\frac{q2}{2}>\frac{q1}{3}.$$

This implies that $`G^{}\mathrm{GL}_8(𝐅_{q^2})`$ satisfies the conditions of Lemma 4.2 with $`d=8`$. In particular, none of representations $`\rho _S`$ of $`G^{}`$ could be realized over $`𝐅_2`$ if $`S`$ is a proper subset of $`\{0,1,\mathrm{},m1\}`$. On the other hand, it is known (, p. 77, Example 2.8.10c) that each absolutely irreducible representation of $`G`$ over $``$ either has dimension divisible by $`3`$ or is isomorphic to the representation obtained from some $`\rho _S`$ via $`GG^{}`$. The rest is clear. ∎

###### Theorem 4.4.

Suppose $`m>1`$ is an integer and let us put $`q=2^m`$. Let $`B`$ be a $`(q^3+1)`$-element set. Let $`G^{}`$ be a group acting faithfully on $`B`$. Assume that $`G^{}`$ contains a subgroup $`G`$ isomorphic to $`𝐔_3(q)`$. Then the $`G^{}`$-module $`Q_B`$ is very simple.

###### Proof.

First, $`𝐔_3(q)`$ is a simple non-abelian group, whose order is $`q^3(q^3+1)(q^21)/(3,q+1)`$ (, p. XVI, Table 6; , pp. 39–40). Second, notice that $`𝐔_3(q)G^{}`$ acts transitively on $`B`$. Indeed, the classification of subgroups of $`𝐔_3(q)`$ (, Th. 6.5.3 and its proof, p. 329–332) implies that each subgroup of $`𝐔_3(q)`$ has index $`q^3+1=\mathrm{\#}(B)`$. This implies that $`𝐔_3(q)`$ acts transitively on $`B`$. Third, we claim that this action is, in fact, doubly transitive. Indeed, the stabilizer $`𝐔_3(q)_b`$ of a point $`bB`$ has index $`q^3+1`$ in $`𝐔_3(q)`$. It follows easily from the same classification that $`𝐔_3(q)_b`$ is (the image of) the stabilizer (in $`\mathrm{SU}_3(𝐅_q)`$) of a proper subspace $`L`$ in $`𝐅_{q^2}^3`$. If $`L`$ is a plane then counting arguments imply that the restriction of the Hermitian form to $`L`$ could not be non-degenerate and therefore $`𝐔_3(q)_b`$ coincides with (the image of) the stabilizer of certain isotropic line $`L^{}L𝐅_{q^2}^3`$. (The line $`L^{}`$ is the orthogonal complement of $`L`$.) If $`L`$ is a line then counting arguments imply that $`L`$ is isotropic. Hence we may always assume that $`𝐔_3(q)_b`$ is (the image of) the stabilizer of an isotropic line in $`𝐅_{q^2}^3`$. Taking into account that the set of isotropic lines in $`𝐅_{q^2}^3`$ has cardinality $`q^3+1=\mathrm{\#}(B)`$, we conclude that $`B=𝐔_3(q)/𝐔_3(q)_b`$ is isomorphic (as $`𝐔_3(q)`$-set) to the set of isotropic lines on which $`𝐔_3(q)`$ acts doubly transitively and we are done.

By Remark 3.2, the double transitivity implies that the $`𝐅_2[𝐔_3(q)]`$-module $`Q_B`$ is absolutely simple. Since $`\mathrm{SU}_3(𝐅_q)𝐔_3(q)`$ is surjective, the $`𝐅_2[\mathrm{SU}_3(𝐅_q)]`$-module $`Q_B`$ is also absolutely simple. Also, in order to prove that $`𝐅_2[𝐔_3(q)]`$-module $`Q_B`$ is very simple, it suffices to check that the $`𝐅_2[\mathrm{SU}_3(q)]`$-module $`Q_B`$ is very simple.

Recall that $`\mathrm{dim}_{𝐅_2}(Q_B)=\mathrm{\#}(B)1=q^3=2^{3m}`$. By Theorem 4.3, there no absolutely simple nontrivial $`𝐅_2[\mathrm{SU}_3(𝐅_q)]`$-modules, whose dimension strictly divides $`2^{3m}`$. This implies that $`Q_B`$ is not isomorphic to a tensor product of absolutely simple $`𝐅_2[\mathrm{SU}_3(𝐅_q)]`$-modules of dimension $`>1`$. Therefore $`Q_B`$ is not isomorphic to a tensor product of absolutely simple $`𝐅_2[𝐔_3(q)]`$-modules of dimension $`>1`$. Recall that all subgroups in $`G=𝐔_3(q)`$ different from $`𝐔_3(q)`$ itself have index $`q^3+1>q^3=\mathrm{dim}_{𝐅_2}(Q_B)`$. It follows from Corollary 3.5 that the $`G`$-module $`Q_B`$ is very simple.

## 5. Proof of Theorems 2.1

Recall that $`\mathrm{Gal}(f)\mathrm{Perm}()`$. It is also known that the natural homomorphism $`\mathrm{Gal}(K)\mathrm{Aut}_{𝐅_2}(J(C)_2)`$ factors through the canonical surjection $`\mathrm{Gal}(K)\mathrm{Gal}(K()/K)=\mathrm{Gal}(f)`$ and the $`\mathrm{Gal}(f)`$-modules $`J(C)_2`$ and $`Q_{}`$ are isomorphic (see, for instance, Th. 5.1 of ). In particular, if the $`\mathrm{Gal}(f)`$-module $`Q_{}`$ is very simple then the $`\mathrm{Gal}(f)`$-modules $`J(C)_2`$ is also very simple and therefore is absolutely simple.

###### Lemma 5.1.

If the $`\mathrm{Gal}(f)`$-module $`Q_{}`$ is very simple then either $`\mathrm{End}(J(C_f))=𝐙`$ or $`\mathrm{char}(K)>0`$ and $`J(C_f)`$ is a supersingular abelian variety.

###### Proof.

This is Corollary 5.3 of . ∎

It follows from Theorem 4.4 that under the assumptions of Theorem 2.1, the $`\mathrm{Gal}(f)`$-module $`Q_{}`$ is very simple. Applying Lemma 5.1, we conclude that either $`\mathrm{End}(J(C_f))=𝐙`$ or $`\mathrm{char}(K)>0`$ and $`J(C_f)`$ is a supersingular abelian variety.