format: text

text:
# Quaternionic Monge-Ampère equations.

## 0 Introduction.

This paper is a continuation of author’s previous paper . In  we have developed the necessary algebraic technique and we have introduced and studied the class of plurisubharmonic functions of quaternionic variables (this class was independently introduced also by G. Henkin ). The main result of the present paper is the existence of a generalized solution of the Dirichlet problem for quaternionic Monge-Ampère equations in quaternionic strictly pseudoconvex bounded domains in $`\text{ }\text{ }\text{H}^n`$. The uniqueness of solution was established in .

The versions of this result for real and complex Monge-Ampère equations were established in classical papers by A.D. Aleksandrov  (the real case) and E. Bedford and B. Taylor  (the complex case). We prove also a result on the regularity of solution in the Euclidean ball. For real Monge-Ampère equations this result was proved by L. Caffarelli, L. Nirenberg, and J. Spruck  for arbitrary strictly convex bounded domains, and for complex Monge-Ampère equations by L. Caffarelli, J. Kohn, L. Nirenberg, and J. Spruck  and N. Krylov  for arbitrary strictly pseudoconvex bounded domains.

The real Monge-Ampère equations appear in various geometric problems such as the Minkowski problem (see A. Pogorelov ) The Dirichlet problem has received considerable study. The interior regularity of the solution of the Dirichlet problem was proved by A. Pogorelov, and the proof was briefly described in -. The complete proof was published in  and , . In  Cheng and Yau gave a different proof of interior regularity; they have also studied some related geometric problems on flat manifolds. Another motivation of studying of the real Monge-Ampére equations is the Monge-Kantorovich problem on the measure transportation (see e.g.  and ).

The complex Monge-Amprère equations were studied in particular in the connection to Kähler geometry. Good references are the books by T. Aubin , A. Besse , D. Joyce .

There is a general philosophy (promoted especially by V. Arnold, see e.g. ) that some mathematical theories should have three versions which are analogous to each other in certain sense: real, complex, and quaternionic. However they should reflect different phenomena. In this paper we present the theory of plurisubharmonic functions of quaternionic variables and the theory of quaternionic Monge-Ampère equations whose study we have started in . Their real and complex analogs are well known (the real analog of the theory of plurisubharmonic functions is the theory of convex functions).

We are going to formulate our main result more precisely and recall the main notions from . Let  H denote the (non-commutative) field of quaternions. Let $`\text{ }\text{ }\text{H}^n`$ denote the space of $`n`$-tuples of quaternions $`(q_1,q_2,\mathrm{},q_n)`$. We consider $`\text{ }\text{ }\text{H}^n`$ as right  H-module (we call it right  H-vector space). An $`n\times n`$ quaternionic matrix $`A=(a_{ij})`$ is called hyperhermitian if $`A^{}=A`$, i.e. $`a_{ij}=\overline{a}_{ji}`$ for all $`i,j`$.

In order to write the classical (real or complex) Monge-Ampère equations one has to use the notion of determinant of matrices. By now there is no construction of determinant of matrices with non-commuting (even quaternionic) entries which would have all the properties of the usual (commutative) determinant. The most general theory of non-commutative determinants is due to Gelfand and Retakh (see -, also , ). However it turns out that on the class of quaternionic hyperhermitian matrices there is a notion of the Moore determinant which has all the properties of the usual determinant of complex (resp. real) hermitian (resp. symmetric) matrices. Some of these properties are reviewed in Section 1, and we refer to  for further details and references. Here we mention only that the Moore determinant depends polynomially on the entries of a hyperhermitian matrix, and the Moore determinant of any complex hermitian matrix $`A`$ considered as quaternionic hyperhermitian coincides with the usual determinant of $`A`$. We denote the Moore determinant of $`A`$ by $`detA`$.

The quaternionic Monge-Ampère equation is written in terms of this determinant. We have also to recall the notion of plurisubharmonic function of quaternionic variables and the definition of quaternionic strictly pseudoconvex domain following . Let $`\mathrm{\Omega }`$ be a domain in $`\text{ }\text{ }\text{H}^n`$.

###### 0.1.1 Definition.

A real valued function $`u:\mathrm{\Omega }\text{ }\text{ }\text{R}`$ is called quaternionic plurisubharmonic (psh) if it is upper semi-continuous and its restriction to any right quaternionic line is subharmonic.

Recall that upper semi-continuity means that $`f(x_0)\underset{xx_0}{lim\; supf(x)}`$ for any $`x_0\mathrm{\Omega }`$.

Also we will call a $`C^2`$-smooth function $`u:\mathrm{\Omega }\text{ }\text{ }\text{R}`$ to be strictly plurisubharmonic if its restriction to any right quaternionic line is strictly harmonic (i.e. the Laplacian is strictly positive).

###### 0.1.2 Definition.

An open bounded domain $`\mathrm{\Omega }\text{ }\text{ }\text{H}^n`$ with a smooth boundary $`\mathrm{\Omega }`$ is called strictly pseudoconvex if for every point $`z_0\mathrm{\Omega }`$ there exists a neighborhood $`𝒪`$ and a smooth strictly psh function $`h`$ on $`𝒪`$ such that $`\mathrm{\Omega }𝒪=\{h<0\}`$, $`h(z_0)=0`$, and $`h(z_0)0`$.

We will write a quaternion $`q`$ in the usual form

$$q=t+xi+yj+zk,$$

where $`t,x,y,z`$ are real numbers, and $`i,j,k`$ satisfy the usual relations

$$i^2=j^2=k^2=1,ij=ji=k,jk=kj=i,ki=ik=j.$$

The Dirac-Weyl operator $`\frac{}{\overline{q}}`$ is defined as follows. For any  H-valued function $`f`$

$$\frac{}{\overline{q}}f:=\frac{f}{t}+i\frac{f}{x}+j\frac{f}{y}+k\frac{f}{z}.$$

Let us also define the operator $`\frac{}{q}`$:

$$\frac{}{q}f:=\overline{\frac{}{\overline{q}}\overline{f}}=\frac{f}{t}\frac{f}{x}i\frac{f}{y}j\frac{f}{z}k.$$

Remarks. (a) The operator $`\frac{}{\overline{q}}`$ is called sometimes the Cauchy-Riemann-Moisil-Fueter operator since it was introduced by Moisil in  and used by Fueter ,  to define the notion of quaternionic analyticity. For further results on quaternionic analyticity we refer e.g. to , , , , and for applications to mathematical physics to . Another used name for this operator is Dirac-Weyl operator. But in fact it was used earlier by J.C. Maxwell in , vol. II, pp.570-576, where he has applied the quaternions to electromagnetism.

(b) Note that

$$\frac{}{\overline{q}}=\frac{}{t}+,$$

where $`=i\frac{}{x}+j\frac{}{y}+k\frac{}{z}`$. The operator $``$ was first introduced by W.R. Hamilton in .

(c) In quaternionic analysis one considers a right version of the operators $`\frac{}{\overline{q}}`$ and $`\frac{}{q}`$ which are denoted respectively by $`\stackrel{}{\frac{}{\overline{q}}}`$ and $`\stackrel{}{\frac{}{q}}`$. The operator $`\stackrel{}{\frac{}{q}}`$ is related to $`\stackrel{}{\frac{}{\overline{q}}}`$ by the same formula as $`\frac{}{q}`$ is related to $`\frac{}{\overline{q}}`$, and $`\stackrel{}{\frac{}{\overline{q}}}`$ is defined as

$$\stackrel{}{\frac{}{\overline{q}}}f:=\frac{f}{t}+\frac{f}{x}i+\frac{f}{y}j+\frac{f}{z}k.$$

For a real valued function $`f`$ the derivatives $`\frac{^2f}{q_i\overline{q}_j}`$ and $`\frac{^2f}{\stackrel{}{q_i}\stackrel{}{\overline{q}_j}}`$ are quaternionic conjugate to each other.

Now we can write the quaternionic Monge-Ampère equation with respect to $`C^2`$\- smooth psh function $`u`$ on $`\mathrm{\Omega }`$:

$$det(\frac{^2u}{\overline{q}_iq_j})=f,$$

where $`f`$ is a given function. Note that the matrix $`(\frac{^2u}{\overline{q}_iq_j})`$ is quaternionic hyperhermitian (since $`u`$ is real valued), $`det`$ means the Moore determinant of this matrix. Note also that since the function $`u`$ is psh, the matrix $`(\frac{^2u}{\overline{q}_iq_j})`$ is non-negative definite, and hence its Moore determinant is non-negative (the notion of positive definiteness of a hyperhermitian matrix is recalled in Section 1, Definition 1.1.4).

One of the main results of Section 2 of  was the definition of non-negative measure also denoted by $`det(\frac{^2u}{\overline{q}_iq_j})`$ for any continuous psh function $`u`$ (which is not necessarily smooth). That construction generalizes to the quaternionic situation the well known constructions in the real and complex cases due respectively to A.D. Aleksandrov  and Chern-Levine-Nirenberg . Now we can formulate the main results of this paper.

###### 0.1.3 Theorem.

Let $`\mathrm{\Omega }\text{ }\text{ }\text{H}^n`$ be a bounded quaternionic strictly pseudoconvex domain. Let $`fC(\overline{\mathrm{\Omega }}),f0`$. Let $`\varphi C(\mathrm{\Omega })`$. Then there exists unique function $`uC(\overline{\mathrm{\Omega }})`$ which is psh in $`\mathrm{\Omega }`$ such that

$$det(\frac{^2u}{\overline{q}_iq_j})=f\text{ in }\mathrm{\Omega },$$

$$u|_\mathrm{\Omega }\varphi .$$

Note that the uniqueness was proved in . Theorem 0.1.3 claims existence of a solution in a generalized sense (e.g. the function $`u`$ does not have to be smooth). It is of interest to prove the regularity of solution $`u`$ under assumptions of regularity of the initial data $`f,\varphi `$. We can prove it when the domain $`\mathrm{\Omega }`$ is the Euclidean ball $`B`$ in $`\text{ }\text{ }\text{H}^n`$. In Section 6 we prove the following result (called Theorem 6.0.1).

###### 0.1.4 Theorem.

Let $`fC^{\mathrm{}}(\overline{B}),f>0`$. Let $`\varphi C^{\mathrm{}}(B)`$. There exists unique psh function $`uC^{\mathrm{}}(\overline{B})`$ which is a solution of the Dirichlet problem

$$det(\frac{^2u}{\overline{q}_iq_j})=f,$$

$$u|_B=\varphi .$$

The real version of this result was proved for arbitrary strictly convex bounded domains in $`\text{ }\text{ }\text{R}^n`$ by Caffarelli, Nirenberg, and Spruck . The complex version of it was proved for arbitrary strictly pseudoconvex bounded domains in $`\text{ }\text{ }\text{C}^n`$ by Caffarelli, Kohn, Nirenberg, and Spruck  and Krylov . Our method is a modification of the method of the last paper . Note also that in the case $`n=1`$ the problem is reduced to the classical Dirichlet problem for the Laplacian in $`\text{ }\text{ }\text{R}^4`$ (which is a linear problem); it was solved in XIX century.

Let us make a few comments why the method of  can not be generalized immediately to arbitrary strictly pseudoconvex bounded domain in $`\text{ }\text{ }\text{H}^n`$. The main difficulty is that in the complex case one uses the holomorphic transformations to make the domain to be (locally) close to the Euclidean ball. In the quaternionic situation it does not work. Indeed in the complex case the class of diffeomorphisms of a domain which are either holomorphic or anti-holomorphic can be characterized as the class of diffeomorphisms preserving the class of psh functions. In the quaternionic situation the class of diffeomorphisms preserving psh functions is very small: all of them must be affine transformations, more precisely modulo translations the corresponding group is equal to $`GL_n(\text{ }\text{ }\text{H})Sp(1)`$. The last fact is proved in Subsection 2.2.

Let us make a few comments on the method of the proof of Theorem 0.1.3. It uses the solution of the Dirichlet problem in the unit ball given by Theorem 0.1.4. The method to deduce the general case from this one follows the lines of the paper  by Bedford and Taylor. It was necessary to generalize to the quaternionic situation many results from the usual (complex) theory of plurisubharmonic functions (this investigation was started in ). Sections 4 and 5 of this paper follow very closely the complex case .

The paper is organized as follows. In Section 1 we review the necessary facts from the theory on non-commutative determinants; the exposition follows . In Section 2 we review the theory of plurisubharmonic functions of quaternionic variables as it was developed in . In Section 3 we construct the matrix valued measure $`\left(\frac{^2u}{\overline{q}_iq_j}\right)`$ for any (finite) psh function $`u`$ on $`\mathrm{\Omega }`$. This construction is a quaternionic version of the well known analogous construction in the complex case (see , p.70). This construction will be used in the proof of Theorem 0.1.3. In Section 4 we construct for any finite psh function $`u`$ an operator $`\mathrm{\Phi }(u)`$ which is essentially $`(det^2u)^{\frac{1}{n}}`$ (following Section 5 of the paper  by Bedford and Taylor). This operator plays an important technical role in the proof of Theorem 0.1.3. In Section 5 we establish several facts on the envelopes of functions from the Perron-Bremermann families following closely again the technique developed in the complex case in . In Section 6 we prove Theorem 0.1.4 on the existence of $`C^{\mathrm{}}`$-regular solution of the Dirichlet problem in the unit ball under appropriate assumptions on the regularity of the initial data. In Section 7 we prove Theorem 0.1.3; the proof uses the results of all previous sections. In Section 8 we discuss further the notion of strictly pseudoconvex domain in the quaternionic space. Thus we introduce the quaternionic analogue of the Levi form and consider some examples. In Subsection 8.3 we state some open questions.

Acknowledgements. We are grateful to P. Biran, M. Gromov, G. Henkin, D. Kazhdan, L. Nirenberg, V. Palamodov, and L. Polterovich for useful and stimulating discussions. We express our gratitude to M. Sodin for numerous explanations of the classical theory of subharmonic functions. We thank J.C. Wood for useful correspondences regarding harmonic morphisms.

## 1 Background from non-commutative linear algebra.

In this section we review some material on non-commutative determinants. More precisely we will recall some facts on the Dieudonné and Moore determinants of quaternionic matrices following .

The Dieudonné determinant of quaternionic matrices behaves exactly like the absolute value of the usual determinant of real or complex matrices from all points of view (algebraic and analytic). Let us denote by $`M_n(\text{ }\text{ }\text{H})`$ the set of all quaternionic $`(n\times n)`$\- matrices. The Dieudonné determinant $`D`$ is defined on this set and takes values in non-negative real numbers:

$$D:M_n(\text{ }\text{ }\text{H})\text{ }\text{ }\text{R}_0.$$

Then one has the following (known) results:

###### 1.1.1 Theorem.

(i) For any complex $`(n\times n)`$\- matrix $`X`$ considered as quaternionic matrix, the Dieudonné determinant $`D(X)`$ is equal to the absolute value of the usual determinant of $`X`$.

(ii) For any quaternionic matrix $`X`$

$$D(X)=D(X^t)=D(X^{}),$$

where $`X^t`$ and $`X^{}`$ denote the transposed and quaternionic conjugate matrices of $`X`$ respectively.

(iii) $`D(XY)=D(X)D(Y).`$

The following result is a weak version of the decomposition of the determinant in row (column).

###### 1.1.2 Theorem.

Let $`A=\left[\begin{array}{ccc}a_{11}& \mathrm{}& a_{1n}\\ \multicolumn{3}{c}{}\\ a_{n1}& \mathrm{}& a_{nn}\end{array}\right]`$ be a quaternionic matrix. Then

$$D(A)\underset{i=1}{\overset{n}{}}|a_{1i}|D(M_{1i}).$$

Similar inequalities hold for any other row or column.

(In this theorem $`|a|`$ denotes the absolute value of a quaternion $`a`$, and $`M_{pq}`$ denotes the minor of the matrix $`A`$ obtained from it by deleting the $`p`$-th row and $`q`$-th column).

In a sense, the Dieudonné determinant provides the theory of absolute value  of determinant. However it is not always sufficient and we loose many of the algebraic properties of the usual determinant. The notion of Moore determinant provides such a theory, but only on the class of quaternionic hyperhermitian  matrices. Remind that a square quaternionic matrix $`A`$ is called hyperhermitian if its quaternionic conjugate $`A^{}`$ is equal to $`A`$. The Moore determinant denoted by $`det`$ is defined on the class of all hyperhermitian matrices and takes real values. For the construction of the Moore determinant we refer to , Subsection 1.1, where one can also find the references to the original papers. The important advantage of the Moore determinant with respect to the Dieudonné determinant is that it depends polynomially on the entries of a matrix; it has already all the algebraic and analytic properties of the usual determinant of real symmetric and complex hermitian matrices. Let us state some of them.

###### 1.1.3 Theorem.

(i) The Moore determinant of any complex hermitian matrix considered as quaternionic hyperhermitian matrix is equal to its usual determinant.

(ii) For any hyperhermitian matrix $`A`$ and any quaternionic matrix $`C`$

$$det(C^{}AC)=detAdet(C^{}C).$$

Examples.

(a) Let $`A=diag(\lambda _1,\mathrm{},\lambda _n)`$ be a diagonal matrix with real $`\lambda _i`$’s. Then $`A`$ is hyperhermitian and its Moore determinant $`detA=_i\lambda _i`$.

(b) A general hyperhermitian $`2\times 2`$ matrix $`A`$ has the form

$$A=\left[\begin{array}{cc}a& q\\ \overline{q}& b\end{array}\right],$$

where $`a,b\text{ }\text{ }\text{R},q\text{ }\text{ }\text{H}`$. Then its Moore determinant is equal to $`detA=abq\overline{q}`$.

Let us remind the definition of positive definiteness of hyperhermitian quaternionic matrix following .

###### 1.1.4 Definition.

Let $`A=(a_{ij})_{i,j=1}^n`$ be a hyperhermitian quaternionic matrix. $`A`$ is called  non-negative definite if for every $`n`$-column of quaternions $`\xi =(\xi _i)_{i=1}^n`$ one has

$$\xi ^{}A\xi =\underset{i,j}{}\overline{\xi _i}a_{ij}\xi _j0.$$

Similarly $`A`$ is called positive definite if the above expression is strictly positive once $`\xi 0`$.

In terms of the Moore determinant one can prove the generalization of the classical Sylvester criterion of positive definiteness of hyperhermitian matrices (Theorem 1.1.13 in ). In terms of the Moore determinant one can introduce the notion of the mixed discriminant and to prove the analogues of Alexandrov’s inequalities for mixed discriminants (Theorem 1.1.15 and Corollary 1.1.16 in ).

The (well known) relation between the Dieudonné and Moore determinants is as follows: for any hyperhermitian matrix $`X`$

$$D(X)=|detX|.$$

Note that the Dieudonné determinant was introduced originally by J. Dieudonné in  (see also  for his theory). It can be defined for arbitrary (non-commutative) field. On more modern language this result can be formulated as a computation of the $`K_1`$-group of a non-commutative field (see e.g. ). Note also that there is a more recent very general theory of non-commutative determinants (or quasideterminants) due to I. Gelfand and V. Retakh generalizing in certain direction the theory of the Dieudonné determinant and many other known theories of non-commutative determinants. First it was introduced in , see also ,  and references therein for further developments and applications. In a recent paper  the connection of the Moore and Dieudonné determinants of quaternionic matrices to the theory of quasideterminants was made very explicit and well understood.

We would also like to mention a different direction of a development of the quaternionic linear algebra started by D. Joyce  and applied by himself to hypercomplex algebraic geometry. We refer also to D. Quillen’s paper  for further investigations in that direction. Another attempt to understand the quaternionic linear algebra from the topological point of view was done by the author in .

## 2 Review of the theory of psh functions of quaternionic variables.

### 2.1 Some results from .

We recall the basic facts from the theory of plurisubharmonic (psh) functions of quaternionic variables established by the author in  (see Definition 0.1.1 in this paper). The operators $`\frac{}{\overline{q}}`$ and $`\frac{}{q}`$ were defined in the introduction.

First one has the following simple fact (see Proposition 2.1.6 in ).

###### 2.1.1 Proposition.

A real valued twice continuously differentiable function $`f`$ on the domain $`\mathrm{\Omega }\text{ }\text{ }\text{H}^n`$ is quaternionic plurisubharmonic if and only if at every point $`q\mathrm{\Omega }`$ the matrix $`(\frac{^2f}{\overline{q}_iq_j})(q)`$ is non-negative definite.

Note that the matrix in the statement of proposition is quaternionic hyperhermitian (since the function $`f`$ is real valued). The more important thing is that in analogy to the real and complex cases one can define for any continuous quaternionic plurisubharmonic function $`f`$ a non-negative measure $`det(\frac{^2f}{\overline{q}_iq_j})`$, where $`det`$ denotes the Moore determinant (this measure is obviously defined for smooth $`f`$). One has the following continuity result.

###### 2.1.2 Theorem.

Let $`\{f_N\}`$ be a sequence of continuous quaternionic plurisubharmonic functions in a domain $`\mathrm{\Omega }\text{ }\text{ }\text{H}^n`$. Assume that this sequence converges uniformly on compact subsets to a function $`f`$. Then $`f`$ is continuous quaternionic plurisubharmonic function. Moreover the sequence of measures $`det(\frac{^2f_N}{\overline{q}_iq_j})`$ weakly converges to the measure $`det(\frac{^2f}{\overline{q}_iq_j})`$.

The proofs of analogous results in real and complex cases can be found in , where the exposition of this topic follows the approach of Chern-Levine-Nirenberg  and Rauch-Taylor . For the complex case we refer also to the classical book by P. Lelong .

The next result is called the minimum principle (Theorem 2.2.1 in ).

###### 2.1.3 Theorem.

Let $`\mathrm{\Omega }`$ be a bounded open set in $`\text{ }\text{ }\text{H}^n`$. Let $`u,v`$ be continuous functions on $`\overline{\mathrm{\Omega }}`$ which are plurisubharmonic in $`\mathrm{\Omega }`$. Assume that

$$det(\frac{^2u}{\overline{q}_iq_j})det(\frac{^2v}{\overline{q}_iq_j})\text{ in }\mathrm{\Omega }.$$

Then

$$min\{u(z)v(z)|z\overline{\mathrm{\Omega }}\}=min\{u(z)v(z)|z\mathrm{\Omega }\}.$$

### 2.2 Diffeomorphisms preserving psh functions.

In this subsection we prove the following proposition.

###### 2.2.1 Proposition.

Let $`\mathrm{\Omega }\text{ }\text{ }\text{H}^n`$ be a domain. Let $`F:\mathrm{\Omega }\mathrm{\Omega }`$ be a diffeomorphism such that for every open set $`𝒪\mathrm{\Omega }`$ and for any psh function $`f`$ on $`𝒪`$ the function $`F^{}f`$ is psh on $`F^1(𝒪)`$. Then $`F`$ is an affine transformation which can be written as a composition of a translation and a linear transformation from the group $`GL_n(\text{ }\text{ }\text{H})Sp(1)`$.

In the statement of the theorem the group $`GL_n(\text{ }\text{ }\text{H})Sp(1)`$ is defined as follows. On the right quaternionic space $`\text{ }\text{ }\text{H}^n`$ there is a left action of the group of  H-linear invertible transformations $`GL_n(\text{ }\text{ }\text{H})`$. Also the group $`Sp(1)`$ of norm one quaternions acts on $`\text{ }\text{ }\text{H}^n`$ from the right. Both actions commute and the group they generate is denoted by $`GL_n(\text{ }\text{ }\text{H})Sp(1)`$. Note that it is isomorphic to $`(GL_n(\text{ }\text{ }\text{H})\times Sp(1))/\{\pm Id\}`$. Now let us prove the proposition. Note also that all such affine transformations preserving the domain $`\mathrm{\Omega }`$ must preserve the class of psh functions (see , Subsection 2.1).

Proof. Let $`U`$ be any domain in $`\text{ }\text{ }\text{H}^1`$ and let $`m:U\mathrm{\Omega }`$ be any  H-linear map. Let $`p:\mathrm{\Omega }\text{ }\text{ }\text{H}^1`$ be any  H-linear projection. Consider the composition $`pFm:U\text{ }\text{ }\text{H}^1`$. It is easy to see that this map preserves the class of psh functions which is one-dimensional case means just that it preserves the class of subharmonic functions. Hence $`pFm`$ preserves the class of harmonic functions (i.e. it is so called harmonic morphism, see e.g.  for more details and references) . However there is a general result of B. Fuglede  which says the following. Let $`g:MN`$ be a smooth map between Riemannian manifolds of the same dimension greater than 2 which preserves the class of harmonic functions (in the above sense). Then $`g`$ is a conformal mapping with the constant coefficient of conformality. When $`M`$ and $`N`$ are linear vector spaces with Euclidean metrics this result together with the classical Liuville theorem imply that $`g`$ is a composition of homothety, translation, and orthogonal transformation.

It easily follows that our original map $`F`$ is an affine transformation. Also it is easy to see that $`F`$ is a composition of a translation and a transformation from $`GL_n(\text{ }\text{ }\text{H})Sp(1)`$. Q.E.D.

## 3 The distribution $`\left(\frac{^2u}{\overline{q}_iq_j}\right)`$.

In this section we will define the matrix valued measure $`\left(\frac{^2u}{\overline{q}_iq_j}\right)`$ for any (finite) psh function $`u`$ on $`\mathrm{\Omega }`$. This construction is a quaternionic version of the well known analogous construction in the complex case (see , p.70).

Let us denote by $`_n`$ the (real) linear space of $`n\times n`$ quaternionic hyperhermitian matrices. Let

$$𝒞:=\{\xi _n|\xi 0\}.$$

Thus $`𝒞`$ is a closed convex cone. On the space $`_n`$ one has the bilinear symmetric form $`(,):_n\times _n\text{ }\text{ }\text{R}`$ defined by

$$(A,B)=ReTr(AB),$$

where for any $`n\times n`$ quaternionic matrix $`X=(x_{ij})`$, $`ReTr(X):=_{i=1}^nRex_{ii}`$. Note also that for any quaternionic matrices $`X`$ and $`C`$ with $`C`$ invertible, one has $`ReTr(CXC^1)=ReTrX`$.

We easily have

###### 3.1.1 Claim.

(i) $`(,)`$ is a perfect pairing on $`_n`$.

(ii) For any matrices $`A,B𝒞`$ one has $`(A,B)0`$.

(iii) The dual cone $`𝒞^o:=\{\xi _n|(\xi ,\eta )0\eta 𝒞\}`$ coincides with $`𝒞`$.

###### 3.1.2 Definition.

One says that an $`_n`$-valued distribution $`\psi `$ on the domain $`\mathrm{\Omega }`$ is non-negative ($`\psi 0`$) if for any smooth compactly supported function $`f`$ on $`\mathrm{\Omega }`$ with values in the cone $`𝒞`$ one has $`\psi (f)0`$. One can call such a $`\psi `$ $`𝒞`$-valued.

As in the usual scalar valued case one has the following result. (For the scalar valued case see .)

###### 3.1.3 Proposition.

Any $`𝒞`$-valued distribution on $`\mathrm{\Omega }`$ is of zero order, i.e. (non-negative) $`_n`$-valued measure.

We also have the following result (which easily follows from the scalar valued case).

###### 3.1.4 Lemma.

Any locally bounded sequence of $`_n`$-valued measures on $`\mathrm{\Omega }`$ has a weakly convergent subsequence.

Note also that for any $`_n`$-valued distribution (resp. measure) $`\mu `$ on $`\mathrm{\Omega }`$ one can define in the obvious way its trace $`Tr\mu (=ReTr\mu )`$, which is a real valued distribution (resp. measure).

###### 3.1.5 Proposition.

The sequence of $`𝒞`$-valued measures $`\{\mu _j\}`$ is (locally) bounded iff the sequence of real valued measures $`\{Tr\mu _j\}`$ is (locally) bounded.

Proof. It immediately follows form the fact that a subset $`X𝒞(_n)`$ is bounded iff the set $`\{Trx|xX\}`$ is bounded. Q.E.D.

Let us now define for any quaternionic psh function $`u`$ on $`\mathrm{\Omega }`$ the $`𝒞`$-valued measure $`(\frac{^2u}{\overline{q}_iq_j})`$ which has the usual meaning for $`C^2`$\- smooth function $`u`$. Let $`u`$ be an arbitrary (finite) quaternionic psh function on $`\mathrm{\Omega }`$. By , Subsection 2.1, $`u`$ is subharmonic. But every finite subharmonic function is locally integrable (see e.g. , Ch.1 §1.4). Hence we can define

$$u_\epsilon :=u\chi _\epsilon ,$$

where $`\chi _\epsilon (z)=\frac{1}{\epsilon ^{4n}}\chi (\frac{z}{\epsilon })0`$ is the usual smoothing kernel (like as in the complex situation, see e.g.  , p.45). Then $`u_\epsilon `$ are $`C^{\mathrm{}}`$\- smooth psh functions. Hence $`(\frac{^2u_\epsilon }{\overline{q}_iq_j})0`$ for all $`\epsilon >0`$.

###### 3.1.6 Proposition-Definition.

For any quaternionic psh function $`u`$ on $`\mathrm{\Omega }`$ the $`_n`$\- valued measures $`(\frac{^2u_\epsilon }{\overline{q}_iq_j})`$ converge weakly to a non-negative $`_n`$\- valued measure as $`\epsilon 0`$. This measure will be denoted by $`(\frac{^2u}{\overline{q}_iq_j})`$.

It is easy to see that if $`uC^2(\mathrm{\Omega })`$ then the limit measure has its usual meaning.

Proof. First let us show that the measures $`\frac{^2u_\epsilon }{\overline{q}_iq_j}`$ are locally bounded. By Proposition 3.1.5 it is sufficient to show that their traces are locally bounded. But $`Tr(\frac{^2u_\epsilon }{\overline{q}_iq_j})=\mathrm{\Delta }u_\epsilon 0`$. Let $`K\mathrm{\Omega }`$ be any compact subset. Let $`\gamma 0`$ be a smooth function with compact support on $`\mathrm{\Omega }`$ which is equal to 1 on $`K`$. Then

$$_K\mathrm{\Delta }u_\epsilon 𝑑vol_\mathrm{\Omega }\mathrm{\Delta }u_\epsilon \gamma 𝑑vol=_\mathrm{\Omega }u_\epsilon \mathrm{\Delta }\gamma 𝑑vol$$

$$\mathrm{\Delta }\gamma _{C(\mathrm{\Omega })}_{supp\gamma }u_\epsilon 𝑑vol\mathrm{\Delta }\gamma _{C(\mathrm{\Omega })}_{supp(\gamma )+\epsilon D}|u|𝑑vol=$$

$$\mathrm{\Delta }\gamma _{C(\mathrm{\Omega })}u_{L^1(supp(\gamma )+\epsilon D)},$$

where $`D`$ denotes the unit Euclidean ball in $`\text{ }\text{ }\text{H}^n`$. This proves the local boundedness of the sequence of measures $`(\frac{^2u_\epsilon }{\overline{q}_iq_j})`$. Hence by Lemma 3.1.4 for any sequence $`\{\epsilon _N\}0`$ the sequence of measures $`(\frac{^2u_{\epsilon _N}}{\overline{q}_iq_j})`$ has a weakly convergent subsequence. It remains to show that the limit does not depend on the choice of subsequence.

Fix an arbitrary $`\varphi C_0^{\mathrm{}}(\mathrm{\Omega })`$. We easily get for any $`i,j`$

$$_\mathrm{\Omega }\varphi \frac{^2u_\epsilon }{\overline{q}_iq_j}𝑑vol=_\mathrm{\Omega }u_\epsilon \frac{^2\varphi }{\overline{q}_iq_j}𝑑vol_\mathrm{\Omega }u\frac{^2\varphi }{\overline{q}_iq_j}𝑑vol.$$

Namely for smooth $`\varphi `$ the limit does not depend on the choice of subsequence. This implies the statement. Q.E.D.

###### 3.1.7 Theorem.

(i) Let $`\{u_N\}`$ be a sequence of quaternionic psh functions on a domain $`\mathrm{\Omega }\text{ }\text{ }\text{H}^n`$ which is uniformly bounded from above on every compact subset of $`\mathrm{\Omega }`$. Then either $`u_N\mathrm{}`$ uniformly on compact subsets of $`\mathrm{\Omega }`$, or else there is a subsequence $`\{u_{N_k}\}`$ which converges in $`L_{loc}^1(\mathrm{\Omega })`$. If $`u_N\mathrm{}`$ for all $`N`$ and $`u_N`$ converge in the sense of distributions to a distribution $`U`$, then $`U`$ is defined by a psh function $`u`$ and $`u_Nu`$ in $`L_{loc}^1(\mathrm{\Omega })`$.

(ii) Assume that a sequence $`\{u_N\}`$ of quaternionic psh functions on $`\mathrm{\Omega }`$ converges in $`L_{loc}^1(\mathrm{\Omega })`$ to a quaternionic psh function $`u`$. Then one has a weak convergence of measures

$$(\frac{^2u_N}{\overline{q}_iq_j})(\frac{^2u}{\overline{q}_iq_j}).$$

To prove this theorem we will need a lemma which is a quaternionic analogue of the corresponding complex result (see , Theorem 4.1.7).

###### 3.1.8 Lemma.

Let $`u`$ be a function defined on a domain $`\mathrm{\Omega }\text{ }\text{ }\text{H}^n`$. Assume that $`u_A(z):=u(Az)`$ is subharmonic in $`\mathrm{\Omega }_A:=\{z|Az\mathrm{\Omega }\}`$ for every invertible linear quaternionic transformation $`A`$ (i.e. $`AGL_n(\text{ }\text{ }\text{H})`$). Then $`u`$ is quaternionic psh.

Assuming this lemma let us prove Theorem 3.1.7.

Proof of Theorem 3.1.7. (i) This part of the theorem is known to be true if one replaces in its statement the word ”psh” by the word ”subharmonic” (see , Theorem 3.2.12). In order to deduce part (i) of the theorem from that result it remains to show that the limit function $`u`$ is psh (and not just subharmonic). But this immediately follows from Lemma 3.1.8.

(ii) First note that the measures $`(\frac{^2u_N}{\overline{q}_iq_j}),N1`$ are uniformly locally bounded in $`\mathrm{\Omega }`$. This is proved exactly as in Proposition 3.1.6.

Hence choosing a subsequence if necessary we may assume that this sequence of measures converges weakly to an $`_n`$-valued measure $`(\nu _{\overline{i}j})`$. We have to prove that $`\nu _{\overline{i}j}=\frac{^2u}{\overline{q}_iq_j}`$. To see it fix an arbitrary function $`\varphi C_0^{\mathrm{}}(\mathrm{\Omega })`$. Then

$$_\mathrm{\Omega }\frac{^2u_N}{\overline{q}_iq_j}\varphi 𝑑vol=_\mathrm{\Omega }u_N\frac{^2\varphi }{\overline{q}_iq_j}𝑑vol_\mathrm{\Omega }u\frac{^2\varphi }{\overline{q}_iq_j}𝑑vol=_\mathrm{\Omega }\frac{^2u}{\overline{q}_iq_j}\varphi 𝑑vol,$$

where the first and the last equalities can be easily deduced from the assumptions. The result follows. Q.E.D.

It remains to prove Lemma 3.1.8.

Proof of Lemma 3.1.8. The proof is an easy modification of the proof of Theorem 4.1.7 in . Fix $`z\mathrm{\Omega }`$. The function $`u(z_1+w_1,z_2+\epsilon w_2,\mathrm{},z_n+\epsilon w_n)`$ is subharmonic in $`w`$ by hypothesis, $`0<\epsilon <1`$. Hence

$$u(z)_{|\zeta |=1}u(z_1+r\zeta _1,z_2+r\epsilon \zeta _2,\mathrm{},z_n+r\epsilon \zeta _n)𝑑\omega (\zeta ),$$

where $`\omega (\zeta )`$ is the normalized Lebesgue measure on the unit sphere. Since $`u`$ is upper semi-continuous and locally bounded above, the Fatou lemma implies as $`\epsilon 0`$ that

$$u(z)_{|\zeta |=1}u(z_1+r\zeta _1,z_2,\mathrm{},z_n)𝑑\omega (\zeta ).$$

The last inequality and Theorem 3.2.3 in  imply that the function $`z_1u(z_1,z_2,\mathrm{},z_n)`$ is subharmonic. The subharmonicity of the restrictions to other quaternionic lines follows form the invariance under quaternionic linear transformations. Q.E.D.

## 4 The operator $`\mathrm{\Phi }(u)`$.

Following Section 5 of the paper  by Bedford and Taylor we will define for any finite psh function $`u`$ an operator $`\mathrm{\Phi }(u)`$ which is essentially $`(det^2u)^{\frac{1}{n}}`$. It is closely related to the operator $`det(^2u)`$, but it is defined for arbitrary finite psh function $`u`$.

As in the previous section we will denote by $`𝒞`$ the cone of non-negative definite quaternionic hyperhermitian $`n\times n`$ matrices. Consider the function

$$\mathrm{\Psi }(\xi )=(det(\xi ))^{\frac{1}{n}},\xi 𝒞.$$

###### 4.1.1 Proposition.

The function $`\mathrm{\Psi }`$ is a continuous, nonnegative, concave function which is homogeneous of degree 1 on the cone $`𝒞`$.

Proof. Concavity follows from Theorem 1.1.17 (ii) of . The other properties are trivial. Q.E.D.

Let $`\mu `$ be a vector valued Borel measure on $`\mathrm{\Omega }\text{ }\text{ }\text{H}^n`$ with values in the cone $`𝒞`$. Let us define a nonnegative Borel measure $`\mathrm{\Psi }(\mu )`$ on $`\mathrm{\Omega }`$ as follows. Choose a scalar valued nonnegative Borel measure $`\lambda `$ on $`\mathrm{\Omega }`$ so that $`\mu `$ is absolutely continuous with respect to $`\lambda `$. Then by the Radon-Nikodim theorem $`d\mu =hd\lambda `$ where $`h`$ is a Borel measurable function on $`\mathrm{\Omega }`$ with values in $`𝒞`$.

###### 4.1.2 Definition.

$`\mathrm{\Psi }(\mu ):=\mathrm{\Psi }(h)\lambda `$.

It is easy to see that this definition is independent of the choice of the measure $`\lambda `$. The following proposition is trivial.

###### 4.1.3 Proposition.

If $`\mu `$ and $`\nu `$ are Borel measures on $`\mathrm{\Omega }`$ with values in $`𝒞`$ then

(1)$`\mathrm{\Psi }(\alpha \mu )=\alpha \mathrm{\Psi }(\mu )`$ if $`\alpha 0`$.

(2)If $`\mu ,\nu `$ are mutually singular then $`\mathrm{\Psi }(\mu +\nu )=\mathrm{\Psi }(\mu )+\mathrm{\Psi }(\nu )`$.

(3) $`\mathrm{\Psi }(\mu )`$ is absolutely continuous with respect to $`\mu `$.

(4)$`\mathrm{\Psi }(t\mu +(1t)\nu )t\mathrm{\Psi }(\mu )+(1t)\mathrm{\Psi }(\nu ),\mathrm{\hspace{0.17em}0}<t<1`$.

###### 4.1.4 Proposition.

If $`\chi 0`$ is a continuous function with compact support then

$$\mathrm{\Psi }(\mu \chi )\mathrm{\Psi }(\mu )\chi $$

on any compact set $`\mathrm{\Omega }^{}`$ with $`\mathrm{\Omega }^{}+support(\chi )\mathrm{\Omega }`$.

Proof. The proof is exactly the same as in the complex case (see Proposition 5.4 in ). It is essentially based on Proposition 4.1.3 and general measure theoretic construction of Goffman and Serrin . We do not reproduce it here. Q.E.D.

###### 4.1.5 Proposition.

Let $`\mu _j`$ be a sequence of Borel measures on $`\mathrm{\Omega }`$ with values in $`𝒞`$ which converges weakly to a Borel measure $`\mu `$. Suppose also that Borel measures $`\mathrm{\Psi }(\mu _j)`$ converge weakly. Then

$$\mathrm{\Psi }(\mu )\underset{j\mathrm{}}{lim}\mathrm{\Psi }(\mu _j).$$

Proof. Again the proof is exactly the same as in the complex case (see Proposition 5.6 of ). Note that in turn this is a special case of , Theorem 3, page 165 with slight modifications. Q.E.D.

To define the operator $`\mathrm{\Phi }(u)`$ for any psh function $`u`$ note that by Proposition-Definition 3.1.6 the matrix of Borel measures $`\left(\frac{^2u}{\overline{q}_iq_j}\right)`$ takes values in the cone $`𝒞`$.

###### 4.1.6 Definition.

$$\mathrm{\Phi }(u):=\mathrm{\Psi }\left(\left(\frac{^2u}{\overline{q}_iq_j}\right)\right).$$

###### 4.1.7 Theorem.

Let $`u,v,u_j`$ be finite psh on $`\mathrm{\Omega }\text{ }\text{ }\text{H}^n`$. Then

(1) $`\mathrm{\Phi }(\alpha u)=\alpha \mathrm{\Phi }(u),\alpha 0,`$

(2) $`\mathrm{\Phi }(tu+(1t)v)t\mathrm{\Phi }(u)+(1t)\mathrm{\Phi }(v),\mathrm{\hspace{0.17em}0}<t<1,`$

(3) If $`\chi 0`$ is a continuous function with compact support then

$$\mathrm{\Phi }(u\chi )\mathrm{\Phi }(u)\chi $$

on any open set $`\mathrm{\Omega }^{}`$ with $`\mathrm{\Omega }^{}+support(\chi )\mathrm{\Omega }`$;

(4) if $`u_ju`$ as distributions on $`\mathrm{\Omega }`$ and if the sequence of measures $`\mathrm{\Phi }(u_j)`$ converges weakly then

$$\mathrm{\Phi }(u)\underset{j\mathrm{}}{lim}\mathrm{\Phi }(u_j).$$

(5) If $`u_\epsilon =u\chi _\epsilon `$ where $`\chi _\epsilon (z)=\frac{1}{\epsilon ^{4n}}\chi (\frac{z}{\epsilon })0`$ is the usual smoothing kernel (like in the complex situation, see e.g. , p.45) then

$$\underset{\epsilon 0}{lim}\mathrm{\Phi }(u_\epsilon )=\mathrm{\Phi }(u).$$

(6) $`\mathrm{\Phi }(max\{u,v\})min\{\mathrm{\Phi }(u),\mathrm{\Phi }(v)\}`$.

Proof. Assertions (1), (2) follow from Proposition 4.1.3. Assertion (3) follows from Proposition 4.1.4. Let us prove (4). By Theorem 3.1.7 $`(^2u_j)(^2u)`$ in the weak topology on the space of $`_n`$-valued measures on $`\mathrm{\Omega }`$. This and Proposition 4.1.5 imply assertion (4). The assertions (5), (6) are proved exactly as in the complex case, and we refer to the proof of Theorem 5.7 in . Q.E.D.

###### 4.1.8 Theorem.

Let $`n>1`$. Let $`u`$ be a finite psh function on $`\mathrm{\Omega }`$ such that the regularizations of $`u`$, $`u_\epsilon =u\chi _\epsilon `$, have the property that $`det(^2u_\epsilon )`$ is a bounded family of Borel measures on each compact subset of $`\mathrm{\Omega }`$. Then

(1) $`\mathrm{\Phi }(u)`$ is absolutely continuous with respect to the Lebesgue measure, and if $`\mathrm{\Phi }(u)=gdvol`$ then $`gL_{loc}^n(\mathrm{\Omega })`$, i.e. $`g^n`$ is locally integrable;

(2) if $`u`$ is continuous and if $`det(^2u)=fdvol+d\nu `$ is the Lebesgue decomposition of the non-negative measure $`det(^2u)`$ into its absolutely continuous and singular parts then $`g^nf`$;

(3) if $`\frac{^2u}{\overline{q}_iq_j}=f_{\overline{i}j}+d\nu _{\overline{i}j}`$ is the Lebesgue decomposition of the Borel measures $`\frac{^2u}{\overline{q}_iq_j}`$ then $`g=(det(f_{\overline{i}j}))^{\frac{1}{n}}`$.

Proof. The proof of this theorem is exactly as in the complex case, and we refer to the proof of Theorem 5.8 in . Q.E.D.

###### 4.1.9 Remark.

The assumptions of Theorem 4.1.8 are satisfied when $`u`$ is a continuous psh function.

## 5 On upper envelopes.

Let $`\mathrm{\Omega }\text{ }\text{ }\text{H}^n`$ be a domain. For functions $`f0`$ on $`\mathrm{\Omega }`$, and $`\varphi C(\mathrm{\Omega })`$ let us denote by

$$(\varphi ,f):=\{v\text{ is finite psh on }\mathrm{\Omega }|\mathrm{\Phi }(v)fvol,\overline{\underset{q\zeta }{lim}}v(q)\varphi (\zeta )\zeta \mathrm{\Omega }\}.$$

The main result of this section is the following result which is a quaternionic analogue of Theorem 6.2 from .

###### 5.1.1 Theorem.

Let $`\mathrm{\Omega }`$ be a strictly pseudoconvex bounded domain with smooth boundary. Let $`\varphi C(\mathrm{\Omega }),fC(\overline{\mathrm{\Omega }}),f0`$. Let

$$u(z):=\underset{v(\varphi ,f)}{supv(z)}.$$

Then $`uC(\overline{\mathrm{\Omega }})`$, $`u`$ is psh in $`\mathrm{\Omega }`$, and $`u|_\mathrm{\Omega }\varphi `$. Moreover $`u(\varphi ,f)`$.

The proof of this theorem closely follows  and ; we are going to present it. As in , we will need two lemmas. Throughout this subsection $`\mathrm{\Omega }`$ will denote a strictly pseudoconvex bounded domain in $`_n`$.

###### 5.1.2 Lemma.

Fix $`\epsilon >0`$. Then for every $`\zeta \mathrm{\Omega }`$ there exists $`v_\zeta (\varphi ,f)C(\overline{\mathrm{\Omega }})`$ such that

$$\varphi (\zeta )\epsilon v_\zeta (\zeta )\varphi (\zeta ).$$

###### 5.1.3 Lemma.

Fix $`\epsilon >0`$. Then for every $`\zeta \mathrm{\Omega }`$ there exists $`h_\zeta C(\overline{\mathrm{\Omega }})`$ which is psh in $`\mathrm{\Omega }`$ such that

1) $`h_\zeta (z)\varphi (z)`$ for all $`z\mathrm{\Omega }`$.

2) $`h_\zeta (\zeta )\varphi (\zeta )\epsilon `$.

First let us show that Lemma 5.1.3 implies Lemma 5.1.2. Choose a large constant $`K0`$ such that $`\mathrm{\Phi }(K|z|^2)=K\mathrm{\Phi }(|z|^2)>f`$. Let $`\stackrel{~}{\varphi }(z):=\varphi (z)K(|z|^2|\zeta |^2)`$. Let $`h_\zeta `$ be as in Lemma 5.1.3 applied for the function $`\stackrel{~}{\varphi }`$ instead of $`\varphi `$. Then the function $`v_\zeta (z):=h_\zeta (z)+K(|z|^2|\zeta |^2)`$ satisfies Lemma 5.1.2.

Proof of Lemma 5.1.3. By assumption (at least locally in a neighborhood of $`\zeta `$) $`\mathrm{\Omega }=\{F<0\}`$ where $`F`$ is twice continuously differentiable function on $`\text{ }\text{ }\text{H}^n`$ which is strictly psh and $`F|_\mathrm{\Omega }0`$. Let $`G(z):=F(z)\delta |z\zeta |^2`$. For small $`\delta `$ the function $`G`$ is psh in a neighborhood $`𝒪`$ of $`\zeta `$. Clearly

$$G(\zeta )=0,G|_{(\overline{\mathrm{\Omega }}\{\zeta \})𝒪}<0.$$

Take small $`\lambda >0`$ and consider

$$F^{}(z):=max\{G,\lambda \}.$$

It is clear that

1) $`F^{}`$ is psh in $`\mathrm{\Omega }`$ and continuous in $`\overline{\mathrm{\Omega }}`$;

2) $`F^{}(\zeta )=0`$;

3) $`F^{}|_{\overline{\mathrm{\Omega }}\zeta }<0.`$
For given $`\epsilon >0`$ there exists a constant $`C0`$ such that

$$CF^{}(z)+\varphi (\zeta )<\varphi (z)+\epsilon \text{ for all }z\mathrm{\Omega }.$$

Let $`h_\zeta (z):=CF^{}(z)+\varphi (\zeta )\epsilon `$. Then

$$h_\zeta (z)<\varphi (z)\text{ for all }z\mathrm{\Omega },$$

$$h_\zeta (\zeta )=\varphi (\zeta )\epsilon .$$

This proves Lemma 5.1.3. Q.E.D.

Proof of Theorem 5.1.1. Let us define the upper regularization of $`u`$ in $`\overline{\mathrm{\Omega }}`$ as usual:

$$u^{}(z):=\underset{z^{}z}{lim\; sup}u(z^{}).$$

It is easy to see that $`u^{}`$ is psh in $`\mathrm{\Omega }`$ (e.g. using Lemma 3.1.8 and the analogous classical result for subharmonic functions, see  Ch. 1, §1.5) . Clearly $`uu^{}`$.

It follows from Lemma 5.1.2 that

$$u(z)\varphi (z)z\mathrm{\Omega }.$$

In order to prove that $`u`$ coincides with $`\varphi `$ on $`\mathrm{\Omega }`$ let us prove the converse inequality (following ). Fix $`\zeta \mathrm{\Omega }`$. As in the proof of Lemma 5.1.3 construct $`F^{}`$. Let $`F^{}:=F^{}`$. Then

1) $`F^{}`$ is super-harmonic;

2) $`F^{}(\zeta )=0`$;

3) $`F^{}(z)>0`$ for $`z\overline{\mathrm{\Omega }}\zeta `$.
In the classical potential theory $`F^{}`$ is called barrier, and hence the classical Dirichlet problem for harmonic functions is solvable on $`\mathrm{\Omega }`$. Hence there exists a harmonic in $`\mathrm{\Omega }`$ function $`hC(\overline{\mathrm{\Omega }})`$ such that $`h|_\mathrm{\Omega }\varphi `$. Since every function from $`(\varphi ,f)`$ is subharmonic we obtain that $`u(z)h(z)z\overline{\mathrm{\Omega }}.`$ Since $`h`$ is continuous we get

$$u^{}(z)h(z)\text{ on }\overline{\mathrm{\Omega }},$$

$$\text{ hence }u^{}(z)\varphi (z)\text{ on }\mathrm{\Omega }.$$

Finally we deduce

$$u(z)=u^{}(z)=\varphi (z),z\mathrm{\Omega }.$$

By H. Cartan’s theorem (see e.g. ) $`u=u^{}`$ almost everywhere in $`\mathrm{\Omega }`$. Since $`(\varphi ,f)`$ is closed under taking finite maximums (Theorem 4.1.7 (6)) Choquet’s lemma (see e.g. ) implies that one can choose an increasing sequence of functions $`u_j(\varphi ,f)`$ which converges to $`u`$ almost everywhere in $`\mathrm{\Omega }`$. But then $`u_ju^{}`$ in $`L_{loc}^1(\mathrm{\Omega })`$. Hence by Theorem 3.1.7 (ii) $`^2u_j^2u^{}`$ weakly. Hence by Proposition 4.1.5

$$\mathrm{\Phi }(u^{})\underset{j\mathrm{}}{lim}\mathrm{\Phi }(u_j)\mu :=fvol=\mu .$$

Hence $`u^{}(\varphi ,f)`$. Since $`uu^{}`$ in $`\overline{\mathrm{\Omega }}`$ we conclude

$$uu^{}.$$

Hence $`u`$ is psh and $`u(\varphi ,f)`$. Hence to finish the proof of Theorem 5.1.1 it remains to prove the continuity of $`u`$ in $`\overline{\mathrm{\Omega }}`$.

First we will prove the following

###### 5.1.4 Claim.

$`u`$ is continuous at all points of the boundary $`\mathrm{\Omega }`$.

Proof. Fix any $`\epsilon >0`$ and any $`\zeta \mathrm{\Omega }`$. By Lemma 5.1.2 there exists a function $`v_\zeta (\varphi ,f)C(\overline{\mathrm{\Omega }})`$ such that $`\varphi (\zeta )\epsilon v_\zeta (\zeta )`$. Since $`v_\zeta `$ is continuous, in a small neighborhood $`U`$ of $`\zeta `$ in $`\overline{\mathrm{\Omega }}`$

$$v_\zeta (z)>\varphi (\zeta )2\epsilon .$$

But $`u(z)v_\zeta (z)`$. Hence $`u(z)>\varphi (\zeta )2\epsilon `$ for $`zU`$. Hence $`\underset{z\zeta }{lim\; inf}u(z)\varphi (\zeta )`$. But $`u`$ is upper semi-continuous in $`\overline{\mathrm{\Omega }}`$ (since $`uu^{}`$), hence $`\underset{z\zeta }{lim\; sup}u(z)u(\zeta )=\varphi (\zeta )`$. Hence $`u`$ is continuous at $`\zeta `$. This proves the claim. Q.E.D.

Now let us continue proving Theorem 5.1.1. Fix $`\epsilon >0`$. Let $`\omega (\epsilon )>0`$ be such that

$$\underset{\underset{|zz^{}|<\omega (\epsilon )}{z,z^{}\overline{\mathrm{\Omega }}}}{sup}|f(z)f(z^{})|<\frac{\epsilon }{2}\text{ and }\underset{\underset{|zz^{}|<\omega (\epsilon )}{z,z^{}\overline{\mathrm{\Omega }},dist(z,\mathrm{\Omega })<3\omega (\epsilon )}}{sup}|u(z)u(z^{})|<\epsilon ,$$

where $`dist(z,\mathrm{\Omega })`$ denotes the shortest distance from $`z`$ to $`\mathrm{\Omega }`$. Existence of such $`\omega (\epsilon )`$ follows from Claim 5.1.4 and the continuity of $`f`$. Let $`\tau \text{ }\text{ }\text{H}^n`$ be any vector with $`|\tau |<\omega (\epsilon )`$ (where $`||`$ denotes the norm of the vector). Let

$$v(z):=u(z+\tau )+\epsilon |z|^2(L+1)\epsilon ,$$

where $`L`$ be any constant satisfying $`L>|z|^2`$ for all $`z\mathrm{\Omega }`$. Let

$$V(z,\tau ):=\{\begin{array}{c}u(z),\text{ if }z\overline{\mathrm{\Omega }},z+\tau \mathrm{\Omega }\\ max\{u(z),v(z)\},\text{ if }z\overline{\mathrm{\Omega }},z+\tau \mathrm{\Omega }\end{array}$$

###### 5.1.5 Lemma.

$`V(z,\tau )(\varphi ,f)`$.

Let us postpone the proof of this lemma and let us finish the proof of Theorem 5.1.1. Lemma 5.1.5 implies in particular that $`V(z,\tau )u(z)`$ for all $`z\mathrm{\Omega }`$. Hence for any $`z,z+\tau \mathrm{\Omega }`$ such that $`|\tau |<\omega (\epsilon )`$ we have

$$u(z+\tau )+\epsilon |z|^2(L+1)\epsilon u(z).$$

Hence for some constant $`C`$

$$u(z+\tau )u(z)<C\epsilon .$$

Replacing $`\tau `$ by $`\tau `$ we get

$$|u(z+\tau )u(z)|<C\epsilon .$$

Hence $`u`$ is continuous. Q.E.D.

Thus it remains to prove Lemma 5.1.5.

Proof of Lemma 5.1.5. Let us check all the conditions in the definition of $`(\varphi ,f)`$.

###### 5.1.6 Claim.

$`V(z,\tau )\varphi (z)`$ for all $`z\mathrm{\Omega }`$.

Proof. Indeed, if $`z+\tau \mathrm{\Omega }`$ then $`V(z,\tau )=u(z)=\varphi (z)`$. If $`z+\tau \mathrm{\Omega }`$ then either $`V(z,\tau )=u(z)=\varphi (z)`$ or

$$V(z,\tau )=u(z+\tau )+\epsilon |z|^2(L+1)\epsilon u(z+\tau )\epsilon =$$

$$u(z)+(u(z+\tau )u(z))\epsilon u(z)=\varphi (z).$$

Q.E.D.

Let us define the subset $`\mathrm{\Gamma }:=\{z\overline{\mathrm{\Omega }}|z+\tau \mathrm{\Omega }\}`$. Note that for any point $`x\mathrm{\Gamma }`$, $`dist(x,\mathrm{\Omega })\omega (\epsilon )`$. Let $`A`$ be the $`\omega (\epsilon )`$-neighborhood of $`\mathrm{\Gamma }`$. Then clearly for all $`xA`$ one has $`dist(x,\mathrm{\Omega })2\omega (\epsilon )`$.

###### 5.1.7 Claim.

For all $`zA`$ one has $`V(z,\tau )=u(z)`$. Hence $`V(z,\tau )`$ is upper semi-continuous in $`\mathrm{\Omega }`$.

Proof. Clearly it is sufficient to prove the first statement. We have to check that for $`zA`$, $`v(z)u(z)`$. We have

$$v(z)=u(z+\tau )+\epsilon |z|^2(L+1)\epsilon u(z+\tau )\epsilon $$

$$u(z)+(u(z+\tau )u(z))\epsilon u(z)$$

where the last inequality follows from the fact that $`dist(z,\mathrm{\Omega })\omega (\epsilon ),|\tau |<\omega (\epsilon )`$ and the definition of $`\omega (\epsilon )`$. Q.E.D.

Since the maximum of two psh functions is psh we can easily get from the last claim

###### 5.1.8 Claim.

$`V(z,\tau )`$ is psh in $`\mathrm{\Omega }`$.

To finish the proof of Lemma 5.1.5 it remains to prove

###### 5.1.9 Claim.

$`\mathrm{\Phi }(V(z,\tau )\mu (=fdvol).`$

Proof. Let us denote by $`\mathrm{\Gamma }_0:=\mathrm{\Gamma }\mathrm{\Omega }`$. By Claim 5.1.7 in a small neighborhood of $`\mathrm{\Gamma }_0`$, or if $`z+\tau \mathrm{\Omega }`$, we have

$$\mathrm{\Phi }(V(z,\tau ))=\mathrm{\Phi }(u(z))\mu .$$

Now it remains to consider domain $`\{z\mathrm{\Omega }|z+\tau \mathrm{\Omega }\}`$. In this domain $`V(z,\tau )=max\{u(z),v(z)\}`$. Hence by Theorem 4.1.7 (6) we get

$$\mathrm{\Phi }(max\{u,v\})min\{\mathrm{\Phi }(u),\mathrm{\Phi }(v)\}.$$

Since $`\mathrm{\Phi }(u)\mu `$ let us prove that $`\mathrm{\Phi }(v)\mu `$. Indeed

$$\mathrm{\Phi }(v(z))\mathrm{\Phi }(u(z+\tau ))+4\epsilon f(z+\tau )+4\epsilon f(z).$$

Q.E.D.

Thus Lemma 5.1.5 and hence Theorem 5.1.1 are proved.

### 5.2 Other Perron-Bremermann families.

Let $`\mathrm{\Omega }`$ be a domain in $`\text{ }\text{ }\text{H}^n`$. For brevity we will denote by $`P(\mathrm{\Omega })`$ the class of psh functions in $`\mathrm{\Omega }`$. Given $`\varphi C(\mathrm{\Omega })`$ and a non-negative measure $`\mu =fdvol`$ on $`\mathrm{\Omega }`$, we define three Perron-Bremermann families of subsolutions to the Monge-Ampère equation (the first one was defined earlier in Subsection 5.1):

$`(\varphi ,f):=\{vP(\mathrm{\Omega })|\mathrm{\Phi }(v)\mu \text{ and }\underset{zz_0}{limsup}v(z)\varphi (z_0),\text{ for all }z_0\mathrm{\Omega }\}`$;

$`C(\varphi ,f):=(\varphi ,f)C(\overline{\mathrm{\Omega }})`$;

$`(\varphi ,\mu ):=\{vP(\mathrm{\Omega })C(\overline{\mathrm{\Omega }})|det(^2v)\mu \text{ and }v(z_0)\varphi (z_0)\text{ for all }z_0\mathrm{\Omega }\}.`$

If $`\mu =fdvol,fL_{loc}^n(\mathrm{\Omega })`$ then let $`\mu ^n:=f^ndvol`$. If $`vP(\mathrm{\Omega })C(\overline{\mathrm{\Omega }})`$ then by Theorem 4.1.8 (2)

$$\mathrm{\Phi }(v)^ndet(^2v),$$

and consequently

$$C(\varphi ,f)(\varphi ,\mu ^n).$$

By Theorem 4.1.8 (1) and Remark 4.1.9 if $`u`$ is continuous then $`\mathrm{\Phi }(u)`$ is absolutely continuous with respect to the Lebesgue measure, $`\mathrm{\Phi }(u)=gdvol`$, and $`gL_{loc}^n(\mathrm{\Omega })`$.

###### 5.2.1 Proposition.

Let $`\mathrm{\Omega }`$ be a bounded domain in $`\text{ }\text{ }\text{H}^n`$ and suppose that $`uP(\mathrm{\Omega })C(\overline{\mathrm{\Omega }})`$ satisfies $`det(^2u)=(\mathrm{\Phi }(u))^n`$. If $`C:=C(\varphi ,\mathrm{\Phi }(u))`$ and $`:=(\varphi ,det(^2u))`$, where $`\varphi =u|_\mathrm{\Omega }`$, then $`sup\{v|v\}=sup\{v|vC\}=u`$.

Proof. The remarks before this proposition and the assumption imply that $`C`$. Hence $`sup\{v|vC\}sup\{v|v\}`$. On the other hand obviously $`uC`$. Hence $`usup\{v|vC\}`$. Thus it remains to show that $`sup\{v|v\}u`$. Fix any $`v`$. By the minimum principle, Theorem 2.1.3, $`uv`$ attains its minimum on $`\mathrm{\Omega }`$. But since $`uv`$ on $`\mathrm{\Omega }`$ we obtain that $`uv`$ in $`\overline{\mathrm{\Omega }}`$. The proposition is proved. Q.E.D.

## 6 The Monge-Ampère equation in the Euclidean ball.

In this section we prove the existence of the solution of the Dirichlet problem for the quaternionic Monge-Ampère equation for the unit Euclidean ball assuming sufficient regularity of the initial data $`f,\varphi `$. First let us introduce some notation.

Let $`B`$ denote the open unit ball in $`\text{ }\text{ }\text{H}^n`$,

$$B:=\{|q|<1\}.$$

Let $`\overline{B}`$ be its closure. The main result of this section is as follows:

###### 6.0.1 Theorem.

Let $`fC^{\mathrm{}}(\overline{B}),f>0`$. Let $`\varphi C^{\mathrm{}}(B)`$. There exists unique psh function $`uC^{\mathrm{}}(\overline{B})`$ which is a solution of the Dirichlet problem

$$det(\frac{^2u}{\overline{q}_iq_j})=f,$$

$$u|_B=\varphi .$$

This (smooth) case will be used in the proof of the general case (Theorem 0.1.3). The method of the proof of this case is a modification of that of the paper by Caffarelli-Kohn-Nirenberg-Spruck .

In this section we will denote by $`g_k`$ the $`C^k`$\- norm of a function $`g`$ in $`\overline{B}`$.

The proof uses the continuity method. In Subsection 6.1 we prove the first order a priori estimates. In Subsection 6.2 we prove the second order a priori estimates. In Subsection 6.3 we obtain $`C^{2,\alpha }`$ a priori estimates as an easy consequence of the results from Subsections 6.1 and 6.2 and a general result from . Then the higher smoothness results follow from these by the standard regularity theory of elliptic equations of second order (see e.g. , ).

### 6.1  First order estimates.

###### 6.1.1 Proposition.

Assume that a psh function $`uC^2(\overline{B})`$ satisfies the quaternionic Monge-Ampère equation with $`f>0`$ in $`\overline{B}`$. Then

$$u_1C,$$

with a constant $`C`$ depending only on $`f_1,\varphi _2`$, and $`f^1_0`$.

Proof. Let $`L`$ be the linearization of the operator $`vlog(det(^2v))`$ at $`u`$. Explicitly this operator can be written

$$Lv=nf^1det(^2v,^2u[n1]).$$

Clearly $`Lu=n`$. Since $`u`$ is strictly psh we have

Claim.The operator $`L`$ is elliptic.
Let $`D`$ be a first order differential operator of the form $`D=\frac{}{x_i},`$ where $`x_i`$ is one of the real coordinate axes in $`\text{ }\text{ }\text{H}^n`$. First let us prove the following lemma.

###### 6.1.2 Lemma.

$$max_{\overline{B}}|Du|max_B|Du|+C,$$

where $`C`$ is a constant depending only on $`f_1,\varphi _1`$, and $`f^1_0`$ .

Proof. We have

$$L(Du)=nf^1det(^2(Du),^2u[n1])=f^1D(det(^2u))=D(\mathrm{log}f).$$

Consider the function $`w(q):=\pm Du+\lambda |q|^2`$, with $`\lambda 0`$. Then we get

$$Lw=\pm D(\mathrm{log}f)+\lambda L(|q|^2).$$

But

$$L(|q|^2)=8nf^1det(I,\underset{n1\text{ times}}{\underset{}{^2u,\mathrm{},^2u}})=8nf^1\underset{i=1}{\overset{n}{}}det(M_{ii}(^2u)),$$

where $`M_{ii}(A)`$ denotes the minor of a matrix $`A`$ obtained from $`A`$ by deleting the $`i`$-th row and the $`i`$-th column.

Claim. Let $`A`$ be an invertible hyperhermitian matrix of order $`n`$. For any $`i,\mathrm{\hspace{0.17em}1}in`$,

$$(A^1)_{ii}=\frac{1}{detA}detM_{ii}(A).$$

Thus using this claim we get

$$Lw=\pm D(\mathrm{log}f)+8n\lambda Tr(^2u)^1.$$

For any hyperhermitian positive definite $`(n\times n)`$ matrix $`C`$ one has $`\frac{1}{n}Tr(C)(detC)^{\frac{1}{n}}`$. Hence we get

$$Lw\pm D(\mathrm{log}f)+8n^2\lambda (det(^2u))^{\frac{1}{n}}=\pm D(\mathrm{log}f)+8n^2\lambda f^{\frac{1}{n}}.$$

Since $`fC^1(\overline{B})`$ and $`f`$ is bounded from below by a positive constant, one can choose a large $`\lambda `$ such that the last expression will be positive. For such a $`\lambda `$ by the maximum principle the function $`w`$ achieves its maximum on the boundary $`B`$. This proves Lemma 6.1.2.

Thus it remains to estimate the gradient $`u`$ on the boundary $`B`$. First let $`\stackrel{~}{\varphi }`$ denote any $`C^2`$\- smooth extension of $`\varphi `$ inside the closed ball $`\overline{B}`$ such that its $`C^2`$-norm can be estimated by the $`C^2`$-norm of $`\varphi `$. Consider the function $`\stackrel{~}{\varphi }+K(|q|^21)`$ for large $`K`$. Let us denote this extension again by $`\varphi `$. Note that on the boundary $`B`$ it coincides with our original $`\varphi `$. Note also that for large $`K`$ the function $`\varphi `$ is psh and moreover

$$det(^2\varphi )f=det(^2u).$$

Hence by the minimum principle $`\varphi u`$ in $`\overline{B}`$. Next let $`h`$ be a harmonic function in $`B`$ which extends $`\varphi `$. Then $`uh`$. Hence on the boundary $`|u|max\{|h|,|\varphi |\}`$. Thus Proposition 6.1.1 is proved. Q.E.D.

### 6.2 Second order estimates.

Let $`D`$ be any real first order differential operator with constant coefficients which are not greater than one. First we need the following result.

###### 6.2.1 Lemma.

For a constant $`C`$ depending only on $`f_2,f^1_0`$

$$max_{\overline{B}}D^2umax_BD^2u+C.$$

Proof. We have

$$D^2(\mathrm{log}f)=D^2(\mathrm{log}det(^2u))=D(\frac{D(det^2u)}{det^2u})=$$

$$f^2\left\{D^2(det^2u)det(^2u)(D(det^2u))^2\right\}=$$

$$f^2\left\{D[ndet(^2(Du),^2u[n1])]det(^2u)[ndet(^2(Du),^2u[n1])]^2\right\}=$$

$$f^1ndet(^2(D^2u),^2u[n1])+$$

$$f^2\left\{n(n1)det(^2(Du)[2],^2u[n2])det(^2u)[ndet(^2(Du),^2u[n1])]^2\right\}.$$

We need the following

###### 6.2.2 Lemma.

Let $`A,B`$ be hyperhermitian $`(n\times n)`$-matrices, $`A>0`$. Then

$$n(n1)det(B[2],A[n2])detA(ndet(B,A[n1]))^20.$$

Assuming this lemma let us finish the proof of Lemma 6.2.1. We get

$$D^2(\mathrm{log}f)f^1ndet(^2(D^2u),^2u[n1])=L(D^2u).$$

Hence we have

$$L(D^2u+\lambda |q|^2)D^2(\mathrm{log}f)+8n^2\lambda f^{\frac{1}{n}},$$

where we have used the lower estimate on $`L(|q|^2)`$ from the previous subsection. For sufficiently large $`\lambda `$ the last expression is positive. Hence by the maximum principle

$$max_B(D^2u+\lambda |q|^2)max_B(D^2u+\lambda |q|^2).$$

Thus Lemma 6.2.1 follows. Q.E.D.

Proof of Lemma 6.2.2. The function $`A\mathrm{log}(detA)`$ is concave on the cone of positive definite hyperhermitian matrices (see , Theorem 1.1.17 (i)). Hence

$$\frac{d^2}{dt^2}(\mathrm{log}det(A+tB))|_{t=0}0.$$

Computing explicitly this derivative we obtain the lemma. Q.E.D.

Note now that in order to prove an estimate on the second derivatives of $`u`$ it is sufficient to prove an upper estimate on it. Indeed let $`q_l=t+ix+jy+kz`$ be one of the quaternionic coordinates. Since $`u`$ is psh we have $`u_{tt}+u_{xx}+u_{yy}+u_{zz}0`$. This and the upper estimates on the second derivatives of the form $`D^2u`$ imply the lower estimates on them. The estimates on the mixed derivatives also can be obtained easily since

$$2u_{tx}=(\frac{}{t}+\frac{}{x})^2uu_{tt}u_{xx}.$$

Hence we have to prove an upper estimate of $`D^2u`$ on $`B`$. Let us introduce additional notation. Let $`r(q)=|q|^21`$. Then

$$B=\{r<0\}.$$

We will denote the quaternionic units as follows:

$$e_0=1,e_1=i,e_2=j,e_3=k.$$

Fix a coordinate system $`(q_1,\mathrm{},q_n)`$ on $`\text{ }\text{ }\text{H}^n`$; we will write $`q_i=_{\epsilon =0}^3e_\epsilon x_i^\epsilon `$. Fix an arbitrary point $`PB`$. We can choose such a coordinate system near this point that the inner normal to $`B`$ at $`P`$ coincides with the axis $`x_n^0`$. Also we will move the center of coordinates to $`P`$, i.e. we will assume that $`P`$ coincides with $`0`$. Let us denote the center of the ball $`B`$ by $`R`$.

First we have the following trivial estimates:

$$|u_{x_i^\epsilon ,x_j^\delta }(P)|C\text{ for }(i,\epsilon ),(j,\delta )(n,0).$$

(Note that here we also use the first order estimates of $`u`$ and $`\varphi `$). Now let us prove the following estimate:

###### 6.2.3 Lemma.

$$|u_{x_i^\epsilon ,x_n^0}(P)|C\text{ for }(i,\epsilon )(n,0),$$

where $`C`$ depends only on $`f_2,\varphi _3,f^1_0`$.

Proof. Clearly one can construct a vector field $`T`$ on $`\text{ }\text{ }\text{H}^n`$ such that

1) $`T(P)=\frac{}{x_i^\epsilon }`$;

2) on the points of $`B`$, $`T`$ is parallel to $`B`$;

3) $`T`$ has the form

$$T=\frac{}{x_i^\epsilon }+a\frac{}{x_n^0},$$

where the function $`a`$ is smooth with estimates on the derivatives depending only on $`n`$, and $`a(P)=0`$.

Consider the function

$$w(q):=\pm T(u\varphi )+(u_{x_n^1}\varphi _{x_n^1})^2+(u_{x_n^2}\varphi _{x_n^2})^2+(u_{x_n^3}\varphi _{x_n^3})^2Ax_n^0+B|qR|^2.$$

We will show that for $`A,B`$ sufficiently large

$`(a)Lw>0;`$
$`(b)w|_B0.`$
If we will prove it then by the maximum principle $`w0`$ in $`\overline{B}`$. Hence

$$|T(u\varphi )|Ax_n^0\text{ in }\overline{B}.$$

Hence at the point $`P`$, $`|\frac{}{x_n^0}T(u\varphi )|A.`$ This will finish the proof of Lemma 6.2.3. Thus let us check the conditions (a) and (b). By a straightforward computation

$$LT(u\varphi )=T(\mathrm{log}f)(L(\varphi _{x_i^\epsilon })+aL(\varphi _{x_n^0}))+(u\varphi )_{x_n^0}La+$$

$$nf^1det((a_{\overline{i}}(u\varphi )_{x_n^0,j})+((u\varphi )_{x_n^0,\overline{i}}a_j),^2u[n1]).$$

where we denote for brevity $`g_i:=\frac{g}{q_i},g_{\overline{i}}:=\frac{g}{\overline{q}_i}`$. However $`\frac{g}{x_n^0}=g_{\overline{n}}e_1\frac{g}{x_n^1}e_2\frac{g}{x_n^2}e_3\frac{g}{x_n^3}`$. Hence

$$LT(u\varphi )=T(\mathrm{log}f)(L(\varphi _{x_i^\epsilon })+aL(\varphi _{x_n^0}))+(u\varphi )_{x_n^0}La+$$

$$nf^1\left(det((a_{\overline{i}}(u\varphi )_{\overline{n},j})+(a_{\overline{i}}(u\varphi )_{\overline{n},j})^{},^2u[n1])\right)$$

$$nf^1\underset{l=1}{\overset{3}{}}det((a_{\overline{i}}e_l(u\varphi )_{x_n^l,j})+(a_{\overline{i}}e_l(u\varphi )_{x_n^l,j})^{},^2u[n1]).$$

Using first and second order estimates on $`a`$ and $`\varphi `$, and first order estimates on $`f`$ and $`u`$ we get the following inequality:

$$|LT(u\varphi )|C+Cnf^1det(I,^2u[n1])+$$

$$nf^1\left(det((a_{\overline{i}}u_{\overline{n},j})+(a_{\overline{i}}u_{\overline{n},j})^{},^2u[n1])\right)+$$

$$nf^1\underset{l=1}{\overset{3}{}}det((a_{\overline{i}}e_l(u\varphi )_{x_n^l,j})+(a_{\overline{i}}e_l(u\varphi )_{x_n^l,j})^{},^2u[n1]).$$

We have the following linear algebraic identity:

Claim.

$$det((a_{\overline{i}}u_{\overline{n},j})+(a_{\overline{i}}u_{\overline{n},j})^{},^2u[n1])=2(Rea_{\overline{n}})det(^2u).$$

It follows from Theorem 1.1.15 (i) of  that for a fixed $`n\times n`$ positive definite hyperhermitian matrix $`A`$ the bilinear form $`det(XX^{},A[n1])`$ is non-negative definite on the space of quaternionic $`n`$-columns. Hence we get

$$|det(XY^{}+YX^{},A[n1])|det(XX^{},A[n1])+det(YY^{},A[n1]).$$
(1)
Using this inequality and the last claim we obtain the following estimate:

$$|LT(u\varphi )|C+Cnf^1det(I,^2u[n1])+2n|Rea_{\overline{n}}|+$$

$$\underset{l=1}{\overset{3}{}}nf^1\left(det(((u\varphi )_{x_n^l,\overline{i}}(u\varphi )_{x_n^lj}),^2u[n1])+det((a_{\overline{i}}a_j),^2u[n1])\right).$$

Using again the first order estimates on $`a`$, we finally get

$$|LT(u\varphi )|C+Cnf^1det(I,^2u[n1])+$$

$$nf^1\underset{l=1}{\overset{3}{}}det(((u\varphi )_{x_n^l,\overline{i}}(u\varphi )_{x_n^lj}),^2u[n1]),$$
$`(2)`$
but now the value of the constant $`C`$ might be different from the previous one.

Now let us compute $`L((u_{x_n^l}\varphi _{x_n^l})^2)`$. By a straightforward computation we have

$$L((u_{x_n^l}\varphi _{x_n^l})^2)=$$

$$2nf^1det(((u\varphi )_{x_n^l,\overline{i}}(u\varphi )_{x_n^l,j})+(u\varphi )_{x_n^l}((u\varphi )_{x_n^l,\overline{i}j}),^2u[n1])=$$

$$2nf^1det(((u\varphi )_{x_n^l,\overline{i}}(u\varphi )_{x_n^l,j}),^2u[n1])+$$

$$2(u\varphi )_{x_n^l}\left((\mathrm{log}f)_{x_n^l}nf^1det(\varphi _{x_n^l,\overline{i}j},^2u[n1])\right).$$

Using this identity and (2) we obtain:

$$LwC+(8BC)nf^1det(I,^2u[n1])+$$

$$\underset{l=1}{\overset{3}{}}nf^1det(((u\varphi )_{x_n^l,\overline{i}}(u\varphi )_{x_n^l,j}),^2u[n1])+$$

$$2\underset{l=1}{\overset{3}{}}(u\varphi )_{x_n^l}\left((\mathrm{log}f)_{x_n^l}nf^1det(\varphi _{x_n^l,\overline{i}j},^2u[n1])\right).$$

But the third summand is non-negative. Hence we get

$$LwC+(8BC)nf^1det(I,^2u[n1])+$$

$$2\underset{l=1}{\overset{3}{}}(u\varphi )_{x_n^l}\left((\mathrm{log}f)_{x_n^l}nf^1det(\varphi _{x_n^l,\overline{i}j},^2u[n1])\right).$$

Using the first order estimates on $`u`$ and $`f`$ and third order estimates on $`\varphi `$ we finally obtain

$$LwC^{}+(8BC^{})nf^1det(I,^2u[n1]).$$

As in the proof of Lemma 6.1.2

$$det(I,^2u[n1])nf^{\frac{n1}{n}}.$$

Thus for large $`B`$ we get

$$LwC^{}+(8BC^{})n^2f^{\frac{1}{n}}>0.$$

Thus the inequality (a) is proved. It remains to prove the inequality (b), namely

$$(b)w|_B0$$

for large $`A,B`$. Note that $`T(u\varphi )|_B0`$. Clearly it is sufficient to prove the inequality (b) only near the point $`P`$. Since $`u\varphi `$ on $`B`$, then using the first order estimates on $`u`$ it is easy to see that for $`l=1,2,3`$

$$|u_{x_n^l}(q)\varphi _{x_n^l}(q)|<C|q|,qB.$$

But for $`qB`$ we have $`|q|K(x_n^0)^{\frac{1}{2}}.`$ Hence $`|u_{x_n^l}(q)\varphi _{x_n^l}(q)|^2<K^{}x_n^0`$. Thus Lemma 6.2.3 is proved. Q.E.D.

Thus to obtain an estimate on all second order derivatives of $`u`$ it remains to prove

$$|u_{x_n^0,x_n^0}(P)|<C.$$

We have proven that $`|u_{x_i^\epsilon ,x_j^\delta }(P)|<C`$ for $`(i,\epsilon ),(j,\delta )(n,0)`$ and

$$|u_{x_n^l,x_n^0}(P)|<C\text{ for }l0.$$
$`(3)`$
It suffices to show that

$$|u_{\overline{n}n}(P)|<C.$$

However by $`(3)`$ it is sufficient to show that for the $`(n1)\times (n1)`$\- matrix

$$(u_{\overline{\alpha },\beta }(P))_{\alpha ,\beta <n}cI$$
$`(4)`$
for some positive constant $`c`$. After subtracting a linear functional we may assume that $`\varphi _{x_j^l}(P)=0`$ for $`(j,l)(n,0)`$. In order to prove $`(4)`$ it is sufficient to prove that

$$\underset{\alpha ,\beta <n}{}\overline{\xi }_\alpha u_{\overline{\alpha }\beta }(P)\xi _\beta c|\xi |^2.$$

Let us prove it for $`\xi =(1,0,\mathrm{},0)`$. Namely $`u_{\overline{1}1}c`$.

Let us write on the boundary $`B`$ the coordinate $`x_n^0`$ as a function of other coordinates:

$$x_n^0=\rho ((x_i^\epsilon )_{(i,\epsilon )(n,0)}).$$

Let $`\stackrel{~}{u}:=u\lambda x_n^0`$ with $`\lambda `$ so chosen that

$$\mathrm{\Delta }_1\stackrel{~}{u}((x_i^\epsilon )_{(i,\epsilon )(n,0)},\rho ((x_i^\epsilon )_{(i,\epsilon )(n,0)}))=0\text{ at }P,$$

where $`\mathrm{\Delta }_1=_{\epsilon =0}^3\frac{^2}{(x_1^\epsilon )^2}`$. Since the first derivatives of $`\rho `$ vanish at $`P`$, the last equality is equivalent to

$$\stackrel{~}{u}_{\overline{1}1}(P)+\stackrel{~}{u}_{x_n^0}\rho _{\overline{1}1}(P)=0.$$
$`(5)`$
Consider the following Taylor decomposition:

$$\stackrel{~}{u}|_B=(\text{ quadratic terms in }x_i^\epsilon x_n^0)+(\text{ 3-order terms })+O(|q|^4)=$$

$$E+F+O(\underset{2jn}{}|q_j|^2)+O(|q|^4),$$

where

$$E:=(\text{ quadratic terms in }x_i^\epsilon x_n^0),$$

$$F:=(\text{ 3-order terms in }x_1^\epsilon ).$$

First let us consider the term $`E`$. We can estimate all the monomials which do not contain $`x_1^\epsilon `$ by $`C^{}_{2jn}|q_j|^2`$. Thus

$$E=\underset{\epsilon ,\delta =0}{\overset{3}{}}\underset{\begin{array}{c}j1\\ (j,\delta )(n,0)\end{array}}{}a_{\epsilon ,\delta ,j}x_1^\epsilon x_j^\delta +Q(x_1^\epsilon )+O(\underset{2jn}{}|q_j|^2),$$

where $`Q`$ is a quadratic polynomial in $`x_1^\epsilon `$ which satisfies $`\mathrm{\Delta }_1Q=0`$.

Now let us consider the expression $`F`$. It is well known (see e.g. ) that for any homogeneous polynomial $`F`$ of degree 3 on a Euclidean space $`\text{ }\text{ }\text{R}^N`$ there exists a unique decomposition $`F(x)=F_0(x)+l(x)|x|^2`$, where $`F_0`$ is a harmonic polynomial, and $`l`$ is a homogeneous polynomial of degree 1. Hence in our case ($`N=4`$) we can write

$$F=F_0+(\underset{\epsilon =0}{\overset{3}{}}b^\epsilon x_1^\epsilon )|q_1|^2,$$

with $`\mathrm{\Delta }_1F_0=0`$.

On the boundary of the unit Euclidean ball $`B`$ we have

$$2x_n^0=|q_1|^2+\underset{2jn1}{}|q_j|^2+\underset{\delta =1}{\overset{3}{}}|x_n^\delta |^2+O(|q|^3).$$

Thus

$$|q_1|^2=2x_n^0(\underset{2jn1}{}|q_j|^2+\underset{\delta =1}{\overset{3}{}}|x_n^\delta |^2)+O(|q|^3).$$

Hence

$$F=F_0+(\underset{\epsilon =0}{\overset{3}{}}b^\epsilon x_1^\epsilon )\left(2x_n^0(\underset{2jn1}{}|q_j|^2+\underset{\delta =1}{\overset{3}{}}|x_n^\delta |^2)+O(|q|^3)\right)=$$

$$F_0+\underset{\epsilon =0}{\overset{3}{}}2b^\epsilon x_1^\epsilon x_n^0+O(\underset{2jn}{}|q_j|^2)+O(|q|^4).$$

Thus we get an estimate

$$\stackrel{~}{u}|_B\underset{\epsilon ,\delta =0}{\overset{3}{}}\underset{\begin{array}{c}j1\\ (j,\delta )(n,0)\end{array}}{}a_{\epsilon ,\delta ,j}x_1^\epsilon x_j^\delta +Q(x_1^\epsilon )+$$

$$(F_0(x_1^\epsilon )+\underset{\epsilon =0}{\overset{3}{}}2b^\epsilon x_1^\epsilon x_n^0)+C\underset{2jn}{}|q_j|^2+O(|q|^4),$$

where $`\mathrm{\Delta }_1F_0=\mathrm{\Delta }_1Q=0`$. If we denote $`G:=F_0+Q`$ then the last estimate can be rewritten

$$\stackrel{~}{u}|_B\underset{\epsilon ,\delta =0}{\overset{3}{}}\underset{j1}{}a_{\epsilon ,\delta ,j}x_1^\epsilon x_j^\delta +G+C\underset{2jn}{}|q_j|^2+O(|q|^4).$$

Let us define

$$\widehat{u}:=\stackrel{~}{u}G.$$

Since $`G`$ depends only on $`q_1`$ and $`\mathrm{\Delta }_1G=0`$ then

$$\widehat{u}_{\overline{i}j}=\stackrel{~}{u}_{\overline{i}j}\text{ for }1i,jn.$$

We have

$$\widehat{u}|_B\underset{\epsilon ,\delta =0}{\overset{3}{}}\underset{j1}{}a_{\epsilon ,\delta ,j}x_1^\epsilon x_j^\delta +C\underset{2jn}{}|q_j|^2+O(|q|^4).$$

Now let us consider the following function

$$h:=\alpha x_n^0+\beta |q|^2+\frac{1}{2D}\underset{\epsilon ,\delta }{}\underset{j1}{}|a_{\epsilon ,\delta ,j}x_1^\epsilon +Dx_j^\delta |^2=$$

$$\alpha x_n^0+\beta |q|^2+\underset{\epsilon ,\delta }{}\underset{j1}{}a_{\epsilon ,\delta ,j}x_1^\epsilon x_j^\delta +D\underset{\epsilon ,\delta }{}\underset{j1}{}|x_j^\delta |^2+\theta ,$$

where $`\alpha ,\beta `$, and $`D`$ will be chosen later, and

$$\theta :=\frac{1}{2D}\underset{\epsilon ,\delta }{}\underset{j1}{}|a_{\epsilon ,\delta ,j}x_1^\epsilon |^20.$$

Hence

$$h|_B\alpha x_n^0+\beta |q|^2+\underset{\epsilon ,\delta }{}\underset{j1}{}a_{\epsilon ,\delta ,j}x_1^\epsilon x_j^\delta +4D\underset{j=2}{\overset{n}{}}|q_j|^2.$$

It is easy to see that for appropriate choices of large $`D`$ and small $`\alpha ,\beta `$ such that $`\alpha x_n^0+\beta |q|^20`$ one can obtain that $`h`$ is psh and

$$h|_B\widehat{u}|_B.$$

Now it is easy to see that the smallest eigenvalue of the matrix $`(h_{\overline{i}j})`$ is equal to $`4\beta `$. Clearly all the elements of this matrix are bounded independently of small $`\beta `$; hence all the other eigenvalues are bounded. Thus choosing sufficiently small $`\beta `$ we may assume that

$$det(h_{\overline{i}j})<f\text{ in }B.$$

Hence by the minimum principle

$$\widehat{u}h\text{ in }\overline{B}.$$

Since $`h(P)=\widehat{u}(P)=0`$ we obtain

$$\widehat{u}_{x_n^0}(P)h_{x_n^0}(P)=\alpha .$$

It is easy to see that $`\stackrel{~}{u}_{x_n^0}(P)=\widehat{u}_{x_n^0}(P)`$. Substituting this equality and the last inequality to $`(5)`$ we get

$$\stackrel{~}{u}_{\overline{1}1}(P)\alpha \rho _{\overline{1}1}(P)=c>0.$$

But $`u_{\overline{1}1}(P)=\stackrel{~}{u}_{\overline{1}1}(P)`$. Thus the second order estimate is proved. Q.E.D.

### 6.3 $`C^{2,\alpha }`$\- estimates.

In this subsection we prove a priori $`C^{2,\alpha }`$\- estimates on solutions of the Dirichlet problem for the quaternionic Monge-Ampère equation. As previously we denote by $`u`$ the solution of this problem. The main result of this subsection is

###### 6.3.1 Theorem.

Let $`\mathrm{\Omega }`$ be strictly pseudoconvex bounded domain in $`\text{ }\text{ }\text{H}^n`$ with smooth boundary. Let $`u`$ be a smooth psh solution of the Dirichlet problem for the quaternionic Monge-Ampère equation with $`f>0`$, and $`f,\varphi `$ be $`C^{\mathrm{}}`$-smooth. Then

$$|u|_{2+\alpha }K\text{ for some }0<\alpha <1,$$

where $`K`$ depends only on $`\mathrm{\Omega }`$ and norms of $`f`$ and $`\varphi `$.

This theorem is an immediate consequence of the following general result due to Caffarelli, Kohn, Nirenberg, and Spruck  and the second order estimates obtained in the previous subsection.

###### 6.3.2 Theorem.

Let $`\mathrm{\Omega }`$ be a bounded domain in $`\text{ }\text{ }\text{R}^N`$ with the smooth boundary $`\mathrm{\Omega }`$. Let $`u`$ be a smooth solution of the elliptic equation

$$F(x,u,Du,D^2u)=0\text{ in }\mathrm{\Omega },$$

$$u\varphi \text{ on }\mathrm{\Omega },$$

$`\varphi `$ is smooth. Assume that $`F`$ is concave in the second derivatives $`u_{ij}`$. Assume that $`u`$ satisfies an estimate

$$|u|_2C^{}.$$

Then

$$|u|_{2+\alpha }K\text{ for some }0<\alpha <1,$$

where $`K`$ depends only on $`\mathrm{\Omega },F,|\varphi |_4,C^{}`$.

Note that this theorem implies Theorem 6.3.1 if one takes $`F(x,u,Du,D^2u)=\mathrm{log}(detu_{\overline{i}j})\mathrm{log}f`$.

Thus Theorem 6.0.1 is proved as well.

## 7 Proof of Theorem 0.1.3.

In this section we will finish the proof of our main result about existence of solution of the Dirichlet problem for quaternionic Monge-Ampère equation (Theorem 0.1.3). Let the case $`n=1`$ is well understood in the literature, we will assume that $`n>1`$ throughout this section. But first we will need the following result.

###### 7.1.1 Theorem.

Suppose $`\mathrm{\Omega }=B`$ is the unit Euclidean ball in $`\text{ }\text{ }\text{H}^n`$, $`\varphi C(B)`$, $`fC(\overline{B}),f0`$. Let $`d\mu =f^{\frac{1}{n}}dvol`$. Then the upper envelopes of the families $`(\varphi ,\mu ),C(\varphi ,\mu ),(\varphi ,\mu ^n)`$ coincide. If $`u`$ denotes the upper envelope, then $`uC(\overline{B})`$ and satisfies

$$\mathrm{\Phi }(u)=f^{\frac{1}{n}}dvol\text{ in }B,$$

$$det(^2u)=fdvol\text{ in }B,$$

$$u=\varphi \text{ in }B.$$

Proof.Remind that the Perron-Bremermann families from the theorem were defined in Subsection 5.2. The argument follows very closely the proof of Theorem 8.2 of . Choose a sequence of functions $`f_j>0`$ with $`f_jC^{\mathrm{}}(\overline{B})`$ decreasing to $`f`$ uniformly on $`\overline{B}`$. Choose also a sequence of functions $`\varphi _jC^{\mathrm{}}(B)`$ such that $`\varphi _j`$ increases to $`\varphi `$ uniformly on $`B`$. By Theorem 6.0.1 there exist unique psh functions $`u_jC(\overline{B})`$ which are solutions of the Dirichlet problem

$$det(^2u_j)=f_j\text{ in }B,u_j=\varphi _j\text{ in }B.$$

By the minimum principle (Theorem 2.1.3) the sequence $`u_j`$ is increasing. We can choose positive numbers $`\eta _j`$ tending to zero so that $`\varphi _j+\eta _j\varphi \text{ on }B`$. Since

$$det(^2(u_k+\epsilon (|z|^21)))det(^2u_k)+\epsilon ^ndet(^2|z|^2)=f_k+\epsilon ^ndet(^2|z|^2),$$

and since $`f_kf`$ uniformly we can choose positive numbers $`\epsilon _j0`$ such that

$$det(^2(u_k+\epsilon _j(|z|^21)))det(^2u_j)\text{ for }kj.$$

By the minimum principle (Theorem 2.1.3) we get

$$u_k+\epsilon _j(|z|^21)u_j(z)+\eta _j\text{ for }kj,z\overline{B}.$$

But $`u_j(z)u_k(z)`$. Hence $`u_ju`$ uniformly on $`\overline{B}`$. By Theorem 2.1.2 $`u`$ is psh and $`det(^2u_j)det(^2u)`$ weakly. Hence $`det(^2u)=f`$. Further, by Theorem 4.1.7(4) $`\mathrm{\Phi }(u)f^{\frac{1}{n}}dvol`$, and by Theorem 4.1.8 and Remark 4.1.9 $`\mathrm{\Phi }(u)^ndet(^2u)`$. Hence $`\mathrm{\Phi }(u)=f^{\frac{1}{n}}dvol`$.

It follows from Proposition 5.2.1 that the upper envelopes of $`C(\varphi ,fdvol)`$ and $`(\varphi ,f^ndvol)`$ coincide with $`u`$. By Theorem 5.1.1 the upper envelopes of $`(\varphi ,fdvol)`$ and $`C(\varphi ,fdvol)`$ coincide. Q.E.D.

###### 7.1.2 Theorem.

Let $`\mathrm{\Omega }`$ be a bounded open set in $`\text{ }\text{ }\text{H}^n`$. Let $`\varphi C(\mathrm{\Omega })`$ and $`d\mu =f^{\frac{1}{n}}dvol`$ with $`f0`$ , $`fC(\mathrm{\Omega })`$. Suppose that

(i) $`(\varphi ,\mu )`$ is nonempty, and

(ii) the upper envelope $`u=sup\{v:v(\varphi ,\mu )\}`$ is continuous on $`\overline{\mathrm{\Omega }}`$ with $`u=\varphi `$ on $`\mathrm{\Omega }`$.
Then $`u`$ is psh and it is the solution to the Dirichlet problem

$$det(^2u)=f\text{ in }\mathrm{\Omega },u=\varphi \text{ on }\mathrm{\Omega }.$$

Also $`\mathrm{\Phi }(u)=f^{\frac{1}{n}}dvol`$.

Proof. It should be checked only that $`det(^2u)=fdvol`$ in $`\mathrm{\Omega }`$. First let us show that $`det(^2u)fdvol`$ in $`\mathrm{\Omega }`$. By Choquet’s lemma there exists an increasing sequence $`u_j(\varphi ,\mu )`$ which converges to $`u`$ almost everywhere, and hence in $`L_{loc}^1(\mathrm{\Omega })`$. Then by Theorem 4.1.7(4) $`\mathrm{\Phi }(u)f^{\frac{1}{n}}`$. Let us write the Lebesgue decomposition

$$det(^2u)=\stackrel{~}{f}dvol+d\nu .$$

By Theorem 4.1.8(2)

$$f=(f^{\frac{1}{n}})^n\stackrel{~}{f}.$$

Hence $`det(^2u)fdvol`$.

To prove the opposite inequality let us fix $`z_0\mathrm{\Omega }`$, and choose $`\epsilon >0`$ so small that the closure of the ball $`B(z_0,\epsilon )=\{|zz_0|<\epsilon \}`$ is contained in $`\mathrm{\Omega }`$. By the previous theorem there is a psh function $`v(z)C(\overline{B(z_0,\epsilon )})`$ such that

$$v(z)=u(z)\text{ on }B(z_0,\epsilon );$$

$$\mathrm{\Phi }(v)=f^{\frac{1}{n}}dvol\text{ on }B(z_0,\epsilon );$$

$$det(^2v)=fdvol\text{ on }B(z_0,\epsilon ).$$

Since $`f=det(^2v)det(^2u)`$ on $`B(z_0,\epsilon )`$, by the minimum principle we have $`vu`$ in $`\overline{B(z_0,\epsilon )}`$. Set $`U(z)=v(z)`$ if $`zB(z_0,\epsilon )`$, and $`U(z)=u(z)`$ if $`z\overline{\mathrm{\Omega }}B(z_0,\epsilon )`$. Then clearly $`U`$ is continuous and psh, and $`U=\varphi `$ on $`\mathrm{\Omega }`$. We also have $`\mathrm{\Phi }(U)f^{\frac{1}{n}}dvol`$. Therefore $`U(\varphi ,f^{\frac{1}{n}}dvol)`$. Hence $`Uu`$. Hence $`Uu`$. In particular in $`B(z_0,\epsilon )`$ we have $`det(^2u)=f`$ and $`\mathrm{\Phi }(u)=f^{\frac{1}{n}}dvol`$. Q.E.D.

Finally let us prove Theorem 0.1.3.

Proof. By Theorem 7.1.2 we have to verify that $`(\varphi ,\mu )`$ is not empty and its upper envelope $`uC(\overline{\mathrm{\Omega }})`$, and $`u=\varphi `$ on $`\mathrm{\Omega }`$. When $`\mathrm{\Omega }`$ is strictly pseudoconvex this is consequence of Theorem 5.1.1. Thus $`u=sup\{v:v(\varphi ,\mu )\}`$ is the solution of the Dirichlet problem. Q.E.D.

## 8 Quaternionic Levi form.

In this section we discuss some additional properties of quaternionic strictly pseudoconvex domains. In Subsection 8.1 we introduce a quaternionic version of the Levi form of a domain with smooth boundary and prove that such a domain is strictly pseudoconvex if and only if its Levi form is positive definite. In Subsection 8.2 we consider some examples and some other analogies with the real and complex cases. In Subsection 8.3 we state some open questions.

### 8.1 The quaternionic Levi form.

In this subsection we introduce the quaternionic version of the Levi form. For the classical complex case we refer to  and . The main result of this subsection is Proposition 8.1.2.

Let $`\mathrm{\Omega }`$ be a domain in $`\text{ }\text{ }\text{H}^n`$ with $`C^2`$-smooth boundary $`\mathrm{\Omega }`$. For any $`z\mathrm{\Omega }`$ let $`T_{\mathrm{\Omega },z}`$ denote the tangent space at $`z`$ to the boundary $`\mathrm{\Omega }`$. The quaternionic tangent space to $`\mathrm{\Omega }`$ at $`z`$ is by definition the maximal quaternionic subspace contained in $`T_{\mathrm{\Omega },z}`$:

$${}_{}{}^{h}T_{\mathrm{\Omega },z}^{}:=T_{\mathrm{\Omega },z}T_{\mathrm{\Omega },z}IT_{\mathrm{\Omega },z}JT_{\mathrm{\Omega },z}K.$$

Let $`\rho C^2`$ be a defining function of $`\mathrm{\Omega }`$, i.e.

$$\rho <0\text{ on }\mathrm{\Omega },\rho =0\text{ and }d\rho 0\text{ on }\mathrm{\Omega }.$$

The Levi form $`L_{\mathrm{\Omega },z}`$ on $`{}_{}{}^{h}T_{\mathrm{\Omega },z}^{}`$ is defined as the restriction of the hyperhermitian quadratic form $`(\frac{^2\rho (z)}{\overline{q}_iq_j})`$ to $`{}_{}{}^{h}T_{\mathrm{\Omega },z}^{}`$ divided by $`|\rho (z)|`$.

###### 8.1.1 Claim.

The Levi form does not depend on the choice of $`\rho `$.

Proof. Let $`\rho ^{}`$ be another defining function of $`\mathrm{\Omega }`$. Then in a small neighborhood of $`z`$ there exists a smooth function $`\alpha ,\alpha (z)>0`$, such that $`\rho ^{}=\alpha \rho `$. But

$$\frac{^2(\alpha \rho )}{\overline{q}_iq_j}=\alpha \frac{^2\rho }{\overline{q}_iq_j}+\frac{\alpha }{\overline{q}_i}\frac{\rho }{q_j}+\frac{\rho }{\overline{q}_i}\frac{\alpha }{q_j}+\rho \frac{^2\alpha }{\overline{q}_iq_j}.$$
$`()`$
Now let us choose the coordinate system such that $`z`$ is at the origin, and $`{}_{}{}^{h}T_{\mathrm{\Omega },z}^{}`$ is spanned by the first $`n1`$ coordinates $`q_1,\mathrm{},q_{n1}`$. If we evaluate this expression at $`z`$ we obtain for $`i,jn1`$:

$$\frac{^2\rho ^{}(z)}{\overline{q}_iq_j}=\alpha \frac{^2\rho (z)}{\overline{q}_iq_j}.$$

Q.E.D.

###### 8.1.2 Proposition.

A $`C^2`$-smooth domain $`\mathrm{\Omega }`$ is strictly pseudoconvex iff the Levi form if positive definite at each point $`z\mathrm{\Omega }`$.

Proof. If $`\mathrm{\Omega }`$ is strictly pseudoconvex then there is nothing to prove. Let us prove the opposite statement. Let us fix a point $`z\mathrm{\Omega }`$ and let us assume that $`L_{\mathrm{\Omega },z}`$ is positive definite. Let us fix any defining function $`\rho `$ of $`\mathrm{\Omega }`$ in a neighborhood of $`z`$. Let us also fix a coordinate system on $`\text{ }\text{ }\text{H}^n`$ so that again $`z`$ is at the origin and $`{}_{}{}^{h}T_{\mathrm{\Omega },z}^{}`$ is spanned by the first $`n1`$ coordinates $`q_1,\mathrm{},q_{n1}`$. From $`()`$ we obtain for any real valued smooth function $`\alpha `$:

$$\frac{^2(\alpha \rho )}{\overline{q}_iq_j}(z)=\alpha (z)\frac{^2\rho (z)}{\overline{q}_iq_j}+\frac{\alpha (z)}{\overline{q}_i}\frac{\rho (z)}{q_j}+\frac{\rho (z)}{\overline{q}_i}\frac{\alpha (z)}{q_j}.$$

Now let us choose $`\alpha `$ such that $`\alpha (q)=1+l(q_n)`$, where $`l`$ is  R-linear real valued functional depending only on $`q_n`$. Then if either $`i<n`$ or $`j<n`$ we get

$$\frac{^2(\alpha \rho )(z)}{\overline{q}_iq_j}=\frac{^2\rho (z)}{\overline{q}_iq_j}.$$

For $`i=j=n`$ we get

$$\frac{^2(\alpha \rho )(z)}{\overline{q}_nq_n}=\frac{^2\rho (z)}{\overline{q}_nq_n}+\frac{l(z)}{\overline{q}_n}\frac{\rho (z)}{q_n}+\frac{\rho (z)}{\overline{q}_n}\frac{l(z)}{q_n}=$$

$$\mathrm{\Delta }_n(\rho )+2Re(\frac{l(z)}{\overline{q}_n}\frac{\rho (z)}{q_n}).$$

If we choose $`l`$ appropriately we can make the last expression arbitrarily large, and then the matrix $`(\frac{^2(\alpha \rho )}{\overline{q}_iq_j})`$ will be positive definite at $`z`$, and hence in some neighborhood of $`z`$, and hence the function $`\alpha \rho `$ will be strictly psh. But $`\alpha \rho `$ is also a defining functional of $`\mathrm{\Omega }`$ near $`z`$. Q.E.D.

### 8.2 Some examples.

In this subsection we present a general construction of quaternionic strictly pseudoconvex domains. It was suggested by M. Gromov  in analogy to the complex case. Then we discuss some differences of the quaternionic situation with the real and complex cases. This part depends very much on discussions with M. Sodin.

###### 8.2.1 Definition.

Let $`S`$ be a real $`3n`$-dimensional linear subspace of $`\text{ }\text{ }\text{H}^n`$. Then $`S`$ is called totally real if

$$S(SI)(SJ)(SK)=\{0\}.$$

Note that a generic real $`3n`$-dimensional linear subspace is totally real. Note also that $`S`$ is totally real if and only if its orthogonal complement $`S^{}(\text{ }\text{ }\text{H}^n)^{}`$ satisfies

$$S^{}+IS^{}+JS^{}+KS^{}=(\text{ }\text{ }\text{H}^n)^{}.$$

###### 8.2.2 Definition.

A smooth $`3n`$-dimensional submanifold of $`\text{ }\text{ }\text{H}^n`$ is called totally real if the tangent space at every point of it is totally real.

###### 8.2.3 Claim.

Let $`M`$ be a $`3n`$-dimensional totally real compact submanifold of $`\text{ }\text{ }\text{H}^n`$. Let $`\mathrm{\Omega }:=M_\epsilon `$ be the $`\epsilon `$-neighborhood of $`M`$. Then for small $`\epsilon >0`$ the domain $`\mathrm{\Omega }`$ is strictly pseudoconvex.

Now let us remind the following characterizations of convex (resp. pseudoconvex) domains in $`\text{ }\text{ }\text{R}^n`$ (resp. $`\text{ }\text{ }\text{C}^n`$) (see e.g. ).

###### 8.2.4 Claim.

Let $`\mathrm{\Omega }`$ be a bounded domain in $`\text{ }\text{ }\text{R}^n`$ (resp. $`\text{ }\text{ }\text{C}^n`$). Then $`\mathrm{\Omega }`$ is convex (resp. pseudoconvex) if and only if the function $`x\mathrm{log}dist(x,\mathrm{\Omega })`$ is convex (resp. plurisubharmonic).

Unfortunately this criterion is not true in the quaternionic situation already in $`\text{ }\text{ }\text{H}^1`$. Indeed by Proposition 8.1.2 any bounded domain with smooth boundary is strictly pseudoconvex in the quaternionic sense. It is not difficult to construct a domain $`\mathrm{\Omega }\text{ }\text{ }\text{H}^1`$ such that the function $`x\mathrm{log}dist(x,\mathrm{\Omega })`$ will be not subharmonic (in the usual sense).

### 8.3 Questions and comments.

We would like to state few questions closely related to the material of this paper.

Question 1. Find a geometric (or any other) interpretation of the quaternionic Monge-Ampère equation (or of an appropriate modification of it).

Remind that the (modified) real Monge-Ampère equations appear in construction of convex hypersurfaces in $`\text{ }\text{ }\text{R}^n`$ with the prescribed conditions on curvature. For this material we refer to , . One of the main applications of (modified) complex Monge-Ampère equations is the construction of Kähler metrics on complex manifolds. After the proof of the Calabi-Yau theorem ,  and the Aubin-Yau theorem ,  they became the key tool in complex differential geometry, see e.g. ,,  for further discussion.

Question 2. (due to L. Polterovich.) Find a geometric characterization of quaternionic strictly pseudoconvex domains. (Note that we have not defined the notion of quaternionic pseudoconvex domain in the non-strict sense.)

Question 3. (due to G. Henkin.) This question is closely related to the previous one. Let $`\mathrm{\Omega }\text{ }\text{ }\text{H}^n`$ be a domain which admits an exhaustion by level sets of plurisubharmonic function; in other words there exists a plurisubharmonic function $`h:\mathrm{\Omega }\text{ }\text{ }\text{R}`$ such that for any number $`c`$ the set $`\{hc\}`$ is compact. (Note that in the classical complex situation this property is one of the equivalent definitions of the pseudoconvex domain.) It was observed by G. Henkin  that if $`h`$ is strictly plurisubharmonic Morse function then $`\mathrm{\Omega }`$ admits a homotopy retraction onto a compact subset of dimension at most $`\frac{3}{4}dim_{\text{ }\text{ }\text{R}}\mathrm{\Omega }=3n`$ (indeed the Morse index of every critical point of such a function is bounded from above by $`3n`$). This implies that the boundary $`\mathrm{\Omega }`$ is connected provided $`n>1`$. These properties are analogous to the corresponding properties of pseudoconvex domains in the complex spaces (where the constant $`\frac{3}{4}`$ is replaced by $`\frac{1}{2}`$). It would be of interest to understand the relation between the class of domains with this property and the class of strictly pseudoconvex domains in the sense of this paper.

Question 4. Generalize Theorem 6.0.1 on the existence of the regular solution (under suitable assumptions on regularity of the initial data) to arbitrary strictly pseudoconvex bounded domains with smooth boundary (and not only for the Euclidean ball).

Note that the real analogue of this result was proved by Caffarelli, Nirenberg, Spruck in , and the complex analogue was proved by Caffarelli, Kohn, Nirenberg, Spruck in  and Krylov in .