text:
# Macdonald functions associated to complex reflection groups

## 0. Introduction

Macdonald functions $`P_\lambda (x;q,t)`$, which were introduced by I.G. Macdonald \[M2\] in 1987, are two variables versions of Hall-Littlewood functions $`P_\lambda (x;t)`$. Those Hall-Littlewood functions and Macdonald functions are parametrized by partitions $`\lambda `$ of $`n`$. Since partitions of $`n`$ parameterize irreducible characters of the symmetric group $`𝔖_n`$, we may say that these functions are associated to symmetric groups. On the other hand, Hall-Littlewood functions are closely related to Green polynomials of a finite general linear group $`GL_n(𝔽_q)`$. In this direction, partitions $`\lambda `$ of $`n`$ occur as unipotent classes of $`GL_n(𝔽_q)`$. Unipotent classes of other finite classical groups such as $`Sp_{2n}(𝔽_q),SO_{2n+1}(𝔽_q)`$ have more complicated patterns. Lusztig introduced in \[L2\] (unipotent) symbols, as a generalization of the notion of partitions, to describe such unipotent classes in connection with Springer representations of Weyl groups. He also introduced in \[L1\] a notion of symbols to parameterize unipotent characters of finite classical groups.

In \[S1\], the author constructed Hall-Littlewood functions associated to complex reflection groups $`W𝔖_n(/e)^n`$. In \[S2\], \[S3\], some related topics are discussed. Our Hall-Littlewood functions are parametrized by $`e`$-tuples of partitions (which parameterize irreducible characters of $`W`$), or rather by various types of $`e`$-symbols. In the case where $`e=2`$, $`W`$ is the Weyl group of type $`B_n`$. In this case, Hall-Littlewood functions attached to unipotent symbols are closely related to Green functions of $`Sp_{2n}(𝔽_q)`$ or $`SO_{2n+1}(𝔽_q)`$. It is also expected that our Hall-Littlewood functions attached to symbols have some connection with unipotent characters.

This paper is an attempt to generalize Hall-Littlewood functions $`P_𝚲^\pm (x;t)`$ (where $`𝚲`$ is an $`e`$-symbol) to the two variables version $`P_𝚲^\pm (x;q,t)`$, just as in the case of $`P_\lambda (x;q,t)`$. We call such functions $`P_𝚲^\pm (x;q,t)`$ Macdonald functions associated to $`W`$. In the case of original Macdonald functions, they are characterized as simultaneous eigenfunctions of various Macdonald operators. We also construct Macdonald operators having (conjecturally) good properties. However, we note that our construction of Macdonald operators works only for a special type of symbols (in the case where $`e=2`$, this is exactly the symbols used to parameterize unipotent characters), though Macdonald functions can be constructed for any type of symbols. In the case of original Macdonald operators, the representation matrix with respect to the basis of Schur functions is a triangular matrix, with distinct eigenvalues. In our case, the matrix of Macdonald operators turns out to be a block triangular matrix, where the blocks correspond to families of symbols. One can conjecture that the matrices appearing in the diagonal blocks have no common eigenvalues. Assuming this conjecture, we show that the Macdonald operator characterizes Macdonald functions, not as eigenfunction, but as a unique solution of linear systems attached to the above diagonal blocks.

The properties of Macdonald functions discussed in this paper are just a part of those established for the original Macdonald functions. We hope to discuss more about them in a subsequent paper.

## 1. Symmetric functions with two parameters

1.1. An $`e`$-tuple of partitions $`𝜶=(\alpha ^{(0)},\mathrm{},\alpha ^{(e1)})`$ is called an $`e`$-partition. We define the size $`|𝜶|`$ of $`𝜶`$ by $`|𝜶|=_{k=0}^{e1}|\alpha ^{(k)}|`$, where $`|\alpha ^{(k)}|`$ is the size of the partition $`\alpha ^{(k)}`$. For a partition $`\alpha :\alpha _1\alpha _2\mathrm{}\alpha _k0`$, let $`l(\alpha )`$ be the number of non-zero parts $`\alpha _k`$. We denote by $`𝒫_n`$ the set of partitions of $`n`$, and $`𝒫_{n,e}`$ the set of $`e`$-partitions of size $`n`$. Let $`W`$ be the complex reflection group $`𝔖_n(/e)^n`$. Then the set of conjugacy classes in $`W`$ is in one to one correspondence with the set $`𝒫_{n,e}`$.

Let us fix a sequence of positive integers $`𝐦=(m_0,\mathrm{},m_{e1})`$, and consider indeterminates $`x_j^{(k)}(0ke1,1jm_k)`$. We denote by $`x=x_𝐦`$ the whole variables $`(x_j^{(k)})`$, and also denote by $`x^{(k)}`$ the variables $`x_1^{(k)},\mathrm{},x_{m_k}^{(k)}`$. Let $`\zeta `$ be a primitive $`e`$-th root of unity in $``$. For each integer $`r1`$ and $`i`$ such that $`0ie1`$, put

$$p_r^{(i)}(x)=\underset{j=0}{\overset{e1}{}}\zeta ^{ij}p_r(x^{(j)}),$$

where $`p_r(x^{(j)})`$ denotes the $`r`$-th power sum symmetric function with respect to the variables $`x^{(j)}`$. We put $`p_r^{(i)}(x)=1`$ for $`r=0`$. For an $`e`$-partition $`𝜶=(\alpha ^{(0)},\mathrm{},\alpha ^{(e1)})`$ with $`\alpha ^{(k)}:\alpha _1^{(k)}\mathrm{}\alpha _{m_k}^{(k)}`$, we define a function $`p_𝜶(x)`$ by

$$p_𝜶(x)=\underset{k=0}{\overset{e1}{}}\underset{j=1}{\overset{m_k}{}}p_{\alpha _j^{(k)}}^{(k)}(x).$$

Next, we define the Schur function $`s_𝜶(x)`$ and monomial symmetric functions $`m_𝜶(x)`$ associated to $`𝜶`$ by

$$s_𝜶(x)=\underset{k=0}{\overset{e1}{}}s_{\alpha ^{(k)}}(x^{(k)}),m_𝜶(x)=\underset{k=0}{\overset{e1}{}}m_{\alpha ^{(k)}}(x^{(k)}),$$

where $`s_{\alpha ^{(k)}}(x^{(k)})`$ (resp. $`m_{\alpha ^{(k)}}(x^{(k)}`$ ) denotes the usual Schur function (resp. monomial symmetric function) associated to the partition $`\alpha ^{(k)}`$ with respect to the variables $`x^{(k)}`$. These are the symmetric functions associated to complex reflection groups $`W𝔖_n(/e)^n`$, as given in \[M1, Appendix B\].

1.2. Put $`𝔖_𝐦=𝔖_{m_0}\times \mathrm{}\times 𝔖_{m_{e1}}`$. We denote by $`\mathrm{\Xi }_𝐦=_{k=0}^{e1}[x_1^{(k)},\mathrm{},x_{m_k}^{(k)}]^{𝔖_{m_k}}`$ the ring of symmetric polynomials (with respect to $`𝔖_𝐦`$) with variables $`x=(x_j^{(k)})`$. $`\mathrm{\Xi }_𝐦`$ has a structure of a graded ring $`\mathrm{\Xi }_𝐦=_{i0}\mathrm{\Xi }_𝐦^i`$, where $`\mathrm{\Xi }_𝐦^i`$ consists of homogeneous symmetric polynomials of degree $`i`$, together with the zero polynomial. We consider the inverse limit

$$\mathrm{\Xi }^i=\underset{\begin{array}{c}\\ 𝐦\end{array}}{lim}\mathrm{\Xi }_𝐦^i$$

with respect to homomorphisms $`\rho _{𝐦^{},𝐦}:\mathrm{\Xi }_𝐦^{}^i\mathrm{\Xi }_𝐦^i`$, where $`𝐦^{}=(m_0^{},\mathrm{},m_{e1}^{})`$ with $`m_k^{}=m_k+l`$ for some integer $`l0`$, and $`\rho _{𝐦^{},𝐦}`$ is induced from the homomorphism $`_k[x_1^{(k)},\mathrm{},x_{m_k^{}}^{(k)}]_k[x_1^{(k)},\mathrm{},x_{m_k}^{(k)}]`$ given by sending $`x_i^{(k)}`$ to 0 for $`i>m_k`$, and leaving the other $`x_j^{(k)}`$ invariant. $`\mathrm{\Xi }=_{i0}\mathrm{\Xi }^i`$ is called the space of symmetric functions. Schur functions $`s_𝜶(x)`$ with infinitely many variables $`x_1^{(k)},x_2^{(k)}\mathrm{}`$ are regarded as elements in $`\mathrm{\Xi }^n`$ with $`n=|𝜶|`$, and the set $`\{s_𝜶(x)𝜶𝒫_{n,e}\}`$ forms a $``$-basis of $`\mathrm{\Xi }^n`$. Similarly, $`\{m_𝜶(x)𝜶𝒫_{n,e}\}`$ gives a $``$-basis of $`\mathrm{\Xi }^n`$. Put $`\mathrm{\Xi }_{}=\mathrm{\Xi }`$. Then $`\{p_𝜶(x)𝜶𝒫_{n,e}\}`$ gives rise to a basis of $`\mathrm{\Xi }_{}`$.

1.3. Let $`q,t`$ be independent indeterminates and let $`F=(q,t)`$ be the field of rational functions in $`q,t`$. We consider the $`F`$-algebra of symmetric functions $`\mathrm{\Xi }_F=F_{}\mathrm{\Xi }`$ with coefficients in $`F`$. Let $`𝜶=(\alpha ^{(0)},\mathrm{},\alpha ^{(e1)})`$ be an $`e`$-partition. For each partition $`\alpha ^{(k)}:\alpha _1^{(k)}\alpha _2^{(k)}\mathrm{}`$, put

$$z_{\alpha ^{(k)}}(q,t)=\underset{j=1}{\overset{l(\alpha ^{(k)})}{}}\frac{1\zeta ^kq^{\alpha _j^{(k)}}}{1\zeta ^kt^{\alpha _j^{(k)}}}.$$

We then define $`z_𝜶(q,t)F`$ by

(1.3.1)
$$z_𝜶(q,t)=z_𝜶\underset{k=0}{\overset{e1}{}}z_{𝜶^{(k)}}(q,t),$$

where $`z_𝜶`$ is the order of the centralizer of $`w_𝜶`$ in $`W`$ (a representative of the conjugacy class of $`W`$ corresponding to $`𝜶𝒫_{n,e}`$). Explicitly, $`z_𝜶`$ is given as follows. For $`𝜶𝒫_{n,e}`$, put $`l(𝜶)=_{k=0}^{e1}l(\alpha ^{(k)})`$. For a partition $`\alpha =(1^{n_1},2^{n_2},\mathrm{})`$, put $`z_\alpha =_{i1}i^{n_i}n_i!`$. Then $`z_𝜶=e^{l(𝜶)}_{k=0}^{e1}z_{\alpha ^{(k)}}`$.

We define a sesquilinear form on $`\mathrm{\Xi }_F`$ by

(1.3.2)
$$\mathrm{}p_𝜶,p_𝜷\mathrm{}=\delta _{𝜶,𝜷}z_𝜶(q,t)$$

for $`𝜶,𝜷𝒫_{n,e}`$. Let $`x=(x_j^{(k)})`$, $`y=(y_j^{(k)})`$ ($`0ke1)`$ be two sequences of infinitely many variables, and we define an infinite product of $`x`$ and $`y`$ by

(1.3.3) $`\mathrm{\Pi }(x,y;q,t)`$ $`={\displaystyle \underset{k=0}{\overset{e1}{}}}{\displaystyle \underset{i,j=1}{\overset{\mathrm{}}{}}}{\displaystyle \underset{r=0}{\overset{\mathrm{}}{}}}{\displaystyle \frac{1tx_i^{(kr1)}y_j^{(k)}q^r}{1x_i^{(kr)}y_j^{(k)}q^r}}`$
$`={\displaystyle \underset{k=0}{\overset{e1}{}}}{\displaystyle \underset{i,j=1}{\overset{\mathrm{}}{}}}{\displaystyle \underset{r=0}{\overset{\mathrm{}}{}}}{\displaystyle \frac{1tx_i^{(k)}y_j^{(k+r+1)}q^r}{1x_i^{(k)}y_j^{(k+r)}q^r}}.`$
(Here the upper indices of the variables in the formula should be read $`mode`$). Note that in the case where $`q=0`$, the product $`\mathrm{\Pi }(x,y;q,t)`$ reduces to the product $`\mathrm{\Omega }(x,y;t)`$ introduced in \[S1, 2.5\], (or rather \[S2, (5.7.1)\], see the remark there).

###### Lemma 1.4.

$`\mathrm{\Pi }(x,y;q,t)`$ has the following expansion.

$$\mathrm{\Pi }(x,y;q,t)=\underset{𝜶}{}z_𝜶(q,t)^1p_𝜶(x)\overline{p_𝜶(y)}$$

where $`𝛂`$ runs over all the $`e`$-partitions of any size.

###### Proof.

Taking the $`\mathrm{log}`$ on both sides of the first formula of (1.3.3),

$$\begin{array}{cc}\hfill \mathrm{log}\mathrm{\Pi }(& x,y;q,t)\hfill \\ & =\underset{k}{}\underset{i,j}{}\underset{r=0}{\overset{\mathrm{}}{}}\underset{m=1}{\overset{\mathrm{}}{}}\left\{\frac{1}{m}(x_i^{(kr)}y_j^{(k)}q^r)^m\frac{t^m}{m}(x_i^{(kr1)}y_j^{(k)}q^r)^m\right\}.\hfill \end{array}$$

By making use of the equation

$$\frac{1}{e}\underset{a=0}{\overset{e1}{}}(\zeta ^k\zeta ^k^{})^a=\delta _{k,k^{}},$$

the above formula can be written as

$$\begin{array}{cc}\hfill \mathrm{log}\mathrm{\Pi }(x,y;q,t)=\underset{a=0}{\overset{e1}{}}& \underset{m1}{}\underset{r0}{}\underset{k,k^{},i,j}{}\{\frac{1}{em}\zeta ^{ar}\zeta ^{a(kr)}\zeta ^{ak^{}}(x_i^{(kr)}y_j^{(k^{})}q^r)^m\hfill \\ & \frac{t^m}{em}\zeta ^{a(r+1)}\zeta ^{a(kr1)}\zeta ^{ak^{}}(x_i^{(kr1)}y_j^{(k^{})}q^r)^m\}.\hfill \end{array}$$

It follows that

$`\mathrm{log}\mathrm{\Pi }(x,y;q,t)`$ $`={\displaystyle \underset{a=0}{\overset{e1}{}}}{\displaystyle \underset{m1}{}}{\displaystyle \underset{r0}{}}{\displaystyle \underset{k,k^{},i,j}{}}{\displaystyle \frac{1\zeta ^at^m}{em}}\zeta ^{ar}q^{rm}(x_i^{(k)}y_j^{(k^{})})^m\zeta ^{ak}\zeta ^{ak^{}}`$
$`={\displaystyle \underset{a=0}{\overset{e1}{}}}{\displaystyle \underset{m1}{}}{\displaystyle \underset{k,k^{},i,j}{}}{\displaystyle \frac{1}{em}}{\displaystyle \frac{1\zeta ^at^m}{1\zeta ^aq^m}}\zeta ^{ak}\zeta ^{ak^{}}(x_i^{(k)}y_j^{(k^{})})^m`$
$`={\displaystyle \underset{a=0}{\overset{e1}{}}}{\displaystyle \underset{m=1}{\overset{\mathrm{}}{}}}{\displaystyle \frac{1}{em}}{\displaystyle \frac{1\zeta ^at^m}{1\zeta ^aq^m}}p_m^{(a)}(x)\overline{p_m^{(a)}(y)}.`$
Hence we have

$`\mathrm{\Pi }(x,y;q,t)`$ $`={\displaystyle \underset{a=0}{\overset{e1}{}}}{\displaystyle \underset{m=1}{\overset{\mathrm{}}{}}}\mathrm{exp}\left\{{\displaystyle \frac{1}{em}}{\displaystyle \frac{1\zeta ^at^m}{1\zeta ^aq^m}}p_m^{(a)}(x)\overline{p_m^{(a)}(y)}\right\}`$
$`={\displaystyle \underset{𝜶}{}}z_𝜶(q,t)^1p_𝜶(x)\overline{p_𝜶(y)}.`$
1.5. For a fixed $`y_j^{(k)}`$, we define a function $`g_{m,\pm }^{(k)}(x;q,t)`$ as the coefficient of $`(y_j^{(k)})^m`$ in

(1.5.1)
$$\underset{i}{}\underset{r0}{}\frac{1tx_i^{(kr1)}y_j^{(k)}q^r}{1x_i^{(kr)}y_j^{(k)}q^r}=\underset{m0}{}g_{m,\pm }^{(k)}(x;q,t)(y_j^{(k)})^m$$

and put, for each $`𝜶=(\alpha _j^{(k)})𝒫_{n,e}`$,

$$g_{𝜶,\pm }(x;q,t)=\underset{j,k}{}g_{\alpha _j^{(k)},\pm }^{(k)}.$$

Then we have

(1.5.2)
$$\mathrm{\Pi }(x,y;q,t)=\underset{𝜶}{}g_{𝜶,+}(x;q,t)m_𝜶(y)=\underset{𝜶}{}m_𝜶(x)g_{𝜶,}(y;q,t).$$

In fact, by comparing the first formula of (1.3.3) and (1.5.1), we have

$`\mathrm{\Pi }(x,y;q,t)`$ $`={\displaystyle \underset{k,j}{}}{\displaystyle \underset{\alpha _j^{(k)}0}{}}g_{\alpha _j^{(k)},+}^{(k)}(x;q,t)(y_j^{(k)})^{\alpha _j^{(k)}}`$
$`={\displaystyle \underset{𝜶}{}}g_{𝜶,+}(x;q,t)m_𝜶(y).`$
This shows the first equality. If we compare the second formula of (1.3.3) and (1.5.1) by replacing $`x`$ and $`y`$, the second equality is obtained in a similar way.

Now by using a similar argument as in \[M1, VI, 2.7\], we see that

(1.5.3)
$$\mathrm{}g_{𝜶,+}(x;q,t),m_𝜷(x)\mathrm{}=\mathrm{}m_𝜶(x),g_{𝜷,}(x;q,t)\mathrm{}=\delta _{𝜶,𝜷}.$$

In particular, the functions $`g_{𝜶,\pm }(x;q,t)`$ form a basis of $`\mathrm{\Xi }_F`$ dual to $`m_𝜶`$. Hence $`\{g_n(x;q,t)n0\}`$ are algebraically independent over $`F`$, and $`\mathrm{\Xi }_F=F[g_1,g_2,\mathrm{}]`$.

The following lemma can be proved in a similar way as \[M1, VI, 2.13\] by using (1.5.2) and (1.5.3).

###### Lemma 1.6.

Let $`E^\pm :\mathrm{\Xi }_F\mathrm{\Xi }_F`$ be two $`F`$-linear operators. Then the following conditions are equivalent.

1. $`\mathrm{}E^+f,g\mathrm{}=\mathrm{}f,E^{}g\mathrm{}`$ for any $`f,g\mathrm{\Xi }_F`$
2. $`E_x^+\mathrm{\Pi }(x,y;q,t)=E_y^{}\mathrm{\Pi }(x,y;q,t)`$, where the suffix $`x`$ indicates the action of $`E^\pm `$ on the $`x`$ variables, and similarly for $`y`$.

1.7. We shall give an explicit form of the function $`g_{𝜶,\pm }(x;q,t)`$. For this, we prepare some notation. For $`\mu =(\mu _0,\mathrm{},\mu _{e1})_0^e`$, put

$$f_{\mu ,\pm }^{(k,i)}(x;q,t)=\underset{a=0}{\overset{e1}{}}\underset{j=1}{\overset{\mu _a}{}}\frac{x_i^{(ka)}tx_i^{(ka1)}q^{e(j1)}}{1q^{ej}}.$$

Let $`_m`$ be the set of sequences $`𝝁=(\mu ^{(1)},\mu ^{(2)},\mathrm{})`$ such that $`\mu ^{(i)}_0^e`$ and that $`_i|\mu ^{(i)}|=m`$. For $`\mu =(\mu _1,\mu _2,\mathrm{},\mu _e)_0^e`$, put $`n(\mu )=_j(j1)\mu _j`$. Then

###### Proposition 1.8.

For each $`m0`$, we have

(1.8.1)
$$g_{m,\pm }^{(k)}(x;q,t)=\underset{𝝁_m}{}\underset{i}{}f_{\mu ^{(i)},\pm }^{(k,i)}(x;q,t)q^{n(\mu ^{(i)})}.$$

###### Proof.

By \[M1, I, §2, Ex. 5\], the following identity of formal power series is known.

$$\underset{i=0}{\overset{\mathrm{}}{}}\frac{1bq^it}{1aq^it}=\underset{m0}{}\left(\underset{i=1}{\overset{m}{}}\frac{abq^{i1}}{1q^i}\right)t^m.$$

Substituting this into

$`A={\displaystyle \underset{r0}{}}{\displaystyle \frac{1tx_i^{(kr1)}yq^r}{1x_i^{(kr)}yq^r}}={\displaystyle \underset{a=0}{\overset{e1}{}}}{\displaystyle \underset{r0}{}}{\displaystyle \frac{1tx_i^{(ka1)}yq^{re+a}}{1x_i^{(ka)}yq^{re+a}}}.`$
with $`a=x_i^{(ka)},b=tx_i^{(ka1)}`$, $`t=q^ay`$, $`q=q^e`$, we see that

$`A`$ $`={\displaystyle \underset{a=0}{\overset{e1}{}}}{\displaystyle \underset{\mu _a=0}{\overset{\mathrm{}}{}}}{\displaystyle \underset{j=1}{\overset{\mu _a}{}}}{\displaystyle \frac{x_i^{(ka)}tx_i^{(ka1)}q^{e(j1)}}{1q^{ej}}}(q^ay)^{\mu _a}`$
$`={\displaystyle \underset{\mu _0^e}{}}f_{\mu ,\pm }^{(k,i)}(x;q,t)q^{n(\mu )}y^{|\mu |}.`$
It follows that

$$\underset{i1}{}\underset{r0}{}\frac{1tx_i^{(kr1)}y_j^{(k)}q^r}{1x_i^{(kr)}y_j^{(k)}q^r}=\underset{m0}{}\underset{𝝁_m}{}\underset{i1}{}f_{\mu ^{(i)},\pm }^{(k,i)}(x;q,t)q^{n(\mu ^{(i)})}(y_j^{(k)})^m.$$

By comparing this with (1.5.1), we obtain the required formula. ∎

Remark 1.9. $`g_{m,\pm }^{(k)}(x;0,t)`$ coincides with the function $`q_{m,\pm }^{(k)}(x;t)`$ introduced in \[S1, 2.2\]. By using a similar, but much simpler arguments as above, one obtains an alternative expression of $`q_{m,\pm }^{(k)}(x;t)`$ as follows.

$$q_{m,\pm }^{(k)}(x;t)=\underset{\mu 𝒫_m}{}|𝔖_\mu |^1\underset{w𝔖_\mu }{}w\left\{\underset{i=1}{\overset{l(\mu )}{}}(x_i^{(k)}tx_i^{(k1)})(x_i^{(k)})^{\mu _i1}\right\},$$

where $`𝔖_\mu `$ is the stabilizer of $`\mu `$ in $`𝔖_m`$. (Here we are considering finite variables $`x_i^{(k)},x_i^{(k1)}(1im)`$, and $`𝔖_m`$ acts on both variables).

1.10. The notion of symbols was introduced in \[S1\]. (Although a more general setting was discussed in \[S2\], we do not use it in the discussion below. We remark that similar symbols were also considered by G. Malle in \[Ma\]). Let $`𝐦=(m_0,\mathrm{},m_{e1})`$ be as before. We denote by $`Z_n^{0,0}=Z_n^{0,0}(𝐦)`$ the set of $`e`$-partitions $`𝜶=(\alpha ^{(0)},\mathrm{},\alpha ^{e1)})𝒫_{n,e}`$ such that each $`\alpha ^{(k)}`$ is written ( as an element in $`^{m_k}`$) in the form $`\alpha ^{(k)}:\alpha _1^{(k)}\mathrm{}\alpha _{m_k}^{(k)}0`$. We express $`𝜶`$ as $`𝜶=(\alpha _j^{(k)})`$ in matrix form. Let us fix integers $`rs0`$ and define an $`e`$-partition $`𝚲^0=𝚲^0(𝐦,s,r)=(\mathrm{\Lambda }^{(0)},\mathrm{},\mathrm{\Lambda }^{(e1)})`$ as follows.

(1.10.1) $`\mathrm{\Lambda }^{(0)}`$ $`:(m_01)r\mathrm{}2rr0,`$
$`\mathrm{\Lambda }^{(i)}`$ $`:s+(m_i1)r\mathrm{}s+2rs+rs`$
for $`i=0,\mathrm{},e1`$. We denote by $`Z_n^{r,s}=Z_n^{r,s}(𝐦)`$ the set of $`e`$-partitions of the form $`𝚲=𝜶+𝚲^0`$, where $`𝜶Z_n^{0,0}`$ and the sum is taken entry-wise. We write $`𝚲=𝚲(𝜶)`$ if $`𝚲=𝜶+𝚲^0`$, and call it the $`e`$-symbol of type $`(r,s)`$ corresponding to $`𝜶`$. We often denote the symbol $`𝚲=(\mathrm{\Lambda }^{(0)},\mathrm{},\mathrm{\Lambda }^{(e1)})`$ in the form $`𝚲=(\mathrm{\Lambda }_j^{(k)})`$ with $`\mathrm{\Lambda }^{(k)}:\mathrm{\Lambda }_1^{(k)}>\mathrm{}>\mathrm{\Lambda }_{m_k}^{(k)}`$ for $`k=0,\mathrm{},e1`$.

Put $`𝐦^{}=(m_0+1,\mathrm{},m_{e1}+1)`$, and define a shift operation $`Z_n^{r,s}(𝐦)Z_n^{r,s}(𝐦^{})`$ by associating $`𝚲^{}=(\mathrm{\Lambda }_0^{},\mathrm{},\mathrm{\Lambda }_{e1}^{})Z_n^{r,s}(𝐦)`$ to $`𝚲=(\mathrm{\Lambda }_0,\mathrm{},\mathrm{\Lambda }_{e1})Z_n^{r,s}(𝐦)`$, where $`\mathrm{\Lambda }_0^{}=(\mathrm{\Lambda }_0+r)\{0\}`$, and $`\mathrm{\Lambda }_k^{}=(\mathrm{\Lambda }_k+r)\{s\}`$ for $`k=0,\mathrm{},e1`$. In other words, for $`𝚲=𝚲(𝜶)`$, $`𝚲^{}`$ is obtained as $`𝚲^{}=𝜶+𝚲^0(𝐦^{},s,r)`$, where $`𝜶`$ is regarded as an element of $`Z_n^{0,0}(𝐦^{})`$ by adding 0 in the entries of $`𝜶`$. We denote by $`\overline{Z}_n^{r,𝐬}`$ the set of classes in $`_𝐦^{}Z_n^{r,𝐬}(𝐦^{})`$ under the equivalence relation generated by shift operations. Note that $`𝒫_{n,e}`$ coincides with the set $`\overline{Z}_n^{0,0}`$. Also note that $`𝚲^0`$ is regarded as a symbol in $`Z_n^{r,s}`$ with $`n=0`$.

Two elements $`𝚲`$ and $`𝚲^{}`$ in $`\overline{Z}_n^{r,𝐬}`$ are said to be similar, and are written as $`𝚲𝚲^{}`$, if there exist representatives in $`Z_n^{r,𝐬}(𝐦)`$ such that all the entries of them coincide with multiplicities. The set of symbols which are similar to a fixed symbol is called a family in $`Z_n^{r,𝐬}`$.

We define a function $`a:Z_n^{r,𝐬}_0`$, for $`𝚲Z_n^{r,𝐬}`$, by

(1.10.2)
$$a(𝚲)=\underset{\lambda ,\lambda ^{}𝚲}{}\mathrm{min}\{\lambda ,\lambda ^{}\}\underset{\mu ,\mu ^{}𝚲^0}{}\mathrm{min}\{\mu ,\mu ^{}\}.$$

The function $`a`$ on $`Z_n^{r,𝐬}`$ is invariant under the shift operation, and it induces a function $`a`$ on $`\overline{Z}_n^{r,𝐬}`$. Clearly, the $`a`$-function takes a constant value on each family in $`Z_n^{r,𝐬}`$. We regard the $`a`$-function as a function on $`Z_n^{0,0}`$ by using the bijection $`Z_n^{0,0}Z_n^{r,s}`$.

1.11. Hall-Littlewood functions $`P_𝚲^\pm (x;t)`$ and $`Q_𝚲^\pm (x;t)`$ attached to symbols $`𝚲`$ were introduced in \[S1\]. We shall now construct a two parameter version of Hall-Littlewood functions. Let us introduce a total order $`𝜶𝜷`$ on $`Z_n^{0,0}`$ such that $`a(𝜶)a(𝜷)`$ whenever $`𝜶𝜷`$ and that each family in $`Z_n^{0,0}`$ forms an interval.

The following proposition is easily obtained by a similar argument as in Remark 4.9 in \[S1\] (i.e., a generalization of Gram-Schmidt orthogonalization process) if one notices that $`\mathrm{\Pi }(x,y;0,t)`$ coincides with $`\mathrm{\Omega }(x,y;t)`$ in \[S1, 2.5\].

###### Proposition 1.12.

There exists a unique function $`P_𝚲^\pm (x;q,t)\mathrm{\Xi }_F`$ for $`𝚲Z_n^{r,s}`$ satisfying the following two properties.

1. $`P_𝚲^\pm (x;q,t)`$ for $`𝚲=𝚲(𝜶)`$ can be expressed in terms of $`s_𝜷(x)`$ as

$$P_𝚲^\pm =s_𝜶+\underset{𝜷}{}u_{𝜶,𝜷}^\pm s_𝜷,$$

with $`u_{𝜶,𝜷}^\pm F`$, where $`u_{𝜶,𝜷}^\pm =0`$ unless $`𝜷𝜶`$ and $`𝜷\sim ̸𝜶`$.
2. $`\mathrm{}P_𝚲^+,P_𝚲^{}^{}\mathrm{}=0`$ unless $`𝚲𝚲^{}`$.

We then define $`Q_𝚲^\pm (x;q,t)`$ as the dual of $`P_𝚲^{}(x;q,t)`$, i.e., by the property that

$$\mathrm{}P_𝚲^+,Q_𝚲^{}^{}\mathrm{}=\mathrm{}Q_𝚲^+,P_𝚲^{}^{}\mathrm{}=\delta _{𝚲,𝚲^{}}.$$

$`P_𝚲^\pm (x;q,t)`$, $`Q_𝚲^\pm (x;q,t)`$ are called the Macdonald functions associated to complex reflection groups $`W`$ (with respect to symbols in $`Z_n^{r,s}`$).

Remark 1.13. (i) The orthogonality relations of Macdonald functions given above imply, by \[M1, VI, 2.7\], that

(1.13.1) $`\mathrm{\Pi }(x,y;q,t)`$ $`={\displaystyle \underset{𝚲}{}}P_𝚲^+(x;q,t)Q_𝚲^{}(y;q,t)`$
$`={\displaystyle \underset{𝚲}{}}Q_𝚲^+(x;q,t)P_𝚲^{}(y;q,t).`$
(ii) In the case where $`q=0`$, the scalar product given in (1.3.2) coincides with the one given in \[S1, 4.7\]. Then by Proposition 4.8 in \[S1\], one sees that

(1.13.2)
$$P_𝚲^\pm (x;0,t)=P_𝚲^\pm (x,t),Q_𝚲^\pm (x;0,t)=Q_𝚲^\pm (x;t),$$

where the right hand sides are the Hall-Littlewood functions defined in \[S1\].

(iii) In the case where $`q=t`$, the scalar product in (1.3.2) coincides with the usual scalar product on the space $`\mathrm{\Xi }_{}`$, where the Schur functions form an orthonormal basis of it. Hence Proposition 1.12 implies that

(1.13.3)
$$P_{𝚲(𝜶)}^\pm (x;t,t)=Q_{𝚲(𝜶)}^\pm (x;t,t)=s_𝜶(x).$$

## 2. Macdonald operators

2.1. The original Macdonald functions related to symmetric groups are characterized as the simultaneous eigenfunctions of Macdonald operators (see \[M1, VI\]). In this section, we shall construct certain operators which can be viewed as a generalization of Macdonald operators to the case of $`W`$. Here we restrict ourselves to the case where symbols are of the type $`(r,s)`$ with $`r=1`$ and $`s=0`$. (We note that the arguments in this section can not be applied to other types of symbols. See Remark 2.9.) In particular, $`𝚲^0=𝜹`$, where $`𝜹=(\delta ^{(0)},\mathrm{},\delta ^{(e1)})`$ with $`\delta ^{(k)}=(m_k1,\mathrm{},1,0)`$. Hence, in the case where $`e=2`$ (i.e., $`W`$ is the Weyl group of type $`B_n`$) with $`m_1=m_0+1`$, these symbols are exactly the ones used to parameterize unipotent characters of finite classical groups $`Sp_{2n}(𝔽_q)`$ or $`SO_{2n+1}(𝔽_q)`$ by Lusztig \[L1\]. Each family $``$ in $`Z_n^{1,0}`$ contains a unique element $`𝚲_{}=(\mathrm{\Lambda }_j^{(k)})`$ with the property

(2.1.1)
$$\mathrm{\Lambda }_1^{(0)}\mathrm{\Lambda }_1^{(1)}\mathrm{}\mathrm{\Lambda }_1^{(e1)}\mathrm{\Lambda }_2^{(0)}\mathrm{\Lambda }_2^{(1)}\mathrm{}.$$

Such an element is called a special symbol associated to the family $``$. The set of families is in bijection with the set of special symbols. Special symbols are regarded as partitions of $`N=\mathrm{\Lambda }_j^{(k)}`$ by (2.1.1).

We shall define a partial order on the set of families in $`Z_n^{1,0}`$. Let $``$ and $`^{}`$ be families in $`Z_n^{1,0}`$ and $`𝚲_{},𝚲_{^{}}`$ be special symbols corresponding to them. We put $`<^{}`$ if $`𝚲_{}<𝚲_{^{}}`$ with respect to the dominance order on $`𝒫_N`$. Recall that for $`\lambda =(\lambda _i),\mu =(\mu _i)𝒫_N`$, the dominance order $`\lambda <\mu `$ is defined by the condition that

$$\underset{i=1}{\overset{k}{}}\lambda _i\underset{i=1}{\overset{k}{}}\mu _i,(1k\underset{j}{}m_j).$$

We define a partial order on $`Z_n^{1,0}`$ by inheriting the partial order on the set of families. For $`\lambda =(\lambda _i)𝒫_N`$, put

$$n(\lambda )=\underset{i1}{}(i1)\lambda _i.$$

Then, for each $`𝚲`$ in a family $``$, the value $`a(𝚲)`$ is given as $`\alpha (𝚲)=n(𝚲_{})n(𝚲^0)`$, where $`𝚲_{}`$ is regarded as an element in $`𝒫_M`$. In particular, we have $`a(𝚲)>a(𝚲^{})`$ if $`𝚲<𝚲^{}`$. As before, by using the bijection $`Z_n^{0,0}Z_n^{1,0}`$, we consider the partial order on $`Z_n^{0,0}`$, which will be denoted by the same symbol.

2.2. In order to construct Macdonald operators, we shall start with finitely many variables $`x=x_𝐦`$. We consider the expansion of $`\mathrm{\Pi }(x,y;q,t)`$ in the case of finitely many variables. Assume that $`x=(x_j^{(k)})=x_𝐦`$ is as in 1.2 and put $`\mathrm{\Xi }_{𝐦,F}=F_{}\mathrm{\Xi }_𝐦`$. Let us denote by $`𝒫_𝐦`$ the set of $`e`$-partitions $`𝜶=(\alpha _j^{(k)})`$ such that $`l(\alpha ^{(k)})m_k`$. Then $`m_𝜶(x)=0`$ unless $`𝜶𝒫_𝐦`$, and those non-zero $`m_𝜶(x)`$ form a basis of $`\mathrm{\Xi }_{𝐦,F}`$. The same is true for Schur functions. Also by a similar argument as in \[M1\] we see that $`g_𝜶(x;q,t)`$ such that $`𝜶𝒫_𝐦`$ form an $`F`$-basis of $`\mathrm{\Xi }_{𝐦,F}`$.

Now by substituting $`x_j^{(k)}=y_j^{(k)}=0`$ for $`j>m_k`$, we have a finite version of (1.5.2), i.e., for $`x=x_𝐦,y=y_𝐦`$, we have

(2.2.1)
$$\mathrm{\Pi }(x,y;q,t)=\underset{𝜶𝒫_𝐦}{}g_{𝜶,+}(x;q,t)m_𝜶(y)=\underset{𝜶𝒫_𝐦}{}m_𝜶(x)g_{𝜶,}(y;q,t).$$

This enables us to define a scalar product on $`\mathrm{\Xi }_{𝐦,F}`$ by

(2.2.2)
$$\mathrm{}g_{𝜶,+},m_𝜷\mathrm{}=\mathrm{}m_𝜶,g_{𝜷,}\mathrm{}=\delta _{𝜶,𝜷}.$$

We now consider the restriction of the functions $`P_𝚲^\pm `$ to $`\mathrm{\Xi }_{𝐦,F}`$. By Proposition 1.12, one sees that $`\{P_{𝚲(𝜶)}^\pm 𝜶𝒫_𝐦\}`$ form a basis of $`\mathrm{\Xi }_{𝐦,F}`$. Moreover, the finite variables version of Proposition 1.12 holds, and $`P_𝚲^\pm \mathrm{\Xi }_F`$ is obtained as the limit of $`P_𝚲^\pm \mathrm{\Xi }_{𝐦,F}`$ determined by these properties.

2.3. Let

$$=(𝐦)=\{𝐢=\left(\begin{array}{c}i_0\\ \mathrm{}\\ i_{e1}\end{array}\right)^e1i_km_k\}.$$

For each $`𝐢`$ and $`uF`$, we define an $`F`$-linear operator $`T_{u,𝐢}^\pm :F[x]F[x]`$ by $`T_{u,𝐢}^\pm f=f^{}`$, where $`f^{}`$ is a polynomial obtained from $`f`$ by replacing the variables $`x_{i_k}^{(k)}`$ by $`ux_{i_{k1}}^{(k1)}`$ for $`k=0,\mathrm{},e1`$. (Here we understand that $`i_e=i_o`$). More generally, for each $`r`$ such that $`1rM_1=\mathrm{min}_k\{m_k\}`$, we define $`_r`$ as the set of $`J=\{𝐢_1,\mathrm{},𝐢_r\}`$ consisting of $`𝐢_k`$ such that any two $`𝐢_k`$ have no common entries (i.e., $`𝐢_j𝐢_k`$ does not contain 0 entries for each $`jk`$ as vectors in $`^e`$). Then we define, for each $`J_r`$, an operator $`T_{u,J}^\pm :F[x]F[x]`$ by $`T_{u,J}^\pm =_{k=1}^rT_{u,𝐢_k}^\pm `$. Note that $`T_{u,𝐢_k}^\pm `$ in the product commute with each other, and so $`T_{u,J}^\pm `$ does not depend on the order of the product.

Let $`Z=Z(𝐦)`$ be the set of sequences $`𝜷=(\beta _j^{(k)})`$ ($`0ke1,1jm_k`$) with $`\beta _i^{(k)}_0`$. For each $`𝜷Z`$, we denote by $`[𝜷]`$ the element in $`Z`$ obtained from $`𝜷`$ by permuting the entries inside each row, so that each row is arranged in decreasing order. We often regard $`𝜷`$ as a matrix, and denote its $`k`$-th row by $`\beta ^{(k)}`$. Then the set $`J`$ is regarded as a subset of the set of indices $`\{(k,j)0ke1,1jm_k\}`$ of $`(\beta _j^{(k)})`$

We put, for each $`𝜷Z`$ and $`J_r`$,

$$\mathrm{}𝜷,J\mathrm{}=\underset{(k,j)J}{}\beta _j^{(k)}.$$

$`𝔖_𝐦`$ acts naturally on $`Z`$ and on $`_r`$, respectively, and this pairing is $`𝔖_𝐦`$-invariant. We also note that the action of $`𝔖_𝐦`$ on $`_r`$ is transitive.

The operation of $`T_{q,J}^\pm `$ on $`x=(x_i^{(k)})`$ also induces an action $`𝜷𝜷_{J\pm }`$ on $`Z`$ by permuting the entries of $`𝜷`$ so that $`T_{q,J}^\pm x^𝜷=q^{\mathrm{}𝜷,J\mathrm{}}x^{𝜷_{J\pm }}`$, where, as usual, $`x^𝜷`$ denotes the monomial $`_{i,k}(x_i^{(k)})^{\beta _i^{(k)}}`$. For each $`𝜷Z`$, we define a function $`a_𝜷(x)`$ by

$$a_𝜷(x)=\underset{w𝔖_𝐦}{}\epsilon (w)w(x^𝜷)$$

We now define, for each $`1rM_1`$, an $`F`$-linear operator $`D_\pm ^r(q,t)`$ on $`F[x]`$ by

(2.3.1)
$$D_\pm ^r(q,t)=a_𝜹(x)^1\underset{w𝔖_𝐦}{}\epsilon (w)\underset{J_r}{}x^{w(𝜹)_{J\pm }}t^{\mathrm{}w(𝜹),J\mathrm{}}T_{q,J}^\pm .$$

Then $`D_\pm ^r(q,t)`$ can be written as

(2.3.2)
$$D_\pm ^r(q,t)=\underset{J_r}{}A_J^\pm (x;t)T_{q,J}^\pm ,$$

with

$`A_J^\pm (x;t)`$ $`=a_𝜹(x)^1{\displaystyle \underset{w𝔖_𝐦}{}}\epsilon (w)x^{w(𝜹)_{J\pm }}t^{\mathrm{}w(𝜹),J\mathrm{}}`$
$`=a_𝜹(x)^1T_{t,J}^\pm a_𝜹(x)`$
(2.3.3) $`=t^{r(r1)/2}{\displaystyle \underset{k=0}{\overset{e1}{}}}{\displaystyle \underset{\begin{array}{c}(k,i)J\\ (k,j)J\end{array}}{}}{\displaystyle \frac{x_i^{(k)}tx_j^{}^{(k1)}}{x_i^{(k)}x_j^{(k)}}},`$
where we write $`J=\{𝐢_1,\mathrm{},𝐢_r\}`$, and take $`(k1,j^{})𝐢_a`$ if $`(k,j)𝐢_a`$ for $`a=1,\mathrm{},r`$. The formulas (2.3.2) and (2.3.3) are the analogue of $`(3.4)_r`$ and $`(3.5)_r`$ in \[M, VI, 3\].

First we show that

###### Lemma 2.4.

1. For $`𝜶𝒫_𝐦`$, we have

$$D_\pm ^r(q,t)m_𝜶(x)=\underset{𝜷}{}\underset{J_r}{}t^{\mathrm{}𝜹,J\mathrm{}}q^{\mathrm{}𝜷,J\mathrm{}}s_{(𝜷+𝜹)_{J\pm }𝜹}(x),$$

where $`𝜷Z`$ runs over all the row permutations of $`𝜶`$.
2. For $`𝜶𝒫_𝐦`$, we have,

$$D_\pm ^r(q,t)m_𝜶(x)=\underset{𝜷Z_n^{0,0}}{}b_{𝜶,𝜷}^{r,\pm }(q,t)m_𝜷(x)$$

with $`b_{𝜶,𝜷}^{r,\pm }(q,t)F`$, where $`b_{𝜶,𝜷}^{r,\pm }(q,t)=0`$ unless $`𝜷𝜶`$ or $`𝜷<𝜶`$. In particular, $`D_\pm ^r`$ is an operator on the space $`\mathrm{\Xi }_{𝐦,F}`$.

###### Proof.

For each $`𝜷Z_n^{0,0}`$, we have

$$D_\pm ^r(q,t)x^𝜷=a_𝜹(x)^1\underset{w_1𝔖_𝐦}{}\epsilon (w_1)\underset{J_r}{}t^{\mathrm{}w_1(𝜹),J\mathrm{}}q^{\mathrm{}𝜷,J\mathrm{}}x^{(𝜷+w_1(𝜹))_{J\pm }}.$$

If we replace $`𝜷`$ by $`w_2(𝜷)`$ for $`w_2𝔖_𝐦`$ and put $`w_2=w_1w`$, the term $`(𝜷+w_1(𝜹))_{J\pm }`$ (resp. $`\mathrm{}𝜷,J\mathrm{}`$ ) is replaced by $`w_1((w(𝜷)+𝜹)_{J^{}\pm })`$ (resp. $`\mathrm{}w(𝜷),J^{}\mathrm{}`$ ), respectively, with $`J^{}=w_1^1(J)`$. It follows that for $`𝜶Z_n^{0,0}`$, we have

$`D_\pm ^r(q,t)m_𝜶(x)`$ $`=|𝔖_𝜶|^1a_𝜹(x)^1{\displaystyle \underset{w,w_1𝔖_𝐦}{}}\epsilon (w_1)`$
$`\times {\displaystyle \underset{J^{}_r}{}}t^{\mathrm{}𝜹,J^{}\mathrm{}}q^{\mathrm{}w(𝜶),J^{}\mathrm{}}x^{w_1((w(𝜶)+𝜹)_{J^{}\pm })}`$
$`=|𝔖_𝜶|^1{\displaystyle \underset{w𝔖_𝐦}{}}{\displaystyle \underset{J_r}{}}t^{\mathrm{}𝜹,J\mathrm{}}q^{\mathrm{}w(𝜶),J\mathrm{}}s_{(w(𝜶)+𝜹)_{J\pm }𝜹}(x).`$
This proves (i).

Next we show (ii). Since $`𝜹=𝚲^0`$ by our assumption, $`𝜶+𝜹`$ coincides with the symbol $`𝚲(𝜶)`$ for $`𝜶Z_n^{0,0}`$. We note that if $`w1`$, then the symbol $`w(𝜶)+𝜹`$ belongs to a family strictly smaller than the family containing $`𝜶`$. On the other hand, if $`w=1`$, $`(𝜶+𝜹)_{J\pm }`$ is obtained from the symbol $`𝜶+𝜹`$ by permuting some entries, and $`s_{(𝜶+𝜹)_{J\pm }𝜹}`$ coincides with $`\pm s_𝜸`$, where $`𝜸+𝜹`$ is obtained from $`(𝜶+𝜹)_{J\pm }`$ by rearranging the rows in decreasing order. It follows that $`𝜸+𝜹`$ is contained in the same family as $`𝜶+𝜹`$. Hence, for $`𝜷Z_n^{0,0}`$ in the expression of (ii), we see that $`𝜷<𝜶`$ if $`w1`$ and $`𝜷𝜶`$ if $`w=1`$. ∎

Next we show

###### Lemma 2.5.

The operators $`D_\pm ^r`$ are adjoint each other, i.e., we have

(2.5.1)
$$\mathrm{}D_+^rf,g\mathrm{}=\mathrm{}f,D_{}^rg\mathrm{},(f,g\mathrm{\Xi }_{𝐦,F}).$$

###### Proof.

By Lemma 1.6, (2.5.1) is equivalent to the formula

(2.5.2)
$$\mathrm{\Pi }^1(D_+^r)_x\mathrm{\Pi }=\mathrm{\Pi }^1(D_{}^r)_y\mathrm{\Pi }.$$

But for $`J_r`$, we have

$`\mathrm{\Pi }^1(T_{q,J}^+)_x\mathrm{\Pi }`$ $`={\displaystyle \underset{k=0}{\overset{e1}{}}}{\displaystyle \underset{j1}{}}{\displaystyle \underset{(k,i)J}{}}{\displaystyle \frac{1x_i^{(k)}y_j^{(k)}}{1tx_i^{(k)}y_j^{(k+1)}}},`$
$`\mathrm{\Pi }^1(T_{q,J}^{})_y\mathrm{\Pi }`$ $`={\displaystyle \underset{k=0}{\overset{e1}{}}}{\displaystyle \underset{i1}{}}{\displaystyle \underset{(k,j)J}{}}{\displaystyle \frac{1x_i^{(k)}y_j^{(k)}}{1tx_i^{(k1)}y_j^{(k)}}}.`$
It follows that both of $`\mathrm{\Pi }^1(D_+^r)_x\mathrm{\Pi }`$ and $`\mathrm{\Pi }^1(D_{}^r)_y\mathrm{\Pi }`$ are independent of $`q`$. Hence in the proof of (2.5.2), we may assume that $`q=t`$. In other words, we have only to prove (2.5.1) under the assumption that $`q=t`$.

Now assume that $`q=t`$. Since

$$T_{t,J}^\pm (x^{w(𝜹)}f)=x^{w(𝜹)_{J\pm }}t^{\mathrm{}w(𝜹),J\mathrm{}}T_{t,J}^\pm f$$

for any polynomial $`fF[x]`$, we have

$$D_\pm ^r(t,t)f=a_𝜹^1\underset{J_r}{}T_{t,J}^\pm (a_𝜹f).$$

It follows that for any $`𝜶Z_n^{0,0}`$,

$`D_\pm ^r(t,t)s_𝜶`$ $`=a_𝜹^1{\displaystyle \underset{J_r}{}}T_{t,J}^\pm (a_{𝜶+𝜹})`$
$`={\displaystyle \underset{J_r}{}}t^{\mathrm{}𝜶+𝜹,J\mathrm{}}s_{(𝜶+𝜹)_{J\pm }𝜹}.`$
As before, $`s_{(𝜶+𝜹)_{J\pm }𝜹}`$ coincides with $`\pm s_𝜷`$, where $`𝜷+𝜹=[(𝜶+𝜹)_{J\pm }]`$ (under the notation in 2.3). Now in the case where $`q=t`$, $`\{s_𝜶(x)𝜶𝒫_𝐦\}`$ is an orthonormal basis of $`\mathrm{\Xi }_{𝐦,F}`$. It follows that

(2.5.3)
$$\mathrm{}D_+^r(t,t)s_𝜶,s_𝜷\mathrm{}=\underset{J}{}\epsilon _{J+}t^{\mathrm{}𝜶+𝜹,J\mathrm{}},$$

where $`J`$ runs over all the elements in $`_r`$ such that $`[(𝜶+𝜹)_{J+}]`$ coincides with $`𝜷+𝜹`$, and $`\epsilon _J=(1)^{ł(w)}`$ with $`w𝔖_𝐦`$ such that $`[(𝜶+𝜹)_{J+}]=w((𝜶+𝜹)_{J+})`$. But if $`𝜷+𝜹=w((𝜶+𝜹)_{J+})`$, we have $`𝜶+𝜹=w^1((𝜷+𝜹)_J^{})=[(𝜷+𝜹)_J^{}]`$ with $`J^{}=w(J)`$. Also in this case,

$$\mathrm{}𝜷+𝜹,J^{}\mathrm{}=\mathrm{}w(𝜶+𝜹)_{J^{}+},J^{}\mathrm{}=\mathrm{}w(𝜶+𝜹),J^{}\mathrm{}=\mathrm{}𝜶+𝜹,J\mathrm{}.$$

Since $`\epsilon _{J+}=\epsilon _J^{}`$, we see that the right hand side of (2.5.3) is equal to

$$\underset{J^{}}{}\epsilon _J^{}t^{\mathrm{}𝜷+𝜹,J^{}\mathrm{}},$$

where $`J^{}_r`$ runs over all the elements such that $`[(𝜷+𝜹)_J^{}]=𝜶+𝜹`$. Clearly this coincides with $`\mathrm{}s_𝜶,D_{}^r(t,t)s_𝜷\mathrm{}`$. So the lemma is proved. ∎

2.6. We fix a total order $``$ on $`Z_n^{0,0}`$ as in 1.11, so that it is compatible with the partial order $`<`$. Let $`B_\pm ^r=(b_{𝜶,𝜷}^{r,\pm })`$ be the matrix consisting of the coefficients in the formula in Lemma 2.4 (ii) with respect to the total order $``$. We consider $`B_\pm ^r`$ as a block matrix with respect to the equivalence relation $`𝜶𝜷`$, and denote it as $`(B_,^{}^{r,\pm })`$, where $`B_,^{}^{r,\pm }`$ is the submatrix of $`B_\pm ^r`$ corresponding to the families $`,^{}`$. Then Lemma 2.4 (ii) implies that $`B_\pm ^r`$ is lower triangular as a block matrix. We consider the Macdonald functions $`P_𝚲^\pm `$ constructed via $``$. The following result shows that the set of Macdonald functions attached to symbols in a fixed family behaves as an eigenfunction for Macdonald operators, where the eigenvalues should be replaced by the diagonal blocks $`B_,^{r,\pm }`$ of $`B_\pm `$.

###### Proposition 2.7.

Let $`P_𝚲^\pm (x;q,t)\mathrm{\Xi }_{𝐦,F}`$ be Macdonald functions attached to $`𝚲=𝚲(𝛂)`$. Then we have

$$D_\pm ^rP_𝚲^\pm =\underset{𝜷𝜶}{}b_{𝜶,𝜷}^{r,\pm }P_{𝚲(𝜷)}^\pm ,$$

where the coefficients $`b_{𝛂,𝛃}^{r,\pm }`$ are the same as in Lemma 2.4 (ii).

###### Proof.

By Proposition 1.12, $`P_𝚲^\pm `$ can be written as

$$P_𝚲^\pm =m_𝜶+\underset{𝜷𝜶,𝜷\sim ̸𝜶}{}u_{𝜶,𝜷}^{}m_𝜷$$

with $`u_{𝜶,𝜷}^{}F`$. It follows, by Lemma 2.4 (ii), that one can write as

$$D_\pm ^rP_𝚲^\pm =\underset{𝜷𝜶}{}b_{𝜶,𝜷}^{r,\pm }P_{𝚲(𝜷)}^\pm +\underset{𝚲^{}𝚲,𝚲^{}\sim ̸𝚲}{}c_{𝚲,𝚲^{}}^\pm P_𝚲^{}^\pm .$$

Hence, for each $`𝚲^{}`$ such that $`𝚲^{}𝚲`$ and that $`𝚲^{}\sim ̸𝚲`$, we have

$$\mathrm{}D_+^rP_𝚲^+,P_𝚲^{}^{}\mathrm{}=c_{𝚲,𝚲^{}}^+.$$

On the other hand thanks to Lemma 2.5, we have

$$\mathrm{}D_+^rP_𝚲^+,P_𝚲^{}^{}\mathrm{}=\mathrm{}P_𝚲^+,D_{}^rP_𝚲^{}^{}\mathrm{}=0$$

since $`D_{}^rP_𝚲^{}^{}`$ is a linear combination of $`P_{𝚲^{\prime \prime }}^{}`$, where $`𝚲^{\prime \prime }𝚲^{}`$ or $`𝚲^{\prime \prime }𝚲^{}`$. It follows that $`c_{𝚲,𝚲^{}}^+=0`$ and the proposition holds for the $`+`$ case. The $``$ case is similar. ∎

In view of Proposition 2.7, it is important to know the diagonal part of $`B_\pm `$. By lemma 2.4, the matrix $`B_,^\pm `$ is described as follows.

###### Lemma 2.8.

For $`𝛂,𝛃Z_n^{0,0}`$ such that $`𝛃𝛂`$, we have

$$b_{𝜶,𝜷}^{r,\pm }(q,t)=\underset{\begin{array}{c}J_r\\ [𝚲_{J\pm }]=𝚲(𝜷)\end{array}}{}\epsilon _{𝚲,J\pm }(tq^1)^{\mathrm{}𝜹.J\mathrm{}}q^{\mathrm{}𝚲,J\mathrm{}},$$

where $`𝚲=𝛂+𝛅Z_n^{1,0}`$ and $`\epsilon _{𝚲,J\pm }=(1)^{l(w)}`$ for $`w𝔖_𝐦`$ such that $`[𝚲_{J\pm }]=w(𝚲_{J\pm })`$.

Remark 2.9. The results in this section work only for a special type of symbols. It seems to be difficult to extend the definition of Macdonald operators in (2.3.1) directly to a more general case. A naive idea for the general situation is to replace $`𝜹`$ by $`𝚲^0`$ in the definition (2.3.1). Then one gets some operator related to the symbols associated with $`𝚲^0`$. However, the thus obtained operator does not preserve the set of polynomials in $`x`$, since it involves the factor $`a_{𝚲^0}(x)^1`$. If one leaves the denominator $`a_𝜹(x)`$ unchanged, and replaces $`𝜹`$ in all other places, then the operator preserves polynomials, but it does not preserve the degree of them, and is not so useful.

## 3. A characterization of Macdonald functions

3.1. We write the operator $`D_\pm ^r`$ as $`D_{𝐦,\pm }^r`$ to indicate the dependence on $`𝐦`$. The operators $`D_{𝐦,\pm }^r`$ are not compatible with the restriction homomorphisms $`\rho _{𝐦^{},𝐦}`$. In the case where $`r=1`$, one can modify $`D_{𝐦,\pm }^1`$ as discussed in \[M1\], so that they are compatible with $`\rho _{𝐦^{},𝐦}`$. Let us define an operator $`E_𝐦^\pm =E_𝐦^\pm (q,t)`$ on $`\mathrm{\Xi }_{𝐦,F}`$ by

(3.1.1)
$$E_𝐦^\pm =t^MD_{𝐦,\pm }^1\underset{𝐢}{}t^{\mathrm{}𝜹,𝐢\mathrm{}M}.$$

We show the following lemma.

###### Lemma 3.2.

The operators $`E_𝐦^\pm `$ are compatible with $`\rho _{𝐦^{},𝐦}`$, i.e., we have

$$\rho _{𝐦^{},𝐦}E_𝐦^{}^\pm =E_𝐦^\pm \rho _{𝐦^{},𝐦}$$

for $`𝐦^{}=(m_0+1,\mathrm{},m_{e1}+1)`$.

###### Proof.

Let us define another operator $`\stackrel{~}{E}_𝐦^\pm :\mathrm{\Xi }_{𝐦,F}\mathrm{\Xi }_{𝐦,F}`$ by

$$\stackrel{~}{E}_𝐦^\pm =t^MD_{𝐦,\pm }^1\underset{𝐢}{}t^{\mathrm{}𝜹,𝐢\mathrm{}M}a_𝜹^1T_{1,𝐢}^\pm a_𝜹.$$

Then for each $`𝜶𝒫_𝐦`$, we have

(3.2.1)
$$\stackrel{~}{E}_𝐦^\pm m_𝜶=\underset{𝜷}{}\underset{𝐢}{}(q^{\mathrm{}𝜷,𝐢\mathrm{}}1)t^{{\scriptscriptstyle i_k}}s_{(𝜷+𝜹)_{𝐢\pm }𝜹},$$

where $`𝜷Z`$ runs over all the row permutations of $`𝜶`$, and $`𝐢=(i_0,\mathrm{},i_{e1})`$. In fact, by applying \[M1, VI, 4\] one can write $`m_𝜶=_𝜷s_𝜷`$ with $`𝜷`$ as above. Since $`a_𝜹^1T_{1,𝐢}^\pm a_𝜹s_𝜷=s_{(𝜷+𝜹)_{𝐢\pm }𝜹}`$, we obtain (3.2.1).

We claim that $`\stackrel{~}{E}_𝐦^\pm `$ is compatible with $`\rho _{𝐦^{},𝐦}`$, i.e.,

(3.2.2)
$$\rho _{𝐦^{},𝐦}\stackrel{~}{E}_𝐦^{}^\pm =\stackrel{~}{E}_𝐦^\pm \rho _{𝐦^{},𝐦}.$$

Recall that the map $`\rho _{𝐦^{},𝐦}:\mathrm{\Xi }_{𝐦^{},F}\mathrm{\Xi }_{𝐦,F}`$ is defined by substituting $`x_{m_k+1}^{(k)}=0`$ for $`k=0,\mathrm{},e1`$. Take $`𝜶Z_n^{0,0}(𝐦)`$ and let $`𝜶^{}Z_n^{0,0}(𝐦^{})`$ be the element obtained by adding 0’s to the last part of $`𝜶`$. We consider the expression of $`\stackrel{~}{E}_𝐦^{}^\pm m_𝜶^{}`$ as given in (3.2.1). Let $`𝜹^{}`$ be the element for $`𝐦^{}`$ corresponding to $`𝜹`$ for $`𝐦`$, and take $`𝜷^{}Z(𝐦^{})`$. We note that $`s_{(𝜷^{}+𝜹^{})_{𝐢\pm }𝜹^{}}`$ goes to zero under $`\rho _{𝐦^{},𝐦}`$ if there exists some $`k`$ such that $`\beta _{m_k+1}^{(k)}0`$ for $`𝜷^{}=(\beta _j^{(k)})`$. In fact, if we write $`𝜷^{}+𝜹^{}=(\gamma _j^{(k)})`$, then $`\gamma _j^{(k)}>0`$ for $`jm_k`$. Hence if $`\beta _{m_k+1}^{(k)}0`$ for some $`k`$, then $`(𝜷^{}+𝜹^{})_{𝐢\pm }=(\gamma _{}^{}{}_{j}{}^{(k)})`$ contains a row $`\gamma _{}^{}{}_{}{}^{(k^{})}`$ whose entries are all non-zero, and so $`s_{(𝜷^{}+𝜹^{})_{𝐢\pm }𝜹^{}}`$ goes to zero.

It follows that in the expression of $`\stackrel{~}{E}_𝐦^{}^\pm m_𝜶^{}`$ in (3.2.1), we may only consider $`𝜷^{}`$ (a row permutation of $`𝜶^{}`$ ) such that the last column consists of zeros. One can embed $`Z(𝐦)`$ into $`Z(𝐦^{})`$ by adding 0 to the last part of $`𝜷Z(𝐦)`$. Then those $`𝜷^{}`$ are identified, under the embedding $`Z_n^{0,0}(𝐦)Z_n^{0,0}(𝐦^{})`$, with a row permutation $`𝜷`$ of $`𝜶`$. Now $`(𝐦)`$ is also embedded into $`(𝐦^{})`$ in the same way. Take $`𝐢(𝐦^{})`$ such that $`𝐢(𝐦)`$. If $`𝐢`$ is not equal to $`𝐢_1={}_{}{}^{t}(m_0+1,\mathrm{},m_{e1}+1)`$, then $`s_{(𝜷^{}+𝜹^{})_{𝐢\pm }𝜹^{}}`$ goes to zero under $`\rho _{𝐦^{},𝐦}`$ by the same reason as before. If $`𝐢=𝐢_1`$, then $`\mathrm{}𝜷^{},𝐢\mathrm{}=0`$. In any case, such $`𝐢`$ does not give a contribution, and we may only consider $`𝐢(𝐦)`$ in (3.2.1). Hence (3.2.2) holds.

Let $`\stackrel{~}{H}_\pm `$ be the coefficient matrix of $`\stackrel{~}{E}_𝐦^\pm m_𝜶`$ in terms of $`m_𝜷`$. Then by Lemma 2.4 (ii), $`\stackrel{~}{H}_\pm `$ can be expressed as a block matrix $`\stackrel{~}{H}_\pm =(\stackrel{~}{H}_,^{}^\pm )`$, where $`\stackrel{~}{H}_,^{}^\pm =0`$ unless $`=^{}`$ or $`^{}<`$. Then by comparing (3.2.1) with Lemma 2.4 (i) on the diagonal parts, we see that

$$\stackrel{~}{H}_,^\pm =t^MB_,^\pm \underset{𝐢}{}t^{\mathrm{}𝜹,𝐢\mathrm{}M}.$$

On the other hand, if we write $`\stackrel{~}{H}_,^\pm =(h_{𝜶,𝜷}^\pm )_{𝜶,𝜷}`$, Proposition 2.7 together with (3.1.1) implies that

$$E_𝐦^\pm P_{𝚲(𝜶)}^\pm =\underset{𝜷𝜶}{}h_{𝜶,𝜷}^\pm P_{𝚲(𝜷)}^\pm .$$

But by (3.2.2), the matrix $`\stackrel{~}{H}_,^\pm `$ does not depend on the shift operation under $`𝐦^{}𝐦`$. This shows that the operator $`E_𝐦^\pm `$ is compatible with $`\rho _{𝐦^{},𝐦}`$. The lemma is proved. ∎

3.3. By Lemma 3.2, one can define an operator $`E^\pm =E^\pm (q,t)`$ on $`\mathrm{\Xi }_F`$ as the limit of $`E_𝐦^\pm `$. Since $`E_𝐦^\pm `$ satisfies a similar formula as given in Lemma 2.5, the operator $`E^\pm `$ also satisfies the adjointness property, i.e., we have

(3.3.1)
$$\mathrm{}E^+f,g\mathrm{}=\mathrm{}f,E^{}g\mathrm{},(f,g\mathrm{\Xi }_F).$$

By Lemma 2.4 (ii), one can write, for each $`𝜶𝒫_{n,e}`$,

(3.3.2)
$$E^\pm (q,t)m_𝜶(x)=\underset{𝜷Z_n^{0,0}}{}h_{𝜶,𝜷}^\pm (q,t)m_𝜷(x)$$

with $`h_{𝜶,𝜷}^\pm (q,t)F`$, where $`h_{𝜶,𝜷}^\pm (q,t)=0`$ unless $`𝜷𝜶`$ or $`𝜷<𝜶`$. Let $`H_\pm =(h_{𝜶,𝜷}^\pm )`$, and write it as a block matrix $`H_\pm =(H_,^{}^\pm )`$ as in 2.6.

In the case of symmetric groups, the matrix $`H_\pm `$ is a triangular matrix, with distinct eigenvalues. This property was used to characterize Macdonald functions as eigenfunctions of Macdonald operators. As an analogy, it is likely that the following property holds for the diagonal parts of the block matrix $`B_\pm `$.

Conjecture A. Let $`,^{}`$ be any distinct families in $`Z_n^{0,0}`$. Then the matrices $`H_,^\pm `$ and $`H_^{},^{}^\pm `$ have no common eigenvalues (according to the sign $`+`$ or $``$, respectively).

We have verified the conjecture in the case where $`e=2`$, and $`n5`$.

3.4. Before giving a characterization of Macdonald functions in terms of Macdonald operators, we prepare an easy lemma. Let $`A=(a_{ij})`$ (resp. $`B=(b_{ij})`$) be a square matrix of degree $`m`$ (resp. $`n`$), and let $`C=(C_{𝜶,𝜷})_{1\alpha ,\beta n}`$ be a block matrix of size $`mn`$, consisting of blocks $`C_{\alpha ,\beta }`$ of size $`m`$, defined by

$$C_{𝜶,𝜷}=\{\begin{array}{cc}Ab_{\alpha ,\alpha }I_m\hfill & \text{ if }\beta =\alpha ,\hfill \\ b_{\alpha ,\beta }I_m\hfill & \text{ otherwise.}\hfill \end{array}$$

We consider a matrix equation $`AX=XB`$, where $`X=(x_{ij})`$ is a $`m\times n`$ matrix of unknown variables. Then this equation can be regarded as a system of linear equations with respect to the $`mn`$ variables $`\{x_{ij}\}`$, whose coefficient matrix is given by the matrix $`C`$. Moreover, if $`B`$ is a triangular matrix, then $`C`$ is block wise triangular, and so $`detC0`$ if and only if $`det(Ab_{\alpha ,\alpha }I_m)0`$ for $`1\alpha n`$. Hence we have the following lemma.

###### Lemma 3.5.

Under the above notation, the following are equivalent.

1. The matrix equation $`AX=XB`$ has a unique solution $`X=0`$.
2. $`detC0`$.
3. The matrices $`A`$ and $`B`$ have no common eigenvalues.

We now show the following.

###### Theorem 3.6.

Suppose that Conjecture A holds. Then the Macdonald functions $`P_𝚲^\pm \mathrm{\Xi }_F`$ are characterized by the following two properties.

(3.6.1) $`P_𝚲^\pm `$ $`=m_𝜶+{\displaystyle \underset{𝜷<𝜶}{}}w_{𝜶,𝜷}^\pm m_𝜷,`$
(3.6.2) $`E^\pm P_𝚲^\pm `$ $`={\displaystyle \underset{𝜷𝜶}{}}h_{𝜶,𝜷}^\pm P_{𝚲(𝜷)}^\pm .`$
In particular, $`P_𝚲^\pm `$ are determined independently from the choice of the total order $``$.

###### Proof.

Let $`\{_iiI\}`$ be the set of families in $`Z_n^{0,0}`$. We give a total order $``$ on the index set $`I`$, according to the total order $``$ on $`Z_n^{0,0}`$, and write $`H_\pm `$ as $`H_\pm =(H_{ij})_{i,jI}`$, where $`H_{ij}=H_{_i,_j}^\pm `$. (Since the following discussion is independent of the sign $`\{\pm \}`$, we omit them). Let $`X=(w_{𝜶,𝜷}^\pm )`$ be the transition matrix between basis $`\{m_𝜶\}`$ and $`\{P_𝚲^\pm \}`$ of $`\mathrm{\Xi }_F`$, and write it as $`X=(X_{ij})`$. By Proposition 1.12, $`X`$ is block wise lower triangular, with identity diagonal blocks. Moreover, by Proposition 2.7, we see that $`XHX^1=G`$, where $`G=(G_{ij})`$ is a block diagonal matrix with diagonal blocks $`G_{ii}=H_{ii}`$. In order to prove the theorem, we have only to show the following.

(3.6.3) Let $`X=(X_{ij})`$ be a block wise lower triangular matrix, with identity diagonal blocks, such that $`XHX^1=G`$. Then $`X`$ is determined uniquely, and $`X_{ij}=0`$ unless $`i=j`$ or $`_j<_i`$.

We show (3.6.3). The equation $`XHX^1=G`$ can be written as

(3.6.4)
$$H_{ii}X_{ij}X_{ij}H_{jj}=\underset{jki}{}X_{ik}H_{kj}$$

for any pair $`ji`$. By backwards induction on $`j`$, we may assume that $`X_{ik}`$ are already determined for $`jki`$. Then (3.6.4) determines $`X_{ij}`$ uniquely, by Conjecture A and Lemma 3.5. Now suppose that $`_j_i`$. Again by induction, we may assume that $`X_{ik}=0`$ unless $`_k<_i`$. Since $`H_{kj}=0`$ unless $`_j<_k`$ by (3.3.2), we must have $`H_{ii}X_{ij}X_{ij}H_{jj}=0`$. This implies that $`X_{ij}=0`$ by Lemma 3.5, and we obtain (3.6.3). Thus the theorem is proved. ∎

Remark 3.7. The Hall-Littlewood function $`P_𝚲^\pm (x;t)`$ given in \[S1\] coincides with $`P_𝚲^\pm (x;0,t)`$. Hence it satisfies similar formulas as (3.6.1), (3.6.2). In particular, $`P_𝚲^\pm (x;t)`$, and so the Kostka functions $`K_{𝜶,𝜷}^\pm (x;t)`$ do not depend on the choice of the total order. This answers the questions posed in \[S1, Remark 4.5, (ii)\] and in \[GM, Remark 2.4\], modulo the truth of the conjecture.

3.8. We give here some examples of the matrices $`B_,^\pm `$ and $`H_,^\pm `$ for some small rank cases with $`e=2`$. Assume that $`W`$ is the Weyl group of type $`C_2`$. Then the symbols and families are given as follows.

$`_1`$ $`=\left\{\left({\displaystyle \genfrac{}{}{0pt}{}{\mathrm{3\; 0}}{0}}\right)\right\},_2=\left\{\left({\displaystyle \genfrac{}{}{0pt}{}{\mathrm{2\; 1\; 0}}{\mathrm{2\; 1}}}\right)\right\},`$
$`_3`$ $`=\{\left({\displaystyle \genfrac{}{}{0pt}{}{\mathrm{2\; 0}}{1}}\right),\left({\displaystyle \genfrac{}{}{0pt}{}{\mathrm{2\; 1}}{0}}\right),\left({\displaystyle \genfrac{}{}{0pt}{}{\mathrm{1\; 0}}{2}}\right)\},`$
which correspond, in this order, to the double partitions of 2,

$$(2;),(;11),(1;1),(11;),(;2)$$

respectively. In the tables below, we express the matrices $`B_,^\pm ,H_,^\pm `$ as $`B_{},H_{}`$, since they are independent of the sign $`\pm `$. We have

$`B__1=(1),B__2=(t^3q+tq),B__3=\left(\begin{array}{ccc}0& q& tq^2\\ q& 0& tq\\ tq^2& q^2& 0\end{array}\right).`$
Up to $`C_5`$, only 3-element or 1-element families occur. The Weyl group of type $`C_6`$ contains a unique 10-element family, which is given as follows,

$$\begin{array}{cc}\hfill _4=\{& \left(\genfrac{}{}{0pt}{}{\mathrm{4\; 2\; 0}}{\mathrm{3\; 1}}\right),\left(\genfrac{}{}{0pt}{}{\mathrm{4\; 2\; 1}}{\mathrm{3\; 0}}\right),\left(\genfrac{}{}{0pt}{}{\mathrm{4\; 3\; 0}}{\mathrm{2\; 1}}\right),\left(\genfrac{}{}{0pt}{}{\mathrm{4\; 3\; 1}}{\mathrm{2\; 0}}\right),\left(\genfrac{}{}{0pt}{}{\mathrm{4\; 3\; 2}}{\mathrm{1\; 0}}\right),\hfill \\ & \left(\genfrac{}{}{0pt}{}{\mathrm{4\; 1\; 0}}{\mathrm{3\; 2}}\right),\left(\genfrac{}{}{0pt}{}{\mathrm{3\; 2\; 0}}{\mathrm{4\; 1}}\right),\left(\genfrac{}{}{0pt}{}{\mathrm{3\; 2\; 1}}{\mathrm{4\; 0}}\right),\left(\genfrac{}{}{0pt}{}{\mathrm{3\; 1\; 0}}{\mathrm{4\; 2}}\right),\left(\genfrac{}{}{0pt}{}{\mathrm{2\; 1\; 0}}{\mathrm{4\; 3}}\right)\}.\hfill \end{array}$$

The corresponding double partitions of 6 are, in this order, as follows.

$`(21;21),(211;2),(22;11),(221;1),(222;),`$
$`(2;22),(11;31),(111;3),(1;32),(;33).`$
The matrix $`B_{}`$ is given by

$$B_{}=\left(\begin{array}{cccccccccc}0& q& t^2q^3& 0& tq^2& tq^2& t^3q^4& 0& 0& t^2q^3\\ q& 0& 0& t^2q^3& tq^3& tq& 0& t^3q^4& 0& t^2q^2\\ t^2q^3& 0& 0& q& tq& tq^3& t^3q^3& 0& t^2q^3& 0\\ 0& t^2q^3& q& 0& tq^2& tq^2& 0& t^3q^3& t^2q^2& 0\\ tq^2& t^2q^2& q^2& tq^2& 0& 0& t^2q^2& t^3q^2& 0& 0\\ tq^2& q^2& t^2q^2& tq^2& 0& 0& 0& 0& t^3q^4& t^2q^4\\ t^3q^4& 0& t^2q^4& 0& tq^3& 0& 0& q& tq^2& t^2q^2\\ 0& t^3q^4& 0& t^2q^4& tq^4& 0& q& 0& tq& t^2q\\ 0& 0& t^2q^3& tq^3& 0& t^3q^4& tq^2& q^2& 0& t^2q^3\\ t^2q^3& tq^3& 0& 0& 0& t^3q^3& tq^3& q^3& t^2q^3& 0\end{array}\right).$$

Throughout the above examples, $`H_{}`$ is given by

$$H_{}=t^{(2m+1)}B_{}t^{(2m+1)}\frac{(1t^m)(1t^{m+1})}{(1t)^2}$$

for symbols of the shape $`𝐦=(m+1,m)`$, (so, $`m=1`$ for $`_1,_3`$, and $`m=2`$ for $`_2,_4`$, respectively). Note that $`B_{}`$ is not necessarily symmetric. However, if we put $`q=t`$, it turns out to be symmetric since $`D^1`$ is a self adjoint operator, and the representation matrix with respect to the orthonormal basis of Schur functions coincides with the diagonal blocks $`(B_{})`$.

3.9. In the case of symmetric groups, Macdonald operators are commuting with each other since they are simultaneously diagonalizable. In our case, Proposition 2.7 shows that $`D_\pm ^r`$ are simultaneously diagonalizable in the sense of block matrices. So they are commuting with each other if and only if the matrices $`B_,^{r,\pm }`$ are commuting with for $`r=1,\mathrm{},M_1`$ (for a fixed $``$). As the following examples show, one might expect that $`B_,^{r,\pm }`$ are commuting with each other in general.

First consider a simple example. Let $`_3=\{𝚲_1,𝚲_2,𝚲_3\}`$ be the 3-element family of $`C_2`$ as in 3.8. We consider its $`m`$ times shifts $`_3^{(m)}=\{𝚲_1^{},𝚲_2^{},𝚲_3^{}\}`$, where

$$𝚲_i^{}=\left(\genfrac{}{}{0pt}{}{x+m,y+m,m1,\mathrm{},0}{z+m,m1,\mathrm{},0}\right)$$

for $`𝚲_i=\left(\genfrac{}{}{0pt}{}{xy}{z}\right)`$. Then the matrices $`B__3^r`$ are given as

$$B__3^r=aB__3^1+b$$

with

$$a=t^{2m}\underset{J^{}}{}t^{\mathrm{}𝜹,J^{}\mathrm{}},b=\underset{J}{}t^{\mathrm{}𝜹,J\mathrm{}},$$

where $`J_r`$ runs over the elements of the form $`J=\{𝐢_{j_1},\mathrm{},𝐢_{j_r}\}`$ for $`𝐢_j=\left(\genfrac{}{}{0pt}{}{j+1}{j}\right)`$ with $`2jm+1`$, and $`J^{}_{r1}`$ runs over the elements having similar properties. This implies that $`B__3^r`$ are all commuting with for $`1rm`$.

In the following, we discuss some related results, i.e., we show that when $`e=2`$, and $`q=t`$, then the operators $`D^r`$ are commuting with each other. First we prepare a lemma. (Since we deal with the case where $`e=2`$, we omit the sign $`\pm `$ in the discussion below.)

###### Lemma 3.10.

Assume that $`e=2`$. Then, for each $`r,r^{}`$, there exists a bijective map $`\phi :_r\times _r^{}_r\times _r^{}`$ satisfying the following properties: let $`J_r,J^{}_r^{}`$ and put $`\phi (J,J^{})=(K,K^{})`$. For each $`𝛂Z`$, we have

1. $`𝜶_{JJ^{}}=𝜶_{K^{}K}`$,
2. $`\mathrm{}𝜶,J\mathrm{}+\mathrm{}𝜶_J,J^{}\mathrm{}=\mathrm{}𝜶,K^{}\mathrm{}+\mathrm{}𝜶_K^{},K\mathrm{}`$.

###### Proof.

Take $`J_r,J^{}_r^{}`$. Let $`J_0`$ be the subset of $`J`$ consisting of $`𝐢=\left(\genfrac{}{}{0pt}{}{a}{b}\right)`$ such that there exists $`𝐢^{}=\left(\genfrac{}{}{0pt}{}{x}{y}\right)J^{}`$ with $`x=a`$ or $`y=b`$, and let $`J_0^{}`$ be the subset of $`J^{}`$ having similar properties. We define an equivalence relation on $`J_0`$ by connecting $`𝐢=\left(\genfrac{}{}{0pt}{}{a}{b}\right),𝐢^{}=\left(\genfrac{}{}{0pt}{}{c}{d}\right)J_0`$ when there exists $`\left(\genfrac{}{}{0pt}{}{x}{y}\right)J^{}`$ such that $`(x,y)=(a,d)`$ or $`(x,y)=(c,b)`$. We denote by $`\{J_CC𝒞\}`$ the set of equivalence classes in $`J_0`$. Then the class $`J_C`$ has the following form.

$`J_C`$ $`=\{\left({\displaystyle \genfrac{}{}{0pt}{}{a_1}{c}}\right),\left({\displaystyle \genfrac{}{}{0pt}{}{a_2}{b_1}}\right),\mathrm{},\left({\displaystyle \genfrac{}{}{0pt}{}{a_{k1}}{b_{k2}}}\right),\left({\displaystyle \genfrac{}{}{0pt}{}{a_k}{b_{k1}}}\right),\left({\displaystyle \genfrac{}{}{0pt}{}{d}{b_k}}\right)\},`$
where $`\left(\genfrac{}{}{0pt}{}{a_i}{b_i}\right)J^{}`$ for $`1ik`$. We put

$`J_C^{}=\{\left({\displaystyle \genfrac{}{}{0pt}{}{a_i}{b_i}}\right)piq\},`$
where $`p=0`$ (resp. $`q=k+1`$) if there exists $`\left(\genfrac{}{}{0pt}{}{a_0}{b_0}\right)J^{}`$ (resp. $`\left(\genfrac{}{}{0pt}{}{a_{k+1}}{b_{k+1}}\right)J^{}`$) such that $`b_0=c`$ (resp. $`a_{k+1}=d`$), and $`p=1,q=k`$ otherwise.

For each $`C𝒞`$, we define the sets $`K_C,K_C^{}`$ as follows:

$`K_C=\{\begin{array}{cc}J_C\hfill & \text{ if }p=1,q=k,\hfill \\ J_C^{}\hfill & \text{ if }p=1,q=k+1,\hfill \\ J_C^{}\hfill & \text{ if }p=0,q=k,\hfill \\ \{\left({\displaystyle \genfrac{}{}{0pt}{}{a_0}{b_1}}\right),\mathrm{},\left({\displaystyle \genfrac{}{}{0pt}{}{a_k}{b_{k+1}}}\right)\}\hfill & \text{ if }p=0,q=k+1.\hfill \end{array}`$
$`K_C^{}`$ $`=\{\begin{array}{cc}\{\left({\displaystyle \genfrac{}{}{0pt}{}{a_2}{c}}\right),\left({\displaystyle \genfrac{}{}{0pt}{}{a_3}{b_1}}\right),\mathrm{},\left({\displaystyle \genfrac{}{}{0pt}{}{a_k}{b_{k2}}}\right),\left({\displaystyle \genfrac{}{}{0pt}{}{d}{b_{k1}}}\right)\}\hfill & \text{ if }p=1,q=k,\hfill \\ J_C\hfill & \text{ if }p=1,q=k+1,\hfill \\ J_C\hfill & \text{ if }p=0,q=k,\hfill \\ J_C^{}\hfill & \text{ if }p=0,q=k+1.\hfill \end{array}`$
Since $`J,J^{}`$ are subsets of the index set of elements in $`Z`$, $`J_C`$, etc. induce permutations on the entries of elements in $`Z`$. We denote by $`x_C,x_C^{}`$ (resp. $`y_C,y_C^{}`$) the permutations in $`𝔖_M`$ corresponding to $`J_C,J_C^{}`$ (resp. $`K_C,K_C^{}`$), respectively. Then it is easy to check that

(3.10.1)
$$x_C^{}x_C=y_Cy_C^{}.$$

We put

$$K=(JJ_0)\underset{C𝒞}{}K_C,K^{}=(J^{}J_0^{})\underset{C𝒞}{}K_C^{}.$$

Components of $`K`$ (resp. $`K^{}`$) are mutually disjoint, and we see that $`K_r,K^{}_r^{}`$. We now define the map $`\phi :_r\times _r^{}_r\times _r^{}`$ by $`\phi (J,J^{})=(K,K^{})`$. Then one can check that $`\phi ^2=\mathrm{id}`$, and so $`\phi `$ is a bijection. The assertion (i) follows from (3.10.1). To show (ii), it is enough to verify the formula in the case where $`J=J_C,J^{}=J_C^{},K=K_C,K^{}=K_C^{}`$. The assertion is clear when $`K_C=J_C^{},K_C^{}=J_C`$. Assume that $`p=1,q=k`$ or $`p=0,q=k+1`$. Then one can check by a direct computation that

$$\mathrm{}𝜶,J_C\mathrm{}=\mathrm{}𝜶_{K_C^{}},K_C\mathrm{},\mathrm{}𝜶_{J_C},J_C^{}\mathrm{}=\mathrm{}𝜶,K_C^{}\mathrm{}.$$

Hence the formula holds in these cases also, and the lemma follows. ∎

###### Proposition 3.11.

Assume that $`e=2`$, and $`q=t`$. Then the operators $`D^r(t,t)`$ are commuting with each other for $`r=1,\mathrm{},M_1`$.

###### Proof.

As remarked in 3.9, it is enough to show that the matrices $`B_{}^r=B_,^{r,\pm }`$ are commuting with each other for $`r=1,\mathrm{},M_1`$. By Lemma 2.4 (i), the part corresponding to the diagonal block in the expression of $`D^r(t,t)s_𝜶`$ is given as

$$\underset{J_r}{}t^{\mathrm{}𝚲,J\mathrm{}}s_{𝚲_J𝜹},$$

where $`𝚲=𝜶+𝜹`$. Let $`𝒳`$ be the part of $`D^r^{}(t,t)D^r(t,t)s_𝜶`$ corresponding to the diagonal blocks. Then we have

$$𝒳=\underset{J_r}{}\epsilon _Jt^{\mathrm{}𝚲,J\mathrm{}}\underset{J^{\prime \prime }_r^{}}{}t^{\mathrm{}[𝚲_J],J^{\prime \prime }\mathrm{}}s_{[𝚲_J]_{J^{\prime \prime }}𝜹}.$$

For each $`J`$, there exists $`w𝔖_𝐦`$ such that $`[𝚲_J]=w(𝚲_J)`$ and $`\epsilon _J`$ is given by $`\epsilon _J=(1)^{l(w)}`$. Then if we put $`J^{}=w^1(J^{\prime \prime })`$, we have $`\mathrm{}[𝚲_J],J^{\prime \prime }\mathrm{}=\mathrm{}𝚲_J,J^{}\mathrm{}`$, and $`[𝚲_J]_{J^{\prime \prime }}=w(𝚲_{JJ^{}})`$. Hence one can write

$$𝒳=\underset{J_r}{}\underset{J^{}_r^{}}{}\epsilon _{J,J^{}}t^{\mathrm{}𝚲,J\mathrm{}+\mathrm{}𝚲_J,J^{}\mathrm{}}s_{[𝚲_{JJ^{}}]𝜹},$$

where $`\epsilon _{J,J^{}}=(1)^{l(w^{})}`$ for $`w^{}𝔖_𝐦`$ such that $`[𝚲_{JJ^{}}]=w^{}(𝚲_{JJ^{}})`$. Now by applying Lemma 3.10 for $`𝜶=𝚲`$, we see that $`𝒳`$ coincides with the diagonal part for $`D^r(t,t)D^r^{}(t,t)s_𝜶`$. The proposition is proved. ∎


format: text

text:
# Interaction of vector solitons with a nonlinear interface

## I Introduction

Spatial solitons are self-trapped optical beams for which diffraction divergence is exactly compensated by nonlinear self-focusing Segev and Stegeman (1998). Such solitons possess many important properties which are believed to find a wide range of potential applications in optical scanning, switching, and processing devices. Spatial solitons can be generated in photorefractive crystals at low powers (of order of $`\mu W`$) Duree et al. (1993) and, therefore, such materials look promising from a viewpoint of their use for the soliton generation in all-optical information circuits.

One of the important properties of spatial solitons in photorefractive media is their stability against collapse and break-up even in a bulk medium due to the saturating nature of nonlinearity. Spatial solitons in (1+1)-dimensional systems can be realized for the light propagation in weakly guiding planar structures, where self-focusing properties of the core material lead to the light localization along the plane of a linear guiding structure.

Usually, a spatial soliton is viewed as a lowest-order (fundamental) mode of an effective waveguide it excites in a nonlinear medium. Moreover, higher-order modes of the induced waveguide, that are associated with an effective soliton-induced potential possessing more than one energy level, can be excited and such modes can be treated as the soliton internal modes. Such internal modes are responsible for oscillations of the soliton amplitude and width, and they have already been studied in a number of theoretical papers Pelinovsky et al. (1999); Etrich et al. (1996). A much more interesting situation takes place in the case of the so-called vector spatial solitons, i.e. multi-component optical solitons which consist of several mutually incoherent (or orthogonally polarized) co-propagating beams. This problem got a little of attention in the past Afanasyev et al. (1989), but it attracts much attention in view of recent experimental studies of multicomponent optical solitons.

The aim of the present paper is twofold. First, we study the internal dynamics of (1+1)-dimensional vector solitons, associated with the transverse relative oscillation of the centers of mass of the partial soliton components. Second, for the first time to our knowledge, we study the scattering of a vector soliton by a nonlinear interface and demonstrate that the scattering process can be modified dramatically by initial excitation of the soliton internal modes.

As an example, we consider the dynamics of solitons in photorefractive nonlinear media taking into account saturation of the medium nonlinearity. A standard way for studying the excited states of solitons is the analysis of the corresponding linear eigenvalue problem for small perturbations. However, in the case of nonintegrable systems, such excited eigenstates can be found only by rather complicated computer simulations (see, for example, Ref. Pelinovsky and Yang (2000) and references therein). In contrast, here we suggest the method of momenta and applied it to analize the scattering problem. As a matter of fact, such a method corresponds to the so-called quasi-classical approximation in quantum mechanics. Regardless of the fact that the method of momenta leads to the continuous spectrum of the eigenstates, it allows us to consider large perturbations (including the vector soliton splitting into individual components), since this approach is dealing with average characteristics of a vector soliton.

## II Vector spatial solitons. <br>The method of momenta

We consider spatial vector solitons consisting of $`N`$ incoherently coupled components, which are described by the following set of Schrödinger-like coupled nonlinear equations (see. e.g. Ref. Ostrovskaya et al. (1999) and references therein):

$$i\frac{\mathrm{\Psi }_j}{x}+\frac{^2\mathrm{\Psi }_j}{z^2}\lambda _j\mathrm{\Psi }_j+ϵ_{NL}\left(I\right)\mathrm{\Psi }_j=0,$$
(1)
where $`x`$ and $`z`$ are the propogation and transverse coordinates, respectively, $`\mathrm{\Psi }_j`$ ($`j=1,\mathrm{}N`$) are the dimensionless slowly varying amplitudes of the beam components, $`\lambda _j`$ are the partial propagation constants, $`ϵ_{NL}\left(I\right)`$ is the nonlinear correction to the dielectric permittivity, $`I=_{j=1}^NI_j`$ is the total intensity of the beam, and $`I_j=\left|\mathrm{\Psi }_j\right|^2`$ are the partial intensities of the soliton components. We assume that the nonlinear correction to the dielectric permittivity depends only on the total intensity of the light, $`I`$, which can be written approximately as follows Luther-Davis et al. (1994):

$$ϵ_{NL}(I)=I/\left(1+sI\right),$$
(2)
where $`s`$ is the saturation parameter. A ground state of a vector spatial soliton corresponds to the stationary solution of eq. (1) with $`/x=0`$ in which $`\mathrm{\Psi }_j=\mathrm{\Psi }_j^{(0)}\left(zZ_j\right)`$, where $`\mathrm{\Psi }_j^{(0)}\left(z\right)`$ are *real* localized functions and $`Z_j`$ is the center of mass of the $`j`$-th component. As a rule, the values $`Z_j`$ for the vector solitons in the ground state are equal to each other and coincide with its center of mass. The dynamical behaviour of an excited vector soliton, such as oscillation of its amplitude and width, oscillation of the centers of mass of its components $`Z_j`$, or superposition of these oscillations, depends on initially excited intrinsic degrees of freedom. Here, it is important to note an analogy between a vector spatial soliton and a bound state of quantum particles. Indeed, the difference lies only in self-consistent nonlinear nature of the potential that provides the confinement for soliton components.

To study excited states of vector solitons we use the method of momenta that has been initially developed for scalar solitons of a Kerr-type nonlinear medium Vlasov et al. (1971). Here we generalize this aproach and technique to study multi-component spatial solitons in photorefractive media.

The method of momenta describes the electromagnetic beams in terms of integral field characteristics, or field momenta Vlasov et al. (1971). The first momentum is defined as

$$P_j=_{\mathrm{}}^{\mathrm{}}I_j𝑑z.$$
(3)
It has the obvious physical meaning of the energy flux in the $`j`$-th soliton component. The second momentum is defined as

$$\overline{Z_j}=_{\mathrm{}}^{\mathrm{}}zI_j𝑑z.$$
(4)
Here the value $`Z_j=\overline{Z_j}/P_j`$ is the center of mass of the $`j`$-th beam component.

The purpose of this section is to derive a close set of effective ordinary evolution equations for the momenta (3, 4). Similar procedure is used in semiclassical description of quantum systems (see, e.g. Ref. Sherwin (1959)). Let us multiply Eq. (1) by $`\mathrm{\Psi }_j^{}`$ to obtain an equation for $`P_j`$, and by $`z\mathrm{\Psi }_j^{}`$ to obtain an equation for $`Z_j`$, and then subtract the corresponding complex conjugated equations. The integration of the resulting equations over $`z`$ yields

$$\frac{dP_j}{dx}=0,$$
(5)
$$iP_j\frac{dZ_j}{dx}=_{\mathrm{}}^{\mathrm{}}\left(\mathrm{\Psi }_j^{}\frac{\mathrm{\Psi }_j}{z}\mathrm{\Psi }_j\frac{\mathrm{\Psi }_j^{}}{z}\right)𝑑z.$$
(6)
Equation (5) means that the energy flux in each individual soliton component is a conserved quantity. Thus, there is no energy exchange between the components. After differentiation of Eq. (6) by $`x`$ and subsequent integration over $`z`$, we obtain

$$P_j\frac{d^2Z_j}{dx^2}=_{\mathrm{}}^{\mathrm{}}I_j\frac{ϵ_{NL}}{z}𝑑z.$$
(7)
The physical meaning of the Eqs. (7) comes from an analogy with the geometrical optics: the Eqs. (7) determine optical rays that correspond to the partial beams forming a vector soliton. From the set of equations (7) one can see that the center of mass of the whole beam moves along a straight line:

$$\frac{d^2Z_c}{dx^2}=\frac{d^2}{dx^2}\left(\frac{Z_1P_1+Z_2P_2+\mathrm{}+Z_NP_N}{P_1+P_2+\mathrm{}+P_N}\right)=0.$$
(8)
The set of Eqs. (7) is not closed. It cannot be used directly since, in order to trace the dynamics of the soliton components, we should know the electric field at each point $`(x,z)`$ in space, i.e. effectively complete solutions of the Eqs. (1) are required. In this sense Eqs. (7) are equivalent to Eqs. (1), and so far we have not gained anything in addition. There are two ways to derive a closed set of equations from Eqs. (7). The first one is to obtain equations for higher order momenta which, nonetheless, does not guarantee that the system of equations will become closed. A simpler way, which allows us to obtain the desired result, is to employ a trial function to be substituted into Eqs. (7) instead of the unknown functions $`\mathrm{\Psi }_j`$. It is clear that, for small deviations of the soliton constituents from the lowest-order (ground) equilibrium state, we can choose the trial functions in the form of unperturbed stationary soliton components. Moreover, since within Eqs. (7) we are operating with the average (integral) characteristics of the soliton, one can guess that the result of the integration will not be sensitive to variations of the transverse soliton structure, even for large deviations from the ground state. This is a rather strong statement, and it will be verified below by means of direct numerical simulations of the Eqs. (1).

## III Two-component solitons

To demonstrate the internal dynamics of the soliton excited states, we consider a particular case of two-component vector solitons. As discussed above, it is natural to choose the trial function for the Eqs. (7) in the form of a stationary solution $`\mathrm{\Psi }_j^{(0)}(zZ_j)`$. According to Eqs. (7), the relative shift between the soliton components, $`\mathrm{\Delta }=Z_2Z_1`$, is governed by the following equation:

$$\overline{P}\frac{d^2\mathrm{\Delta }}{dx^2}=F\left(\mathrm{\Delta }\right),$$
(9)
where $`\overline{P}=P_1P_2/(P_1+P_2)`$ is the reduced mass, and

$$F(\mathrm{\Delta })=2_{\mathrm{}}^{\mathrm{}}I_2(z+\mathrm{\Delta })\frac{ϵ_{NL}\left[I_1(z)+I_2(z+\mathrm{\Delta })\right]}{z}𝑑z.$$
(10)
Let us now introduce the effective potential of the beams interaction as follows

$$F(\mathrm{\Delta })=\frac{dV(\mathrm{\Delta })}{d\mathrm{\Delta }}.$$
(11)
We have now come to the simple mechanical model for the internal dynamics of the two-component soliton, which describes two interacting quasi-particles in a bound state. This state can be thought of as a ”photon molecule”. Fig. 1 shows the effective potential $`V(\mathrm{\Delta })`$ for numerically calculated one-hump soliton with two identical components. The minimum of the effective potential corresponds to the stationary soliton in a ground state. The internal oscillations of excited soliton components deducted from the solution of Eq. (9) (shown in Fig. 1 (c) by solid lines) are compared to the numerical solution of eqs.(1) (dashed lines in Fig. 1 (c)). The damping of the oscillations occurs due to radiation losses of the energy from the soliton originating from bends of the soliton component trajectories, which are not taken into account in the method of momenta.

The next example deals with the two-hump solitons with the transverse structures of the components shown in Fig. 2 (a,c) Ostrovskaya and Kivshar (1998). One can see that, when the amplitude of the two-hump component is significantly smaller than the amplitude of the one-hump one, the stationary state with coaxial constituents is unstable because of the maximum of the potential at $`\mathrm{\Delta }=0`$ (see Fig. 2 (b)). It is interesting to note that, in this case, the ground equilibrium state of the soliton corresponds to the soliton components with displaced centers. Such (1+1)D asymmetric vector solitons are well known from the theory of the two-component Kerr solitons described by Manakov model ($`s0`$ in eqn. 2) Akhmediev et al. (1999) and for the partially coherent multi-component solitons in photorefractive medium Litchinitser et al. (1999). However, when the amplitudes of the components are comparable, the coaxial state becomes stable again (see Fig. 2 (d)). The dynamics of the unstable coaxial soliton state calculated by solving eqs.(9), as well as eqs.(1), is shown in Fig. 3. One can see that, in this case, the agreement between the method of momenta and numerical simulations is much worse than for the one-hump solitons because of the strong reshaping of the two-hump beams. This reshaping, along with the radiation processes, violates the assumptions of the analytical method.

## IV Interaction of vector solitons with a nonlinear interface

The method of momenta looks promising for treating the problem of vector soliton dynamics in inhomogeneous media. In this section we consider interaction of a two-component vector soliton with an interface between two photorefractive media. Both linear and nonlinear properties of the materials are discontinuous on the interface. The problem of one-component photorefractive soliton interaction with nonlinear interface had recently been studied in Boardman et al. (2000), soliton interaction with interface in the Kerr-like medium can be found in Newell and Moloney (1992); Aceves et al. (1989). Let us represent the coordinate dependences of the partial propagation constants and nonlinear correction to dielectric permittivity by the following form:

$`\lambda _{1,2}(z)=\lambda _{1,2}+\delta \lambda _{1,2}1(z),`$
$`ϵ_{NL}(z)=ϵ_{NL}(I)\left[1+\delta ϵ_{NL}1(z)\right]`$ (12)
where $`1(z)=\{\begin{array}{cc}\hfill 1,z>0& \\ \hfill 0,z<0& \end{array}`$ is the unit step function. By the method of momenta one can obtain that the energy fluxes in both beams are conserved. For the centers of mass of the components we have:

$`P_{1,2}{\displaystyle \frac{d^2Z_{1,2}}{dx^2}}={\displaystyle _{\mathrm{}}^{\mathrm{}}}\left[2+\delta ϵ_{NL}1(z)\right]I_{1,2}{\displaystyle \frac{ϵ_{NL}(I)}{z}}𝑑z`$
$`\delta ϵ_{NL}I_{1,2}(Z_{1,2})+\delta \lambda _{1,2}I_{1,2}(Z_{1,2})\times `$
$`\times ϵ_{NL}\left[I_{1,2}(Z_{1,2})+I_{2,1}(Z_{2,1})\right]`$
All the terms in the Eqs. (IV) have clear physical meaning. The terms that do not contain the jumps of the medium parameters represent the internal forces acting between soliton components. They are responsible for an internal dynamics of the soliton and, in the homogeneous case, can be reduced to the equation (9). The terms proportional to $`\delta \lambda _{1,2}`$ and $`\delta ϵ_{NL}`$ are the external forces perturbing the internal oscillations of the soliton components at the collisions of quasi-particles with an interface.

To derive the closed set of equations describing the interaction of a vector soliton with an interface we substitute into Eqs. (IV) trial functions in the form of the stationary unperturbed soliton component (see Section II). One can expect that this ansatz will be sufficiently accurate whenever transverse structures of the beams are not strongly affected by large discontinuities of the medium parameters. We will verify this assumption by comparing direct computer simulation of the wave equations (1) (in the case of inhomogeneous media) with the results of quasi-particle theory obtained with the method of momenta. In what follows, we study the following effects of soliton interaction with an interface: (a) excitation of internal degrees of freedom; (b) the change of the type of soliton interaction with an interface (i.e. reflection, transmission and capture) as function of the initial internal energy of the vector soliton launched under different initial conditions; (c) the splitting of pre-excited soliton into individual components at the collision with an interface. It should be noted, with regard to the point (a) above, that a spatial soliton consisting of identical components, being initially in ground state, does not become excited at the collision with an interface. This happens because, due to identical character of interaction of every component with interface, the coaxial structure of the soliton state is preserved.

The results that we present in this section are obtained for the one-hump soliton shown in the inset in Fig. 4 (a) and interface parameters are as follows: $`\delta \lambda _{1,2}=0.15`$; $`\delta ϵ_{NL}=0.2`$; $`ds=0`$.

The solutions of the ordinary differential equations (IV) are shown in Fig. 4. Excitation of internal oscillations after soliton reflection from an interface is demonstrated in Fig. 4 (a). Soliton dynamics near the interface depends dramatically on the initial energy level of the internal oscillations of the components. In Fig. 4 (b) soliton components were launched to the interface at the same angle $`\left(dZ_{1,2}/dx=0.5\right)`$ and from the same position of the common center of mass. The only adjustable parameter, which determines the initial internal energy of oscillations, was the relative shift between components $`\mathrm{\Delta }`$. One can see that, depending on the value of $`\mathrm{\Delta }`$, the transmission (curve 5 – $`\mathrm{\Delta }=0.12`$), reflection (curves 2,3,4 – $`\mathrm{\Delta }=0.4,0.8,1.2`$, respectively) and capture into the unstable nonlinear surface mode (curve 5 – $`\mathrm{\Delta }=0.0`$) take place.

It is interesting to note that the character of soliton interaction depends upon the parameter $`\mathrm{\Delta }`$ non-monotonically, which can be explained by considering the phase of the internal oscillations at the interface. This phase depends on the initial value of $`\mathrm{\Delta }`$ by virtue of anharmonicity of internal oscillations in the effective potential well. Fig. 5 demonstrates the results of numerical calculations of equations (1), in which $`\lambda _{1,2}`$ and $`ϵ_{NL}`$ are taken according to eq. (12). Excitation of the internal degree of freedom takes place at the reflection from the interface Fig. 5 (a). The calculations shown in Fig. 5 (b) were carried out for the same initial conditions as for curves 1,2, and 3 in Fig. 4 (b). The higher values of the initial relative shift $`\mathrm{\Delta }`$ between individual components do not result in the same behaviour of the trajectories as in Fig. 4 (b). It can be easily explained by the strong damping of oscillations due to the radiation effects, which takes place for large values $`\mathrm{\Delta }`$. As a result, the amplitude of the internal oscillations is small at the point where soliton meets the interface and the outcome of interaction is very similar to the one for the smaller initial values of $`\mathrm{\Delta }`$. To conclude, one can check that method of momenta leads to the satisfactory agreement with the direct numerical solution of nonlinear Schrödinger equations for the values of $`\mathrm{\Delta }`$ up to the half width of the soliton.

The splitting of the vector soliton into individual components is shown in Fig. 4. We have revealed that the splitting occurs at the angles of incidence of the soliton beam close to the angle of total internal reflection and for large enough values of initial relative shift between components ($`\mathrm{\Delta }=2.2`$, $`dZ_1/dx=dZ_2/dx=0.5`$). Without interface, the soliton components make up a bound state (Fig. 4 (c)). At the presence of interface the method of momenta predicts that splitting into components takes place (Fig. 4 (d)). This effect can be treated in terms of the quasi-particle theory. Indeed, when the depth of effective potential well is of the same order that the energy level of the bound state, than even a small perturbation (at the interface) of this state can lead to its destruction.

We have obtained the similar splitting effect \[shown in Fig. 5 (c)\] by solving the equations (1) numerically for the initial conditions used in the method of momenta. In numerical simulations, we can observe the separation of the beam components, but in the case of such large values of the initial transverse shift, the value of the first momentum (4) fails to determine the position of the center of mass of the localized beams. Significant part of the energy is transferred into radiation, and the integration in (4) gives us the coordinate of the common center of mass of both localized and nonlocalized waves. It is clear that the less the radiation losses are, the better the position of the localized beam is described by (4).

Finally, it is neccessary to note that the dramatic reshaping of the multi-hump solitons at the interaction with the interface takes place. Numerical simulations show that these complex beams can be transformed into a set of single hump solitons and a considerable part of the electromagnetic energy goes into the radiation even for a very small change of the parameters of the medium at the interface (see Fig. 6). This reshaping makes it impossible to apply the method of momenta to the problems of multi-hump soliton dynamics in the inhomogeneous media.

## V Conclusions

We have developed the method of momenta to analyze the dynamics of vector solitons consisting of two (or more) incoherently coupled components, in both homogeneous and inhomogeneous photorefractive nonlinear media. First, we have demonstrated that this method provides an effective analytical tool for the study of internal oscillations of vector solitons. Second, we have studied the scattering of a vector soliton by a nonlinear interface, and we have demonstrated that this type of the soliton interaction can lead to the excitation of the soliton’s internal oscillations. In particular, we have demonstrated that the same vector soliton incident onto an interface can become reflected, transmitted, captured by the interface, or even split into its constituents, depending solely on the energy of the initially excited internal oscillations.

###### Acknowledgements.

Authors wish to thank Yu.S.Kivshar and E.A.Ostrovskaya for the help in the preparation of the article.