text:
# Kazhdan-Lusztig polynomials and character formulae for the Lie superalgebra 𝖌𝖑(𝒎|𝒏)

## 1. Introduction

The problem of computing the characters of the finite dimensional irreducible representations of the Lie superalgebra $`𝔤𝔩(m|n)`$ over $``$ was raised originally by V. Kac in 1977 \[Ka2, Ka3\]. Kac proved that the finite dimensional universal highest weight modules, known nowadays as Kac modules, are irreducible for so-called typical highest weights, and gave a formula for their characters. After that, there were several conjectures and partial results dealing with atypical highest weights \[BL, BR, Sg1, JHKT1, JHKT2, HKJ, KaW, PS2\], before the complete solution to the problem was given by V. Serganova \[S2, S3\] in 1995 using a mixture of algebraic and geometric techniques.

In this article, we present a different, purely algebraic solution of the problem. One consequence is a proof of a conjecture made by van der Jeugt and Zhang \[JZ\], which is apparently closely related to the conjectures made in \[HKJ\]. In particular the composition multiplicities of the Kac modules are all either $`0`$ or $`1`$, a fact which does not seem to follow easily from Serganova’s formula since that involves certain alternating sums. We also formulate for the first time a conjecture for the characters of the infinite dimensional irreducible representations in the analogue of category $`𝒪`$ for the Lie superalgebra $`𝔤𝔩(m|n)`$.

Inspired by ideas of Lascoux, Leclerc and Thibon \[LLT\], our approach is to relate the finite dimensional representation theory of $`𝔤𝔩(m|n)`$ to the structure of the module

$$^{m|n}:=\stackrel{m}{}𝒱^{}\stackrel{n}{}𝒱,$$

where $`𝒱`$ denotes the natural representation of the quantized enveloping algebra $`U_q(𝔤𝔩_{\mathrm{}})`$. By work of Lusztig \[L, ch. 27\], the module $`^{m|n}`$ possesses a canonical basis $`\{U_\lambda \}`$ and a dual canonical basis $`\{L_\lambda \}`$, see Theorems 3.6 and 3.13, which for the purpose of this introduction we parametrize via (4.4) by the set $`X^+(m|n)`$ of dominant integral weights for $`𝔤𝔩(m|n)`$. The entries of the transition matrices between these bases and the natural monomial basis $`\{K_\lambda \}`$ of $`^{m|n}`$ define polynomials $`u_{\mu ,\lambda }(q)`$ and $`l_{\mu ,\lambda }(q)`$ for each $`\mu ,\lambda X^+(m|n)`$:

$$U_\lambda =\underset{\mu X^+(m|n)}{}u_{\mu ,\lambda }(q)K_\mu ,L_\lambda =\underset{\mu X^+(m|n)}{}l_{\mu ,\lambda }(q)K_\mu .$$

The remarkable thing is that it is quite easy to compute these polynomials explicitly, because all the $`𝔰𝔩_2`$-strings in the crystal graph underlying the module $`^{m|n}`$ are of length $`2`$, see Corollary 3.39 for the explicit formulae. The main result of the article shows that the polynomials $`u_{\mu ,\lambda }(q)`$ evaluated at $`q=1`$ compute the composition multiplicities of the Kac modules, see Theorem 4.37 and (4.39). Moreover, the polynomials $`l_{\mu ,\lambda }(q^1)`$ coincide with the Kazhdan-Lusztig polynomials $`K_{\lambda ,\mu }(q)`$ defined originally by Serganova \[S2, S3\], so can be used to compute $`\mathrm{Ext}`$’s between Kac modules and irreducible modules, see Theorem 4.51 and Corollary 4.52.

The module $`^{m|n}`$ is a summand of the tensor space

$$𝒯^{m|n}:=\stackrel{m}{}𝒱^{}\stackrel{n}{}𝒱.$$

The latter also possesses a canonical basis $`\{T_\lambda \}`$ and a dual canonical basis $`\{L_\lambda \}`$, see Theorems 2.17 and 2.23, which we parametrize via (4.4) by the set $`X(m|n)`$ of all integral weights for $`𝔤𝔩(m|n)`$. For $`\lambda X^+(m|n)X(m|n)`$, the elements $`L_\lambda `$ here coincide with the images of the elements with the same name in the previous paragraph under the embedding $`^{m|n}𝒯^{m|n}`$. The entries of the transition matrices between these bases and the natural monomial basis $`\{M_\lambda \}`$ of $`𝒯^{m|n}`$ give us polynomials $`t_{\mu ,\lambda }(q)`$ and $`l_{\mu ,\lambda }(q)`$ for each $`\mu ,\lambda X(m|n)`$:

$$T_\lambda =\underset{\mu X(m|n)}{}t_{\mu ,\lambda }(q)M_\mu ,L_\lambda =\underset{\mu X(m|n)}{}l_{\mu ,\lambda }(q)M_\mu .$$

These should be viewed as the true combinatorial analogues for $`𝔤𝔩(m|n)`$ of the Kazhdan-Lusztig polynomials of \[KL, Deo\]. We explain an explicit algorithm to compute $`t_{\mu ,\lambda }(q)`$ in 2, and conjecture based on calculations with this algorithm that our polynomials share the positivity properties of the usual Kazhdan-Lusztig polynomials, see Conjecture 2.28. We conjecture moreover that the polynomials $`t_{\mu ,\lambda }(q)`$ evaluated at $`q=1`$ compute the composition multiplicities of the Verma modules in category $`𝒪`$, see Conjecture 4.32 and (4.34). This conjecture is true in the case $`m=0`$ by the original Kazhdan-Lusztig conjecture \[KL\] for the Lie algebra $`𝔤𝔩(n)`$ proved in \[BB, BrK\], see Theorem 4.31(i).

Returning to finite dimensional representations, let us now formulate the conjecture of van der Jeugt and Zhang proved here precisely, to give the flavor of the combinatorics that arises. So let $`𝔤`$ denote the Lie superalgebra $`𝔤𝔩(m|n)`$ over $``$, labeling rows and columns of matrices in $`𝔤`$ by the ordered index set $`I(m|n)=\{m,\mathrm{},1,1,\mathrm{},n\}`$. We work always with the standard choices $`𝔥`$ and $`𝔟`$ of Cartan and Borel subalgebras, consisting of diagonal and upper triangular matrices respectively. For $`iI(m|n)`$, let $`\delta _i𝔥^{}`$ denote the function picking out the $`i`$th diagonal entry of a diagonal matrix. Put a symmetric bilinear form $`(.|.)`$ on $`𝔥^{}`$ by setting $`(\delta _i|\delta _j)=1`$ if $`i=j>0`$, $`1`$ if $`i=j<0`$, and 0 otherwise. Let $`WS_m\times S_n`$ denote the Weyl group associated to $`𝔤_{\overline{0}}`$, acting naturally on $`𝔥^{}`$. We also need the dot action of $`W`$ on $`𝔥^{}`$: $`w\lambda =w(\lambda +\rho )\rho `$ where $`\rho =_{iI(m|n)}i\delta _i`$.

Let $`X(m|n)𝔥^{}`$ denote the set of all $``$-linear combinations of the weights $`\{\delta _i\}_{iI(m|n)}`$, and let $`X^+(m|n)X(m|n)`$ denote the dominant integral weights, namely, the weights $`\lambda =_{iI(m|n)}\lambda _i\delta _iX(m|n)`$ with $`\lambda _m\mathrm{}\lambda _1,\lambda _1\mathrm{}\lambda _n`$. Associated to $`\lambda X^+(m|n)`$, we have the Kac module $`K(\lambda )`$ and its unique irreducible quotient $`L(\lambda )`$, see 4. We should note that there is no loss of generality in restricting our attention to integral weights, since an arbitrary finite dimensional irreducible representation of $`𝔤𝔩(m|n)`$ is either typical or can be obtained from $`L(\lambda )`$ for some $`\lambda X^+(m|n)`$ by tensoring with a one dimensional representation.

Main Theorem. Let $`\lambda X^+(m|n)`$. Let $`r`$ be maximal such that there exist $`mi_1<\mathrm{}<i_r<0<j_r<\mathrm{}<j_1n`$ with $`(\lambda +\rho |\delta _{i_s}\delta _{j_s})=0`$ for each $`s=1,\mathrm{},r`$. Let $`(k_1,\mathrm{},k_r)`$ be the lexicographically smallest tuple of strictly positive integers such that for all $`\theta =(\theta _1,\mathrm{},\theta _r)\{0,1\}^r`$, $`\lambda +_{s=1}^r\theta _sk_s(\delta _{i_s}\delta _{j_s})`$ is conjugate under the dot action of $`W`$ to a dominant weight, denoted $`𝚁_\theta (\lambda )X^+(m|n)`$. Then, for each $`\mu X^+(m|n)`$,

$$[K(\mu ):L(\lambda )]=\{\begin{array}{cc}1\hfill & \text{if }\mu =𝚁_\theta (\lambda )\text{ for some }\theta =(\theta _1,\mathrm{},\theta _r)\{0,1\}^r\text{,}\hfill \\ 0\hfill & \text{otherwise.}\hfill \end{array}$$

To prove the Main Theorem, we work with a different family of modules $`\{U(\lambda )\}_{\lambda X^+(m|n)}`$ called indecomposable tilting modules, following the general framework developed by Soergel \[So2\] and extended to Lie superalgebras in \[B2\]. The problem of computing the multiplicities of Kac modules in indecomposable tilting modules is roughly speaking transpose to the problem of computing the composition multiplicities of Kac modules, see (4.16) for the precise relationship (a twisted BGG reciprocity). The main step in the proof gives an explicit inductive construction of the $`U(\lambda )`$’s starting from the typical case, when $`U(\lambda )=K(\lambda )`$, and applying certain special translation functors that arise from tensoring with the natural module and its dual. Actually, we see eventually that the indecomposable tilting modules in this finite dimensional setting coincide with the indecomposable projectives (also injectives), but they are parametrized by highest weight rather than by their irreducible quotients. Though one could just as well choose to work with the latter more familiar labeling, the alternate parameterization seems to be the one that emerges naturally when considering canonical bases. There are also indecomposable tilting modules denoted $`\{T(\lambda )\}_{\lambda X(m|n)}`$ in category $`𝒪`$, where again they seem to correspond most directly to the canonical basis.

We now explain how the remainder of the article is organized. In sections 2 and 3, we give the construction and properties of the canonical bases of the modules $`𝒯^{m|n}`$ and $`^{m|n}`$ from a purely combinatorial standpoint. Then in section 4 we describe the representation theory of $`𝔤𝔩(m|n)`$, working in two natural categories $`𝒪_{m|n}`$ and $`_{m|n}`$ whose Grothendieck groups are identified with the spaces $`𝒯^{m|n}`$ and $`^{m|n}`$ respectively. In sections 2 and 3 we work exclusively in a $`\rho `$-shifted notation which is more convenient for the combinatorics, replacing the set $`X(m|n)`$ of weights with the set $`^{m|n}`$ of functions $`I(m|n)`$. See (4.4) for the rule to translate between the two notations.

Acknowledgements.  I would like to thank Arkady Berenstein and Jon Kujawa for helpful conversations, Barrie Hughes for pointing out the conjectures made in \[HKJ, JZ\], and Nathan Geer for asking questions that pushed me to think about the results in 4.

## 2. Tensor algebra

In this section, we define and study the canonical basis of the tensor space $`𝒯^{m|n}`$. We will work throughout over the field $`(q)`$ of rational functions, where $`q`$ is an indeterminate.

§2-a. Combinatorial notation. For $`m,n0`$, let $`S_{m|n}`$ denote the symmetric group $`S_m\times S_n`$ acting on the set $`I(m|n)=\{m,\mathrm{},1,1,\mathrm{},n\}`$ so that $`S_m`$ permutes $`\{m,\mathrm{},1\}`$ and $`S_n`$ permutes $`\{1,\mathrm{},n\}`$. Thus $`S_{m|n}`$ is generated by the basic transpositions

$$s_{m+1}=(mm+1),\mathrm{},s_1=(21),s_1=(\mathrm{1\hspace{0.25em}\hspace{0.25em}2}),\mathrm{},s_{n1}=(n1n).$$

Let $`^{m|n}`$ be the set of all functions $`I(m|n)`$. We call $`f^{m|n}`$ antidominant if $`f(m)\mathrm{}f(1),f(1)\mathrm{}f(n)`$. Note $`S_{m|n}`$ acts on the right on $`^{m|n}`$ by composition of functions, and every $`f^{m|n}`$ is conjugate under this action to a unique antidominant function. We also have the ‘flip’ $`\omega :^{m|n}^{n|m}`$, where $`\omega (f)`$ is the function $`I(n|m),if(i)`$.

Let $`P`$ denote the free abelian group on basis $`\{\epsilon _a|a\}`$ endowed with a symmetric bilinear form $`(.,.)`$ for which the $`\epsilon _a`$ form an orthonormal basis. We view $`P`$ as the integral weight lattice associated to the Lie algebra $`𝔤𝔩_{\mathrm{}}`$. The simple roots are the elements $`\epsilon _a\epsilon _{a+1}P`$ for $`a`$. The dominance ordering on $`P`$ is defined by $`\mu \nu `$ if $`(\nu \mu )`$ is an $``$-linear combination of simple roots (here and later $`=\{0,1,2,\mathrm{}\}`$). Equivalently, $`\mu \nu `$ if

$$\underset{ba}{}(\mu ,\epsilon _b)\underset{ba}{}(\nu ,\epsilon _b)$$
(2.1)
for all $`a`$ with equality for $`a0`$.

For $`f^{m|n}`$ and $`jI(m|n)`$, define

$$\mathrm{wt}(f):=\underset{iI(m|n)}{}\mathrm{sgn}(i)\epsilon _{f(i)},\mathrm{wt}_j(f):=\underset{jiI(m|n)}{}\mathrm{sgn}(i)\epsilon _{f(i)},$$
(2.2)
where $`\mathrm{sgn}(i)\{\pm 1\}`$ denotes the sign of $`i`$. The degree of atypicality of $`f^{m|n}`$ is defined to be

$$\mathrm{\#}f:=\frac{1}{2}\left(m+n\underset{a}{}\left|(\mathrm{wt}(f),\epsilon _a)\right|\right).$$
(2.3)
If $`\mathrm{\#}f=0`$, then $`f`$ is called typical. So $`f`$ is typical if and only if

$$\{f(m),\mathrm{},f(1)\}\{f(1),\mathrm{},f(n)\}=\mathrm{}.$$

§2-b. Bruhat ordering. Introduce a partial ordering on $`^{m|n}`$ by declaring that $`gf`$ if $`\mathrm{wt}(g)=\mathrm{wt}(f)`$ and $`\mathrm{wt}_j(g)\mathrm{wt}_j(f)`$ for all $`jI(m|n)`$. It is immediate that if $`gf`$ then $`\mathrm{\#}g=\mathrm{\#}f`$. Using (2.1), we see that $`gf`$ if and only if

$$\underset{\stackrel{jiI\left(m|n\right)}{g(i)a}}{}\mathrm{sgn}(i)\underset{\stackrel{jiI\left(m|n\right)}{f(i)a}}{}\mathrm{sgn}(i)$$
(2.4)
for all $`a`$ and $`jI(m|n)`$, with equality if either $`a0`$ or $`j=m`$. From this, one gets in particular that $`gf`$ if and only if $`\omega (g)\omega (f)`$. In proofs, it will be convenient to have a shorthand for the sums appearing in the inequality (2.4), so for $`f^{m|n},a`$ and $`jI(m|n)`$ we abbreviate

$$\mathrm{\#}(f,a,j)=\underset{\begin{array}{c}jiI(m|n)\\ f(i)a\end{array}}{}\mathrm{sgn}(i).$$

Thus, $`gf`$ if and only if $`\mathrm{\#}(g,a,j)\mathrm{\#}(f,a,j)`$ for all $`a`$ and $`jI(m|n)`$, with equality for $`a0`$ or $`j=m`$.

Here is another description of the partial order. Let $`d_i^{m|n}`$ be the function $`j\mathrm{sgn}(i)\delta _{i,j},`$ for each $`iI(m|n)`$. Write $`fg`$ if one of the following holds:

* $`g=fd_i+d_j`$ for some $`mi1,1jn`$ such that $`f(i)=f(j)`$;
* $`g=f(ij)`$ for some $`1i<jn`$ such that $`f(i)>f(j)`$;
* $`g=f(ij)`$ for some $`mi<j1`$ such that $`f(i)<f(j)`$.

Then:

###### Lemma 2.5.

$`fg`$ if and only if there is a sequence $`h_1,\mathrm{},h_r^{m|n}`$ such that $`f=h_1\mathrm{}h_r=g`$.

###### Proof.

($``$) Obvious.

($``$) We show by induction on $`(m+n)`$ that if $`fg`$ are neigbors in the ordering, then $`fg`$. The case $`m+n=0`$ is vacuous, so suppose $`m+n>0`$. Replacing $`f,g`$ by $`\omega (f),\omega (g)`$ if necessary, we may assume in fact that $`n>0`$. If $`f(n)=g(n)`$, then we are done by induction, so we may assume that $`a=f(n)<g(n)=b`$. We consider two cases.

Case one: there exists $`0<i<n`$ with $`a<f(i)b`$. Pick the greatest such $`i`$, so each $`f(j)`$ for $`j=i+1,\mathrm{},n`$ is either $`a`$ or $`>b`$, and set $`c=f(i)`$. We claim that $`ff(in)g`$, whence $`fg`$ as required since $`f`$ and $`g`$ are neighbors. For $`i<j`$ and $`ad<c`$, we have that $`\mathrm{\#}(f(in),d,j)=\mathrm{\#}(f,d,j)1`$, while $`\mathrm{\#}(f(in),d,j)=\mathrm{\#}(f,d,j)`$ for all other $`j,d`$. Therefore to prove the claim, we just need to show that $`\mathrm{\#}(f,d,j)>\mathrm{\#}(g,d,j)`$ for each $`i<j`$ and each $`ad<c`$. But by the choice of $`i`$, we have that $`\mathrm{\#}(f,d,j)=\mathrm{\#}(f,b,j)\mathrm{\#}(g,b,j)>\mathrm{\#}(g,d,j)`$ since $`g(n)=b`$.

Case two: each $`f(j)`$ for $`j=1,\mathrm{},n`$ is either $`a`$ or $`>b`$. From $`(\mathrm{wt}(g),\epsilon _b)=(\mathrm{wt}(f),\epsilon _b)0`$, we deduce that there must exist $`mi<0`$ with $`g(i)=b`$. Take the greatest such $`i`$. Now we claim that $`fg+d_id_ng`$, so again $`fg`$ as they are neighbors. To prove the claim, note that $`\mathrm{\#}(g+d_id_n,d,j)=\mathrm{\#}(g,d,j)`$ unless $`j>i`$ and $`d=b1`$, while $`\mathrm{\#}(g+d_id_n,b1,j)=\mathrm{\#}(g,b1,j)+1`$ for $`j>i`$. Therefore we need to show that $`\mathrm{\#}(f,b1,j)>\mathrm{\#}(g,b1,j)`$ for each $`j>i`$. Now observe that $`\mathrm{\#}(f,b1,j)\mathrm{\#}(f,b,j)\mathrm{\#}(g,b,j)>\mathrm{\#}(g,b1,j)`$. ∎

For example, writing elements of $`^{2|2}`$ as tuples,

$$(1,2|2,1)(1,2|1,2)(1,3|1,3)(3,1|1,3).$$

It is worth pointing out that $`f^{m|n}`$ is minimal with respect to the ordering just defined if and only if $`f`$ is typical and antidominant.

§2-c. The quantum group. Recall that the quantum integer associated to $`n0`$ is $`[n]:=(q^nq^n)/(qq^1)`$ and the quantum factorial is $`[n]!:=[n][n1]\mathrm{}[2][1].`$ Let $`:(q)(q)`$ be the field automorphism induced by $`qq^1`$. We will call an additive map $`f:VW`$ between $`(q)`$-vector spaces antilinear if $`f(cv)=\overline{c}f(v)`$ for all $`c(q),vV`$.

Let $`𝒰`$ denote the quantum group $`U_q(𝔤𝔩_{\mathrm{}})`$. By definition, this is the $`(q)`$-algebra on generators $`E_a,F_a,K_a,K_a^1(a)`$ subject to relations

$`K_aK_a^1`$ $`=K_a^1K_a=1,`$
$`K_aK_b`$ $`=K_bK_a,`$
$`K_aE_bK_a^1`$ $`=q^{(\epsilon _a,\epsilon _b\epsilon _{b+1})}E_b,`$
$`K_aF_bK_a^1`$ $`=q^{(\epsilon _a,\epsilon _{b+1}\epsilon _b)}F_b,`$
$`E_aF_bF_bE_a`$ $`=\delta _{a,b}{\displaystyle \frac{K_{a,a+1}K_{a+1,a}}{qq^1}},`$
$`E_aE_b`$ $`=E_bE_a`$ $`\text{if }|ab|>1,`$
$`E_a^2E_b+E_bE_a^2`$ $`=(q+q^1)E_aE_bE_a`$ $`\text{if }|ab|=1,`$
$`F_aF_b`$ $`=F_bF_a`$ if $`|ab|>1`$,
$`F_a^2F_b+F_bF_a^2`$ $`=(q+q^1)F_aF_bF_a`$ if $`|ab|=1`$.
Here, for any $`a,b`$, $`K_{a,b}`$ denotes $`K_aK_b^1`$. Also introduce the divided powers $`F_a^{(r)}:=F_a^r/[r]!`$ and $`E_a^{(r)}:=E_a^r/[r]!`$. We have the bar involution on $`𝒰`$, namely, the unique antilinear automorphism such that $`\overline{E_a}=E_a,\overline{F_a}=F_a,\overline{K_a}=K_a^1.`$

We regard $`𝒰`$ as a Hopf algebra with comultiplication $`\mathrm{\Delta }:𝒰𝒰𝒰`$ defined on generators by

$`\mathrm{\Delta }(E_a)`$ $`=1E_a+E_aK_{a+1,a},`$
$`\mathrm{\Delta }(F_a)`$ $`=K_{a,a+1}F_a+F_a1,`$
$`\mathrm{\Delta }(K_a)`$ $`=K_aK_a.`$
This is the comultiplication from Kashiwara \[K2\], and is different from the one in Lusztig’s book \[L\]. The counit $`\epsilon `$ is defined by $`\epsilon (E_a)=\epsilon (F_a)=0`$, $`\epsilon (K_a)=1`$, the antipode $`S`$ by $`S(E_a)=E_aK_{a,a+1},S(F_a)=K_{a+1,a}F_a,S(K_a)=K_a^1.`$

§2-d. The space $`𝒯^{m\mathbf{|}n}`$. Let $`𝒱`$ be the natural $`𝒰`$-module, with basis $`\{v_a\}_a`$ and action defined by

$$K_av_b=q^{\delta _{a,b}}v_b,E_av_b=\delta _{a+1,b}v_a,F_av_b=\delta _{a,b}v_{a+1}.$$

Let $`𝒲=𝒱^{}`$ be the dual $`𝒰`$-module, with basis $`\{w_a\}_a`$ related to the basis of $`𝒱`$ by $`w_a,v_b=(q)^a\delta _{a,b}`$. The action of $`𝒰`$ on $`𝒲`$ satisfies

$$K_aw_b=q^{\delta _{a,b}}w_b,E_aw_b=\delta _{a,b}w_{a+1},F_aw_b=\delta _{a+1,b}w_a.$$

Let $`𝒯^{m|n}:=𝒲^m𝒱^n,`$ viewed as a $`𝒰`$-module in the natural way. Recall that $`^{m|n}`$ denotes the set of all functions $`I(m|n)`$. For $`f^{m|n}`$, we let

$$M_f=w_{f(m)}\mathrm{}w_{f(1)}v_{f(1)}\mathrm{}v_{f(n)}.$$

The vectors $`\{M_f\}_{f^{m|n}}`$ give a basis for $`𝒯^{m|n}`$. A vector $`v`$ in a $`𝒰`$-module $`M`$ is said to be of weight $`\nu P`$ if $`K_av=q^{(\nu ,\epsilon _a)}v`$ for all $`a`$. The weight of the vector $`M_f`$ is $`\mathrm{wt}(f)`$, as defined in (2.2).

We will often work with a completion $`\widehat{𝒯}^{m|n}`$ of $`𝒯^{m|n}`$. To define this formally, let $`_d^{m|n}`$ denote the set of all $`f^{m|n}`$ with $`f(i)d`$ for all $`iI(m|n)`$. Let $`𝒯_d^{m|n}`$ denote the subspace of $`𝒯^{m|n}`$ spanned by $`\{M_f\}_{f_d^{m|n}}`$, and let $`\pi _d:𝒯^{m|n}𝒯_d^{m|n}`$ denote projection along the basis. The filtration $`(\mathrm{ker}\pi _d)_d`$ induces a topology on the abelian group $`𝒯^{m|n}`$, see \[Bou, ch.III, $`\mathrm{\S }`$2.5\]. Let

$$\widehat{𝒯}^{m|n}=\underset{}{\mathrm{lim}}𝒯_d^{m|n}$$

denote the corresponding completion, and identify $`𝒯^{m|n}`$ with its image in $`\widehat{𝒯}^{m|n}`$. The projections $`\pi _d`$ extend by continuity to give maps $`\pi _d:\widehat{𝒯}^{m|n}𝒯_d^{m|n}`$. As usual, we will view elements of $`\widehat{𝒯}^{m|n}`$ as infinite $`(q)`$-linear combinations of the basis elements $`\{M_f\}_{f^{m|n}}`$ whose projections onto each $`𝒯_d^{m|n}`$ are finite sums. A homomorphism $`\theta :𝒯^{m|n}\widehat{𝒯}^{m|n}`$ of abelian groups satisfying the compatibility condition

$`\pi _d(u)=0`$ implies $`\pi _d(\theta (u))=0`$ for all $`u𝒯^{m|n}`$ and all $`d0`$
is automatically continuous, hence extends uniquely to a continuous endomorphism of $`\widehat{𝒯}^{m|n}`$. In particular, the action of $`𝒰`$ lifts uniquely to a continuous action on $`\widehat{𝒯}^{m|n}`$, since $`E_a,F_a`$ and $`K_a`$ commute with $`\pi _d`$ for all $`d>a`$.

§2-e. The Iwahori-Hecke algebra. Associated to the symmetric group $`S_{m|n}`$ we have the Iwahori-Hecke algebra $`_{m|n}`$. This is defined as the $`(q)`$-algebra on generators $`H_{m+1},\mathrm{},H_1,H_1,\mathrm{},H_{n1}`$ subject to relations

$`H_i^2=1(qq^1)H_i,`$
$`H_iH_{i+1}H_i=H_{i+1}H_iH_{i+1},`$
$`H_iH_j=H_jH_i\text{if }|ij|>1.`$
For $`xS_{m|n}`$, we have the corresponding element $`H_x_{m|n}`$, where $`H_x=H_{i_1}\mathrm{}H_{i_r}`$ if $`x=s_{i_1}\mathrm{}s_{i_r}`$ is a reduced expression for $`x`$. The bar involution on $`_{m|n}`$ is the unique antilinear automorphism such that $`\overline{H_x}=H_{x^1}^1,`$ in particular $`\overline{H_i}=H_i+(qq^1)`$.

We define a linear right action of $`_{m|n}`$ on $`𝒯^{m|n}`$ by the formulae

$$M_fH_i=\{\begin{array}{cc}M_{fs_i}\hfill & \text{if }ffs_i\text{,}\hfill \\ q^1M_f\hfill & \text{if }f=fs_i\text{,}\hfill \\ M_{fs_i}(qq^1)M_f\hfill & \text{if }ffs_i\text{.}\hfill \end{array}$$

Since the action of $`_{m|n}`$ commutes with all $`\pi _d`$, it lifts by continuity to $`\widehat{𝒯}^{m|n}`$. As is well-known, see e.g. \[D2\], the actions of $`𝒰`$ and $`_{m|n}`$ on $`𝒯^{m|n}`$ commute with one another, hence the actions on the completion $`\widehat{𝒯}^{m|n}`$ also commute.

§2-f. Some (anti)automorphisms. Let $`\sigma ,\tau :𝒰𝒰`$ be the antiautomorphisms and $`\omega :𝒰𝒰`$ be the automorphism defined by

$`\sigma (E_a)=E_{1a},`$ $`\sigma (F_a)=F_{1a},\sigma (K_a)=K_a,`$
$`\tau (E_a)=q^1K_{a+1,a}F_a,`$ $`\tau (F_a)=qE_aK_{a,a+1},\tau (K_a)=K_a,`$
$`\omega (E_a)=F_a,`$ $`\omega (F_a)=E_a,\omega (K_a)=K_a^1.`$
Let $`\tau :_{m|n}_{m|n}`$ be the antiautomorphism and $`\omega :_{m|n}_{n|m}`$ be the isomorphism defined by $`\tau (H_i)=H_i`$ and $`\omega (H_i)=H_i`$ for $`iI(m1|n1)`$. Introduce the linear map

$$\omega :𝒯^{m|n}𝒯^{n|m},M_fM_{\omega (f)},$$
(2.6)
where $`\omega (f)`$ is as in 2. Note $`\omega `$ extends by continuity to a linear map $`\widehat{𝒯}^{m|n}\widehat{𝒯}^{n|m}`$. Next let $`(.,.)__𝒯`$ be the symmetric bilinear form on $`𝒯^{m|n}`$ defined by

$$(M_f,M_g)__𝒯=\delta _{f,g}$$
(2.7)
for $`f,g^{m|n}`$. Finally, define an antilinear map

$$\sigma :𝒯^{m|n}𝒯^{m|n},M_fM_f.$$
(2.8)
The form $`(.,.)__𝒯`$ and the map $`\sigma `$ do not extend to the completion.

###### Lemma 2.9.

* $`\omega (XuH)=\omega (X)\omega (u)\omega (H)`$ for all $`X𝒰,H_{m|n}`$ and $`u\widehat{𝒯}^{m|n}`$.
* $`(XuH,v)__𝒯=(u,\tau (X)v\tau (H))__𝒯`$ for all $`X𝒰,H_{m|n}`$ and $`u,v𝒯^{m|n}`$.
* $`\sigma (XuH)=\tau (\overline{\sigma (X)})\sigma (u)\overline{H}`$ for all $`X𝒰,H_{m|n}`$ and $`u𝒯^{m|n}`$.

###### Proof.

These are all checked directly for $`_{m|n}`$. To prove them for $`𝒰`$, one first checks that $`\tau `$ and $`\sigma `$ are coalgebra automorphisms and $`\omega `$ is a coalgebra antiautomorphism of $`𝒰`$. Hence it suffices to check (i)–(iii) when $`m+n=1`$. ∎

§2-g. Generation. We proceed to prove that $`\widehat{𝒯}^{m|n}`$ is generated as a topological $`𝒰`$-module by the vectors $`M_f`$ for typical $`f^{m|n}`$.

###### Lemma 2.10.

Suppose that $`f^{m|n}`$ and $`1i_1<\mathrm{}<i_rn`$ are such that $`f(i_1)=\mathrm{}=f(i_r)=a+1`$ and $`f(j)a,a+1`$ for all $`j\{i_1,i_1+1,\mathrm{},n\}\{i_1,\mathrm{},i_r\}`$. Let $`f^{}`$ be the function with $`f^{}(i_1)=\mathrm{}=f^{}(i_r)=a`$ and $`f^{}(j)=f(j)`$ for all $`ji_1,\mathrm{},i_r`$. Then, for any $`gf`$,

$$E_a^{(r)}M_g\delta _{f,g}M_f^{}+\underset{g^{}f^{}}{}[q,q^1]M_g^{}.$$

###### Proof.

Take $`gf`$. Recall the definition of $`d_j^{m|n}`$ from 2. Note $`E_a^{(r)}M_g`$ is a linear combination of $`M_g^{}`$’s where $`g^{}=gd_{j_1}\mathrm{}d_{j_r}`$ for $`j_1<\mathrm{}<j_rI(m|n)`$ such that

$$g(j_s)=\{\begin{array}{cc}a\hfill & \text{if }j_s<0,\hfill \\ a+1\hfill & \text{if }j_s>0\text{.}\hfill \end{array}$$

Let us show that for such a $`g^{}`$, we have that $`g^{}f^{}`$. By (2.4), we need to show that $`\mathrm{\#}(g^{},b,j)\mathrm{\#}(f^{},b,j)`$ for all $`b`$ and $`jI(m|n)`$. Since $`gf`$, we know that $`\mathrm{\#}(g,b,j)\mathrm{\#}(f,b,j)`$. So we are done except possibly for $`b=a`$. Suppose then that $`\mathrm{\#}(g^{},a,j)>\mathrm{\#}(f^{},a,j)`$ for some $`j`$. Say $`i_1,\mathrm{},i_s<ji_{s+1},\mathrm{},i_r`$ and $`j_1,\mathrm{},j_t<jj_{t+1},\mathrm{},j_r`$. Then,

$`\mathrm{\#}(f^{},a,j)`$ $`<\mathrm{\#}(g^{},a,j)=\mathrm{\#}(g,a,j)+(rt)\mathrm{\#}(f,a,j)+(rt)`$
$`=\mathrm{\#}(f^{},a,j)(rs)+(rt)=\mathrm{\#}(f^{},a,j)+st.`$
Hence, we must have that $`s>t`$. This implies in particular that $`j>0`$, and using this we get that

$$\mathrm{\#}(g,a+1,j)\mathrm{\#}(g^{},a,j)>\mathrm{\#}(f^{},a,j)=\mathrm{\#}(f,a+1,j),$$

which is a contradiction. So indeed we must have that $`g^{}f^{}`$. Finally suppose that $`g^{}=f^{}`$. The assumption that $`f(j)a,a+1`$ for $`j\{i_1,i_1+1,\mathrm{},n\}\{i_1,\mathrm{},i_r\}`$ means that we must have $`j_1i_1,\mathrm{},j_ri_r`$. Hence, $`fg`$. Since we started with the assumption that $`gf`$, we therefore have $`g=f`$ which completes the proof. ∎

Twisting with $`\omega `$ using Lemma 2.9(i), we also have the analogous statement for $`F_a^{(r)}`$:

###### Lemma 2.11.

Suppose that $`f^{m|n}`$ and $`mi_r<\mathrm{}<i_11`$ are such that $`f(i_1)=\mathrm{}=f(i_r)=a+1`$ and $`f(j)a,a+1`$ for all $`j\{m,1m,\mathrm{},i_1\}\{i_1,\mathrm{},i_r\}`$. Let $`f^{}`$ be the function with $`f^{}(i_1)=\mathrm{}=f^{}(i_r)=a`$ and $`f^{}(j)=f(j)`$ for all $`ji_1,\mathrm{},i_r`$. Then, for any $`gf`$,

$$F_a^{(r)}M_g\delta _{f,g}M_f^{}+\underset{g^{}f^{}}{}[q,q^1]M_g^{}.$$

###### Theorem 2.12.

We can write each $`M_f`$ as a (possibly infinite) $`[q,q^1]`$-linear combination of terms of the form $`F_{a_1}^{(r_1)}\mathrm{}F_{a_s}^{(r_s)}M_g`$ for $`a_1,\mathrm{},a_s,r_1,\mathrm{},r_s1`$ and typical $`g^{m|n}`$.

###### Proof.

To prove the theorem, it suffices to show for each $`d`$ and $`f^{m|n}`$ that we can write $`M_f`$ as a finite linear combination of terms of the form $`F_{a_1}^{(r_1)}\mathrm{}F_{a_s}^{(r_s)}M_g`$ for typical $`g^{m|n}`$ modulo $`\mathrm{ker}\pi _d`$. So fix $`d`$ and $`f^{m|n}`$. There are only finitely many $`gf`$ with $`\pi _dM_g0`$, So proceeding by induction on the dominance ordering, we may assume that every $`M_g`$ with $`gf`$ can be expressed as a finite linear combination of terms of the form $`F_{a_1}^{(r_1)}\mathrm{}F_{a_s}^{(r_s)}M_g`$ for typical $`g^{m|n}`$ modulo $`\mathrm{ker}\pi _d`$.

Let $`\{a_1<a_2<\mathrm{}<a_s\}=\{f(m),\mathrm{},f(1)\}`$ and let $`r_t=\mathrm{\#}\{iI(m|0)|f(i)=a_t\}`$ for each $`t=1,\mathrm{},s`$. Choose $`k0`$ so that every element of the set $`\{f(m)+k,\mathrm{},f(1)+k\}`$ exceeds every element of the set $`\{f(1),\mathrm{},f(n)\}`$. Define $`g^{m|n}`$ by

$$g(i)=\{\begin{array}{cc}f(i)\hfill & \text{if }i>0\text{,}\hfill \\ f(i)+k\hfill & \text{if }i<0\text{.}\hfill \end{array}$$

Note $`g`$ is typical by the choice of $`k`$. Now consider

$$F_{a_s}^{(r_s)}\mathrm{}F_{a_s+k1}^{(r_s)}F_{a_{s1}}^{(r_{s1})}\mathrm{}F_{a_{s1}+k1}^{(r_{s1})}\mathrm{}F_{a_1}^{(r_1)}\mathrm{}F_{a_1+k1}^{(r_1)}M_g.$$

One checks using Lemma 2.11 that this equals $`M_f`$ plus a $`[q,q^1]`$-linear combination of $`M_h`$’s with $`hf`$. So we are done by the induction hypothesis. ∎

###### Corollary 2.13.

Suppose $`\theta :\widehat{𝒯}^{m|n}\widehat{𝒯}^{m|n}`$ is a continuous $`𝒰,_{m|n}`$-bimodule endomorphism fixing $`M_f`$ for all typical antidominant $`f^{m|n}`$. Then $`\theta `$ is the identity map.

###### Proof.

If $`f`$ is antidominant, then $`M_fH_x=M_{fx}`$ for all $`xS_{m|n}`$. So for typical antidominant $`f`$ we have that $`\theta (M_{fx})=\theta (M_fH_x)=\theta (M_f)H_x=M_fH_x=M_{fx}`$. This shows that $`\theta `$ fixes $`M_g`$ for all typical $`g^{m|n}`$. Now using the continuity of $`\theta `$ and Theorem 2.12, we get that $`\theta `$ fixes all $`M_f`$. Hence by continuity again, $`\theta `$ is the identity map. ∎

§2-h. Canonical bases. We now follow ideas of Lusztig \[L, ch. 27\] to define a canonical topological basis of $`\widehat{𝒯}^{m|n}`$. We should note that in loc. cit., Lusztig only considers finite dimensional quantum groups, but the techniques generalize to our situation on passing to the completion. The first step in the construction is to introduce a bar involution on the space $`\widehat{𝒯}^{m|n}`$ that is compatible with the bar involutions on $`𝒰`$ and on $`_{m|n}`$. The definition of this in Lusztig’s work involves the quasi-$`R`$-matrix associated to $`𝒰`$. One gets from \[L, $`\mathrm{\S }`$27.3\] a bar involution $`:\widehat{𝒯}^{m|n}\widehat{𝒯}^{m|n}`$ that satisfies property (iv), hence (i), in the theorem below and that is compatible with the bar involution on $`𝒰`$. One then checks easily using Lusztig’s definition that it is also compatible with the bar involution on $`_{m|n}`$, giving the existence half of the proof of the theorem. We will sketch a direct construction of the bar involution on $`\widehat{𝒯}^{m|n}`$ below, independent of Lusztig’s work.

###### Theorem 2.14.

There exists a unique continuous, antilinear map $`:\widehat{𝒯}^{m|n}\widehat{𝒯}^{m|n}`$ such that

* $`\overline{M_f}=M_f`$ for all typical antidominant $`f^{m|n}`$;
* $`\overline{XuH}=\overline{X}\overline{u}\overline{H}`$ for all $`X𝒰,u\widehat{𝒯}^{m|n}`$, $`H_{m|n}`$.

Moreover,

* bar is an involution;
* $`\overline{M_f}=M_f+()`$ where $`()`$ is a (possibly infinite) $`[q,q^1]`$-linear combination of $`M_g`$’s for $`gf`$;
* $`\overline{\omega (u)}=\omega (\overline{u})`$ for all $`u\widehat{𝒯}^{m|n}`$.

###### Proof.

Let us first explain how to prove the uniqueness, and also the fact that bar is necessarily an involution. Suppose that we are given two continuous antilinear maps $`,:\widehat{𝒯}^{m|n}\widehat{𝒯}^{m|n}`$ satisfying properties (i) and (ii). Then we can consider the composite map

$$\phi :\widehat{𝒯}^{m|n}\widehat{𝒯}^{m|n},u\stackrel{~}{\overline{u}}.$$

This is a continuous $`𝒰,_{m|n}`$-bimodule endomorphism of $`\widehat{𝒯}^{m|n}`$ fixing $`M_f`$ for all typical, antidominant $`f`$. Hence, $`\phi `$ is the identity map by Corollary 2.13. In particular, this gives that $`\overline{\overline{u}}=u`$ and $`\stackrel{~}{\stackrel{~}{u}}=u`$ for each $`u\widehat{𝒯}^{m|n}`$. Finally, applying $``$ to both sides of the equation $`\stackrel{~}{\overline{u}}=u`$ gives that $`\overline{u}=\stackrel{~}{u}`$ for all $`u`$, whence uniqueness.

To get existence without appealing directly to Lusztig’s work, we need a little more notation. For a partition $`\lambda =(\lambda _1\lambda _2\mathrm{})`$, let $`|\lambda |`$ denote the sum of its parts, $`r(\lambda )`$ denote the total number of non-zero parts, and $`r_s(\lambda )`$ denote the total number of parts equal to $`s`$. Also let

$$p(\lambda )=(1q^2)^{r(\lambda )}(q)^{|\lambda |}\underset{s1}{}q^{r_s(\lambda )(r_s(\lambda )1)/2}[r_s(\lambda )]!.$$

Finally, for $`a`$ and a partition $`\lambda `$, let $`a_\lambda ^{m|n}^{m|n}`$ denote the function $`ia+\lambda _{|i|}`$. Let $`D_\lambda ^{m|n}`$ denote the set of minimal length $`\mathrm{stab}_{S_{m|n}}(a_\lambda ^{m|n})\backslash S_{m|n}`$-coset representatives. For example, if $`\lambda =(2,1,1)`$, then

$`M_{0_\lambda ^{3|2}}`$ $`=w_1w_1w_2v_2v_1,`$
$`D_\lambda ^{3|2}`$ $`=\{1,s_1,s_1s_2,s_1,s_1s_1,s_1s_2s_1\}.`$
Now define $`:𝒯^{m|n}\widehat{𝒯}^{m|n}`$ to be the unique antilinear map satisfying the following properties:

* For $`a`$, $`\overline{w_a^mv_a^n}={\displaystyle \underset{\stackrel{\lambda \text{ with}}{r(\lambda )m,n}}{}}p(\lambda )\left[{\displaystyle \underset{xD_\lambda ^{m|n}}{}}q^{\mathrm{}(x)}M_{a_\lambda ^{m|n}x}\right].`$
* If $`xS_{m|n}`$ is the unique element of minimal length such that $`fx`$ is antidominant then $`\overline{M_f}=\overline{M_{fx}}H_x^1`$.
* If all elements of $`\{f(m),\mathrm{},f(k1),f(l+1),\mathrm{},f(n)\}`$ are strictly greater than all elements of $`\{f(k),\mathrm{},f(1),f(1),\mathrm{},f(l)\}`$ for some $`1km,1ln`$, then

$$\overline{M_f}=\underset{\stackrel{gI\left(mk|nl\right)}{hI(k|l)}}{}a_gb_hw_{g(km)}\mathrm{}w_{g(1)}M_hv_{g(1)}\mathrm{}v_{g(nl)}$$

where

$`\overline{w_{f(m)}\mathrm{}w_{f(k1)}v_{f(l+1)}\mathrm{}v_{f(n)}}`$ $`={\displaystyle \underset{gI(mk|nl)}{}}a_gM_g,`$
$`\overline{w_{f(k)}\mathrm{}w_{f(1)}v_{f(1)}\mathrm{}v_{f(l)}}`$ $`={\displaystyle \underset{hI(k|l)}{}}b_hM_h.`$

The following property is a consequence of (B1), and is useful in inductive arguments. We omit the proof.

* For $`m,n>0`$ and $`a`$,

$$\begin{array}{c}\overline{w_a^mv_a^n}=w_a\overline{w_a^{(m1)}v_a^n}+\hfill \\ \hfill q^{mn}(qq^1)\underset{l1}{}(q)^{1l}w_{a+l}\overline{w_a^{(m1)}v_a^{(n1)}}v_{a+l}X\end{array}$$

where $`X=1+qH_{n1}^1+\mathrm{}+q^{n1}H_{n1}^1\mathrm{}H_1^1.`$

For example:

$`\overline{w_a^m}`$ $`=w_a^m,\overline{v_a^n}=v_a^n,`$
$`\overline{w_av_a}`$ $`=w_av_a+(qq^1){\displaystyle \underset{l1}{}}(q)^{1l}w_{a+l}v_{a+l}.`$
Now one checks easily that the map $`:𝒯^{m|n}\widehat{𝒯}^{m|n}`$ just defined satisfies (iv), hence (i), and (v). In particular, (iv) implies that bar is continuous, so it extends uniquely to a continuous antilinear map $`:\widehat{𝒯}^{m|n}\widehat{𝒯}^{m|n}`$. One finally needs to show that it satisfies (ii). This is done by a lengthy – but elementary – verification, using (B1), (B1), (B2) and (B3) directly. We omit the details. ∎

Now we appeal to the following general lemma originating in \[KL\], also used implicitly in \[L, $`\mathrm{\S }`$27.3\]. See \[D1, 1.2\] for a short proof.

###### Lemma 2.15.

Let $`(I,)`$ be a partially ordered set with the property that $`\{jI|ji\}`$ is finite for all $`iI`$. Suppose that $`M`$ is a $`(q)`$-vector space with basis $`\{u_i\}_{iI}`$ equipped with an antilinear involution $`:MM`$ such that $`\overline{u_i}=u_i+()`$ for each $`iI`$, where $`()`$ is a $`[q,q^1]`$-linear combination of $`u_j`$’s for $`ji`$. Then there exist unique bases $`\{x_i\}_{iI}`$, $`\{y_i\}_{iI}`$ for $`M`$ such that

* $`\overline{x_i}=x_i`$ and $`\overline{y_i}=y_i`$;
* $`x_iu_i+_{jI}q[q]u_j`$ and $`y_iu_i+_{jI}q^1[q^1]u_j,`$

for each $`iI`$. Moreover, the coefficient of $`u_j`$ in $`x_i`$ (resp. $`y_i`$) is zero unless $`ji`$.

Applying this to the space $`𝒯_d^{m|n}`$ for fixed $`d`$, the basis $`\{M_f\}_{f_d^{m|n}}`$ and the antilinear involution $`\pi _d:𝒯_d^{m|n}𝒯_d^{m|n}`$, we deduce:

###### Lemma 2.16.

There exist unique bases $`\{T_f^{(d)}\}_{f_d^{m|n}}`$, $`\{L_f^{(d)}\}_{f_d^{m|n}}`$ for $`𝒯_d^{m|n}`$ such that

* $`\pi _d(\overline{T_f^{(d)}})=T_f^{(d)}`$ and $`\pi _d(\overline{L_f^{(d)}})=L_f^{(d)}`$;
* $`T_f^{(d)}M_f+_{g_d^{m|n}}q[q]M_g`$ and $`L_f^{(d)}M_f+_{g_d^{m|n}}q^1[q^1]M_g`$.

Moreover, the coefficient of $`M_g`$ in $`T_f^{(d)}`$ (resp. $`L_f^{(d)}`$) is zero unless $`gf`$.

Passing to the completion gives us bases for $`\widehat{𝒯}^{m|n}`$:

###### Theorem 2.17.

There exist unique topological bases $`\{T_f\}_{f^{m|n}}`$, $`\{L_f\}_{f^{m|n}}`$ for $`\widehat{𝒯}^{m|n}`$ such that

* $`\overline{T_f}=T_f`$ and $`\overline{L_f}=L_f`$;
* $`T_fM_f+\widehat{}_{g^{m|n}}q[q]M_g`$ and $`L_fM_f+\widehat{}_{g^{m|n}}q^1[q^1]M_g`$.

Moreover,

* $`T_f=M_f+()`$ and $`L_f=M_f+()`$ where $`()`$ and $`()`$ are (possibly infinite) linear combinations of $`M_g`$’s for $`gf`$;
* $`\omega (T_f)=T_{\omega (f)}`$ and $`\omega (L_f)=L_{\omega (f)}`$.

###### Proof.

Take $`ed`$ and $`f_d^{m|n}`$. Consider the elements $`T_f^{(e)}𝒯_e^{m|n}`$ and $`T_f^{(d)}𝒯_d^{m|n}`$ given by Lemma 2.16. We know that if $`\pi _d(u)=0`$ then $`\pi _d(\overline{u})=0`$. Hence we have that $`\pi _d(\overline{\pi _d(u)u})=0`$ for all $`u\widehat{𝒯}^{m|n}`$. Applying this to $`u=T_f^{(e)}`$ we deduce that $`\pi _d(\overline{\pi _d(T_f^{(e)})})=\pi _d(T_f^{(e)}).`$ Hence by the uniqueness in Lemma 2.16, we have that $`\pi _d(T_f^{(e)})=T_f^{(d)}.`$ Similarly, $`\pi _d(L_f^{(e)})=L_f^{(d)}.`$ Hence, for all $`f^{m|n}`$ there exist unique elements $`T_f,L_f\widehat{𝒯}^{m|n}`$ such that

$$\pi _d(T_f)=T_f^{(d)},\pi _d(L_f)=L_f^{(d)}$$

for all $`d`$ and all $`f_d^{m|n}`$. Using the lemma for each $`d`$, one now easily checks that these satisfy (i)–(iii), while (iv) follows from Theorem 2.14(v). ∎

We call the basis $`\{T_f\}_{f^{m|n}}`$ the canonical basis of $`\widehat{𝒯}^{m|n}`$, and $`\{L_f\}_{f^{m|n}}`$ is the dual canonical basis. Let us introduce notation for the coefficients: let

$$T_f=\underset{g^{m|n}}{}t_{g,f}(q)M_g,L_f=\underset{g^{m|n}}{}l_{g,f}(q)M_g$$
(2.18)
for polynomials $`t_{g,f}(q)[q]`$ and $`l_{g,f}(q)[q^1]`$. We know that $`t_{g,f}(q)=l_{g,f}(q)=0`$ unless $`gf`$, and that $`t_{f,f}(q)=l_{f,f}(q)=1`$.

###### Example 2.19.

If $`m=n=1`$, the bases $`\{T_f\}_{f^{m|n}}`$ and $`\{L_f\}_{f^{m|n}}`$ are

$`\{w_av_b|a,b,ab\}`$ $`\{w_av_a+qw_{a+1}v_{a+1}|a\},`$
$`\{w_av_b|a,b,ab\}`$ $`\{w_av_aq^1w_{a+1}v_{a+1}`$
$`+q^2w_{a+2}v_{a+2}\mathrm{}|a\}`$
respectively.

§2-i. Duality. We wish next to explain the relationship between the bases $`\{T_f\}_{f^{m|n}}`$ and $`\{L_f\}_{f^{m|n}}`$. Recall the definitions (2.7) and (2.8). Define a new bilinear form $`.,.__𝒯`$ on $`\widehat{𝒯}^{m|n}`$ by

$$u,v__𝒯=(u,\sigma (\overline{v}))__𝒯$$
(2.20)
for $`u,v\widehat{𝒯}^{m|n}`$. Note this makes sense, even though the expression $`\sigma (\overline{v})`$ may not make sense in its own right. Indeed, it is clear that on expanding $`u`$ and $`\overline{v}`$ in terms of the basis $`\{M_f\}`$, there are only finitely many $`f`$ such that $`M_f`$ is involved in $`u`$ and $`M_f`$ is involved in $`\overline{v}`$. So we can interpret $`(u,\sigma (\overline{v}))__𝒯`$ as

$$\underset{f^{m|n}}{}(u,M_f)__𝒯\overline{(M_f,\overline{v})__𝒯},$$

all but finitely many terms in the sum being zero.

###### Lemma 2.21.

$`XuH,v__𝒯=u,\sigma (X)v\tau (H)__𝒯`$ for all $`X𝒰,H_{m|n}`$ and $`u,v\widehat{𝒯}^{m|n}`$.

###### Proof.

According to Lemma 2.9, we have that

$`XuH,v__𝒯`$ $`=(XuH,\sigma (\overline{v}))__𝒯=(u,\tau (X)\sigma (\overline{v})\tau (H))__𝒯`$
$`=(u,\tau (\overline{\sigma \overline{(\sigma (X))}})\sigma (\overline{v})\tau (H))__𝒯=(u,\sigma (\overline{\sigma (X)}\overline{v}\overline{\tau (H)}))__𝒯`$
$`=(u,\sigma (\overline{\sigma (X)v\tau (H)}))__𝒯=u,\sigma (X)v\tau (H)__𝒯.`$
###### Lemma 2.22.

The bilinear form $`.,.__𝒯`$ is symmetric.

###### Proof.

Let us first show that $`u,M_f__𝒯=M_f,u__𝒯`$ for all $`u\widehat{𝒯}^{m|n}`$ and typical antidominant $`f^{m|n}`$. We need to show that $`(u,M_f)__𝒯=\overline{(\overline{u},M_f)__𝒯},`$ for which it suffices to consider the special case $`u=M_g`$. Then, $`(M_g,M_f)__𝒯=\delta _{g,f}`$. Consider $`(\overline{M_g},M_f)__𝒯`$. By Theorem 2.14(iv), it is zero unless $`\mathrm{wt}(g)=\mathrm{wt}(f)`$. So since $`f`$ is typical, $`g`$ must be too. By Theorem 2.14(iv),

$$\overline{M_g}=M_g+(\text{a linear combination of }M_h\text{’s with }hg).$$

Since $`f`$ is antidominant, we deduce that $`(\overline{M_g},M_f)__𝒯=(M_g,M_f)__𝒯=\delta _{g,f}`$.

Now we show that $`u,M_f__𝒯=M_f,u__𝒯`$ for all $`u\widehat{𝒯}^{m|n}`$ and all typical $`f^{m|n}`$. Let $`xS_{m|n}`$ be of minimal length such that $`fx`$ is antidominant. Then, by the previous paragraph and Lemma 2.21, we have that

$`u,M_f__𝒯`$ $`=u,M_{fx}T_{x^1}__𝒯=uT_x,M_{fx}__𝒯`$
$`=M_{fx},uT_x__𝒯=M_{fx}T_{x^1},u__𝒯=M_f,u__𝒯,`$
as required.

Now let us consider the general case. In view of Theorem 2.12, we may assume that $`v=XM_g`$ for typical $`g`$ and $`X=F_{a_1}^{(r_1)}\mathrm{}F_{a_s}^{(r_s)}𝒰`$. Then, by the previous paragraph and Lemma 2.21, we have that

$`u,v__𝒯`$ $`=u,XM_g__𝒯=\sigma (X)u,M_g__𝒯`$
$`=M_g,\sigma (X)u__𝒯=XM_g,u__𝒯=v,u__𝒯.`$
This completes the proof. ∎

The following theorem characterizes $`\{L_f\}_{f^{m|n}}`$ uniquely as the dual basis to $`\{T_g\}_{g^{m|n}}`$ under the bilinear form $`.,.__𝒯`$.

###### Theorem 2.23.

For $`f,g^{m|n}`$, $`L_f,T_g__𝒯=\delta _{f,g}`$.

###### Proof.

Consider $`L_f,T_g__𝒯=(L_f,\sigma (T_g))__𝒯.`$ We observe that $`L_f`$ is equal to $`M_f`$ plus a $`q^1[q^1]`$-linear combination of $`M_h`$’s with $`hf`$. Also $`\sigma (T_g)`$ equals $`M_g`$ plus a $`q^1[q^1]`$-linear combination of $`M_h`$’s with $`hg`$. Hence, $`L_f,T_g__𝒯`$ is zero unless $`fg`$, it is $`1`$ if $`f=g`$ and it is in $`q^1[q^1]`$ if $`fg`$.

On the other hand, by Lemma 2.22, $`L_f,T_g__𝒯=(\sigma (L_f),T_g)__𝒯`$. Hence, arguing as in the previous paragraph, $`L_f,T_g__𝒯`$ is zero unless $`fg`$, it is $`1`$ if $`f=g`$ and it is in $`q[q]`$ if $`fg`$. Since $`q[q]q^1[q^1]=\{0\}`$, this completes the proof. ∎

###### Corollary 2.24.

For $`f^{m|n}`$,

$$M_f=\underset{g^{m|n}}{}t_{f,g}(q^1)L_g=\underset{g^{m|n}}{}l_{f,g}(q^1)T_g.$$

###### Proof.

By the theorem, we can write $`M_f=_{g^{m|n}}M_f,T_g__𝒯L_g`$. Now a computation from the definition (2.20) of the form $`.,.__𝒯`$ gives that $`M_f,T_g__𝒯=t_{f,g}(q^1)`$. The second equality is proved similarly. ∎

§2-j. An algorithm. The goal in this subsection is to explain an algorithm to compute $`T_f^{(d)}`$ (cf. Lemma 2.16) for each $`d`$ and $`f_d^{m|n}`$. Assuming a certain positivity conjecture which ensures that the $`T_f^{(d)}`$ converge to $`T_f`$ in finitely many steps, the algorithm can be modified to actually compute the canonical basis elements $`T_f`$ themselves.

The algorithm proceeds by induction on the degree of atypicality $`\mathrm{\#}f`$ of $`f`$. To begin with, we describe the base of the induction by explaining how to compute $`T_f`$ (hence all $`T_f^{(d)}`$) for typical $`f`$. If $`f`$ is typical and antidominant, then we have that $`T_f=M_f`$ and we are done. Otherwise, we can find $`iI(m1|n1)`$ such that $`fs_if`$. We may assume by induction that $`T_{fs_i}`$ is already known, and consider the bar invariant element $`T_{fs_i}(H_i+q),`$ which we view as a first approximation to $`T_f`$. It equals $`M_f`$ plus a sum of terms $`p_{g,f}(q)M_g`$ for polynomials $`p_{g,f}(q)[q]`$ and $`g`$ with $`gf`$ (there being only finitely many such $`g`$’s since $`f`$ is typical). For each such $`g`$ with $`p_{g,f}(0)0`$, we make a correction by subtracting $`p_{g,f}(0)T_g`$ from our first approximation. The result is a bar invariant expression that equals $`M_f`$ plus a $`q[q]`$-linear combination of $`M_g`$’s. This must be $`T_f`$ by the uniqueness in Theorem 2.17.

We have just described the usual algorithm to compute the parabolic Kazhdan-Lusztig polynomials associated to the Hecke algebra $`_{m|n}`$ \[KL, Deo\]. To make this precise, let us recall the definition of the latter, following \[So1\]. Let $`f^{m|n}`$ be antidominant. Then, $`S_f:=\mathrm{stab}_{S_{m|n}}(f)`$ is a parabolic subgroup of $`S_{m|n}`$. Let $`_f`$ be the corresponding parabolic subalgebra of $`_{m|n}`$. Let $`\mathrm{𝟏}__f`$ denote the one dimensional right $`_f`$-module on basis $`1`$ with action $`1H_i=q^11`$ for each $`H_i_f`$. We consider the induced module $`\mathrm{𝟏}__f__f_{m|n}.`$ This has a basis given by the elements $`M_x^{(f)}:=1H_x`$ as $`x`$ runs over the set $`D_f`$ of minimal length $`S_f\backslash S_{m|n}`$-coset representatives. The bar involution on $`\mathrm{𝟏}__f__f_{m|n}`$ is the antilinear map defined by $`\overline{1H_x}:=1\overline{H_x}.`$ The Kazhdan-Lusztig basis of $`\mathrm{𝟏}__f__f_{m|n}`$ is the unique bar invariant basis $`\{C_x^{(f)}\}_{xD_f}`$ such that

$$C_x^{(f)}M_x^{(f)}+\underset{yD_f}{}q[q]M_y^{(f)}.$$

The corresponding parabolic Kazhdan-Lusztig polynomials are defined from

$$C_x^{(f)}=\underset{yD_f}{}m_{y,x}^{(f)}(q)M_y^{(f)}.$$

The relationship to our situation is as follows:

###### Lemma 2.25.

Suppose that $`f`$ is typical and antidominant. Then, for $`xD_f`$ and $`g^{m|n}`$,

$$t_{g,fx}(q)=\{\begin{array}{cc}m_{y,x}^{(f)}(q)\hfill & \text{if }g=fy\text{ for some }yD_f,\hfill \\ 0\hfill & \text{otherwise.}\hfill \end{array}$$

###### Proof.

Note that $`M_fH_i=q^1M_f`$ for all $`H_i_f`$. So we get from Frobenius reciprocity a unique $`_{m|n}`$-module homomorphism $`\theta :\mathrm{𝟏}__f__f_{m|n}𝒯^{m|n}`$ under which $`M_x^{(f)}`$ maps to $`M_{fx}`$. By (B2) from the proof of Theorem 2.14, $`\overline{M_{fx}}=M_f\overline{H_x}`$. So $`\theta (\overline{u})=\overline{\theta (u)}`$ for each $`u\mathrm{𝟏}__f__f_{m|n}`$. Therefore, $`\theta (C_x^{(f)})`$ is bar invariant, and it equals $`M_{fx}`$ plus a $`q[q]`$-linear combination of other $`M_g`$’s. Hence, $`\theta (C_x^{(f)})=T_{fx}`$ by the uniqueness. This shows that for each $`xD_f`$,

$$T_{fx}=\underset{yD_f}{}m_{y,x}^{(f)}(q)M_{fy}.$$

The lemma follows. ∎

###### Remark 2.26.

In the case $`m=0`$, this lemma shows that in type $`A`$ the parabolic Kazhdan-Lusztig polynomials coincide with the coefficients of the canonical basis of $`𝒱^n`$. This is a well-known observation, see for example \[FKK\].

Now we describe the algorithm to compute $`T_f^{(d)}`$ for atypical $`f`$. We assume therefore that we are given $`d`$ and $`f_d^{m|n}`$ with $`\mathrm{\#}f>0`$, and that we have already constructed an algorithm to compute $`T_g^{(e)}`$ for each $`e`$ and $`g`$ with $`\mathrm{\#}g<\mathrm{\#}f`$. Let us write $`f_{}=(f(m),\mathrm{},f(1))`$ and $`f_+=(f(1),\mathrm{},f(n))`$. Define $`a_1`$ to be the greatest integer that appears in both the tuples $`f_{}`$ and $`f_+`$. Now we iterate a certain bumping procedure:

Let $`n_{1,1}`$ be the number of entries equal to $`a_1`$ appearing in the tuple $`f_+`$, and label all such entries. If there are no entries equal to $`(a_1+1)`$ appearing to the right of labeled $`a_1`$’s, move on to the next paragraph. Otherwise, let $`n_{1,2}`$ be the number of entries equal to $`(a_1+1)`$ appearing to the right of labeled $`a_1`$’s, and label all such $`(a_1+1)`$’s. Next, if there are no $`(a_1+2)`$’s to the right of labeled $`(a_1+1)`$’s, move on to the next paragraph. Otherwise let $`n_{1,3}`$ be the number of $`(a_1+2)`$’s to the right of labeled $`(a_1+1)`$’s, and label all such $`(a_1+2)`$’s. Continue in this way.

When the process just described terminates, we are left with a sequence $`n_{1,1},n_{1,2},\mathrm{},n_{1,k_1}`$ for some $`k_11`$, where there are $`n_{1,i}`$ labeled $`(a_1+i1)`$’s in the tuple $`f_+`$. We define $`X_1:=E_{a_1+k_11}^{(n_{1,k_1})}\mathrm{}E_{a_1+1}^{(n_{1,2})}E_{a_1}^{(n_{1,1})}`$ and $`a_2:=a_1+k_1`$. If there are no entries equal to $`a_2`$ in the tuple $`f_{}`$, the bumping procedure is finished. Otherwise, we need to repeat the bumping procedure but applied to $`f_{}`$ instead, as follows. Let $`n_{2,1}`$ be the number of entries equal to $`a_2`$ appearing in the tuple $`f_{}`$, and label all such entries. If there are no entries equal to $`(a_2+1)`$ appearing to the left of labeled $`a_2`$’s, move on to the next paragraph. Otherwise, let $`n_{2,2}`$ be the number of entries equal to $`(a_2+1)`$ appearing to the left of labeled $`a_2`$’s, and label all such $`(a_2+1)`$’s. Continue in this way until the process terminates.

We are left with a sequence $`n_{2,1},n_{2,2},\mathrm{},n_{2,k_1}`$ for some $`k_21`$, where there are $`n_{2,i}`$ labeled $`(a_2+i1)`$’s in the tuple $`f_{}`$. Let $`X_2:=F_{a_2+k_21}^{(n_{2,k_2})}\mathrm{}F_{a_2+1}^{(n_{2,2})}F_{a_2}^{(n_{2,1})}`$ and $`a_3:=a_2+k_2`$. This time if there are no entries equal to $`a_3`$ in the tuple $`f_+`$, the bumping procedure is finished. Otherwise, we repeat the whole process once more from the beginning, but using $`a_3`$ in place of $`a_1`$, to construct $`X_3,a_4,X_4,\mathrm{}`$ and so on.

When the bumping procedure finally ends, we are left with a sequence of monomials $`X_1,\mathrm{},X_N`$ and integers $`a_1<a_2<\mathrm{}<a_{N+1}`$. Increase all labeled entries in the tuples $`f_{},f_+`$ by $`1`$ and let $`h^{m|n}`$ be the corresponding function. Note that $`\mathrm{\#}h<\mathrm{\#}f`$. So by induction, we can compute $`T_h^{(e)}`$, where $`e=\mathrm{max}(d,a_{N+1})`$. Now consider the bar invariant element $`\pi _d(X_N\mathrm{}X_1T_h^{(e)})𝒯_d^{m|n},`$ which is our first approximation to $`T_f^{(d)}`$. By Lemmas 2.10 and 2.11, it equals $`M_f`$ plus a finite linear combination of terms $`p_{g,f}(q)M_g`$ for polynomials $`p_{g,f}(q)[q,q^1]`$ and $`g_d^{m|n}`$ with $`gf`$. Now we make corrections to the first approximation. Let $`gf`$ be maximal such that $`p_{g,f}(q)q[q]`$. Let $`p_{g,f}^{}(q)`$ be the unique bar invariant element of $`[q,q^1]`$ such that $`p_{g,f}^{}(q)p_{g,f}(q)(modq[q])`$. Proceeding by induction on the ordering on $`_d^{m|n}`$, we may assume that $`T_g^{(d)}`$ is already known. Subtract $`p_{g,f}^{}(q)T_g^{(d)}`$ from the first approximation, to obtain a second approximation to $`T_f^{(d)}`$. Repeating the correction procedure, we reduce in finitely many steps to a bar invariant expression that equals $`M_f`$ plus a $`q[q]`$-linear combination of $`M_g`$’s. This must be $`T_f^{(d)}`$ by the uniqueness. We are done.

###### Example 2.27.

We explain how to compute $`T_{(0,4,1|0,2,3)}^{(4)}`$ using the algorithm. The bumping procedure proceeds as follows:

$`(0,4,1|0,2,3)`$ $`\stackrel{a_1=0}{}(0,4,1|\underset{¯}{0},2,3)\stackrel{a_2=1}{}(0,4,\underset{¯}{1}|\underset{¯}{0},2,3)`$
$`\stackrel{a_3=2}{}(0,4,\underset{¯}{1}|\underset{¯}{0},\underset{¯}{2},\underset{¯}{3})\stackrel{a_4=4}{}(0,\underset{¯}{4},\underset{¯}{1}|\underset{¯}{0},\underset{¯}{2},\underset{¯}{3}).`$
Now, $`(0,5,2|1,3,4)`$ is typical, so we can compute $`\pi _4(F_4E_3E_2F_1E_0T_{(0,5,2|1,3,4))}^{(5)})`$ using the Kazhdan-Lusztig algorithm. It turns out that this equals $`M_{(0,4,1|0,2,3)}+M_{(1,4,1|1,2,3)}`$ plus a $`q[q]`$-linear combination of lower terms. Now one computes $`T_{(1,4,1|1,2,3)}^{(4)}`$ needed for the correction procedure by repeating the algorithm (which is rather lengthy). Finally one obtains

$$\begin{array}{c}T_{(0,4,1|0,2,3)}^{(4)}=M_{(0,4,1|0,2,3)}+qM_{(1,4,0|0,2,3)}+qM_{(4,0,1|0,2,3)}\hfill \\ \hfill +q^2M_{(1,4,1|1,2,3)}+q^2M_{(4,1,0|0,2,3)}+q^3M_{(4,1,1|1,2,3)}.\end{array}$$

Note there is no reason why we chose to start the bumping procedure with $`f_+`$ in describing the algorithm. One could also start the bumping procedure with $`f_{}`$, increasing all entries in $`f_{}`$ equal to $`a_1`$ by $`1`$ and so on …. In practice, one should always choose to start with the side for which the resulting word $`X_N\mathrm{}X_1𝒰`$ is as short as possible. In the present example, it is better to start the bumping procedure with $`f_{}`$, since then there is only one step:

$$(0,4,1|0,2,3)\stackrel{a_1=0}{}(\underset{¯}{0},4,1|0,2,3).$$

Thus, we need to compute $`\pi _4(F_0T_{(1,4,1|0,2,3)}^{(4)})`$ instead, which is much quicker as only one generator of $`𝒰`$ needs to be applied. It turns out that this equals $`T_{(0,4,1|1,2,3)}^{(4)}`$ directly (indeed it already equals $`T_{(0,4,1|1,2,3)}`$), with no corrections needed.

Computer calculations using the above algorithm support the following positivity conjecture:

###### Conjecture 2.28.

Let $`f^{m|n}`$.

* The coefficients $`t_{g,f}(q)`$ of $`T_f`$ when expanded in the basis $`\{M_g\}_{g^{m|n}}`$ belong to $`[q]`$.
* The coefficients $`l_{g,f}(q)`$ of $`L_f`$ when expanded in the basis $`\{M_g\}_{g^{m|n}}`$ belong to $`[q^1]`$.
* For each $`a`$, $`f^{m|n}`$ and $`r1`$, the coefficients of $`E_a^{(r)}T_f`$ and $`F_a^{(r)}T_f`$ when expanded in the basis $`\{T_g\}_{g^{m|n}}`$ belong to $`[q,q^1]`$.
* For each $`a`$, $`f^{m|n}`$ and $`r1`$, the coefficients of $`E_a^{(r)}L_f`$ and $`F_a^{(r)}L_f`$ when expanded in the basis $`\{L_g\}_{g^{m|n}}`$ belong to $`[q,q^1]`$.

If this positivity conjecture is true, it follows in particular that each $`T_f`$ belongs to $`𝒯^{m|n}`$ rather than the completion $`\widehat{𝒯}^{m|n}`$, i.e. each $`T_f`$ is a finite linear combination of $`M_g`$’s. To see this, we modify the above algorithm to obtain an algorithm that computes $`T_f`$ itself (not just the $`T_f^{(d)}`$’s) in finitely many steps, as follows. To start with, one follows the bumping procedure to obtain $`h`$ and the elements $`X_1,\mathrm{},X_N𝒰`$ exactly as above. Since $`\mathrm{\#}h<\mathrm{\#}f`$, we may assume that $`T_h`$ is known inductively and is a finite sum of $`M_g`$’s. Consider $`X_N\mathrm{}X_1T_h𝒯^{m|n}`$, and choose $`e`$ to be minimal so that $`X_N\mathrm{}X_1T_h𝒯_e^{m|n}`$. In view of Conjecture 2.28(iii), $`X_N\mathrm{}X_1T_h`$ equals $`T_f`$ plus a $`[q,q^1]`$-linear combination of $`T_g`$’s. So by Conjecture 2.28(i), we must have that that $`T_f𝒯_e^{m|n}`$, hence $`T_f=T_f^{(e)}`$. Now follow the above algorithm to compute $`T_f^{(e)}`$.

§2-k. Crystal structures. Finally in this section, we review some results of Kashiwara, see e.g. \[K2\] for the basic language used here. Let $`𝒜`$ be the subring of $`(q)`$ consisting of rational functions having no pole at $`q=0`$. Evaluation at $`q=0`$ induces an isomorphism $`𝒜/q𝒜`$.

Let $`𝒱_𝒜`$ be the $`𝒜`$-lattice in $`𝒱`$ spanned by the $`v_a`$’s, and let $`𝒲_𝒜`$ be the $`𝒜`$-lattice in $`𝒲`$ spanned by the $`w_a`$’s. Then, $`𝒱_𝒜`$ together with the basis of the $``$-vector space $`𝒱_𝒜/q𝒱_𝒜`$ given by the images of the $`v_a`$’s is a lower crystal basis for $`𝒱`$ at $`q=0`$ in the sense of \[K2, 4.1\]. Similarly, $`𝒲_𝒜`$ together with the basis for $`𝒲_𝒜/q𝒲_𝒜`$ given by the images of the $`w_a`$’s is a lower crystal basis for $`𝒲`$ at $`q=0`$. Let $`𝒯_𝒜^{m|n}=𝒲_𝒜^m_𝒜𝒱_𝒜^n`$ be the $`𝒜`$-lattice in $`𝒯^{m|n}`$ spanned by the $`M_f`$’s. Then, by \[K2, Theorem 4.1\], $`𝒯_𝒜^{m|n}`$ together with the basis for $`𝒯_𝒜^{m|n}/q𝒯_𝒜^{m|n}`$ given by the images of the $`M_f`$ for $`f^{m|n}`$ is a lower crystal basis for $`𝒯^{m|n}`$ at $`q=0`$. Moreover, we can easily describe the associated crystal graph using Kashiwara’s tensor product rule.

To do this, let us identify the set $`\{M_f+q𝒯_𝒜^{m|n}\}_{f^{m|n}}`$ underlying the crystal basis with the set $`^{m|n}`$ in the obvious way. Then, Kashiwara’s crystal operators induce maps $`\stackrel{~}{E}_a^{},\stackrel{~}{F}_a^{}:^{m|n}^{m|n}\{\mathrm{}\}`$. (We are using $`\stackrel{~}{E}_a^{},\stackrel{~}{F}_a^{}`$ because $`\stackrel{~}{E}_a,\stackrel{~}{F}_a`$ are used for something else later on.) Fix $`a`$ and $`f^{m|n}`$. The $`a`$-signature $`(\sigma _m,\mathrm{},\sigma _1,\sigma _1,\mathrm{},\sigma _n)`$ of $`f`$ is defined by

$$\sigma _i=\{\begin{array}{cc}+\hfill & \text{if }i>0\text{ and }f(i)=a\text{, or if }i<0\text{ and }f(i)=a+1\text{,}\hfill \\ \hfill & \text{if }i>0\text{ and }f(i)=a+1\text{, or if }i<0\text{ and }f(i)=a\text{,}\hfill \\ 0\hfill & \text{otherwise.}\hfill \end{array}$$
(2.29)
From this, we form the reduced $`a`$-signature by successively replacing subsequences of the form $`+`$ (possibly separated by $`0`$’s) in the signature with $`00`$ until no $``$ appears to the right of a $`+`$. Recall the definition of $`d_j^{m|n}`$ from 2. We define

$$\stackrel{~}{E}_a^{}(f)=\{\begin{array}{cc}\mathrm{}\hfill & \text{if there are no }\text{’s in the reduced }a\text{-signature,}\hfill \\ fd_j\hfill & \text{if the rightmost }\text{ is in position }jI(m|n)\text{,}\hfill \end{array}$$

and

$$\stackrel{~}{F}_a^{}(f)=\{\begin{array}{cc}\mathrm{}\hfill & \text{if there are no }+\text{’s in the reduced }a\text{-signature,}\hfill \\ f+d_j\hfill & \text{if the leftmost }+\text{ is in position }jI(m|n)\text{.}\hfill \end{array}$$

Also let

$`\epsilon _a^{}(f)`$ $`=\mathrm{max}\{r0|(\stackrel{~}{E}_a^{})^r(f)0\}`$
$`=\text{the total number of }\text{’s in the reduced }a\text{-signature},`$
$`\phi _a^{}(f)`$ $`=\mathrm{max}\{r0|(\stackrel{~}{F}_a^{})^r(f)0\}`$
$`=\text{the total number of }+\text{’s in the reduced }a\text{-signature}.`$
Then, the datum $`(^{m|n},\stackrel{~}{E}_a^{},\stackrel{~}{F}_a^{},\epsilon _a^{},\phi _a^{},\mathrm{wt})`$ is the crystal associated to the module $`𝒯^{m|n}`$.

###### Example 2.30.

Consider the function $`f=(3,6,2,0,2,1|3,2,1)^{6|3}`$. The $`2`$-signature is $`(+,0,,0,,0|,+,0)`$. Cancelling off $`+`$ pairs, we deduce that the reduced $`2`$-signature is $`(0,0,0,0,,0|,+,0)`$. Hence, the $`2`$-string through $`f`$ in the crystal graph is

$`(3,6,2,0,3,1|2,2,1)\stackrel{\stackrel{~}{F}_2^{}}{}(3,6,2,0,2,1|2,2,1)`$ $`\stackrel{\stackrel{~}{F}_2^{}}{}(3,6,2,0,2,1|3,2,1)`$
$`\stackrel{\stackrel{~}{F}_2^{}}{}(3,6,2,0,2,1|3,3,1).`$
###### Theorem 2.31.

Let $`f^{m|n}`$ and $`a`$.

* $`E_aT_f=[\phi _a^{}(f)+1]T_{\stackrel{~}{E}_a^{}(f)}+\widehat{}_{g^{m|n}}u_{g,f}^aT_g`$ where the coefficient $`u_{g,f}^a`$ belongs to $`q^{2\phi _a^{}(g)}[q]`$ and is zero unless $`\epsilon _b^{}(g)\epsilon _b^{}(f)`$ for all $`b`$.
* $`F_aT_f=[\epsilon _a^{}(f)+1]T_{\stackrel{~}{F}_a^{}(f)}+\widehat{}_{g^{m|n}}v_{g,f}^aT_g`$ where the coefficient $`v_{g,f}^a`$ belongs to $`q^{2\epsilon _a^{}(g)}[q]`$ and is zero unless $`\phi _b^{}(g)\phi _b^{}(f)`$ for all $`b`$.

(In (i) resp. (ii), the first term on the right hand side should be omitted if $`\stackrel{~}{E}_a^{}(f)`$ resp. $`\stackrel{~}{F}_a^{}(f)`$ equals $`\mathrm{}`$.)

###### Proof.

Fix $`d`$, and consider $`𝒯=𝒯_d^{m|n}`$, which is an integrable module in the sense of \[K1, 1.3\] with respect to the subalgebra of $`𝒰`$ generated by all $`E_a,F_a,K_b^{\pm 1}`$ for $`a<d,bd`$. Let $`𝒯_0`$ (resp. $`𝒯_{}`$) be the $`𝒜`$ (resp. $`[q,q^1]`$) -lattice in $`𝒯`$ spanned by the basis elements $`\{M_f\}_{f_d^{m|n}}`$. Let $`𝒯_{\mathrm{}}=\pi _d(\overline{𝒯_0})`$, an $`\overline{𝒜}`$-lattice in $`𝒯`$. The canonical map $`𝒯_{}𝒯_0𝒯_{\mathrm{}}𝒯_0/q𝒯_0`$ is an isomorphism; this follows at once from Lemma 2.16 since that shows that all three lattices are generated by the elements $`\{T_f^{(d)}\}_{f_d^{m|n}}`$. The preimage of the crystal basis element $`M_f+q𝒯_0`$ is $`T_f^{(d)}`$, for each $`f_d^{m|n}`$. In the language of Kashiwara \[K1\], this shows that $`(𝒯_{},𝒯_0,𝒯_{\mathrm{}})`$ is a balanced triple, and that $`\{T_f^{(d)}\}_{f_d^{m|n}}`$ is a lower global crystal basis for $`𝒯`$ at $`q=0`$.

Now we get from \[K1, Proposition 5.3.1\] (which is about upper global crystal bases) and an argument involving duality \[K1, $`\mathrm{\S }`$3.2\], that

$$E_aT_f^{(d)}=[\phi _a^{}(f)+1]T_{\stackrel{~}{E}_a^{}(f)}^{(d)}+\underset{g_d^{m|n}}{}u_{g,f}^aT_g^{(d)}$$

where the coefficient $`u_{g,f}^a`$ belongs to $`q^{2\phi _a^{}(g)}[q]`$ and is zero unless $`\epsilon _b^{}(g)\epsilon _b^{}(f)`$ for all $`b<d`$. Taking the limit as $`d\mathrm{}`$, we get (i). The proof of (ii) is similar. ∎

We will also meet certain dual crystal operators on $`^{m|n}`$. Define

$`\stackrel{~}{E}_a^{}(f):=\stackrel{~}{F}_{1a}^{}(f),`$ $`\stackrel{~}{F}_a^{}(f):=\stackrel{~}{E}_{1a}^{}(f),`$ (2.32)
$`\epsilon _a^{}(f):=\phi _{1a}^{}(f),`$ $`\phi _a^{}(f):=\epsilon _{1a}^{}(f).`$ (2.33)
These can be described explicitly in a similar way to the above: for fixed $`a`$ and $`f^{m|n}`$, let $`(\sigma _m,\mathrm{},\sigma _1,\sigma _1,\mathrm{},\sigma _n)`$ be the $`a`$-signature as defined in (2.29). Form the dual reduced $`a`$-signature by successively replacing sequences of the form $`+`$ (possibly separated by $`0`$’s) with $`00`$ until no $``$ appears to the left of a $`+`$. Then:

$$\stackrel{~}{E}_a^{}(f)=\{\begin{array}{cc}0\hfill & \text{if there are no }\text{’s in the dual reduced }a\text{-signature,}\hfill \\ fd_j\hfill & \text{if the leftmost }\text{ is in position }jI(m|n)\text{,}\hfill \end{array}$$

and

$$\stackrel{~}{F}_a^{}(f)=\{\begin{array}{cc}0\hfill & \text{if there are no }+\text{’s in the dual reduced }a\text{-signature,}\hfill \\ f+d_j\hfill & \text{if the rightmost }+\text{ is in position }jI(m|n)\text{.}\hfill \end{array}$$

Also

$`\epsilon _a^{}(f)`$ $`=\text{the total number of }\text{’s in the dual reduced }a\text{-signature},`$
$`\phi _a^{}(f)`$ $`=\text{the total number of }+\text{’s in the dual reduced }a\text{-signature}.`$
In this way, we obtain the dual crystal structure $`(^{m|n},\stackrel{~}{E}_a^{},\stackrel{~}{F}_a^{},\epsilon _a^{},\phi _a^{},\mathrm{wt})`$ on the underlying set $`^{m|n}`$.

###### Theorem 2.34.

Let $`f^{m|n}`$ and $`a`$.

* $`E_aL_f=[\epsilon _a^{}(f)]L_{\stackrel{~}{E}_a^{}(f)}+\widehat{}_{g^{m|n}}w_{g,f}^aL_g`$ where the coefficient $`w_{g,f}^a`$ belongs to $`q^{2\epsilon _a^{}(f)}[q]`$ and is zero unless $`\phi _b^{}(g)\phi _b^{}(f)`$ for all $`b`$.
* $`F_aL_f=[\phi _a^{}(f)]L_{\stackrel{~}{F}_a^{}(f)}+\widehat{}_{g^{m|n}}x_{g,f}^aL_g`$ where the coefficient $`x_{g,f}^a`$ belongs to $`q^{2\phi _a^{}(f)}[q]`$ and is zero unless $`\epsilon _b^{}(g)\epsilon _b^{}(f)`$ for all $`b`$.

###### Proof.

Dualize Theorem 2.31 using Theorem 2.23 and Lemma 2.21. ∎

## 3. Exterior algebra

Now we descend from the tensor space $`𝒯^{m|n}`$ to $`^{m|n}`$. We continue with the same notation as in section 2.

§3-a. The space $`^{m\mathbf{|}n}`$. Let $`w_0`$ denote the longest element of $`S_{m|n}`$. Let

$$H_0:=\underset{xS_{m|n}}{}(q)^{\mathrm{}(x)\mathrm{}(w_0)}H_x_{m|n}.$$
(3.1)
The first lemma summarizes some elementary properties.

###### Lemma 3.2.

The following properties hold:

* $`H_iH_0=qH_0=H_0H_i`$ for any $`iI(m1|n1)`$;
* $`\overline{H_0}=H_0`$;
* $`H_0^2=[m]![n]!H_0`$;
* $`H_0=\tau (H_0)`$;
* the map $`\omega :_{m|n}_{n|m}`$ maps $`H_0_{m|n}`$ to $`H_0_{n|m}`$.

###### Proof.

Part (i) is an easy exercise. For (ii), use \[So1, Proposition 2.9\] and apply the map $`dia`$ there. For (iii), one gets at once using (i) that

$$H_0^2=\underset{xS_{m|n}}{}(q)^{\mathrm{}(x)\mathrm{}(w_0)}H_xH_0=\underset{xS_{m|n}}{}(q)^{2\mathrm{}(x)\mathrm{}(w_0)}H_0.$$

Now use the well-known formula for the Poincaré polynomial of $`S_{m|n}`$ to rewrite the sum. Finally, (iv) and (v) are obvious. ∎

Let $`^{m|n}:=𝒯^{m|n}H_0,`$ a $`𝒰`$-submodule of $`𝒯^{m|n}`$. Note $`^{m|n}`$ is the $`q`$-analogue of the exterior power $`^m𝒲^n𝒱.`$ Form the completion $`\widehat{}^{m|n}=\widehat{𝒯}^{m|n}H_0`$ as in 2. By Lemmas 2.9(i) and 3.2(v), the restriction of the map $`\omega :\widehat{𝒯}^{m|n}\widehat{𝒯}^{n|m}`$ is an isomorphism $`\omega :\widehat{}^{m|n}\widehat{}^{n|m}`$.

We will call $`f^{m|n}`$ dominant if $`f(m)<\mathrm{}<f(1),f(1)>\mathrm{}>f(n).`$ We warn the reader that the inequality signs here are strict, unlike in the earlier definition of antidominant! Let $`_+^{m|n}`$ denote the set of all dominant $`f^{m|n}`$. For $`f_+^{m|n}`$, let

$$K_f:=M_{fw_0}H_0^{m|n}.$$
(3.3)
The following lemma implies that the $`\{K_f\}_{f_+^{m|n}}`$ form a basis for $`^{m|n}`$.

###### Lemma 3.4.

Suppose $`f^{m|n}`$ and let $`xS_{m|n}`$ be the unique element of minimal length such that $`fx`$ is antidominant. Then,

$$M_fH_0=\{\begin{array}{cc}(q)^{\mathrm{}(x)}K_{fxw_0}\hfill & \text{if }fxw_0\text{ is dominant,}\hfill \\ 0\hfill & \text{otherwise.}\hfill \end{array}$$

###### Proof.

We have that $`M_f=M_{fx}H_{x^1}`$. So applying Lemma 3.2(i), $`M_fH_0=M_{fx}H_{x^1}H_0=(q)^{\mathrm{}(x)}M_{fx}H_0`$. Finally, note that if $`fxw_0`$ is not dominant, then $`M_{fx}H_0=0`$. ∎

§3-b. Canonical bases. Since $`\overline{H_0}=H_0`$ by Lemma 3.2(ii), the bar involution on $`\widehat{𝒯}^{m|n}`$ leaves $`\widehat{}^{m|n}`$ invariant. Moreover, for dominant $`f`$, $`\overline{K_f}=\overline{M_{fw_0}}H_0`$. So using Lemma 3.4 and the explicit description of $`\overline{M_{fw_0}}`$ given by (B1) and (B3) in the proof of Theorem 2.14, we see that $`\overline{K_f}=K_f+()`$ where $`()`$ is a (possibly infinite) $`[q,q^1]`$-linear combination of $`K_g`$’s for $`g_+^{m|n}`$ with $`gf`$. Moreover, for typical dominant $`f`$, we have that $`\overline{K_f}=K_f`$. As in Theorem 2.14, these properties uniquely characterize the bar involution on $`\widehat{}^{m|n}`$:

###### Theorem 3.5.

There exists a unique continuous, antilinear map $`:\widehat{}^{m|n}\widehat{}^{m|n}`$ such that

* $`\overline{K_f}=K_f`$ for all typical $`f_+^{m|n}`$;
* $`\overline{Xu}=\overline{X}\overline{u}`$ for all $`X𝒰`$ and $`u\widehat{}^{m|n}`$.

Moreover,

* bar is an involution;
* $`\overline{K_f}=K_f+()`$ where $`()`$ is a (possibly infinite) $`[q,q^1]`$-linear combination of $`K_g`$’s for dominant $`gf`$;
* $`\overline{\omega (u)}=\omega (\overline{u})`$ for all $`u\widehat{}^{m|n}`$.

###### Proof.

We have already proved existence above. For uniqueness, note on applying $`H_0`$ to the conclusion of Theorem 2.12 that we can write each $`K_f`$ as a possibly infinite linear combination of $`X_gK_g`$’s for $`X_g𝒰`$ and typical $`g_+^{m|n}`$. Hence, as in Corollary 2.13, the only continuous $`𝒰`$-endomorphism of $`\widehat{}^{m|n}`$ that fixes $`K_f`$ for all typical $`f_+^{m|n}`$ is the identity map. Now using this one gets uniqueness by exactly the same argument as in the proof of Theorem 2.14. ∎

Now applying the general principles used in the proof of Theorem 2.17, we deduce:

###### Theorem 3.6.

There exist unique topological bases $`\{U_f\}_{f_+^{m|n}}`$, $`\{L_f\}_{f_+^{m|n}}`$ for $`\widehat{}^{m|n}`$ such that

* $`\overline{U_f}=U_f`$ and $`\overline{L_f}=L_f`$;
* $`U_fK_f+\widehat{}_{g_+^{m|n}}q[q]K_g`$ and $`L_fK_f+\widehat{}_{g_+^{m|n}}q^1[q^1]K_g`$.

Moreover,

* $`U_f=K_f+()`$ and $`L_f=K_f+()`$ where $`()`$ and $`()`$ are (possibly infinite) linear combinations of $`K_g`$’s for dominant $`gf`$;
* $`\omega (U_f)=U_{\omega (f)}`$ and $`\omega (L_f)=L_{\omega (f)}`$.

We use the following notation for the coefficients:

$$L_f=\underset{g_+^{m|n}}{}l_{g,f}(q)K_g,U_f=\underset{g_+^{m|n}}{}u_{g,f}(q)K_g,$$
(3.7)
for polynomials $`l_{g,f}(q)[q^1],u_{g,f}(q)[q]`$. We know that $`l_{g,f}(q)=u_{g,f}(q)=0`$ unless $`gf`$, and $`l_{f,f}(q)=u_{f,f}(q)=1`$.

Note that $`K_f=M_f+()`$ where $`()`$ is a $`q^1[q^1]`$-linear combination of $`M_g`$’s. So the element $`L_f`$ defined in Theorem 3.6 is bar invariant and equals $`M_f`$ plus a $`q^1[q^1]`$-linear combination of $`M_g`$’s. So by the uniqueness in Theorem 2.17, the elements $`L_f`$ and the polynomials $`l_{g,f}(q)`$ defined here are the same as the ones defined in 2, for dominant $`g,f`$. Thus our notation is consistent with the earlier notation. The relationship between the elements $`U_f`$ here and the $`T_f`$’s from before is given by:

###### Lemma 3.8.

For $`f_+^{m|n}`$, $`U_f=T_{fw_0}H_0`$.

###### Proof.

Note that $`T_{fw_0}H_0`$ is a bar invariant element of $`\widehat{}^{m|n}`$. Recall that $`T_{fw_0}`$ equals $`M_{fw_0}`$ plus a $`q[q]`$-linear combination of $`M_g`$’s. So applying Lemma 3.4, $`T_{fw_0}H_0`$ equals $`K_f`$ plus a $`q[q]`$-linear combination of $`K_g`$’s. Hence $`T_{fw_0}H_0=U_f`$ by the uniqueness in Theorem 3.6. ∎

§3-c. Duality. Recall the antilinear involution $`\sigma :𝒯^{m|n}𝒯^{m|n}`$ defined in (2.8). In view of Lemmas 2.9(iii) and 3.2(ii), this leaves the subspace $`^{m|n}`$ invariant. Indeed, by Lemma 3.4, we have that

$$\sigma (K_f)=(q)^{\mathrm{}(w_0)}K_{fw_0}$$
(3.9)
for each $`f_+^{m|n}`$. Let $`(.,.)_{_{}}`$ be the bilinear form on $`^{m|n}`$ defined so that the elements $`\{K_f\}_{f_+^{m|n}}`$ are orthonormal. Note by Lemmas 2.9(ii) and 3.2(iii) that

$`(K_f,K_g)__𝒯`$ $`=(M_{fw_0}H_0,M_{gw_0}H_0)__𝒯=[m]![n]!(M_{fw_0}H_0,M_{gw_0})__𝒯`$
$`=(q)^{\mathrm{}(w_0)}[m]![n]!\delta _{f,g}.`$
Hence,

$$(u,v)_{_{}}=\frac{(q)^{\mathrm{}(w_0)}}{[m]![n]!}(u,v)__𝒯$$
(3.10)
for all $`u,v^{m|n}`$. Finally, define a bilinear form $`.,._{_{}}`$ on $`\widehat{}^{m|n}`$ by setting

$$u,v_{_{}}:=(q)^{\mathrm{}(w_0)}(u,\sigma (\overline{v}))_{_{}}$$
(3.11)
for all $`u,v\widehat{}^{m|n}`$. Comparing with the definition of the form $`.,.__𝒯`$ from (2.20) and using (3.10), one sees immediately that

$$u,v_{_{}}=\frac{1}{[m]![n]!}u,v__𝒯$$
(3.12)
for all $`u,v\widehat{}^{m|n}`$. Hence in particular we get from Lemma 2.22 that the form $`.,._{_{}}`$ is symmetric.

###### Theorem 3.13.

For $`f,g_+^{m|n}`$, $`L_f,U_{gw_0}_{_{}}=\delta _{f,g}`$

###### Proof.

Since $`L_f\widehat{}^{m|n}`$, we have by Lemma 3.2(iii) that $`L_fH_0=[m]![n]!L_f`$. So applying Theorem 2.23 and Lemmas 3.8 and 2.21, we have that

$`L_f,U_{gw_0}_{_{}}`$ $`={\displaystyle \frac{1}{[m]![n]!}}L_f,T_gH_0__𝒯={\displaystyle \frac{1}{[m]![n]!}}L_fH_0,T_g__𝒯`$
$`=L_f,T_g__𝒯=\delta _{f,g}.`$
By the theorem and the argument used to prove Corollary 2.24, we get:

###### Corollary 3.14.

For $`f_+^{m|n}`$,

$$K_f=\underset{g_+^{m|n}}{}u_{fw_0,gw_0}(q^1)L_g=\underset{g_+^{m|n}}{}l_{fw_0,gw_0}(q^1)U_g.$$

§3-d. Crystal structures. Next we describe the crystal structure on $`\widehat{}^{m|n}`$, following the same language as in 2. Recalling (3.3), let

$`\stackrel{~}{E}_a(f):=(\stackrel{~}{E}_a^{}(fw_0))w_0,`$ $`\stackrel{~}{F}_a(f):=(\stackrel{~}{F}_a^{}(fw_0))w_0,`$ (3.15)
$`\epsilon _a(f):=\epsilon _a^{}(fw_0),`$ $`\phi _a(f):=\phi _a^{}(fw_0)`$ (3.16)
for $`f_+^{m|n}`$. Then, $`(_+^{m|n},\stackrel{~}{E}_a,\stackrel{~}{F}_a,\epsilon _a,\phi _a,\mathrm{wt})`$ is the crystal associated to the $`𝒰`$-module $`^{m|n}`$. Actually, the crystal structure is so simple in this case, that we can actually list all the possibilities explicitly. There are ten possible configurations for edges of color $`a`$ in the crystal graph, listed below. Here, $`\mathrm{}`$ denotes entries different from $`a,a+1`$.

* $`(\mathrm{},a,a+1,\mathrm{}|\mathrm{},a+1,a,\mathrm{})`$;
* $`(\mathrm{},a,a+1,\mathrm{}|\mathrm{},a,\mathrm{})\stackrel{\stackrel{~}{F}_a}{}(\mathrm{},a,a+1,\mathrm{}|\mathrm{},a+1,\mathrm{})`$;
* $`(\mathrm{},a+1,\mathrm{}|\mathrm{},a+1,a,\mathrm{})\stackrel{\stackrel{~}{F}_a}{}(\mathrm{},a,\mathrm{}|\mathrm{},a+1,a,\mathrm{})`$;
* $`(\mathrm{},a+1,\mathrm{}|\mathrm{},a,\mathrm{})\stackrel{\stackrel{~}{F}_a}{}(\mathrm{},a,\mathrm{}|\mathrm{},a,\mathrm{})\stackrel{\stackrel{~}{F}_a}{}(\mathrm{},a,\mathrm{}|\mathrm{},a+1,\mathrm{})`$;
* $`(\mathrm{},a+1,\mathrm{}|\mathrm{},a+1,\mathrm{})`$;
* $`(\mathrm{},a,a+1,\mathrm{}|\mathrm{})`$;
* $`(\mathrm{}|\mathrm{},a+1,a,\mathrm{})`$;
* $`(\mathrm{}|\mathrm{},a,\mathrm{})\stackrel{\stackrel{~}{F}_a}{}(\mathrm{}|\mathrm{},a+1,\mathrm{})`$;
* $`(\mathrm{},a+1,\mathrm{}|\mathrm{})\stackrel{\stackrel{~}{F}_a}{}(\mathrm{},a,\mathrm{}|\mathrm{})`$;
* $`(\mathrm{}|\mathrm{})`$.

We also have the dual crystal $`(_+^{m|n},\stackrel{~}{E}_a^{},\stackrel{~}{F}_a^{},\epsilon _a^{},\phi _a^{},\mathrm{wt})`$, where $`\stackrel{~}{E}_a^{},\stackrel{~}{F}_a^{},\epsilon _a^{}`$ and $`\phi _a^{}`$ are the restrictions of the functions from 2 to $`_+^{m|n}`$. Combining (2.32) and (3.15), we have that

$$\stackrel{~}{E}_a^{}(fw_0)=\stackrel{~}{F}_{1a}(f)w_0,\stackrel{~}{F}_a^{}(fw_0)=\stackrel{~}{E}_{1a}(f)w_0.$$
(3.17)
Again, there are ten possible configurations for the edges in the corresponding dual crystal graph, all of which are exactly the same as (1)–(10) above (replacing $`\stackrel{~}{F}_a`$ with $`\stackrel{~}{F}_a^{}`$) with the exception of (4) and (5) which change to

* $`(\mathrm{},a+1,\mathrm{}|\mathrm{},a,\mathrm{})\stackrel{\stackrel{~}{F}_a^{}}{}(\mathrm{},a+1,\mathrm{}|\mathrm{},a+1,\mathrm{})\stackrel{\stackrel{~}{F}_a^{}}{}(\mathrm{},a,\mathrm{}|\mathrm{},a+1,\mathrm{})`$;
* $`(\mathrm{},a,\mathrm{}|\mathrm{},a,\mathrm{})`$.

###### Remark 3.18.

In (3.31) and Lemma 3.32(v) below we will define mutually inverse bijections $`𝙻,𝚁:_+^{m|n}_+^{m|n}`$. By considering all the above cases (1)–(10) one by one, it is not hard to check that $`𝙻`$ satisfies, indeed is characterized uniquely by, the following properties

* if $`f_+^{m|n}`$ is typical then $`𝙻(f)=f`$;
* for every $`a`$ and $`f_+^{m|n}`$, $`\stackrel{~}{E}_a^{}𝙻(f)=𝙻(\stackrel{~}{E}_af)`$ and $`\stackrel{~}{F}_a^{}𝙻(f)=𝙻(\stackrel{~}{F}_af)`$;
* for every $`f_+^{m|n}`$, $`\mathrm{wt}(𝙻(f))=\mathrm{wt}(f)`$.

Hence, $`𝙻:(_+^{m|n},\stackrel{~}{E}_a,\stackrel{~}{F}_a,\epsilon _a,\phi _a,\mathrm{wt})(_+^{m|n},\stackrel{~}{E}_a^{},\stackrel{~}{F}_a^{},\epsilon _a^{},\phi _a^{},\mathrm{wt})`$ is an isomorphism of crystals with inverse $`𝚁`$.

The crucial observation to be made from the above description of the crystal graph is that all $`a`$-strings are of length at most $`2`$. The following lemma is a consequence of this particularly simple structure.

###### Lemma 3.19.

Let $`f_+^{m|n}`$ and $`a`$.

* If $`\epsilon _a(f)>0`$, then $`E_aU_f=[\phi _a(f)+1]U_{\stackrel{~}{E}_a(f)}`$.
* If $`\phi _a(f)>0`$, then $`F_aU_f=[\epsilon _a(f)+1]U_{\stackrel{~}{F}_a(f)}`$.

###### Proof.

We prove (i), (ii) being similar. Dualizing Theorem 2.34(i) using Theorem 3.13 and Lemma 2.21 (or by considering the effect of the Kashiwara operators directly and arguing as in the proof of Theorem 2.31) gives us that $`E_aU_f=[\phi _a(f)+1]U_{\stackrel{~}{E}_a(f)}+\widehat{}_{g_+^{m|n}}y_{g,f}^aU_g`$ where $`y_{g,f}^a`$ belongs to $`q^{2\phi _a(g)}[q]`$ and is zero unless $`\epsilon _b(g)\epsilon _b(f)`$ for all $`b`$. Suppose that $`y_{g,f}^a0`$ for some $`g`$. By assumption, $`\epsilon _a(g)\epsilon _a(f)1`$, so $`\phi _a(g)1`$ since all $`a`$-strings are of length $`2`$. So $`0y_{g,f}^aq[q]`$. But $`y_{g,f}^a`$ is bar invariant, so this is a contradiction. ∎

§3-e. Two algorithms. In this subsection, we describe algorithms to compute the canonical basis $`\{U_f\}`$ and the coefficients $`\{l_{g,f}(q)\}`$ of the dual canonical basis of $`^{m|n}`$ explicitly. The first algorithm computes $`U_f`$, and is similar to the algorithm for computing the $`T_f`$’s explained in 2 — but it is much simpler since no corrections are needed thanks to Lemma 3.19.

###### Procedure 3.20.

Suppose we are given $`f_+^{m|n}`$ with $`\mathrm{\#}f>0`$. Compute $`h_+^{m|n}`$ and operators $`X_a,Y_a\{E_a,F_a\}_a`$ by following the instructions below starting at step (0).

* Choose the largest $`i\{m,\mathrm{},1\}`$ such that $`f(i)=f(j)`$ for some $`j\{1,\mathrm{},n\}`$. Go to step (1).
* If $`i<1`$ and $`f(i+1)=f(i)+1`$, replace $`i`$ by $`(i+1)`$ and repeat step (1). Otherwise, go to step (2).
* If $`f(i)+1=f(j)`$ for some (necessarily unique) $`j\{1,\mathrm{},n\}`$ go to step (1). Otherwise, set $`X_a=F_{f(i)},Y_a=E_{f(i)}`$ and $`h=fd_i`$. Stop.
* If $`j>1`$ and $`f(j1)=f(j)+1`$, replace $`j`$ by $`(j1)`$ and repeat step (1). Otherwise, go to step (2).
* If $`f(j)+1=f(i)`$ for some (necessarily unique) $`i\{m,\mathrm{},1\}`$ go to step (1). Otherwise, set $`X_a=E_{f(j)},Y_a=F_{f(j)}`$ and $`h=f+d_j`$. Stop.

The following lemma follows immediately from the nature of the above procedure and Lemma 3.19.

###### Lemma 3.21.

Take $`f_+^{m|n}`$ with $`\mathrm{\#}f>0`$. Define $`h`$ and operators $`X_a,Y_a\{E_a,F_a\}_a`$ according to Procedure 3.20. Then, one of the following holds:

* $`\mathrm{\#}h=\mathrm{\#}f`$. In this case, the $`a`$-string through $`f`$ is $`h\stackrel{\stackrel{~}{X}_a}{}f`$, of length $`1`$. Moreover, $`X_aU_h=U_f`$, $`Y_aU_f=U_h`$ and $`X_aK_h=K_f`$.
* $`\mathrm{\#}h=\mathrm{\#}f1`$. In this case, the $`a`$-string through $`f`$ is $`h\stackrel{\stackrel{~}{X}_a}{}f\stackrel{\stackrel{~}{X}_a}{}g`$, of length $`2`$. Moreover $`X_aU_h=U_f`$, $`Y_aU_f=[2]U_h`$ and $`X_aK_h=K_f+qK_{\stackrel{~}{X}_a^{}(h)}`$.

Case (ii) (when the atypicality gets strictly smaller) must occur after at most $`(m+n1)`$ repetitions of the procedure. Hence after finitely many recursions, the procedure reduces $`f`$ to a typical weight.

Lemma 3.21 implies the following algorithm for computing $`U_f`$. If $`f_+^{m|n}`$ is typical then $`U_f=K_f`$, since such $`f`$’s are minimal in the ordering $``$ in $`_+^{m|n}`$. Otherwise, apply Procedure 3.20 to get $`h_+^{m|n}`$ and $`X_a\{E_a,F_a\}_a`$. Since the procedure always reduces $`f`$ to a typical weight in finitely many steps, we may assume $`U_h`$ is known recursively. Then $`U_f=X_aU_h`$.

###### Example 3.22.

Applying the algorithm repeatedly, we get that

$`U_{(0,1,3,4|2,1,0)}`$ $`=F_4F_3E_2F_1F_5F_4E_3F_2E_1F_0K_{(1,3,5,6|4,2,0)}`$
$`=K_{(0,1,3,4|2,1,0)}+qK_{(1,3,4,6|6,2,1)}+qK_{(0,3,4,5|5,2,0)}+q^2K_{(3,4,5,6|6,5,2)}.`$
In the next subsection, we will apply the above algorithm to derive a closed formula for $`U_f`$. We turn now to describing the second algorithm, which computes the polynomials $`l_{g,f}(q)`$. It will not be needed until 4 below. First we state a variation on Procedure 3.20.

###### Procedure 3.23.

Suppose we are given $`g_+^{m|n}`$ with $`\mathrm{\#}g>0`$. Compute $`h_+^{m|n}`$ and operators $`X_a,Y_a\{E_a,F_a\}_a`$ by following the instructions below starting at step (0).

* Choose the smallest $`i\{m,\mathrm{},1\}`$ such that $`g(i)=g(j)`$ for some $`j\{1,\mathrm{},n\}`$. Go to step (1).
* If $`i>m`$ and $`g(i1)=g(i)1`$, replace $`i`$ by $`(i1)`$ and repeat step (1). Otherwise, go to step (2).
* If $`g(i)1=g(j)`$ for some (necessarily unique) $`j\{1,\mathrm{},n\}`$ go to step (1). Otherwise, set $`h=g+d_i`$, $`X_a=E_{h(i)}`$ and $`Y_a=F_{h(i)}`$. Stop.
* If $`j<n`$ and $`g(j+1)=g(j)1`$, replace $`j`$ by $`(j+1)`$ and repeat step (1). Otherwise, go to step (2).
* If $`g(j)1=g(i)`$ for some (necessarily unique) $`i\{m,\mathrm{},1\}`$ go to step (1). Otherwise, set $`h=gd_j`$, $`X_a=F_{h(j)}`$ and $`Y_a=E_{h(j)}`$. Stop.

###### Lemma 3.24.

Suppose $`g,f_+^{m|n}`$ with $`\mathrm{\#}g>0`$. Define $`h`$ and operators $`X_a,Y_a\{E_a,F_a\}_a`$ according to Procedure 3.23. Then,

$$l_{g,f}(q^1)=\{\begin{array}{cc}l_{h,\stackrel{~}{Y}_a^{}(f)}(q^1)\hfill & \text{if }\mathrm{\#}h=\mathrm{\#}g\text{,}\hfill \\ l_{h,\stackrel{~}{Y}_a^{}(f)}(q^1)+ql_{\stackrel{~}{X}_a(h),f}(q^1)\hfill & \text{if }\mathrm{\#}h=\mathrm{\#}g1\text{,}\hfill \end{array}$$

interpreting $`l_{h,\stackrel{~}{Y}_a^{}(f)}(q^1)`$ as $`0`$ if $`\stackrel{~}{Y}_a^{}(f)=\mathrm{}`$.

###### Proof.

Let $`f,g_+^{m|n}`$ with $`\mathrm{\#}f>0`$. Apply Procedure 3.20 to construct $`h`$ and operators $`X_a,Y_a\{E_a,F_a\}_a`$. Apply the operator $`X_a`$ to both sides of the equation

$$K_h=\underset{k_+^{m|n}}{}l_{hw_0,kw_0}(q^1)U_k$$

from Corollary 3.14. In the case that $`\mathrm{\#}h=\mathrm{\#}f`$, we know by Lemma 3.21 that $`h`$, hence also all $`k_+^{m|n}`$ with the same weight as $`h`$, is at one end of an $`a`$-string of length 1. So by Lemma 3.19, $`X_aU_k=U_{\stackrel{~}{X}_a(k)}`$ for all $`k`$ with $`l_{hw_0,kw_0}(q^1)0`$. Hence,

$$K_f=X_aK_h=\underset{k_+^{m|n}}{}l_{hw_0,kw_0}(q^1)U_{\stackrel{~}{X}_a(k)}.$$

On the other hand, if $`\mathrm{\#}h=\mathrm{\#}f1`$, then $`h`$, hence also all $`k`$ of the same weight as $`h`$, lies at one end of an $`a`$-string of length 2, so $`X_aU_k=U_{\stackrel{~}{X}_a(k)}`$. We also know from Lemma 3.21 that $`X_aK_h=K_f+qK_{\stackrel{~}{X}_a^{}(h)}`$. So in this case,

$$K_f=\underset{k_+^{m|n}}{}l_{hw_0,kw_0}(q^1)U_{\stackrel{~}{X}_a(k)}q\underset{h_+^{m|n}}{}l_{\stackrel{~}{X}_a^{}(h)w_0,gw_0}(q^1)U_g.$$

Now compute the coefficient of $`U_g`$ in the above expressions for $`K_f`$ using Corollary 3.14 again, to get

$$l_{fw_0,gw_0}(q^1)=\{\begin{array}{cc}l_{hw_0,\stackrel{~}{Y}_a(g)w_0}(q^1)\hfill & \text{if }\mathrm{\#}h=\mathrm{\#}f\text{,}\hfill \\ l_{hw_0,\stackrel{~}{Y}_a(g)w_0}(q^1)\hfill & \text{if }\mathrm{\#}h=\mathrm{\#}f1\text{,}\hfill \\ ql_{\stackrel{~}{X}_a^{}(h)w_0,gw_0}(q^1)\hfill & \end{array}$$

interpreting $`l_{hw_0,\stackrel{~}{Y}_a(g)w_0}(q)`$ as $`0`$ if $`\stackrel{~}{Y}_a(g)=\mathrm{}`$. The lemma follows from this, replacing $`f`$ by $`gw_0`$ and $`g`$ by $`fw_0`$ and using (3.17), since Procedure 3.23 is just Procedure 3.20 twisted by the involution $`ffw_0`$. ∎

Now to compute $`l_{g,f}(q^1)`$, we have that $`l_{g,f}(q^1)=\delta _{g,f}`$ if $`g`$ is typical, and it is $`0`$ if $`gf`$. Otherwise, if $`\mathrm{\#}g>0`$ and $`gf`$, apply Procedure 3.23 and Lemma 3.24 to write $`l_{g,f}(q^1)`$ in terms of $`l_{h,\stackrel{~}{Y}_a^{}(f)}(q^1)`$ and (in case $`\mathrm{\#}h=\mathrm{\#}g1`$) $`l_{\stackrel{~}{X}_a(h),f}(q^1)`$, and repeat. This process terminates in finitely many steps, because $`h`$ is closer to being typical than $`g`$ in the sense of Procedure 3.23, and $`\stackrel{~}{X}_a(h)`$ is closer than $`g`$ to failing the condition $`gf`$. Note this algorithm shows in particular that $`l_{g,f}(q^1)[q]`$, as also follows from the explicit description given in Corollary 3.39(ii) below.

§3-f. Combinatorial description of canonical bases. We now introduce some combinatorics to enable us to write down closed formulae for the canonical basis and dual canonical basis elements. The material in this subsection was inspired originally by \[JZ\]. Suppose $`f^{m|n}`$ is conjugate under the action of $`S_{m|n}`$ to an element of $`_+^{m|n}`$. We will denote this “dominant conjugate” of $`f`$ by $`f^+`$. For $`mi<0<jn`$ with $`f(i)=f(j)`$, let

$`𝙻_{i,j}(f)`$ $`:=fa(d_id_j),`$ (3.25)
where $`a`$ is the smallest positive integer such that $`fa(d_id_j)`$ and all $`𝙻_{k,l}(f)a(d_id_j)`$ for $`i<k<0<l<j`$ with $`f(k)=f(l)`$ are conjugate to elements of $`_+^{m|n}`$. Similarly, let
$`𝚁_{i,j}(f)`$ $`:=f+b(d_id_j),`$ (3.26)
where $`b`$ is the smallest positive integer such that $`f+b(d_id_j)`$ and all $`𝚁_{k,l}(f)+b(d_id_j)`$ for $`mk<i,j<ln`$ with $`f(k)=f(l)`$ are conjugate to elements of $`_+^{m|n}`$.

Now take $`f_+^{m|n}`$. Let $`r=\mathrm{\#}f`$ and $`mi_1<\mathrm{}<i_r<0<j_r<\mathrm{}<j_1n`$ be the unique integers with $`f(i_s)=f(j_s)`$ for each $`s=1,\mathrm{},r`$. For a tuple $`\theta =(\theta _1,\mathrm{},\theta _r)^r`$, $`|\theta |`$ denotes $`\theta _1+\mathrm{}+\theta _r`$. Let

$`𝙻_\theta (f)`$ $`=\left(𝙻_{i_r,j_r}^{\theta _r}𝙻_{i_{r1},j_{r1}}^{\theta _{r1}}\mathrm{}𝙻_{i_1,j_1}^{\theta _1}(f)\right)^+,`$ (3.27)
$`𝙻_\theta ^{}(f)`$ $`=\left(𝙻_{i_1,j_1}^{\theta _1}𝙻_{i_2,j_2}^{\theta _2}\mathrm{}𝙻_{i_r,j_r}^{\theta _r}(f)\right)^+,`$ (3.28)
$`𝚁_\theta (f)`$ $`=\left(𝚁_{i_1,j_1}^{\theta _1}𝚁_{i_2,j_2}^{\theta _2}\mathrm{}𝚁_{i_r,j_r}^{\theta _r}(f)\right)^+,`$ (3.29)
$`𝚁_\theta ^{}(f)`$ $`=\left(𝚁_{i_r,j_r}^{\theta _r}𝚁_{i_{r1},j_{r1}}^{\theta _{r1}}\mathrm{}𝚁_{i_1,j_1}^{\theta _1}(f)\right)^+.`$ (3.30)
Note that $`𝙻_\theta ^{}(f),𝙻_\theta (f)f𝚁_\theta (f),𝚁_\theta ^{}(f)`$. The operators $`𝙻_\theta `$ and $`𝚁_\theta `$ will only ever be used for $`\theta `$ belonging to the set $`\{0,1\}^r`$. In the special case that $`\theta _1=\mathrm{}=\theta _r=1`$, we let

$$𝙻(f):=𝙻_\theta (f),𝚁(f):=𝚁_\theta (f).$$
(3.31)
The following combinatorial lemma lists some elementary properties of the lowering and raising operators, which follow immediately from the definition.

###### Lemma 3.32.

Let $`f_+^{m|n}`$ and $`r=\mathrm{\#}f`$.

* Suppose $`\theta ^r`$ and let $`\phi =(\theta _r,\mathrm{},\theta _1)`$. Then, $`𝚁_\theta (fw_0)=𝙻_\phi (f)w_0`$ and $`𝚁_\theta ^{}(fw_0)=𝙻_\phi ^{}(f)w_0.`$
* The sets $`\{𝙻_\theta (f)\}_{\theta \{0,1\}^r}`$ and $`\{𝚁_\theta (f)\}_{\theta \{0,1\}^r}`$ contain $`2^r`$ distinct elements.
* Suppose $`\theta \{0,1\}^r`$ and let $`\phi =(1\theta _r,\mathrm{},1\theta _1)`$. Then, $`𝙻_\theta (𝚁(f))=𝚁_\phi (f)`$ and $`𝚁_\theta (𝙻(f))=𝙻_\phi (f)`$. In particular, taking $`\theta _1=\mathrm{}=\theta _r=1`$, the maps $`𝙻,𝚁:_+^{m|n}_+^{m|n}`$ are mutually inverse bijections.

###### Example 3.33.

Take $`f=(0,1,3,4|2,1,0)`$ as in Example 3.22, so $`\mathrm{\#}f=2`$. Then we have that $`𝙻_{(0,0)}(f)=(0,1,3,4|2,1,0)`$, $`𝙻_{(1,0)}(f)=(1,3,4,6|6,2,1)`$, $`𝙻_{(0,1)}(f)=(0,3,4,5|5,2,0)`$ and $`𝙻_{(1,1)}(f)=(3,4,5,6|6,5,2)`$. Observe these are exactly the $`K_g`$’s appearing in the expression for $`U_f`$ computed in Example 3.22.

The main theorem of the subsection is the following.

###### Theorem 3.34.

For $`f_+^{m|n}`$ and $`r=\mathrm{\#}f`$,

* $`U_f={\displaystyle \underset{\theta \{0,1\}^r}{}}q^{|\theta |}K_{𝙻_\theta (f)};`$
* $`K_f={\displaystyle \underset{\theta ^r}{}}(q)^{|\theta |}U_{𝙻_\theta ^{}(f)}.`$

###### Proof.

(i) If $`f`$ is typical, then $`U_f=K_f`$ and there is nothing to prove. So suppose that $`\mathrm{\#}f>0`$ and define $`h`$ and $`X_a,Y_a\{E_a,F_a\}_a`$ according to Procedure 3.20. We may assume by induction that the result has already been established for $`h`$. Recalling Lemma 3.21, we need to consider two cases. In the first case $`\mathrm{\#}h=\mathrm{\#}f`$, we know that $`U_h=_{\theta \{0,1\}^r}q^{|\theta |}K_{𝙻_\theta (h)}.`$ Applying $`X_a`$ to both sides, noting that $`X_aU_h=U_f`$ and that $`X_aK_{𝙻_\theta (h)}=K_{𝙻_\theta (f)}`$ for each $`\theta `$, gives the desired conclusion. In the second case $`\mathrm{\#}h=\mathrm{\#}f1`$. This time, we know that $`U_h=_{\theta \{0,1\}^{r1}}q^{|\theta |}K_{𝙻_\theta (h)}.`$ For each $`\theta `$, $`𝙻_\theta (h)`$ here has the form $`(\mathrm{},a+1,\mathrm{}|\mathrm{},a,\mathrm{})`$, so

$$X_aK_{𝙻_\theta (h)}=K_{(\mathrm{},a,\mathrm{}|\mathrm{},a,\mathrm{})}+qK_{(\mathrm{},a+1,\mathrm{}|\mathrm{},a+1,\mathrm{})}=K_{𝙻_{\theta 0}(f)}+qK_{𝙻_{\theta 1}(f)},$$

where $`\theta x`$ denotes $`(\theta _1,\mathrm{},\theta _{r1},x)\{0,1\}^r`$. So again we see on applying $`X_a`$ to both sides that $`U_f=X_aU_h=_{\theta \{0,1\}^r}q^{|\theta |}K_{𝙻_\theta (f)}`$.

(ii) To deduce this from (i), we will work in the free $`[q,q^1]`$-module $`^{m|n}`$ on basis $`\{[f]\}_{f_+^{m|n}S_{m|n}}`$, completed to a topological $`[q,q^1]`$-module $`\widehat{}^{m|n}`$ exactly as in 2 so that expressions of the form $`[f]+`$(a possibly infinite linear combination of $`[g]^{}s`$ with $`gf`$) make sense. We define continuous linear maps $`U,K:\widehat{}^{m|n}\widehat{}^{m|n}`$ by letting $`U([f])=U_{f^+},K([f])=K_{f^+}`$. These maps have the right inverses $`U^1,K^1:\widehat{}^{m|n}\widehat{}^{m|n}`$ with $`U^1(U_f)=[f],K^1(K_f)=[f]`$ for each $`f_+^{m|n}`$. Finally, define continuous linear operators $`\lambda _{i,j}:\widehat{}^{m|n}\widehat{}^{m|n}`$ for each $`mi<0<jn`$ by

$$\lambda _{i,j}([f])=\{\begin{array}{cc}[𝙻_{i,j}(f)]\hfill & \text{if }f(i)=f(j)\text{,}\hfill \\ 0\hfill & \text{if }f(i)f(j)\text{,}\hfill \end{array}$$

for each $`f_+^{m|n}S_{m|n}`$. Now consider the maps

$`P:=K\left({\displaystyle \stackrel{}{\underset{mi<0<jn}{}}}(1+q\lambda _{i,j})\right)U^1`$ $`:\widehat{}^{m|n}\widehat{}^{m|n},`$
$`Q:=U\left({\displaystyle \stackrel{}{\underset{mi<0<jn}{}}}{\displaystyle \frac{1}{1+q\lambda _{i,j}}}\right)K^1`$ $`:\widehat{}^{m|n}\widehat{}^{m|n},`$
where $`\stackrel{}{}`$ is taken in some ordering with $`i`$’s decreasing and $`j`$’s increasing from left to right and $`\stackrel{}{}`$ is taken in the opposite ordering, and $`\frac{1}{1+q\lambda _{i,j}}`$ denotes $`(1q\lambda _{i,j}+q^2\lambda _{i,j}\mathrm{})`$. By (i) and the definition of the operator $`𝙻_{i,j}`$, the map $`P`$ sends $`U_f`$ to $`U_f`$, so $`P=\mathrm{id}`$. On the other hand the result we are trying to prove is equivalent to the statement that $`Q`$ sends $`K_f`$ to $`K_f`$. Therefore we will be done if we can show that $`PQ=\mathrm{id}`$, i.e. that for every $`f_+^{m|n}`$,

$$K\left(\stackrel{}{\underset{mi<0<jn}{}}(1+q\lambda _{i,j})\right)(U^1U)\left(\stackrel{}{\underset{mi<0<jn}{}}\frac{1}{1+q\lambda _{i,j}}\right)([f])=K_f.$$

This is obvious if we can show that the inside map $`(U^1U):[g][g^+]`$ on the left hand side can be omitted. For this, we check that

$$K\left(\stackrel{}{\underset{mi<0<jn}{}}(1+q\lambda _{i,j})\right)([g^+])=K\left(\stackrel{}{\underset{mi<0<jn}{}}(1+q\lambda _{i,j})\right)([g])$$
(3.35)
for every $`g^{m|n}`$ such that $`[g]`$ is involved in $`\left(\stackrel{}{}\frac{1}{1+q\lambda _{i,j}}\right)([f])`$ with non-zero coefficient. Suppose we have such a $`g`$. The crucial observation is that whenever there exist $`mi^{}<i<0<j<j^{}n`$ with $`g(j)=g(i)<g(i^{})=g(j^{})`$, one can find $`c`$ with $`g(i)<c<g(i^{})`$ that does not arise in the tuple $`g`$. Given this it is not hard to see that (3.35) holds. ∎

###### Corollary 3.36.

For $`f_+^{m|n}`$,

* $`K_f=_gq^{|\theta _g|}L_g`$ where the sum is over all $`g_+^{m|n}`$ such that $`𝚁_{\theta _g}(g)=f`$ for some (unique) $`\theta _g\{0,1\}^{\mathrm{\#}g}`$;
* $`L_f=_{g,\theta }(q)^{|\theta |}K_g`$ where the sum is over all $`g_+^{m|n}`$ and $`\theta ^{\mathrm{\#}g}`$ such that $`𝚁_\theta ^{}(g)=f`$.

###### Proof.

(i) Recall from Corollary 3.14 that the coefficient of $`L_g`$ in $`K_f`$ is equal to $`u_{fw_0,gw_0}(q^1)`$. By Theorem 3.34(i) and Lemma 3.32(ii), $`u_{fw_0,gw_0}(q^1)=q^{|\theta |}`$ if $`fw_0=𝙻_\theta (gw_0)`$ for some (necessarily unique) $`\theta \{0,1\}^{\mathrm{\#}g}`$, and is zero otherwise. Equivalently, invoking Lemma 3.32(i), $`u_{fw_0,gw_0}(q^1)=q^{|\theta |}`$ if $`f=𝚁_\theta (g)`$ for some $`\theta `$, and is zero otherwise.

(ii) By Corollary 3.14 again, $`l_{g,f}(q^1)`$ is equal to the coefficient of $`U_{fw_0}`$ in $`K_{gw_0}`$. By Theorem 3.34(ii), this equals $`_\theta (q)^{|\theta |}`$ where the sum is over all $`\theta ^{\mathrm{\#}g}`$ with $`𝙻_\theta ^{}(gw_0)=fw_0`$, equivalently, $`𝚁_\theta ^{}(g)=f`$. ∎

###### Example 3.37.

Suppose $`f=(m,\mathrm{},2,1|1,2,\mathrm{},n)`$, so $`r=\mathrm{\#}f=\mathrm{min}(m,n)`$. We observe that any $`gf`$ can be represented as $`𝙻_\theta ^{}(f)`$ for a unique element $`\theta ^r`$ with $`\theta _1\mathrm{}\theta _r`$. Moreover, this $`\theta `$ is also the unique element of $`^r`$ with the property that $`f=𝚁_\theta ^{}(g)`$. We deduce from Corollary 3.36(ii) that

$$L_f=\underset{\theta =(\theta _1\mathrm{}\theta _r)^r}{}(q)^{|\theta |}K_{𝙻_\theta ^{}(f)}.$$
(3.38)
Recalling the definitions from (3.7), we can restate Theorem 3.34(i) and Corollary 3.36(ii) as follows:

###### Corollary 3.39.

For $`g,f_+^{m|n}`$,

* $`u_{g,f}(q)=q^{|\theta |}`$ if $`g=𝙻_\theta (f)`$ for some $`\theta \{0,1\}^{\mathrm{\#}f}`$, $`u_{g,f}(q)=0`$ otherwise;
* $`l_{g,f}(q^1)=_\theta q^{|\theta |}`$ summing over all $`\theta ^{\mathrm{\#}g}`$ with $`𝚁_\theta ^{}(g)=f`$.

###### Example 3.40.

Using Corollary 3.39(ii) and arguing by induction on $`n`$, one gets that $`l_{g,f}(q^1)=q^2(1+q^2)^{n1}`$, for $`f=(0,2,\mathrm{},2n2|2n2,\mathrm{},2,0)`$ and $`g=(2,4,\mathrm{},2n|2n,\mathrm{},4,2)`$.

###### Corollary 3.41.

For $`f,g_+^{m|n}`$, $`u_{gw_0,fw_0}(q)=q^{\mathrm{\#}f}u_{g,𝚁(f)}(q^1).`$

###### Proof.

Let $`r=\mathrm{\#}f`$. By Corollary 3.39(i) and Lemma 3.32(i), we know that $`u_{gw_0,fw_0}(q)=q^{|\theta |}`$ if $`g=𝚁_\theta (f)`$ for some $`\theta \{0,1\}^r`$ and is zero otherwise. Similarly, by Lemma 3.32(iii), $`u_{g,𝚁(f)}(q)`$ is $`q^{|\theta |}=q^{r|\phi |}`$ if $`g=𝙻_\theta (𝚁(f))=𝚁_\phi (f)`$ for some $`\theta \{0,1\}^r`$ and $`\phi =(1\theta _r,\mathrm{},1\theta _1)`$ and is zero otherwise. ∎

§3-g. Length function. We now consider some further properties of the polynomials $`l_{g,f}(q)`$.

###### Lemma 3.42.

Let $`g,f_+^{m|n}`$ with $`gf`$ and set $`r=\mathrm{\#}g=\mathrm{\#}f`$. There exists a unique $`\theta =\theta (g,f)^r`$ such that

* $`f=𝚁_\theta ^{}(g)`$;
* if $`f=𝚁_\phi ^{}(g)`$ for some $`\theta \phi ^r`$ then $`|\phi |<|\theta |`$ and $`|\phi ||\theta |(mod2)`$.

Given in addition $`h_+^{m|n}`$ with $`hgf`$, $`\theta (h,f)=\theta (h,g)+\theta (g,f)`$.

###### Proof.

We just explain how to construct $`\theta `$, and leave the rest of the proof to the reader. Define $`mi_1<\mathrm{}<i_r<0<j_r<\mathrm{}<j_1n`$ such that $`g(i_s)=g(j_s)`$ for each $`s=1,\mathrm{},r`$, and $`mi_1^{}<\mathrm{}<i_r^{}<0<j_r^{}<\mathrm{}<j_1^{}n`$ such that $`f(i_s^{})=f(j_s^{})`$ for each $`s=1,\mathrm{},r`$. For $`0sr`$, let

$$g_s=g+\underset{t=1}{\overset{s}{}}(g(i_s)f(i_s^{}))(d_{i_s}d_{j_s}),$$

so $`g_0=g`$ and $`(g_r)^+=f`$. Now for each $`s=1,\mathrm{},r`$, let $`\theta _s`$ be the unique non-negative integer such that $`𝚁_{i_s,j_s}^{\theta _s}(g_{s1})=g_s`$, recalling (3.26), and take $`\theta =\theta (g,f)`$ to be the tuple $`(\theta _1,\mathrm{},\theta _r)`$. ∎

Using Lemma 3.42, we can introduce a length function on $`_+^{m|n}`$. Suppose to start with that $`gf`$. Let $`\mathrm{}(g,f)=|\theta (g,f)|`$, where $`\theta (g,f)`$ is the tuple defined in the lemma. Notice that if $`hgf`$, then

$$\mathrm{}(h,g)+\mathrm{}(g,f)=\mathrm{}(h,f),$$
(3.43)
as follows from the stronger fact that $`\theta (h,g)+\theta (g,f)=\theta (h,f)`$ established by Lemma 3.42. This allows us to extend the notion of length to arbitrary $`g,f_+^{m|n}`$ with $`\mathrm{wt}(g)=\mathrm{wt}(f)`$: pick $`h_+^{m|n}`$ with $`hg`$ and $`hf`$ and set $`\mathrm{}(g,f)=\mathrm{}(h,f)\mathrm{}(h,g)`$. To check that this is well-defined, suppose $`h^{}_+^{m|n}`$ also satisfies $`h^{}g`$ and $`h^{}f`$. Choose another $`k_+^{m|n}`$ with $`kh`$ and $`kh^{}`$. Then using (3.43),

$`\mathrm{}(h,f)\mathrm{}(h,g)`$ $`=(\mathrm{}(k,h)+\mathrm{}(h,f))(\mathrm{}(k,h)+\mathrm{}(h,g))`$
$`=\mathrm{}(k,f)\mathrm{}(k,g)=\mathrm{}(h^{},f)\mathrm{}(h^{},g),`$
as required. So we have now defined $`\mathrm{}(g,f)`$, the length of $`f`$ relative to $`g`$, for arbitrary $`g,f_+^{m|n}`$ with $`\mathrm{wt}(g)=\mathrm{wt}(f)`$. It is immediate from the definition that (3.43) holds for all $`h,g,f_+^{m|n}`$ with $`\mathrm{wt}(h)=\mathrm{wt}(g)=\mathrm{wt}(f)`$.

Finally we can somewhat arbitrarily introduce an absolute notion of length. For each weight $`\gamma `$ of $`^{m|n}`$, we fix a choice of “origin” $`o_\gamma _+^{m|n}`$ with $`\mathrm{wt}(o_\gamma )=\gamma `$. Then for any $`f_+^{m|n}`$, we define

$$\mathrm{}(f):=\mathrm{}(o_\gamma ,f)$$
(3.44)
where $`\gamma =\mathrm{wt}(f)`$. The important thing is that if $`\mathrm{wt}(g)=\mathrm{wt}(f)`$, then $`\mathrm{}(g,f)=\mathrm{}(f)\mathrm{}(g)`$, so we can recover the length of $`f`$ relative to $`g`$ from the absolute lengths of $`f`$ and $`g`$. In this notation, Lemma 3.42(ii) and Corollary 3.39(ii) combine to show:

###### Corollary 3.45.

For $`g,f_+^{m|n}`$ with $`gf`$, the polynomial $`l_{g,f}(q^1)`$ belongs to $`q^{\mathrm{}(f)\mathrm{}(g)}[q^2]`$, and the coefficient of $`q^{\mathrm{}(f)\mathrm{}(g)}`$ is $`1`$.

## 4. Representations of $`𝔤𝔩(m|n)`$

We now relate the combinatorics developed in sections 2 and 3 to two natural categories $`𝒪_{m|n}`$ and $`_{m|n}`$ of representations of $`𝔤𝔩(m|n)`$. For basic notions regarding Lie superalgebras, we follow \[Ka1\]. We denote the parity of a vector $`v`$ in a vector superspace by $`\overline{v}_2`$. For a Lie superalgebra $`𝔤=𝔤_{\overline{0}}𝔤_{\overline{1}}`$ and $`𝔤`$-supermodules $`M,N`$, the space $`\mathrm{Hom}_𝔤(M,N)`$ has a canonical $`_2`$-grading, and the category of all $`𝔤`$-supermodules is a superadditive category in the sense of \[M, ch.3,$`\mathrm{\S }`$2.7\]. We will use the notation $`MN`$ as opposed to the usual $`MN`$ to indicate that there is an even isomorphism between $`M`$ and $`N`$. Also $`\mathrm{\Pi }`$ denotes the parity change functor.

§4-a. Two categories. From now on, we let $`𝔤`$ denote the Lie superalgebra $`𝔤𝔩(m|n)`$. So $`𝔤`$ consists of $`(m+n)\times (m+n)`$ matrices over $``$, where we label rows and columns of such matrices by the ordered index set $`I(m|n)=\{m,\mathrm{},1,1,\mathrm{},n\}`$ as in the introduction. For $`iI(m|n)`$, let $`\overline{i}=\overline{0}`$ if $`i>0`$ and $`\overline{1}`$ if $`i<0`$. Then, the parity of the $`ij`$-matrix unit $`e_{i,j}𝔤`$ is $`\overline{i}+\overline{j}`$, and the superbracket satisfies

$$[e_{i,j},e_{k,l}]=\delta _{j,k}e_{i,l}(1)^{(\overline{i}+\overline{j})(\overline{k}+\overline{l})}\delta _{i,l}e_{k,j}.$$
(4.1)
Note that the subalgebra $`𝔤_{\overline{0}}`$ of $`𝔤`$ is isomorphic to $`𝔤𝔩(m)𝔤𝔩(n)`$. We will need some other important subalgebras: let $`𝔥`$ denote the standard Cartan subalgebra of $`𝔤`$ consisting of all diagonal matrices, let $`𝔟`$ be the standard Borel subalgebra of all upper triangular matrices, and let $`𝔭=𝔤_{\overline{0}}+𝔟`$.

For $`\lambda 𝔥^{}`$ and a $`𝔤`$-supermodule $`M`$, we define the $`\lambda `$-weight space $`M_\lambda `$ of $`M`$ with respect to $`𝔥`$ as usual: $`M_\lambda =\{mM|hm=\lambda (h)m\text{ for all }h𝔥\}.`$ Given a $`𝔤`$-supermodule $`M`$ such that $`M=_{\lambda 𝔥^{}}M_\lambda `$, we can consider the graded dual $`M^{}:=_{\lambda 𝔥^{}}\mathrm{Hom}_{}(M_\lambda ,)`$ with the usual $`_2`$-grading and $`𝔤`$-action. Twisting the $`𝔤`$-action on $`M^{}`$ with the automorphism $`XX^{st}`$, where $`st:𝔤𝔤`$ is the supertranspose $`e_{i,j}(1)^{\overline{i}(\overline{i}+\overline{j})}e_{j,i}`$, we obtain a new $`𝔤`$-supermodule denoted $`M^\tau `$. If all the weight spaces are finite dimensional, then there are natural isomorphisms $`(M^{})^{}M`$ and $`(M^\tau )^\tau M`$.

Let $`\{\delta _i\}_{iI(m|n)}`$ be the basis for $`𝔥^{}`$ dual to the basis $`\{e_{i,i}\}_{iI(m|n)}`$ for $`𝔥`$. Define a symmetric bilinear form $`(.|.)`$ on $`𝔥^{}`$ by declaring that $`(\delta _i|\delta _j)=(1)^{\overline{i}}\delta _{i,j}`$. The Weyl group $`W`$ associated to the reductive Lie algebra $`𝔤_{\overline{0}}`$ can be identified with the symmetric group $`S_{m|n}`$ from 2. It acts linearly on $`𝔥^{}`$ so that $`x\delta _i=\delta _{xi}`$ for $`xW,iI(m|n)`$. As before, we write $`w_0`$ for the longest element of $`W`$. We will also need the dot action of $`W`$ on $`X(m|n)`$ defined by $`x\lambda :=x(\lambda +\rho )\rho `$ where

$$\rho =\underset{iI(m|n)}{}i\delta _i.$$
(4.2)
The root system of $`𝔤`$ is the set $`R=\{\delta _i\delta _j|i,jI(m|n),ij\}.`$ We write $`R=R_{\overline{0}}R_{\overline{1}}`$ where $`R_{\overline{0}}`$ consists of all even roots $`\delta _i\delta _j`$ with $`\overline{i}=\overline{j}`$, and $`R_{\overline{1}}`$ consists of the remaining odd roots. Corresponding to the Borel subalgebra $`𝔟`$, we have the standard choice of positive roots $`R^+=R_{\overline{0}}^+R_{\overline{1}}^+=\{\delta _i\delta _j|i,jI(m|n),i<j\}`$. The dominance ordering on $`𝔥^{}`$ is defined by $`\lambda \mu `$ if $`(\mu \lambda )`$ is an $``$-linear combination of positive roots.

From now on, we will restrict our attention to the integral weights, i.e. the weights belonging to the subset $`X(m|n)`$ of $`𝔥^{}`$ consisting of all $``$-linear combinations of $`\{\delta _i\}_{iI(m|n)}`$. For $`\lambda =_{iI(m|n)}\lambda _i\delta _iX(m|n)`$, we define its parity

$$\overline{\lambda }:=\lambda _m+\mathrm{}+\lambda _2+\lambda _1_2.$$
(4.3)
Let $`X^+(m|n)`$ be the set of all dominant integral weights, namely, the $`\lambda =_{iI(m|n)}\lambda _i\delta _iX(m|n)`$ such that $`\lambda _m\mathrm{}\lambda _1`$ and $`\lambda _1\mathrm{}\lambda _n`$. Define a bijection

$$X(m|n)^{m|n},\lambda f_\lambda $$
(4.4)
where $`f_\lambda ^{m|n}`$ is the function defined by $`f_\lambda (i)=(\lambda +\rho |\delta _i)`$ for $`iI(m|n)`$. Under this bijection, $`X^+(m|n)`$ maps onto $`_+^{m|n}`$, see 3. Also $`f_{x\lambda }=f_\lambda x^1`$ for each $`xW=S_{m|n}`$, i.e. the dot action of $`W`$ on $`X(m|n)`$ corresponds to the action of $`S_{m|n}`$ on $`^{m|n}`$ introduced in 2. Now we lift all the remaining combinatorial definitions involving $`^{m|n}`$ directly to $`X(m|n)`$. For instance, recalling (2.3), we define the degree of atypicality $`\mathrm{\#}\lambda `$ of $`\lambda X(m|n)`$ by $`\mathrm{\#}\lambda :=\mathrm{\#}f_\lambda `$; this is the same notion as in \[S3, (1.1)\]. Similarly, let $`\mathrm{wt}(\lambda ):=\mathrm{wt}(f_\lambda )`$, an element of the weight lattice $`P`$, see (2.2), and write $`\lambda \mu `$ if $`f_\lambda f_\mu `$, see 2. This ordering on $`X(m|n)`$ plays the role of the Bruhat ordering, see e.g. Theorem 4.31(ii) below. It should not be confused with the dominance ordering $``$: we have that $`\lambda \mu \lambda \mu `$ but not conversely.

We are ready to introduce two categories of representations of $`𝔤`$. All the results summarized in the remainder of this subsection are taken from \[B2, section 7\], where they are deduced from a general framework for representations of graded Lie superalgebras similar to that of Soergel \[So2\].

The first category is denoted $`𝒪_{m|n}`$, and is the (integral weight) analogue of the \[BGG\] category $`𝒪`$ for a semisimple Lie algebra. By definition, $`𝒪_{m|n}`$ is the category of all finitely generated $`𝔤`$-supermodules $`M`$ that are locally finite dimensional over $`𝔟`$ and satisfy

$$M=\underset{\lambda X(m|n)}{}M_\lambda .$$
(4.5)
An object $`P𝒪_{m|n}`$ is projective if every (not necessarily even) morphism from $`P`$ to a quotient of an object $`M𝒪_{m|n}`$ lifts to a morphism from $`P`$ to $`M`$. By \[B2, Lemma 7.3\], the category $`𝒪_{m|n}`$ has enough projectives, i.e. every object is a quotient of a projective object. Moreover, $`𝒪_{m|n}`$ is finite, i.e. every object has a composition series. For each $`\lambda X(m|n)`$, we have the Verma module

$$M(\lambda ):=U(𝔤)_{U(𝔟)}_\lambda ,$$
(4.6)
where $`_\lambda `$ is the one dimensional $`𝔟`$-module of weight $`\lambda `$ concentrated in degree $`\overline{\lambda }`$. The significance of the choice of parity here will be explained in 4 below. As usual, $`M(\lambda )`$ has a unique irreducible quotient denoted $`L(\lambda )`$, and $`\{L(\lambda )\}_{\lambda X(m|n)}`$ is a complete set of pairwise non-isomorphic irreducibles in $`𝒪_{m|n}`$.

We say that an object $`M𝒪_{m|n}`$ has a Verma flag if it has a filtration $`0=M_0<\mathrm{}<M_r=M`$ such that each $`M_i/M_{i1}`$ is $`M(\lambda _i)`$ for some $`\lambda _iX(m|n)`$. If $`M`$ has a Verma flag and $`\mu X(m|n)`$, we let

$$(M:M(\mu ))=dim\mathrm{Hom}_{𝒪_{m|n}}(M,M(\mu )^\tau ).$$
(4.7)
By \[B2, (6.1)\], this computes the number of subquotients of a Verma flag of $`M`$ that are $`M(\mu )`$. There is an obvious refinement of these multiplicities: for $`p_2`$,

$$(M:M(\mu ))_p:=dim\mathrm{Hom}_{𝒪_{m|n}}(M,M(\mu )^\tau )_p$$
(4.8)
counts the number of subquotients of a Verma flag of $`M`$ that are $`\mathrm{\Pi }^pM(\mu )`$.

By \[B2, Theorem 6.3\], there is for each $`\lambda X(m|n)`$ a unique (up to even isomorphism) indecomposable module $`T(\lambda )𝒪_{m|n}`$ satisfying the following properties:

* $`T(\lambda )`$ has a Verma flag starting with $`M(\lambda )`$ at the bottom;
* $`\mathrm{Ext}_{𝒪_{m|n}}^1(M(\mu ),T(\lambda ))=0`$ for all $`\mu X(m|n)`$.

Moreover, by \[B2, (7.4)\], the multiplicity of $`M(\mu )`$ in a Verma flag of $`T(\lambda )`$ is equal to the composition multiplicity of $`L(\lambda 2\rho )`$ in $`M(\mu 2\rho )`$, i.e.

$$(T(\lambda ):M(\mu ))=[M(\mu 2\rho ):L(\lambda 2\rho )],$$
(4.9)
for $`\lambda ,\mu X(m|n)`$. In particular, $`(T(\lambda ):M(\lambda ))=1`$ and $`(T(\lambda ):M(\mu ))=0`$ unless $`\mu \lambda `$. Consequently, we call $`T(\lambda )`$ the infinite dimensional tilting module of highest weight $`\lambda `$. Finally, note that for every $`\lambda X(m|n)`$,

$$f_{\lambda 2\rho }=f_\lambda ,$$
(4.10)
so the involution $`\lambda \lambda 2\rho `$ on $`X(m|n)`$ appearing in the formula (4.9) corresponds to the involution $`ff`$ on $`^{m|n}`$ in $`\rho `$-shifted notation.

The second category we shall consider is the category $`_{m|n}`$ of all finite dimensional $`𝔤`$-supermodules satisfying (4.5). Again, this is finite and has enough projectives. As is well-known, the irreducible finite dimensional $`𝔤_{\overline{0}}`$-supermodules with integral highest weights are parametrized by the set $`X^+(m|n)`$. For $`\lambda X^+(m|n)`$, let us write $`L^{}(\lambda )`$ for the corresponding irreducible highest weight representation of $`𝔤_{\overline{0}}`$ concentrated in degree $`\overline{\lambda }`$. Then, for each $`\lambda X^+(m|n)`$, we have the Kac module

$$K(\lambda ):=U(𝔤)_{U(𝔭)}L^{}(\lambda ),$$
(4.11)
where we are viewing $`L^{}(\lambda )`$ here as a $`𝔭`$-supermodule with elements of $`𝔭_{\overline{0}}=𝔤_{\overline{0}}`$ acting as given and elements of $`𝔭_{\overline{1}}`$ acting trivially. For $`\lambda X^+(m|n)`$, the irreducible module $`L(\lambda )`$ defined earlier can also be realized as the unique irreducible quotient of $`K(\lambda )`$, and $`\{L(\lambda )\}_{\lambda X^+(m|n)}`$ is a complete set of pairwise non-isomorphic irreducibles in $`_{m|n}`$.

When working in $`_{m|n}`$, we will talk about Kac flags in place of Verma flags. If $`M`$ has a Kac flag, the number of subquotients of a Kac flag of $`M`$ that are $`K(\mu )`$ is denoted $`(M:K(\mu ))`$, and can be computed by

$$(M:K(\mu ))=dim\mathrm{Hom}_{_{m|n}}(M,K(\mu )^\tau ).$$
(4.12)
Like in (4.8), there is a refinement denoted $`(M:K(\mu ))_p`$ for $`p_2`$, counting the number of subquotients of a Kac flag of $`M`$ that are $`\mathrm{\Pi }^pK(\mu )`$. By \[B2, (7.6)\] or \[Z, Proposition 2.5\], the projective cover $`P(\lambda )`$ of $`L(\lambda )`$ in the category $`_{m|n}`$ has a Kac flag with $`K(\lambda )`$ appearing at the top, satisfying the BGG reciprocity

$$(P(\lambda ):K(\mu ))=[K(\mu ):L(\lambda )].$$
(4.13)
There are also indecomposable tilting modules in category $`_{m|n}`$, denoted $`U(\lambda )`$ for $`\lambda X^+(m|n)`$. Here, by \[B2, Theorem 6.3\], $`U(\lambda )_{m|n}`$ is the unique (up to even isomorphism) indecomposable object such that

* $`U(\lambda )`$ has a Kac flag starting with $`K(\lambda )`$ at the bottom;
* $`\mathrm{Ext}_{_{m|n}}^1(K(\mu ),U(\lambda ))=0`$ for all $`\mu X^+(m|n)`$.

Let $`\beta =n(\delta _m+\mathrm{}+\delta _1)m(\delta _1+\mathrm{}+\delta _n)`$ be the sum of the positive odd roots. Then, by \[B2, (7.7)–(7.8)\] and parity considerations, we have that

$`K(\lambda )^{}`$ $`K(\beta w_0\lambda ),`$ (4.14)
$`U(\lambda )^{}`$ $`P(\beta w_0\lambda ).`$ (4.15)
Note (4.13), (4.14) and (4.15) together imply

$$(U(\lambda ):K(\mu ))=[K(\beta w_0\mu ):L(\beta w_0\lambda )],$$
(4.16)
for $`\lambda ,\mu X^+(m|n)`$. In particular, $`(U(\lambda ):K(\lambda ))=1`$ and $`(U(\lambda ):K(\mu ))=0`$ unless $`\mu \lambda `$. Accordingly, we will call $`U(\lambda )`$ the finite dimensional tilting module of highest weight $`\lambda `$. We remark finally that

$$f_{\beta w_0\lambda }=f_\lambda w_0(m+n+1)\mathrm{𝟏},$$
(4.17)
where $`\mathrm{𝟏}^{m|n}`$ is the constant function $`i1`$. Thus, up to a constant shift which can usually be ignored, the involution $`\lambda \beta w_0\lambda `$ on $`X^+(m|n)`$ appearing in the formula (4.16) corresponds to the involution $`ffw_0`$ on $`_+^{m|n}`$.

§4-b. Translation functors. We need some basic facts about central characters. Let $`Z`$ be the (even) center of $`U(𝔤)`$. The fixed choices of $`𝔥𝔟`$ determine a Harish-Chandra homomorphism $`\phi :ZU(𝔥),`$ see \[Dix, 7.4.3\]. Each $`\lambda 𝔥^{}`$ yields a central character $`\chi _\lambda `$ defined by $`\chi _\lambda (z)=\lambda (\phi (z))`$. To parametrize the integral central characters, i.e. the $`\chi _\lambda `$ for $`\lambda X(m|n)`$, we use the following consequence of results of Sergeev \[Sg2\], see \[S3, Corollary 1.9\]:

###### Lemma 4.18.

Given $`\lambda ,\mu X(m|n)`$, we have that $`\chi _\lambda =\chi _\mu `$ if and only if $`\mathrm{wt}(\lambda )=\mathrm{wt}(\mu )`$ (where $`\mathrm{wt}(\lambda )=\mathrm{wt}(f_\lambda )`$, see (2.2)).

For each central character $`\chi `$, let $`𝒪_\chi `$ denote the full subcategory of $`𝒪_{m|n}`$ consisting of the modules all of whose composition factors have central character $`\chi `$. We have the block decomposition

$$𝒪_{m|n}=\underset{\chi }{}𝒪_\chi $$

as $`\chi `$ runs over all integral central characters. Lemma 4.18 shows that we can parametrize the integral characters $`\chi `$ instead by the weights $`\gamma P`$ arising non-trivially in the tensor space $`𝒯^{m|n}`$ of 2. Let us introduce some notation to do this formally. Suppose that $`\gamma P`$. Let $`𝒪_\gamma =\{0\}`$ if $`\gamma `$ is not a weight of $`𝒯^{m|n}`$, else let $`𝒪_\gamma =𝒪_{\chi _\lambda }`$ where $`\lambda X(m|n)`$ is such that $`\mathrm{wt}(\lambda )=\gamma `$. Then, we can rewrite the above block decomposition as

$$𝒪_{m|n}=\underset{\gamma P}{}𝒪_\gamma ,$$
(4.19)
where $`𝒪_\gamma `$ is non-zero if and only if $`\gamma `$ is a weight of $`𝒯^{m|n}`$. We let $`\mathrm{pr}_\gamma :𝒪_{m|n}𝒪_\gamma `$ be the natural projection functor. In an entirely similar way, we have the block decomposition of $`_{m|n}`$,

$$_{m|n}=\underset{\gamma P}{}_\gamma ,$$
(4.20)
where this time $`_{m|n}`$ is non-zero if and only if $`\gamma `$ is a weight of $`^{m|n}`$, see 3.

Let $`V`$ be the natural $`𝔤`$-supermodule. So, $`V`$ is the vector superspace on basis $`\{v_i\}_{iI(m|n)}`$, where $`\overline{v}_i:=\overline{i}`$, and the action of the matrix unit $`e_{i,j}𝔤`$ is given by $`e_{i,j}v_k=\delta _{j,k}v_i.`$ For $`r0`$, let $`S^rV`$ be the $`r`$th supersymmetric power of $`V`$, a finite dimensional irreducible representation of $`𝔤`$. Let $`S^rV^{}=S^r(V^{})(S^rV)^{}`$. For $`a`$ and $`r0`$, we define additive functors $`F_a^{(r)},E_a^{(r)}:𝒪_{m|n}𝒪_{m|n}`$ as follows. It suffices by additivity to define them on objects belonging to $`𝒪_\gamma `$ for each $`\gamma P`$. So if $`M𝒪_\gamma `$, we let

$`F_a^{(r)}M`$ $`:=\mathrm{pr}_{\gamma r(\epsilon _a\epsilon _{a+1})}(MS^rV),`$ (4.21)
$`E_a^{(r)}M`$ $`:=\mathrm{pr}_{\gamma +r(\epsilon _a\epsilon _{a+1})}(MS^rV^{}).`$ (4.22)
On a morphism $`\theta :MN`$, $`F_a^{(r)}\theta `$ and $`E_a^{(r)}\theta `$ are defined simply to be the restrictions of the natural maps $`\theta \mathrm{id}`$. Clearly, the restrictions of $`F_a^{(r)}`$ and $`E_a^{(r)}`$ to $`_{m|n}`$ give functors $`F_a^{(r)},E_a^{(r)}:_{m|n}_{m|n}`$ too. The first well-known lemma gives the elementary properties.

###### Lemma 4.23.

On either category $`𝒪_{m|n}`$ or $`_{m|n}`$, $`F_a^{(r)}`$ and $`E_a^{(r)}`$ are exact functors, they commute with the $`\tau `$-duality, and are both left and right adjoint to each other.

The next lemma is also quite standard, though we have included a proof since we wish to keep track of parity information.

###### Lemma 4.24.

Let $`\nu _1,\mathrm{},\nu _N`$ be the set of weights of $`S^rV`$ ordered so that $`\nu _i>\nu _ji<j`$. Let $`\lambda X(m|n)`$.

* $`M(\lambda )S^rV`$ has a multiplicity-free Verma flag with subquotients $`M(\lambda +\nu _1),\mathrm{},M(\lambda +\nu _N)`$ in order from bottom to top.
* $`M(\lambda )S^rV^{}`$ has a multiplicity-free Verma flag with subquotients $`M(\lambda \nu _1),\mathrm{},M(\lambda \nu _N)`$ in order from top to bottom.

###### Proof.

We prove (i), (ii) being entirely similar. By the tensor identity,

$$M(\lambda )S^rV=(U(𝔤)_{U(𝔟)}_\lambda )S^rVU(𝔤)_{U(𝔟)}(_\lambda S^rV).$$

So it suffices by exactness of the functor $`U(𝔤)_{U(𝔟)}\mathrm{?}`$ to show that $`M:=_\lambda S^rV`$ has a filtration $`0=M_0<M_1<\mathrm{}<M_N=M`$ as a $`𝔟`$-module with $`M_i/M_{i1}_{\lambda +\nu _i}`$. Let $`x_1,\mathrm{},x_N`$ be a basis for $`S^rV`$, where $`x_i`$ is of weight $`\nu _i`$. Then $`1x_i_\lambda S^rV`$ is of weight $`\lambda +\nu _i`$ and degree $`\overline{\lambda }+\overline{x}_i=\overline{\lambda }+\overline{\nu }_i=\overline{\lambda +\nu _i}`$ (recall (4.3)). So taking $`M_i`$ to be the subspace spanned by $`1x_1,\mathrm{},1x_i`$ gives the required filtration. ∎

###### Corollary 4.25.

Let $`\lambda X(m|n)`$ and $`a`$. Let $`(\sigma _m,\mathrm{},\sigma _1,\sigma _1,\mathrm{},\sigma _n)`$ be the $`a`$-signature of $`f_\lambda `$, see (2.29).

* $`F_a^{(r)}M(\lambda )`$ has a multiplicity-free Verma flag with subquotients $`M(\lambda +\delta _{i_1}+\mathrm{}+\delta _{i_r})`$ for all distinct $`i_1,\mathrm{},i_rI(m|n)`$ such that $`\sigma _{i_1}=\mathrm{}=\sigma _{i_r}=+`$.
* $`E_a^{(r)}M(\lambda )`$ has a multiplicity-free Verma flag with subquotients $`M(\lambda \delta _{j_1}\mathrm{}\delta _{j_r})`$ for all distinct $`j_1,\mathrm{},j_rI(m|n)`$ such that $`\sigma _{j_1}=\mathrm{}=\sigma _{j_r}=`$.

In both (i) and (ii), the Verma flag can be chosen so that subquotients appear in order refining dominance, most dominant at the bottom.

###### Proof.

The Verma module $`M(\lambda )`$ has central character $`\chi _\lambda `$ so belongs to $`𝒪_{\mathrm{wt}(\lambda )}`$ by Lemma 4.18. Applying the exact functor $`\mathrm{pr}_{\mathrm{wt}(\lambda )r(\epsilon _a\epsilon _{a+1})}`$ to the filtration in Lemma 4.24(i), we deduce that $`F_a^{(r)}M(\lambda )`$ has a Verma flag with subquotients being the $`M(\lambda +\nu _i)`$ such that $`\mathrm{wt}(\lambda +\nu _i)=\mathrm{wt}(\lambda )r(\epsilon _a\epsilon _{a+1})`$. This implies that $`\nu _i=\delta _{i_1}+\mathrm{}+\delta _{i_s}`$ for distinct $`i_1,\mathrm{},i_sI(m|n)`$ such that $`\sigma _{i_1}=\mathrm{}=\sigma _{i_s}=+`$, giving (i). Part (ii) is similar. ∎

There is an analogous statement in the finite dimensional setting.

###### Corollary 4.26.

Let $`\lambda X^+(m|n)`$ and $`a`$. Let $`(\sigma _m,\mathrm{},\sigma _1,\sigma _1,\mathrm{},\sigma _n)`$ be the $`a`$-signature of $`f_\lambda `$, see (2.29).

* $`F_a^{(r)}K(\lambda )`$ has a multiplicity-free Kac flag with subquotients $`K(\lambda +\delta _{i_1}+\mathrm{}+\delta _{i_r})`$ for all distinct $`i_1,\mathrm{},i_rI(m|n)`$ such that $`\lambda +\delta _{i_1}+\mathrm{}+\delta _{i_r}X^+(m|n)`$ and $`\sigma _{i_1}=\mathrm{}=\sigma _{i_r}=+`$.
* $`E_a^{(r)}K(\lambda )`$ has a multiplicity-free Kac flag with subquotients $`K(\lambda \delta _{j_1}\mathrm{}\delta _{j_r})`$ for all distinct $`j_1,\mathrm{},j_rI(m|n)`$ such that $`\lambda \delta _{j_1}\mathrm{}\delta _{j_r}X^+(m|n)`$ and $`\sigma _{j_1}=\mathrm{}=\sigma _{j_r}=`$.

In both (i) and (ii), the Kac flag can be chosen so that subquotients appear in order refining dominance, most dominant at the bottom.

###### Proof.

We prove (i). By universal properties, $`K(\lambda )`$ is the largest finite dimensional quotient of $`M(\lambda )`$. So since $`F_a^{(r)}`$ is exact, $`F_a^{(r)}K(\lambda )`$ is a quotient of $`F_a^{(r)}M(\lambda )`$ and Corollary 4.25 implies that $`F_a^{(r)}K(\lambda )`$ has a filtration with subquotients being finite dimensional quotients of $`M(\lambda +\delta _{i_1}+\mathrm{}+\delta _{i_r})`$ for all distinct $`i_1,\mathrm{},i_rI(m|n)`$ such that $`\sigma _{i_1}=\mathrm{}=\sigma _{i_r}=+`$. But such a quotient is zero unless $`\lambda +\delta _{i_1}+\mathrm{}+\delta _{i_r}X^+(m|n)`$. Hence, $`F_a^{(r)}K(\lambda )`$ has a filtration with subquotients being quotients of the Kac modules $`K(\lambda +\delta _{i_1}+\mathrm{}+\delta _{i_r})`$ for all distinct $`i_1,\mathrm{},i_rI(m|n)`$ such that $`\lambda +\delta _{i_1}+\mathrm{}+\delta _{i_r}X^+(m|n)`$ and $`\sigma _{i_1}=\mathrm{}=\sigma _{i_r}=+`$. Finally the fact that each factor is actually isomorphic to the corresponding Kac module, rather than a proper quotient, follows by a character calculation using the Kac character formula for $`K(\mu )`$, the Pieri formulae \[Mac, (5.16),(5.17)\] and Lemma 4.18. ∎

###### Corollary 4.27.

Let $`a`$ and $`r1`$.

* For each $`\lambda X(m|n)`$, each indecomposable summand of $`F_a^{(r)}T(\lambda )`$ or of $`E_a^{(r)}T(\lambda )`$ is $`T(\mu )`$ for $`\mu X(m|n)`$.
* For each $`\lambda X^+(m|n)`$, each indecomposable summand of $`F_a^{(r)}U(\lambda )`$ or of $`E_a^{(r)}U(\lambda )`$ is $`U(\mu )`$ for $`\mu X^+(m|n)`$.

###### Proof.

We prove (i) for $`E_a^{(r)}`$, the other cases being similar. Let $`T`$ be an indecomposable summand of $`E_a^{(r)}T(\lambda )`$. We need to show that it has a Verma flag with subquotients $`M(\nu )`$ for various $`\nu X(m|n)`$, and that $`\mathrm{Ext}_{𝒪_{m|n}}^1(M(\mu ),T)=0`$ for all $`\mu X(m|n)`$. The first statement is immediate since $`E_a^{(r)}T(\lambda )`$ has such a Verma flag by Corollary 4.25, and summands of modules with a Verma flag also have a Verma flag, see \[B2, Corollary 4.3\]. For the second statement, Lemma 4.23 and a standard argument, see e.g. \[J2, I.4.4\], shows that $`\mathrm{Ext}_{𝒪_{m|n}}^1(M(\mu ),E_a^{(r)}T(\lambda ))\mathrm{Ext}_{𝒪_{m|n}}^1(F_a^{(r)}M(\mu ),T(\lambda )).`$ To see that the right hand side is zero, note that $`F_a^{(r)}M(\mu )`$ has a Verma flag by Corollary 4.25. By induction on length using the long exact sequence and the defining property (T2) of $`T(\lambda )`$, $`\mathrm{Ext}_{𝒪_{m|n}}^1(M,T(\lambda ))=0`$ for every $`M𝒪_{m|n}`$ with a Verma flag. ∎

Let $`𝒪_{m|n}^\mathrm{\Delta }`$ be the full subcategory of $`𝒪_{m|n}`$ consisting of all modules possessing a Verma flag. Let $`K(𝒪_{m|n}^\mathrm{\Delta })`$ denote the Grothendieck group of the superadditive category $`𝒪_{m|n}^\mathrm{\Delta }`$ in the sense of \[BK, $`\mathrm{\S }`$2-c\]. Note $`K(𝒪_{m|n}^\mathrm{\Delta })`$ is a free $``$-module on basis $`\{[M(\lambda )]\}_{\lambda X(m|n)}`$. Similarly, let $`_{m|n}^\mathrm{\Delta }`$ be the full subcategory of $`_{m|n}`$ consisting of all modules possessing a Kac flag, and let $`K(_{m|n}^\mathrm{\Delta })`$ denote its Grothendieck group. So $`K(_{m|n}^\mathrm{\Delta })`$ is a free $``$-module on basis $`\{[K(\lambda )]\}_{\lambda X^+(m|n)}`$. In view of Corollary 4.25 and 4.26, the functors $`F_a^{(r)}`$ and $`E_a^{(r)}`$ map objects in $`𝒪_{m|n}^\mathrm{\Delta }`$ resp. $`_{m|n}^\mathrm{\Delta }`$ to objects in $`𝒪_{m|n}^\mathrm{\Delta }`$ resp. $`_{m|n}^\mathrm{\Delta }`$. Moreover, they preserve short exact sequences in $`𝒪_{m|n}^\mathrm{\Delta }`$ resp. $`_{m|n}^\mathrm{\Delta }`$. Hence they induce $``$-linear operators on $`K(𝒪_{m|n}^\mathrm{\Delta })`$ and on $`K(_{m|n}^\mathrm{\Delta })`$.

Now we make the connection to the modules $`𝒯^{m|n}`$ and $`^{m|n}`$ from sections 2 and 3 of the article. Actually we need to specialize these modules at $`q=1`$. So let $`𝒯_{[q,q^1]}^{m|n}`$ be the $`[q,q^1]`$-lattice in $`𝒯^{m|n}`$ spanned by $`\{M_f\}_{f^{m|n}}`$, in the notation of 2. Let $`_{[q,q^1]}^{m|n}`$ be the $`[q,q^1]`$-lattice in $`^{m|n}`$ spanned by $`\{K_f\}_{f_+^{m|n}}`$, in the notation of 3. Viewing $``$ as a $`[q,q^1]`$-module so that $`q`$ acts as $`1`$, we define

$`𝒯_{}^{m|n}`$ $`:=_{[q,q^1]}𝒯_{[q,q^1]}^{m|n},`$
$`_{}^{m|n}`$ $`:=_{[q,q^1]}_{[q,q^1]}^{m|n}.`$
We write $`M_f(1)`$ (resp. $`K_f(1)`$) for the basis element $`1M_f`$ of $`𝒯_{}^{m|n}`$ (resp. $`1K_f`$ of $`_{}^{m|n}`$). Similarly, we define $`T_f(1)=1T_f`$ and $`U_f(1)=1U_f`$ (in case of $`T_f(1)`$, recall that as a consequence of Conjecture 2.28 we expect it is a finite sum of $`K_g(1)`$’s so belongs to $`𝒯_{}^{m|n}`$, but without this we mean here to work in the completion $`\widehat{𝒯}_{}^{m|n}`$ constructed as in 2).

Note the generators $`E_a^{(r)}`$ and $`F_a^{(r)}`$ of $`𝒰=U_q(𝔤𝔩_{\mathrm{}})`$ specialize at $`q=1`$ to the usual divided powers $`E_a^r/r!`$ and $`F_a^r/r!`$ in the Chevalley generators of the Lie algebra $`𝔤𝔩_{\mathrm{}}`$, so we can view $`𝒯_{}^{m|n}`$ resp. $`_{}^{m|n}`$ as modules over the Kostant $``$-form $`𝒰_{}`$ for the universal enveloping algebra $`U(𝔤𝔩_{\mathrm{}})`$.

###### Theorem 4.28.

Identify $`K(𝒪_{m|n}^\mathrm{\Delta })`$ with $`𝒯_{}^{m|n}`$ via the $``$-module isomorphism

$$i:K(𝒪_{m|n}^\mathrm{\Delta })𝒯_{}^{m|n},[M(\lambda )]M_{f_\lambda }(1).$$

Then, the representation theoretically defined operators $`F_a^{(r)},E_a^{(r)}`$ act in the same way as the Chevalley generators $`F_a^{(r)},E_a^{(r)}`$ of $`𝒰_{}`$.

###### Proof.

Corollary 4.25 shows that the operators induced by the functors $`F_a^{(r)},E_a^{(r)}`$ act on $`[M(\lambda )]K(𝒪_{m|n}^\mathrm{\Delta })`$ in exactly the same way on $`K(𝒪_{m|n}^\mathrm{\Delta })`$ as $`F_a^{(r)},E_a^{(r)}𝒰_{}`$ act on $`M_{f_\lambda }(1)𝒯_{}^{m|n}`$. ∎

An entirely similar argument, using Corollary 4.26 instead, gives the analogous theorem for category $`_{m|n}^\mathrm{\Delta }`$:

###### Theorem 4.29.

Identify $`K(_{m|n}^\mathrm{\Delta })`$ with $`_{}^{m|n}`$ via the $``$-module isomorphism

$$j:K(_{m|n}^\mathrm{\Delta })_{}^{m|n},[K(\lambda )]K_{f_\lambda }(1).$$

Then, the representation theoretically defined operators $`F_a^{(r)},E_a^{(r)}`$ act in the same way as the Chevalley generators $`F_a^{(r)},E_a^{(r)}`$ of $`𝒰_{}`$.

§4-c. Tilting modules in category $`𝒪_{m\mathbf{|}n}`$. We proceed to prove some results and formulate some conjectures about the infinite dimensional tilting modules $`T(\lambda )`$. For $`\lambda X(m|n)`$, write

$$M^{}(\lambda ):=U(𝔤_{\overline{0}})_{𝔟_{\overline{0}}}_\lambda $$

for the purely even Verma module for $`𝔤_{\overline{0}}`$ concentrated in degree $`\overline{\lambda }`$, and $`L^{}(\lambda )`$ for its unique irreducible quotient. We will need the following result of Kac \[Ka3, Proposition 2.9\]. Actually in loc. cit., Kac is only concerned with finite dimensional representations, but the same argument works for the general case stated here.

###### Lemma 4.30.

If $`\lambda X(m|n)`$ is typical, then $`L(\lambda )U(𝔤)_{U(𝔭)}L^{}(\lambda )`$.

Recall the definition of the polynomials $`t_{g,f}(q)`$ and $`l_{g,f}(q)`$ from (2.18). We use the bijection (4.4) to shift notation, letting $`t_{\mu ,\lambda }(q):=t_{f_\mu ,f_\lambda }(q)`$ and $`l_{\mu ,\lambda }(q):=l_{f_\mu ,f_\lambda }(q)`$. The first part of the following theorem is a reformulation of the Kazhdan-Lusztig conjecture \[KL\] for $`𝔤𝔩(m)𝔤𝔩(n)`$, proved in \[BB, BrK\].

###### Theorem 4.31.

Let $`\lambda X(m|n)`$.

* If $`\lambda `$ is typical then $`(T(\lambda ):M(\mu ))=t_{\mu ,\lambda }(1)`$ for each $`\mu X(m|n)`$.
* For arbitrary $`\lambda `$, each subquotient of a Verma flag of $`T(\lambda )`$ is $`M(\mu )`$ for $`\mu \lambda `$.

###### Proof.

(i) For the proof, we will assume instead that $`\lambda X(m|n)`$ is typical with $`\lambda +\rho X^+(m|n)`$. Let $`W_\lambda `$ be the stabilizer in $`WS_{m|n}`$ of $`\lambda `$ under the dot action, and let $`D^\lambda `$ be the set of all maximal length $`W/W_\lambda `$-coset representatives. Let $`w_\lambda `$ be the longest element of $`W_\lambda `$. By the Kazhdan-Lusztig conjecture for the Lie algebra $`𝔤_{\overline{0}}`$ proved in \[BB, BrK\] combined with the translation principle \[J1\], see also \[BGS, Theorem 3.11.4\], we have that

$$[M^{}(x\lambda ):L^{}(y\lambda )]=P_{x,y}(1)$$

for arbitrary $`x,yD^\lambda `$. Here, $`P_{x,y}(1)`$ denotes the usual Kazhdan-Lusztig polynomial associated to $`x,yW`$ evaluated at $`1`$, see \[KL\].

We claim that $`P_{x,y}(1)=t_{x\lambda 2\rho ,y\lambda 2\rho }(1)`$ for all $`x,yD^\lambda `$. To see this, let $`f:=f_\lambda `$, which is antidominant in the sense of 2. Define $`S_f,D_f`$ as in 2. We will use the fact that the map $`D^\lambda D_f,xw_\lambda x^1`$ is a bijection. Observe using (4.10) that $`f_{y\lambda 2\rho }=fw_\lambda y^1`$. So by Lemma 2.25, $`t_{x\lambda 2\rho ,y\lambda 2\rho }(1)=t_{fw_\lambda x^1,fw_\lambda y^1}(1)=m_{w_\lambda x^1,w_\lambda y^1}^{(f)}(1)`$. Noting $`m_{w_\lambda x^1,w_\lambda y^1}^{(f)}(1)`$ is the same as the element with the same name in \[So1\], \[So1, Remark 2.6\] and \[So1, Proposition 3.4\] show that $`m_{w_\lambda x^1,w_\lambda y^1}^{(f)}(1)=P_{x^1,y^1}(1)=P_{x,y}(1)`$. This proves the claim.

Now $`M(x\lambda )U(𝔤)_{U(𝔭)}M^{}(x\lambda )`$ by associativity of tensor product, while Lemma 4.30 shows that $`L(y\lambda )U(𝔤)_{U(𝔭)}L^{}(y\lambda )`$. So as the functor $`U(𝔤)_{U(𝔭)}\mathrm{?}`$ is exact, we deduce from the previous two paragraphs that

$$[M(x\lambda ):L(y\lambda )]=t_{x\lambda 2\rho ,y\lambda 2\rho }(1).$$

Note moreover that this argument shows that every subquotient of $`M(x\lambda )`$ that is $`L(y\lambda )`$ is actually $`L(y\lambda )`$. Finally applying (4.16) gives that

$$(T(y\lambda 2\rho ):M(x\lambda 2\rho ))=t_{x\lambda 2\rho ,y\lambda 2\rho }(1).$$

Part (i) of the theorem follows easily from this and central character considerations. Moreover, by an obvious refinement of (4.16) keeping track of parity information too, we see that every subquotient of a Verma flag of $`T(y\lambda 2\rho )`$ that is $`M(x\lambda 2\rho )`$ is actually $`M(x\lambda 2\rho )`$.

(ii) We proceed by induction on $`\mathrm{\#}\lambda `$. The case that $`\lambda `$ is typical follows from (i). So suppose that $`\mathrm{\#}\lambda >0`$ and the theorem has been proved for all $`\mu `$ with $`\mathrm{\#}\mu <\mathrm{\#}\lambda `$. Let $`i:K(𝒪_{m|n}^\mathrm{\Delta })𝒯_{}^{m|n}`$ be the map defined in Theorem 4.28. Apply the algorithm explained in 2 to $`f=f_\lambda `$ to construct $`h=f_\nu `$ for $`\nu X(m|n)`$ with $`\mathrm{\#}\nu <\mathrm{\#}\lambda `$ and a sequence $`X_1,\mathrm{},X_N`$ of monomials in $`E_a^{(r)}`$ and $`F_a^{(r)}`$. Let $`M:=X_N\mathrm{}X_1T(\nu )`$. Note by Corollary 4.25 that $`M`$ has a Verma flag, and each subquotient of a Verma flag of $`M`$ is $`M(\mu )`$ for some $`\mu X(m|n)`$. By the induction hypothesis, $`i([T(\nu )])`$ equals $`M_h(1)`$ plus a linear combination of $`M_g(1)`$’s with $`gh`$. By Lemmas 2.10 and 2.11, we deduce that $`X_N\mathrm{}X_1i([T(\nu )])`$ equals $`M_f(1)`$ plus a linear combination of $`M_g(1)`$’s with $`gf`$. So by Theorem 4.28,

$$[M]=[M(\lambda )]+(\text{a linear combination of }[M(\mu )]\text{’s with }\mu \lambda ).$$

By Corollary 4.27(i), $`T(\lambda )`$ is a summand of $`M`$, and the result follows. ∎

Motivated by the theorem, we formulate the following conjecture.

###### Conjecture 4.32.

Let $`i:K(𝒪_{m|n}^\mathrm{\Delta })𝒯_{}^{m|n}`$ be the map defined in Theorem 4.28. Then, $`i([T(\lambda )])=T_{f_\lambda }(1)`$ for each $`\lambda X(m|n)`$.

In view of (4.9) this conjecture is equivalent to either of the statements

$`(T(\lambda ):M(\mu ))`$ $`=t_{\mu ,\lambda }(1),`$ (4.33)
$`[M(\lambda ):L(\mu )]`$ $`=t_{\lambda 2\rho ,\mu 2\rho }(1)`$ (4.34)
for all $`\lambda ,\mu X(m|n)`$. By Corollary 2.24 and (4.10), the unitriangular matrices $`(l_{\mu ,\lambda }(1))_{\mu ,\lambda X^(m|n)}`$ and $`(t_{\lambda 2\rho ,\mu 2\rho }(1))_{\mu ,\lambda X^+(m|n)}`$ are inverse to each other, so inverting (4.34) also gives that

$$\mathrm{ch}L(\lambda )=\underset{\lambda X(m|n)}{}l_{\mu ,\lambda }(1)\mathrm{ch}M(\mu ).$$
(4.35)
Although the summation is infinite here, it involves only finitely many non-zero contributions to the dimensions of each fixed weight space of $`L(\lambda )`$, thus it can be viewed as a conjectural character formula for irreducibles in $`𝒪_{m|n}`$.

Further evidence for Conjecture 4.32 is given by the main theorem in the next subsection. We finally mention one other result which is in keeping with the conjecture, compare in particular with Theorem 2.34. Recall the definition of the dual crystal operators $`\stackrel{~}{E}_a^{},\stackrel{~}{F}_a^{},\epsilon _a^{}`$ and $`\phi _a^{}`$ from 2. Again, we lift these directly to $`X(m|n)`$ via the bijection (4.4).

###### Theorem 4.36 (Kujawa).

Let $`\lambda X(m|n)`$ and $`a`$.

* $`F_aL(\lambda )0`$ if and only if $`\phi _a^{}(\lambda )0`$, in which case it is a $`\tau `$-self-dual indecomposable module with irreducible socle and cosocle $`L(\stackrel{~}{F}_a^{}(\lambda ))`$. Moreover, $`F_aL(\lambda )`$ is irreducible if and only if $`\phi _a^{}(\lambda )=1`$.
* $`E_aL(\lambda )0`$ if and only if $`\epsilon _a^{}(\lambda )0`$, in which case it is a $`\tau `$-self-dual indecomposable module with irreducible socle and cosocle $`L(\stackrel{~}{E}_a^{}(\lambda ))`$. Moreover, $`E_aL(\lambda )`$ is irreducible if and only if $`\epsilon _a^{}(\lambda )=1`$.

Theorem 4.36 is a result of Jon Kujawa that will form part of his PhD thesis \[Ku\]. The proof, which will hopefully appear elsewhere, is similar to the proof given in \[B1\] of Kleshchev’s modular branching rules from \[Kv\]. It involves some explicit calculations with certain lowering operators in $`U(𝔤)`$.

§4-d. Tilting modules in category $`_{m\mathbf{|}n}`$. Now we study the finite dimensional tilting modules $`U(\lambda )`$. Lift the crystal operators $`\stackrel{~}{E}_a,\stackrel{~}{F}_a,\epsilon _a,\phi _a`$ from 3 to $`X^+(m|n)`$ through the bijection (4.4), as well as the mutually inverse bijections $`𝙻`$ and $`𝚁`$ from (3.31).

###### Theorem 4.37.

Let $`j:K(_{m|n}^\mathrm{\Delta })_{}^{m|n}`$ be the map defined in Theorem 4.29. Then, $`j([U(\lambda )])=U_{f_\lambda }(1)`$ for each $`\lambda X^+(m|n)`$. Moreover:

* Each subquotient of a Kac flag of $`U(\lambda )`$ is $`K(\mu )`$ for $`𝙻(\lambda )\mu \lambda `$;
* $`U(\lambda )P(𝙻(\lambda ))`$;
* $`U(\lambda )U(\lambda )^\tau `$.

###### Proof.

If $`\mathrm{\#}\lambda =0`$ then Lemma 4.30 implies that $`U(\lambda )=P(\lambda )=K(\lambda )=L(\lambda )`$ and the theorem follows in this case. Now suppose that $`\mathrm{\#}\lambda >0`$. Let $`f=f_\lambda `$ and define $`h=f_\nu `$ for $`\nu X^+(m|n)`$ and operators $`X_a\{E_a,F_a\}_a`$ according to Procedure 3.20. We may assume by induction that the theorem has been proved for $`\nu `$.

Consider $`X_aU(\nu )`$. Theorem 4.29, Lemma 3.21 and the induction hypothesis shows that $`j([X_aU(\nu )])=X_aU_{f_\nu }(1)=U_{f_\lambda }(1)`$. So we get from the explicit description of $`U_{f_\lambda }(1)`$ in Theorem 3.34(i) that $`[X_aU(\nu )]=[K(\lambda )]+()+[K(𝙻(\lambda ))]`$ where $`()`$ is a sum of $`[K(\mu )]`$’s for $`𝙻(\lambda )\mu \lambda `$. Using Corollary 4.26, we deduce from this and the induction hypothesis that $`X_aU(\nu )`$ has a Kac flag with subquotients $`K(\lambda )`$, $`K(𝙻(\lambda ))`$ and all other subquotients $`K(\mu )`$ for $`𝙻(\lambda )\mu \lambda `$. So $`X_aU(\nu )`$ must have a summand that is $`U(\lambda )`$, recalling Lemma 4.27(ii). Also $`U(\nu )U(\nu )^\tau `$ is projective by the induction hypothesis, hence $`X_aU(\nu )(X_aU(\nu ))^\tau `$ is projective by Lemma 4.23. So $`X_aU(\nu )`$ must have a summand that is $`P(𝙻(\lambda ))`$. To complete the proof, it just remains to show that $`X_aU(\nu )`$ is indecomposable. For this, we give two different arguments, the first based on Theorem 4.36 and the second using instead a fundamental fact proved by Serganova in \[S3\].

Method one. Suppose the space

$$\mathrm{Hom}_{_{m|n}}(X_aU(\nu ),L(\mu ))\mathrm{Hom}_{_{m|n}}(U(\nu ),Y_aL(\mu ))$$

is non-zero for some $`\mu X^+(m|n)`$. By the choice of $`a`$ in Procedure 3.20, $`\lambda `$ is not at the end of an $`a`$-string of length $`2`$ in the crystal graph. Since we must have that $`\mathrm{wt}(\mu )=\mathrm{wt}(\lambda )`$ by Lemma 4.18, it follows that $`\mu `$ is also not at the end of an $`a`$-string of length $`2`$. Theorem 4.36 now implies immediately that $`Y_aL(\mu )L(\stackrel{~}{Y}_a^{}(\mu ))`$. By the induction hypothesis, $`U(\nu )`$ is the projective cover of $`L(𝙻(\nu ))`$, so we deduce from the non-vanishing of the right hand hom space above that $`\stackrel{~}{Y}_a^{}(\mu )=𝙻(\nu )`$. Hence, $`\mu =\stackrel{~}{X}_a^{}(𝙻(\nu ))=𝙻(\stackrel{~}{X}_a(\nu ))=𝙻(\lambda )`$, using Remark 3.18(2) for the penultimate equality. We have now shown that $`\mathrm{cosoc}_𝔤(X_aU(\nu ))L(𝙻(\lambda ))`$, so it is indecomposable.

Method two. Suppose $`X_aU(\nu )`$ is decomposable. Then, by what we have shown already, we can write $`X_aU(\nu )=T_1T_2`$ where $`T_1U(\lambda )`$ and $`T_20`$ is a direct sum of indecomposable tilting modules. Note that $`Y_aT_i0`$ for each $`i`$, indeed we have by adjointness that

$$\mathrm{Hom}_{_{m|n}}(U(\nu ),Y_aT_i)\mathrm{Hom}_{_{m|n}}(X_aU(\nu ),T_i)0.$$

Recalling Lemma 3.21, we now consider two cases. First, suppose that $`\nu `$ is at the end of an $`a`$-string of length $`1`$. Then, we have that $`Y_aU_{f_\lambda }(1)=U_{f_\nu }(1)`$, i.e. $`[Y_aX_aU(\nu )]=[U(\nu )]`$. Since $`Y_aX_aU(\nu )`$ is a direct sum of indecomposable tilting modules, we deduce that $`U(\nu )Y_aX_aU(\nu )Y_aT_1Y_aT_2`$, a contradiction since $`U(\nu )`$ is indecomposable. Otherwise, we have that $`\nu `$ is at the end of an $`a`$-string of length $`2`$, and $`[Y_aX_aU(\nu )]=2[U(\nu )]`$. Hence this time we must have that $`Y_aT_1Y_aT_2U(\nu )`$. In particular, we get that $`[Y_aU(\lambda ):L(\nu )]=1`$. We now show that $`[Y_aU(\lambda ):L(\nu )]2`$, to get the desired contradiction.

Let $`\mu =\stackrel{~}{X}_a^{}(\nu )`$, so $`\mu =\lambda \alpha `$ for some $`\alpha R_{\overline{1}}^+`$ with $`(\lambda +\rho |\alpha )=0`$. By \[S3, Theorem 5.5\] and (4.16), we have that $`[K(\lambda ):L(\mu )]1`$ and that $`(U(\lambda ):K(\mu ))=[K(\beta w_0\mu ):L(\beta w_0\lambda )]1`$. Hence, $`[U(\lambda ):L(\mu )]2`$, since it has a Kac flag involving both $`K(\lambda )`$ and $`K(\mu )`$, each of which have $`L(\mu )`$ as a composition factor. Now $`X_aK(\nu )`$ has a two-step filtration with $`K(\lambda )`$ at the bottom and $`K(\mu )`$ at the top, hence

$$\mathrm{Hom}_{_{m|n}}(K(\nu ),Y_aL(\mu ))\mathrm{Hom}_{_{m|n}}(X_aK(\nu ),L(\mu ))0.$$

This shows that $`[Y_aL(\mu ):L(\nu )]1`$. Finally applying the exact functor $`Y_a`$ to $`U(\lambda )`$ and combining our two facts $`[U(\lambda ):L(\mu )]2`$ and $`[Y_aL(\mu ):L(\nu )]1`$ gives that $`[Y_aU(\lambda ):L(\nu )]2`$ as required. ∎

Now recall the definition of the polynomials $`u_{g,f}(q)`$ and $`l_{g,f}(q)`$ from (3.7). As usual we shift notation, writing $`u_{\mu ,\lambda }(q):=u_{f_\mu ,f_\lambda }(q)`$ and $`l_{\mu ,\lambda }(q):=l_{f_\mu ,f_\lambda }(q)`$. Combining the theorem with (4.16), we get that

$`(U(\lambda ):K(\mu ))`$ $`=u_{\mu ,\lambda }(1),`$ (4.38)
$`[K(\lambda ):L(\mu )]`$ $`=u_{\beta w_0\lambda ,\beta w_0\mu }(1).`$ (4.39)
The Main Theorem stated in the introduction follows immediately from the second of these formulae and Corollary 3.36(i), since in view of (4.17) and Corollary 3.14 that gives an explicit formula for $`u_{\beta w_0\lambda ,\beta w_0\mu }(1)`$. In particular, $`[K(\lambda ):L(\mu )]1`$ for all $`\lambda ,\mu X^+(m|n)`$, as was conjectured in \[HKJ, Conjecture 7.2\], and $`L(\mu )`$ appears as a composition factor in exactly $`2^{\mathrm{\#}\mu }`$ different Kac modules $`K(\lambda )`$, as was conjectured in \[HKJ, Corollary 7.3\].

By Corollary 3.14 and (4.17), the unitriangular matrix $`(l_{\mu ,\lambda }(1))_{\mu ,\lambda X^+(m|n)}`$ is the inverse of $`(u_{\beta w_0\lambda ,\beta w_0\mu }(1))_{\mu ,\lambda X^+(m|n)}`$. So on inverting (4.39), we also get that

$$\mathrm{ch}L(\lambda )=\underset{\mu X^+(m|n)}{}l_{\mu ,\lambda }(1)\mathrm{ch}K(\mu ).$$
(4.40)
This can be viewed as a character formula for the finite dimensional irreducible $`𝔤𝔩(m|n)`$-supermodules with integral highest weight. The explicit description of the coefficients $`l_{\mu ,\lambda }(1)`$ given by Corollary 3.39(ii) seems to be quite different from the explicit description given by Serganova \[S3, Theorem 2.3\], and I have been unable to prove combinatorially that they are equivalent.

To conclude the subsection, let us record one more consequence of Theorem 4.37.

###### Corollary 4.41.

For $`\lambda X^+(m|n)`$, $`L(\lambda )^{}L(\beta w_0𝚁(\lambda ))`$.

###### Proof.

By (4.15) and Theorem 4.37, $`P(\beta w_0𝚁(\lambda ))U(𝚁(\lambda ))^{}P(\lambda )^{}`$ and it is self-dual under the duality $`\tau `$. Hence $`L(\beta w_0𝚁(\lambda ))\mathrm{soc}_𝔤P(\beta w_0𝚁(\lambda ))\mathrm{soc}_𝔤P(\lambda )^{}(\mathrm{cosoc}_𝔤P(\lambda ))^{}L(\lambda )^{}`$. ∎

###### Remark 4.42.

A different description of the highest weight of $`L(\lambda )^{}`$ can be derived using Serganova’s odd reflections, see \[S1\], \[PS1, Lemma 0.3\] and \[BKu, Theorem 4.5\]. In view of (4.39) and (4.14), Corollary 4.41 implies (indeed is equivalent to) the equality $`u_{\beta w_0\mu ,\beta w_0\lambda }(1)=u_{\mu ,𝚁(\lambda )}(1)`$, see Corollary 3.41 for a stronger statement.

§4-e. Highest weight categories. Let $`_{m|n}^{\overline{0}}`$ (resp. $`_{m|n}^{\overline{1}}`$) be the full subcategory of $`_{m|n}`$ consisting of the modules all of whose composition factors are $`L(\lambda )`$ (resp. $`\mathrm{\Pi }L(\lambda )`$) for $`\lambda X^+(m|n)`$. Obviously, the parity change functor $`\mathrm{\Pi }`$ defines an isomorphism between $`_{m|n}^{\overline{0}}`$ and $`_{m|n}^{\overline{1}}`$. Since each $`\mathrm{End}_{_{m|n}}(L(\lambda ))`$ is concentrated in degree $`\overline{0}`$, each $`\mathrm{Hom}_{_{m|n}}(M,N)`$ for $`M,N_{m|n}^{\overline{0}}`$ is also concentrated in degree $`\overline{0}`$, hence $`_{m|n}^{\overline{0}}`$ is an abelian category.

###### Lemma 4.43.

For $`\lambda X^+(m|n)`$, each of the objects $`U(\lambda ),P(\lambda ),K(\lambda )`$ and $`L(\lambda )`$ belong to $`_{m|n}^{\overline{0}}`$. Moreover, the dualities $``$ and $`\tau `$ and the functors $`F_a^{(r)}`$ and $`E_a^{(r)}`$ map objects in $`_{m|n}^{\overline{0}}`$ to objects in $`_{m|n}^{\overline{0}}`$.

###### Proof.

By Theorem 4.37(i), each subquotient of a Kac flag of $`U(\lambda )`$ is $`K(\mu )`$ for some $`\mu X^+(m|n)`$. We deduce using (4.14)–(4.15) that each subquotient of a Kac flag of $`P(\lambda )`$ is $`K(\mu )`$ for some $`\mu X^+(m|n)`$. By the obvious refinement of (4.13) keeping track of parities, it follows that each composition factor of $`K(\mu )`$ is $`L(\lambda )`$ for some $`\lambda X^+(m|n)`$. Combining these statements shows that all of $`U(\lambda ),P(\lambda ),K(\lambda )`$ and $`L(\lambda )`$ belong to $`_{m|n}^{\overline{0}}`$. For the remaining statement, we obviously have that $`L(\lambda )^\tau L(\lambda )`$, hence $`\tau `$ leaves $`_{m|n}^{\overline{0}}`$ invariant. The same thing for $``$ follows from Corollary 4.41. Finally, Corollary 4.26 shows that the exact functors $`F_a^{(r)}`$ and $`E_a^{(r)}`$ send $`K(\lambda )`$ to an object in $`_{m|n}^{\overline{0}}`$, and $`L(\lambda )`$ is a quotient of $`K(\lambda )`$ so they must also send $`L(\lambda )`$ to an object in $`_{m|n}^{\overline{0}}`$. ∎

###### Corollary 4.44.

For any $`M,N_{m|n}^{\overline{0}}`$ and $`i0`$, the space $`\mathrm{Ext}_{_{m|n}}^i(M,N)`$ is concentrated in degree $`\overline{0}`$.

###### Proof.

We have already noted this is the case if $`i=0`$. To get the general case from this, note by the lemma that every composition factor of every term of the obvious minimal projective resolution of $`M`$ belongs to $`_{m|n}^{\overline{0}}`$. ∎

It follows easily from the corollary that every object $`M_{m|n}`$ decomposes uniquely as $`M=M^{\overline{0}}M^{\overline{1}}`$ with $`M^p_{m|n}^p`$ for each $`p_2`$. We deduce that there is a decomposition $`_{m|n}=_{m|n}^{\overline{0}}\mathrm{\Pi }_{m|n}^{\overline{0}}`$ allowing us to reconstruct the superadditive category $`_{m|n}`$ from the additive category $`_{m|n}^{\overline{0}}`$. For example, for $`M,N_{m|n}`$ and $`i0`$, we have that

$`\mathrm{Ext}_{_{m|n}}^i(M,N)_{\overline{0}}`$ $`=\mathrm{Ext}_{_{m|n}^{\overline{0}}}^i(M^{\overline{0}},N^{\overline{0}})\mathrm{Ext}_{_{m|n}^{\overline{0}}}^i(\mathrm{\Pi }M^{\overline{1}},\mathrm{\Pi }N^{\overline{1}}),`$ (4.45)
$`\mathrm{Ext}_{_{m|n}}^i(M,N)_{\overline{1}}`$ $`=\mathrm{Ext}_{_{m|n}^{\overline{0}}}^i(M^{\overline{0}},\mathrm{\Pi }N^{\overline{1}})\mathrm{Ext}_{_{m|n}^{\overline{0}}}^i(\mathrm{\Pi }M^{\overline{1}},N^{\overline{0}}).`$ (4.46)
At this point, we refer the reader to \[CPS1, CPS2\] for the definition of a highest weight category with duality.

###### Theorem 4.47.

The category $`_{m|n}^{\overline{0}}`$ is a highest weight category with weight poset $`(X^+(m|n),)`$ and duality $`\tau `$. For $`\lambda X^+(m|n)`$, $`U(\lambda ),P(\lambda ),K(\lambda )`$ and $`L(\lambda )`$ are the indecomposable tilting, projective, standard and irreducible modules parametrized by $`\lambda `$, respectively.

###### Proof.

We have seen in Theorem 4.37 that $`(P(\lambda ):K(\mu ))0\lambda \mu `$. Given this and (4.13) it is a routine matter to check that $`_{m|n}^{\overline{0}}`$ satisfies the axioms for a highest weight category with duality. ∎

###### Remark 4.48.

In an entirely similar fashion, we define $`𝒪_{m|n}^{\overline{0}}`$ to be the full subcategory of $`𝒪_{m|n}`$ consisting of the objects $`M`$ all of whose composition factors are $`L(\lambda )`$ for $`\lambda X(m|n)`$. Using Theorem 4.31(ii) and the refined versions of (4.9) and BGG reciprocity \[B2, (6.6)\] keeping track of parity, one can prove analogues of all the results in this subsection for $`𝒪_{m|n}^{\overline{0}}`$: there is a decomposition $`𝒪_{m|n}=𝒪_{m|n}^{\overline{0}}\mathrm{\Pi }𝒪_{m|n}^{\overline{0}}`$, and $`𝒪_{m|n}^{\overline{0}}`$ is a highest weight category with weight poset $`(X(m|n),)`$ and duality $`\tau `$.

§4-f. Kazhdan-Lusztig polynomials. In this subsection, we explain the true significance of the polynomials $`l_{\mu ,\lambda }(q)=l_{f_\mu ,f_\lambda }(q)`$ for $`\mu ,\lambda X^+(m|n)`$.

###### Lemma 4.49.

Let $`\mu X^+(m|n)`$. Then, $`K(\mu )`$ has a projective resolution $`\mathrm{}P_1(\mu )P_0(\mu )K(\mu )0`$ in $`_{m|n}^{\overline{0}}`$ such that for every $`\lambda X^+(m|n)`$,

$$\underset{i0}{}dim\mathrm{Hom}_{_{m|n}}(P_i(\mu ),L(\lambda ))q^i=l_{\mu ,\lambda }(q^1).$$

###### Proof.

We first explain how to construct for fixed $`d0`$ an exact sequence $`P_d(\mu )\mathrm{}P_0(\mu )K(\mu )0`$ with each $`P_i(\mu )`$ projective. In case $`\mathrm{\#}\mu =0`$, $`K(\mu )`$ is already projective, so we can simply take $`P_0(\mu )=K(\mu )`$ and $`P_i(\mu )=0`$ for $`i>0`$. Now suppose $`\mathrm{\#}\mu >0`$. Let $`g=f_\mu `$ and apply Procedure 3.23 to construct $`h=f_\nu `$ and operators $`X_a,Y_a\{E_a,F_a\}_a`$. Since Procedure 3.23 reduces $`\mu `$ to a typical weight in finitely many steps, we may assume inductively that we have already constructed an exact sequence

$$P_d(\nu )\mathrm{}P_0(\nu )K(\nu )0.$$
(4.50)
Now we consider two cases. Suppose first that $`\mathrm{\#}\nu =\mathrm{\#}\mu `$. Then $`X_aK(\nu )K(\mu )`$, so applying $`X_a`$ to (4.50) gives us the desired sequence with $`P_i(\mu )=X_aP_i(\nu )`$. In the second case, $`\mathrm{\#}\nu =\mathrm{\#}\mu 1`$, and $`X_aK(\nu )`$ has a two step filtration with $`K(\mu )`$ at the top and $`K(\stackrel{~}{X}_a(\nu ))`$ at the bottom. Applying $`X_a`$ to (4.50) gives us an exact sequence $`X_aP_d(\nu )\mathrm{}X_aP_0(\nu )X_aK(\nu )0.`$ By induction on $`d`$, we may assume in addition that we have already constructed an exact sequence $`P_{d1}(\stackrel{~}{X}_a(\nu ))\mathrm{}P_0(\stackrel{~}{X}_a(\nu ))K(\stackrel{~}{X}_a(\nu ))0.`$ Applying the comparison theorem \[W, 2.2.6\] to the embedding $`i:K(\stackrel{~}{X}_a(\nu ))X_aK(\nu )`$, we get vertical maps making the diagram commute:

$$\begin{array}{ccccccccc}\mathrm{}& & P_1(\stackrel{~}{X}_a(\nu ))& & P_0(\stackrel{~}{X}_a(\nu ))& & K(\stackrel{~}{X}_a(\nu ))& & 0\\ & & & & & & i& & \\ \mathrm{}& & X_aP_1(\nu )& & X_aP_0(\nu )& & X_aK(\nu )& & 0\end{array}$$

The total complex of this double complex is exact by the acyclic assembly lemma \[W, 2.7.3\]. Factoring out $`K(\stackrel{~}{X}_a(\mu ))`$ yields the required exact sequence

$$\mathrm{}X_aP_1(\nu )P_0(\stackrel{~}{X}_a(\nu ))X_aP_0(\nu )K(\mu )0$$

This time, $`P_i(\mu )=X_aP_i(\nu )P_{i1}(\stackrel{~}{X}_a(\nu ))`$.

Replacing $`d`$ by $`(d+1)`$, the same procedure constructs an exact sequence $`P_{d+1}(\mu )P_d(\mu )\mathrm{}P_0(\mu )K(\mu )0,`$ where we can always ensure that the first $`d`$ terms are the same as the ones constructed before. Now letting $`d\mathrm{}`$ we get a projective resolution of $`K(\mu )`$. We note moreover by the construction that whenever $`P(\lambda )`$ is a summand of $`P_i(\mu )`$ for some $`i0`$, i.e. $`\mathrm{Hom}_{_{m|n}}(P_i(\mu ),L(\lambda ))0`$, we must have that $`\mu \lambda `$.

Finally let $`p_{\mu ,\lambda }(q)=_{i0}dim\mathrm{Hom}_{_{m|n}}(P_i(\mu ),L(\lambda ))q^i`$. To complete the proof, we need to show that $`p_{\mu ,\lambda }(q)=l_{\mu ,\lambda }(q^1)`$ for each $`\mu ,\lambda X^+(m|n)`$. For this, we show that the polynomials $`p_{\mu ,\lambda }(q)`$ satisfy the same relations as the polynomials $`l_{\mu ,\lambda }(q^1)`$ in Lemma 3.24. Once this is established, the algorithm explained at the end of 3 to compute $`l_{\mu ,\lambda }(q^1)`$ also computes $`p_{\mu ,\lambda }(q)`$, hence $`p_{\mu ,\lambda }(q)=l_{\mu ,\lambda }(q^1)`$. So take $`\lambda ,\mu X^+(m|n)`$ with $`\mathrm{\#}\mu >0`$, where we may assume that $`\mathrm{wt}(\lambda )=\mathrm{wt}(\mu )`$ since otherwise $`p_{\mu ,\lambda }(q)=0`$ and the conclusion holds trivially. Apply Procedure 3.23 to $`g=f_\mu `$ to get $`h=f_\nu `$ and operators $`X_a,Y_a`$, and consider the two cases $`\mathrm{\#}\nu =\mathrm{\#}\mu `$ or $`\mathrm{\#}\nu =\mathrm{\#}\mu 1`$, just like above. Let us just explain the argument in the second case, the first case being easier. Since $`\mathrm{wt}(\lambda )=\mathrm{wt}(\mu )`$ and $`\mu `$ is not at the end of an $`a`$-string of length 2 in the dual crystal graph, Theorem 4.36 shows that $`Y_aL(\lambda )`$ equals $`L(\stackrel{~}{Y}_a^{}(\lambda ))`$, interpreted as $`0`$ if $`\stackrel{~}{Y}_a^{}(\lambda )=\mathrm{}`$. So we get that

$$\begin{array}{c}\underset{i0}{}dim\mathrm{Hom}_{_{m|n}}(X_aP_i(\nu ),L(\lambda ))q^i=\underset{i0}{}dim\mathrm{Hom}_{_{m|n}}(P_i(\nu ),Y_aL(\lambda ))q^i\hfill \\ \hfill =\underset{i0}{}dim\mathrm{Hom}_{_{m|n}}(P_i(\nu ),L(\stackrel{~}{Y}_a^{}(\lambda )))q^i=p_{\nu ,\stackrel{~}{Y}_a^{}(\lambda )}(q),\end{array}$$

interpreted as $`0`$ in case $`\stackrel{~}{Y}_a^{}(\lambda )=\mathrm{}`$. We noted above that $`P_i(\mu )=X_aP_i(\nu )P_{i1}(\stackrel{~}{X}_a(\nu ))`$, hence we get that $`p_{\mu ,\lambda }(q)=p_{\nu ,\stackrel{~}{Y}_a^{}(\lambda )}(q)+qp_{\stackrel{~}{X}_a(\nu ),\lambda }(q),`$ which is what we wanted in this case, cf. Lemma 3.24. ∎

Now choose a length function on $`_+^{m|n}`$ as explained in 3, and lift it to $`X^+(m|n)`$ by setting $`\mathrm{}(\lambda ):=\mathrm{}(f_\lambda )`$.

###### Theorem 4.51.

For $`\mu ,\lambda X^+(m|n)`$, the superspace $`\mathrm{Ext}_{_{m|n}}^i(K(\mu ),L(\lambda ))`$ is concentrated in degree $`\overline{0}`$, and

$$\underset{i0}{}dim\mathrm{Ext}_{_{m|n}}^i(K(\mu ),L(\lambda ))q^i=l_{\mu ,\lambda }(q^1).$$

Hence, $`\mathrm{Ext}_{_{m|n}}^{}(K(\mu ),L(\lambda ))0`$ if and only if $`\mu \lambda `$, in which case

* $`\mathrm{Ext}_{_{m|n}}^i(K(\mu ),L(\lambda ))0i\mathrm{}(\lambda )\mathrm{}(\mu ),i\mathrm{}(\lambda )\mathrm{}(\mu )(mod2)`$;
* $`\mathrm{Ext}_{_{m|n}}^{\mathrm{}(\lambda )\mathrm{}(\mu )}(K(\mu ),L(\lambda ))`$ is exactly one dimensional;
* $`\mathrm{Ext}_{_{m|n}}^1(K(\mu ),L(\lambda ))`$ is at most one dimensional.

###### Proof.

Apply the functor $`\mathrm{Hom}_{_{m|n}}(\mathrm{?},L(\lambda ))`$ to the projective resolution constructed in Lemma 4.49 and use Corollaries 3.45, 3.39(ii) and 4.44. ∎

By \[Z, Theorem 7.6\], Theorem 4.51 shows that the polynomials $`l_{\mu ,\lambda }(q^1)`$ defined here agree with the Kazhdan-Lusztig polynomials $`K_{\lambda ,\mu }(q)`$ defined by Serganova in \[S2, S3\]. (It also proves \[JZ, Conjecture 4.4\], and answers a question raised at the end of \[Z\].) Thus we have a cohomological interpretation of the polynomial $`l_{\mu ,\lambda }(q^1)`$, analogous to Vogan’s interpretation \[V, Conjecture 3.4\] of Kazhdan-Lusztig polynomials in category $`𝒪`$ for a semisimple Lie algebra. The even-odd vanishing established in Theorem 4.51(i) is especially important: in the language of \[CPS3\], it shows that the highest weight category $`_{m|n}^{\overline{0}}`$ has a “Kazhdan-Lusztig theory”. Applying \[CPS4, Corollary 3.9\] (and Corollary 4.44 again), we obtain:

###### Corollary 4.52.

For $`\mu ,\lambda X^+(m|n)`$, the superspace $`\mathrm{Ext}_{_{m|n}}^i(L(\mu ),L(\lambda ))`$ is concentrated in degree $`\overline{0}`$, and

$$\underset{i0}{}dim\mathrm{Ext}_{_{m|n}}^i(L(\mu ),L(\lambda ))q^i=\underset{\nu X^+(m|n)}{}l_{\nu ,\mu }(q^1)l_{\nu ,\lambda }(q^1).$$

In particular, $`\mathrm{Ext}_{_{m|n}}^i(L(\mu ),L(\lambda ))0i\mathrm{}(\lambda )\mathrm{}(\mu )(mod2)`$.

###### Example 4.53.

Take $`\mu =\lambda =0`$ and let $`r=\mathrm{min}(m,n)`$. In this case, the polynomials $`l_{\nu ,\mu }(q^1)`$ are computed explicitly in Example 3.37. Combining this with Corollary 4.52, one deduces that $`dim\mathrm{Ext}_{_{m|n}}^{2i}(,)`$ equals the number of partitions of $`i`$ with at most $`r`$ non-zero parts. Hence:

$$\underset{i0}{}dim\mathrm{Ext}_{_{m|n}}^i(,)q^i=\frac{1}{(1q^2)(1q^4)\mathrm{}(1q^{2r})}.$$
(4.54)