text:
# Howe Duality for Lie Superalgebras

## 1. Introduction

Howe duality is a way of relating representation theory of a pair of reductive Lie groups/algebras \[H1, H2\]. It has found many applications to invariant theory, real and complex reductive groups, $`p`$-adic groups and infinite-dimensional Lie algebras etc.

As an example we consider one of the fundamental cases—the $`(gl(m),gl(n))`$ Howe duality. The symmetric algebra $`S(^m^n)`$ and the skew-symmetric algebra $`\mathrm{\Lambda }(^m^n)`$ admit remarkable multiplicity-free decompositions under the natural actions of $`gl(m)\times gl(n)`$. The highest weight vectors of $`gl(m)\times gl(n)`$ inside the symmetric algebra are given by products of certain determinants (see (2.2)) and form a free abelian semi-group while those inside the skew-symmetric algebra are given by Grassmann monomials, cf. \[H2, KV, GW\].

Our present paper is devoted to the study of Howe duality for Lie superalgebras and its applications. It is by now a well established fact that one should put the Grassmann variables on the same footing as Cartesian variables and hence it is natural to consider the supersymmetric algebra, which is a mixed tensor of symmetric and skew-symmetric algebras. In this paper we give a complete description of the Howe duality in a symmetric111In this paper we will freely suppress the term super. So in case when a superspace is involved, the terms symmetric, commute etc. mean supersymmetric, supercommute etc unless otherwise specified. algebra under the action of a dual pair of general linear Lie superalgebras and find explicit formulas for the highest weight vectors inside our symmetric model. A dual pair consisting of a general linear Lie superalgebra and a general linear Lie algebra was discussed in \[H1\]. We also study in detail some other multiplicity-free actions of the general linear Lie superalgebras as specified below.

Our motivation is manifold. First, our work is motivated by an attempt to unify the Howe duality in the symmetric and skew-symmetric models \[H2\] which have many differences and similarities. Specialization of our results gives rise to the Howe duality for general linear Lie algebras in both symmetric and skew-symmetric models. Secondly, we are motivated by our study of the duality in the infinite-dimensional setup (see the review \[W\] and references therein) and our work in progress on its generalization to the superalgebra case. We realize that we have to understand the finite-dimensional picture better first in order to have a more complete description of the infinite-dimensional picture. Thirdly, there exists a new type of Howe duality which is of pure superalgebra phenomenon which is treated in \[CW\].

Let us discuss the contents of the paper. The generalization of Schur duality for the superspace was given by Sergeev \[Se\]. For lack of an analog in the super setup of the criterion of multiplicity-free action in terms of the existence of a dense open orbit of a Borel subgroup (cf. \[V\] and \[H2\]), we use Sergeev’s result to derive the decomposition of the symmetric algebra $`S(^{p|q}^{m|n})`$ with respect to the action of the sum of two general linear Lie superalgebras $`gl(p|q)\times gl(m|n)`$. We see that a representation of $`gl(p|q)`$ is paired with a representation of $`gl(m|n)`$ parameterized by the same Young diagram. On the other hand one can show that our Howe duality for superalgebras implies Sergeev’s Schur duality as well. We also obtain an explicit multiplicity-free decomposition of a skew-symmetric algebra $`\mathrm{\Lambda }(^{p|q}^{m|n})`$ as $`gl(p|q)\times gl(m|n)`$-modules. In particular it follows that $`gl(p|q)`$ and $`gl(m|n)`$ when acting on $`S(^{p|q}^{m|n})`$ and respectively on $`\mathrm{\Lambda }(^{p|q}^{m|n})`$ are mutual (super)centralizers. A remarkable phenomenon is the complete reducibility of the symmetric model under the action of the dual pair, which is quite unusual for Lie superalgebras.

In a purely combinatorial way, Brini, Palareti and Teolis \[BPT\] were indeed the first to obtain an explicit decomposition of $`S(^{p|q}^{m|n})`$ under the action of $`gl(p|q)\times gl(m|n)`$. In addition, their combinatorial approach exhibits explicit bases parameterized by so-called left (or right) symmetrized bitableuax between two “standard Young diagrams” (see \[BPT\] for definition). However, Brini et al did not identify the highest weights for these $`gl(p|q)\times gl(m|n)`$-modules inside $`S(^{p|q}^{m|n})`$.

We also obtain an explicit decomposition into irreducible $`gl(m|n)`$-modules of the symmetric algebra $`S(S^2^{m|n})`$ and respectively skew-symmetric algebra $`\mathrm{\Lambda }(S^2^{m|n})`$ of the symmetric square of the natural representation of $`gl(m|n)`$. These results unifies and generalizes several classical results and they can be proved in an analogous way as in the classical case \[H2, GW\].

Associated to the Howe duality and the above $`gl(m|n)`$-module decompositions, we obtain, by taking characters, various combinatorial identities involving the so-called hook Schur functions. Being generalization of Schur functions, these hook Schur functions have been studied in \[BR\]. The decompositions mentioned above in turn provide the representation theoretic realization of the corresponding combinatorial identities. For example, the $`(gl(m|n),gl(p|q))`$-duality gives rise to a combinatorial identity for the hook Schur functions which generalizes the Cauchy identity for Schur functions, cf. \[H2\]. Specializations and variations of these combinatorial identities are well known and other proofs can be found in Macdonald \[M\].

However it is a much more difficult problem to find explicit formulas for the highest weight vectors of $`gl(p|q)\times gl(m|n)`$-modules inside the symmetric algebra. We first find formulas for the highest weight vectors in the case for $`q=0`$ (and so for $`n=0`$ by symmetry). A main ingredient in the formulas for the highest weight vectors is given by the determinant of a matrix which involves both Cartesian variables $`x_j^i`$’s and Grassmann variables $`\eta ^i`$’s of the form:

$$\left(\begin{array}{cccc}x_1^1& x_1^2& \mathrm{}& x_1^r\\ x_2^1& x_2^2& \mathrm{}& x_2^r\\ \mathrm{}& \mathrm{}& \mathrm{}& \mathrm{}\\ x_m^1& x_m^2& \mathrm{}& x_m^r\\ \eta ^1& \eta ^2& \mathrm{}& \eta ^r\\ \mathrm{}& \mathrm{}& \mathrm{}& \mathrm{}\\ \eta ^1& \eta ^2& \mathrm{}& \eta ^r\end{array}\right).$$

We remark that the rows involving Grassmann variables are the same but the determinant is nonzero (one needs to overcome some psychological barriers). Note that when $`m=r`$ the Grassmann variables disappear and the above determinant reduces to those mentioned earlier which occur in the formulas for highest weight vectors in the symmetric algebra case of the classical Howe duality. When $`m=0`$, the Cartesian variables disappear and the above determinant is equal to (up to a scalar multiple) a Grassmann monomial which shows up in the formulas for highest weight vectors in the classical skew-symmetric algebra case.

We show that the $`gl(p|q)\times gl(m|n)`$ highest weight vectors form an abelian semigroup in the case when $`p=m`$. However in contrast to the Lie algebra case this semigroup is not free in general. We find that the generators of the semigroup are given by highest weight vectors associated to rectangular Young diagrams of length not exceeding $`m+1`$. This way we are able to find explicit formulas for all highest weight vectors in the case when $`q=0`$ (or $`m=0`$), or $`p=m`$.

In the general case the highest weight vectors no longer form a semigroup. We find a nice way to overcome this difficulty by introducing some extra variables which, roughly speaking, help us to reduce the general case to the case $`p=m`$. Then we use a simple method to get rid of the extra variables to obtain the genuine highest weight vectors we are looking for.

In contrast to the Howe duality in the Lie algebra setup, it is difficult to check directly the highest weight condition of the vectors we have obtained. We use instead the multiplicity-free decomposition of the symmetric algebra to get around this difficulty. As highest weight constraint we obtain interesting non-trivial polynomial identities typically involving various minors of a matrix.

We also find explicit formulas for the highest weight vectors appearing in the $`gl(m|n)`$-module decomposition of $`S(S^2^{m|n})`$. These highest weight vector formulas, which constitute a mixture of determinants and Pfaffians, have somewhat similar features as those found in the Howe duality for the general linear Lie superalgebras.

A formula for highest weight vectors in the decomposition of $`S(^{p|q}^{m|n})`$ as $`gl(p|q)\times gl(m|n)`$ modules may also be obtained in principle using the combinatorial approach of Brini et al \[BPT\], and in this way the highest weights for these $`gl(p|q)\times gl(m|n)`$ modules can be identified. However this way one can neither expect to obtain formulas as explicit as ours, nor can one see the semigroup structure of the set of the highest weight vectors which is the guiding principle for us to find these vectors. It is also interesting to see whether our results concerning the decomposition of $`S(S^2^{m|n})`$ (and respectively $`\mathrm{\Lambda }(S^2^{m|n})`$) and the highest weight vectors in these models may also be obtained with extra insights from the combinatorial approach in \[BPT\] as well.

The plan of the paper goes as follows. In Section 2 we review the classical dual pairs of general linear Lie algebras and Schur duality. In Section 3 we present various multiplicity-free actions for Lie superalgebras and obtain the corresponding symmetric function identities. Section 4 is devoted to the construction of the $`gl(p|q)\times gl(m|n)`$ highest weight vectors inside $`S(^{p|q}^{m|n})`$. More precisely, in Section 4.1, Section 4.2, and Section 4.3, we find explicit formulas of highest weight vectors in the case $`q=0`$, $`p=m`$, and the general case, respectively. Finally in Section 5 we construct the $`gl(m|n)`$ highest weight vectors inside $`S(S^2^{m|n})`$.

Acknowledgment. Our work is greatly influenced by the beautiful article \[H2\] of R. Howe to whom we are grateful. The results in this paper and and its sequel \[CW\] are based on our two preprints under the same title (part one and two) as the current paper. After we submitted part one and finished part two, we came across two preprints of Sergeev: “An analog of the classical invariant theory for Lie superlagebras”, I, II, math.RT/9810113 and math.RT/9904079, which have overlaps with our work. More precisely, Sergeev obtained independently the $`gl(p|q)\times gl(m|n)`$-module decomposition of the symmetric algebra $`S(^{p|q}^{m|n})`$ (i.e. our Theorem 3.2), and the $`gl(m|n)`$-module decomposition of $`S(S^2^{m|n})`$ (i.e. our Theorem 3.4). Finally we also like to thank the referee for bringing the paper \[BPT\], which is discussed in the introduction, to our attention. Therefore we have made corresponding changes on our preprints which result in the current version of this paper and \[CW\].

## 2. The Classical Picture

In this section we will review some classical multiplicity-free actions of the general linear Lie algebra. We begin with the classical $`gl(m)\times gl(n)`$-duality, cf. Howe \[H2\].

Let $`\lambda =(\lambda _1,\lambda _2,\mathrm{},\lambda _l)`$ be a partition of the integer $`|\lambda |=\lambda _1+\mathrm{}+\lambda _l`$, where $`\lambda _1\mathrm{}\lambda _l>0`$. The integer $`|\lambda |`$ is called the size, $`l`$ is called the length (denoted by $`l(\lambda )`$), and $`\lambda _1`$ is called the width of the partition $`\lambda `$. Let $`\lambda ^{}`$ denote the Young diagram obtained from $`\lambda `$ by transposing. We will often denote $`\lambda _1`$ by $`t`$ and write $`\lambda ^{}=(\lambda _1^{},\lambda _2^{},\mathrm{},\lambda _t^{})`$. For example, the Young diagram

(2.1) ....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................$`r\{\text{ }`$$`\underset{t}{\underset{}{\text{ }}}`$
stands for the partition $`(5,3,2,1)`$ and its transpose is the partition $`(4,3,2,1,1)`$.

Given a partition $`\lambda =(\lambda _1,\lambda _2,\mathrm{},\lambda _l)`$ satisfying $`lm`$, we may regard $`\lambda `$ as a highest weight of $`gl(m)`$ by identifying $`\lambda `$ with the $`m`$-tuple $`(\lambda _1,\lambda _2,\mathrm{},\lambda _l,0,\mathrm{},0)`$ by adding $`ml`$ zeros to $`\lambda `$. We denote the irreducible finite-dimensional highest weight module of $`gl(m)`$ by $`V_m^\lambda `$.

Consider the natural action of the complex general linear Lie groups $`GL(m)`$ and $`GL(n)`$ on the space $`^m^n`$. If we identify $`^m^n`$ with $`M_{mn}`$, the space of all $`m\times n`$ matrices, then the actions of $`GL(m)`$ and $`GL(n)`$ are given by left and right multiplications:

$$(g_1,g_2)(T)=(g_1^t)^1Tg_2^1g_1GL(m),g_2GL(n),TM_{mn}.$$

The Lie algebras $`gl(m)`$ and $`gl(n)`$ act on $`^m^n`$ accordingly. Denoting by $`S(^m^n)`$ the symmetric tensor algebra of $`^m^n`$ with an induced action of $`gl(m)\times gl(n)`$, we have the following multiplicity-free decomposition of $`S(^m^n)`$ as a $`gl(m)\times gl(n)`$-module:

$$S(^m^n)\underset{\lambda }{}V_m^\lambda V_n^\lambda ,$$

where the sum above is over Young diagrams $`\lambda `$ of length not exceeding $`\mathrm{min}(m,n)`$.

One can find an explicit formula for the $`gl(m)\times gl(n)`$ highest weight vectors in this decomposition. Let us denote a basis of $`^m`$ by $`x_1,x_2,\mathrm{},x_m`$ and a basis of $`^n`$ by $`x^1,x^2,\mathrm{},x^n`$. Then the vectors $`x_i^j:=x_ix^j`$, for $`i=1,\mathrm{},m`$ and $`j=1,\mathrm{},n`$ form a basis for $`^m^n`$ so that we may identify $`S(^m^n)`$ with $`[x_1^1,\mathrm{},x_1^n,\mathrm{},x_m^1,\mathrm{},x_m^n]`$. Using this identification the standard Borel subalgebra of $`gl(m)`$ is a sum of the Cartan subalgebra generated by

$$\underset{j=1}{\overset{n}{}}x_i^j\frac{}{x_i^j},1im,$$

and the nilpotent radical generated by

$$\underset{j=1}{\overset{n}{}}x_{i1}^j\frac{}{x_i^j},2im.$$

Similarly the Borel subalgebra of $`gl(n)`$ is the sum of the Cartan subalgebra generated by

$$\underset{i=1}{\overset{m}{}}x_i^j\frac{}{x_i^j},1jn,$$

and the nilpotent radical generated by

$$\underset{i=1}{\overset{m}{}}x_i^{j1}\frac{}{x_i^j},2jn.$$

Let $`ϵ_i`$ for $`i=1,\mathrm{},m`$ (respectively $`\stackrel{~}{ϵ}_j`$ for $`j=1,\mathrm{},n`$) be the fundamental weights corresponding to the Cartan subalgebra of $`gl(m)`$ (respectively $`gl(n)`$) above. For $`1r\mathrm{min}(m,n)`$ define

(2.2)
$$\mathrm{\Delta }_r:=\mathrm{det}\left(\begin{array}{cccc}x_1^1& x_2^1& \mathrm{}& x_r^1\\ x_1^2& x_2^2& \mathrm{}& x_r^2\\ \mathrm{}& \mathrm{}& \mathrm{}& \mathrm{}\\ x_1^r& x_2^r& \mathrm{}& x_r^r\end{array}\right).$$

It is easy to see that $`\mathrm{\Delta }_r`$ is a highest weight vector for both $`gl(m)`$ and $`gl(n)`$ and its weights are respectively $`_{i=1}^rϵ_i`$ and $`_{i=1}^r\stackrel{~}{ϵ}_i`$. This weight corresponds to the Young diagram

..........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................$`\mathrm{}`$$`\text{ }\}r`$
That is, $`\mathrm{\Delta }_r`$ is the highest weight vector for $`\mathrm{\Lambda }^r(^m)\mathrm{\Lambda }^r(^n)`$ inside $`S(^m^n)`$, the tensor product of the $`r`$-th fundamental representations of $`gl(m)`$ and $`gl(n)`$.

Let $`\lambda `$ be a Young diagram as in (2.1) with length not exceeding $`\mathrm{min}(m,n)`$. The set of highest weight vectors in $`S(C^m^n)`$ form an abelian semigroup, and the product $`\mathrm{\Delta }_{\lambda _1^{}}\mathrm{\Delta }_{\lambda _2^{}}\mathrm{}\mathrm{\Delta }_{\lambda _t^{}}`$ is a highest weight vector for the irreducible representation in $`S(C^m^n)`$ corresponding to the Young diagram $`\lambda `$.

On the other hand, the skew-symmetric algebra $`\mathrm{\Lambda }(^m^n)`$ admits an induced $`gl(m)\times gl(n)`$ action. Following Howe \[H2\], we have the multiplicity-free decomposition

$$\mathrm{\Lambda }(^m^n)\underset{\lambda }{}V_m^\lambda V_n^\lambda ^{},$$

where the summation runs over Young diagrams $`\lambda `$ of length not exceeding $`m`$ and of width not exceeding $`n`$.

Denote by $`\eta _i^j,1im,1jn`$ the standard basis for $`^m^n`$ in the consideration of skew-symmetric algebra. The highest weight vector for the $`gl(m)\times gl(n)`$-module $`V_m^\lambda V_n^\lambda ^{}`$ inside $`\mathrm{\Lambda }(^m^n)`$ is given by

$`\eta _1^1\eta _1^2\mathrm{}\eta _1^{\lambda _1}\eta _2^1\eta _2^2\mathrm{}\eta _2^{\lambda _2}\mathrm{}\eta _l^1\eta _l^2\mathrm{}\eta _l^{\lambda _l}`$
where $`l`$ is the length of $`\lambda `$.

Intimately related to the Howe duality is the Schur duality, which we review below. Consider the standard representation of $`GL(m)`$ on $`^m`$. It induces an action on the $`k`$-th tensor power $`^k^m`$. Now the symmetric group $`S_k`$ in $`k`$ letters acts on $`^k^m`$ in a natural way. These two actions commute and we may thus decompose $`^k^m`$ into a direct sum of irreducible $`GL(m)\times S_k`$-module. Recalling that the irreducible representations of symmetric group $`S_k`$ admit parameterization by Young diagrams of weight $`k`$, Schur duality states that

$$^k^m\underset{\lambda }{}V_m^\lambda M_k^\lambda ,$$

where the summation is over Young diagrams $`\lambda `$ of size $`k`$ and of length not exceeding $`m`$. Here $`M_k^\lambda `$ is the irreducible representation of $`S_k`$ corresponding to the Young diagram $`\lambda `$.

Further well known examples of a multiplicity-free action of $`gl(m)`$ that are of interest to us are as follows: consider the action of $`gl(m)`$ on the symmetric square $`S^2^m`$ and skew-symmetric square $`\mathrm{\Lambda }^2^m`$. We have an induced action on their respective symmetric algebras $`S(S^2^m)`$ and $`S(\mathrm{\Lambda }^2^m)`$. Explicitly, the decomposition of these spaces as $`gl(m)`$-modules is as follows (cf. \[H2\], \[GW\]):

(2.3) $`S(S^2^m)`$ $`{\displaystyle \underset{l(\lambda )m}{}}V_m^{2\lambda },`$
(2.4) $`S(\mathrm{\Lambda }^2^m)`$ $`{\displaystyle \underset{l(\lambda )\frac{m}{2}}{}}V_m^{(2\lambda )^{}}.`$
Explicit formulas for the highest weight vectors in either cases are well known (cf. \[H2\]) and are given in Remark 5.1.

One may also consider the decompositions of the skew-symmetric algebra of $`S^2^m`$ and $`\mathrm{\Lambda }^2^m`$. In order to describe the highest weights that appear in these decompositions we need a few terminology. The Young diagram associated to the partition $`\lambda =(k+1,1,,\mathrm{},1)`$ of length $`k1`$ is called a $`(k+1,k)`$-hook. We will sometimes also call this $`(k+1,k)`$-hook a hook of shape $`(k+1,k)`$. Assuming that $`k>l`$ we may form a new Young diagram by “nesting” the $`(l+1,l)`$-hook inside the $`(k+1,k)`$-hook. The resulting partition of length $`k`$ is $`(k+1,l+2,2,\mathrm{},2,1,\mathrm{},1)`$, where $`2`$ appears $`l1`$ times and $`1`$ appears $`kl1`$ times. Similarly a sequence of hooks of shapes $`(k_1+1,k_1),\mathrm{},(k_s+1,k_s)`$ with $`k_i>k_{i+1}`$ for $`i=1,\mathrm{},s1`$ may be nested, and the resulting partition has length $`k_1`$. In consistency with the terminology used we call the partition $`(k,1,\mathrm{},1)`$ of length $`k+1`$ a hook of shape $`(k,k+1)`$ or a $`(k,k+1)`$-hook. Nesting of hooks of shapes $`(k_1,k_1+1),\mathrm{},(k_s,k_s+1)`$ with $`k_i>k_{i+1}`$ for $`i=1,\mathrm{},s1`$ is done in an analogous fashion.

Now we can state the following multiplicity-free decompositions of $`gl(m)`$-modules (cf. \[H2\], \[GW\]):

(2.5) $`\mathrm{\Lambda }(S^2^m){\displaystyle \underset{\lambda }{}}V_m^\lambda ,`$
(2.6) $`\mathrm{\Lambda }(\mathrm{\Lambda }^2^m){\displaystyle \underset{\mu }{}}V_m^\mu ,`$
where $`\lambda `$ (respectively $`\mu `$) is over all partitions with $`l(\lambda )m`$ (respectively $`l(\mu )m)`$ such that $`\lambda `$ (respectively $`\mu `$) is obtained by nesting a sequence of $`(k+1,k)`$-hooks (respectively of $`(k,k+1)`$-hooks).

## 3. Multiplicity-free Actions of the general linear Lie superalgebra

Let $`^{m|n}`$ denote the superspace of superdimension $`m|n`$. Recall that this means that $`^{m|n}`$ is a $`_2`$-graded space, where the even subspace has dimension $`m`$ and the odd subspace has dimension $`n`$. The space of linear maps from $`^{m|n}`$ to itself can be regarded as the space of $`(m+n)\times (m+n)`$ matrices with an induced $`_2`$-gradation, which gives it a natural structure as a Lie superalgebra, denoted by $`gl(m|n)`$. We have a triangular decomposition $`gl(m|n)=gl(m|n)_1+gl(m|n)_0+gl(m|n)_1`$, where $`gl(m|n)_{\pm 1}`$ denote the set of upper and lower triangular matrices and $`gl(m|n)_0`$ denotes the set of diagonal matrices. Given an $`m+n`$ tuple of complex numbers $`(a_1,\mathrm{},a_m;b_1,\mathrm{},b_n)`$, we associate an irreducible $`gl(m|n)`$-module $`V_{m|n}`$ of highest weight $`(a_1,\mathrm{},a_m;b_1,\mathrm{},b_n)`$ (with respect to the standard Borel subalgebra $`gl(m|n)_0+gl(m|n)_1`$). It is well known (cf. e.g. \[K\]) that the module $`V_{m|n}`$ is finite-dimensional if and only if $`(a_1,\mathrm{},a_m;b_1,\mathrm{},b_n)`$ satisfies the conditions $`a_ia_{i+1},b_jb_{j+1}_+`$, for all $`i=1,\mathrm{},m1`$ and $`j=1,\mathrm{},n1`$.

Let $`^{p|q}`$ and $`^{m|n}`$ denote complex superspaces of superdimensions $`p|q`$ and $`m|n`$, respectively. We will now describe a duality between the Lie superalgebras $`gl(p|q)`$ and $`gl(m|n)`$. Our starting point is Schur duality for Lie superalgebra $`gl(m|n)`$.

Schur duality for the Lie superalgebra $`gl(m|n)`$ was studied in \[Se\]. Below we will recall the main result for the convenience of the reader. Let $`^{m|n}`$ denote the standard $`gl(m|n)`$-module. We may, as in the classical case, consider the $`k`$-th tensor power $`^k^{m|n}`$ which admits a natural action of the Lie superalgebra $`gl(m|n)`$. On the other hand the symmetric group $`S_k`$ acts naturally on $`^k^{m|n}`$ by permutations with appropriate signs (corresponding to the permutations of odd elements in $`^{m|n}`$). It is easy to check that the actions of $`gl(m|n)`$ and $`S_k`$ commute with each other, cf. Sergeev \[Se\] (also see Berele-Regev \[BR\] for a more detailed study).

###### Theorem 3.1 (Sergeev).

As a $`gl(m|n)\times S_k`$-module we have

$$^k^{m|n}\underset{\lambda }{}V_{m|n}^\lambda M_k^\lambda ,$$

where $`\lambda `$ is summed over Young diagrams of size $`k`$ such that $`\lambda _{m+1}n`$, $`M_k^\lambda `$ is the irreducible $`S_k`$-module parameterized by $`\lambda `$, and $`V_{m|n}^\lambda `$ denotes the irreducible $`gl(m|n)`$-module with highest weight $`(\lambda _1,\lambda _2,\mathrm{},\lambda _m;\lambda _1^{}m,\mathrm{},\lambda _n^{}m)`$, where we denote $`l=l`$ for $`l_+`$ and $`l=0`$, otherwise.

The symmetric algebra $`S(^{p|q}^{m|n})`$ is by definition equal to the tensor product of the symmetric algebra of the even part of $`^{p|q}^{m|n}`$ and the skew-symmetric algebra of the odd part of $`^{p|q}^{m|n}`$. It admits a natural gradation

$`S(^{p|q}^{m|n})={\displaystyle \underset{k0}{}}S^k(^{p|q}^{m|n})`$
by letting the degree of the basis elements of $`^{p|q}^{m|n}`$ be $`1`$. The natural actions of $`gl(p|q)`$ on $`^{p|q}`$ and $`gl(m|n)`$ on $`^{m|n}`$ induce commuting actions on the $`k`$-th symmetric algebra $`S^k(^{p|q}^{m|n})`$. Indeed $`gl(p|q)`$ and $`gl(m|n)`$ are mutual centralizers in $`gl(^{p|q}^{m|n})`$. We obtain the following theorem by an analogous argument as in \[H2\].

###### Theorem 3.2.

The symmetric algebra $`S(^{p|q}^{m|n})`$ is multiplicity-free as a module over $`gl(p|q)\times gl(m|n)`$. More explicitly, we have the following decomposition

$$S(^{p|q}^{m|n})\underset{\lambda }{}V_{p|q}^\lambda V_{m|n}^\lambda ,$$

where the sum is over Young diagrams $`\lambda `$ satisfying $`\lambda _{p+1}q`$ and $`\lambda _{m+1}n`$. Here the highest weight of the module $`V_{p|q}^\lambda `$ (respectively $`V_{m|n}^\lambda `$) is given by $`(\lambda _1,\mathrm{},\lambda _p;\lambda _1^{}p,\mathrm{},\lambda _q^{}p)`$ (resp. $`(\lambda _1,\mathrm{},\lambda _m;\lambda _1^{}m,\mathrm{},\lambda _n^{}m)`$).

###### Proof.

By the definition of the $`k`$-th supersymmetric algebra we have

$$S^k(^{m|n}^{p|q})((^k^{m|n})(^k^{p|q}))^{\mathrm{\Delta }_k},$$

where $`\mathrm{\Delta }_k`$ is the diagonal subgroup of $`S_k\times S_k`$. By Theorem 3.1 we have therefore

$`S(^{m|n}^{p|q})`$ $`{\displaystyle \underset{k=0}{\overset{\mathrm{}}{}}}(({\displaystyle \underset{|\lambda |=k}{}}V_{m|n}^\lambda M_k^\lambda )({\displaystyle \underset{|\mu |=k}{}}V_{p|q}^\mu M_k^\mu ))^{\mathrm{\Delta }_k}`$
$`{\displaystyle \underset{k=0}{\overset{\mathrm{}}{}}}{\displaystyle \underset{|\lambda |=|\mu |=k}{}}(V_{m|n}^\lambda V_{p|q}^\mu )(M_k^\lambda M_k^\mu )^{\mathrm{\Delta }_k}`$
$`{\displaystyle \underset{k=0}{\overset{\mathrm{}}{}}}{\displaystyle \underset{|\lambda |=k}{}}(V_{m|n}^\lambda V_{p|q}^\lambda )`$
$`{\displaystyle \underset{\lambda }{}}(V_{m|n}^\lambda V_{p|q}^\lambda ),`$
where $`\lambda `$ in the previous line is summed over all Young diagrams satisfying the conditions $`\lambda _{m+1}n`$ and $`\lambda _{p+1}q`$. The second to last equality follows from the well-known fact that $`M_k^\lambda `$ is a self-contragredient module. ∎

###### Remark 3.1.

1. This theorem (except the explicit formula for the highest weights) was first obtained in Brini et al \[BPT\] in a combinatorial approach. It is also obtained independently recently by Sergeev.
2. When $`n=q=0`$, we recover the $`(gl(p),gl(m))`$-duality in the symmetric algebra case. When $`q=m=0`$ we recover the $`(gl(p),gl(n))`$-duality in the skew-symmetric algebra case.
3. One can easily show that the $`(gl(m|n),gl(k))`$-duality implies the $`(gl(m|n),S_k)`$ Schur duality (Theorem 3.1), using an argument of Howe (cf. 2.4, \[H2\]).

The next corollary is immediate from Theorem 3.2.

###### Corollary 3.1.

The image of the action of the universal enveloping algebras of $`gl(p|q)`$ and $`gl(m|n)`$ on $`S^k(^{p|q}^{m|n})`$ are double commutants.

###### Theorem 3.3.

The skew-symmetric algebra $`\mathrm{\Lambda }(^{p|q}^{m|n})`$ is multiplicity-free as a module over $`gl(p|q)\times gl(m|n)`$. More explicitly, we have the following decomposition

$$\mathrm{\Lambda }(^{p|q}^{m|n})\underset{\lambda }{}V_{p|q}^\lambda V_{m|n}^\lambda ^{},$$

where the sum is over Young diagrams $`\lambda `$ satisfying $`\lambda _{p+1}q`$ and $`\lambda _{m+1}^{}n`$. Here the highest weight of the module $`V_{p|q}^\lambda `$ (respectively $`V_{m|n}^\lambda ^{}`$) is given by $`(\lambda _1,\mathrm{},\lambda _p;\lambda _1^{}p,\mathrm{},\lambda _q^{}p)`$ (resp. $`(\lambda _1^{},\mathrm{},\lambda _m^{};\lambda _1m,\mathrm{},\lambda _nm)`$).

###### Proof.

By the definition of the $`k`$-th skew-symmetric algebra we have

$$\mathrm{\Lambda }^k(^{p|q}^{m|n})((^k^{p|q})(^k^{m|n}))^{\mathrm{\Delta }_k},$$

where $`\mathrm{\Delta }_k`$ is the diagonal subgroup of $`S_k\times S_k`$ and $`(^k^{m|n}))^{\mathrm{\Delta }_k}`$ is the subspace of $`(^k^{m|n}))`$ that transforms according to the sign character of $`\mathrm{\Delta }_k`$. By Theorem 3.1 we have therefore

$`\mathrm{\Lambda }(^{p|q}^{m|n})`$ $`{\displaystyle \underset{k=0}{\overset{\mathrm{}}{}}}\left(({\displaystyle \underset{|\lambda |=k}{}}V_{p|q}^\lambda M_k^\lambda )({\displaystyle \underset{|\mu |=k}{}}V_{m|n}^\mu M_k^\mu )\right)^{\mathrm{\Delta }_k}`$
$`{\displaystyle \underset{k=0}{\overset{\mathrm{}}{}}}{\displaystyle \underset{|\lambda |=|\mu |=k}{}}(V_{p|q}^\lambda V_{m|n}^\mu )(M_k^\lambda M_k^\mu )^{\mathrm{\Delta }_k}`$
$`{\displaystyle \underset{k=0}{\overset{\mathrm{}}{}}}{\displaystyle \underset{|\lambda |=k}{}}V_{p|q}^\lambda V_{m|n}^\lambda ^{}`$
$`{\displaystyle \underset{\lambda }{}}V_{p|q}^\lambda V_{m|n}^\lambda ^{},`$
where $`\lambda `$ in the previous line is summed over all Young diagrams satisfying the conditions $`\lambda _{p+1}q`$ and $`\lambda _{m+1}^{}n`$. The second to last equality follows from the following well known facts: $`M_k^\lambda `$ is a self-contragredient module and tensoring the module $`M_k^\lambda `$ with the sign character yields the module $`M_k^\lambda ^{}`$. ∎

###### Remark 3.2.

Of course it follows from Theorem 3.3 that the image of the action of the universal enveloping algebras of $`gl(p|q)`$ and $`gl(m|n)`$ on $`\mathrm{\Lambda }^k(^{p|q}^{m|n})`$ are also double commutants.

The following corollary turns out to be very useful later on in order to check that a given vector is indeed a highest weight vector inside $`S(^{p|q}^{m|n})`$.

###### Corollary 3.2.

Assume a vector $`vS(^{p|q}^{m|n})`$ has the weight $`\lambda `$ with respect to $`gl(p|q)\times gl(m|n)`$ associated to a Young diagram $`\lambda `$ satisfying $`\lambda _{p+1}q`$ and $`\lambda _{m+1}n`$. If $`v`$ is a highest weight vector for $`gl(p|q)`$, then it is for $`gl(m|n)`$ as well.

###### Proof.

Since $`v`$ is a highest weight vector for $`gl(p|q)`$ with weight $`\lambda `$, it belongs to the subspace $`W`$ of $`S(^{p|q}^{m|n})`$ which consists of vectors with weight $`\lambda `$ annihilated by the standard Borel in $`gl(p|q)`$. By Theorem 3.2, $`W`$ is isomorphic to $`V_{m|n}^\lambda `$ as a $`gl(m|n)`$-module. There exists a unique vector (up to scalar multiple) in $`V_{m|n}^\lambda `$ which has weight $`\lambda `$, which is the highest weight vector. By assumption $`v`$ has weight $`\lambda `$ as a $`gl(m|n)`$-module, so it is a highest weight vector for $`gl(m|n)`$. ∎

The description of highest weight vectors of the irreducible $`gl(p|q)\times gl(m|n)`$-modules in the symmetric algebra turns out to be much more subtle than in the classical Howe duality case and we will deal with this question in Section 4.

Next consider the symmetric square $`S^2^{m|n}`$ of the natural representation of $`gl(m|n)`$. The following theorem can be proved by an analogous argument as in \[H2\]. This result was also obtained independently recently by Sergeev. We omit the proof since it is in any case parallel to the proof of Theorem 3.5 below.

###### Theorem 3.4.

The symmetric algebra of the symmetric square of the natural representation $`^{m|n}`$ of the Lie superalgebra $`gl(m|n)`$ is a completely reducible multiplicity-free $`gl(m|n)`$-module. More precisely we have the following decomposition

$$S^k(S^2^{m|n})=\underset{\lambda }{}V_{m|n}^\lambda ,$$

where the summation is over all partitions $`\lambda `$ into even parts of size $`2k`$ and $`\lambda _{m+1}n`$.

Now $`S^2^{m|n}`$ reduces to $`S^2^m`$ in the case when $`n=0`$, and to $`\mathrm{\Lambda }^2^n`$ in the case when $`m=0`$, the symmetric and skew-symmetric square of the natural representation of $`gl(m)`$ and $`gl(n)`$, respectively. Thus one obtains as a corollary the classical multiplicity-free decompositions of their respective symmetric algebras, namely (2.3) and (2.4). Again the question of obtaining explicit formulas for the highest weight vectors inside $`S(S^2^{m|n})`$ is substantially more subtle than in the non-super case. We will give these in Section 5.

###### Theorem 3.5.

The skew-symmetric algebra of the symmetric square of the natural representation $`^{m|n}`$ of the Lie superalgebra $`gl(m|n)`$ is a completely reducible multiplicity-free $`gl(m|n)`$-module. More precisely we have the following decomposition

$$\mathrm{\Lambda }^k(S^2^{m|n})=\underset{\lambda }{}V_{m|n}^\lambda ,$$

where the summation is over all partitions $`\lambda `$ of size $`2k`$, which are obtained by nesting $`(l+1,l)`$-hooks with $`\lambda _{m+1}n`$.

###### Proof.

Our argument follows closely the one given in the proof of Theorem 4.4.2 in \[H2\] with Theorem 3.1 replacing the classical Schur duality. Let $`D_k`$ denote the the subgroup of $`S_{2k}`$, which preserves the partition $`\{\{1,2\},\{3,4\},\mathrm{},\{2k1,2k\}\}`$ of $`2k`$. Note that $`D_k`$ is isomorphic to a semidirect product of $`S_k`$ and $`(_2)^k`$, where $`_2`$ acts by interchanging $`2j1`$ with $`2j`$ and $`S_k`$ acts by permuting the pairs. Let $`\mathrm{sign}`$ denote the character on $`D_k`$ which is trivial on $`(_2)^k`$, but transforms by the sign character on $`S_k`$. We observe that

$$\mathrm{\Lambda }^{2k}(S^2^{m|n})(\stackrel{2k}{}^{m|n})^{D_k,\mathrm{sign}}.$$

Thus using Theorem 3.1 we obtain

$$\mathrm{\Lambda }^{2k}(S^2^{m|n})\underset{|\lambda |=2k}{}(V_{m|n}^\lambda M_{2k}^\lambda )^{D_k,\mathrm{sign}}\underset{|\lambda |=2k}{}V_{m|n}^\lambda (M_{2k}^\lambda )^{D_k,\mathrm{sign}}.$$

Now by Theorem A1.4 of \[H2\] the space $`(M_{2k}^\lambda )^{D_k,\mathrm{sign}}`$ is non-zero if and only if $`\lambda `$ is constructed from nesting hooks of types $`(l+1,l)`$, in which case it is one-dimensional. ∎

Similarly we obtain as a corollary the classical multiplicity-free decompositions (2.5) and (2.6).

###### Remark 3.3.

The character of $`V_{m|n}^\lambda `$ is defined as the trace of the action of the diagonal matrix $`\mathrm{diag}(x_1,\mathrm{},x_m;y_1,\mathrm{},y_n)`$ in $`gl(m|n)`$ on $`V_{m|n}^\lambda `$ and according to \[BR\] is given by so-called hook Schur functions $`\mathrm{HS}_\lambda (x,y)`$ (see \[BR\] for definition). Thus, comparing the characters of both sides of Theorem 3.2 and Theorem 3.3, respectively, with $`x=(x_1,\mathrm{},x_p)`$, $`y=(y_1,\mathrm{},y_q)`$, $`u=(u_1,\mathrm{},u_m)`$ and $`v=(v_1,\mathrm{},v_n)`$ we obtain the following combinatorial identities:

$`{\displaystyle \underset{\lambda }{}}\mathrm{HS}_\lambda (x,y)\mathrm{HS}_\lambda (u,v)`$ $`={\displaystyle \underset{i,j,k,l}{}}(1x_iu_k)^1(1y_jv_l)^1(1+x_iv_l)(1+y_ju_k),`$
$`{\displaystyle \underset{\lambda }{}}\mathrm{HS}_\lambda (x,y)\mathrm{HS}_\lambda ^{}(u,v)`$ $`={\displaystyle \underset{i,j,k,l}{}}(1+x_iu_k)(1+y_jv_l)(1x_iv_l)^1(1y_ju_k)^1,`$
where $`1ip`$, $`1jq`$, $`1km`$ and $`1ln`$ with summation in the first identity over $`\lambda `$ such that $`\lambda _{p+1}q`$ and $`\lambda _{m+1}n`$ and in the second one over $`\lambda `$ such that $`\lambda _{p+1}q`$ and $`\lambda _{m+1}^{}n`$. Now putting $`y=v=0`$ in the first identity we obtain the classical Cauchy identity, while putting respectively $`y=u=0`$ the dual Cauchy identity (see e.g. \[M\]):

$`{\displaystyle \underset{\lambda }{}}s_\lambda (x)s_\lambda (y)={\displaystyle \underset{i,j}{}}(1x_iy_j)^1,`$
$`{\displaystyle \underset{\mu }{}}s_\mu (x)s_\lambda ^{}(y)={\displaystyle \underset{i,j}{}}(1+x_iy_j).`$
###### Remark 3.4.

Similarly Theorem 3.4 and Theorem 3.5 give rise to the following combinatorial identities ($`x=(x_1,\mathrm{},x_m)`$ and $`y=(y_1,\mathrm{},y_n)`$):

$`{\displaystyle \underset{\lambda }{}}\mathrm{HS}_\lambda (x,y)`$ $`={\displaystyle \underset{ii^{},j<j^{}}{}}(1x_ix_i^{})^1(1y_jy_j^{})^1{\displaystyle \underset{i,j}{}}(1+x_iy_j),`$
$`{\displaystyle \underset{\mu }{}}\mathrm{HS}_\mu (x,y)`$ $`={\displaystyle \underset{ii^{},j<j^{}}{}}(1+x_ix_i^{})(1+y_jy_j^{}){\displaystyle \underset{i,j}{}}(1x_iy_j)^1,`$
where in the first identity the sum is over all partitions $`\lambda `$ with even rows such that $`\lambda _{m+1}n`$ and in the second over all partitions $`\mu `$ that can be obtained by nesting $`(k+1,k)`$-hooks such that $`\mu _{m+1}n`$ and $`1i,i^{}m`$, $`1j,j^{}n`$. Putting either $`x=0`$ or $`y=0`$ in these two identities we obtain the following classical Schur function identities (see e.g. \[M\]), which correspond to the decompositions in (2.3), (2.4), (2.5) and (2.6), respectively:

$`{\displaystyle \underset{l(\lambda )m}{}}s_{2\lambda }={\displaystyle \underset{1im}{}}(1x_i^2)^1{\displaystyle \underset{1i<jm}{}}(1x_ix_j)^1,`$
$`{\displaystyle \underset{l(\mu ^{})\frac{m}{2}}{}}s_{(2\mu )^{}}={\displaystyle \underset{1i<jm}{}}(1x_ix_j)^1,`$
$`{\displaystyle \underset{\rho }{}}s_\rho ={\displaystyle \underset{1im}{}}(1+x_i^2){\displaystyle \underset{1i<jm}{}}(1+x_ix_j),`$
$`{\displaystyle \underset{\pi }{}}s_\pi ={\displaystyle \underset{1i<jm}{}}(1+x_ix_j),`$
where $`\rho `$ (respectively $`\pi `$) above is summed over all nested sequences of hooks of shape $`(k+1,k)`$ with $`km`$ (respectively of hooks of shape $`(k,k+1)`$ with $`km1`$).

## 4. Construction of highest weight vectors in $`S(^{p|q}^{m|n})`$

This section is devoted to the construction of the highest weight vectors of $`gl(p|q)\times gl(m|n)`$ inside the symmetric algebra of $`^{p|q}^{m|n}`$. We will divide this section into several cases. Before we embark on this task we will set the notation to be used throughout this section.

We let $`e^1,\mathrm{},e^p;f^1,\mathrm{},f^q`$ denote the standard homogeneous basis for the standard $`gl(p|q)`$-module. Here $`e^i`$ are even, while $`f^j`$ are odd basis elements. Similarly we let $`e_1,\mathrm{},e_m;f_1,\mathrm{},f_n`$ denote the standard homogeneous basis for the standard $`gl(m|n)`$-module. The weights of $`e^i,f^j,e_l`$ and $`f_k`$ are denoted by $`\stackrel{~}{ϵ}_i`$, $`\stackrel{~}{\delta }_j`$, $`ϵ_l`$ and $`\delta _k`$, for $`1ip`$, $`1jq`$, $`1lm`$ and $`1kn`$, respectively. We set

(4.1)
$$x_l^i:=e_le^i;\xi _l^j:=e_lf^j;\eta _k^i:=f_ke^i;y_k^j:=f_kf^j.$$

We will denote by $`[𝐱,\xi ,\eta ,𝐲]`$ the polynomial superalgebra generated by (4.1). The commuting pair of $`gl(p|q)`$ and $`gl(m|n)`$ may be realized as first order differential operators as follows ($`1i,i^{}p;1l,l^{}q`$ and $`1s,s^{}m;1k,k^{}n`$):

(4.2) $`{\displaystyle \underset{j=1}{\overset{m}{}}}x_j^i{\displaystyle \frac{}{x_j^i^{}}}+{\displaystyle \underset{j=1}{\overset{n}{}}}\eta _j^i{\displaystyle \frac{}{\eta _j^i^{}}},{\displaystyle \underset{j=1}{\overset{m}{}}}\xi _j^l{\displaystyle \frac{}{\xi _j^l^{}}}+{\displaystyle \underset{j=1}{\overset{n}{}}}y_j^l{\displaystyle \frac{}{y_j^l^{}}},`$
$`{\displaystyle \underset{j=1}{\overset{m}{}}}x_j^i{\displaystyle \frac{}{\xi _j^l^{}}}+{\displaystyle \underset{j=1}{\overset{n}{}}}\eta _j^i{\displaystyle \frac{}{y_j^l^{}}},{\displaystyle \underset{j=1}{\overset{m}{}}}\xi _j^l{\displaystyle \frac{}{x_j^i^{}}}+{\displaystyle \underset{j=1}{\overset{n}{}}}y_j^l{\displaystyle \frac{}{\eta _j^i^{}}},`$
(4.3) $`{\displaystyle \underset{j=1}{\overset{p}{}}}x_s^j{\displaystyle \frac{}{x_s^{}^j}}+{\displaystyle \underset{j=1}{\overset{q}{}}}\xi _s^j{\displaystyle \frac{}{\xi _s^{}^j}},{\displaystyle \underset{j=1}{\overset{p}{}}}\eta _k^{}^j{\displaystyle \frac{}{\eta _k^j}}+{\displaystyle \underset{j=1}{\overset{q}{}}}y_k^{}^j{\displaystyle \frac{}{y_k^j}},`$
$`{\displaystyle \underset{j=1}{\overset{p}{}}}x_s^j{\displaystyle \frac{}{\eta _k^j}}{\displaystyle \underset{j=1}{\overset{q}{}}}\xi _s^j{\displaystyle \frac{}{y_k^j}},{\displaystyle \underset{j=1}{\overset{p}{}}}\eta _k^j{\displaystyle \frac{}{x_s^j}}{\displaystyle \underset{j=1}{\overset{q}{}}}y_k^j{\displaystyle \frac{}{\xi _s^j}}.`$
(4.2) spans a copy of $`gl(p|q)`$, while (4.3) spans a copy of $`gl(m|n)`$.

Our Cartan subalgebras of $`gl(p|q)`$ and $`gl(m|n)`$ are spanned, respectively, by

$$\underset{j=1}{\overset{m}{}}x_j^i\frac{}{x_j^i}+\underset{j=1}{\overset{n}{}}\eta _j^i\frac{}{\eta _j^i},\underset{j=1}{\overset{m}{}}\xi _j^l\frac{}{\xi _j^l}+\underset{j=1}{\overset{n}{}}y_j^l\frac{}{y_j^l},$$

and

$$\underset{j=1}{\overset{p}{}}x_s^j\frac{}{x_s^j}+\underset{j=1}{\overset{q}{}}\xi _s^j\frac{}{\xi _s^j},\underset{j=1}{\overset{p}{}}\eta _k^j\frac{}{\eta _k^j}+\underset{j=1}{\overset{q}{}}y_k^j\frac{}{y_k^j},$$

while the nilpotent radicals are respectively generated by the simple root vectors

$`{\displaystyle \underset{j=1}{\overset{m}{}}}x_j^{i1}{\displaystyle \frac{}{x_j^i}}+{\displaystyle \underset{j=1}{\overset{n}{}}}\eta _j^{i1}{\displaystyle \frac{}{\eta _j^i}},{\displaystyle \underset{j=1}{\overset{m}{}}}\xi _j^{l1}{\displaystyle \frac{}{\xi _j^l}}+{\displaystyle \underset{j=1}{\overset{n}{}}}y_j^{l1}{\displaystyle \frac{}{y_j^l}},`$
(4.4) $`{\displaystyle \underset{j=1}{\overset{m}{}}}x_j^p{\displaystyle \frac{}{\xi _j^1}}+{\displaystyle \underset{j=1}{\overset{n}{}}}\eta _j^p{\displaystyle \frac{}{y_j^1}},1<ip,1<lq.`$
and

$`{\displaystyle \underset{j=1}{\overset{p}{}}}x_{s1}^j{\displaystyle \frac{}{x_s^j}}+{\displaystyle \underset{j=1}{\overset{q}{}}}\xi _{s1}^j{\displaystyle \frac{}{\xi _s^j}},{\displaystyle \underset{j=1}{\overset{p}{}}}\eta _{k1}^j{\displaystyle \frac{}{\eta _k^j}}+{\displaystyle \underset{j=1}{\overset{q}{}}}y_{k1}^j{\displaystyle \frac{}{y_k^j}},`$
(4.5) $`{\displaystyle \underset{j=1}{\overset{p}{}}}x_m^j{\displaystyle \frac{}{\eta _1^j}}{\displaystyle \underset{j=1}{\overset{q}{}}}\xi _m^j{\displaystyle \frac{}{y_1^j}},1<sm,1<kn.`$
With these conventions, we may thus identify $`S(^{p|q}^{m|n})`$ with the polynomial superalgebra $`[𝐱,\xi ,\eta ,𝐲]`$ (as $`gl(p|q)\times gl(m|n)`$-modules).

### 4.1. Highest Weight Vectors: the Case $`q=0`$

In this section we will describe the highest weight vectors for $`gl(p)\times gl(m|n)`$ in the symmetric algebra $`S(^p^{m|n})`$, i.e. $`q=0`$ case. The space $`S(^p^{m|n})`$ is identified with $`[𝐱,\eta ]`$, and (4) and (4) reduce to

(4.6) $`{\displaystyle \underset{j=1}{\overset{m}{}}}x_j^{i1}{\displaystyle \frac{}{x_j^i}}+{\displaystyle \underset{j=1}{\overset{n}{}}}\eta _j^{i1}{\displaystyle \frac{}{\eta _j^i}},`$
(4.7) $`{\displaystyle \underset{j=1}{\overset{p}{}}}x_{s1}^j{\displaystyle \frac{}{x_s^j}},{\displaystyle \underset{j=1}{\overset{p}{}}}\eta _{k1}^j{\displaystyle \frac{}{\eta _k^j}},{\displaystyle \underset{j=1}{\overset{p}{}}}x_m^j{\displaystyle \frac{}{\eta _1^j}},`$
respectively. Now by Theorem 3.2 a highest weight representation $`V_p^\lambda V_{m|n}^\lambda `$ of $`gl(p)\times gl(m|n)`$ appears in the decomposition of $`S^k(^p^{m|n})`$ if and only if $`\lambda `$ is of size $`k`$ and of length at most $`p`$ such that $`\lambda _{m+1}n`$.

We will consider two cases separately, namely $`mp`$ and $`m<p`$.

We begin with the case of $`mp`$. Here the condition $`\lambda _{m+1}n`$ is an empty condition. So we are looking for homogeneous polynomials of degree $`k`$ in $`[𝐱,\eta ]`$, annihilated by all vectors of (4.6) and (4.7), and having $`gl(p)`$\- and $`gl(m|n)`$-weight $`\lambda `$ of length not exceeding $`p`$. If $`\lambda `$ is such a weight, then $`\lambda _i^{}p`$, where we recall that $`\lambda ^{}=(\lambda _1^{},\mathrm{},\lambda _t^{})`$ denotes the transpose of $`\lambda `$. It is easy to see that the product $`\mathrm{\Delta }_{\lambda _1^{}}\mathrm{}\mathrm{\Delta }_{\lambda _t^{}}`$ is annihilated by all vectors of (4.6) and (4.7), where we recall that $`\mathrm{\Delta }_r`$ is defined in (2.2). It is straightforward to check that its weight is exactly $`\lambda `$.

###### Theorem 4.1.

In the case when $`mp`$, all $`gl(p)\times gl(m|n)`$ highest weight vectors in $`[𝐱,\eta ]`$ form an abelian semigroup generated by $`\mathrm{\Delta }_r`$, for $`r=1,\mathrm{},p`$. The highest weight vector associated to the weight $`\lambda `$ is given by the product $`\mathrm{\Delta }_{\lambda _1^{}}\mathrm{}\mathrm{\Delta }_{\lambda _t^{}}`$.

We now consider the case $`p>m`$. In this case the condition $`\lambda _{m+1}n`$ is no longer an empty condition. Obviously the highest weight vectors associated to Young diagrams $`\lambda `$ with $`\lambda _{m+1}=0`$ can be obtained just as in the previous case.

Now suppose $`\lambda `$ is a diagram of length exceeding $`m`$. Let $`\lambda _1^{},\lambda _2^{},\mathrm{},\lambda _t^{}`$ denote its column lengths as usual. We have $`p\lambda _1^{}\lambda _2^{}\mathrm{}\lambda _t^{}`$ and $`m\lambda _{n+1}^{}`$. For $`mrp`$, the following determinant of an $`r\times r`$ matrix plays a fundamental role in this paper:

(4.8)
$$\mathrm{\Delta }_{k,r}:=\mathrm{det}\left(\begin{array}{cccc}x_1^1& x_1^2& \mathrm{}& x_1^r\\ x_2^1& x_2^2& \mathrm{}& x_2^r\\ \mathrm{}& \mathrm{}& \mathrm{}& \mathrm{}\\ x_m^1& x_m^2& \mathrm{}& x_m^r\\ \eta _k^1& \eta _k^2& \mathrm{}& \eta _k^r\\ \eta _k^1& \eta _k^2& \mathrm{}& \eta _k^r\\ \mathrm{}& \mathrm{}& \mathrm{}& \mathrm{}\\ \eta _k^1& \eta _k^2& \mathrm{}& \eta _k^r\end{array}\right),k=1,\mathrm{},n.$$

That is, the first $`m`$ rows are filled by the vectors $`(x_j^1,\mathrm{},x_j^r)`$, for $`j=1,\mathrm{},m`$, in increasing order and the last $`rm`$ rows are filled with the same vector $`(\eta _k^1,\mathrm{},\eta _k^r)`$. Since the matrix entries involve Grassmann variables $`\eta _k^i`$, we must specify what we mean by the determinant. By the determinant of a matrix

$$A:=\left(\begin{array}{cccc}a_1^1& a_1^2& \mathrm{}& a_1^r\\ a_2^1& a_2^2& \mathrm{}& a_2^r\\ \mathrm{}& \mathrm{}& \mathrm{}& \mathrm{}\\ a_r^1& a_p^2& \mathrm{}& a_r^r\end{array}\right),$$

whose matrix entries involve Grassmann variables $`\eta _k^i`$, we will always mean the expression $`_{\sigma S_r}(1)^{p(\sigma )}a_1^{\sigma (1)}a_2^{\sigma (2)}\mathrm{}a_r^{\sigma (r)}`$, where $`p(\sigma )`$ is the length of $`\sigma `$ in the symmetric group $`S_r`$. In general it is not true that $`\mathrm{det}A=\mathrm{det}A^t`$.

###### Remark 4.1.

The determinant (4.8) is always nonzero. It reduces to (2.2) when $`m=r`$, and reduces to (up to a scalar multiple) $`\eta _k^1\mathrm{}\eta _k^r`$ when $`m=0`$.

Now let $`\lambda `$ be a diagram of length at most $`p`$ such that $`\lambda _{m+1}n`$. It is thus of the following shape:

..............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................$`\mathrm{}`$$`\mathrm{}`$$`\mathrm{}`$$`\mathrm{}`$$`\mathrm{}`$$`m\{\text{ }`$$`pm\{\text{ }`$$`\underset{r}{\underset{}{\text{ }}}`$$`\underset{tr}{\underset{}{\text{ }}}`$
where $`r`$ is defined by $`\lambda _r^{}>m`$ and $`\lambda _{r+1}^{}m`$. We can divide such a diagram into two diagrams, namely

(4.9) ......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................$`m\{\text{ }`$$`pm\{\text{ }`$$`\underset{r}{\underset{}{\text{ }}}`$........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................$`m\{\text{ }`$$`\underset{tr}{\underset{}{\text{ }}}`$
Now the second diagram in (4.9) has length not exceeding $`m`$, so its associated highest weight vector is given by the product $`\mathrm{\Delta }_{\lambda _{r+1}^{}}\mathrm{}\mathrm{\Delta }_{\lambda _t^{}}`$. A formula for the highest weight vector associated to the first diagram in (4.9) is given by the following proposition. We will denote by $`_{k=1}^r\mathrm{\Delta }_{k,\lambda _k^{}}`$ the (ordered) product $`\mathrm{\Delta }_{1,\lambda _1^{}}\mathrm{\Delta }_{2,\lambda _2^{}}\mathrm{}\mathrm{\Delta }_{r,\lambda _r^{}}`$.

###### Proposition 4.1.

Let $`p\lambda _1^{}\lambda _2^{}\mathrm{}\lambda _r^{}>m`$. Then $`_{k=1}^r\mathrm{\Delta }_{k,\lambda _k^{}}`$ is a highest weight vector associated to the first Young diagram in (4.9).

###### Proof.

Observe that $`_{k=1}^r\mathrm{\Delta }_{k,\lambda _k^{}}`$ has the same $`gl(p)\times gl(m|n)`$-weight as the first Young diagram of (4.9). Clearly $`_{k=1}^r\mathrm{\Delta }_{k,\lambda _k^{}}`$ is non-zero. It is straightforward to verify that $`_{k=1}^r\mathrm{\Delta }_{k,\lambda _k^{}}`$ is annihilated by the operators in (4.6). It follows from Corollary 3.2 that $`_{k=1}^r\mathrm{\Delta }_{k,\lambda _k^{}}`$ is also a highest weight vector for $`gl(m|n)`$. ∎

Our next theorem follows by observing that the product of the highest weight vectors corresponding to the two Young diagrams in (4.9) is non-zero and is a highest weight vector associated to the Young diagram $`\lambda `$.

###### Theorem 4.2.

Suppose that $`m<p`$. An irreducible highest weight module $`V_p^\lambda V_{m|n}^\lambda `$ appearing in $`[𝐱,\eta ]`$ if and only if $`\lambda `$ corresponds to a Young diagram $`\lambda `$ of length not exceeding p and $`\lambda _{m+1}n`$. Furthermore a highest weight vector associated to such a $`\lambda `$ is given by

$$\underset{k=1}{\overset{r}{}}\mathrm{\Delta }_{k,\lambda _k^{}}\underset{j=r+1}{\overset{t}{}}\mathrm{\Delta }_{\lambda _j^{}},$$

where $`r`$ is defined by $`\lambda _r^{}>m`$ and $`\lambda _{r+1}^{}m`$.

As a corollary we obtain the following useful combinatorial identity, which will play an important role later on.

###### Corollary 4.1.

Let $`x_l^i`$ be even variables for $`i=1,\mathrm{},p`$ and $`l=1,\mathrm{},m`$ with $`pq>m`$. Let $`\eta _1^i`$ and $`\eta _2^i`$ be odd variables for $`i=1,\mathrm{},p`$. Then

$$\mathrm{det}\left(\begin{array}{cccc}x_1^1& x_1^2& \mathrm{}& x_1^p\\ x_2^1& x_2^2& \mathrm{}& x_2^p\\ \mathrm{}& \mathrm{}& \mathrm{}& \mathrm{}\\ x_m^1& x_m^2& \mathrm{}& x_m^p\\ \eta _1^1& \eta _1^2& \mathrm{}& \eta _1^p\\ \eta _1^1& \eta _1^2& \mathrm{}& \eta _1^p\\ \mathrm{}& \mathrm{}& \mathrm{}& \mathrm{}\\ \mathrm{}& \mathrm{}& \mathrm{}& \mathrm{}\\ \eta _1^1& \eta _1^2& \mathrm{}& \eta _1^p\end{array}\right)\mathrm{det}\left(\begin{array}{cccc}x_1^1& x_1^2& \mathrm{}& x_1^q\\ x_2^1& x_2^2& \mathrm{}& x_2^q\\ \mathrm{}& \mathrm{}& \mathrm{}& \mathrm{}\\ x_m^1& x_m^2& \mathrm{}& x_m^q\\ \eta _1^1& \eta _1^2& \mathrm{}& \eta _1^q\\ \eta _2^1& \eta _2^2& \mathrm{}& \eta _2^q\\ \mathrm{}& \mathrm{}& \mathrm{}& \mathrm{}\\ \eta _2^1& \eta _2^2& \mathrm{}& \eta _2^q\end{array}\right)=0.$$

###### Proof.

Consider $`\mathrm{\Delta }_{1,p}`$ and $`\mathrm{\Delta }_{2,q}`$ where $`pq`$. By Theorem 4.2 the product $`\mathrm{\Delta }_{1,p}\mathrm{\Delta }_{2,q}`$ is a highest weight vector and thus is annihilated by all operators in (4.7). In particular applying the operator $`_{j=1}^p\eta _1^j\frac{}{\eta _2^j}`$ to $`\mathrm{\Delta }_{1,p}\mathrm{\Delta }_{2,q}`$ and dividing by $`(qm)`$ we obtain the desired identity. ∎

###### Remark 4.2.

The above corollary gives rise to identities involving minors in even variables $`x`$’s by looking at the coefficient of a fixed Grassmann monomial involving $`\eta `$’s. We do not know of other direct proof of these identities.

It is well known (cf. \[OV\]) that as a $`gl(p)`$-module $`S^i(^p)\mathrm{\Lambda }^j(^p)`$, for $`j=1,\mathrm{},p`$, decomposes into a direct sum two irreducible components of highest weights $`iϵ_1+_{k=1}^jϵ_k`$ and $`iϵ_1+_{k=2}^{j+1}ϵ_k`$, respectively. We can also get this result from Theorem 4.2 and in addition obtain explicit formulas of the highest weight vectors. To do so consider $`S^k(^p^{1|1})S^k(^{p|p})_{i+j=k}S^i(^{p|0})\mathrm{\Lambda }^j(^{0|p})`$. Now according to Theorem 4.2 all the $`gl(p)\times gl(1|1)`$ highest weight vectors inside $`S^k(^p^{1|1})`$ are given by $`(x_1^1)^i\mathrm{\Delta }_{1,j}`$, where $`j=1,\mathrm{},p`$ and $`i+j=k`$. These vectors are of course $`gl(p)`$ highest weight vectors. Now a simple calculation shows that applying the negative root vector of $`gl(1|1)`$ to $`(x_1^1)^i\mathrm{\Delta }_{1,j}`$ we obtain a non-zero multiple of $`(x_1^1)^i\eta _1^1\eta _1^2\mathrm{}\eta _1^j`$, while applying the negative root vector again gives zero. Thus the vectors $`(x_1^1)^i\mathrm{\Delta }_{1,j}`$ and $`(x_1^1)^i\eta _1^1\eta _1^2\mathrm{}\eta _1^j`$ exhaust all $`gl(p)`$ highest weight vectors inside the space $`_{i+j=k}S^i(^p)\mathrm{\Lambda }^j(^p)`$. To conclude the proof we observe that the vectors $`(x_1^1)^{i1}\mathrm{\Delta }_{1,j+1}`$ and $`(x_1^1)^i\eta _1^1\eta _1^2\mathrm{}\eta _1^j`$ lie in $`S^i(^p)\mathrm{\Lambda }^j(^p)`$, with weights $`iϵ_1+_{k=1}^jϵ_k`$ and $`iϵ_1+_{k=2}^{j+1}ϵ_k`$, respectively.

### 4.2. Highest Weight Vectors: the Case $`p=m`$

In this section we shall find $`gl(m|q)\times gl(m|n)`$ highest weight vectors that appear in the decomposition of $`[𝐱,\xi ,\eta ,𝐲]`$. By Theorem 3.2 we need to construct a vector in $`[𝐱,\xi ,\eta ,𝐲]`$ annihilated by all operators in (4) and (4) of weight corresponding to the Young diagram $`\lambda `$

(4.10) .........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................$`\mathrm{}`$$`\mathrm{}`$$`\mathrm{}`$$`\mathrm{}`$$`\mathrm{}`$$`m\{\text{ }`$$`s\{\text{ }`$$`\underset{tr}{\underset{}{\text{ }}}`$$`\underset{r}{\underset{}{\text{ }}}`$
where $`rmin(q,n)`$ (which we will always assume for this section).

First we remark that if $`\lambda `$ has length less than or equal to $`m`$ then it is easy to check that a formula for the corresponding highest weight vector is given by $`\mathrm{\Delta }_{\lambda _1^{}}\mathrm{}\mathrm{\Delta }_{\lambda _t^{}}`$. So we may assume that the length of $`\lambda `$ exceeds $`m`$.

As before we cut up this Young diagram into two diagrams, namely

(4.11) .........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................$`\mathrm{}`$$`\mathrm{}`$$`m\{\text{ }`$$`s\{\text{ }`$$`\underset{r}{\underset{}{\text{ }}}`$...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................$`\mathrm{}`$$`\mathrm{}`$$`m\{\text{ }`$$`\underset{tr}{\underset{}{\text{ }}}`$
Denoting the second diagram by $`\mu `$ and $`v`$ a highest weight vector associated to the first diagram, it is easy to see that the product $`v\mathrm{\Delta }_{\mu _1^{}}\mathrm{}\mathrm{\Delta }_{\mu _{tr}^{}}`$is a highest weight vector for the diagram $`\lambda `$. Thus our task reduces to finding a highest weight vector associated to the first diagram in (4.11).

We claim that a highest weight vector associated to the first diagram in (4.11) can be essentially obtained by taking a product of those associated to $`s`$ diagrams of rectangular shape

(4.12) ..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................$`m\{\text{ }`$$`1\{\text{ }`$$`\underset{\lambda _{m+i}}{\underset{}{\text{ }}}`$
and dividing by a suitable power of $`\mathrm{\Delta }_m`$. Indeed taking the product of two highest weight vectors for the Young diagram of shape (4.12) of widths $`\lambda _{m+i}`$ and $`\lambda _{m+i+1}`$ respectively produces a highest weight vector for the Young diagram

.......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................$`m\{\text{ }`$$`2\{\text{ }`$$`\underset{\lambda _{m+i+1}}{\underset{}{\text{ }}}`$$`\underset{\lambda _{m+i+1}}{\underset{}{\text{ }}}`$$`\underset{\lambda _{m+i}}{\underset{}{\text{ }}}`$
Once we verify that the product is non-zero, we may divide it by $`(\mathrm{\Delta }_m)^{\lambda _{m+i+1}}`$ and the resulting vector is a highest weight vector for the diagram

.........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................$`m\{\text{ }`$$`2\{\text{ }`$$`\underset{\lambda _{m+i+1}}{\underset{}{\text{ }}}`$$`\underset{\lambda _{m+i}}{\underset{}{\text{ }}}`$
Similarly by taking a product of $`s`$ such vectors associated to the $`s`$ diagrams of the form (4.12) of widths $`\lambda _{m+1},\mathrm{},\lambda _{m+s}`$, respectively, and dividing by $`\mathrm{\Delta }_m^{\lambda _{m+2}+\mathrm{}+\lambda _{m+s}}`$ we obtain a highest weight vector associated to the first diagram of (4.11). So our task now is to find a formula for a highest weight vector corresponding to a Young diagram of shape (4.12). (From the explicit formula it will follow immediately that a product of $`s`$ vectors of such type is non-zero).

Let us put $`r=\lambda _{m+i}`$ in (4.12). We define the $`r\times r`$ matrix $`Y`$ and the $`m\times m`$ matrix $`X`$ as follows:

(4.13)
$$Y:=\left(\begin{array}{cccc}y_1^1& y_1^2& \mathrm{}& y_1^r\\ y_2^1& y_2^2& \mathrm{}& y_2^r\\ \mathrm{}& \mathrm{}& \mathrm{}& \mathrm{}\\ y_r^1& y_r^2& \mathrm{}& y_r^r\end{array}\right),X:=\left(\begin{array}{cccc}x_1^1& x_1^2& \mathrm{}& x_1^m\\ x_2^1& x_2^2& \mathrm{}& x_2^m\\ \mathrm{}& \mathrm{}& \mathrm{}& \mathrm{}\\ x_m^1& x_m^2& \mathrm{}& x_m^m\end{array}\right).$$

Given a Young diagram $`\lambda `$ of rectangular shape (see (4.12)) consisting of $`m`$ rows and $`r`$ columns, we consider marked diagrams $`D`$ obtained by marking the boxes in $`\lambda `$ subject to the restriction that each column can contain no more than one marked box. For example the following is a marked diagram in the case $`r=6`$ and $`m=4`$:

(4.14) ..............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................$`1`$$`2`$$`3`$$`4`$$`5`$$`6`$$`1`$$`2`$$`3`$$`4`$XXXXX
To each such a marked diagram $`D`$ we may associate an $`r\times r`$ matrix $`Y_D`$ obtained from $`Y`$ as follows. For each marked box, say in the $`i`$-th column and $`j`$-th row, we replace the $`i`$-th row of the matrix $`Y`$ by the vector $`(\xi _j^1,\xi _j^2,\mathrm{},\xi _j^r)`$. The resulting matrix will be denoted by $`Y_D`$. For instance in our example (4.14) the matrix $`Y_D`$ is

$$Y_D=\left(\begin{array}{cccccc}\xi _1^1& \xi _1^2& \xi _1^3& \xi _1^4& \xi _1^5& \xi _1^6\\ y_1^1& y_1^2& y_1^3& y_1^4& y_1^5& y_1^6\\ \xi _3^1& \xi _3^2& \xi _3^3& \xi _3^4& \xi _3^5& \xi _3^6\\ \xi _2^1& \xi _2^2& \xi _2^3& \xi _2^4& \xi _2^5& \xi _2^6\\ \xi _2^1& \xi _2^2& \xi _2^3& \xi _2^4& \xi _2^5& \xi _2^6\\ \xi _3^1& \xi _3^2& \xi _3^3& \xi _3^4& \xi _3^5& \xi _3^6\end{array}\right).$$

To each such diagram $`D`$ we may also associate $`r`$ $`m\times m`$ matrices $`X_i`$ $`(i=1,\mathrm{},r)`$ obtained from $`X`$ as follows. If the $`i`$-th column of $`D`$ is not marked, then $`X_i=X`$. If the $`i`$-th column is marked at the $`j`$-th row, then $`X_i`$ is the matrix obtained from $`X`$ by replacing its $`j`$-th row by the vector $`(\eta _i^1,\eta _i^2,\mathrm{},\eta _i^m)`$. As an illustration, the diagram in our example (4.14) gives rise to the matrices

$$X_1=\left(\begin{array}{cccc}\eta _1^1& \eta _1^2& \eta _1^3& \eta _1^4\\ x_2^1& x_2^2& x_2^3& x_2^4\\ x_3^1& x_3^2& x_3^3& x_3^4\\ x_4^1& x_4^2& x_4^3& x_4^4\end{array}\right),X_2=X,X_3=\left(\begin{array}{cccc}x_1^1& x_1^2& x_1^3& x_1^4\\ x_2^1& x_2^2& x_2^3& x_2^4\\ \eta _3^1& \eta _3^2& \eta _3^3& \eta _3^4\\ x_4^1& x_4^2& x_4^3& x_4^4\end{array}\right)\mathrm{etc}.$$

Let $`|D|`$ denote the total number of marked boxes in the diagram $`D`$. Set

$$\mathrm{\Delta }_D:=\mathrm{det}X_D\mathrm{det}Y_D,$$

where by $`\mathrm{det}X_D`$ we mean $`_{i=1}^r\mathrm{det}X_i`$ arranged in increasing order. We can now state the following theorem.

###### Theorem 4.3.

The vector $`_D(1)^{\frac{1}{2}|D|(|D|1)}\mathrm{\Delta }_D`$ is a $`gl(m|q)\times gl(m|n)`$ highest weight vector in $`[𝐱,\xi ,\eta ,𝐲]`$ corresponding to the rectangular Young diagram of length $`m+1`$ and width $`r`$, where the summation over $`D`$ ranges over all possible marked $`m\times r`$ diagrams.

###### Proof.

We first show that $`_D(1)^{\frac{1}{2}|D|(|D|1)}\mathrm{\Delta }_D`$ indeed has the correct weight.

First note that diagram (4.12) corresponds to the $`gl(m|q)\times gl(m|n)`$-weight $`_{i=1}^mrϵ_i+_{i=1}^mr\stackrel{~}{ϵ}_i+_{j=1}^r\delta _j+_{j=1}^r\stackrel{~}{\delta }_j`$. Let $`\stackrel{~}{D}_i`$, $`j=1,\mathrm{},m`$, denote the $`m`$ disjoint subsets of $`\{1,\mathrm{},r\}`$ defined by the condition that $`j\stackrel{~}{D}_i`$ if and only if $`D`$ contains a marked box at its $`j`$-th column and $`i`$-th row. Put $`\stackrel{~}{D}=_{i=1}^m\stackrel{~}{D}_i`$ and $`\stackrel{~}{D}^c=\{1,\mathrm{},r\}\stackrel{~}{D}`$. The weight of $`\mathrm{det}Y_D`$ is $`_{j\stackrel{~}{D}^c}\delta _j+_{i=1}^m|\stackrel{~}{D}_i|ϵ_i+_{j=1}^r\stackrel{~}{\delta }_j`$. Now $`\mathrm{det}X_D`$ has weight $`r_{i=1}^mϵ_i_{i=1}^m|\stackrel{~}{D}_i|ϵ_i+_{j\stackrel{~}{D}}\delta _j+r_{i=1}^m\stackrel{~}{ϵ}_i`$. Hence each $`\mathrm{det}Y_D\mathrm{det}X_D`$ has weight $`_{i=1}^mrϵ_i+_{i=1}^mr\stackrel{~}{ϵ}_i+_{j=1}^r\delta _j+_{j=1}^r\stackrel{~}{\delta }_j`$, as required.

Hence by Corollary 3.2 it is sufficient to show that (4) annihilates it, namely

(4.15) $`\left({\displaystyle \underset{j=1}{\overset{m}{}}}x_{s1}^j{\displaystyle \frac{}{x_s^j}}+{\displaystyle \underset{j=1}{\overset{q}{}}}\xi _{s1}^j{\displaystyle \frac{}{\xi _s^j}}\right)({\displaystyle \underset{D}{}}(1)^{\frac{1}{2}|D|(|D|1)}\mathrm{\Delta }_D)`$ $`=`$ $`0,`$
(4.16) $`\left({\displaystyle \underset{j=1}{\overset{m}{}}}\eta _{s1}^j{\displaystyle \frac{}{\eta _s^j}}+{\displaystyle \underset{j=1}{\overset{q}{}}}y_{s1}^j{\displaystyle \frac{}{y_s^j}}\right)({\displaystyle \underset{D}{}}(1)^{\frac{1}{2}|D|(|D|1)}\mathrm{\Delta }_D)`$ $`=`$ $`0,`$
(4.17) $`\left({\displaystyle \underset{j=1}{\overset{m}{}}}x_m^j{\displaystyle \frac{}{\eta _1^j}}{\displaystyle \underset{j=1}{\overset{q}{}}}\xi _m^j{\displaystyle \frac{}{y_1^j}}\right)({\displaystyle \underset{D}{}}(1)^{\frac{1}{2}|D|(|D|1)}\mathrm{\Delta }_D)`$ $`=`$ $`0.`$
We will first establish (4.15). Note that the simple root vector $`_{j=1}^mx_{s1}^j\frac{}{x_s^j}+_{j=1}^q\xi _{s1}^j\frac{}{\xi _s^j}`$ maps the vectors $`(x_s^1,\mathrm{},x_s^m)`$ to $`(x_{s1}^1,\mathrm{},x_{s1}^m)`$ and $`(\xi _s^1,\mathrm{},\xi _s^q)`$ to $`(\xi _{s1}^1,\mathrm{},\xi _{s1}^q)`$. For a diagram $`D`$, let us denote by $`D_{_s^{s1}}`$ a diagram obtained from $`D`$ by moving each marked box in its $`s`$-th row to the box above it in the $`s1`$-st row. Analogously we define $`D_{_s^{s1}}`$ a diagram obtained from $`D`$ by moving each marked box in the $`s1`$-st row to the box below it in the $`s`$-th row. It is easy to check

$`\left({\displaystyle \underset{j=1}{\overset{m}{}}}x_{s1}^j{\displaystyle \frac{}{x_s^j}}+{\displaystyle \underset{j=1}{\overset{q}{}}}\xi _{s1}^j{\displaystyle \frac{}{\xi _s^j}}\right)`$ $`(\mathrm{\Delta }_D)=`$
$`{\displaystyle \underset{D_{_s^{s1}}}{}}\mathrm{det}X_D\mathrm{det}Y_{D_{_s^{s1}}}{\displaystyle \underset{D_{_s^{s1}}}{}}\mathrm{det}X_{D_{_s^{s1}}}\mathrm{det}Y_D.`$
Thus we have

$`({\displaystyle \underset{j=1}{\overset{m}{}}}x_{s1}^j{\displaystyle \frac{}{x_s^j}}`$ $`+{\displaystyle \underset{j=1}{\overset{q}{}}}\xi _{s1}^j{\displaystyle \frac{}{\xi _s^j}})({\displaystyle \underset{|D|=k}{}}\mathrm{\Delta }_D)=`$
(4.18) $`{\displaystyle \underset{|D|=k}{}}\left({\displaystyle \underset{D_{_s^{s1}}}{}}\mathrm{det}X_D\mathrm{det}Y_{D_{_s^{s1}}}{\displaystyle \underset{D_{_s^{s1}}}{}}\mathrm{det}X_{D_{_s^{s1}}}\mathrm{det}Y_D\right).`$
But evidently $`_{|D|=k}\mathrm{det}X_{D_{_s^{s1}}}\mathrm{det}Y_D=_{|D|=k}\mathrm{det}X_D\mathrm{det}Y_{D_{_s^{s1}}}`$ thanks to the equality $`(D_{_s^{s1}})_{_s^{s1}}=D`$. Hence the right-hand side of (4.18) is zero, proving (4.15).

Our next step is to prove (4.16). In this case $`_{j=1}^m\eta _{s1}^j\frac{}{\eta _s^j}+_{j=1}^qy_{s1}^j\frac{}{y_s^j}`$ maps the vectors $`(\eta _s^1,\mathrm{},\eta _s^m)`$ to $`(\eta _{s1}^1,\mathrm{},\eta _{s1}^m)`$ and $`(y_s^1,\mathrm{},y_s^q)`$ to $`(y_{s1}^1,\mathrm{},y_{s1}^q)`$. For a diagram $`D`$ such that $`j\stackrel{~}{D}_i`$ and $`l\stackrel{~}{D}_i`$ we denote by $`D_{jl}`$ the diagram obtained from $`D`$ by removing $`j`$ from $`\stackrel{~}{D}_i`$ and adding $`l`$ to $`\stackrel{~}{D}_i`$. For a fixed $`k`$ we write

$$\underset{|D|=k}{}\mathrm{\Delta }_D=\underset{|D|=k}{}\left(\underset{s,s1\stackrel{~}{D}}{}\mathrm{\Delta }_D+\underset{s\stackrel{~}{D}\mathrm{or}s1\stackrel{~}{D}}{}\mathrm{\Delta }_D\right).$$

First observe that

$`\left({\displaystyle \underset{j=1}{\overset{m}{}}}\eta _{s1}^j{\displaystyle \frac{}{\eta _s^j}}+{\displaystyle \underset{j=1}{\overset{q}{}}}y_{s1}^j{\displaystyle \frac{}{y_s^j}}\right)({\displaystyle \underset{|D|=k}{}}{\displaystyle \underset{s,s1\stackrel{~}{D}}{}}\mathrm{\Delta }_D)`$
$`=`$ $`{\displaystyle \underset{|D|=k}{}}{\displaystyle \underset{s,s1\stackrel{~}{D}}{}}\left(\left({\displaystyle \underset{j=1}{\overset{m}{}}}\eta _{s1}^j{\displaystyle \frac{}{\eta _s^j}}+{\displaystyle \underset{j=1}{\overset{q}{}}}y_{s1}^j{\displaystyle \frac{}{y_s^j}}\right)(\mathrm{det}X_D)\right)\mathrm{det}Y_D`$
$`=`$ $`0.`$
This is because if $`s,s1\stackrel{~}{D}_i`$, for some $`i`$, then the term

$`\left({\displaystyle \underset{j=1}{\overset{m}{}}}\eta _{s1}^j{\displaystyle \frac{}{\eta _s^j}}+{\displaystyle \underset{j=1}{\overset{q}{}}}y_{s1}^j{\displaystyle \frac{}{y_s^j}}\right)(\mathrm{det}X_D)`$
$`=`$ $`\mathrm{}\mathrm{det}X_{s1}(i)\mathrm{det}X_{s1}(i)\mathrm{}`$
$`=`$ $`0,`$
where in general $`X_b(a)`$ is the matrix obtained from $`X`$ by replacing the $`a`$-th row with the vector $`(\eta _b^1,\mathrm{},\eta _b^m)`$.

Now if $`s\stackrel{~}{D}_i`$ and $`s1\stackrel{~}{D}_l`$, then

$$\left(\underset{j=1}{\overset{m}{}}\eta _{s1}^j\frac{}{\eta _s^j}+\underset{j=1}{\overset{q}{}}y_{s1}^j\frac{}{y_s^j}\right)(\mathrm{det}X_D)=\mathrm{}\mathrm{det}X_{s1}(l)\mathrm{det}X_{s1}(i)\mathrm{}.$$

Let $`D^{}`$ be the same diagram as $`D`$, except $`s\stackrel{~}{D^{}}_l`$ and $`s1\stackrel{~}{D^{}}_i`$. Then we have

$$\left(\underset{j=1}{\overset{m}{}}\eta _{s1}^j\frac{}{\eta _s^j}+\underset{j=1}{\overset{q}{}}y_{s1}^j\frac{}{y_s^j}\right)(\mathrm{det}X_D^{})=\mathrm{}\mathrm{det}X_{s1}(i)\mathrm{det}X_{s1}(l)\mathrm{}.$$

Of course $`Y_D=Y_D^{}`$ and $`\mathrm{det}X_{s1}(i)`$ anticommutes with $`\mathrm{det}X_{s1}(l)`$, so

$$\left(\underset{j=1}{\overset{m}{}}\eta _{s1}^j\frac{}{\eta _s^j}+\underset{j=1}{\overset{q}{}}y_{s1}^j\frac{}{y_s^j}\right)(\mathrm{det}X_D\mathrm{det}Y_D+\mathrm{det}X_D^{}\mathrm{det}Y_D^{})=0.$$

Next we observe that if $`D`$ is a diagram such that $`s,s1\stackrel{~}{D}`$ then

$$\left(\underset{j=1}{\overset{m}{}}\eta _{s1}^j\frac{}{\eta _s^j}+\underset{j=1}{\overset{q}{}}y_{s1}^j\frac{}{y_s^j}\right)(\mathrm{det}X_D\mathrm{det}Y_D)=0,$$

so that our task of proving (4.16) reduces to proving that

$`\left({\displaystyle \underset{j=1}{\overset{m}{}}}\eta _{s1}^j{\displaystyle \frac{}{\eta _s^j}}+{\displaystyle \underset{j=1}{\overset{q}{}}}y_{s1}^j{\displaystyle \frac{}{y_s^j}}\right)`$ $`\left({\displaystyle \underset{\stackrel{s\stackrel{~}{D}}{s1\stackrel{~}{D}}}{}}\mathrm{det}X_D\mathrm{det}Y_D+{\displaystyle \underset{\stackrel{s1\stackrel{~}{D}}{s\stackrel{~}{D}}}{}}\mathrm{det}X_D\mathrm{det}Y_D\right)`$
(4.19) $`=`$ $`0,`$
where the sum is over all diagrams $`D`$ with $`|D|=k`$. But the left-hand side of (4.2) is equal to

$$\underset{s\stackrel{~}{D},s1\stackrel{~}{D}}{}\mathrm{det}X_{D_{ss1}}\mathrm{det}Y_D\underset{s1\stackrel{~}{D},s\stackrel{~}{D}}{}\mathrm{det}X_D\mathrm{det}Y_{D_{s1s}}=0.$$

To complete the proof we now need to verify (4.17). The odd simple root vector $`_{j=1}^mx_m^j\frac{}{\eta _1^j}_{j=1}^q\xi _m^j\frac{}{y_1^j}`$ has the effect of changing the vectors $`(\eta _1^1,\mathrm{},\eta _1^m)`$ to $`(x_m^1,\mathrm{},x_m^m)`$ and $`(y_1^1,\mathrm{},y_1^q)`$ to $`(\xi _m^1,\mathrm{},\xi _m^q)`$. If $`D`$ is a diagram such that $`1D_j`$ with $`jm`$, then

$$\left(\underset{j=1}{\overset{m}{}}x_m^j\frac{}{\eta _1^j}\underset{j=1}{\overset{q}{}}\xi _m^j\frac{}{y_1^j}\right)\mathrm{\Delta }_D=0.$$

Thus

$`\left({\displaystyle \underset{j=1}{\overset{m}{}}}x_m^j{\displaystyle \frac{}{\eta _1^j}}{\displaystyle \underset{j=1}{\overset{q}{}}}\xi _m^j{\displaystyle \frac{}{y_1^j}}\right)({\displaystyle \underset{D}{}}(1)^{\frac{1}{2}|D|(|D|1)}\mathrm{\Delta }_D)=`$
(4.20) $`\left({\displaystyle \underset{j=1}{\overset{m}{}}}x_m^j{\displaystyle \frac{}{\eta _1^j}}{\displaystyle \underset{j=1}{\overset{q}{}}}\xi _m^j{\displaystyle \frac{}{y_1^j}}\right)({\displaystyle \underset{D,1\stackrel{~}{D}_m}{}}(1)^{\frac{1}{2}|D|(|D|1)}\mathrm{\Delta }_D+{\displaystyle \underset{D,1\stackrel{~}{D}}{}}(1)^{\frac{1}{2}|D|(|D|1)}\mathrm{\Delta }_D).`$
For a diagram $`D`$ with $`1\stackrel{~}{D}`$ (resp. with $`1\stackrel{~}{D}_m`$) we denote by $`D^+`$ (resp. $`D^{}`$) the diagram obtained from $`D`$ by adding $`1`$ to $`\stackrel{~}{D}_m`$ (resp. by removing $`1`$ from $`\stackrel{~}{D}`$). Then (4.20) becomes

(4.21)
$$\underset{D,1\stackrel{~}{D}_m}{}(1)^{\frac{1}{2}|D|(|D|1)}\mathrm{det}X_D^{}\mathrm{det}Y_D\underset{D,1\stackrel{~}{D}}{}(1)^{\frac{1}{2}|D|(|D|1)+|D|}\mathrm{det}X_D\mathrm{det}Y_{D^+}.$$

Setting $`D^{}=D^{}`$ in the first sum of (4.21) we may rewrite (4.21) as

$`{\displaystyle \underset{D^{},1\stackrel{~}{D^{}}}{}}(1)^{\frac{1}{2}|D^{}|(|D^{}|+1)}`$ $`\mathrm{det}X_D^{}\mathrm{det}Y_{D_{}^{}{}_{}{}^{+}}`$
$``$ $`{\displaystyle \underset{D,1\stackrel{~}{D}}{}}(1)^{\frac{1}{2}|D|(|D|1)+|D|}\mathrm{det}X_D\mathrm{det}Y_{D^+}=0.`$
We will denote the vector $`_D(1)^{\frac{1}{2}|D|(|D|1)}\mathrm{\Delta }_D`$ by $`\mathrm{\Gamma }_r`$. It is clear that a product of $`\mathrm{\Gamma }_r`$’s (not necessary for the same value $`r`$) remains nonzero. Thus a highest weight vector for an arbitrary Young diagram of shape (4.10) can be constructed using such vectors, as described earlier in this section. We summarize the results in this section in the following theorem.

###### Theorem 4.4.

An irreducible representation $`V_{m|q}^\lambda V_{m|n}^\lambda `$ of $`gl(m|q)\times gl(m|n)`$ appears in the decomposition of $`[𝐱,\xi ,\eta ,𝐲]`$ if and only if $`\lambda `$ is associated to a Young diagram with $`\lambda _{m+1}\mathrm{min}(q,n)`$. Let $`t`$ be the length of $`\lambda ^{}`$. Then

* if the length of $`\lambda `$ does not exceed $`m`$, then a highest weight vector is given by $`\mathrm{\Delta }_{\lambda _1^{}}\mathrm{}\mathrm{\Delta }_{\lambda _t^{}}`$.
* if the length of $`\lambda `$ is $`m+s`$, $`s1`$, let $`0r\mathrm{min}(q,n)`$ be such that $`\lambda _r^{}>m`$ and $`\lambda _{r+1}^{}m`$, then a highest weight vector corresponding to $`\lambda `$ is given by

(4.22)
$$(\mathrm{\Delta }_m)^{(\lambda _{m+2}+\mathrm{}+\lambda _{m+s})}\mathrm{\Gamma }_{\lambda _{m+1}}\mathrm{\Gamma }_{\lambda _{m+2}}\mathrm{}\mathrm{\Gamma }_{\lambda _{m+s}}\mathrm{\Delta }_{\lambda _{r+1}^{}}\mathrm{}\mathrm{\Delta }_{\lambda _t^{}}.$$

We will obtain a more explicit formula for (4.22) in the next section.

### 4.3. Highest Weight Vectors: the General Case

We consider now the general case. Without loss of generality we may assume $`pm`$.

According to Theorem 3.2 an irreducible $`gl(p|q)\times gl(m|n)`$-module $`V_{p|q}^\lambda V_{m|n}^\lambda `$ appears in the decomposition of $`[𝐱,\xi ,\eta ,𝐲]`$ if and only if $`\lambda _{m+1}n`$ and $`\lambda _{p+1}q`$. If the length of the Young diagram $`\lambda `$ is less than or equal to $`m`$, then $`\mathrm{\Delta }_{\lambda _1^{}}\mathrm{}\mathrm{\Delta }_{\lambda _t^{}}`$ is the desired highest weight vector, where $`t`$ is the length of $`\lambda ^{}`$. If the length of $`\lambda `$ exceeds $`m`$, but is less than or equal to $`p`$, then we see that the vector given in Theorem 4.2 provides a formula for the highest weight vector in this case as well. Thus it remains to study the case when the length of $`\lambda `$ exceeds $`p`$.

So we are to consider a Young diagram of the form:

(4.23) ..........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................$`\mathrm{}`$$`\mathrm{}`$$`\mathrm{}`$$`\mathrm{}`$$`m\{\text{ }`$$`pm\{\text{ }`$$`s\{\text{ }`$$`\underset{r^{}r}{\underset{}{\text{ }}}`$$`\underset{tr^{}}{\underset{}{\text{ }}}`$$`\underset{r}{\underset{}{\text{ }}}`$
In the case when $`qn`$, the numbers $`r,r^{}`$ satisfying the conditions $`0rq`$, $`0r^{}n`$ and $`rr^{}`$ are determined as follows: $`\lambda _r^{}>p`$ and $`\lambda _{r+1}^{}p`$, $`\lambda _r^{}^{}>m`$ and $`\lambda _{r^{}+1}^{}m`$. In the case when $`qn`$, the numbers $`r,r^{}`$ satisfying $`0rr^{}n`$ are defined in exactly the same way. In either case we may split (4.23) into three diagrams:

(4.24) ...........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................$`m\{\text{ }`$$`pm\{\text{ }`$$`m\{\text{ }`$$`pm\{\text{ }`$$`s\{\text{ }`$$`m\{\text{ }`$$`\underset{r^{}r}{\underset{}{\text{ }}}`$$`\underset{tr^{}}{\underset{}{\text{ }}}`$$`\underset{r}{\underset{}{\text{ }}}`$
We associate the vectors $`\mathrm{\Delta }_{r+1,\lambda _{r+1}^{}}\mathrm{}\mathrm{\Delta }_{r^{},\lambda _r^{}^{}}`$ and $`\mathrm{\Delta }_{\lambda _{r^{}+1}^{}}\mathrm{}\mathrm{\Delta }_{\lambda _t^{}}`$ to the second and third diagrams in (4.24) respectively. Below we will construct a highest weight vector for the first diagram in (4.24). From the formula it will be easy to see that the product of these three vectors is a highest weight vector for the Young diagram (4.23).

The above discussion thus reduces our question to finding a $`gl(p|q)\times gl(m|n)`$-highest weight vector corresponding to a Young diagram of type:

(4.25) ......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................$`m\{\text{ }`$$`pm\{\text{ }`$$`s\{\text{ }`$$`\underset{r}{\underset{}{\text{ }}}`$
where $`r\mathrm{min}(q,n)`$. The difficulty of finding a highest weight vector associated to such a diagram lies in the fact that the highest weight vectors in $`[𝐱,\xi ,\eta ,𝐲]`$ no longer form a semigroup in general.

We will now outline our strategy. We need to find highest weight vectors in $`[𝐱,\xi ,\eta ,𝐲]`$, annihilated by (4) and having weight corresponding to the Young diagram in (4.25). Recall that $`𝐱`$ denotes the set of even variables $`\{x_l^i|1ip,1lm\}`$. We introduce a new set of even variables $`x_l^i,`$ $`1ip,m<lp`$, and denote by $`𝐱^{}=\{x_l^i|1i,lp\}`$, the union of our old set with this new set. We shall construct certain vectors in $`[𝐱^{},\xi ,\eta ,𝐲]`$, which can been shown, using our results in the previous section, that they are annihilated by (4). A priori these vectors lie in $`[𝐱^{},\xi ,\eta ,𝐲]`$ so that such vectors do not make sense. However, we will show that these vectors, after dividing by a suitable power of the determinant of the $`p\times p`$ matrix $`(x_l^i)`$, are in fact independent of the variables $`\{x_l^i|1ip,m+1lp\}`$, and thus lie in $`[𝐱,\xi ,\eta ,𝐲]`$.

Consider a marked diagram having $`m`$ rows and $`r`$ columns with at most $`r`$ marked boxes subject to the constraint that at most one marked box appears on each column. To such a diagram $`D`$ we have associated in the previous section a matrix $`Y_D`$, which is obtained from the $`r\times r`$ matrix $`Y`$ (see (4.13)) by suitably replacing its rows. To each such diagram we now associate $`p\times p`$ matrices $`X_i`$, for $`1ir`$, similar to the ones in the previous section: Let $`X`$ denote the $`p\times p`$ matrix:

(4.26)
$$X:=\left(\begin{array}{cccc}x_1^1& x_1^2& \mathrm{}& x_1^p\\ x_2^1& x_2^2& \mathrm{}& x_2^p\\ \mathrm{}& \mathrm{}& \mathrm{}& \mathrm{}\\ x_m^1& x_m^2& \mathrm{}& x_m^p\\ x_{m+1}^1& x_{m+1}^2& \mathrm{}& x_{m+1}^p\\ \mathrm{}& \mathrm{}& \mathrm{}& \mathrm{}\\ x_p^1& x_1^2& \mathrm{}& x_p^p\end{array}\right).$$

If the $`i`$-th column of $`D`$ is not marked, then $`X_i=X`$. If the $`i`$-th column is marked at the $`j`$-th row, then $`X_i`$ is the matrix obtained from $`X`$ by replacing its $`j`$-th row by the vector $`(\eta _i^1,\eta _i^2,\mathrm{},\eta _i^p)`$. Note that none of its $`m+1`$-st to $`p`$-th rows are replaced. As before we define $`\mathrm{det}X_D:=_{i=1}^r\mathrm{det}X_i`$ (arranged in increasing order) and define $`\mathrm{\Gamma }_r=_D(1)^{\frac{1}{2}|D|(D|1)}\mathrm{det}X_D\mathrm{det}Y_D`$.

The proof of Theorem 4.3 carries over word for word to prove

###### Proposition 4.2.

The vector $`\mathrm{\Gamma }_r`$ is annihilated by (4).

In the case when $`p=m`$ a highest weight vector for the first Young diagram in (4.11) is obtained essentially by taking product of highest weight vectors for the Young diagram of type (4.12). In the case when $`p>m`$ one can verify that this procedure cannot be carried out, as such products are necessarily zero. Hence in this case we will need to find a general formula for the Young diagram $`\lambda `$ of shape (4.25). To do so we will first consider matrices that will play the same role in the case of $`pm`$ as the $`X_i`$’s play in the case $`p=m`$. As we will generalize diagrams to include those that allow more than one marked box on each column, we are led to study combinatorial identities of determinants of matrices obtained from $`X`$ that have more than one row replaced by an odd vector. This leads us to define the following types of determinants.

Let $`X`$ be the $`p\times p`$ matrix as in (4.26) and let $`(\eta _j^1,\mathrm{},\eta _j^p)`$ be an odd vector. Let $`I`$ be a subset of $`\{1,\mathrm{},p\}`$ and define $`X_j(I)`$ to be the matrix obtained from $`X`$ by replacing its $`i`$-th row by the vector $`(\eta _j^1,\mathrm{},\eta _j^p)`$, for all $`iI`$. If $`I=\{i_1,\mathrm{},i_l\}`$, we write $`X_j(I)=X_j(i_1,\mathrm{},i_l)`$ as well.

###### Lemma 4.1.

We have

(4.27)
$$\mathrm{det}X_1(1)\mathrm{det}X_1(2)\mathrm{}\mathrm{det}X_1(p)=\frac{1}{p!}(\mathrm{det}X)^{p1}\mathrm{det}X_1(1,\mathrm{},p).$$

###### Proof.

Denoting by $`R(X)`$ and $`L(X)`$ the right-hand side and the left-hand side of (4.27), respectively, we may regard $`R`$ and $`L`$ as functions of $`X`$. Since the group $`GL(p)`$ acts on $`X`$, the space of $`p\times p`$ matrices, by left multiplication, it acts on functions of $`X`$. To be more precise if $`AGL(p)`$, then $`(AL)(X):=L(A^1X)`$ and $`(AR)(X):=R(A^1X)`$. We want to study the effect of this action on $`R`$ and $`L`$. In order to do so, consider first the action of the three kinds of elementary matrices on them. Namely, those that interchanges any two rows, that multiplies a row by a scalar, and those that add a scalar multiple of a row to another. It is subject to a direct verification that if $`A`$ is any of the three types of elementary matrices, we have

(4.28)
$$(AR)(X)=(\mathrm{det}A)^{1p}R(X),(AL)(X)=(\mathrm{det}A)^{1p}L(X).$$

Since every element in $`GL(p)`$ is a product of elementary matrices, we conclude that (4.28) holds for every $`AGL(p)`$ as well. Putting $`X=1_p`$, the identity $`p\times p`$ matrix, we see that $`R(1_p)=L(1_p)`$ so that $`R(A)=L(A)`$ by (4.28) for all $`AGL(p)`$. As $`GL(p)`$ is a Zariski open set in the space of $`p\times p`$ matrices we have $`R(X)=L(X)`$ for any $`p\times p`$ matrix $`X`$. ∎

###### Remark 4.3.

An alternative proof of the above lemma can be given as follows. Let $`X_{ij}`$ denote the $`(i,j)`$-th minor of $`X`$. It is known that $`\mathrm{det}(X_{ij})=\mathrm{det}X^{p1}`$ which follows directly from a form of the Cramer’s formula $`X(X_{ij})=(\mathrm{det}X)1_p`$. The above lemma follows from this identity by expanding each determinant in the left-hand side of (4.27) by the row $`(\eta _1^1,\mathrm{},\eta _1^p)`$ and noting that $`\mathrm{det}X_1(1,\mathrm{},p)`$ is equal to $`p!\eta _1^1\eta _1^2\mathrm{}\eta _1^p`$.

###### Corollary 4.2.

Let $`I=\{i_1,\mathrm{},i_{|I|}\}`$ and $`J=\{j_1,\mathrm{},j_{|J|}\}`$ be two subsets of $`\{1,\mathrm{},p\}`$ arranged in increasing order. Then

* $`\mathrm{det}X_1(I)\mathrm{det}X_1(J)=0`$ if and only if $`IJ\mathrm{}`$.
* $`\mathrm{det}X_1(i_1)\mathrm{det}X_1(i_2)\mathrm{}\mathrm{det}X_1(i_{|I|})=\frac{1}{|I|!}(\mathrm{det}X)^{|I|1}\mathrm{det}X_1(I).`$
* For $`IJ=\mathrm{}`$ we have

$$\mathrm{det}X_1(I)\mathrm{det}X_1(J)=ϵ_{IJ}\frac{|I|!|J|!}{(|I|+|J|)!}\mathrm{det}X\mathrm{det}X_1(IJ),$$

where $`ϵ_{IJ}`$ is the sign of the permutation that arranges the ordered tuple $`(i_1,\mathrm{},i_{|I|},j_1,\mathrm{},j_{|J|})`$ in increasing order.

###### Proof.

(i) is an obvious consequence of Lemma 4.1.

For (ii) let $`I^c=\{k_1,\mathrm{},k_{|I^c|}\}`$ denote the complementary subset of $`I`$ in $`\{1,\mathrm{},p\}`$ put in increasing order. We apply successively the differential operators $`_{j=1}^px_{k_1}^j\frac{}{\eta _1^j}`$, $`_{j=1}^px_{k_2}^j\frac{}{\eta _1^j}`$, $`\mathrm{}`$, $`_{j=1}^px_{k_{|I^c|}}^j\frac{}{\eta _1^j}`$ to (4.27) and find that

$$(\mathrm{det}X)^{p|I|}\mathrm{det}X_1(i_1)\mathrm{}\mathrm{det}X_1(i_{|I|})=\frac{1}{|I|!}(\mathrm{det}X)^{p1}\mathrm{det}X_1(I).$$

Dividing by $`(\mathrm{det}X)^{p|I|}`$ we obtain (ii).

By (ii) we have

$`{\displaystyle \frac{1}{|I|!}}(\mathrm{det}X)^{|I|1}{\displaystyle \frac{1}{|J|!}}(\mathrm{det}`$ $`X)^{|J|1}\mathrm{det}X_1(I)\mathrm{det}X_1(J)=`$
$`\mathrm{det}X_1(i_1)\mathrm{}\mathrm{det}X_1(i_{|I|})\mathrm{det}X_1(j_1)\mathrm{}\mathrm{det}X_1(j_{|J|}).`$
Thus if $`ϵ_{IJ}`$ is the permutation arranging $`IJ`$ in increasing order, then

$$\frac{1}{|I|!|J|!}(\mathrm{det}X)^{|I|+|J|2}\mathrm{det}X_1(I)\mathrm{det}X_1(J)=\frac{ϵ_{IJ}}{(|I|+|J|)!}(\mathrm{det}X)^{|I|+|J|1}\mathrm{det}X_1(IJ).$$

Now (iii) follows from dividing the above equation by $`(\mathrm{det}X)^{|I|+|J|2}`$ and multiplying by $`|I|!|J|!`$. ∎

Returning to the problem of finding the highest weight vector associated to the Young diagram (4.25), our first task is to present a more explicit expression for a product of the form $`\mathrm{\Gamma }_{\lambda _{p+1}}\mathrm{}\mathrm{\Gamma }_{\lambda _{p+s}}`$. We associate to such a Young diagram a collection $`D`$ of $`s`$ marked diagrams $`D_i`$, $`i=1,\mathrm{},s`$, with $`D_i`$ having $`\lambda _{p+i}`$ columns and $`m`$ rows. We arrange these $`D_i`$ in the form:

(4.29) ......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................$`m\{\text{ }`$$`m\{\text{ }`$$`m\{\text{ }`$$`\underset{\lambda _{p+1}}{\underset{}{\text{ }}}`$$`\underset{\lambda _{p+2}}{\underset{}{\text{ }}}`$$`\underset{\lambda _{p+s}}{\underset{}{\text{ }}}`$$`D_1:`$$`D_2:`$$`D_s:`$
Marked boxes are put into $`D`$ subject to the following constraint: in each $`D_i`$ a column has at most one marked box. If a diagram $`D_i`$ contains a marked box in its $`k`$-th row and $`s`$-th column, then no other $`D_j`$ $`(ji)`$ contains a marked box in its $`k`$-th row and $`s`$-th column. From now on $`D`$ will denote such a collection of marked diagrams.

Now suppose that $`D`$ is a collection of diagrams $`D_i`$, $`i=1,\mathrm{},s`$. To each $`D_i`$ we may associate a $`\lambda _{p+i}\times \lambda _{p+i}`$ matrix $`Y_{D_i}`$ as in the previous section. We let $`\mathrm{det}Y_D:=\mathrm{det}Y_{D_1}\mathrm{det}Y_{D_2}\mathrm{}\mathrm{det}Y_{D_s}`$. Now to each column $`j`$ of $`D`$ $`(1j\lambda _{p+1})`$, we may associate a $`p\times p`$ matrix $`X_j(I_j)`$ obtained from the matrix $`X`$ as follows. Let $`I_j`$ be the subset of $`\{1,\mathrm{},p\}`$ consisting of the numbers of the marked rows on the column $`j`$. We define $`X_j(I_j)`$ to be the matrix obtained from $`X`$ by replacing the rows of $`X`$ corresponding to $`I_j`$ by the vector $`(\eta _j^1,\mathrm{},\eta _j^p)`$. We then define $`X_D:=X_1(I_1)X_2(I_2)\mathrm{}X_{\lambda _{p+1}}(I_{\lambda _{p+1}})`$.

Suppose we have a marked box in $`D_i`$ appearing in its $`k`$-th row and $`s`$-th column. We associate an odd indeterminate $`a_i^{ks}`$. Consider the product of all $`a_i^{ks}`$ arranged in increasing order following the lexicographical ordering of $`(i,s,k)`$. Now we may also consider the product arranged in increasing order following the lexicographical ordering $`(s,k,i)`$. These two products differ by a sign, and this sign is denoted by $`ϵ_D`$. Furthermore we let $`d_i=|D_i|`$, the number of marked boxes in $`D_i`$, and $`e_j`$ be the number of marked boxes in the $`j`$-th column of $`D`$.

###### Proposition 4.3.

With notation as above we have

$$\frac{\mathrm{\Gamma }_{\lambda _{p+1}}\mathrm{}\mathrm{\Gamma }_{\lambda _{p+s}}}{(\mathrm{det}X)^{\lambda _{p+2}+\mathrm{}+\lambda _{p+s}}}=\underset{D}{}(1)^{\frac{1}{2}((_{i,j}d_id_j)|D|)}\frac{ϵ_D}{e_1!\mathrm{}e_{\lambda _{p+1}}!}\mathrm{det}X_D\mathrm{det}Y_D.$$

###### Proof.

Given diagram $`D_i`$ with $`m`$ rows and $`\lambda _{p+i}`$ columns, for $`i=1,\mathrm{},s`$, we want to know how to simplify the expression

$$(1)^{\frac{1}{2}_{i=1}^sd_i(d_i1)}\mathrm{det}X_{D_1}\mathrm{det}Y_{D_1}\mathrm{}\mathrm{det}X_{D_s}\mathrm{det}Y_{D_s}.$$

We move all $`\mathrm{det}X_{D_i}`$ to the left and get

(4.30)
$$(1)^{\frac{1}{2}(_{i,j=1}^sd_id_j)\frac{1}{2}|D|}\mathrm{det}X_{D_1}\mathrm{}\mathrm{det}X_{D_s}\mathrm{det}Y_{D_1}\mathrm{}\mathrm{det}Y_{D_s}.$$

Now each $`\mathrm{det}X_{D_i}`$ is a product of $`\mathrm{det}X_b(a)`$. We arrange all $`X_1(a)`$ together so that $`X_1(a_1)`$ appears to the left of $`X_1(a_2)`$ if and only if $`a_1<a_2`$ and call the resulting expression $`X_D^{\eta _1}`$ and move it to the left. We do the same thing to $`X_2(a)`$ and move $`X_D^{\eta _2}`$ to the right of $`X_D^{\eta _1}`$ etc. Then (4.30) becomes

(4.31)
$$(1)^{\frac{1}{2}(_{i,j=1}^sd_id_j)\frac{1}{2}|D|}ϵ_DX_D^{\eta _1}X_D^{\eta _2}\mathrm{}X_D^{\eta _{\lambda _{p+1}}}\mathrm{det}Y_{D_1}\mathrm{}\mathrm{det}Y_{D_s}.$$

We apply now Corollary 4.2 to (4.31) and obtain

$$(1)^{\frac{1}{2}(_{i,j=1}^sd_id_j)\frac{1}{2}|D|}\frac{ϵ_D}{e_1!\mathrm{}e_{\lambda _{p+1}}!}(\mathrm{det}X)^{(\lambda _{p+1}^{}p)+\mathrm{}+(\lambda _{\lambda _{p+1}}^{}p)\lambda _{p+1}}\mathrm{det}X_D\mathrm{det}Y_D.$$

Since $`_{i=1}^s\lambda _{p+i}=_{i=1}^{\lambda _{p+1}}(\lambda _i^{}p)`$, the proposition follows. ∎

It follows immediately from Corollary 4.1 that

###### Proposition 4.4.

The vector

$$Z:=\mathrm{det}X_1(m+1,\mathrm{},p)\mathrm{det}X_2(m+1,\mathrm{},p)\mathrm{}\mathrm{det}X_{\lambda _{p+1}}(m+1,\mathrm{},p)$$

is annihilated by (4).

###### Proposition 4.5.

The expression

$$\underset{D}{}(1)^{\frac{1}{2}((_{i,j}d_id_j)|D|)}\frac{ϵ_D}{e_1!\mathrm{}e_{\lambda _{p+1}}!}\mathrm{det}X_D\mathrm{det}Y_D\mathrm{det}Z$$

is divisible by $`(\mathrm{det}X)^{\lambda _{p+1}}`$. Furthermore the resulting expression is independent of the variables $`x_l^i`$ $`(l=m+1,\mathrm{},p)`$ and is annihilated by (4).

###### Proof.

Since for a subset $`I`$ of $`\{1,\mathrm{},m\}`$, $`\mathrm{det}X_i(I)\mathrm{det}X_i(m+1,\mathrm{},p)`$ is a scalar multiple of $`\mathrm{det}X\mathrm{det}X_i(I\{m+1,\mathrm{},p\})`$ by Corollary 4.2 (iii), it follows that the expression is divisible by $`(\mathrm{det}X)^{\lambda _{p+1}}`$ and independent of $`x_l^i`$, for $`l=m+1,\mathrm{},p`$, after division. It is clear that it is annihilated by (4). ∎

The vector

$$\underset{D}{}(1)^{\frac{1}{2}((_{i,j}d_id_j)|D|)}(e_1!\mathrm{}e_{\lambda _{p+1}}!)^1ϵ_D(\mathrm{det}X)^{\lambda _{p+1}}\mathrm{det}Z\mathrm{det}X_D\mathrm{det}Y_D$$

depends only on the $`s`$-tuple $`(\lambda _{p+1},\mathrm{},\lambda _{p+s})`$ and thus we will denote this vector by $`\mathrm{\Gamma }(\lambda _{p+1},\mathrm{},\lambda _{p+s})`$.

###### Proposition 4.6.

The vector $`\mathrm{\Gamma }(\lambda _{p+1},\mathrm{},\lambda _{p+s})`$ has weight corresponding to the Young diagram (4.25).

###### Proof.

Let $`f_j`$, $`j=1,\mathrm{},m`$ denote the number of marked boxes in $`D`$ that appear in the $`j`$-th row of some diagram $`D_i`$. Then $`\mathrm{det}Y_D`$ has weight

$$\underset{j=1}{\overset{\lambda _{p+1}}{}}(\lambda _j^{}p)\delta _j+\underset{j=1}{\overset{m}{}}f_jϵ_j\underset{j=1}{\overset{\lambda _{p+1}}{}}e_j\delta _j+\underset{j=1}{\overset{\lambda _{p+1}}{}}(\lambda _j^{}p)\stackrel{~}{\delta }_j,$$

while the expression $`(\mathrm{det}X)^{\lambda _{p+1}}\mathrm{det}X_D\mathrm{det}Z`$ has weight

$$\lambda _{p+1}\underset{j=1}{\overset{m}{}}ϵ_j+(pm)\underset{j=1}{\overset{\lambda _{p+1}}{}}\delta _j+\underset{j=1}{\overset{\lambda _{p+1}}{}}e_j\delta _j\underset{j=1}{\overset{m}{}}f_jϵ_j+\lambda _{p+1}\underset{j=1}{\overset{p}{}}\stackrel{~}{ϵ_j}.$$

So the combined weight is

$$\lambda _{p+1}\underset{j=1}{\overset{m}{}}ϵ_j+\lambda _{p+1}\underset{j=1}{\overset{p}{}}\stackrel{~}{ϵ_j}+\underset{j=1}{\overset{\lambda _{p+1}}{}}(\lambda _j^{}m)\delta _j+\underset{j=1}{\overset{\lambda _{p+1}}{}}(\lambda _j^{}p)\stackrel{~}{\delta }_j,$$

which of course is the weight of the Young diagram (4.25). ∎

Combining our results in this section we have proved

###### Theorem 4.5.

In the case when $`pm`$ an irreducible $`gl(p|q)\times gl(m|n)`$ module $`V_{p|q}^\lambda V_{m|n}^\lambda `$ appears in $`[𝐱,\xi ,\eta ,𝐲]`$ if and only if $`\lambda _{m+1}n`$ and $`\lambda _{p+1}q`$. The following are highest weight vectors corresponding to such a Young diagram $`\lambda `$ ($`t`$ is the length of $`\lambda ^{}`$):

* In the case when $`\lambda _{m+1}=0`$ it is given by

$$\underset{i=1}{\overset{t}{}}\mathrm{\Delta }_{\lambda _i^{}}.$$
* In the case when $`\lambda _{m+1}>0`$ and $`\lambda _{p+1}=0`$ it is given by

$$\underset{i=1}{\overset{r}{}}\mathrm{\Delta }_{i,\lambda _i^{}}\underset{i=r+1}{\overset{t}{}}\mathrm{\Delta }_{\lambda _i^{}},$$

where $`0rn`$ is defined by $`\lambda _r^{}>m`$ and $`\lambda _{r+1}^{}m`$.
* In the case when $`\lambda _{p+1}>0`$ it is given by

$$\mathrm{\Gamma }(\lambda _{p+1},\mathrm{},\lambda _{p+s})\underset{i=r+1}{\overset{r^{}}{}}\mathrm{\Delta }_{i,\lambda _i^{}}\underset{i=r^{}+1}{\overset{t}{}}\mathrm{\Delta }_{\lambda _i^{}},$$

where $`rr^{}`$ are defined as in (4.23) and $`p+s`$ is the length of $`\lambda `$.

###### Proof.

The only thing that remains to prove is that the vector in (iii) is indeed killed by (4). But this is because of the presence of $`Z`$ in the formula of $`\mathrm{\Gamma }(\lambda _{p+1},\mathrm{},\lambda _{p+s})`$ and so is an immediate consequence of Corollary 4.1. ∎

## 5. Construction of highest weight vectors in $`S(S^2^{m|n})`$

In this section we will give an explicit formula for a highest weight vector of each irreducible $`gl(m|n)`$-module that appear in the symmetric algebra of the symmetric square of the natural $`gl(m|n)`$-module. According to Theorem 3.4 we have the following decomposition of $`S(S^2^{m|n})`$ as a $`gl(m|n)`$-module:

$$S(S^2^{m|n})\underset{\lambda }{}V_{m|n}^\lambda ,$$

where the summation is over all partitions $`\lambda `$ with even rows and $`\lambda _{m+1}n`$.

We let $`\{x_1,\mathrm{},x_m;\xi _1,\mathrm{},\xi _n\}`$ be the standard basis for $`^{m|n}`$, with $`x_i`$ denoting even, and $`\xi _j`$ odd vectors. Regarding $`x_i`$ as even and $`\xi _j`$ as odd variables the Lie supealgebra $`gl(m|n)`$ has a natural identification with the space of first order differential operators over $`[x_i,\xi _j]`$. The Cartan subalgebra of diagonal matrices is then spanned by its standard basis $`x_i\frac{}{x_i}`$ and $`\xi _j\frac{}{\xi _j}`$, for $`i=1,\mathrm{},m`$ and $`j=1,\mathrm{},n`$. The nilpotent radical is generated by the simple root vectors

(5.1)
$$x_i\frac{}{x_{i+1}},\xi _j\frac{}{\xi _{j+1}},x_m\frac{}{\xi _1},i=1,\mathrm{},m1;j=1,\mathrm{},n1.$$

$`S^2^{m|n}`$ then is spanned by the vectors $`x_{ij}=x_{ji}=x_ix_j`$, $`y_{kl}=y_{lk}=\xi _k\xi _l`$ and $`\eta _{ki}=\xi _kx_i`$, where $`1i,jm`$ and $`1k,ln`$. This allows us to identify $`S(S^2^{m|n})`$ with the polynomial algebra over $``$ in the even variables $`x_{ij}`$ and $`y_{kl}`$ and odd variables $`\eta _{ki}`$, with $`1ijm`$ and $`1k<ln`$, which we denote by $`[x,y,\eta ]`$.

A convention of notation we will use throughout this section is the following: By $`x_i(x_1,x_2,\mathrm{},x_m)`$ we will mean the row vector $`(x_{i1},x_{i2},\mathrm{},x_{im})`$. So by the expression

$$\left(\begin{array}{c}x_1(x_1,x_2,\mathrm{},x_m)\\ x_2(x_1,x_2,\mathrm{},x_m)\\ \mathrm{}\\ x_m(x_1,x_2,\mathrm{},x_m)\end{array}\right)$$

we mean the matrix whose $`i`$-th row entries equals to $`(x_{i1},x_{i2},\mathrm{},x_{im})`$, i.e. the matrix

$$X:=\left(\begin{array}{ccc}x_{11}& x_{12}& \mathrm{}x_{1m}\\ x_{21}& x_{22}& \mathrm{}x_{2m}\\ \mathrm{}& \mathrm{}& \mathrm{}\mathrm{}\\ x_{m1}& x_{m2}& \mathrm{}x_{mm}\end{array}\right).$$

Similarly by an expression of the form

$$X_i(\xi _j):=\left(\begin{array}{c}x_1(x_1,x_2,\mathrm{},x_m)\\ \mathrm{}\\ x_{i1}(x_1,x_2,\mathrm{},x_m)\\ \xi _j(x_1,x_2,\mathrm{},x_m)\\ x_{i+1}(x_1,x_2,\mathrm{},x_m)\\ \mathrm{}\\ x_m(x_1,x_2,\mathrm{},x_m)\end{array}\right)$$

we mean to replace the $`i`$-th row of the matrix $`X`$ by the vector $`(\eta _{j1},\eta _{j2},\mathrm{},\eta _{jm})`$. In these forms the action of (5.1) will be more transparent.

Consider the first $`r\times r`$ minor $`\mathrm{\Delta }_r`$ of the $`m\times m`$ matrix $`X`$, for $`1rm`$. It is easily seen to be a highest weight vector in $`[x,y,\eta ]`$ of highest weight $`2(_{i=1}^rϵ_i)`$, where as before we use $`ϵ_i`$ and $`\delta _k`$ to denote the fundamental weights of $`gl(m|n)`$. Hence if $`\lambda `$ is a Young diagram with even rows of length not exceeding $`m`$, then its corresponding highest weight vector is a product of $`\mathrm{\Delta }_r`$’s. To be explicit note that since $`\lambda `$ has even rows, $`\lambda _1=2t`$ is an even number. Furthermore we also have $`\lambda _{2i1}^{}=\lambda _{2i}^{}`$, for all $`i=1,\mathrm{},t`$. Then the highest weight vector is given by $`_{i=1}^t\mathrm{\Delta }_{\lambda _{2i}^{}}`$.

Consider now a diagram of the form

(5.2) .....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................$`m\{\text{ }`$$`1\{\text{ }`$$`\underset{2l}{\underset{}{\text{ }}}`$
The product of highest weight vectors of two such diagrams, if non-zero, gives a highest weight vector of a diagram of the form

............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................$`m\{\text{ }`$$`2\{\text{ }`$$`\underset{2l_1}{\underset{}{\text{ }}}`$$`\underset{2l_2}{\underset{}{\text{ }}}`$$`\underset{2l_2}{\underset{}{\text{ }}}`$
Dividing by the determinant $`\mathrm{det}X^{l_2}`$ we obtain a highest weight vector for the diagram

.......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................$`m\{\text{ }`$$`2\{\text{ }`$$`\underset{2l_1}{\underset{}{\text{ }}}`$$`\underset{2l_2}{\underset{}{\text{ }}}`$
Thus it is enough to construct vectors associated to the Young diagrams of the form (5.2). To do so we first consider the case when $`l=1`$ in (5.2).

Consider the expression

(5.3) $`\mathrm{\Delta }(\xi _1,\xi _2):=`$ $`(\mathrm{det}X)(\xi _1\xi _2)+(\mathrm{det}X_1(\xi _1))(\xi _2x_1)`$
$`+(\mathrm{det}X_2(\xi _1))(\xi _2x_2)+\mathrm{}+(\mathrm{det}X_m(\xi _1))(\xi _2x_m),`$
where by $`(\xi _1\xi _2)`$ and $`(\xi _2x_i)`$ we mean $`y_{12}`$ and $`\eta _{2i}`$, respectively. The following lemma will be useful later on.

###### Lemma 5.1.

Let $`A=(a_{ij})`$ be a complex symmetric $`m\times m`$ matrix and $`\theta _1,\theta _2,\mathrm{},\theta _m`$ be odd variables. Then

$$\mathrm{det}\left(\begin{array}{cc}0& \theta _1\mathrm{}\theta _m\\ \theta _1& \\ \mathrm{}& A\\ \theta _m& \end{array}\right)=0.$$

###### Proof.

It is enough to restrict ourselves to real symmetric $`m\times m`$ matrices $`A`$. Let $`U`$ be an orthogonal $`m\times m`$ matrix such that $`U^tAU=D`$, where $`D`$ is a diagonal matrix. We compute

(5.4)
$$\left(\begin{array}{cc}1& 0\mathrm{}0\\ 0& \\ \mathrm{}& U^t\\ 0& \end{array}\right)\left(\begin{array}{cc}0& \theta _1\mathrm{}\theta _m\\ \theta _1& \\ \mathrm{}& A\\ \theta _m& \end{array}\right)\left(\begin{array}{cc}1& 0\mathrm{}0\\ 0& \\ \mathrm{}& U\\ 0& \end{array}\right)=\left(\begin{array}{cc}0& \zeta _1\mathrm{}\zeta _m\\ \zeta _1& \\ \mathrm{}& D\\ \zeta _m& \end{array}\right),$$

where $`\zeta _k=_{j=1}^mu_{jk}\theta _j`$ and $`U=(u_{ij})`$. But the determinant of the matrix on the right-hand side of (5.4) is zero. ∎

The next lemma is straightforward.

###### Lemma 5.2.

$`\mathrm{\Delta }(\xi _1,\xi _2)`$ has weight $`2(_{i=1}^mϵ_i)+\delta _1+\delta _2`$ and hence its weight corresponds to the weight of the Young diagram (5.2) with $`l=1`$.

###### Lemma 5.3.

$`\mathrm{\Delta }(\xi _1,\xi _2)`$ is annihilated by all operators in (5.1) and hence is a highest weight vector in $`S(S^2^{m|n})`$ corresponding to the Young diagram (5.2) with $`l=1`$.

###### Proof.

First consider the action of the operator $`x_{i1}\frac{}{x_i}`$, for $`i=2,\mathrm{},m`$, on $`\mathrm{\Delta }(\xi _1,\xi _2)`$ given as in (5.3). Certainly $`x_{i1}\frac{}{x_i}`$ annihilates the first summand of (5.3), and furthermore it takes the summand $`X_j(\xi _1)(\xi _2x_j)`$ for $`ji1,i`$ to

$$\mathrm{det}\left(\begin{array}{c}x_1(x_1,\mathrm{},x_{i1},x_{i1},\mathrm{},x_m)\\ \mathrm{}\\ x_{j1}(x_1,\mathrm{},x_{i1},x_{i1},\mathrm{},x_m)\\ \xi _1(x_1,\mathrm{},x_{i1},x_{i1},\mathrm{},x_m)\\ x_{j+1}(x_1,\mathrm{},x_{i1},x_{i1},\mathrm{},x_m)\\ \mathrm{}\\ x_m(x_1,\mathrm{},x_{i1},x_{i1},\mathrm{},x_m)\end{array}\right)(\xi _2x_j)+\mathrm{det}\left(\begin{array}{c}x_1(x_1,\mathrm{},x_m)\\ \mathrm{}\\ x_{j1}(x_1,\mathrm{},x_m)\\ \xi _1(x_1,\mathrm{},x_m)\\ x_{j+1}(x_1,\mathrm{},x_m)\\ \mathrm{}\\ x_{i1}(x_1,\mathrm{},x_m)\\ x_{i1}(x_1,\mathrm{},x_m)\\ \mathrm{}\\ x_m(x_1,\mathrm{},x_m)\end{array}\right)(\xi _2x_j),$$

which is zero. $`x_{i1}\frac{}{x_i}`$ takes $`X_i(\xi _1)(\xi _2x_i)`$ to

$$\mathrm{det}\left(\begin{array}{c}x_1(x_1,\mathrm{},x_{i1},x_{i1},\mathrm{},x_m)\\ \mathrm{}\\ x_{ii}(x_1,\mathrm{},x_{i1},x_{i1},\mathrm{},x_m)\\ \xi _1(x_1,\mathrm{},x_{i1},x_{i1},\mathrm{},x_m)\\ x_{i+1}(x_1,\mathrm{},x_{i1},x_{i1},\mathrm{},x_m)\\ \mathrm{}\\ x_m(x_1,\mathrm{},x_{i1},x_{i1},\mathrm{},x_m)\end{array}\right)(\xi _2x_i)+\mathrm{det}\left(\begin{array}{c}x_1(x_1,\mathrm{},x_m)\\ \mathrm{}\\ x_{i1}(x_1,\mathrm{},x_m)\\ \xi _1(x_1,\mathrm{},x_m)\\ x_{i+1}(x_1,\mathrm{},x_m)\\ \mathrm{}\\ x_m(x_1,\mathrm{},x_m)\end{array}\right)(\xi _2x_{i1}).$$

The first summand is zero, while the second summand remains. Now we verify similarly that $`x_{i1}\frac{}{x_i}`$ takes $`X_{i1}(\xi _1)(\xi _2x_{i1})`$ to the identical expression as the second summand above with the difference that the $`i1`$-st and $`i`$-th rows are interchanged. Thus $`x_{i1}\frac{}{x_i}(\mathrm{\Delta }(\xi _1,\xi _2))=0`$.

Consider now the action of $`x_m\frac{}{\xi _1}`$ on $`\mathrm{\Delta }(\xi _1,\xi _2)`$. Note that $`x_m\frac{}{\xi _1}`$ kills every term in (5.3) except for the first and the last. The contribution from the first summand is $`\mathrm{det}X(\xi _2x_m)`$, while that from the last summand is $`\mathrm{det}X(\xi _2x_m)`$, and hence $`x_m\frac{}{\xi _1}(\mathrm{\Delta }(\xi _1,\xi _2))=0`$.

Finally we consider the action of $`\xi _1\frac{}{\xi _2}`$ on $`(\mathrm{\Delta }(\xi _1,\xi _2))`$ as in (5.3). $`\xi _1\frac{}{\xi _2}`$ kills the first term in (5.3) and the resulting vector is

(5.5)
$$\underset{i=1}{\overset{m}{}}(\mathrm{det}X_i(\xi _1))(\xi _1x_i),$$

which can be in a consistent form with our earlier notation written as $`\mathrm{\Delta }(\xi _1,\xi _1)`$. Expanding along the first row we see that (5.5) is the same as

$$\mathrm{det}\left(\begin{array}{cc}0& (\xi _1x_1)\mathrm{}(\xi _1x_m)\\ (\xi _1x_1)& \\ \mathrm{}& X\\ (\xi _1x_m)& \end{array}\right),$$

which is zero by Lemma 5.1. ∎

The proof of the above theorem gives us certain identities that will be used later on. We will collect them here for the convenience of the reader:

(5.6) $`x_{i1}{\displaystyle \frac{}{x_i}}(\mathrm{\Delta }(\xi _k,\xi _l))`$ $`=0,`$
(5.7) $`\xi _{j1}{\displaystyle \frac{}{\xi _j}}(\mathrm{\Delta }(\xi _j,\xi _l))`$ $`=\mathrm{\Delta }(\xi _{j1},\xi _l),`$
(5.8) $`\xi _{j1}{\displaystyle \frac{}{\xi _j}}(\mathrm{\Delta }(\xi _s,\xi _j))`$ $`=\mathrm{\Delta }(\xi _s,\xi _{j1}),`$
(5.9) $`\mathrm{\Delta }(\xi _j,\xi _j)`$ $`=0.`$
We now turn our attention to the general case of a Young diagram of the form (5.2) with general $`l`$. Of course we have the restriction that $`2ln`$.

Let $`\sigma =\{(i_1,i_2),(i_3,i_4),\mathrm{},(i_{2l1},i_{2l})\}`$ be a partition of the set $`\{1,2,\mathrm{},2l\}`$. Assuming that we have arranged $`\sigma `$ in the form so that $`i_1<i_2,i_3<i_4,\mathrm{},i_{2l1}<i_{2l}`$, we may define $`ϵ_\sigma `$ to be the sign of the permutation taking $`k`$ to $`i_k`$ for all $`1k2l`$. We may associate to $`\sigma `$ a vector $`\mathrm{\Delta }(\xi _{i_1},\xi _{i_2})\mathrm{}\mathrm{\Delta }(\xi _{i_{2l1}},\xi _{i_{2l}})`$ in $`S(S^2^{m|n})`$ and define

$$\mathrm{\Gamma }(2l)=\underset{\sigma }{}ϵ_\sigma \mathrm{\Delta }(\xi _{i_1},\xi _{i_2})\mathrm{}\mathrm{\Delta }(\xi _{i_{2l1}},\xi _{i_{2l}}),$$

where the sum is taken over all partition $`\sigma =\{(i_1,i_2),(i_3,i_4),\mathrm{},(i_{2l1},i_{2l})\}`$ of the set $`\{1,2,\mathrm{},2l\}`$ arranged in the form that $`i_1<i_2,i_3<i_4,\mathrm{},i_{2l1}<i_{2l}`$. The following lemma is again a straightforward computation.

###### Lemma 5.4.

The weight of $`\mathrm{\Gamma }(2l)`$ is $`2l(_{i=1}^mϵ_i)+_{j=1}^{2l}\delta _j`$ and hence corresponds to the weight of the Young diagram (5.2).

###### Lemma 5.5.

$`\mathrm{\Gamma }(2l)`$ is annihilated by (5.1) and hence a highest weight vector in $`S(S^2^{m|n})`$.

###### Proof.

The fact that $`\mathrm{\Gamma }(2l)`$ is annihilated by $`x_{i1}\frac{}{x_i}`$ for $`i=2,\mathrm{},m`$ is a consequence of (5.6). Now the proof of Lemma 5.3 shows that $`x_m\frac{}{\xi _1}(\mathrm{\Delta }(\xi _1,\xi _s))=0`$, for every $`s=1,\mathrm{},n`$. Thus $`x_m\frac{}{\xi _1}`$ annihilates $`\mathrm{\Gamma }(2l)`$ as well. So it remains to show that $`\xi _{j1}\frac{}{\xi _j}`$ kills $`\mathrm{\Gamma }(2l)`$.

Given a summand in $`\mathrm{\Gamma }(2l)`$ of the form $`ϵ_\sigma \mathrm{}\mathrm{\Delta }(\xi _{j1},\xi _l)\mathrm{}\mathrm{\Delta }(\xi _j,\xi _i)\mathrm{}`$ there exists a summand of the form $`ϵ_\sigma ^{}\mathrm{}\mathrm{\Delta }(\xi _{j1},\xi _i)\mathrm{}\mathrm{\Delta }(\xi _j,\xi _l)\mathrm{}`$, which is identical to it except at these two places. Now $`\xi _{j1}\frac{}{\xi _j}`$ takes the first of the two summands above to

$$ϵ_\sigma \mathrm{}\mathrm{\Delta }(\xi _{j1},\xi _l)\mathrm{}\mathrm{\Delta }(\xi _{j1},\xi _i)\mathrm{},$$

and the second summand to

$$ϵ_\sigma ^{}\mathrm{}\mathrm{\Delta }(\xi _{j1},\xi _i)\mathrm{}\mathrm{\Delta }(\xi _{j1},\xi _l)\mathrm{}.$$

But $`\sigma `$ and $`\sigma ^{}`$ differ by a transposition $`(i,l)`$ and hence $`ϵ_\sigma =ϵ_\sigma ^{}`$ and so these two terms cancel.

Consider a summand in $`\mathrm{\Gamma }(2l)`$ of the form $`ϵ_\sigma \mathrm{}\mathrm{\Delta }(\xi _l,\xi _{j1})\mathrm{}\mathrm{\Delta }(\xi _j,\xi _i)\mathrm{}`$. But in $`\mathrm{\Gamma }(2l)`$ we also have a summand of the form $`ϵ_\sigma ^{}\mathrm{}\mathrm{\Delta }(\xi _l,\xi _j)\mathrm{}\mathrm{\Delta }(\xi _{j1},\xi _i)\mathrm{}`$. Applying $`\xi _{j1}\frac{}{\xi _j}`$ to these two terms, we again see that they cancel by the same reasoning as before.

Now we look at a term of the form $`ϵ_\sigma \mathrm{}\mathrm{\Delta }(\xi _j,\xi _l)\mathrm{}\mathrm{\Delta }(\xi _i,\xi _{j1})\mathrm{}`$. We also have a term of the form $`ϵ_\sigma ^{}\mathrm{}\mathrm{\Delta }(\xi _{j1},\xi _l)\mathrm{}\mathrm{\Delta }(\xi _i,\xi _j)\mathrm{}`$. Again they will cancel each other after applying $`\xi _{j1}\frac{}{\xi _j}`$.

Finally a term of the form $`ϵ_\sigma ^{}\mathrm{}\mathrm{\Delta }(\xi _{j1},\xi _j)\mathrm{}`$ is killed by $`\xi _{j1}\frac{}{\xi _j}`$ by (5.9). This completes the proof. ∎

It is clear that a product of $`\mathrm{\Gamma }(2l)`$’s (not necessarily the same $`l`$) is non-zero, which therefore allows us to construct all other highest weight vectors, as discussed in the beginning of this section. Below we summarize the results of this section.

###### Theorem 5.1.

The $`gl(m|n)`$-highest weight vectors of $`S(S^2^{m|n})`$ form an abelian semigroup generated by $`\mathrm{\Gamma }(2),\mathrm{},\mathrm{\Gamma }(2[\frac{n}{2}])`$ and $`\mathrm{\Delta }_1,\mathrm{},\mathrm{\Delta }_m`$, where $`[\frac{n}{2}]`$ denotes the largest integer not exceeding $`\frac{n}{2}`$. Furthermore this semigroup is free if and only if $`n=0,1`$. More precisely a highest weight vector associated to an even partition $`\lambda =(\lambda _1,\mathrm{},\lambda _l)`$ with $`\lambda _{m+1}n`$ is given by

$$(\mathrm{det}X)^{_{i=m+2}^l\lambda _i}\underset{m+1}{\overset{l}{}}\mathrm{\Gamma }(\lambda _i)\underset{j=r+1}{\overset{l}{}}(\mathrm{\Delta }_{\lambda _j^{}})^{\frac{1}{2}},$$

where the non-negative integer $`r`$ is defined by $`\lambda _r^{}>m`$ and $`\lambda _{r+1}^{}m`$.

###### Remark 5.1.

From Theorem 5.1 we may recover the highest weight vectors in $`S(S^2^m)`$ and $`S(\mathrm{\Lambda }^2^n)`$ by putting $`n=0`$ and $`m=0`$, respectively. Namely, identifying $`S^2^m`$ (respectively $`\mathrm{\Lambda }^2^n`$) with the space of symmetric $`m\times m`$ (respectively skew-symmetric $`n\times n`$) matrices, we see that in the first case the highest weight vectors are generated by the leading minors of the determinant of the typical element of $`S^2^m`$, while in the second case they are generated by the Pfaffians of the leading $`2l\times 2l`$ minors of the the typical element of $`\mathrm{\Lambda }^2^n`$, where $`2ln`$. (cf. \[H2\]).