text:
# Contents

## 0 Introduction

Classically, for a smooth projective curve $`C`$ one defines a principally polarized abelian variety, its Jacobian $`JC`$. If $`C`$ has nodes, one has, in fact, two analogues: the Picard variety, which in this case is a semiabelian group variety and usually not complete, and a ”compactified Jacobian”, which is a projective variety. Oda and Seshadri \[OS\] define several such compactifications. Namikawa \[N2\] defines one variety $`\overline{J}C`$, and in \[A1\] it is shown how to construct a theta divisor $`\mathrm{\Theta }`$ on it and how to obtain the pair $`(\overline{J},\mathrm{\Theta })`$ as a stable semiabelic pair. According to Mumford and Namikawa \[N2\], by using methods of toric geometry, the Torelli map from the moduli space of curves $`M_g`$ to the moduli space $`A_g`$ of principally polarized abelian varieties can be extended to a morphism from the Deligne-Mumford compactification $`\overline{M}_g`$ to the toroidal compactification of $`A_g`$ for the 2nd Voronoi fan. The latter compactification appears in \[A2\] as the closure of $`A_g`$ in the moduli of stable semiabelic pairs and \[A1\] gives a moduli interpretation of the extended map.

Again, classically, a smooth projective curve $`C`$ with an involution $`\iota `$ defines a polarized abelian variety, the Prym variety $`P(C,\iota )`$. When the involution has 0 or 2 fixed points, the polarization is (twice) a principal polarization. What happens if we now consider a nodal curve $`(C,\iota )`$ with an involution? The quotient curve $`C^{}=C/\iota `$ is again a nodal curve and the identity component of the kernel of the norm map $`\mathrm{Nm}:JCJC^{}`$ is a semiabelian variety. One is naturally led to the question whether this variety has ”good” compactifications. The naive approach would be to take the closure of the identity component of the kernel of the norm map in $`\overline{J}C`$. Examples quickly show that this naive approach does not always give the right answer. Indeed, the question allows more than one interpretation. The restriction of the theta divisor of a Jacobian to the Prym in the case of a fixed point free involution of a smooth curve defines twice a principal polarization on the Prym variety. On a smooth abelian variety this defines uniquely a principal polarization. The situation is more complicated for degenerations of abelian varieties. There are very well behaved degenerations of polarized abelian varieties with a divisible polarization such that the limit polarization is not divisible (e.g. the degeneration of elliptic curves with a degree $`n`$ divisor to a fibre of type $`I_n`$). Hence we must expect different answers whether we consider the degeneration problem for principally polarized Pryms or for Prym varieties with the induced polarization. We will address both of these questions in this paper.

The first question is closely related to the Prym map from the space $`_g`$ of unramified double covers of a curve of genus $`g`$ to $`A_{g1}`$. The space $`_g`$ has a natural normal compactification $`\overline{}_g`$ parametrizing admissible double covers of stable curves and the question arises whether the Prym map extends to a regular morphism from $`\overline{}_g`$ to the second Voronoi compactification of $`A_{g1}`$ (or some other suitable compactification). Beauville has constructed a partial extension of the Prym map in the case where the kernel of the norm map has no toric part, i.e. where the image of the Prym map is still contained in $`A_{g1}`$. The preprint version of the paper by Friedman and Smith \[FS\] contains some examples of admissible double covers where the Prym map does not extend to any reasonable toroidal compactification of $`A_{g1}`$. The main result of the present paper is a combinatorial criterion (see Theorem 3.2) which characterizes those admissible double covers which lie in the indeterminacy locus of the Prym map. In the appendix to this paper by Vologodsky \[V\] this combinatorial criterion is translated into a more geometric criterion which says that the locus of indeterminacy is the closure of the locus given by the examples found by Friedman and Smith. It should be pointed out, that the fact that the Prym map has indeterminacies, is not related to singularities of $`\overline{}_g`$. Indeed, the variety $`\overline{}_g`$ is smooth at a generic point representing an example of Friedman-Smith type.

The contents of this paper is organized as follows. In Section 1 we study the kernel of the norm map $`\mathrm{Nm}:JCJC^{}`$. Its identity component is a semiabelian variety $`P`$, whose classifying homomorphism we determine. Moreover we compute the number of components of the kernel of the norm map (Proposition 1.7) and determine the induced polarization type of the abelian part of $`P`$ (Lemma 1.8, Proposition 1.9, and the remarks following these statements).

Section 2 contains, for the reader’s convenience, a brief summary of the relevant parts of the degeneration theory of abelian varieties in a form which is relevant for this paper. What is new here is a discussion of induced degeneration data.

In Section 3 we consider degenerations of Jacobian varieties and of principally polarized Prym varieties. Here we shall relate the degeneration data which we obtain to the results from section 1. As the main application we determine the precise locus of indeterminacy of the extended Prym map (Theorem 3.2). Our answer is given in terms of combinatorial data of the graph $`(\mathrm{\Gamma },\iota )`$ associated to the curve $`C`$. Given a concrete example it is easy to check whether the point $`[(C,\iota )]`$ lies in the indeterminacy locus or not.

In Section 4 we consider the non-principally polarized case. For any nodal curve with an involution (including those to which the Prym map does not extend) we construct a “middle” compactified Prym variety of which we show that it is the limit of non-principally polarized Pryms (Theorem 4.6). This construction is closer in spirit to the naive approach of taking the closure of $`P`$ in $`\overline{J}C`$. It is, therefore, natural to compare this compactification of the Prym variety to the compactified Jacobian $`\overline{J}C`$. In order to do this we construct several finite morphisms $`f_v`$ from the “middle” compactified Prym variety to the compactified Jacobian of $`C`$ (see Theorem 4.1). Among these morphisms we identify the best one, which corresponds to a “maximal half-integral shift” $`v`$.

Section 5 is devoted to a series of examples which illustrate various aspects of our theory. In particular, example 5.2.3 is devoted to the Friedman-Smith examples. In the simplest case where the Prym map is not defined we obtain a $`^1`$ of different degenerations. In example 5.2.4 we illustrate in a simple case how the degenerations in the principally polarized and in the non-principally polarized case are related to each other.

Acknowledgement. We would like to thank Professors R.Smith and R.Varley for helpful discussions. The first named author was supported by the NSF. The last named author is grateful to the University of Georgia at Athens for hospitality during two visits. This project was further supported by the DFG special research project “Global methods in complex geometry”, grant Hu 337/5-1.

## 1 The semiabelian part of the Prym variety

### 1.1 Construction of $`P`$

Throughout this paper (with the exception of sections $`2`$ and $`3`$ where we will work with families) we will denote by $`C`$ a connected nodal curve defined over an algebraic field $`k`$ of characteristic different from $`2`$ , i.e. a projective curve whose singularities are at most nodes.

To every such curve we can associate a graph $`\mathrm{\Gamma }=\mathrm{\Gamma }(C)`$ whose vertices $`\{v_i\}_{iI}`$ correspond to the irreducible components $`C_i`$ of $`C`$ and whose edges $`\{e_j\}_{jJ}`$ correspond to the nodes $`Q_j`$ of $`C`$. After choosing an orientation (it will not matter which one we choose) for $`\mathrm{\Gamma }`$ one can define a chain complex $`C_{}(\mathrm{\Gamma },)`$ where

$$C_0(\mathrm{\Gamma },)=\underset{iI}{}v_i,C_1(\mathrm{\Gamma },)=\underset{jJ}{}e_j$$

and where $`(e_j)=v_iv_i^{}`$ if $`e_j`$ is an edge going from $`v_i`$ to $`v_i^{}`$. Moreover we choose pairings $`[,]`$ on $`C_0(\mathrm{\Gamma },)`$ and $`(,)`$ on $`C_1(\mathrm{\Gamma },)`$ by $`[v_i,v_j]=\delta _{ij}`$ and $`(e_i,e_j)=\delta _{ij}`$. One can then identify the cochain complex $`C^{}(\mathrm{\Gamma },)`$ with the chain complex $`C_{}(\mathrm{\Gamma },)`$ via these pairings. Thus the coboundary map $`\delta `$ becomes $`\delta v_i=\underset{jJ}{}[v_i,e_j]e_j`$. The homology and cohomology groups $`H_i(\mathrm{\Gamma },)`$ and $`H^i(\mathrm{\Gamma },);i=0,1`$ are defined in the usual way. For details of this see \[OS, chapter I\].

The Jacobian $`JC`$ is defined as the group of line bundles on $`C`$ whose multidegree is $`0`$. This defines a group scheme which coincides with the usual Jacobian if $`C`$ is smooth, but which is not necessarily compact otherwise. We denote by $`\nu :NC`$ the normalization of $`C`$. By \[OS, Proposition 10.2\] the group scheme $`JC`$ is an extension

$`1H^1(\mathrm{\Gamma },k^{})JC\stackrel{\nu ^{}}{}JN0`$ (1)
where $`JN=\underset{iI}{}JN_i`$ is the product of the Jacobians of the normalizations of the components $`C_i`$ of $`C`$. This extension defines an element in the group $`\text{Ext}^1(JN,H^1(\mathrm{\Gamma },k^{}))=\text{Hom}(H_1(\mathrm{\Gamma },),{}_{}{}^{t}JN).`$

It is not difficult to make this explicit: Every edge $`e_j`$ corresponds to a double point $`Q_j`$ and over $`Q_j`$ lie two points $`Q_j^+`$ and $`Q_j^{}`$ in $`N`$. If $`e_j`$ is not a loop, i.e. an edge going from $`v_i`$ to $`v_i^{}`$ with $`i^{}i`$ we can distinguish $`Q_j^+`$ and $`Q_j^{}`$ by saying that $`Q_j^+`$ lies on $`N_i`$ and $`Q_j^{}`$ lies on $`N_i^{}`$. Otherwise we choose one of the points above $`Q_j`$ arbitrarily as $`Q_j^+`$ and the other as $`Q_j^{}`$. Then we can associate to $`e_j`$ the line bundle $`𝒪_N(Q_j^+)𝒪_N(Q_j^{})`$ whose total degree is $`0`$, but whose multidegree is different from $`0`$ if $`e_j`$ is not a loop. By linearity we obtain a map

$$u:C_1(\mathrm{\Gamma },)\underset{iI}{}\text{Pic}(N_i).$$

If we restrict this map to $`H_1(\mathrm{\Gamma },)`$ then the images are line bundles of multidegree $`0`$, and using the identification $`JN={}_{}{}^{t}JN`$ we thus have a homomorphism

$$c:H_1(\mathrm{\Gamma },)JN={}_{}{}^{t}JN$$

which we call the classifying map of the Jacobian $`JC`$. Note that $`JC`$ is given by the classifying homomorphism via the negative of the pushout. It is not difficult to check that this map corresponds to the extension (1) (see \[N1, Proposition 7\]). (Note that our definition of the classifying map depends on the orientation of the graph $`\mathrm{\Gamma }`$ and on the choice of $`Q_j^+`$ and $`Q_j^{}`$ if $`e_j`$ is a loop. Both choices also appear in the identification of the kernel of (1) with $`H^1(\mathrm{\Gamma },k^{})`$ and correspond to an automorphism $`zz^1`$ in some factor of the torus $`H^1(\mathrm{\Gamma },k^{}`$).)

Let $`\iota :CC`$ be an involution with quotient $`C^{}=C/\iota `$. We assume that $`\iota `$ is not the identity on any of the components $`C_i`$ of $`C`$. The quotient $`C^{}`$ is again a nodal curve. We denote the quotient map by $`\pi :CC^{}`$. If $`C`$ (and hence $`C^{}`$) is smooth, the norm map is defined by

$$\begin{array}{ccc}\hfill \pi _{}=\mathrm{Nm}:JC& & JC^{}\hfill \\ \hfill 𝒪(\mathrm{\Sigma }n_iP_i)& & 𝒪(\mathrm{\Sigma }n_i\pi (P_i)).\hfill \end{array}$$

Recall that there is a norm map

$$\pi _{}=\mathrm{Nm}:JCJC^{}.$$

also in the case where $`C`$ is a nodal curve (for a general definition see e.g. \[EGA, Proposition II.6.5\]). One can show easily that the norm map is given by

$$\mathrm{Nm}:(\iota ^{})/\iota .$$

This coincides with the norm map described above when $`C`$ is smooth.

###### Lemma 1.1

The norm map $`\mathrm{Nm}:JCJC^{}`$ is surjective.

Proof. If $`JC^{}`$ then $`:=\pi ^{}JC`$ and $`\mathrm{Nm}()=^2`$. Surjectivity then follows since $`JC^{}`$ is 2-divisible. ∎

We first want to understand the structure of the group scheme

$$P=\mathrm{ker}(\mathrm{Nm}:JCJC^{})_0,$$

i.e. the component of the kernel of the norm map containing the identity. We shall then ask about good “compactifications”of $`P`$. As in the classical case, $`P`$ can be alternatively defined as

$$P=\mathrm{ker}((1+\iota ):JCJC)_0=\mathrm{im}((1\iota ):JCJC).$$

###### Definition

We call $`P`$ the (open) Prym variety associated to the double cover $`\pi :CC^{}`$.

We shall see later that $`P`$ carries the structure of a semiabelian variety.

We have 3 possible types of double points, namely:

1. Fixed points of $`\iota `$ where the 2 branches are not exchanged. Then the image under $`\pi `$ is again a node. We shall call these nodes (branchwise) fixed.
2. Fixed points of $`\iota `$ where the 2 branches are exchanged. Then the image of this node is a smooth point of $`C^{}`$ and we shall refer to such nodes as swapping nodes.
3. Nodes which are exchanged under $`\iota `$. These we shall call non-fixed nodes.

Recall that the group of Cartier divisors on $`C`$ is

$$\text{Div}C=\underset{xC_{\mathrm{reg}}}{}x+\underset{Q\mathrm{singular}}{}K_Q^{}/𝒪_Q^{}.$$

After choosing local parameters for the 2 branches of $`C`$ which intersect at a point $`Q`$ we have an identification (see also \[B, p.158\])

$$K_Q^{}/𝒪_Q^{}k^{}\times \times .$$

Here $`K`$ denotes the ring of rational functions on the normalization $`N`$ and the two integers are the multiplicities of the divisors on the branches. For the three types of fixed points we have:

1. In this case $`\pi (Q)`$ is again a node. The involution $`\iota ^{}`$ and the map $`\pi _{}`$ are given by

$$\begin{array}{ccc}\iota ^{}(c,m,n)\hfill & =& ((1)^{m+n}c,m,n),\hfill \\ \pi _{}(c,m,n)\hfill & =& ((1)^{m+n}c^2,m,n)).\hfill \end{array}$$
2. Here $`\pi (Q)`$ is a smooth point and we have

$$\begin{array}{ccc}\iota ^{}(c,m,n)\hfill & =& (\frac{1}{c},n,m)\hfill \\ \pi _{}(c,m,n)\hfill & =& (m+n)\pi (Q).\hfill \end{array}$$
3. If the nodes $`Q`$ and $`Q^{}`$ are interchanged, then

$$\iota ^{}:K_Q^{}^{}/𝒪_Q^{}^{}K_Q^{}/𝒪_Q^{}$$

and after identifying $`K_Q^{}/𝒪_Q^{}`$ with $`K_{\pi (Q)}^{}/𝒪_{\pi (Q)}^{}`$ the map $`\pi _{}`$ induces the identity.

We shall denote the edges, resp. vertices of the graph $`\mathrm{\Gamma }^{}=\mathrm{\Gamma }(C^{})`$ by $`f_j^{},j^{}J^{}`$ resp. $`w_i^{},i^{}I^{}`$. The involution $`\iota `$ acts on the sets $`I`$ and $`J`$ and hence by linearity on $`C_0(\mathrm{\Gamma },)`$ and $`C_1(\mathrm{\Gamma },)`$ and we can choose orientations of the graphs $`\mathrm{\Gamma }`$ and $`\mathrm{\Gamma }^{}`$ which are compatible with $`\iota `$ in the following sense: If $`e_j`$ is an edge in $`\mathrm{\Gamma }`$ from $`v_i`$ to $`v_i^{}`$, $`ii^{}`$ with $`e_je_{\iota (j)}`$, then $`e_{\iota (j)}`$ goes from $`v_{\iota (i)}`$ to $`v_{\iota (i^{})}`$. Otherwise we choose an arbitrary orientation on $`e_j`$. If $`f_j^{}`$ is an edge of $`\mathrm{\Gamma }^{}`$ which is not a loop, it is either the image of an edge $`e_j`$ from $`v_i`$ to $`v_i^{}`$ where $`\iota `$ fixes $`j,i`$ and $`i^{}`$ or it is the image of two edges $`e_j`$ and $`e_{\iota (j)}`$ with $`j\iota (j)`$. In the first case we orient $`f_j^{}`$ in such a way that it goes from $`w_{\pi (i)}`$ to $`w_{\pi (i^{})}`$. In the second case we choose the orientation in such a way that $`f_j^{}=f_{\pi (j)}`$ goes from $`w_{\pi (i)}`$ to $`w_{\pi (i^{})}`$ if $`e_j`$ goes from $`v_i`$ to $`v_i^{}`$. By our choice of the orientation on $`\mathrm{\Gamma }`$ this is well defined. For a loop in $`\mathrm{\Gamma }^{}`$ we choose an arbitrary orientation.

We define homomorphisms

$$\iota _l:C_l(\mathrm{\Gamma },)C_l(\mathrm{\Gamma },),l=0,1.$$

For $`l=0`$ this is defined by $`\iota _0(v_i)=v_{\iota (i)}`$. For $`l=1`$ we set $`\iota _1(e_j)=\pm e_{\iota (j)}`$ where we always use the $`+`$-sign with the following two exceptions: we set $`\iota _1(e_j)=e_j`$ if either $`e_j`$ comes from a loop which corresponds to a swapping node or if $`e_j`$ is an edge from $`v_i`$ to $`v_i^{},ii^{}`$ and $`j=\iota (j),i^{}=\iota (i)`$. It follows directly from the definitions that $`\iota _0`$ and $`\iota _1`$ commute with the boundary map $``$ and dually with the coboundary map $`\delta `$. Hence $`\iota `$ induces involutions

$$\iota :H_1(\mathrm{\Gamma },)H_1(\mathrm{\Gamma },),\iota :H^1(\mathrm{\Gamma },)H^1(\mathrm{\Gamma },).$$

The involution $`\iota :CC`$ also induces an involution on the extension

$$1T_C=H^1(\mathrm{\Gamma },)k^{}JCJN0.$$

###### Lemma 1.2

Under the identification $`T_C=H^1(\mathrm{\Gamma },)k^{}`$ the restriction of the involution $`\iota :JCJC`$ to $`T_C`$ is induced by $`\iota :H^1(\mathrm{\Gamma },)H^1(\mathrm{\Gamma },)`$.

Proof. Recall from the proof of \[OS, Proposition 10.2\] that we have an exact sequence

$$1𝒪_C^{}𝒪_N^{}\stackrel{\alpha }{}\underset{jJ}{}k_{Q_j}^{}1$$

where $`k_{Q_j}^{}`$ is the skyscraper sheaf with fibre $`k^{}`$ at the node $`Q_j`$. (The identification of the stalk of the cokernel over a node $`Q_j`$ with $`k^{}`$ depends on choosing an order of the two branches at $`Q_j`$. Whenever $`Q_j`$ defines an edge in $`\mathrm{\Gamma }`$ which is not a loop then we shall choose this order according to the chosen orientation of $`\mathrm{\Gamma }`$. If $`Q_j`$ gives rise to a loop then we can choose the order arbitrarily.) The involution $`\iota `$ acts on $`𝒪_C^{}`$ and on $`𝒪_N^{}`$ and hence also on the quotient. If $`Q_j`$ is a branchwise fixed node, then $`\iota `$ acts trivially on $`k_{Q_j}^{}`$, if $`Q_j`$ is a swapping node, then $`\iota `$ acts by $`zz^1`$ and otherwise $`\iota `$ interchanges $`k_{Q_j}^{}`$ and $`k_{Q_{\iota (j)}}^{}`$. The above sequence induces a long exact sequence

$$1H^0(𝒪_C^{})H^0(𝒪_N^{})\stackrel{\alpha }{}\underset{jJ}{}k_{Q_j}^{}JCJN0.$$

The extension (1) is then a consequence of the observation that the map $`\alpha :H^0(𝒪_N^{})_{jJ}k_{Q_j}^{}`$ coincides with the coboundary map $`\delta :C_0(\mathrm{\Gamma },k^{})C_1(\mathrm{\Gamma },k^{})`$. Under this identification the action of $`\iota `$ coincides with the action induced by $`\iota _0`$ and $`\iota _1`$ and this proves the claim. ∎

The norm map $`\mathrm{Nm}:JCJC^{}`$ induces a diagram

$$\text{}(\text{D1})$$

Note that all vertical maps are surjective. As before this follows since $`\mathrm{Nm}(\pi ^{})=^2`$ for $`JC^{}`$ and since the groups involved are 2-divisible.

Adding the kernels of the vertical maps in diagram (D1) we obtain the following diagram with exact rows and columns:

$$\text{}(\text{D}2)$$

None of the group schemes in the top row need be connected. We had already defined $`P`$ as the identity component of $`P^{}`$. Similarly we denote the identity components of the other groups in the top row by

$$K_N=(K_N^{})_0,T_P=(T_P^{})_0.$$

The group scheme $`K_N`$ is an abelian variety which is the Prym variety for the double cover $`NN^{}`$, and we have a morphism $`PK_N`$ whose fibers, however, need not be connected. The identity component of the kernel $`K_P`$ of this map is isomorphic to $`T_P`$ and we have a commutative diagram

$$\begin{array}{ccccccccc}& & & & 0& & 0& & \\ & & & & & & & & \\ & & & & T_P& & K_P& & \\ & & & & & & & & \\ & & & & P& =& P& & \\ & & & & & & & & \\ 0& & H& & A& & K_N& & 0\\ & & & & & & & & \\ & & & & 0& & 0& & \end{array}$$

where $`HK_P/T_P`$ is a finite group. In particular $`A`$ is an abelian variety and the left hand column gives $`P`$ the structure of a semiabelian variety. For future use we have to determine the classifying homomorphism of this semiabelian variety. Before we do this we have to describe the various lattices which will play a role.

The involution $`\iota `$ acting on the lattices $`H^1(\mathrm{\Gamma },)`$, resp. $`H_1(\mathrm{\Gamma },)`$ defines ($`\pm )1`$-eigenspaces $`[H^1(\mathrm{\Gamma },)]^\pm `$, resp. $`[H_1(\mathrm{\Gamma },)]^\pm `$.

###### Proposition 1.3

$`T_P=[H^1(\mathrm{\Gamma },)]^{}k^{}`$.

Proof. We first observe that $`T_P\text{Ker}((1+\iota )\text{id}_k^{})`$. This follows since $`\mathrm{Nm}()=(\iota ^{})/\iota =𝒪_C^{}`$, implies that $`\iota ^{}=𝒪_C`$. The identity component of $`\text{Ker}((1+\iota )\text{ id}_k^{})`$ is $`\text{Ker}(1+\iota )k^{}=[H^1(\mathrm{\Gamma },)]^{}k^{}`$, hence $`T_P[H^1(\mathrm{\Gamma },)]^{}k^{}`$. Next we claim that for every $``$ with $`\iota ^{}=𝒪_C`$ the image $`=\mathrm{Nm}()`$ is 2-torsion. This follows since $`^2=\mathrm{Nm}(\pi ^{})=\mathrm{Nm}(\iota ^{})=\mathrm{Nm}(𝒪_C)=𝒪_C^{}`$. Since the 2-torsion elements in $`JC^{}`$ are discrete the inclusion $`[H^1(\mathrm{\Gamma },)]^{}k^{}T_P`$ follows if there is one element in $`[H^1(\mathrm{\Gamma },)]^{}k^{}`$ whose image under the norm map is $`𝒪_C^{}`$. Since $`𝒪_C`$ is in $`[H^1(\mathrm{\Gamma },)]^{}k^{}`$ we are done. ∎

###### Remark

The above proof also shows that $`K_P=[H^1(\mathrm{\Gamma },)]^{}k^{}\times (\mu _2)^N`$ for some integer $`N`$, where $`\mu _2=\{\pm 1\}`$.

For future use we fix the notation

$$X=H_1(\mathrm{\Gamma },),[X]^{}=[H_1(\mathrm{\Gamma },)]^{}.$$

Note that the dual lattice $`X^{}`$ is the group of $`1`$-parameter subgroups of the torus $`T_C`$, i.e. $`T_C=X^{}k^{}`$.

Recall (cf.\[CR, Theorem (74.3)\]) that every involution $`\iota `$ on a lattice of finite rank has, with respect to a suitable basis, the form

$$\iota =k(1)+l(1)+m\left(\begin{array}{cc}0& 1\\ 1& 0\end{array}\right).$$

Such a decomposition is not canonical, but the integers $`k,l`$ and $`m`$ are independent of the given decomposition. We consider the homomorphism

$$\begin{array}{ccc}\hfill \pi ^{}:H_1(\mathrm{\Gamma },)& & H_1(\mathrm{\Gamma },\frac{1}{2})\hfill \\ \hfill h& & \frac{1}{2}(h\iota (h))\hfill \end{array}$$

and define

$$X^{}=\pi ^{}(H_1(\mathrm{\Gamma },)).$$

(Note that $`X^{}`$ is not the $`(1)`$-eigenspace of the lattice $`X`$.) Then there is an exact sequence

$$0[H_1(\mathrm{\Gamma },)]^+H_1(\mathrm{\Gamma },)\stackrel{\pi ^{}}{}X^{}0$$

and the quotient map $`\pi ^{}:H_1(\mathrm{\Gamma },)X^{}`$ is the dual to the inclusion $`[H^1(\mathrm{\Gamma },)]^{}H^1(\mathrm{\Gamma },)`$. Also note that after tensoring with $``$ the map $`\pi ^{}`$ identifies the $`(1)`$-eigenspace $`H_1(\mathrm{\Gamma },)^{}`$ with $`X_{}^{}=X^{}`$. Over the rationals $`\pi ^{}`$ is nothing but the projection onto the $`(1)`$-eigenspace. Via the map $`\pi ^{}`$ we can consider $`[X]^{}`$ as a sublattice of $`X^{}`$ with

$$X^{}/[X]^{}(/2)^m.$$

Finally note that since $`X^{}`$ is dual to $`[H^1(\mathrm{\Gamma },)]^{}`$, it is the character group of the torus $`T_P`$ (cf Proposition 1.3).

The abelian base of the semiabelian variety $`P`$ is $`A`$ and the character group of its toric part is $`X^{}`$. By the general theory of semiabelian varieties the group scheme $`P`$ is defined by its classifying homomorphism $`c^{}:X^{}{}_{}{}^{t}A`$. We have to determine this homomorphism. In order to do this recall the following facts about homomorphisms of semiabelian varieties. Assume that $`1T_iG_iA_i0`$; $`i=1,2`$ are two semiabelian varieties with classifying homomorphisms $`c_i:X_i{}_{}{}^{t}A_{i}^{}`$. Then giving a homomorphism $`\phi :G_1G_2`$ is equivalent to giving two homomorphisms $`{}_{}{}^{t}\phi _{T}^{},{}_{}{}^{t}\phi _{A}^{}`$ making the following diagram commutative:

The following is a standard lemma whose proof we omit. In this lemma we assume for simplicity that the characteristic of the field $`k`$ does not divide the order of any of the finite groups appearing. Note that we will be in this position since we have assumed that the characteristic of $`k`$ is different from $`2`$ and since the finite groups which will appear are all $`2`$-groups.

###### Lemma 1.4

Let $`\phi :G_1G_2`$ be a homomorphism of semiabelian varieties.

1. The identity component of the kernel $`(\mathrm{ker}\phi )_0`$ is a semiabelian variety defined by the last column in the diagram
2. The image $`\mathrm{im}\phi `$ is a semiabelian variety defined by the third column in the diagram

Here $`(\mathrm{ker}{}_{}{}^{t}\phi _{A}^{})_0((\mathrm{ker}{}_{}{}^{t}\phi _{A}^{})_0+c_2(\mathrm{ker}{}_{}{}^{t}\phi _{T}^{}))\mathrm{ker}{}_{}{}^{t}\phi _{A}^{}`$ with finite cokernels.

We can now obtain a description of the classifying homomorphism of $`P`$. For this we apply the first part of this lemma to $`P=\left(\mathrm{ker}(1+\iota )\right)_0`$. Then we have

$`X\stackrel{1+\iota }{}XX/(1+\iota )(X)(X/(1+\iota )(X))/\mathrm{Torsion}=X^{}`$ (2)
and the torsion group is identified with the image of $`X^+`$ and, moreover, is isomorphic to $`(/2)^k`$, where $`k`$ is the number of $`(+1)`$-blocks in the decomposition of the involution $`\iota `$. This immediately gives

###### Proposition 1.5

The classifying homomorphism of the semiabelian variety $`P`$ is given by the right hand vertical map of the following diagram

The abelian variety $`A`$ is a $`2^a`$ cover of the Prym variety $`K_N`$ of the normalization, where $`ak`$ and $`k`$ is the number of $`(+1)`$-blocks in the decomposition of the involution $`\iota :XX`$.

### 1.2 Geometric properties of the kernel of the norm map

in this subsection we shall study the geometric properties of the kernel of the norm map. In particular, we shall determine the number of its connected components and the type of the induced polarization on the abelian part $`A`$ of $`P`$. We write the set $`S`$ of nodes of $`C`$ as

$$S=S_e\iota (S_e)S_fS_f^{}$$

where $`S_e\iota (S_e)`$ is the set of nodes which are exchanged under $`\iota `$, $`S_f`$ is the set of nodes which are branchwise fixed and $`S_f^{}`$ is the set of swapping nodes. For every node $`s`$ we have (after a choice of local parameters) a homomorphism

$$\begin{array}{ccc}\hfill k_s^{}& & JC\hfill \\ \hfill c_s& & 𝒪(c_s,0,0).\hfill \end{array}$$

(Note that we are not claiming that this map is injective. If $`s`$ is a disconnecting node, then the above homomorphism is the constant homomorphism mapping $`k_s^{}`$ to the trivial line bundle.) Recall that $`P^{}`$ is the kernel of the norm map.

###### Proposition 1.6

Every element $`P^{}`$ is of the form

$$\underset{sS_f^{}}{}𝒪(c_s,0,0)\iota ^{}^1$$

where $`\mathrm{Pic}(C)`$ is a line bundle on $`C`$ whose degree on every component of $`C`$ is $`0`$ or $`1`$.

Proof. As in \[M, Lemma 1\] or \[B, Lemma (3.3)\] we can assume that $`=𝒪(D)`$ where $`D`$ is a Cartier divisor with $`\pi _{}(D)=0`$. For branchwise fixed nodes we have

$$(1,0,0)=(1,0,1)\iota ^{}(1,0,1)$$

and for swapping nodes we have

$$(1,1,1)=(1,1,0)\iota ^{}(1,1,0).$$

Hence we have a decomposition

$$D=D^{}\iota ^{}D^{}+\mathrm{\Sigma }_{sS_f^{}}(c_s,0,0).$$

This shows the claim apart from the assertion on the degrees of $``$. We can always replace $``$ by $`\pi ^{}𝒩`$ for some line bundle $`𝒩`$ on $`C^{}`$. This shows immediately that we can assume that the degree of $``$ is $`0`$ or $`1`$ on components $`C_i`$ of $`C`$ which are fixed under $`\iota `$. Now assume that $`C_i`$ and $`C_{\iota (i)}`$ are two components which are interchanged. Since $`P^{}JC`$ we have $`\mathrm{deg}|_{C_i}=\mathrm{deg}|_{C_{\iota (i)}}=0`$ and this implies that $`\mathrm{deg}|_{C_i}=\mathrm{deg}|_{C_{\iota (i)}}`$. But then, after replacing $``$ by $`\pi ^{}𝒩`$ for some line bundle $`𝒩`$ on $`C^{}`$ we can in fact assume that $`\mathrm{deg}|_{C_i}=0`$ for all components $`C_i`$ which are not fixed under the involution $`\iota `$. ∎

In order to compute the number of components of $`P^{}`$ we have to determine in how far we can normalize the multidegree of the line bundles $``$ in Proposition 1.6. The proof of this proposition shows that we can assume that the degree of $``$ is $`0`$ on components of $`C`$ which are not fixed under $`\iota `$. As before we denote by $`B`$ the union of the components of $`C`$ which are fixed by $`\iota `$. Let $`N_B^{}`$ be the partial normalization of $`B`$ obtained by blowing up the nodes of $`B`$ which are not branchwise fixed. Then $`N_B^{}`$ decomposes into connected components

$$N_B^{}=N_B^1\mathrm{}N_B^{n_B}L_B^1\mathrm{}L_B^{l_B}$$

where the components $`N_B^i`$ have no smooth fixed points with respect to $`\iota `$, whereas the components $`L_B^j`$ have such a fixed point.

###### Proposition 1.7

The number of components of the kernel $`P^{}`$ of the norm map equals $`2^{n_B}`$ where $`n_B`$ is the number of connected components of the curve $`N_B^{}`$ on which the involution $`\iota `$ has no smooth fixed points.

Proof. We first show that the number of components of $`P^{}`$ is at most $`2^{n_B}`$. The involution $`\iota `$ acts on each of the components $`N_B^i`$, $`L_B^j`$ defining varieties $`P_{N_B^i}`$, $`Q_{L_B^j}`$ which are defined as the kernel of the restricted norm maps. It was shown by Beauville \[B, Lemma 3.3\] that $`P_{N_B^i}`$ has $`2`$ components. In particular Beauville showed that one can normalize the multidegree of $`|_{N_B^i}`$ to $`(0,0,\mathrm{},0)`$ or $`(1,0,\mathrm{},0)`$ depending on whether the sum of the degrees is even or odd. The curves $`L_B^j`$ have a smooth fixed point $`x=\iota (x)`$. Since $`𝒪(x)\iota ^{}(𝒪(x))^1`$ is trivial we can change the parity of the multidegree of $``$ in this case and hence in this case $`Q_{L_B^j}`$ is irreducible. We had already remarked that we can assume that the multidegree on $`D\iota (D)`$ is $`(0,0,\mathrm{},0)`$. This shows that it suffices to consider at most $`2^{n_B}`$ possible multidegrees. Since the variety of line bundles on $`C`$ with fixed multidegree is irreducible, it follows from Proposition 1.6 that the number of components of $`P^{}`$ is at most $`2^{n_B}`$.

To prove that the number of components is at least $`2^{n_B}`$ it is sufficient to show that the restriction $`P^{}\underset{i}{}P_{N_B^i}\times \underset{j}{}Q_{L_B^j}`$ is surjective. We choose line bundles $`_iP_{N_B^i}`$, resp. $`_jQ_{L_B^j}`$ and the trivial bundle $`𝒪_{D\iota (D)}`$. We want to glue these line bundles to a line bundle $``$ on $`C`$ which is contained in $`P^{}`$. To do this we have to specify the gluing at the nodes which are not branchwise fixed. The gluing over the swapping nodes can be chosen arbitrarily. Since we can always choose the gluing at a node $`s`$ and its image $`\iota (s)`$ in such a way that $`\mathrm{Nm}()=𝒪_C`$ the claim follows. ∎

Our next aim is to determine the type of the polarization on $`A`$ which is induced from the principal polarization on $`JN`$ via the map $`AK_NJN`$. Roughly speaking $`A`$ is made up from $`3`$ types of building blocks which arise as follows:

1. Let $`D_i`$ and $`D_{\iota (i)}`$ be $`2`$ components of $`C`$ which are exchanged under the involution $`\iota `$ and denote the normalizations of these components by $`N_{D_i}`$ and $`N_{D_{\iota (i)}}`$. Then the kernel of the norm map defined by the quotient map $`N_{D_i}N_{D_{\iota (i)}}N_{D_i}=N_{D_{\iota (i)}}`$ is $`JD_{N_i}`$ embedded as the anti-diagonal in $`JN_{D_i}\times JN_{D_i}`$. The restriction of the product polarization on $`JN_{D_i}\times JN_{D_i}`$ to the (anti-)diagonal is twice a principal polarization.
2. The involution $`\iota `$ on a component $`N_B^i`$ defines an abelian variety $`P_{N_B^i}`$ and the induced polarization is twice a principal polarization. This was shown by Beauville \[B\].
3. The third building block comes from the components $`L_B^j`$. To simplify the notation we consider a connected nodal curve $`L`$ together with an involution $`\iota `$ where we assume that all nodes are branchwise fixed. We do, however, allow that $`\iota `$ has smooth fixed points. Let $`L^{}=L/\iota `$ and we denote by $`N_L`$ the normalization of $`L`$, resp. by $`N_L^{}^{}`$ that of $`L^{}`$. Then we have varieties

$$\begin{array}{ccccc}Q\hfill & =& \text{Ker}(\mathrm{Nm}:JL\hfill & & JL^{}),\hfill \\ R\hfill & =& \text{Ker}(\mathrm{Nm}:JN_L\hfill & & JN_L^{}^{}).\hfill \end{array}$$

We have already observed that $`Q`$ and $`R`$ are connected. Since all nodes are branchwise fixed the map

$$QRJN_L$$

is finite and hence $`Q`$ is an abelian variety.

###### Lemma 1.8

Assume that $`L`$ is a connected nodal curve with an involution $`\iota :LL`$ which has only branchwise fixed nodes and let $`r`$ be the number of smooth fixed points of $`\iota `$. Let $`Q`$ be the abelian variety defined above. Then

1. The induced polarization on $`Q`$ is of type $`(1,\mathrm{},1,2,\mathrm{},2)`$.
2. It is of type $`(2,\mathrm{},2)`$ if and only if $`r=0`$ or $`2`$.

Proof. The number of smooth fixed points of $`\iota `$ is even. By identifying these points pairwise we obtain a curve $`\overline{C}`$ together with an involution $`\overline{\iota }`$ which is of Beauville type, i.e. all double points are branchwise fixed and there are no smooth fixed points. We can consider $`C`$ as a partial normalization of $`\overline{C}`$. The claim then follows from \[DL, Lemma 1\]. ∎

###### Remark

The number of entries equal to $`1`$ in the above lemma is $`r^{}1`$ if there are $`r=2r^{}>0`$ smooth fixed points.

We can now prove the

###### Proposition 1.9

The polarization of the abelian variety $`A`$ which is induced from the map $`AJN`$ is always of type $`(1,\mathrm{},1,2,\mathrm{},2)`$. It is twice a principal polarization if and only if the following condition ($``$) holds:
($``$) Every connected component of the partial blow-up $`N_B^{}`$ has at most $`2`$ smooth fixed points with respect to $`\iota `$.

Proof. We consider

$$A^{}:=JN_{D_1}\times \mathrm{}\times JN_{D_{n_e}}\times P_{N_B^1}^0\times \mathrm{}\times P_{N_B^{n_B}}^0\times Q_{L_B^1}\times \mathrm{}\times Q_{L_B^{l_B}}$$

where $`P_{N_B^k}^0`$ is the identity component of $`P_{N_B^k}`$. By Lemma (1.8) and the discussion preceding this lemma the polarization induced by the map $`A^{}JN`$ is twice a principal polarization if and only if ($``$) holds. We claim that $`A^{}=A`$. Pulling back line bundles to the partial normalization defined by blowing up all nodes which are not branchwise fixed, resp. to the normalization $`N`$ defines a diagram:

The map $`A^{}K_N`$ is finite. In order to show that $`A^{}=A`$ it is, therefore, enough to prove that the kernel of the map $`PA^{}`$ is connected. Let $``$ be an element in this kernel. Then we can write $`=𝒪_C(D)`$ with $`D=D_1+D_2+D_3`$ where the support of the divisor $`D_k`$ is contained in the set of nodes of type $`(k)`$. Again we can argue as in \[M, Lemma 1\] to conclude that we may assume that $`\pi _{}(D)=0`$. Moreover we claim that we may assume $`D_1=0`$. Indeed, the restriction of $`𝒪(D)`$ to each of the connected components $`N_B^i`$ or $`L_B^j`$ is trivial. Hence there exists for each of these components a rational function $`g^i`$ or $`h^j`$ such that the restriction of $`D`$ to $`N_B^i`$ or $`L_B^j`$ is equal to the principal divisor $`(g^i)`$ or $`(h^j)`$. We can assume that $`g^i`$ and $`h^j`$ are $`\pm 1`$ on each irreducible component of $`N_B^i`$ and $`L_B^j`$. We extend these rational functions to a rational function on $`C`$ by setting it equal to $`1`$ for every component which is not fixed under $`\iota `$. The divisor

$$D^{}:=D\underset{i}{}(g^i)\underset{j}{}(h^j)D$$

is only supported on nodes which are either swapping or non-fixed nodes. We claim that we have still $`\pi _{}(D^{})=0`$. There is nothing to check for swapping nodes. Exchanged nodes come in pairs, say $`\{Q,Q^{}\}`$, which are interchanged by $`\iota `$ and there are the following possibilities. The first possibility is that $`\{Q,Q^{}\}`$ is contained in $`1`$ or $`2`$ curves which are invariant under $`\iota `$. The second possibility is that $`\{Q,Q^{}\}`$ is contained in the intersection of a curve $`B_i`$ with a curve $`D_j`$ and its image $`D_{\iota (j)}`$ under the involution $`\iota `$. The third possibility is that none of the curves containing $`\{Q,Q^{}\}`$ is fixed under $`\iota `$. In either case the property $`\pi _{}(D)=0`$ is not effected by changing $`D`$ to $`D^{}`$. We claim that the line bundles $`𝒪(D^{})`$ with $`\pi _{}(D^{})=0`$ and $`D^{}`$ supported at swapping nodes and exchanged nodes such that the pull back of $`D^{}`$ to $`N`$ is trivial are parameterized by an irreducible variety. This follows since the variety of line bundles of the form

$$\underset{sS_f^{}}{}𝒪(c_s,0,0)\underset{sS_e}{}𝒪(d_s,0,0)\underset{sS_e}{}𝒪(d_{\iota (s)}^1,0,0)$$

is the image of a finite product of tori $`k^{}`$ and hence irreducible. ∎

###### Remark

The description of $`A`$ in the proof of the above theorem together with the remark after Lemma 1.8 allows us to compute the degree of the induced polarization on $`A`$.

###### Remark

Geometrically the most interesting case occurs when $`(C,\iota )`$ can be smoothed to a curve $`(C(t),\iota (t))`$, where $`\iota (t)`$ has $`0`$ or $`2`$ smooth fixed points. This is the case if $`(C,\iota )`$ has either no smooth fixed points and at most one swapping node or $`2`$ smooth fixed points and no swapping nodes. In either case the above proposition shows that the induced polarization is twice a principal polarization, i.e. of type $`(2,\mathrm{},2)`$.

## 2 Theory of degenerations

### 2.1 General theory of degenerations of abelian varieties

For the reader’s convenience we shall review the basic facts about degenerations of abelian varieties in a form relevant to this article. Unfortunately, a certain amount of technicalities is unavoidable in this subject. We shall begin with the Mumford-Faltings-Chai uniformization of abelian varieties over the quotient field of a complete normal ring, \[FC, Ch.II\].

The general setup is that $`R`$ is a noetherian normal integral domain complete w.r.t. an ideal $`I=\sqrt{I}`$, which is a completion of a normal excellent ring. Let $`S=\mathrm{Spec}R`$, $`S_0=\mathrm{Spec}R/I`$ and let $`\eta `$ be the generic point of $`S`$ and $`K`$ be the fraction field of $`R`$. Assume we are given the following data:

1. Let $`G/S`$ be a semiabelian scheme whose generic fiber $`G_\eta `$ is abelian such that $`G_{S_0}`$ is an extension of an abelian scheme $`A_0`$ by a split torus $`T_0`$,
2. Let $``$ be an invertible sheaf on $`G`$ rigidified at the zero section such that $`_\eta `$ is ample.

The main result of \[FC, Ch.II\] is that the pair $`(G,)`$ is equivalent to a set of *degeneration data* (d0) - (d4) which we will describe below. Here small letters will indicate that we describe the degeneration data of a semiabelian scheme over the base $`S`$ and capital letters will later indicate that we describe the degeneration data of the central fiber over $`S_0`$. The degeneration data of the pair $`(G,)`$ given above are as follows:

1. An abelian scheme $`A/S`$ with an ample rigidified sheaf $``$. This sheaf determines a polarization $`\lambda :A{}_{}{}^{t}A`$, where $`{}_{}{}^{t}A`$ is the dual abelian scheme over $`S`$.
2. 1. A semiabelian scheme, otherwise known as Raynaud extension

    $$1T\stackrel{~}{G}A0$$

    over $`S`$ with a torus $`T`$. Since the torus $`T_0`$ is split, the character group $`X`$ of $`T`$ is a constant group scheme over $`S`$, i.e. $`X^r`$. Then $`\stackrel{~}{G}`$ corresponds uniquely to a homomorphism $`c:X{}_{}{}^{t}A`$ via the negative of the pushout.
    2. A second semiabelian scheme

    $$1{}_{}{}^{t}T{}_{}{}^{t}\stackrel{~}{G}{}_{}{}^{t}A0$$

    with a split torus part, given by a homomorphism $`{}_{}{}^{t}c:YA`$.
3. An inclusion of lattices $`\varphi :YX`$ with a finite cokernel such that $`c\varphi =\lambda {}_{}{}^{t}c`$. This corresponds to a homomorphism $`\stackrel{~}{G}{}_{}{}^{t}\stackrel{~}{G}`$.
4. A bihomomorphism $`\tau :Y\times X({}_{}{}^{t}c\times c)^{}𝒫_\eta ^1`$, where $`𝒫_\eta `$ is a Poincaré biextension on $`(A\times {}_{}{}^{t}A)_\eta `$, the universal Poincaré bundle with the zero section removed. Explicitly, this means that for every $`yY`$, $`xX`$ we are given a nonzero section $`\tau (y,x)`$ of the sheaf $`c(x)_\eta ^1`$ over $`{}_{}{}^{t}c(y)_\eta \mathrm{Spec}K`$, and that

$$\tau (y_1+y_2,x)\tau (y_1,x)^1\tau (y_2,x)^1=1$$

and that a similar identity holds for the linearity in the second variable. The fact that $`𝒫`$ is a biextension means that the left-hand side of the above identity is an element of the trivial $`𝔾_m`$-torsor over $`K`$, so the identity makes sense.

Here $`\tau `$ is required to satisfy the following positivity condition. The element $`\tau (y_1,\varphi (y_2))`$ can be understood as a section of $`(\mathrm{id}_A,\lambda )^{}𝒫_\eta ^1=\mathrm{\Lambda }(_\eta ^{}{}_{}{}^{1})`$ on the pullback by $`({}_{}{}^{t}c\times {}_{}{}^{t}c):Y\times Y(A\times A)_\eta `$. Here $`^{}`$ is the $`𝔾_m`$-torsor associated to the invertible sheaf $``$ and $`\mathrm{\Lambda }(^{})=m^{}^{}p_1^{}^{}{}_{}{}^{1}p_2^{}^{}^1`$ is the canonical symmetric biextension on $`A_\eta \times A_\eta `$, $`p_i`$ are the projections, and $`m`$ is the multiplication map. The positivity condition is that for all $`y`$ the element $`\tau (y,\varphi (y))`$ is defined over the whole of $`S=\mathrm{Spec}R`$ and for every $`n0`$ it is 0 modulo $`I^n`$ for all but finitely many $`y`$.
5. A cubical morphism $`\psi :Y({}_{}{}^{t}c)^{}_\eta ^{}^1`$. Explicitly, this means that for every $`yY`$ we a have a nonzero section $`\psi (y)`$ of the sheaf $`_\eta ^1`$ over $`{}_{}{}^{t}c(y)_\eta `$, and that

$$\psi (y_1+y_2+y_3)\psi (y_2+y_3)^1\psi (y_1+y_3)^1\psi (y_1+y_2)^1\psi (y_1)\psi (y_2)\psi (y_3)=1$$

(in particular, $`\psi (0)=1`$). Again, the fact that $``$ has a canonical cubical structure implies that the value of the left-hand-side is in the trivial $`𝔾_m`$-torsor, and so the identity makes sense.

Moreover, $`\psi `$ is required to satisfy the identity

$$\psi (y_1+y_2)\psi (y_1)^1\psi (y_2)^1=\tau (y_1,\varphi (y_2))=\tau (y_2,\varphi (y_1))$$

on $`({}_{}{}^{t}c\times {}_{}{}^{t}c)^{}\mathrm{\Lambda }(_\eta ^{}{}_{}{}^{1})`$. It follows that for all but finitely many $`y`$ the element $`\psi (y)`$ is a section of $`^1`$ over the whole of $`S`$, and, moreover, for each $`n0`$ all but finitely many $`\psi (y)`$ are 0 modulo $`I^n`$.

The meaning of these degeneration data is as follows: $`\tau (y,x)`$ is canonically an element of $`\mathrm{Iso}(T_{{}_{}{}^{t}c(y)}^{}c(x),c(x))_\eta `$, i.e. an invertible element of

$`\mathrm{Hom}(T_{{}_{}{}^{t}c(y)}^{}c(x),c(x))_\eta =\left(T_{{}_{}{}^{t}c(y)}^{}c(x)_\eta \right)^1|_{\{0\}}c(x)_\eta |_{\{0\}}=`$
$`=\left(T_{{}_{}{}^{t}c(y)}^{}c(x)_\eta \right)^1|_{\{0\}}=𝒫_\eta ^1({}_{}{}^{t}c(y),c(x))`$
(since $`c(x)`$ is linearized at 0), and similarly $`\psi (y)`$ is an element of $`\mathrm{Iso}(T_{c^t(y)}^{},c(\varphi (y)))_\eta ,`$ see \[FC, p.44\] for more details. Therefore, $`\tau `$ describes an embedding $`Y\stackrel{~}{G}_\eta =\mathrm{Spec}_{xX}c(x)`$ which via $`\pi :\stackrel{~}{G}A`$ sits over $`{}_{}{}^{t}c:YA_\eta `$, and $`\psi `$ describes an action of $`Y`$ on the sheaf $`\pi _\eta ^{}_\eta `$. The uniformization means that the abelian variety $`G_\eta `$ can be thought of as the quotient $`\stackrel{~}{G}_\eta /Y`$ and the sheaf $`_\eta `$ as the quotient $`(\pi _\eta ^{}_\eta )/Y`$.

Moreover, by \[FC, Thm.5.1\] any section $`s\mathrm{\Gamma }(G_\eta ,_\eta )`$ can be written as a partial Fourier series $`s=_{xX}\sigma _x(s)`$, with $`\sigma _x(s)\mathrm{\Gamma }(A,c(x))_\eta `$ which satisfy the following identity:

$$\sigma _{x+\varphi (y)}(s)=\psi (y)\tau (y,x)T_{{}_{}{}^{t}c(y)}^{}\sigma _x(s).$$

This makes sense since $`\psi (y)\tau (y,x)`$ is canonically an element of the group $`\mathrm{Iso}(T_{c^t(y)}^{}(c(x)),c(x+\varphi (y)))_\eta .`$

Note also that the degree of the polarization $`\lambda _\eta :G_\eta {}_{}{}^{t}G_{\eta }^{}`$ is the product of the degree of the polarization $`\lambda _A:A{}_{}{}^{t}A`$ and $`|X/Y|`$. Moreover, one can show that there is the following exact sequence:

$$0\mathrm{ker}(\lambda _A)_\eta \mathrm{ker}\lambda _\eta /(Y^{}/X^{})(X/Y)0,$$

where we denote $`X^{}=\mathrm{Hom}(X,)`$ and $`Y^{}=\mathrm{Hom}(Y,)`$.

The degeneration data (d0)-(d4) give corresponding degeneration data for the “central fiber” over $`S_0=\mathrm{Spec}R/I`$. We shall later on describe how to reconstruct the “central fiber” from these data. To begin with the data (d0) - (d2) give corresponding data (D0) - (D2) for an arbitrary base $`S`$. Namely we get

1. An abelian scheme $`A_0/S_0`$ with an ample rigidified sheaf $`_0`$. This sheaf determines a polarization $`\lambda _0:A_0{}_{}{}^{t}A_{0}^{}`$.
2. 1. A semiabelian scheme

    $$1T_0\stackrel{~}{G}_0A_00$$

    with a split torus part, given by a homomorphism $`c_0:X{}_{}{}^{t}A_{0}^{}`$.
    2. A second semiabelian scheme

    $$1{}_{}{}^{t}T_{0}^{}{}_{}{}^{t}\stackrel{~}{G}_{0}^{}{}_{}{}^{t}A_{0}^{}0$$

    with a split torus part, given by a homomorphism $`{}_{}{}^{t}c_{0}^{}:YA_0`$.
3. An inclusion of lattices $`\varphi :YX`$ with a finite cokernel such that $`c_0\varphi =\lambda _0{}_{}{}^{t}c_{0}^{}`$.

We shall now restrict ourselves to the special situation of a one-parameter degeneration. In this case $`R`$ is a complete discrete valuation ring, $`R/I=k`$ is a field (not necessarily algebraically closed) and $`S_0=\mathrm{Spec}k`$ is a one-point scheme. In addition to the pair $`(G_\eta ,_\eta )`$ we shall also pick a section $`\theta _\eta \mathrm{\Gamma }(G_\eta ,_\eta )`$. In other words, we shall look at a pair $`(G_\eta ,\mathrm{\Theta }_\eta )`$, where $`\mathrm{\Theta }_\eta `$ is the Cartier divisor determined by $`\theta _\eta `$. Then by \[A2, Thm.5.7.1\] this pair can, possibly after a base change, be extended uniquely to a *stable semiabelic pair* $`(\stackrel{~}{G}\overline{P}\mathrm{\Theta })`$ over $`S`$. This means in particular, that $`\overline{P}_0`$ is a projective seminormal variety over $`k`$ with a $`G_0`$-action and finitely many orbits, $`\mathrm{\Theta }_0`$ is an ample Cartier divisor on it not containing any orbits entirely, and they satisfy certain conditions listed in\[A2, 1.1\]. (In \[A2\] one starts with a $`G_\eta `$-torsor $`P_\eta `$, but we will take $`P_\eta =G_\eta `$ here for simplicity). The central fiber $`(\overline{P}_0,\mathrm{\Theta }_0)`$ is described by the data (D0)-(D2) and the additional data (D3)-(D6) which we list below, following \[A2\].

Since $`R`$ is now a DVR and the ideal $`I=(t)`$ is principal, we can write the bihomomorphism $`\tau :Y\times X({}_{}{}^{t}c\times c)^{}𝒫_\eta ^1`$ as

$$\tau (y,x)=t^{B(y,x)}\tau ^{}(y,x),$$

where $`B(y,x)`$ is the valuation of $`\tau (y,x)`$. This defines a bilinear form $`B:Y\times X`$ such that $`B|_{Y\times Y}`$ is symmetric and positive definite. Considering $`\tau `$ modulo the ideal $`I`$ we obtain the datum

1. A bihomomorphism $`\tau _0=\tau modI:Y\times X({}_{}{}^{t}c\times c)^{}𝒫_0^1`$ on the central fiber.

Applying the same procedure to the cubical morphism $`\psi `$ we get a quadratic nonhomogeneous function $`A:Y`$ such that $`A(0)=0`$, whose quadratic part is $`\frac{1}{2}B|_{Y\times Y}`$. Taking $`\psi `$ modulo $`I`$ we obtain

1. A cubical morphism $`\psi _0:Y({}_{}{}^{t}c)^{}_0^{}^1`$ on the central fiber.

Finally, applying this to $`\theta _\eta `$ we obtain

1. For each $`xX`$, a section $`\theta _{x,0}\mathrm{\Gamma }(A_0,c(x))`$, such that

$$\theta _{x+\varphi (y),0}=\psi _0(y)\tau _0(y,x)T_{{}_{}{}^{t}c_{0}^{}(y)}^{}\theta _{x,0}.$$

We also get a function $`H:X`$, $`H(x)=\mathrm{val}_t(\theta _x)`$. One easily checks the following: The function $`A:Y`$ extends uniquely to a quadratic nonhomogeneous function $`A:X`$, and $`H(x)=A(x)+r(xmodY)`$, where the last function depends only on the residue of $`x`$ modulo $`Y`$. The function $`H`$ determines the last part of the data:

1. A cell decomposition $`\mathrm{\Delta }`$ of the vector space $`X`$ into polytopes. The vertices of these polytopes are in $`X`$, and it is periodic with respect to the lattice $`Y`$. The construction is as follows. $`H`$ is the “height function”, i.e. one considers the convex hull of the countably many points $`(x,H(x))`$, $`xX`$. Since the leading term of $`H`$ is a positive definite quadratic form (whose first derivative is linear), it is easy to see that the lower envelope of this hull consists of countably many polytopes and that the projection of these polytopes to $`X`$ gives a $`Y`$-periodic decomposition. This is the decomposition $`\mathrm{\Delta }`$.

###### Remark

1. Only the equivalence classes of $`\tau _0,\psi _0,\theta _0`$ in the cohomology groups $`H^1(\mathrm{\Delta },𝕋)`$ and $`Z^1(\widehat{𝕄^{}})`$ matter, see \[A2\] for more details.
2. Changing the sheaf $`_\eta `$ by a translated sheaf (hence, in the same polarization class), and the divisor $`\mathrm{\Theta }_\eta `$ by the corresponding translated divisor, gives an isomorphic pair $`(\overline{P}_0,\mathrm{\Theta }_0)`$ if we do not choose an “origin” in $`\overline{P}_0`$.
3. The function $`\tau _0`$ (resp. $`B`$) plays the role of a 1-cohomology class, and the function $`\psi _0`$ (resp. $`A`$) the role of a 1-cocycle restricting to this 1-cohomology class. So, changing $`\psi _0`$ without changing $`\tau _0`$ leads to an isomorphic pair $`(\stackrel{~}{G}_0\overline{P}_0,\mathrm{\Theta }_0)`$, again, if we do not fix the “origin” in $`\overline{P}_0`$.
4. If the polarization $`\lambda _\eta `$ is principal, then the divisor $`\mathrm{\Theta }_\eta `$ is superfluous, since for any sheaf $`_\eta `$ defining $`\lambda _\eta `$ the section $`\theta _\eta `$ is unique, up to a multiplicative constant.
5. In the case of principal polarization we must have $`Y=X`$, so for the decomposition $`\mathrm{\Delta }`$ the two lattices of vertices and of periods coincide, and the periodic remainder function $`r(xmodY)`$ disappears. In this case, $`\mathrm{\Delta }`$ is a classical Delaunay decomposition for the quadratic form $`B`$. By analogy, we will christen the decompositions appearing in the general case as *semi-Delaunay decompositions*.
6. The whole construction can be repeated in entirely the same manner in the complex-analytic setting. The base $`S`$ then has to be replaced by the germ of a normal analytic space, and the ring $`R`$ by the corresponding ring of convergent power series.

Vice versa, starting from the data (D0)-(D6), one can construct a stable semiabelic variety $`V`$ together with an ample sheaf $`𝒪_V(1)`$ and a global section of this sheaf, according to a general construction of \[A2\]. Since this construction is crucial to us, we briefly recall it.

For each cell $`\delta `$ of the cell decomposition $`\mathrm{\Delta }`$ we consider $`\mathrm{Cone}\delta `$, the cone defined by $`1\times \delta `$ in $`\times X_{}`$. and to each $`\chi =(d,x)𝕏=X`$ in this cone we associate the sheaf $`_\chi =^dc(x)`$, where $`c(x)`$ is the line bundle given by $`c(x){}_{}{}^{t}A`$. The line bundle $`_\chi `$ is rigidified at the origin of $`A`$. Since $`_{\chi _1}_{\chi _2}`$ and $`_{\chi _1+\chi _2}`$ are naturally isomorphic as rigidified sheaves, we have a semigroup algebra of $`𝒪_A`$-invertible sheaves labeled by $`\mathrm{Cone}\delta `$. The variety $`\stackrel{~}{V}_\delta `$ is defined as the $`\mathrm{Proj}`$ over $`A`$ of this algebra. This variety $`\stackrel{~}{V}_\delta `$ has a natural ample sheaf $`𝒪_{\stackrel{~}{V}_\delta }(1)`$ and a projection onto $`A`$. If two cells $`\delta _1`$ and $`\delta _2`$ intersect along $`\delta _{12}`$, then the subvariety $`\stackrel{~}{V}_{\delta _{12}}`$ is contained in both varieties $`\stackrel{~}{V}_{\delta _1}`$ and $`\stackrel{~}{V}_{\delta _2}`$, and we can glue them along it, together with their sheaves. This will be done in such a way that the resulting variety is seminormal. In terms of the $`\mathrm{Proj}`$ of a ring this can be done as follows. Introduce formal variables $`\zeta _\chi `$ to label the sheaves $`_\chi `$ that belong to the cell $`\delta `$. Then define a big semigroup algebra $`R`$, this time labeled by the union of all the cones $`\mathrm{Cone}\mathrm{\Delta }:=_{\delta \mathrm{\Delta }}\mathrm{Cone}\delta `$ (i.e. all $`(d,x)`$ with $`d>0`$ and $`(0,0)`$), and define the multiplication by setting

$$\zeta _{\chi _1}\zeta _{\chi _2}=\zeta _{\chi _1+\chi _2}$$

if $`\chi _1`$, $`\chi _2`$ belong to a cone over a common cell of $`\mathrm{\Delta }`$ (in which case one speaks of cellmates with respect to $`\mathrm{\Delta }`$), and $`0`$ otherwise. Let $`\stackrel{~}{V}=\mathrm{Proj}_AR`$. This scheme carries a natural sheaf $`𝒪_{\stackrel{~}{V}}(1)`$ which restricts to each $`\stackrel{~}{V}_\delta `$ as the sheaf $`𝒪_{\stackrel{~}{V}_\delta }(1)`$.

As we explained above, $`\tau `$ and $`\psi `$ define an action of $`Y`$ on the algebra $`_{(d,x)X}^dc(x)`$. This induces a $`Y`$-action on the algebra $`R`$, and therefore on $`\stackrel{~}{V}=\mathrm{Proj}_AR`$. The variety $`(V,𝒪_V(1))`$ is then defined as the quotient of $`(\stackrel{~}{V},𝒪_{\stackrel{~}{V}}(1))`$ by $`Y`$. This action is properly discontinuous in the Zariski topology and hence $`𝒪_V(1)`$ is ample due to the Nakai-Moishezon or the Kleiman criterion.

Finally, the formal power series $`\theta =\theta _x\zeta ^{(1,x)}`$ defines a section of $`𝒪_V(1)`$ as follows. The formal restriction of $`\theta `$ to the algebra for the cone $`\delta `$ defines a section of $`𝒪_{V_\delta }(1)`$. These sections coincide on the intersections and, by the basic identity in (D5), are compatible with the $`Y`$-action. Therefore, they glue to a global section of $`𝒪_V(1)`$.

For what follows it is important to recall that the variety $`V`$ has the following properties (see \[A2, 1.1\] for more details):

1. $`V`$ is seminormal.
2. $`V`$ is naturally stratified into locally closed strata, and there is a 1-to-1 correspondence between the strata of dimension $`dimA+i`$ and $`i`$-dimensional cells $`\delta `$ of the decomposition $`\mathrm{\Delta }`$ modulo $`Y`$.
3. Maximal cells correspond to irreducible components of $`V`$.

### 2.2 Induced degeneration data

Going back to the general setup of the previous subsection, let us now assume that we have, in addition, a semiabelian variety $`G^{}/S`$ with abelian generic fiber $`G_\eta ^{}`$, and an injective homomorphism $`\phi :G^{}G`$. In this case, the uniformization data for the variety $`(G_\eta ^{},_\eta ^{}=\phi _\eta ^{}(_\eta ))`$ can be read off directly from those of the variety $`G_\eta `$. We would like to write this down explicitly, since this is exactly the situation which we encounter with Jacobians and Prym varieties.

By the functoriality of the varieties $`\stackrel{~}{G},{}_{}{}^{t}\stackrel{~}{G}`$, we get homomorphisms $`\stackrel{~}{G}^{}\stackrel{~}{G}`$, $`{}_{}{}^{t}\stackrel{~}{G}{}_{}{}^{t}\stackrel{~}{G}_{}^{}`$ which are encoded by the following commutative diagrams of group schemes over $`S`$:

The homomorphism $`T^{}T`$ on the torus parts is injective, therefore $`XX^{}`$ is surjective. One has $`Y^{}=Y\stackrel{~}{G}^{}`$, hence $`Y^{}Y`$ is injective. The bihomomorphism $`\tau :Y\times X({}_{}{}^{t}c\times c)^{}𝒫_\eta ^1`$ encodes the embedding $`Y\stackrel{~}{G}_\eta `$, and the same is true for $`\tau ^{}`$. Therefore, $`\tau ^{}`$ is merely the “restriction” of $`\tau `$ in the following sense:

$$\tau ^{}(y^{},x^{})=\tau (y^{},x)\text{for any }xx^{}.$$

This statement makes sense for the following reason: $`\tau (y^{},x)`$ is an element of $`c(x)^1({}_{}{}^{t}c(y^{}))=c(x)^1\left(\phi _A({}_{}{}^{t}c_{}^{}(y^{}))\right)`$, and similarly $`\tau ^{}(y^{},x^{})`$ is an element of $`\phi _A^{}(c(x)^1)\left({}_{}{}^{t}c_{}^{}(y^{})\right)`$. These are canonically isomorphic.

The map $`\varphi ^{}:Y^{}X^{}`$ is the composition $`Y^{}Y\stackrel{\mathit{\varphi }}{}XX^{}`$ and it is injective. Indeed, if $`\varphi ^{}(y^{})=0`$ then $`\tau ^{}(y^{},\varphi (y^{}))=1`$ by bilinearity. On the other hand, $`\tau ^{}(y^{},\varphi ^{}(y^{}))=\tau (y^{},\varphi (y^{}))`$, and for $`y^{}0`$ we get a contradiction to the positivity condition on $`\tau `$.

The sheaf $`^{}`$ is $`\phi _A^{}()`$, and we automatically have $`c^{}\varphi ^{}=\lambda ^{}{}_{}{}^{t}c_{}^{}`$, where $`\lambda ^{}=\lambda (^{}):A^{}{}_{}{}^{t}A_{}^{}`$. The cubical homomorphism $`\psi ^{}`$ is obtained by restriction: $`\psi ^{}(y^{})=\psi (y^{})`$. Again, both maps take values in canonically isomorphic $`𝔾_m`$-torsors.

On the central fiber, the data (D0)-(D2), and in the case of a one parameter degeneration also (D3) and (D4), are obtained by straightforward restriction.

Finally, given a Fourier series $`\theta =_{xX}\theta _x`$ with $`\theta _x\mathrm{\Gamma }(G_\eta ,_\eta )`$, converging in the $`I`$-adic topology, the restriction to $`G_\eta ^{}`$ is given by

$$\theta ^{}=\underset{x^{}X^{}}{}\theta _x^{}^{}=\underset{x^{}X^{}}{}\underset{xX,xx^{}}{}\phi _A^{}(\theta _x).$$

In this expression, each $`\theta _x^{}^{}`$ is itself a power series converging in the $`I`$-adic topology, representing an element of $`\mathrm{\Gamma }(A_\eta ^{},_\eta ^{}c^{}(x^{}))`$. For as long as $`\mathrm{\Theta }_\eta `$ does not contain $`G_\eta ^{}`$, one has $`\theta ^{}0`$, and this can always be achieved by choosing a translated divisor.

###### Lemma 2.1

For a generic choice of a divisor $`\mathrm{\Theta }_\eta `$ in a given polarization class the induced height function is given by $`H^{}(x^{})=\mathrm{min}_{xx^{}}H(x).`$

Proof. Because of the identity connecting $`\theta _x^{}^{}`$ and $`\theta _{x^{}+\varphi ^{}(y^{})}^{}`$ we only have to check this statement for finitely many representatives of elements in $`X^{}/Y^{}`$. For each $`x^{}`$ there are only finitely many $`x`$ mapping to it with the minimal height $`H(x)`$. We are free to change the cocycle $`\psi _0`$ by a linear function on $`Y`$, and we are free to multiply the $`\theta `$’s in different classes modulo $`Y`$ by different constants. Hence, it is easy to arrange that the sections $`\phi _A^{}(\theta _x)`$ of the minimal height do not cancel each other. ∎

Note that in general we only have an inequality $`H^{}(x^{})\mathrm{min}_{xx^{}}H(x)`$.

## 3 Degenerations of Jacobians and principally polarized Prym varieties

### 3.1 Degenerations of Jacobians

Let $`C_0`$ be a stable curve of genus $`g`$ and let $`\widehat{U}`$ be the base of the universal deformation. Then we have a curve $`C/\widehat{U}`$ and a semiabelian family of Jacobians $`JC/\widehat{U}`$, $`JC=\mathrm{Pic}_{C/\widehat{U}}^0`$, with an abelian generic fiber. After a finite base change, one can choose an ample sheaf $``$ on $`JC`$ giving a principal polarization on the generic fiber. We can then apply the general construction outlined above to this family. First we obtain the data (J0)-(J2) which do not depend on the choice of a one-parameter family. The data (J3)-(J6) are *a priori* defined only for a particular choice of a one-parameter degeneration. However, for the Jacobians the limit variety is independent of such a choice, and so the data (J3)-(J6) depend only on the curve $`C_0`$ itself, see \[A1\]. (This will also follow from Theorem 3.2 below). The resulting data are as follows:

1. The abelian variety $`A_0=JN_0`$, where $`N_0=N_i`$ is the normalization of the nodal curve $`C_0`$. This is a principally polarized abelian variety and the sheaf $`_0`$ in this polarization class can be chosen arbitrarily. Alternatively, on the $`A_0`$-torsor $`\mathrm{Pic}^{g_i1}N_i`$ this sheaf is defined canonically.
2. The lattice $`X=H_1(\mathrm{\Gamma },)`$, where $`\mathrm{\Gamma }`$ is the dual graph of $`C_0`$, and the same semiabelian variety $`G_0=JC_0`$ as in the first section, fitting into an exact sequence

$$1H^1(\mathrm{\Gamma },k^{})G_0=JCJN0,$$

plus the dual semiabelian variety $`{}_{}{}^{t}G_{0}^{}=G_0`$.
3. The inclusion $`\varphi :YX`$ is the identity and $`{}_{}{}^{t}c:YA_0`$ is defined as $`\lambda _{A_0}^1c\varphi `$.
4. The equivalence class of the bihomomorphism $`\tau _0:Y\times X({}_{}{}^{t}c\times c)^{}𝒫^1`$ is given by the so called Deligne symbol, a generalization of the double ratio (see \[BM\]).

The bilinear function $`B:X\times X`$ is determined by the monodromy, which is given by the Picard-Lefschetz formula. Explicitly, we have an embedding

$$H_1(\mathrm{\Gamma })C_1(\mathrm{\Gamma })=_{\mathrm{edges}}e_j.$$

Let $`z_j`$, $`j=1\mathrm{}q`$ be the coordinate functions on $`C_1(\mathrm{\Gamma })`$. On $`\widehat{U}`$ there are $`q`$ transversally intersecting divisors $`D_j`$, each corresponding to a node of $`C_0`$. A one-parameter family $`C/S`$ gives rise to a map $`S\widehat{U}`$ and this determines valuations $`\alpha _j>0`$ along the divisors $`D_j`$. The Picard-Lefschetz formula says that the monodromy corresponds to the form $`B=\alpha _jz_j^2`$. The positive integers $`\alpha _j`$ are not determined by the central fiber $`C_0`$ alone, but also depend on the degenerating family.

1. The cell decomposition $`\mathrm{\Delta }`$ defined by $`B`$ does, however, not depend on the constants $`\alpha _j`$, which are determined by the choice of the $`1`$-parameter family. It is obtained by taking the intersections of the standard cubes in $`C_1(\mathrm{\Gamma },)`$ with the subspace $`H_1(\mathrm{\Gamma },)`$. This follows from the simple combinatorial lemma 3.1 below and the well-known fact that the functions $`z_j`$ restricted to $`H_1(\mathrm{\Gamma },)`$ form a unimodular system of vectors. The corresponding statement in graph theory says the following: If $`z_{i_1},\mathrm{},z_{i_n}`$ are linearly independent then $`\mathrm{\Gamma }\{e_{i_1},\mathrm{},e_{i_n}\}`$ is a spanning tree. This spanning tree defines a basis of $`H_1(\mathrm{\Gamma },)`$ with the property $`z_{i_s}(e_{i_t})=\delta _{i_si_t}`$. Therefore $`z_{i_1},\mathrm{},z_{i_n}`$ span $`H_1(\mathrm{\Gamma },)^{}`$ over the integers $``$.

Since we are in the principally polarized case the rest of the data have an auxiliary character, as we have explained in the remarks at the end of subsection 2.1.

###### Lemma 3.1

Let $`X`$ be a lattice and $`l_j:X`$ be linear functions such that the quadratic form $`l_j^2`$ is positive definite. Then the Delaunay decomposition for the quadratic form $`\alpha _jl_j^2`$ is independent of the choice of positive constants $`\alpha _j`$ if and only if the forms $`l_j`$ define a dicing: the $`0`$-skeleton of the cell decomposition given by the hyperplanes $`\{l_j(x)=n\}`$ coincides with the lattice $`X`$.

Proof. It is easy to see that the Delaunay decomposition for the quadratic form $`l_{j_0}^2+\epsilon _{jj_0}l_j^2,0\epsilon 1`$, or, what is the same, for the form $`Nl_{j_0}^2+_{jj_0}l_j^2,N0`$ is a subdecomposition of the Delaunay decomposition for the form $`l_{j_0}^2`$. The latter is, of course, formed by the hyperplanes $`\{l_{j_0}=n;n\}`$. Therefore, if the Delaunay decomposition does not depend on the choice of constants $`\alpha _j>0`$, then it must be a refinement of the intersections of dice $`\{l_j=n_j;n_j\}`$. Hence this intersection cannot have extra vertices, i.e. must be a dicing.

Vice versa, assume that the linear forms $`l_j`$ define a dicing. We claim that it is a Delaunay decomposition for the quadratic form $`q=\alpha _jl_j^2`$ for any $`\alpha _j>0`$. Indeed, let $`\delta `$ be the lattice polytope which is the intersection of sets $`\{n_jl_jn_j+1\}`$ for some $`n_j`$. Then for every vertex of $`\delta `$ one has $`l_j(P)=n_j`$ or $`n_j+1`$ and for any lattice point $`Q`$, which is not a vertex of $`\delta `$, at least one function $`l_j`$ has a different value. This means that the vertices of $`\delta `$ all lie on an ”empty ellipsoid”

$$\alpha _j((l_jn_j1/2)^2(1/2)^2)=0$$

and that all other lattice points lie outside of this ellipsoid. This is Delaunay’s original definition of a Delaunay cell for the form $`q`$. ∎

The data above define the unique “canonical compactified Jacobian” $`(\overline{J}_{g1}C,\mathrm{\Theta }_{g1})`$ of \[A1\], which is a stable semiabelic pair. Moreover, this pair defines a point in the second Voronoi compactification $`A_g^{\mathrm{Vor}}`$ of $`A_g`$. If $`C_0`$ is a stable curve, then this is the image of $`C_0`$ under the Torelli map.

### 3.2 Degenerations of principally polarized Pryms

Let $`C_0`$ be a stable curve with an involution $`\iota _0`$. The involution $`\iota _0`$ gives an involution on the universal deformation space $`\widehat{U}`$ of $`C_0`$, and the fixed locus $`\widehat{W}`$ of this involution, which is regular itself, represents the deformation of $`(C_0,\iota _0)`$ together with an involution. We have a family $`(C,\iota )/\widehat{W}`$ with central fiber $`(C_0,\iota _0)`$ and a smooth generic fiber $`C_\eta `$. Over $`\widehat{W}`$ there are two semiabelian families with abelian generic fibers, namely $`JC`$ and the subfamily $`P(C,\iota )`$. The first one is principally polarized, and that gives the induced polarization on the second family.

We shall apply subsection 2.2 to the subfamily $`P(C,\iota )`$. This gives us first of all degeneration data (p0)-(p2), and for the central fiber the corresponding data (P0)-(P2), which we shall now describe. We shall see that we have encountered most of these data already in the first section.

Taking the connected component of the identity of $`\mathrm{ker}(1+\iota )`$ and specializing to the central fiber commute, hence the semiabelian variety $`\stackrel{~}{G}_0^{}`$ in (P1) is the same as the Prym variety $`P_0`$ of the first section (there denoted by $`P`$), and the abelian variety $`A_0^{}`$ in (P0) is its abelian part. The latter was simply denoted by $`A`$ in the first section. The ample sheaf $`_0^{}`$ is the pullback of $`_0`$ from the Jacobian $`JC_0`$. As in section 1 we denote by $`[X]^+`$, resp. $`[X]^{}`$ the sublattices of $`X=H_1(\mathrm{\Gamma },)`$ corresponding to the $`(\pm 1)`$-eigenspaces. Then $`\stackrel{~}{G}_0^{}`$ is given by the classifying homomorphism $`c^{}:X^{}A_0^{}`$ where $`X^{}`$ is $`X/[X]^+`$. This is the classifying homomorphism which we described in lemma 1.5.

There appears a second lattice in (P1), which we will denote $`Y^{}`$. It is given by $`Y^{}=Y\stackrel{~}{G}^{}`$ in $`\stackrel{~}{G}`$ and we have not yet encountered this in the first section. The lattice $`Y^{}`$ contains $`2X^{}`$, and we have an injective morphism $`Y^{}[X]^{}=[H_1(\mathrm{\Gamma },)]^{}`$ which, however, need not be an isomorphism. In the complex analytic case $`Y^{}`$ is the image of the natural homomorphism $`[H_1(C_t,)]^{}X=H_1(\mathrm{\Gamma },)`$, where $`(C_t,\iota _t)`$ is a topological smoothing of $`(C_0,\iota _0)`$. In particular, $`Y^{}`$ can be computed explicitly in any given concrete example (see also section 5). For curves $`(C_0,\iota _0)`$ which correspond to points in $`\overline{}_g`$ we have $`Y^{}=2X^{}`$ for degree reasons.

In (P2), the inclusion $`\varphi ^{}:Y^{}X^{}`$ is the composition $`Y^{}[X]^{}X^{}`$ and, as in the general case, we have $`c^{}\varphi ^{}=\lambda ^{}{}_{}{}^{t}c_{}^{}`$.

The rest of the data (P4)-(P6) require looking at one-parameter degenerations. To get any kind of uniqueness we must look at the situation when the induced polarization on the Prym variety is twice the principal polarization, so that we can analyze the degeneration of principally polarized Pryms. We shall, therefore, for the rest of this section assume that $`(C_0,\iota _0)`$ is a nodal curve with an involution which is smoothable to a nonsingular curve with a fixed-point-free involution. It is easy to show that in this case $`C_0`$ has no swapping nodes in the terminology of the first section, and that the only points fixed by the involution are the branchwise fixed nodes. If the genus of the quotient curve $`C_0^{}`$ is $`g`$ then the genus of $`C_0`$ must be $`2g1`$. We will denote by $`\overline{}_g`$ the coarse moduli space of admissible covers, or equivalently of pairs $`(C,\iota )`$ as above where $`C`$ is assumed to be stable. It is not hard to show that $`\overline{}_g`$ exists, that it is a compactification of the moduli space of the moduli space $`_g`$ of smooth genus $`g`$ curves with an irreducible étale degree-two cover, and that it is proper and normal.

So, let us fix a one-parameter family $`(C,\iota )/S`$ so that on the generic fiber we have a smooth curve with base point free involution. Our general theory gives us the degeneration data (p0)-(p4) for the abelian variety $`G_\eta ^{}=P(C_\eta ,\iota _\eta )`$ with the polarization $`\lambda _\eta ^{}:G_\eta ^{}{}_{}{}^{t}G_{\eta }^{}`$ and correspondingly degeneration data (P0)-(P4) for the central fiber. In our situation these are degeneration data for a twice principally polarized abelian variety, whereas we are interested in the degeneration data of the principally polarized abelian variety whose polarization is given by $`\lambda _\eta ^{}=2[\lambda /2]_\eta ^{}`$. We shall denote the latter data by (pp), resp. (PP), standing for principally polarized Prym.

We can approach this problem backwards. Assuming that we have data for an ample sheaf $`_\eta `$ on $`G_\eta `$, what are the data for the sheaf $`_\eta ^2`$? We still have the uniformization of $`G_\eta `$ as $`\stackrel{~}{G}_\eta /Y`$. So, the scheme $`\stackrel{~}{G}`$ and the embedding $`Y\stackrel{~}{G}`$ do not change. Therefore, the schemes $`A,{}_{}{}^{t}A`$, homomorphisms $`c`$, $`{}_{}{}^{t}c`$, and the bihomomorphism $`\tau `$ do not change. It is only the cocycle for the sheaf $``$ which is doubled. Which means $`\varphi =2[\varphi /2]`$ and $`\lambda _A=2[\lambda /2]_A`$. Therefore, we get the following data for the central fiber of the principally polarized Prym:

1. The same abelian variety $`A_0^{}`$ as before. The polarization $`[\lambda /2]_0^{}:A_0^{}{}_{}{}^{t}A_{0}^{}`$ is the unique solution to $`\lambda _0^{}=2[\lambda /2]_0^{}`$.
2. The same lattices $`Y^{}`$ and $`X^{}`$ and homomorphisms $`c^{}:X^{}{}_{}{}^{t}A_{0}^{}`$ and $`{}_{}{}^{t}c_{0}^{}:Y^{}A_0`$.
3. $`[\varphi /2]^{}=(1/2)\varphi ^{}:Y^{}X^{}`$. This is now an isomorphism.
4. The same $`\tau _0^{}`$ as in the non-principally polarized case.
5. $`[\psi /2]_0^{}`$ is a solution of the equation $`\psi _{}^{}{}_{0}{}^{}(y)=([\psi /2]_0^{}(y))^2`$. (This is not unique, and may exist only after a finite field extension if the residue field $`k`$ is not algebraically closed.)
6. Since the polarization is now principal, the section is essentially unique and plays an auxiliary role.
7. The cell decomposition, which we will denote by $`\mathrm{\Delta }^{}`$, is the Delaunay decomposition for the bilinear form $`[B/2]^{}:Y^{}\times X^{}`$, $`[B/2]^{}(y,x)=B^{}(y,x)`$.

We have now all the tools to approach the question of the uniqueness of the limit of the principally polarized Pryms. The Prym map $`_gA_{g1}`$, associating to a smooth curve with a base-point-free involution its principally polarized Prym variety, extends to a rational map from $`\overline{}_g`$ to any compactification of $`A_{g1}`$. Of the infinitely many toroidal compactifications one has a particular functorial importance, namely the second Voronoi compactification $`\overline{A}_{g1}^{\mathrm{Vor}}`$. By \[A2\] it is the closure of $`A_{g1}`$ in the moduli space $`\overline{\mathrm{AP}}_{g1}`$ of principally polarized stable semiabelic pairs, and the latter space is projective. In \[FS\] Friedman and Smith have given a series of examples of pairs $`[(C_0,\iota _0)]\overline{}_g`$ near which the Prym map does not extend to a regular morphism to $`\overline{A}_{g1}^{\mathrm{Vor}}`$ (or any other reasonable toroidal compactification). (The preprint version of \[FS\] contains more details than the published version.) In the theorem below, we give the complete answer to the extendibility question. The answer we give is in terms of a combinatorial condition on the dual graph $`(\mathrm{\Gamma },\iota )`$ of the curve $`C_0`$, together with an involution.

Recall that we have $`X^{}XC_1(\mathrm{\Gamma },)`$. As before, for an edge $`e_j`$, let $`z_j`$ be the corresponding coordinate function on $`C_1(\mathrm{\Gamma },)`$. We have three possibilities:

1. $`z_j`$ is identically zero on $`X^{}`$.
2. $`z_j:X^{}`$. This happens if for every simple oriented cycle $`wH_1(\mathrm{\Gamma },)`$ in the graph $`\mathrm{\Gamma }`$ $`\mathrm{mult}_{e_j}w=1`$ implies $`\mathrm{mult}_{e_{\iota (j)}}w=1`$.
3. $`z_j:X^{}\frac{1}{2}`$. This happens if there exists a simple oriented cycle $`wH_1(\mathrm{\Gamma },)`$ with $`\mathrm{mult}_{e_j}w=1`$ but $`\mathrm{mult}_{e_{\iota (j)}}w=0`$.

The first case is immaterial for us. Define $`m_j`$ to be 1 or 2 respectively in the second or third case so that we have an epimorphism from $`X^{}`$ to $``$. The combinatorial condition is

* The linear functions $`m_jz_j`$ define a dicing of the lattice $`X^{}`$.

Geometrically, the condition $`()`$ can be described as follows. Consider all translations of the hyperplanes $`H_j=\{z_j=0\}`$ in $`C_1(\mathrm{\Gamma },)`$ through points in $`X^{}`$ and take the intersection with $`X_{}^{}`$. This defines a cell decomposition and $`()`$ is fulfilled if the vertices of this cell decomposition are exactly the points of the lattice $`X^{}`$. In concrete examples this condition can be checked easily (see section 5 for a number of worked out examples). Vologodsky \[V\] has shown that a curve $`(C,\iota )`$ fails to fulfill condition $`()`$ if and only if it is the limit of curves of Friedman-Smith type with at least $`4`$ nodes. Here Friedman-Smith type means that $`C`$ has two smooth irreducible components $`C_1`$ and $`C_2`$ which are fixed under the involution $`\iota `$ and that, moreover, the nodes $`C_1C_2`$ of $`C`$ are pairwise interchanged. We are now ready to formulate the main application of our theory.

###### Theorem 3.2

The following are equivalent:

1. In a neighborhood of the point $`0=[(C_0,\iota _0)]`$ the Prym map $`_gA_{g1}`$ extends to a morphism from $`\overline{}_g`$ to the second Voronoi toroidal compactification $`\overline{A}_{g1}^{\mathrm{Vor}}`$.
2. The limit variety $`(\overline{P}_0,\mathrm{\Theta }_0)`$ depends only on the pair $`(C_0,\iota _0)`$ and not on the choice of a one-parameter degeneration.
3. The cell decomposition $`\mathrm{\Delta }^{}`$ depends only on the pair $`(C_0,\iota _0)`$ and not on the choice of a one-parameter degeneration.
4. Condition (\*) holds. In this case, the decomposition $`\mathrm{\Delta }^{}`$ coincides with the dicing.

Proof. By the properness of $`\overline{\mathrm{AP}}_{g1}`$, for every morphism $`(S,0)(\overline{}_g,0)`$ from a regular one-dimensional $`S`$ with $`(S0)_g`$, there exists a unique morphism $`S\overline{A}_{g1}^{\mathrm{Vor}}`$. Therefore, (1) implies (2), and clearly (2) implies (3). The equivalence of (3) and (4) follows by Lemma 3.1. So, we only have to show that (3) implies (1).

Let $`Z\overline{A}_{g1}^{\mathrm{Vor}}`$ be the “image” of the point $`[(C_0,\iota _0)]`$ under the rational map $`\overline{}_g\overline{A}_{g1}^{\mathrm{Vor}}`$. In other words, if $`p:T\overline{}_g`$, $`q:T\overline{A}_{g1}^{\mathrm{Vor}}`$ is a resolution of this rational map, with $`p`$ proper and birational and $`q`$ proper, then $`Z=q(p^1(0))`$. $`Z`$ is closed, and points of $`Z`$ correspond to all possible limits of Pryms for all one-parameter families $`(S,0)(\overline{}_g,0)`$. Since $`\overline{}_g`$ is normal, the Zariski Main Theorem implies that $`Z`$ is connected and that the rational map in question is regular in a neighborhood of $`[(C_0,\iota _0)]`$ if and only if $`Z`$ is a point.

The structure of any toroidal compactification of $`A_{g1}`$ is as follows. There is a natural stratification such that each stratum is fibered over the moduli of the data (D0)-(D2), and each fiber is a quotient of a torus by a finite group. In particular, it is affine. In addition, the strata of the second Voronoi compactification correspond to Delaunay decompositions modulo $`\mathrm{GL}(g1,)`$. Since the data (D0)-(D2) depend only on $`(C_0,\iota _0)`$ the image of $`Z`$ lies in the union of affine sets as above. If the cell decomposition $`\mathrm{\Delta }^{}`$ is unique, then there is only one such affine set. Since $`Z`$ is proper and connected it must then be a point and hence the Prym map is regular at the point $`0=[(C_0,\iota _0)]`$. ∎

###### Remark

In order to make the Prym map regular, it is necessary to blow up $`\overline{}_g`$. This means that we have infinitely many possible limits of Prym at the points of indeterminacy. For a description of all possible limits in the simplest case see the example in 5.2.3.

## 4 Degeneration of Pryms with the induced polarization

Restricting the theta divisor of the Jacobian to the Prym variety defines a polarization which is usually not a principal polarization. In this section we shall study possible limits of Prym varieties with the induced, i.e. in most cases non-principal, polarization. Since we now work with the induced polarization we should expect some relationship with the compactified Jacobian. This we will investigate in the second part of this section.

### 4.1 Construction of the “middle” compactified Prym

Each family of nodal curves with an involution gives us induced data (p0)-(p4). The problem is that there is no canonical choice of a section $`\theta _\eta `$ and correspondingly we have no canonical choice for a cell decomposition for the central fiber. One obtains a natural candidate for such a cell decomposition by taking the cell decomposition $`\mathrm{\Delta }`$ for the Jacobian which is given by (J6) and then intersecting it with $`X_{}^{}`$. As we shall see, this leads indeed to a degeneration of smooth Prym varieties. In view of the construction of the cell decomposition, which is obtained by slicing with $`X_{}^{}`$ (one could also consider slicing with translates of this subspace), we shall refer to this as the “middle” Prym.

Since degenerating families will not appear in the rest of this section, we will suppress the subscript <sub>0</sub> from now on. Our first aim is to describe data (PM0)-(PM6) which define this middle Prym. Throughout we will use the letter “m” in our notation in order to indicate “middle”.

For (PM0)-(PM2) we take the data (P0)-(P2) which, we recall, are universal, and depend only on the pair $`(C,\iota )`$. We have already explained our choice of decomposition, namely we choose

1. Define $`[\mathrm{\Delta }]^{}`$ to be the cell decomposition obtained by intersecting $`\mathrm{\Delta }`$ with the subspace $`X_{}^{}`$.

Clearly, the cells of $`[\mathrm{\Delta }]^{}`$ are intersections with $`X_{}^{}`$ of cells $`\delta `$ of $`\mathrm{\Delta }`$ such that $`\iota (\delta )=\delta `$. As a consequence, the 0-skeleton of $`[\mathrm{\Delta }]^{}`$ is contained in the lattice $`X^{}`$. In addition, $`[\mathrm{\Delta }]^{}`$ is invariant with respect to the lattice $`[X]^{}=H_1(\mathrm{\Gamma },)H_1(\mathrm{\Gamma },)^{}`$. Note that $`Y^{}[X]^{}`$ and hence $`[\mathrm{\Delta }]^{}`$ is also $`Y^{}`$-invariant.

1. For the bihomomorphism $`\tau _m^{}:Y^{}\times X^{}({}_{}{}^{t}c_{}^{}\times c^{})^{}𝒫_{A^{}\times {}_{}{}^{t}A_{}^{}}^1`$ we take the restriction of the bihomomorphism $`\tau :Y\times X({}_{}{}^{t}c\times c)^{}𝒫_{A\times {}_{}{}^{t}A}^1`$ from the datum (J3) for the Jacobian $`\overline{J}C`$.
2. We also define the cubical morphism $`\psi _m^{}`$ to be the restriction of the morphism $`\psi `$ in (J4).

It is important to note that, since $`[\mathrm{\Delta }]^{}`$ is a subdivision of $`\mathrm{\Delta }`$, the $`\tau `$’s and $`\psi `$’s in the same equivalence class restrict to equivalent $`\tau _m^{}`$’s and $`\psi _m^{}`$’s.

We will not choose a particular divisor on our middle Prym, so the datum (PM5) is not needed.

The above data define a stable semiabelic variety $`[\overline{P}]`$ together with an ample sheaf $`[]^{}`$ according to the construction we reviewed in subsection 2.1. It is not a priori clear that this a degeneration of Prym varieties. This, however, follows from

###### Theorem 4.1

Let $`(𝒞,i)/S`$ be any one-parameter family of curves with involution whose generic fiber is smooth and with special fiber isomorphic to $`(C,\iota )`$. Then, possibly after a finite base change $`S^{}S`$, one can choose a divisor $`\mathrm{\Theta }_\eta `$ on the generic fiber in the same polarization class as induced by the Jacobian, so that the limit stable semiabelic variety of the family $`(𝒞_\eta ,\mathcal{\Theta }_\eta )`$ is the middle Prym $`[\overline{P}]`$ defined by $`(𝒞,i)/S`$.

Proof. We first look at the degeneration of Jacobians. We are free to adjust the function $`A=\mathrm{val}_t\psi `$ by any linear function on $`Y=X`$ without changing the polarization class. After a finite base change of degree $`2`$ (whose effect is multiplying $`A`$ and $`B`$ by 2), we can assume that $`A^{}(y)=B^{}(\varphi ^{}(y),y)/2`$. Now consider the convex hull of the points $`(x,H(x))`$, $`xX`$. It is symmetric with respect to the involution $`\iota `$. If we intersect the lower envelope of this hull with the vector space $`X_{}^{}`$, we will get a height function $`H^{\prime \prime }`$ on $`X^{}`$ which gives the decomposition $`[\mathrm{\Delta }]^{}`$. By Lemma 2.1 it will coincide with an induced height $`H^{}`$ for a generic choice of the equation $`\theta `$. ∎

In section 2.1 (see remark 5 after (D6)) we had introduced the notion of semi-Delaunay decompositions. These are the decompositions which play for degenerations with a non-principal polarization the role which in the case of principal polarizations are played by Delaunay decompositions. Our above theorem, therefore, has the following consequence

###### Corollary 4.2

The decomposition $`[\mathrm{\Delta }]^{}`$ is semi-Delaunay.

If $`(C,\iota )`$ is a nodal curve which can be smoothed to a curve with an involution which has at most $`2`$ fixed points, then the Prym of the deformation carries twice a principal polarization. In this case we have two possible degenerations of Prym varieties, namely the principally polarized Prym variety $`\overline{P}`$ and the middle Prym variety $`[\overline{P}]`$ with the induced degeneration. In good cases the limit $`\overline{P}`$ will be unique and isomorphic to $`[\overline{P}]`$ with the only difference that $`[\overline{P}]`$ carries twice the polarization of $`\overline{P}`$. The following result characterizes when this happens

###### Proposition 4.3

Let $`(C,\iota )`$ be a nodal curve which can be smoothed to a curve with an involution with at most $`2`$ fixed points. Then the following conditions are equivalent:

1. The middle Prym variety $`[\overline{P}]`$ is as a stable semiabelic variety isomorphic to $`\overline{P}`$ and carries twice its polarization
2. The following condition is fulfilled:

* $`[\mathrm{\Delta }]^{}`$ is a dicing with respect to the lattice $`2X^{}`$.

Proof. By the assumption that $`(C,\iota )`$ can be smoothed to a curve $`C`$ with an involution with at most $`2`$ fixed points we have the equality of lattices $`Y^{}=[X]^{}=2X^{}`$. Clearly the second condition is equivalent to $`[\mathrm{\Delta }]^{}=2\mathrm{\Delta }^{}`$. On the other hand, following the discussion of conditions (PP0-PP6) in section 3.2, the principally polarized Prym $`\overline{P}`$ with twice the principal polarization is given by the cell decomposition $`2\mathrm{\Delta }^{}`$ and $`Y^{}=2X^{}`$. Moreover, by construction the bilinear forms $`\tau _m^{}`$ and $`\tau |_{2X^{}\times X^{}}`$ coincide. Hence, $`[\overline{P}]\overline{P}`$ if and only if $`[\mathrm{\Delta }]^{}=2\mathrm{\Delta }^{}`$. ∎

###### Remarks

1. Obviously condition (\**) is stronger than (\*). For a discussion of an example where (\*) holds, but condition (\**) fails, see example 5.2.4.
2. For any given nodal curve $`(C,\iota )`$ it is easy to verify whether (\**) is fulfilled.
3. In the appendix Vologodsky \[V\] shows that a stable curve $`(C,\iota )`$ fails condition (\**) if and only if it is a degeneration of a Friedman-Smith type curves with at least 2 nodes.

### 4.2 Relation with the compactified Jacobian

The last question which we would like to address is the relationship between the variety $`[\overline{P}]`$ and $`\overline{J}C`$, as motivated by the smooth case.

We start with some remarks concerning maps of toric varieties. Let $`\delta `$ be a polytope in $`\mathrm{\Delta }`$ and consider the lattice polytope $`\delta ^{}=\pi ^{}(\delta )`$ in the lattice $`X^{}=\pi ^{}(X)`$. Then we have two associated algebras:

1. The $`𝒪_A`$-algebra $`R=R_\delta `$ which is the subalgebra in $`_{(d,x)X}^dc(x)`$ corresponding to the lattice points in the cone over the polytope $`\delta (1,X)`$ (here $`A=JN`$, the Jacobian of the normalization of $`C`$). This algebra is graded by the semigroup $`\mathrm{Cone}\delta X`$.
2. The analogous $`𝒪_A^{}`$-algebra $`R^{}=R_\delta ^{}`$.

Since the morphism $`A^{}A`$ is finite (recall from section 1.1 that its kernel is $`\mathrm{im}(/2)^k`$, where $`k`$ is the number of $`(+1)`$-blocks in the decomposition of the involution $`\iota `$ on $`X`$), $`A^{}`$ is affine over $`A`$. As an $`𝒪_A`$-algebra, $`R^{}`$ is graded by the semigroup $`\mathrm{Cone}\delta ^{}\mathrm{im}(/2)^kX^{}\mathrm{im}(/2)^k`$. The homomorphism $`RR^{}`$ is a homomorphism of graded algebras, and the homomorphism $`XX^{}\mathrm{im}(/2)^k`$ comes from equation (2) after Lemma 1.4. For both gradings we have an invertible $`𝒪_A`$, resp. $`𝒪_{\mathrm{im}A^{}}`$-module ($`\mathrm{im}A^{}=K_N`$ as in the first section) in every homogeneous degree.

###### Lemma 4.4

1. $`\mathrm{Spec}R^{}\mathrm{Spec}R`$ is a closed embedding if and only if the semigroup homomorphism $`\mathrm{Cone}\delta \mathrm{Cone}\delta ^{}\mathrm{im}(/2)^k`$ is surjective.
2. $`V_\delta ^{}=\mathrm{Proj}R^{}V_\delta =\mathrm{Proj}R`$ is a closed embedding if and only if the image of $`\mathrm{Cone}\delta \mathrm{Cone}\delta ^{}\mathrm{im}(/2)^k`$ gives everything in high enough degrees $`dd_0`$.
3. The morphism from the main, i.e. the dense, stratum of $`V_\delta ^{}`$ to the main stratum of $`V_\delta `$ is injective if and only if the homomorphism $`\delta X(\delta ^{}X^{})\mathrm{im}(/2)^k`$ of abelian groups is surjective.
4. the morphism $`V_\delta ^{}V_\delta `$ is finite.

Proof. The first three statements are applications of basic facts about $`\mathrm{Proj}`$’s of graded algebras and toric varieties. The fourth one follows since, in any case, $`\delta \delta ^{}`$ is a surjective map of polytopes. Hence the saturation of the image of the corresponding homomorphism of semigroups is everything. Thus the algebra $`R^{}`$ is finite over $`R`$, and the morphism of $`\mathrm{Proj}`$’s is well defined and finite. ∎

For every polytope $`\delta ^{}`$ in $`[\mathrm{\Delta }]^{}`$ there exists a unique minimal polytope $`\delta `$ in $`\mathrm{\Delta }`$ such that $`\delta ^{}=\delta X_{}^{}`$. Under the involution $`\iota `$ the polytope $`\delta ^{}`$ and the whole decomposition $`\mathrm{\Delta }`$ map to themselves. Therefore, the polytope $`\delta `$ has to be invariant under the involution as well. This implies that under $`\pi ^{}:X_{}X_{}^{}`$ the cell $`\delta `$ maps surjectively to $`\delta ^{}`$, and so we are in the situation of the above lemma. Putting all polytopes together and applying the compatible actions of the lattices $`Y`$ and $`Y^{}`$ now leads to a finite morphism from $`[\overline{P}]`$ to $`\overline{J}C=\overline{J}_{g1}C`$. Already in simple examples, however, it turns out that this morphism in not optimal. The basic example is the situation considered by Beauville \[B\] (see also Example 5.1 in the next section) when the Prym variety is in fact abelian. Without a shift we get the finite morphism $`PK_N`$ from section 1, whose degree is some power of 2, and which need not be an embedding. In order to discuss the optimal choice for the shift $`v`$ we start with the following

###### Lemma 4.5

Let $`v`$ be any vector in $`(1/2)(X[C_1(\mathrm{\Gamma },)]^+)`$. Then the cell decompositions $`[\mathrm{\Delta }]^{}+v`$ and $`\mathrm{\Delta }(X_{}^{}+v)`$ coincide.

Proof. Let $`uH_1(\mathrm{\Gamma },)C_1(\mathrm{\Gamma },)`$ be an arbitrary vector. Write it as $`u_je_j`$ using the standard basis in $`C_1(\mathrm{\Gamma },)`$, labeled by the edges $`e_j`$. Let us first determine the unique cell $`\delta (u)`$ in $`\mathrm{\Delta }`$ which contains $`u`$ in its relative interior. After shifting $`u`$ by an element of $`H_1(\mathrm{\Gamma },)`$ we can assume that $`1<u_j<1`$ for all $`j`$. Then $`u`$ lies in the relative interior of a standard Euclidean cube in $`C_1(\mathrm{\Gamma },)`$ determined by a system of $`|J|`$ equalities or inequalities which are: for each $`j`$, $`z_j=0`$ if $`u_j=0`$, $`1z_j0`$ if $`u_j`$ is negative, and $`0z_j1`$ if $`u_j`$ is positive. Obviously, the cell $`\delta (u)`$ is the intersection of this cube with $`H_1(\mathrm{\Gamma },)`$. Denote by $`\mathrm{supp}(u)`$ the collection of edges $`e_j`$ with $`u_j0`$, and by $`\mathrm{\Gamma }(u)`$ the spanning subgraph of $`\mathrm{\Gamma }`$ which has these edges. Then it is easy to see that the cell $`\delta (u)`$ spans the sublattice $`H_1(\mathrm{\Gamma }(u),)H_1(\mathrm{\Gamma },)`$.

Now fix the vector $`v`$. The two decompositions of our lemma coincide if and only if for every $`uX_{}^{}`$ the negative parts of $`H_1(\mathrm{\Gamma }(u))`$ and $`H_1\left(\mathrm{\Gamma }(u+v)\right)`$ coincide. But this is clear: for every $`e_s`$ appearing in $`v=v_je_j`$ with a non-zero coefficient, one has $`\iota (e_s)=e_s`$ and such an edge does not appear in the support of any $`u`$ with $`\iota (u)=u`$. ∎

The cubical morphism $`\psi _m^{}`$ is well suited for the inclusion $`Y^{}Y`$ but not for the inclusion shifted by $`v`$. Since $`v`$ is half-integral, we either have to make a choice out of $`2^{dimX^{}}`$ possibilities or work with the sheaf $`^2`$. For the latter purpose, we define

1. the cubical morphism $`\psi _{m,2}^{}:Y^{}({}_{}{}^{t}c_{}^{})^{}^2`$ is given by the formula $`\psi _{m,2}^{}(y^{})=\psi _m^2(y^{})\tau (y^{},2v).`$

Note that this choice only affects the ample sheaf on $`[\overline{P}]`$, but not the variety itself.

###### Theorem 4.6

For any vector in $`(1/2)(X[C_1(\mathrm{\Gamma },)]^+)`$ there exists a finite morphism $`f_v`$ from the “middle” Prym variety $`[\overline{P}]`$ to the canonically compactified Jacobian $`\overline{J}C`$.

Proof. Take a cell $`\delta ^{}`$ of $`[\mathrm{\Delta }]^{}`$ and let $`\delta `$ be the minimal cell in $`\mathrm{\Delta }`$ containing $`\delta ^{}+v`$. As before, we see that under $`\pi ^{}`$ the polytope $`\delta `$ surjects to $`\delta ^{}`$. Therefore, we have a finite morphism $`V_\delta ^{}V_\delta `$. By construction, these morphisms coincide on the intersections of cells, and so glue to a finite morphism $`\stackrel{~}{V}^{}\stackrel{~}{V}`$. The functions $`\tau ,\psi ^2`$ define the $`Y`$-action on $`(\stackrel{~}{V},𝒪(2))`$ and, in particular, the action by any subgroup $`Y^{}Y`$ such that $`Y^{}[X]^{}=Y^{}`$. Similarly, the functions $`\tau _m^{},\psi _{m,2}^{}`$ define the $`Y^{}`$-action on $`(\stackrel{~}{V},𝒪(2))`$. Since these functions were chosen to be compatible, we have a finite morphism on the quotients $`V^{}=\stackrel{~}{V}^{}/Y^{}\stackrel{~}{V}/Y^{}`$. This gives a finite map from $`[\overline{P}]`$ to a $`[X]^{}/Y^{}`$-Galois cover of $`\overline{J}C`$. Dividing by $`[X]^{}/Y^{}`$, we get the required finite morphism. ∎

Of course, two half-integral shifts that differ by an integral vector give the same morphism $`f_v`$. As we said before, the basic motivation for making a shift $`v`$ is the situation considered in \[B\] where the Prym variety is in fact abelian and where, without a shift, we will in general only obtain a finite map, but not an embedding. In this case we obtain an embedding if we shift by a vector $`v=_{\text{all }j}e_j/2`$. With this in mind, we give the following

###### Definition

We call a vector in $`(1/2)(X[C_1(\mathrm{\Gamma },)]^+)`$ a *maximal half-shift* if the support of $`v`$ consists of all edges $`e_j`$ with $`\iota (e_j)=e_j`$, i.e. if it spans $`[C_1(\mathrm{\Gamma },)]^+`$.

###### Lemma 4.7

Assume that the only fixed points of the involution on $`C`$ are branchwise fixed nodes. Then a maximal half-shift exists, and it is unique up to an element of $`H_1(\mathrm{\Gamma },)`$.

Proof. Under our assumption the fixed points of the involution on the normalization $`N`$ are precisely the preimages of the branchwise fixed nodes. On the graph $`\mathrm{\Gamma }`$ these nodes correspond to edges $`e_j`$ which are fixed under $`\iota `$. Let $`\mathrm{\Gamma }^+`$ be the spanning subgraph of $`\mathrm{\Gamma }`$ which has these nodes. Since the number of fixed points of an involution on a smooth curve is even, the degrees of the vertices in $`\mathrm{\Gamma }^+`$ are all even. By a basic fact in graph theory there exists an oriented eulerian cycle $`e_j`$, and half of it gives us a maximal half-shift. Moreover, two such cycles obviously differ by an even integral vector in $`H_1(\mathrm{\Gamma },)`$, so half of it will be integral. ∎

As a consequence of condition (3) of Lemma 4.4 we obtain

###### Proposition 4.8

Assume the map $`X[C_1(\mathrm{\Gamma },)]^+\mathrm{im}(/2)^k`$ to be surjective. Then the morphism $`f_v`$ for a maximal half-shift $`v`$ is generically injective.

In general, the question of when the morphism $`f_v`$, in particular for a maximal half-shift, is an embedding, appears to be combinatorially quite involved, and is deserving of a separate study. We leave this question for another place and time.

## 5 Examples

In this section we illustrate our theory by discussing degenerations of Prym varieties in several examples. We shall mostly concentrate on a description of the combinatorial data, i.e. the lattices $`Y^{},X^{}`$ and $`[X]^{}`$ and the cell decompositions $`\mathrm{\Delta }^{}`$ and $`[\mathrm{\Delta }]^{}`$. This gives the combinatorial structure of the toric part of the degenerate Prym variety.

The examples 5.1 to 5.2.4 all correspoind to points in $`\overline{}_g`$ (at least if there are no smooth fixed ponts). The examples 5.2.1 and 5.2.2 are optimally behaved in the sense that condition (\**), and hence also (\*), is fulfilled. These examples demonstrate different behaviour of the toric parts. Example 5.2.3 explains the Friedman-Smith examples and example 5.2.4 is the easiest example in $`\overline{}_g`$ where (\*) holds, but (\**) fails. Here we explain the relationship between the principally polarized and the middle Prym. The remaining examples treat mixed cases.

### 5.1 Beauville type examples

Let $`C`$ be a nodal curve with an involution $`\iota `$ which has only branchwise fixed nodes. In particular $`\iota `$ maps every component $`C_i`$ of $`C`$ to itself and the involution $`\iota `$ acts as the identity on $`C_1(\mathrm{\Gamma },)`$ and hence also on $`H_1(\mathrm{\Gamma },)`$. In this case $`X^{}=Y^{}=\{0\}`$ and the Prym variety is an abelian variety. If there are no smooth fixed points, then these are the principal examples treated by Beauville in \[B\]. The presence of smooth fixed points does not change $`X^{}=Y^{}=\{0\}`$, but if we have more than 2 smooth fixed points the polarization on the Prym variety will no longer be twice a principal polarization.

### 5.2 Non-fixed nodes only

#### 5.2.1  Case of irreducible $`C`$

Let $`C`$ be an irreducible nodal curve with an even number of double points $`Q_1,\mathrm{},Q_{2a}`$ which are pairwise exchanged by $`\iota `$, say $`\iota (Q_j)=Q_{2aj+1}`$. The graph $`\mathrm{\Gamma }(C)`$ has one vertex $`v`$ and $`2a`$ loops $`e_1,\mathrm{},e_{2a}`$ around $`v`$.

In this case the involution $`\iota `$ acts on $`H_1(\mathrm{\Gamma },)=C_1(\mathrm{\Gamma },)`$ by $`\iota (e_j)=e_{2aj+1}`$ and is, therefore, with respect to the basis $`e_1,e_{2a},e_2,e_{2a1},\mathrm{},e_a,e_{a+1}`$ given by the matrix

$$\left(\begin{array}{cccc}\left(\begin{array}{cc}0& 1\\ 1& 0\end{array}\right)& & & \\ & \left(\begin{array}{cc}0& 1\\ 1& 0\end{array}\right)& & \\ & & \mathrm{}& \\ & & & \left(\begin{array}{cc}0& 1\\ 1& 0\end{array}\right)\end{array}\right).$$

The decomposition $`\mathrm{\Delta }`$ of $`H_1(\mathrm{\Gamma },)=C_1(\mathrm{\Gamma },)`$ is given by the standard cubes. In this case we have

$$[H_1(\mathrm{\Gamma },)]^{}=[X]^{}=e_1e_{2a},\mathrm{},e_ae_{a+1}=2X^{}.$$

The induced decomposition $`[\mathrm{\Delta }]^{}`$ consists also of standard cubes, i.e. the cube spanned by $`e_1e_{2a},\mathrm{},e_ae_{a+1}`$ its faces and the $`2X^{}`$-translates of these cells. It is Delaunay with respect to $`2X^{}`$. This implies that condition $`()`$, and hence also condition $`()`$, is fulfilled. Since the smoothing of non-fixed nodes does not introduce fixed points on $`C_t`$, we have already for degree reasons that $`Y^{}=2X^{}=[X]^{}`$. The decomposition $`\mathrm{\Delta }^{}`$ consists of cubes with vertices in $`X^{}`$ and we have $`[\mathrm{\Delta }]^{}=2\mathrm{\Delta }^{}`$. As semiabelic varieties (without the polarization) $`\overline{P}=[\overline{P}]`$ and if there is no abelian part this variety is isomorphic to $`(^1)^a`$ where “opposite”sides are glued and the gluing is defined by the form $`\tau ^{}`$. If there is an abelian part, then the normalization of $`\overline{P}=[\overline{P}]`$ is a $`(^1)^a`$-bundle over the abelian part.

#### 5.2.2 Two exchanged components

Assume that $`C=C_1C_2`$ with $`C_1`$ and $`C_2`$ intersecting in $`Q_1,\mathrm{},Q_{2a}`$. The involution $`\iota `$ interchanges $`C_1`$ and $`C_2`$ and acts on the nodes by $`\iota (Q_j)=Q_{2a+1j}`$. Then the graph $`\mathrm{\Gamma }(C)`$ has 2 vertices and $`2a`$ edges and looks as follows

The involution $`\iota `$ acts by $`\iota (e_j)=e_{2a+1j}`$ and the following elements define a basis of $`H_1(\mathrm{\Gamma },)`$:

$$\begin{array}{c}h_1=e_1e_{2a},\hfill \\ h_{2m}=e_me_{2am};m=1,\mathrm{},a1\hfill \\ h_{2m+1}=e_{m+1}e_{2am+1};m=1,\mathrm{},a1.\hfill \end{array}$$

With respect to this basis the involution $`\iota `$ is given by the matrix

$$\iota =\left(\begin{array}{cccc}1& & & \\ & \left(\begin{array}{cc}0& 1\\ 1& 0\end{array}\right)& & \\ & & \mathrm{}& \\ & & & \left(\begin{array}{cc}0& 1\\ 1& 0\end{array}\right)\end{array}\right).$$

We find that

$$[H_1(\mathrm{\Gamma },)]^{}=[X]^{}=h_2h_3,\mathrm{},h_{2a2}h_{2a1}=2X^{}.$$

If we set

$$l_1=\frac{1}{2}(h_2h_3),l_2=\frac{1}{2}(h_4h_5),\mathrm{},l_{a1}=\frac{1}{2}(h_{2a2}h_{2a1}),$$

then

$$X^{}=l_1,\mathrm{},l_{a1}.$$

Since non-fixed nodes do not contribute to fixed points on $`C_t`$ we find that $`Y^{}=2X^{}`$ in all of these cases. In case $`a=1`$ we have $`X^{}=\{0\}`$. If $`a=2`$ we find the following picture

In this case $`[\mathrm{\Delta }]^{}`$ is the unique Delaunay decomposition with respect to the rank 1 lattice $`2X^{}`$ and similarly $`\mathrm{\Delta }^{}`$ is the unique Delaunay decomposition for $`X^{}`$. As semiabelic varieties $`\overline{P}=[\overline{P}]`$ and this is a nodal curve (with a polarization of degree $`1`$, resp. $`2`$).

Let $`a=3`$. Then $`X^{}=l_1,l_2`$ and the decomposition $`[\mathrm{\Delta }]^{}`$ of $`X_{}^{}`$ looks as follows (the broken lines indicate $`\mathrm{\Delta }^{}`$)

Again $`[\mathrm{\Delta }]^{}=2\mathrm{\Delta }^{}`$ and both decompositions are Delaunay with respect to $`2X^{}`$, resp. $`X^{}`$. If the curves $`C_1`$ and $`C_2`$ are rational we obtain $`2`$ copies of $`^2`$ each with a polarization of degree $`2`$, resp. $`1`$. Note that the $`2`$ projective planes are glued in exactly the same way as in the case of the compactified Jacobian of the dollar sign curve of genus $`2`$. Here condition (\**), and hence also (\*), is fulfilled.

For general $`a`$ let $`y_1,\mathrm{},y_{a1}`$ be the dual coordinates with respect to $`l_1,\mathrm{},l_{a1}`$. Then $`[\mathrm{\Delta }]^{}`$ is given by

$$\begin{array}{cc}y_i=c2;\hfill & i=1,\mathrm{},a1,\hfill \\ y_iy_{i+1}=d2;\hfill & i=1,\mathrm{},a2.\hfill \end{array}$$

This gives a Delaunay decomposition for $`2X^{}`$ and, in particular, condition (\**), and thus also (\*), is fulfilled. The building blocks of the toric part are projective spaces $`^{a1}`$.

#### 5.2.3 The Friedman-Smith examples

Let $`C=C_1C_2`$ consist of two irreducible components intersecting in an even number of nodes $`Q_1,\mathrm{},Q_{2a}`$. Assume that $`C_1`$ and $`C_2`$ are fixed by the involution $`\iota `$ which, however, interchanges the nodes pairwise, i.e. $`\iota (Q_j)=Q_{2aj+1}`$. This situation can, for example, be realized as follows. Let $`C_1`$ and $`C_2`$ be two elliptic curves and choose non-zero 2-torsion points $`\tau _1,\tau _2`$. Choose general points $`R_1,\mathrm{},R_a`$ on $`C_1`$ and $`S_1,\mathrm{},S_a`$ an $`C_2`$. Next identify the points $`R_k`$ and $`S_k`$ for $`k=1,\mathrm{},a`$ as well as $`R_k+\tau _1`$ and $`S_k+\tau _2`$ for $`k=1,\mathrm{},a`$. (We choose the points $`R_k`$ such that $`R_{k_1}+\tau _1R_{k_2}`$ for all $`k_1,k_2`$ and similarly for the points $`S_k`$). Then the involutions $`xx+\tau _i;i=1,2`$ on $`C_i`$ define an involution $`\iota `$ on $`C`$ which has the required properties.

As before the graph $`\mathrm{\Gamma }(C)`$ has 2 vertices and $`2a`$ edges

The difference is that this time $`\iota (e_j)=e_{2aj+1}`$. We can define a basis of $`H_1(\mathrm{\Gamma },)`$ by setting

$$\begin{array}{c}h_1=e_1e_{2a},\hfill \\ h_{2m}=e_me_{2am};m=1,\mathrm{},a1\hfill \\ h_{2m+1}=e_{2am+1}e_{m+1},m=1,\mathrm{},a1.\hfill \end{array}$$

With respect to this basis the involution $`\iota `$ is given by the matrix

$$\iota =\left(\begin{array}{cccc}1& & & \\ & \left(\begin{array}{cc}0& 1\\ 1& 0\end{array}\right)& & \\ & & \mathrm{}& \\ & & & \left(\begin{array}{cc}0& 1\\ 1& 0\end{array}\right)\end{array}\right).$$

To describe the lattices $`[X]^{}`$ and $`X^{}`$ we define

$$l_0=h_1,l_j=\frac{1}{2}(h_{2j}h_{2j+1});j=1,\mathrm{},a1.$$

Then

$$[X]^{}=l_0,2l_1,\mathrm{},2l_{a1},X^{}=l_0,l_1,\mathrm{},l_{a1}.$$

We first consider the case $`a=1`$. This case is special in the sense that $`X^{}=[X]^{}`$. The decomposition $`[\mathrm{\Delta }]^{}`$ is the unique Delaunay decomposition with respect to $`X^{}`$. In particular it is not Delaunay with respect to $`2X^{}`$. On the other hand condition (\*) is fulfilled since $`X^{}`$ has rank 1.

Let us now consider the case $`a=2`$. The decomposition $`[\mathrm{\Delta }]^{}`$ then looks as follows

We can see immediately from this picture that condition (\*) is not fulfilled. Similarly for arbitrary integers $`a2`$ the decomposition $`[\mathrm{\Delta }]^{}`$ is not Delaunay with respect to $`2X^{}`$, nor is (\*) fulfilled. If we have no smooth fixed points on $`C_1`$ and $`C_2`$, then $`Y^{}=2X^{}`$. This is not true in general. If e.g. $`C_1`$ and $`C_2`$ are genus $`2`$ curves and $`\iota `$ has $`2`$ smooth fixed points on each of the curves $`C_i`$, then the abelian part of the Prym is $`(2,2)`$-polarized and $`X^{}/Y^{}=/2`$.

In this example one can see easily that there are different possible limits for the principally polarized Prym variety depending on the degenerating family. Recall that the form $`B`$ on the lattice $`X=e_1+e_2+e_3+e_4`$ is of the form $`B=_{j=1}^4\alpha _jz_j^2`$, where the $`\alpha _j`$ depend on the degenerating family. The induced form on the lattice $`X^{}=l_1+l_2`$ is given by the matrix

$$B^{}=\left(\begin{array}{cc}\alpha _1+\alpha _4& (\alpha _1+\alpha _4)/2\\ (\alpha _1+\alpha _4)/2& (\alpha _1+\alpha _2+\alpha _3+\alpha _4)/4\end{array}\right).$$

If $`\alpha _j=1`$ for $`j=1,\mathrm{},4`$ this leads to $`B^{}=\left(\begin{array}{cc}2& 1\\ 1& 1\end{array}\right)`$ which is the sum of two semi-positive definite matrices and hence the corresponding Delaunay decomposition of the plane consists of squares. Geometrically this means the following. Let us, for simplicity, assume that we have no abelian part (i.e., the curves $`C_i`$ are elliptic). Then the normalization of $`\overline{P}`$ is a quadric $`^1\times ^1`$ and $`\overline{P}`$ is obtained from this quadric by identifying opposite sides of the square given by the coordinate points via the relation $`xbx`$, where $`b`$ is a non-zero constant. (If there is an abelian part we have a fibration of quadrics over it.) Incidentally, the constant $`b`$ corresponds to the form $`\tau `$. On the other hand if $`\alpha _1=\alpha _4=1`$ and $`\alpha _2=\alpha _3=3`$ then $`B^{}=\left(\begin{array}{cc}2& 1\\ 1& 2\end{array}\right)`$ which is the sum of three semi-positive definite forms and the corresponding Delaunay decomposition of the plane consists of triangles. Geometrically this correspond to two copies of $`^2`$ glued as in the Jacobian of a dollar sign curve of genus $`2`$. The parameters $`b`$ and $`b^1`$ give isomorphic varieties. Altogether we obtain, as $`b`$ varies, a $`^1`$ worth of degenerations, where $`b=0,\mathrm{}`$ belong to the case of the union of two planes.

#### 5.2.4 An example with $`3`$ components

Consider a nodal curve $`C`$ which looks as follows

For the involution $`\iota `$ we want to assume that $`\iota (C_1)=C_2,\iota (Q_1)=Q_2`$ and $`\iota (Q_3)=Q_4`$. Then we must have $`\iota (E)=E`$. This is easy to obtain if we take e.g. $`E`$ to be an elliptic curve and $`\iota `$ a fixed point free involution on $`E`$ which interchanges the points $`Q_3`$ and $`Q_4`$. In this case $`\iota `$ has no smooth fixed points. We orient the graph $`\mathrm{\Gamma }(C)`$ of $`C`$ as follows

Then the involution is given by $`\iota (e_1)=e_2`$ and $`\iota (e_3)=e_4`$. The elements

$$h_1=e_1+e_3e_4,h_2=e_2+e_3e_4$$

form a basis of $`H_1(C,)`$ and with respect to this basis

$$\iota =\left(\begin{array}{cc}0& 1\\ 1& 0\end{array}\right).$$

It follows that

$$X^{}=\frac{1}{2}(h_1+h_2);[X]^{}=2X^{}=h_1+h_2.$$

The picture below shows the Delaunay decomposition $`\mathrm{\Delta }`$ of $`X_{}`$ and its intersection with $`X_{}^{}`$.

The decomposition $`[\mathrm{\Delta }]^{}`$ is the Delaunay decomposition with respect to $`X^{}`$. In particular it is not Delaunay with respect to $`2X^{}`$. On the other hand condition (\*) is fulfilled for $`X^{}`$ since the rank of the lattice is 1. Again in this case $`Y^{}=2X^{}`$, since we can deform $`(C,\iota )`$ to a smooth curve $`(C_t,\iota )`$ where the involution has no smooth fixed points and whose Prym, therefore, has twice a principal polarization. We want to use this example to discuss the relationship between the degenerating families of the principally polarized and the twice principally polarized Prym. The two families coincide outside the central fiber, but have different central fibers, namely a nodal elliptic curve in the first case and a union of $`2^1`$’s intersecting in two points in the second case. We start with a generic smoothing of the pair $`(C,\iota )`$. In this case the monodromy corresponds to the form $`B=\underset{j=1}{\overset{3}{}}z_j^2`$. This defines a bilinear form on $`H_1(C,)`$ which, with respect to the basis $`h_1,h_2`$ of $`H_1(C,)`$, is given by the matrix

$$B|_{H_1(C,)}=\left(\begin{array}{cc}3& 2\\ 2& 3\end{array}\right).$$

Let $`l=1/2(h_1+h_2)`$ be a generator of $`X^{}`$. The form $`[B/2]^{}:Y^{}\times X^{}`$ is then given by $`[B/2]^{}(l,2l)=5`$. In order to describe degenerating families we have to specify height functions $`H_i:X^{};i=1,2`$ which define the decompositions $`\mathrm{\Delta }^{}`$ and $`[\mathrm{\Delta }]^{}`$. We define $`A^{}:X^{}`$ by $`A^{}(ml)=5/4m^2+1/2m`$ and set

$$H_1(ml)=A^{}(ml)+c_1(\overline{m})$$

where

$$c_1(\overline{m})=\{\begin{array}{cccc}0\hfill & \mathrm{if}m\hfill & & 0mod2\hfill \\ 5/4\hfill & \mathrm{if}m\hfill & & 1mod2,\hfill \end{array}$$

respectively

$$c_2(\overline{m})=\{\begin{array}{cccc}0\hfill & \mathrm{if}m\hfill & & 0mod2\hfill \\ 1/4\hfill & \mathrm{if}m\hfill & & 1mod2.\hfill \end{array}$$

Note that only the quadratic part of $`A^{}`$ is uniquely determined by $`[B/2]^{}`$. Given these data we can construct the degeneration families (for details see(\[AN, A2\]). For this we consider the algebra $`k[[t]][z][\zeta ^\pm ]`$ where $`t`$ plays the role of the variable in the base and $`z`$ is a formal variable giving the grading. We then consider the subalgebras $`\stackrel{~}{R_i}k[[t]][z][\zeta ^\pm ]`$ generated by the elements $`z\zeta ^xt^{H_i(x)},xX^{}`$. The group $`Y^{}`$ acts on $`\stackrel{~}{R}_i`$ by

$$y^{}:z\zeta ^xt^{H_i(x)}z\zeta ^{x+y}t^{H_i(x+y)}.$$

One then obtains the degenerating families as $`𝒜_i=\mathrm{Proj}(\stackrel{~}{R_i})/Y^{}`$. A straight forward calculation shows that $`\mathrm{Proj}(\stackrel{~}{R_i})`$ is the torus embedding of a fan $`\mathrm{\Delta }_i`$ which looks as follows

for $`\mathrm{\Delta }_1`$, resp.

for $`\mathrm{\Delta }_2`$. The projection onto the base is given by projecting onto $`_0f`$. From this we can read off that $`𝒜_1`$ has one $`A_5`$-singularity and that $`𝒜_2`$ has one $`A_2`$\- and one $`A_3`$-singularity. The fan

is a common refinement and corresponds to blowing up $`𝒜_1`$ and $`𝒜_2`$ in their singular points. The central fibre is then of type $`I_6`$, i.e. a $`6`$-gon of $`^1`$’s.

Note that all the examples which we have discussed so far lie in $`\overline{}_g`$, provided we have no smooth fixed points.

### 5.3 Mixed types of nodes

#### 5.3.1 Branchwise fixed nodes and non-fixed nodes

(a) Let $`C`$ be an irreducible curve with $`2a`$ non-fixed nodes $`Q_1,\mathrm{},Q_{2a}`$ and $`b`$ branchwise fixed nodes $`Q_1,\mathrm{},Q_b`$. Again we choose the indexing such that $`\iota (Q_j)=Q_{2aj+1}`$. We denote the edges corresponding to $`Q_j`$ by $`e_j`$ and those corresponding to $`Q_k`$ by $`f_k`$. Then $`\iota (e_j)=e_{2aj+1}`$ and $`\iota (f_k)=f_k`$. Hence

$$[H_1(\mathrm{\Gamma }())]^{}=[X]^{}=e_1e_{2a},\mathrm{},e_ae_{a+1}=2X^{}$$

and the discussion is very similar to that of case (2.1). In particular $`[\mathrm{\Delta }]^{}`$ is Delaunay with respect to $`2X^{}`$, conditions (\**) and (\*) are fulfilled and $`Y^{}=2X^{}`$.

(b) We can also have a combination of branchwise fixed and non-fixed nodes if the curve $`C`$ is reducible. E.g., we can have $`C=C_1C_2`$ where $`C_1`$ and $`C_2`$ intersect in $`2a`$ nodes $`Q_1,\mathrm{},Q_{2a}`$ which are non-fixed and $`b`$ branchwise fixed nodes $`Q_1,\mathrm{},Q_b`$ which implies that necessarily $`\iota (C_i)=C_i`$. For simplicity we shall discuss the case $`a=2`$ and $`b=1`$. This implies the existence of a smooth fixed point on each of the components and we shall assume that there is exactly one such point on each $`C_i`$. Then the graph $`\mathrm{\Gamma }(C)`$ of $`C`$ looks as follows

where $`f`$ belongs to the branchwise fixed node $`Q_1`$. As in the discussion before we shall label the nodes in such a way that $`\iota (e_1)=e_4,\iota (e_2)=e_3`$. Since $`Q_1`$ is branchwise fixed we have $`\iota (f)=f`$. The elements

$$h_1=e_1e_3,h_2=e_4e_2,h_3=fe_1,h_4=fe_4$$

form a basis of $`H_1(C,)`$ and with respect to this basis

$$\iota =\left(\begin{array}{cc}\left(\begin{array}{cc}0& 1\\ 1& 0\end{array}\right)& \\ & \left(\begin{array}{cc}0& 1\\ 1& 0\end{array}\right)\end{array}\right).$$

Let

$$l_1=\frac{1}{2}(h_1h_2),l_2=\frac{1}{2}(h_3h_4).$$

Then

$$[X]^{}=2l_1,2l_2=2X^{}.$$

The dicing which appears in condition (\*) now looks as follows

and this shows that (\*) is fulfilled. We even have that $`[\mathrm{\Delta }]^{}`$ is Delaunay with respect to $`2X^{}`$, i.e. (\**) holds. Comparing this to example 5.2.3 shows that the extra node $`Q_1`$ ”improves” the situation in an essential way. The building blocks of the toric part are quadrics $`^1\times ^1`$.

#### 5.3.2 Swapping nodes and non-fixed nodes

(a) We first assume that $`C`$ is irreducible and that it has $`2a`$ non-fixed nodes $`Q_1,\mathrm{},Q_{2a}`$ and $`b`$ swapping nodes $`R_1,\mathrm{},R_b`$. We shall assume that $`\iota (Q_j)=Q_{2aj+1}`$ and correspondingly $`\iota (e_j)=e_{2a+j1}`$. We will denote the edges corresponding to the nodes $`R_j`$ by $`f_j`$. The involution acts on these edges by $`\iota (f_j)=f_j`$. This case is related to case (5.2.1). We find

$$\begin{array}{ccc}[X]^{}\hfill & =& f_1,\mathrm{},f_b,e_1e_{2a},\mathrm{},e_ae_{a+1},\hfill \\ 2X^{}\hfill & =& 2f_1,\mathrm{},2f_b,e_1e_{2a},\mathrm{},e_ae_{a+1}[X]^{}\text{ if }b1.\hfill \end{array}$$

The decomposition $`[\mathrm{\Delta }]^{}`$ is not Delaunay with respect to $`2X^{}`$. Recall that swapping nodes contribute $`2`$ fixed points on a smoothing $`C_t`$. We can compute the polarization of $`P(C_t,\iota )`$ and compare this with Proposition 1.9. We find that $`|X^{}/Y^{}|=2`$ if we have no smooth fixed points and $`X^{}=Y^{}`$ if we have smooth fixed points. In particular $`Y^{}2X^{}`$ unless $`b=1`$ and we have no smooth fixed points. Strictly speaking we cannot speak about condition (\*) since this is not a point in $`\overline{}_g`$, but the dicing condition still holds. The building blocks of the toric part are all of the form $`(^1)^b\times (^1)^a`$ and the number of these building blocks in $`[\overline{P}]`$ is equal to $`2^b`$.
(b) Now assume that $`C=C_1C_2`$. We shall assume that all nodes lie on the intersection of $`C_1`$ and $`C_2`$. If we want swapping nodes, then we must necessarily assume that $`\iota (C_1)=C_2`$ and this in turn rules out branchwise fixed nodes. For the sake of simplicity we shall assume that we have no smooth fixed points. Again we assume that we have non-fixed nodes $`Q_1,\mathrm{},Q_{2a}`$ and swapping nodes $`R_1,\mathrm{},R_b`$ . If we choose the orientation in such a way that all edges start at the same vertex, then $`\iota (e_j)=e_{2aj+1}`$ and $`\iota (f_j)=f_j`$. To obtain a basis of $`H_1(C,)`$ we define

$$\begin{array}{c}h_0=f_1e_1\hfill \\ h_1=e_{2a}f_1\hfill \\ h_{2m}=e_me_{2am};m=1,\mathrm{},a1\hfill \\ h_{2m+1}=e_{m+1}e_{2am+1};m=1,\mathrm{},a1\hfill \\ w_k=f_lf_1;l=2,\mathrm{},b.\hfill \end{array}$$

With respect to this basis

$$\iota =\left(\begin{array}{cccccc}\left(\begin{array}{cc}0& 1\\ 1& 0\end{array}\right)& & & & & \\ & \mathrm{}& & & & \\ & & \left(\begin{array}{cc}0& 1\\ 1& 0\end{array}\right)& & & \\ & & & 1& & \\ & & & & \mathrm{}& \\ & & & & & 1\end{array}\right)$$

where we have $`a`$ blocks of type $`\left(\begin{array}{cc}0& 1\\ 1& 0\end{array}\right)`$ and $`b1`$ entries $`1`$. Let

$$l_i=\frac{1}{2}(h_{2i}h_{2i+1});i=0,\mathrm{},a1.$$

Then

$$[X]^{}=2l_0,\mathrm{},2l_{a1},w_2,\mathrm{},w_b,X^{}=l_0,\mathrm{},l_{a1},w_2,\mathrm{},w_b.$$

The abelian part of the degenerate Prym carries twice a principal polarization. On the other hand the Prym of a smoothing $`(C_t,\iota )`$ has a polarization of type $`(1,\mathrm{},1,2,\mathrm{},2)`$ where the number of $`1`$’s is $`b1`$. Hence $`|X^{}/Y^{}|=a`$ and, in particular $`X^{}2Y^{}`$ if $`b2`$.

As a special case we consider $`a=2,b=1`$. Then the decomposition $`[\mathrm{\Delta }]^{}`$ looks as follows

This shows that $`[\mathrm{\Delta }]^{}`$ is not Delaunay with respect to $`2X^{}`$ The curve $`(C,\iota )`$ does not belong to a point in $`\overline{}_g`$, but the dicing condition of (\*) is fulfilled. The building blocks for the toric part of the “middle” Prym $`[\overline{P}]`$ are $`^2`$’s and Hirzebruch surfaces $`F_1`$.

Finally we consider the case $`a=1`$ and $`b=2`$. In this case both $`\mathrm{\Delta }^{}`$ and $`[\mathrm{\Delta }]^{}`$ looks as follows

The dicing condition of (\*) is fulfilled, but $`[\mathrm{\Delta }]^{}`$ is clearly not Delaunay with respect to $`2X^{}=2l_0,2w_2`$. The building blocks of the toric part are projective planes.

#### 5.3.3 Branchwise fixed nodes and swapping nodes

(a) Assume that $`C`$ is irreducible and that it has $`a`$ branchwise fixed nodes $`Q_1,\mathrm{},Q_a`$ and $`b`$ swapping nodes $`R_1,\mathrm{},R_b`$. Moreover we assume that there are $`2r`$ smooth fixed points. Let $`e_1,\mathrm{},e_a,f_1,\mathrm{}f_b`$ denote the corresponding edges. Then $`\iota (e_i)=e_i`$ and $`\iota (f_j)=f_j`$. Hence

$$[X]^{}=X^{}=f_1,\mathrm{},f_b.$$

The decompositions $`\mathrm{\Delta }^{}`$ and $`[\mathrm{\Delta }]^{}`$ are the standard cubes with vertices in $`X^{}`$. In particular $`\mathrm{\Delta }^{}`$ is Delaunay with respect to $`X^{}`$ and the dicing condition of (\*) is fulfilled. Computing the degree of the polarization by looking at a smoothing we find that $`|X^{}/Y^{}|=2`$ if $`r=0`$ and $`X^{}=Y^{}`$ otherwise.

(b) Finally assume that $`C=C_1C_2`$ and that all nodes lie in the intersection of the irreducible components $`C_1`$ and $`C_2`$. Then we cannot have branchwise fixed nodes and swapping nodes simultaneously. Let us assume that we have $`b`$ swapping nodes $`Q_1,\mathrm{},Q_b`$ with corresponding edges $`f_1,\mathrm{},f_b`$. Then $`\iota (f_i)=f_i`$ and $`H_1(\mathrm{\Gamma },)=[H_1(\mathrm{\Gamma },)]^{}`$. Again $`[\mathrm{\Delta }]^{}`$ is the Delaunay decomposition into standard cubes with respect to $`[H_1(\mathrm{\Gamma },)]^{}=[X]^{}`$. As before we have $`|X^{}/Y^{}|=2`$ if we have no smooth fixed points and $`X^{}=Y^{}`$ otherwise. In particular $`Y^{}2X^{}`$ if $`b2`$ or $`b=1`$ and we have smooth fixed points.

V. Alexeev, valery@math.uga.edu
Department of Mathematics, University of Georgia
Athens, Georgia 30602, USA

CH. Birkenhake, birken@mi.uni-erlangen.de
Fachbereich Mathematik, Universität Mainz
Staudingerweg 9, D 55099 Mainz, Germany

K. Hulek, hulek@math.uni-hannover.de
Institut für Mathematik, Universität Hannover
Welfengarten 1, D 30060 Hannover, Germany