text:
# Polaron Crossover and Bipolaronic Metal-Insulator Transition in the half-filled Holstein model.

## Abstract

The formation of a finite density polaronic state is analyzed in the context of the Holstein model using the Dynamical Mean-Field Theory. The spinless and spinful fermion cases are compared to disentangle the polaron crossover from the bipolaron formation. The exact solution of Dynamical Mean-Field Theory is compared with weak-coupling perturbation theory, non-crossing (Migdal), and vertex correction approximations. We show that polaron formation is not associated to a metal-insulator transition, which is instead due to bipolaron formation.

Recent experiments strongly suggest that the electron-phonon (e-ph) interaction is relevant in many materials, ranging from high Tc superconducting cuprates Lanzara , to colossal magnetoresistance manganites manga , from the fullerenes Gunn-review  to magnesium diboride mgb2 . While the specific role of the e-ph interaction is certainly very different from compound to compound, the properties of these materials are hardly explained in terms of standard approaches to the e-ph problem, like the Migdal-Eliashberg theory, and pose a serious challenge to theories.

More specifically, polaronic features have been observed in lightly doped cuprates calvani , in the manganites manga , up to some indication in the fullerenes Ricco . A small polaron is a carrier so tightly interacting with the lattice that its effective mass is strongly enhanced therefore reducing its mobilitynotapolaroni . The single polaron problem has been extensively studied allowing to understand in detail the polaron physics csgcdff ; cgc . Nevertheless, the experimental findings of polaronic effects obviously deal with finite densities of carriers, and prompt for an analogous understanding of the finite density polaron problem, where the polarons are able to interact and to change drastically the phonon properties. In this regard, it is important to stress the distinction between polaronic and bipolaronic states. If repulsion between the electrons is neglected, two polarons (with opposite spin) tend in fact to bind, giving rise to a (local) pair, which is called bipolaron. The bipolarons in turn may undergo a superconducting transition of the Bose-Einstein type alex-ranningr . If superconductivity is not allowed, bipolarons give rise to an insulating state of localized pairs dambrumenil ; bulla  which may eventually condense in a charge-ordered state cdp . In some important compounds bipolaronic states are indeed unlikely formed. In cuprates and fullerenes, the strong electron correlation forbids bipolaron formation, while in the manganites, the double-exchange mechanism favors ferromagnetic states at low temperature, in which no bipolaron can be formed.

In order to disentangle the polaron effects from bipolaron formation, we compare the spinless fermion case, in which bipolarons are forbidden by Pauli principle, with the spinful case, which has been extensively studied in the recent past FJS+strong ; freericks-weak ; millis-shraiman ; pata ; millis-deppler . To be more explicit, we show that the polaron crossover at finite density is not by itself a metal insulator transition (MIT), and insulating behavior can only be associated to localized bipolarons. This effect shows up in the non-vanishing of both the quasiparticle renormalization factor, and of the renormalized phonon frequency in the spinless case. In the spinful case the quasiparticle weight vanishes at some definite coupling while the renormalized phonon frequency remains finite.

We consider the Holstein molecular crystal model, in which tight-binding electrons interact with local modes of constant frequency. The Hamiltonian is

$$H=t\underset{i,j,\sigma }{}c_{i,\sigma }^{}c_{j,\sigma }+H.c.g\underset{i}{}n_i(a_i+a_i^{})+\omega _0\underset{i}{}a_i^{}a_i,$$
(1)
where $`c_{i,\sigma }`$ ($`c_{i,\sigma }^{}`$) and $`a_i`$ ($`a_i^{}`$) are destruction (creation) operators for fermions and for phonons of frequency $`\omega _0`$, $`n_i`$ the electron density, $`t`$ is the hopping amplitude, $`g`$ is an e-ph coupling. The density is fixed to $`n=0.5`$ ($`n=1`$) in the spinless (spinful) system, which corresponds to the particle-hole symmetric half-filled case. In analogy with the studies of the Mott transition in the Hubbard model, in which antiferromagnetism is neglected dmft , we restrict ourselves to the state with no charge order bulla . Despite in the particle-hole symmetric case the ground state is always ordered, the homogeneous solution may become representative of the groundstate whenever charge ordering is spoiled by some frustration effect like a next-nearest-neighbor hopping. Moreover, this study allows to characterize how strong e-ph interaction may lead to the destruction of the metallic state (just like electron correlation leads to the Mott state). The adiabatic ratio $`\gamma =\omega _0/t`$ has been shown to be an important parameter for the single polaron formation. In the adiabatic regime $`\gamma <1`$, the polaron crossover occurs when $`\lambda =g^2/\omega _0t1`$, while in the antiadiabatic regime $`\gamma >1`$, the condition is instead $`(g/\omega _0)^21`$ csgcdff ; cgc .

We solve the model using the Dynamical Mean-Field Theory (DMFT), a non-perturbative approach which becomes exact in the limit of infinite dimensions dmft . We notice that in such a limit the spinless fermion case does not coincide with the infinite correlation limit. Nonetheless, this system represents an instructive playground where polaronic effects can be observed without too many competing phases. In DMFT, the lattice model is mapped onto an impurity problem subject to a self-consistency condition, which contains the information about the lattice. In our case the impurity model is

$`H`$ $`=`$ $`{\displaystyle \underset{k,\sigma }{}}V_kc_{k,\sigma }^{}f_\sigma +H.c.+{\displaystyle \underset{k,\sigma }{}}E_kc_{k,\sigma }^{}c_{k,\sigma }`$ (2)
$``$ $`g(a+a^{}){\displaystyle \underset{\sigma }{}}f_\sigma ^{}f_\sigma +\omega _0a^{}a,`$
where the phonons live only on the impurity ($`f`$) site, $`E_k`$ and $`V_k`$ are the energy levels and the hybridization parameters of the conduction bath. For the $`z`$-coordination Bethe lattice of half-bandwidth $`D=2t\sqrt{z}=1`$ the self-consistency equation in the $`z\mathrm{}`$ limit is

$$\frac{D^2}{4}G(i\omega _n)=\underset{k}{}\frac{V_k^2}{i\omega _nE_k}.$$
(3)
In this work we use Exact Diagonalization (ED) to solve the impurity model (2) caffarel . This method requires to restrict the sum in Eqs. (2,3) to a finite small number of levels $`N_s1`$. The discretized model can then be solved at $`T=0`$ using the Lanczos method. The convergence is exponential in $`N_s`$, and a few levels are enough to give converged results. Results are obtained taking typically $`N_s=10`$, having checked that no significant change occurs for larger $`N_s`$ note-ph . Error bars can be evaluated by linearly extrapolating (in $`1/N_s`$) to $`N_s\mathrm{}`$, and are of the size of symbols in the figures.

We compare the ED solution of DMFT, with two approximate schemes: a self-consistent Non Crossing Approximation (NCA) dambrumenil ; freericks-weak  (diagrams (a) in Fig. 1) and a self-consistent approximation including the first Vertex Correction beyond NCA (VCA) freericks-weak  (diagrams (a) and (b) in Fig. 1). Notice that in the standard Migdal-Eliashberg approximation the phonon spectrum is not self-consistently evaluated, but it is taken “from experiments”, while in our NCA and VCA the phonon self-energy is self-consistently evaluated (on the imaginary frequency axis) through an iterative scheme, just like the electron self-energy. Actually, to allow for a safer truncation on the frequency axis, the quantity which is iteratively determined is not the full self-energy, but the difference between this quantity and the result from second order perturbation theory. Zero temperature results are obtained by lowering the temperature until the physically relevant quantities converge (consistently increasing the number of Matsubara frequecies $`N_{max}=6/(2T\pi )`$). We checked the convergence to $`T=0`$ by monitoring the quasiparticle spectral weight $`Z`$ defined as $`Z^1=(1(\mathrm{\Sigma }(i\omega _{n=1})\mathrm{\Sigma }(i\omega _{n=0})/2\pi T)`$ where $`\omega _n=(2n1)\pi T`$. The vanishing of $`Z`$ is used within DMFT to characterize the Mott MIT in the repulsive dmft , and the pairing transition in the attractive Hubbard models attractive . A vanishing $`Z`$ has been found also in the spinful Holstein model bulla . The convergence to $`T=0`$ turns out to depend on both $`\gamma `$ (as detailed below, we study $`\gamma =0.1`$ and $`\gamma =1`$ as representative of adiabatic and nonadiabatic regimes, respectively) and $`\lambda `$. Within VCA $`T/t=3\times 10^3`$ is sufficient to get results representative of the ground state for weak/intermediate coupling. In the spinful case for $`\gamma =0.1`$ $`T=10^3`$ is instead necessary since the polaron crossover is approached for smaller coupling (see below). Within NCA it is possible to span the strong coupling regime, making an extrapolation to $`T=0`$ necessary in the adiabatic regime for $`\lambda >1`$ ($`\lambda >0.5`$) in the spinless (spinful) case. In the non-adiabatic regime the extrapolation is required for $`\lambda >1.6`$ ($`\lambda >1.2`$) in the spinless (spinful) case.

We first discuss the spinless case. Exact DMFT results for $`Z`$ are shown in Fig. 2. The logarithmic scale on the y-axis evidences that in both cases $`Z`$, even if exponentially reduced, does never vanish by increasing $`\lambda `$, indicating that no MIT is taking place. It is important to observe that $`Z`$ increases when the truncation error is reduced increasing $`N_s`$.

The comparison with NCA and VCA shows non trivial tendencies. Both approximations are accurate at weak coupling, but VCA remains closer to ED for relatively large coupling even in the adiabatic regime, where the Migdal approximation would be expected to hold. Quite surprisingly even in the non-adiabatic regime VCA improves NCA only at weak coupling. At strong coupling NCA predicts a polaronic crossover, in qualitative agreement with exact results, even if, for $`\gamma =1`$, the crossover coupling is strongly underestimated by NCA. On the other hand, VCA drastically diverges from ED, and gives unphysical negative mass renormalization at strong coupling both for $`\gamma =0.1`$ and $`1`$, in agreement with the divergence of vertex corrections predicted in Ref. dambrumenil . The coupling at which the approximate methods deviate from exact results decreases with increasing $`\gamma `$, and it is unrelated with the polaron crossover coupling which is instead larger for $`\gamma =1`$, as shown in the inset of Fig. 1, and in agreement with the single polaron case csgcdff ; cgc .

An important difference between the finite-density situation and the single polaron problem is that many electrons are able to renormalize the phonon properties. The renormalized phonon frequency can be obtained from the phonon propagator $`D(i\omega _n)=_0^\beta 𝑑\tau e^{i\omega _n\tau }T(a_0(\tau )+a_0^{}(\tau ))(a_0+a_0^{})`$, as $`(\mathrm{\Omega }/\omega _0)^2=2/\omega _0D(i\omega _n=0)^1`$. As it is shown in Fig. 3, $`\mathrm{\Omega }/\omega _0`$ never vanishes as a function of the coupling, but rather exponentially decreases.

The way NCA and VCA results for $`\mathrm{\Omega }`$ compare with ED is analogous to the results for $`Z`$. In the adiabatic regime NCA strongly overestimates $`\mathrm{\Omega }/\omega _0`$ around the polaron crossover. In the nonadiabatic regime NCA again predicts a phonon softening which does not actually occur at such small values of the coupling. In the same regime VCA improves NCA result only at weak coupling and gives a phonon hardening at strong coupling. It is worth to note that for both $`Z`$ and $`\mathrm{\Omega }`$ the coupling at which NCA and VCA deviate from ED is lower in nonadiabatic than in adiabatic regime. This “non uniform” behavior is not present for a single polaron cgc , and represents a first peculiarity of the finite-density case.

Now we compare our findings for spinless fermions with the spinful fermion case. In the adiabatic regime a MIT has been found around $`\lambda \lambda _{MIT}0.68`$ pata , close to the value ($`\lambda =0.664`$) at which the density of state at Fermi energy vanishes in the adiabatic limit ($`\gamma =0`$) millis-shraiman . This transition has a precursor in the phonon softening bulla  but, contrary to the spinless case, the spin degrees of freedom prevent electron coherent hopping at strong coupling through a Kondo-like mechanism Moeller . A MIT has been claimed to occur also within NCA dambrumenil .

In Fig. 4 we show results for $`\gamma =0.1`$. The logarithmic plot clearly shows that $`Z`$ vanishes faster than exponential, signaling a MIT at $`\lambda 0.76`$, in agreement with Ref. bulla . Contrary to the spinless case, here $`Z`$ decreases by increasing $`N_s`$. On the other hand, $`\mathrm{\Omega }/\omega _0`$ decreases with $`\lambda `$ as in the spinless case, suggesting that, even if strongly softened, the phonon mode never becomes completely soft. This behavior is more evident for larger phonon frequency $`\gamma =1`$, as reported in Fig. 5. Also in this case $`Z`$ vanishes much faster than exponential at $`\lambda =1.44`$, where the MIT takes place, while the phonon renormalization is much less effective and leads to quite a large value of $`\mathrm{\Omega }/\omega _0`$ even at the MIT point (see inset).

The comparison with approximate schemes qualitatively resembles the spinless case. NCA underestimates $`Z`$ for $`\lambda `$ below the MIT, but it gives a finite weight also above the exact MIT point. Our extrapolation to $`T=0`$ from finite temperatures does not allow us to definitively rule out the existence of a MIT within NCA even if, in contrast with a claim in Ref. dambrumenil , $`Z`$ seems to decrease exponentially with $`\lambda `$ within this approximation note-ambrumenil . Again VCA gives better results than NCA even in the adiabatic regime up to intermediate coupling $`\lambda 0.55`$, but it becomes completely unreliable at strong coupling.

We have studied the formation of a polaronic state in the Holstein model at half-filling within DMFT. For spinless fermions, a continuous crossover leads to a polaronic state by increasing the coupling constant. Despite the electron effective mass becomes exponentially large in the strong-coupling regime, the ground state is always metallic. The crossover is more abrupt in the adiabatic case. In the spinful case, the polarons can bind to form bipolarons, leading to a real MIT. The phonon renormalization is much stronger in the adiabatic regime than in the nonadiabatic case, but the phonons do not become completely soft at the MIT. Approximate treatments (NCA and VCA) strongly deviate from exact DMFT above a coupling which diminishes with increasing $`\omega _0/t`$. At weak coupling VCA correctly reproduces the qualitative trends of exact results, and improves on NCA.

M. C. thanks the hospitality and financial support of the Physics Department of the University of Rome “La Sapienza”, and of INFM, Unitá Roma 1. We acknowledge financial support of MIUR Cofin 2001 and enlightening discussions with C. Castellani.


format: text

text:
# 1 Introduction

## 1 Introduction

Let $`C_g`$ be a general curve of genus $`g2`$ (over $``$). Then $`C_g`$ has a cover to $`^1`$ of degree $`n`$ if and only if $`2(n1)g`$. This is a classical fact of algebraic geometry. (It is part of Brill-Noether theory, which more generally considers maps of a curve to $`^m`$, see \[HM\], Ch. 5). If $`C_g`$ has a cover to $`^1`$ of degree $`n`$, then there is such a cover that is simple, i.e., has monodromy group $`S_n`$ and all inertia groups are generated by transpositions. The question arises whether $`C_g`$ admits other types of covers to $`^1`$.

If there is a cover $`C_g^1`$ branched at $`r`$ points of $`^1`$ and $`g2`$ then $`r3g`$ (see Remark 2.2 below). Zariski \[Za\] used this to show that if $`g>6`$ then there is no such cover with solvable monodromy group. He made a conjecture on the existence of such covers for $`g6`$, but there is a counterexample to that, see Fried \[Fr2\], Fried/Guralnick \[FrGu\].

The condition $`r3g`$ was further used by Guralnick to restrict the possibilities for the monodromy group $`G`$ of a cover $`C_g^1`$ of degree $`n`$. Assume the cover does not factor non-trivially, i.e., $`G`$ is a primitive subgroup of $`S_n`$. (Knowledge of this case is sufficient to know all types of covers $`C_g^1`$; this was already observed by Zariski \[Za\], see \[GM\]). If further $`g>3`$, then $`G=S_n`$ or $`G=A_n`$. For $`g=3`$ there are 3 additional cases, with $`n=7,8,16`$ and $`G=GL_3(2),AGL_3(2),AGL_4(2)`$, respectively. This was proved by Guralnick and Magaard \[GM\] and Guralnick and Shareshian \[GS\], using the classification of finite simple groups. There is also a corresponding result for $`g=2`$, but it is less definitive.

As noted in \[GM\], it was not known whether the case $`G=A_n`$ actually occurs. This is answered in the affirmative in this paper. More precisely, we prove the following: Let $`g3`$ and $`n2`$. Then the general curve of genus $`g`$ admits a cover to $`^1`$ of degree $`n`$ with monodromy group $`A_n`$ such that all inertia groups are generated by double transpositions if and only if $`n2g+1`$. The same statement holds when we replace double transpositions by $`3`$-cycles (see Theorem 3.3). We refine the latter result in Theorem 4.1 by showing that both of the two types of $`3`$-cycle covers occur for the general curve. (See Fried \[Fr1\] and Serre \[Se1\], \[Se2\] for this type distinction). We also study the exceptional cases in genus 3.

A preliminary version of this paper has been circulated since October 2001. It was brought to our attention that in a recent preprint S. Schröer \[Schr\] proves a weaker version of our result on $`3`$-cycles (which, however, also holds in positive characteristic): The locus in $`_g`$ of curves admitting a cover to $`^1`$ with only triple ramification points has dimension $`max(2g3,g)`$.

The authors are grateful to Bob Guralnick for raising the question of moduli dimension for alternating groups. We further gratefully acknowledge the kind support of Gerhard Frey without whose expert advice this paper may have never been finished.

## 2 Moduli dimension of a tuple in $`S_n`$

Let $`^1=_{}^1`$ the Riemann sphere. Let $`𝒰^{(r)}`$ be the open subvariety of $`(^1)^r`$ consisting of all $`(p_1,\mathrm{},p_r)`$ with $`p_ip_j`$ for $`ij`$. Consider a cover $`f:X^1`$ of degree $`n`$, with branch points $`p_1,\mathrm{},p_r^1`$. Pick $`p^1\{p_1,\mathrm{},p_r\}`$, and choose loops $`\gamma _i`$ around $`p_i`$ such that $`\gamma _1,\mathrm{},\gamma _r`$ is a standard generating system of the fundamental group $`\mathrm{\Gamma }:=\pi _1(^1\{p_1,\mathrm{},p_r\},p)`$ (see \[V\], Thm. 4.27); in particular, we have $`\gamma _1\mathrm{}\gamma _r=1`$. Such a system $`\gamma _1,\mathrm{},\gamma _r`$ is called a homotopy basis of $`^1\{p_1,\mathrm{},p_r\}`$. The group $`\mathrm{\Gamma }`$ acts on the fiber $`f^1(p)`$ by path lifting, inducing a transitive subgroup $`G`$ of the symmetric group $`S_n`$ (determined by $`f`$ up to conjugacy in $`S_n`$). It is called the monodromy group of $`f`$. The images of $`\gamma _1,\mathrm{},\gamma _r`$ in $`S_n`$ form a tuple of permutations called a tuple of branch cycles of $`f`$.

Let $`\sigma _1,\mathrm{},\sigma _r`$ be elements $`1`$ of the symmetric group $`S_n`$ with $`\sigma _1\mathrm{}\sigma _r=1`$, generating a transitive subgroup. Let $`\sigma =(\sigma _1,\mathrm{},\sigma _r)`$. We call such a tuple admissible. We say a cover $`f:X^1`$ of degree $`n`$ is of type $`\sigma `$ if it has $`\sigma `$ as tuple of branch cycles relative to some homotopy basis of $`^1`$ minus the branch points of $`f`$. The genus $`g`$ of $`X`$ depends only on $`\sigma `$ (by the Riemann-Hurwitz formula); we write $`g=g_\sigma `$.

Let $`_\sigma `$ be the set of pairs $`([f],(p_1,\mathrm{},p_r))`$, where $`[f]`$ is an equivalence class of covers of type $`\sigma `$, and $`p_1,\mathrm{},p_r`$ is an ordering of the branch points of $`f`$. We use the usual notion of equivalence of covers, see \[V\], p. 67. Let $`\mathrm{\Psi }_\sigma :_\sigma 𝒰^{(r)}`$ be the map forgetting $`[f]`$. The Hurwitz space $`_\sigma `$ carries a natural structure of quasiprojective variety such that $`\mathrm{\Psi }_\sigma `$ is an algebraic morphism, and an unramified covering in the complex topology (see \[FrV\],\[V\], \[We\]). We also have the morphism

$$\mathrm{\Phi }_\sigma :_\sigma _g$$

mapping $`([f],(p_1,\mathrm{},p_r))`$ to the class of $`X`$ in the moduli space $`_g`$ (where $`g=g_\sigma `$). Each irreducible component of $`_\sigma `$ has the same image in $`_g`$ (since the action of $`S_r`$ permuting $`p_1,\mathrm{},p_r`$ induces a transitive action on the components of $`_\sigma `$). Hence this image, i.e., the locus of genus $`g`$ curves admitting a cover to $`^1`$ of type $`\sigma `$, is irreducible.

###### Definition 2.1

(a) The moduli dimension of $`\sigma `$, denoted by $`\text{mod-dim}(\sigma )`$, is the dimension of the image of $`\mathrm{\Phi }_\sigma `$; i.e., the dimension of the locus of genus $`g`$ curves admitting a cover to $`^1`$ of type $`\sigma `$. We say $`\sigma `$ has full moduli dimension if $`\text{mod-dim}(\sigma )=dim_g`$.

(b) We say $`\sigma `$ has infinite moduli degree if the following holds: If $`f:X^1`$ is a cover of type $`\sigma `$ with general branch points then $`X`$ has infinitely many covers to $`^1`$ of (the same) type $`\sigma `$ such that the corresponding subfields of the function field of $`X`$ are all different. (This terminology is further discussed at the end of this section).

A curve is called a general curve of genus $`g`$ if it corresponds to a point of $`_g`$ that does not lie in any proper closed subvariety of $`_g`$ defined over $`\overline{}`$ (the algebraic closure of the rationals). Clearly, an admissible tuple $`\sigma `$ has full moduli dimension if and only if each general curve of genus $`g_\sigma `$ admits a cover to $`^1`$ of type $`\sigma `$.

Part (a) of the following Remark is the necessary condition for full moduli dimension used by Guralnick, Fried and Zariski. We indicate the proof at the end of this section.

###### Remark 2.2

Let $`\sigma `$ be an admissible tuple of length $`r`$ in $`S_n`$, and $`g:=g_\sigma `$.
(a) Suppose $`\sigma `$ has full moduli dimension. Then $`r3dim_g`$, thus if $`g2`$ then $`r3g`$.
(b) If $`r3>dim_g`$ then $`\sigma `$ has infinite moduli degree.

Here is a simple but crucial lemma that allows us to make use of the hypothesis of infinite moduli degree.

###### Lemma 2.3

Suppose $`f_i:X^1`$ is an infinite collection of covers such that the corresponding subfields of the function field of $`X`$ are all different. Let $`S`$ be the set of $`(x,y)X\times X`$ with $`f_i(x)=f_i(y)`$ for some $`i`$. Then $`S`$ is Zariski-dense in $`X\times X`$.

Proof : Let $`S_i`$ be the curve on $`X\times X`$ consisting of all $`(x,y)`$ with $`f_i(x)=f_i(y)`$. The set $`S`$ is the union of all $`S_i`$. If $`S`$ is not Zariski-dense in $`X\times X`$ then it must be the union of finitely many $`S_i`$; then the curves $`S_i`$ cannot be all distinct. But if $`S_i=S_j`$ then the subfields of $`(X)`$ corresponding to $`f_i`$ and $`f_j`$ coincide. This contradicts the hypothesis.

Here is our sufficient condition for full moduli dimension.

###### Lemma 2.4

Let $`n3`$. Given an admissible tuple $`\sigma =(\sigma _1,\mathrm{},\sigma _r)`$ in $`S_n`$ with $`g_\sigma >0`$, define $`\stackrel{~}{\sigma }=(\sigma _1,\mathrm{},\sigma _{r+2})`$, where either

$$\sigma _{r+1}=\sigma _{r+2}=(1,2)(n,n+1)$$

is a double transposition or

$$\sigma _{r+1}=\sigma _{r+2}^1=(n1,n,n+1)$$

is a 3-cycle. Then $`\stackrel{~}{\sigma }`$ is an admissible tuple in $`S_{n+1}`$ with $`g_{\stackrel{~}{\sigma }}=g_\sigma +1`$. If $`\sigma `$ has infinite moduli degree then

$$\text{mod-dim}(\stackrel{~}{\sigma })\text{mod-dim}(\sigma )+\{\begin{array}{cc}3\hfill & \text{ if }g_\sigma >1\hfill \\ 2\hfill & \text{ if }g_\sigma =1\hfill \end{array}$$

Proof : Let $`g:=g_\sigma `$. Then $`g_{\stackrel{~}{\sigma }}=g+1`$ by Riemann-Hurwitz. Let $`\mathrm{\Phi }:=\mathrm{\Phi }_{\stackrel{~}{\sigma }}`$ and $`:=_{\stackrel{~}{\sigma }}`$. The map $`\mathrm{\Phi }`$ extends to $`\overline{\mathrm{\Phi }}:\overline{}\overline{}_{g+1}`$, where $`\overline{}_{g+1}`$ is the stable compactification of $`_{g+1}`$, and $`\overline{}`$ is $``$ plus that piece $``$ of the boundary where the last two branch points come together (see \[We\]); thus $`\overline{}`$ covers the set of $`(p_1,\mathrm{},p_{r+2})`$ in $`(^1)^{r+2}`$ with $`p_ip_j`$ for $`ij`$ unless $`\{i,j\}=\{r+1,r+2\}`$, and $``$ is the inverse image of the subset defined by the condition $`p_{r+1}=p_{r+2}`$.

If we coalesce the last two entries of $`\stackrel{~}{\sigma }`$ we obtain $`\sigma `$, which has orbits of length $`n`$ and 1 on $`\{1,\mathrm{},n+1\}`$. For a cover $`X_{g+1}^1`$ of type $`\stackrel{~}{\sigma }`$, this means the following: When coalescing the last two branch points, $`X_{g+1}`$ degenerates into a nodal curve $`\overline{X}`$ with two components linked at one point $`P`$. One component is a non-singular curve covering $`^1`$ of degree 1. The other component $`\overline{X}_g`$ is a singular curve whose only singularity is a node $`N`$. Its normalization $`X_g`$ covers $`^1`$ of type $`\sigma `$. If $`\sigma _{r+1}=(1,2)(n,n+1)`$ then $`N`$ corresponds to the cycle $`(1,2)`$ and $`P`$ to the cycle $`(n,n+1)`$. If $`\sigma _{r+1}=(n1,n,n+1)`$ then $`N=P`$.

The nodal curve $`\overline{X}`$ is stably equivalent to the stable curve $`\overline{X}_g`$, and the latter constitutes the image in $`\overline{}_{g+1}`$ of the element of $``$ corresponding to $`\overline{X}^1`$ (see \[HM\], Th. 3.160). Thus the image of $``$ in $`\overline{}_{g+1}`$ lies in the boundary component consisting of irreducible curves with one node whose normalization has genus $`g`$. We can identify this boundary component with $`_{g,2}`$ (= moduli space of genus $`g`$ curves with two unordered marked points). The two marked points correspond to the node. Thus we have the commutative diagram

$$\begin{array}{ccc}_\sigma & \stackrel{\mathrm{\Phi }_\sigma }{}& _g\\ & & \\ & & _{g,2}\\ & & \\ \overline{}& \stackrel{\overline{\mathrm{\Phi }}}{}& \overline{}_{g+1}\end{array}$$

where the vertical arrows on the lower level are inclusion. The map $`_{g,2}_g`$ is the natural projection (forgetting the marked points), and the map $`_\sigma `$ sends the point corresponding to the cover $`\overline{X}^1`$ to that corresponding to the cover $`X_g^1`$ of type $`\sigma `$ (see the previous paragraph).

The image of $`\overline{}`$ in $`\overline{}_{g+1}`$ is irreducible (as remarked above). Its intersection with the boundary of $`\overline{}_{g+1}`$ is a closed proper subvariety, hence has codimension at least 1. This subvariety contains the image of $``$, which we denote by $`\text{Im}()`$. Thus $`\text{mod-dim}(\stackrel{~}{\sigma })1+dim\text{Im}()`$.

The fiber $`F`$ in $`_{g,2}`$ of the point of $`_g`$ corresponding to $`X_g`$ can be identified with the set of unordered pairs $`(x,y)`$ of distinct points of $`X_g`$, modulo Aut$`(X_g)`$. The intersection $`F_\sigma `$ of this fiber with $`\text{Im}()`$ consists of those $`(x,y)`$ such that there is a cover $`f:X_g^1`$ of type $`\sigma `$ with $`f(x)=f(y)`$ and $`f(x)`$ not a branch point of $`f`$. If $`X_g`$ is a general curve with the property that it admits a cover to $`^1`$ of type $`\sigma `$, then by Lemma 2.3 and the hypothesis of infinite moduli degree, $`F_\sigma `$ is Zariski-dense in $`F`$. Since $`F_\sigma `$ is the general fiber of the surjective map $`\text{Im}()\mathrm{\Phi }_\sigma (_\sigma )`$, it follows that $`dim\text{Im}()=dimF+dim\mathrm{\Phi }_\sigma (_\sigma )=dimF+\text{mod-dim}(\sigma )`$. This completes the proof.

Consider the natural action of $`\text{PGL}_2()`$ on $`^1`$ (by fractional linear transformations). It induces an action on $`_\sigma `$, with $`\lambda \text{PGL}_2()`$ mapping $`([f],(p_1,\mathrm{},p_r))`$ to $`([\lambda f],(\lambda (p_1),\mathrm{},\lambda (p_r)))`$. The closed subspace of $`_\sigma `$ defined by the conditions $`p_1=0`$, $`p_2=1`$, $`p_3=\mathrm{}`$ maps bijectively to the quotient $`_\sigma /\text{PGL}_2()`$. Hence this quotient carries a natural structure of quasi-projective variety, and the map $`\mathrm{\Phi }_\sigma :_\sigma _g`$ induces a morphism $`_\sigma /\text{PGL}_2()_g`$. (Clearly $`\mathrm{\Phi }_\sigma `$ is constant on $`\text{PGL}_2()`$-orbits).

The dimension of (each component of) $`_\sigma /\text{PGL}_2()`$ is $`r3`$. Thus if $`\mathrm{\Phi }_\sigma `$ is dominant then $`r3dim_g`$. This proves Remark 2.2(a). If $`r3>dim_g`$ then the general fiber of the map $`_\sigma /\text{PGL}_2()_g`$ is infinite. This proves Remark 2.2(b) (since two covers $`f_1,f_2:X^1`$ correspond to the same subfield of the function field of $`X`$ if and only if $`f_1`$ is the composition of $`f_2`$ with an element of $`\text{PGL}_2()`$).

For clarification, we now briefly discuss the general concept of moduli degree. This will not be needed elsewhere in the paper. The map $`_\sigma /\text{PGL}_2()_g`$ factorizes further over the action of $`S_r`$ permuting the branch points (i.e., one can drop the ordering of the branch points. Actually, the version of the Hurwitz space without ordering of the branch points is more natural, see \[V\], Ch. 10, but for the purpose of this paper we need the ordering). Anyway, the natural definition of the moduli degree of $`\sigma `$ is as follows: The degree of the induced map from the (irreducible) variety $`_\sigma /(\text{PGL}_2()\times S_r)`$ to $`_g`$. Thus the moduli degree of $`\sigma `$ is the number of covers $`f:X^1`$ of type $`\sigma `$ modulo $`\text{PGL}_2()`$, where $`X`$ corresponds to a (fixed) general point in the image of $`\mathrm{\Phi }_\sigma `$.

## 3 Covers with monodromy group $`A_n`$

We consider admissible tuples $`\sigma =(\sigma _1,\mathrm{},\sigma _r)`$ in $`S_n`$ such that each $`\sigma _i`$ is a double transposition (resp., 3-cycle). Then $`r=n+g1n1`$, where $`g:=g_\sigma `$ (by Riemann-Hurwitz). Let DT$`(n,g)`$ (resp., TC$`(n,g)`$ ) be the set of these tuples $`\sigma `$; and let DTA$`(n,g)`$ (resp., TCA$`(n,g)`$ ) be the subset consisting of those $`\sigma `$ that generate $`A_n`$ (the alternating group).

###### Lemma 3.1

(i) For each $`n4`$ (resp., $`n6`$) the set DT$`(n,0)`$ (resp., DTA$`(n,0)`$ ) is non-empty.
(ii) The set TCA$`(n,0)`$ is non-empty for each $`n3`$.

Proof : (i) For $`n=4`$ take $`\sigma `$ to consist of all double transpositions in $`A_4`$. For $`n=5`$ take $`\sigma =(\sigma _1,\mathrm{},\sigma _4)`$ such that $`\sigma _1\sigma _2`$ ($`=(\sigma _3\sigma _4)^1`$) is a 5-cycle. For $`n=6`$ use GAP (or check otherwise).

Assume now $`\sigma `$ is in DTA$`(n,0)`$, and $`n6`$. We may assume $`\sigma _r=(1,2)(3,4)`$. Replacing $`\sigma _r`$ by the two elements $`(1,2)(n,n+1)`$ and $`(3,4)(n,n+1)`$ yields a tuple in DTA$`(n+1,0)`$. This proves (i).

(ii) For $`n=3`$ take $`\sigma =((1,2,3),(1,2,3)^1)`$. Assume now $`\sigma `$ is in TCA$`(n,0)`$, $`n3`$. We may assume $`\sigma _1=(1,2,3)`$. Replacing $`\sigma _1`$ by the two elements $`(n+1,3,1)`$ and $`(3,n+1,2)`$ yields a tuple in TCA$`(n+1,0)`$.

###### Lemma 3.2

Both of DTA$`(n,g)`$ and TCA$`(n,g)`$ contain a tuple of full moduli dimension if one of the following holds:
(i) $`g=1`$ and $`n5`$.
(ii) $`g=2`$ and $`n6`$.
(iii) $`g>2`$ and $`n2g+1`$.

Proof : (i) See \[FKK\] for a proof of the TCA$`(n,1)`$ case that does not use the stable compactification. For the DTA$`(n,1)`$ case, we use induction on $`n`$.

We anchor our induction at $`n=5`$. We choose $`\sigma =(\sigma _1,\sigma _2,\sigma _3,\sigma _4,\sigma _5)`$, where $`\sigma _1=\sigma _2=(1,2)(3,4)`$, $`\sigma _3=(1,2)(4,5)`$, $`\sigma _4=(1,4)(2,5)`$, and $`\sigma _5=(1,5)(2,4)`$. If we coalesce the last two entries of $`\sigma `$ we obtain $`(\sigma _1,\sigma _1,\sigma _3,\sigma _3)`$, which has orbits $`\{1,2\}`$ and $`\{3,4,5\}`$. For a cover $`X_1^1`$ of type $`\sigma `$, this means the following: When coalescing the last two branch points, $`X_1`$ degenerates into a nodal curve $`\overline{X}`$ with two components linked at one point $`P`$. Both components are non-singular curves of genus 1 (resp., 0). They both cover $`^1`$ with four branch points and of degree 2 (resp. 3). The point $`P`$ ramifies in both covers. The nodal curve $`\overline{X}`$ is stably equivalent to its genus 1 component, and the latter constitutes the image in $`_1`$ of the cover $`\overline{X}^1`$ (as in the proof of lemma 2.4). Clearly, every element of $`_1`$ can be obtained in this fashion. Thus the map $`_\sigma _1`$ is dominant because the boundary of $`_\sigma `$ already maps surjectively to $`_1`$.

Now assume $`\sigma =(\sigma _1,\mathrm{},\sigma _n)`$ is a tuple in DTA$`(n,1)`$, $`n5`$, of full moduli dimension. Write $`\sigma _n=st`$ where $`s,t`$ are double transpositions in $`S_{n+1}S_n`$. Let $`\sigma ^{}=(\sigma _1,\mathrm{},\sigma _{n1},s,t)`$, a tuple in DTA$`(n+1,1)`$. Moreover, $`\sigma ^{}`$ has full moduli dimension because $`\mathrm{\Phi }_\sigma ^{}`$ restricted to the boundary component of $`_\sigma ^{}`$ isomorphic to $`_\sigma `$ already maps dominantly to $`_1`$.

(ii) Same for both cases. So we only do the DT case. By (i), there is a tuple in DTA$`(n1,1)`$ of full moduli dimension. Its length equals $`n1`$, and $`(n1)3>1=dim_1`$; thus the tuple has infinite moduli degree by Remark 2.2(b). Then Lemma 2.4 produces a tuple in DTA$`(n,2)`$ of full moduli dimension.

(iii) Same for both cases. So we only do the DT case. First we settle the case $`g=3`$, $`n7`$. By (ii), there is a tuple in DTA$`(n1,2)`$ and of moduli dimension 3. Its length is $`n`$, and $`n3>3=dim_2`$; the claim follows from Remark 2.2(b) and Lemma 2.4.

Now suppose $`g>3`$, $`n2g+1`$. Then $`n12(g1)+2`$. By induction we may assume there is a tuple in DTA$`(n1,g1)`$ and of full moduli dimension. Its length is $`r:=n+g3`$, and $`r3>3(g1)3=dim_{g1}`$; the claim follows again from Remark 2.2(b) and Lemma 2.4.

###### Theorem 3.3

(i) Let $`g3`$. Then each general curve of genus $`g`$ admits a cover to $`^1`$ of degree $`n`$ with monodromy group $`A_n`$ such that all inertia groups are generated by double transpositions if and only if $`n2g+1`$.
(ii) For $`n6`$ (resp., $`n5`$), each general curve of genus 2 (resp., 1) admits a cover to $`^1`$ of degree $`n`$ with monodromy group $`A_n`$ such that all inertia groups are generated by double transpositions.
(iii) Assertions (i) and (ii) also hold for 3-cycles instead of double transpositions.

Proof : In view of Lemma 3.2, it only remains to show that the condition $`n2g+1`$ in (i) is necessary. Indeed, if the general curve of genus $`g`$ admits such a cover then an associated tuple of branch cycles is in DTA$`(n,g)`$ and of full moduli dimension. Thus the claim follows from the necessary condition $`r3g`$ (Remark 2.2) since $`r=n+g1`$. The proof of (iii) is the same.

###### Corollary 3.4

Let $`C`$ be a general curve of genus $`g4`$. Then the monodromy groups of primitive covers $`C^1`$ are among the symmetric and alternating groups, and up to finitely many, all of these groups occur.

Here a cover is called primitive if it does not factor non-trivially. The first assertion in the Corollary follows from \[GM\], and the second from the Theorem plus Brill-Noether theory.

## 4 Braid orbits of admissible tuples

The braid orbit of a tuple $`\sigma `$ in $`S_n`$ is the smallest set of tuples in $`S_n`$ that contains $`\sigma `$ and is closed under (component-wise) conjugation and under the braid operations

$$(g_1,\mathrm{},g_r)^{Q_i}=(g_1,\mathrm{},g_{i+1},g_{i+1}^1g_ig_{i+1},\mathrm{},g_r)$$

for $`i=1,\mathrm{},r1`$.

Let $`\sigma `$, $`\sigma ^{}`$ be admissible tuples in $`S_n`$ of length $`r`$. Let $`f:X^1`$ be a cover of type $`\sigma `$. Then $`f`$ is of type $`\sigma ^{}`$ if and only if $`\sigma ^{}`$ lies in the braid orbit of $`\sigma `$. In other words, for the associated Hurwitz spaces we have $`_\sigma =_\sigma ^{}`$ if and only if $`\sigma ^{}`$ lies in the braid orbit of $`\sigma `$ (see \[FrV\], \[V\], Ch. 10). Thus the above notions of moduli dimension, moduli degree etc. depend only on the braid orbit of $`\sigma `$. So from now on we will speak of the moduli dimension of a braid orbit, etc.

### 4.1 Braid orbits of 2-cycle tuples

Admissible tuples in $`S_n`$ of fixed length that consist only of transpositions form a single braid orbit (by Clebsch 1872, see \[V\], Lemma 10.15). They correspond to the so-called simple covers. Their braid orbit has full moduli dimension if and only if $`2(n1)g`$, where $`g=g_\sigma `$ (see the remarks in the Introduction).

### 4.2 Braid orbits of 3-cycle tuples

Now consider tuples that consist only of 3-cycles. Recall our notation TC$`(n,g)`$ for the set of those (admissible) tuples with fixed parameters $`n`$, $`g`$. Assume $`n5`$. Note that TC$`(n,g)=`$ TCA$`(n,g)`$ (i.e., each such tuple generates $`A_n`$) by \[Hup\], Satz 4.5.c and the fact that a transitive group generated by $`3`$-cycles must be primitive. The corresponding covers have been studied by Fried \[Fr1\]. Serre \[Se1\], \[Se2\] considered certain generalizations. Fried proved that TC$`(n,g)`$ (is non-empty and) consists of exactly two braid orbits (resp., one braid orbit) if $`g>0`$ (resp., $`g=0`$). Let

$$\{\pm 1\}\widehat{A}_nA_n$$

be the unique non-split degree 2 extension of $`A_n`$. Each 3-cycle $`tA_n`$ has a unique lift $`\widehat{t}\widehat{A}_n`$ of order 3. For $`\sigma =(\sigma _1,\mathrm{},\sigma _r)`$ TC$`(n,g)`$ we have $`\widehat{\sigma }_1\mathrm{}\widehat{\sigma }_r=\pm 1`$. The value of this product is called the lifting invariant of $`\sigma `$. It depends only on the braid orbit of $`\sigma `$. For $`g=0`$ the lifting invariant is $`+1`$ if and only if $`n`$ is odd (by \[Fr1\] and \[Se1\]). For $`g>0`$ the two braid orbits on TC$`(n,g)`$ have distinct lifting invariant.

Now we can refine Theorem 3.3 as follows.

###### Theorem 4.1

Assume $`n6`$, $`g>0`$ and $`n2g+1`$. Then both braid orbits on TC$`(n,g)`$ have full moduli dimension.

Proof : The claim holds for $`g=1`$ by \[FKK\], Comment 0. Now suppose in the situation of Lemma 2.4, $`\stackrel{~}{\sigma }`$ is a tuple in $`A_n`$ with $`\sigma _{r+1}=\sigma _{r+2}^1=(n1,n,n+1)`$. Then clearly $`\sigma `$ and $`\stackrel{~}{\sigma }`$ have the same lifting invariant. Thus the proof of Lemma 3.2 also shows the present refinement, since it iterates the construction of Lemma 2.4.

## 5 The exceptional cases in genus 3

Let $`\sigma =(\sigma _1,\mathrm{},\sigma _r)`$ be an admissible tuple in $`S_n`$, and $`g:=g_\sigma 3`$. Assume $`\sigma `$ satisfies the necessary condition $`r3g`$ for full moduli dimension. Assume further $`\sigma `$ generates a primitive subgroup $`G`$ of $`S_n`$. If $`g4`$ then $`G=S_n`$ or $`G=A_n`$ by \[GM\] and \[GS\]. If $`g=3`$ and $`G`$ is not $`S_n`$ or $`A_n`$ then one of the following holds (see \[GM\], Theorem 2):

1. $`n=7`$, $`GGL_3(2)`$
2. $`n=8`$, $`GAGL_3(2)`$ (the affine group)
3. $`n=16`$, $`GAGL_4(2)`$

Recall that $`GL_3(2)`$ is a simple group of order 168. It acts doubly transitively on the 7 non-zero elements of $`(𝔽_2)^3`$. The affine group $`AGL_m(2)`$ is the semi-direct product of $`GL_m(2)`$ with the group of translations; it acts triply transitively on the affine space $`(𝔽_2)^m`$.

In cases (1) and (3), the tuple $`\sigma `$ consists of 9 transvections of the respective linear or affine group. (A transvection fixes a hyperplane of the underlying linear or affine space point-wise). In case (2), either $`\sigma `$ consists of 10 transvections or it consists of 8 transvections plus an element of order 2,3 or 4 (where the element of order 2 is a translation).

###### Remark 5.1

The tuples in case (1) form a single braid orbit on DT$`(7,3)`$. This braid orbit has full moduli dimension by the Theorem below.

Proof : We show that tuples of 9 involutions generating $`G=GL_3(2)`$ (with product 1) form a single braid orbit. This uses the BRAID program \[MSV\]. Direct application of the program is not possible because the number of tuples is too large.

We first note that if 9 involutions generate $`G`$, then there are 6 among them that generate already (since the lattice of subgroups of $`G`$ has length 6). We can move these 6 into the first 6 positions of the tuple by a sequence of braids. Now we apply the BRAID program to 6-tuples of involutions generating $`G`$ (but not necessarily with product 1). We find that such tuples with any prescribed value of their product form a single braid orbit. By inspection of these braid orbits, we find that each contains a tuple whose first two involutions are equal, and the remaining still generate $`G`$. This reduces the original problem to showing that tuples of 7 involutions with product 1, generating $`G`$, form a single braid orbit. The BRAID program did that.

In cases (1) and (2), the transvections yield double transpositions in $`S_n`$. Thus again Lemma 2.4 can be used to show there actually exist such tuples that have full moduli dimension. Case (3) requires a more complicated argument which will be worked out later.

###### Theorem 5.2

Each general curve of genus 3 admits a cover to $`^1`$ of degree 7 (resp., 8) and monodromy group $`GL_3(2)`$ (resp., $`AGL_3(2)`$ ), branched at 9 (resp., 10) points of $`^1`$, such that all inertia groups are generated by double transpositions.

Proof : Let $`G`$ be a (doubly) transitive subgroup of $`S_7`$ isomorphic to $`GL_3(2)`$. Let $`H`$ ($`S_4`$) be a point stabilizer in $`G`$. View $`H`$ as a subgroup of $`S_6`$ via its (transitive) action on the other 6 points. In 5.1 below, we show there is a tuple $`\tau `$ in $`\text{DT}(6,2)`$ of full moduli dimension that generates this subgroup $`H`$ of $`S_6`$. This tuple has length 7, hence has infinite moduli degree by Remark 2.2(b). Choose a double transposition in $`G`$ that is not in $`H`$, and append two copies of it to the tuple $`\tau `$. By Lemma 2.4, this yields a tuple $`\sigma \text{DT}(7,3)`$ of full moduli dimension, satisfying (1).

The group $`GL_3(2)`$ is the stabilizer of 0 in the transitive action of $`AGL_3(2)`$ on the 8 points of $`(𝔽_2)^3`$. Replacing the last entry $`\sigma _9`$ of the above tuple $`\sigma `$ by two double transpositions from $`AGL_3(2)`$ that are not in $`GL_3(2)`$ and have product $`\sigma _9`$, yields a tuple in $`\text{DT}(8,3)`$ satisfying (2). This tuple has full moduli dimension because already the boundary of the corresponding Hurwitz space maps dominantly to $`_3`$.

### 5.1 Certain covers of degree 6 from the general curve of genus 2 to $`^1`$

Let $`\tau _1,\tau _2,\tau _3`$ be the three double transpositions in $`H:=S_4`$. Let $`\rho _1`$ and $`\rho _2`$ be transpositions in $`H`$ generating an $`S_3`$-subgroup. Then the tuple

$$\tau =(\tau _1,\tau _2,\tau _3,\rho _1,\rho _1,\rho _2,\rho _2)$$

generates $`H`$. View $`H`$ as a subgroup of $`S_6`$ as in the proof of Theorem 5.2. Then $`\tau `$ becomes an element of $`\text{DT}(6,2)`$ (since all involutions of $`GL_3(2)`$ act as double transpositions on the 7 points).

Now consider a cover $`f:X^1`$ of type $`\tau `$. Note that $`H`$ is an imprimitive subgroup of $`S_6`$, permuting 3 blocks of size 2. The kernel of the action of $`H`$ on these 3 blocks equals $`\{1,\tau _1,\tau _2,\tau _3\}`$. Thus $`f`$ factors as $`f=hg`$ where $`g:X^1`$ is of degree 2 (the hyperelliptic map on the genus 2 curve $`X`$) and $`h:^1^1`$ is a simple cover of degree 3 (i.e., its tuple of branch cycles consists of 4 involutions in $`S_3`$). Let $`p_i^1`$, $`i=1,2,3`$ be the branch point of $`f`$ corresponding to $`\tau _i`$. Then $`p_i`$ has 3 distinct pre-images $`x_i,y_i,z_i`$ under $`h`$. We may assume $`p_1=0=x_3`$, $`p_2=\mathrm{}=y_3`$, $`p_3=1=x_1`$. Then $`h`$ is of the form

$$h(x)=\frac{(x1)(xy_1)(xz_1)}{(xx_2)(xy_2)(xz_2)}$$

Exactly one of $`x_i,y_i,z_i`$, say $`z_i`$, is unramified under $`g`$. Thus $`x_1=1,y_1,x_2,y_2,x_3=0,y_3=\mathrm{}`$ are the 6 branch points of the hyperelliptic map $`g`$. It is well-known that (the $`\text{PGL}_2()`$-orbit of) this 6-set determines the isomorphism class of the genus 2 curve $`X`$. Now we are ready to prove:

###### Lemma 5.3

The tuple $`\tau `$ has full moduli dimension.

Proof : It suffices to show that for each choice of $`y_1^{},x_2^{},y_2^{}`$ sufficiently close to $`y_1,x_2,y_2`$, respectively (in the complex topology), the following holds: There are $`z_1^{},z_2^{}`$ close to $`z_1,z_2`$, respectively, such that the map

$$h^{}(x)=\frac{(x1)(xy_1^{})(xz_1^{})}{(xx_2^{})(xy_2^{})(xz_2^{})}$$

composed with the double cover $`g^{}:X^{}^1`$ branched at $`y_1^{},x_2^{},y_2^{},0,\mathrm{},1`$ is a cover of type $`\tau `$. This follows by continuity once we know that the condition $`h^{}(0)=1(=h^{}(\mathrm{}))`$ is preserved. But this condition $`h^{}(0)=1`$ is easy to achieve: We can view it as defining $`z_2^{}`$ (after free choice of $`z_1^{}`$).


text:
# Heat Kernels and the Index Theorems onEven and Odd Dimensional ManifoldsPartially supported by the MOEC and the 973 Project.

## 1. Introduction

As is well-known, the index theorem proved by Atiyah and Singer \[AS1\] in 1963, which expresses the analytically defined index of elliptic differential operators through purely topological terms, has had a wide range of implications in mathematics as well as in mathematical physics. Moreover, there have been up to now many different proofs of this celebrated result.

The existing proofs of the Atiyah-Singer index theorem can roughly be divided into three categories:

(i) The cobordism proof: this is the proof originally given in \[AS1\]. It uses the cobordism theory developed by Thom and modifies Hirzebruch’s proof of his Signature theorem as well as his Riemann-Roch theorem;

(ii) The $`K`$-theoretic proof: this is the proof given by Atiyah and Singer in \[AS2\]. It modifies Grothendieck’s proof of the Hirzebruch-Riemann-Roch theorem and relies on the topological $`K`$-theory developed by Atiyah and Hirzebruch. The Bott periodicity theorem plays an important role in this proof;

(iii) The heat kernel proof: this proof originates from a simple and beautiful formula due to Mckean and Singer \[MS\], and has closer relations with differential geometry as well as mathematical physics. It also lead directly to the important Atiyah-Patodi-Singer index theorem for Dirac operators on manifolds with boundary.

In this article, we will survey some of the developments concerning the heat kernel proofs of various index theorems, including a recent result with Dai \[DZ2\] on an index theorem for Toeplitz operators on odd dimensional manifolds with boundary.

## 2. Heat kernels and the index theorems on even dimensional manifolds

We start with a smooth closed oriented $`2n`$-dimensional manifold $`M`$ and two smooth complex vector bundles $`E`$, $`F`$ over $`M`$, on which there is an elliptic differential operator between the spaces of smooth sections, $`D_+:\mathrm{\Gamma }(E)\mathrm{\Gamma }(F).`$

If we equip $`TM`$ with a Riemannian metric and $`E`$, $`F`$ with Hermitian metrics repectively, then $`\mathrm{\Gamma }(E)`$ and $`\mathrm{\Gamma }(F)`$ will carry canonically induced inner products.

Let $`D_{}:\mathrm{\Gamma }(F)\mathrm{\Gamma }(E)`$ be the formal adjoint of $`D_+`$ with respect to these inner products. Then the index of $`D_+`$ is given by

$$\mathrm{ind}D_+=dim\left(\mathrm{ker}D_+\right)dim\left(\mathrm{ker}D_{}\right).$$
$`(2.1)`$
It is a topological invariant not depending on the metrics on $`TM`$, $`E`$ and $`F`$.

The famous Mckean-Singer formula \[MS\] says that $`\mathrm{ind}D_+`$ can also be computed by using the heat operators associated to the Laplacians $`D_{}D_+`$ and $`D_+D_{}`$. That is, for any $`t>0`$, one has

$$\mathrm{ind}D_+=\mathrm{Tr}\left[\mathrm{exp}\left(tD_{}D_+\right)\right]\mathrm{Tr}\left[\mathrm{exp}\left(tD_+D_{}\right)\right].$$
$`(2.2)`$
By introducing the $`𝐙_2`$-graded vector bundle $`EF`$ and setting $`D=\left(\begin{array}{cc}0& D_{}\\ D_+& 0\end{array}\right)`$, we can rewrite the difference of the two traces in the right hand side of (2.2) as a single “supertrace” as follows,

$$\mathrm{ind}D_+=\mathrm{Tr}_s\left[\mathrm{exp}\left(tD^2\right)\right],\text{for any }t>0.$$
$`(2.2)^{}`$
Let $`P_t(x,y)`$ be the smooth kernel of $`\mathrm{exp}(tD^2)`$ with respect to the volume form on $`M`$. For any $`f\mathrm{\Gamma }(EF)`$, one has

$$\mathrm{exp}\left(tD^2\right)f(x)=_MP_t(x,y)f(y)𝑑y.$$
$`(2.3)`$
In particular,

$$\mathrm{Tr}_s\left[\mathrm{exp}\left(tD^2\right)\right]=_M\mathrm{Tr}_s\left[P_t(x,x)\right]𝑑x.$$
$`(2.4)`$
Now, for simplicity, we assume that the elliptic operator $`D`$ is of order one. Then by a standard result, which goes back to Minakshisundaram and Pleijiel \[MP\], one has that when $`t>0`$ tends to $`0`$,

$$P_t(x,x)=\frac{1}{(4\pi t)^n}\left(a_n+a_{n+1}t+\mathrm{}+a_0t^n+o_x\left(t^n\right)\right),$$
$`(2.5)`$
where $`a_i\mathrm{End}((EF)_x)`$, $`i=n,\mathrm{},0`$.

By (2.2)’, (2.4) and (2.5), and by taking $`t>0`$ small enough, one deduces that

$$_M\mathrm{Tr}_s[a_i]𝑑x=0,ni<0,$$

$$\mathrm{ind}D_+=\left(\frac{1}{4\pi }\right)^n_M\mathrm{Tr}_s[a_0]𝑑x.$$
$`(2.6)`$
Mckean and Singer conjectured in \[MS\] that for certain geometric operators, there should be some “fantastic cancellation” so that the following far reaching refinement of (2.6) holds,

$$\mathrm{Tr}_s[a_i]=0,ni<0,$$

and moreover, $`\mathrm{Tr}_s[a_0]`$ can be calculated simply in the Chern-Weil geometric theory of characteristic classes.

In fact, as a typical example, let $`M`$ be an even dimensional compact smooth oriented spin manifold carrying a Riemannian metric $`g^{TM}`$. Let $`R^{TM}`$ be the curvature of the Levi-Civita connection associated to $`g^{TM}`$. Let $`S(TM)=S_+(TM)`$ $`S_{}(TM)`$ be the Hermitian vector bundle of $`(TM,g^{TM})`$-spinors, and $`D_+:\mathrm{\Gamma }(S_+(TM))\mathrm{\Gamma }(S_{}(TM))`$ the associated Dirac operator.

One then has the formula (cf. \[BGV, Chap. 4, 5\]),

$$\underset{t0}{lim}\mathrm{Tr}_s\left[P_t(x,x)\right]dx=\left\{\widehat{A}\left(\frac{R^{TM}}{2\pi }\right)\right\}^{\mathrm{max}}:=\left\{\stackrel{1/2}{det}\left(\frac{\frac{\sqrt{1}}{4\pi }R^{TM}}{\mathrm{sinh}\left(\frac{\sqrt{1}}{4\pi }R^{TM}\right)}\right)\right\}^{\mathrm{max}},$$
$`(2.7)`$
which implies the Atiyah-Singer index theorem \[AS1\] for $`D_+`$:

$$\mathrm{ind}D_+=\widehat{A}(M):=_M\widehat{A}\left(\frac{R^{TM}}{2\pi }\right).$$
$`(2.8)`$
A result of type (2.7) is called a local index theorem. The first proof of such a local result was given by V. K. Patodi \[P\] for the de Rham-Hodge operator $`d+d^{}`$. Other direct heat kernel proofs of (2.7) have been given by Berline-Vergne, Bismut, Getzler and Yu respectively. We refer to \[BGV\] and \[Yu\] for more details.

The heat kernel proof of the local index theorem leads to a generalization of the index theorem for Dirac operators to the case of manifolds with boundary. This was achieved by Atiyah, Patodi and Singer in \[APS\], and will be reviewed in the next section.

## 3. The index theorem for Dirac operators on even dimensional manifolds with boundary

Let $`M`$ be a smooth compact oriented even dimensional spin manifold with (nonempty) smooth boundary $`M`$. Then $`M`$ is again oriented and spin.

Let $`g^{TM}`$ be a metric on $`TM`$. Let $`g^{TM}`$ be its restriction on $`TM`$. We assume for simplicity that $`g^{TM}`$ is of product structure near the boundary $`M`$. Let $`S(TX)=S_+(TX)S_{}(TX)`$ be the $`𝐙_2`$-graded Hermitian vector bundle of $`(TX,g^{TX})`$-spinors.

Since now $`M`$ has a nonempty boundary $`M`$, the associated Dirac operator $`D_+:\mathrm{\Gamma }(S_+(TM))\mathrm{\Gamma }(S_{}(TM))`$ is not elliptic. To get an elliptic problem, one needs to introduce an elliptic boundary condition for $`D_+`$, and this was achieved by Atiyah, Patodi and Singer in \[APS\]. It is remarkable that this boundary condition, to be described right now, is global in nature.

First of all, the Dirac operator $`D_+`$ induces canonically a formally self-adjoint first order elliptic differential operator

$$D_M:\mathrm{\Gamma }\left(S_+(TM)|_M\right)\mathrm{\Gamma }\left(S_+(TM)|_M\right),$$

which is called the induced Dirac operator on the boundary $`M`$.

Clearly, the $`L^2`$-completion of $`S_+(TM)|_M`$ admits an orthogonal decomposition

$$L^2\left(S_+(TM)|_X\right)=\underset{\lambda \mathrm{Spec}(D_M)}{}E_\lambda ,$$
$`(3.1)`$
where $`E_\lambda `$ is the eigenspace of $`\lambda `$.

Let $`L_0^2(S_+(TM)|_M)`$ denote the direct sum of the eigenspaces $`E_\lambda `$ associated to the eigenvalues $`\lambda 0`$. Let $`P_0`$ denote the orthogonal projection from $`L^2(S_+(TM)|_M)`$ to $`L_0^2(S_+(TM)|_M)`$. We call $`P_0`$ the Atiyah-Patodi-Singer projection associated to $`D_M`$, to emphasize its role in \[APS\].

Then by \[APS\], the boundary problem

$$(D_+,P_0):\{u:u\mathrm{\Gamma }(S_+(TM)),P_0\left(u|_M\right)=0\}\mathrm{\Gamma }(S_{}(TM)),$$
$`(3.2)`$
is Fredholm. We call this elliptic boundary problem the Atiyah-Patodi-Singer boundary problem associated to $`D_+`$. We denote by $`\mathrm{ind}(D_+,P_0)`$ the index of the Fredholm operator (3.2).

The Atiyah-Patodi-Singer index theorem The following identity holds,

$$\mathrm{ind}(D_+,P_0)=_M\widehat{A}\left(\frac{R^{TM}}{2\pi }\right)\overline{\eta }\left(D_M\right).$$
$`(3.3)`$
The boundary correction term $`\overline{\eta }(D_M)`$ appearing in the right hand side of (3.3) is a spectral invariant associated to the induced Dirac operator $`D_M`$ on $`M`$. It is defined as follows: for any complex number $`s𝐂`$ with $`\mathrm{Re}(s)>dimM`$, define

$$\eta (D_M,s)=\underset{\lambda \mathrm{Spec}(D_M)}{}\frac{\mathrm{sgn}(\lambda )}{|\lambda |^s}.$$
$`(3.4)`$
By using the heat kernel method, one can show easily that $`\eta (D_M,s)`$ can be extended to a meromorphic function on $`𝐂`$, which is holomorphic at $`s=0`$. Following \[APS\], we then define

$$\overline{\eta }\left(D_M\right)=\frac{dim\left(\mathrm{ker}D_M\right)+\eta (D_M,0)}{2}$$
$`(3.5)`$
and call it the (reduced) eta invariant of $`D_M`$.

The eta invariants of Dirac operators have played important roles in many aspects of topology, geometry and mathematical physics.

In the next sections, we will discuss the role of eta invariants in the heat kernel approaches to the index theorems on odd dimensional manifolds.

## 4. Heat kernels and the index theorem on odd dimensional manifolds

Let $`M`$ be now an odd dimensional smooth closed oriented spin manifold. Let $`g^{TM}`$ be a Riemannian metric on $`TM`$ and $`S(TM)`$ the associated Hermitian vector bundle of $`(TM,g^{TM})`$-spinors.111Since now $`M`$ is of odd dimension, the bundle of spinors does not admit a $`𝐙_2`$-graded structure. In this case, the associated Dirac operator $`D:\mathrm{\Gamma }(TM)\mathrm{\Gamma }(TM)`$ is (formally) self-adjoint.222In fact, if $`M`$ bounds an even dimensional spin manifold, then $`D`$ can be thought of as the induced Dirac operator on boundary appearing in the previous section. Thus, one can proceed as in Section 3 to construct the Atiyah-Patodi-Singer projection

$$P_0:L^2(S(TM))L_0^2(S(TM)).$$

Now consider the trivial vector bundle $`𝐂^N`$ over $`M`$. We equip $`𝐂^N`$ with the canonical trivial metric and connection. Then $`P_0`$ extends naturally to an orthogonal projection from $`L^2(S(TM)𝐂^N)`$ to $`L_0^2(S(TM)𝐂^N)`$ by acting as identity on $`𝐂^N`$. We still denote this extension by $`P_0`$.

On the other hand, let

$$g:MU(N)$$

be a smooth map from $`M`$ to the unitary group $`U(N)`$. Then $`g`$ can be interpreted as automorphism of the trivial complex vector bundle $`𝐂^N`$. Moreover $`g`$ extends naturally to an action on $`L^2(S(TM)𝐂^N)`$ by acting as identity on $`L^2(S(TM))`$. We still denote this extended action by $`g`$.

With the above data given, one can define a Toeplitz operator $`T_g`$ as follows,

$$T_g=P_0gP_0:L_0^2\left(S(TM)𝐂^N\right)L_0^2\left(S(TM)𝐂^N\right).$$
$`(4.1)`$
The first important fact is that $`T_g`$ is a Fredholm operator. Moreover, it is equivalent to an elliptic pseudodifferential operator of order zero. Thus one can compute its index by using the Atiyah-Singer index theorem \[AS2\], as was indicated in the paper of Baum and Douglas \[BD\], and the result is

$$\mathrm{ind}T_g=\widehat{A}(TM)\mathrm{ch}(g),[M],$$
$`(4.2)`$
where $`\mathrm{ch}(g)`$ is the odd Chern character associated to $`g`$.

There is also an analytic proof of (4.2) by using heat kernels. For this one first applies a result of Booss and Wojciechowski (cf. \[BW\]) to show that the computation of $`\mathrm{ind}T_g`$ is equivalent to the computation of the spectral flow of the linear family of self-adjoint elliptic operators, acting of $`\mathrm{\Gamma }(S(TM)𝐂^N)`$, which connects $`D`$ and $`gDg^1`$. The resulting spectral flow can then be computed by variations of $`\eta `$-invariants, where the heat kernels are naturally involved.

The above ideas have been extended in \[DZ1\] to give a heat kernel proof of a family extension of (4.2).

## 5. An index theorem for Toeplitz operators on odd dimensional manifolds with boundary

In this section, we describe an extension of (4.2) to the case of manifolds with boundary, which was proved recently in my paper with Xianzhe Dai \[DZ2\]. This result can be thought of as an odd dimensional analogue of the Atiyah-Patodi-Singer index theorem described in Section 3.

This section is divided into three subsections. In Subsection 4.1, we extend the definition of Toeplitz operators to the case of manifolds with boundary. In Subsection 4.2, we define an $`\eta `$-invariant for cylinders which will appear in the statement of the main result to be described in Subsection 4.3.

### 5.1. Toeplitz operators on manifolds with boundary

Let $`M`$ be an odd dimensional oriented spin manifold with (nonempty) boundary $`M`$. Then $`M`$ is also oriented and spin. Let $`g^{TM}`$ be a Riemannian metric on $`TM`$ such that it is of product structure near the boundary $`M`$. Let $`S(TM)`$ be the Hermitian bundle of spinors associated to $`(M,g^{TM})`$. Since $`M\mathrm{}`$, the Dirac operator $`D:\mathrm{\Gamma }(S(TM))\mathrm{\Gamma }(S(TM))`$ is no longer elliptic. To get an elliptic operator, one needs to impose suitable boundary conditions, and it turns out that again we will adopt the boundary conditions introduced by Atiyah, Patodi and Singer \[APS\].

Let $`D_M:\mathrm{\Gamma }(S(TM)|_M)\mathrm{\Gamma }(S(TM)|_M)`$ be the canonically induced Dirac operator on the boundary $`M`$. Then $`D_M`$ is elliptic and (formally) self-adjoint. For simplicity, we assume here that $`D_M`$ is invertible, that is, $`\mathrm{ker}D_M=0`$.

Let $`P_{M,0}`$ denote the Atiyah-Patodi-Singer projection from $`L^2(S(TM)|_M)`$ to $`L_0^2(S(TM)|_M)`$. Then $`(D,P_{M,0})`$ forms a self-adjoint elliptic boundary problem. We will also denote the corresponding elliptic self-adjoint operator by $`D_{P_{M,0}}`$.

Let $`L_{P_{M,0},0}^2(S(TM))`$ be the space of the direct sum of eigenspaces of non-negative eigenvalues of $`D_{P_{M,0}}`$. Let $`P_{P_{M,0},0}`$ denote the orthogonal projection from $`L^2(S(TM))`$ to $`L_{P_{M,0},0}^2(S(TM))`$.

Now let $`𝐂^N`$ be the trivial complex vector bundle over $`M`$ of rank $`N`$, which carries the trivial Hermitian metric and the trivial Hermitian connection. We extend $`P_{P_{M,0},0}`$ to act as identity on $`𝐂^N`$.

Let $`g:MU(N)`$ be a smooth unitary automorphism of $`𝐂^N`$. Then $`g`$ extends to an action on $`S(TM)𝐂^N`$ by acting as identity on $`S(TM)`$.

Since $`g`$ is unitary, one verifies easily that the operator $`gP_{M,0}g^1`$ is an orthogonal projection on $`L^2((S(TM)𝐂^N)|_M)`$, and that $`gP_{M,0}g^1P_{M,0}`$ is a pseudodifferential operator of order less than zero. Moreover, the pair $`(D,gP_{M,0}g^1)`$ forms a self-adjoint elliptic boundary problem. We denote its associated elliptic self-adjoint operator by $`D_{gP_{M,0}g^1}`$.

Let $`L_{gP_{M,0}g^1,0}^2(S(TM)𝐂^N)`$ be the space of the direct sum of eigenspaces of nonnegative eigenvalues of $`D_{gP_{M,0}g^1}`$. Let $`P_{gP_{M,0}g^1,0}`$ denote the orthogonal projection from $`L^2(S(TM)𝐂^N)`$ to $`L_{gP_{M,0}g^1,0}^2(S(TM)𝐂^N)`$.

Clearly, if $`sL^2(S(TM)𝐂^N)`$ verifies $`P_{M,0}(s|_M)=0`$, then $`gs`$ verifies

$$gP_{M,0}g^1\left((gs)|_M\right)=0.$$

Definition 5.1 The Toeplitz operator $`T_g`$ is defined by

$$T_g=P_{gP_{M,0}g^1,0}gP_{P_{M,0},0}:$$

$$L_{P_{M,0},0}^2\left(S(TM)𝐂^N\right)L_{gP_{M,0}g^1,0}^2\left(S(TM)𝐂^N\right).$$

One verifies that $`T_g`$ is a Fredholm operator. The main result of this section evaluates the index of $`T_g`$ by more geometric quantities.

### 5.2. An $`𝜼`$-invariant associated to $`𝒈`$

We consider the cylinder $`[0,1]\times M`$. Clearly, the restriction of $`g`$ on $`M`$ extends canonically to this cylinder.

Let $`D|_{[0,1]\times M}`$ be the restriction of $`D`$ on $`[0,1]\times M`$. We equip the boundary condition $`P_{M,0}`$ at $`\{0\}\times M`$ and the boundary condition $`\mathrm{Id}gP_{M,0}g^1`$ at $`\{1\}\times M`$. Then $`(D|_{[0,1]\times M},P_{M,0},\mathrm{Id}gP_{M,0}g^1)`$ forms a self-adjoint elliptic boundary problem. We denote the corresponding elliptic self-adjoint operator by $`D_{P_{M,0},gP_{M,0}g^1}`$.

Let $`\eta (D_{P_{M,0},gP_{M,0}g^1},s)`$ be the $`\eta `$-function of $`s𝐂`$ which, when $`\mathrm{Re}(s)>>0`$, is defined by

$$\eta (D_{P_{M,0},gP_{M,0}g^1},s)=\underset{\lambda 0}{}\frac{\mathrm{sgn}(\lambda )}{|\lambda |^s},$$

where $`\lambda `$ runs through the nonzero eigenvalues of $`D_{P_{M,0},gP_{M,0}g^1}`$.

It is proved in \[DZ2\] that under our situation, $`\eta (D_{P_{M,0},gP_{M,0}g^1},s)`$ can be extended to a meromorphic function on $`𝐂`$ which is holomorphic at $`s=0`$.

Let $`\overline{\eta }(D_{P_{M,0},gP_{M,0}g^1})`$ be the reduced $`\eta `$-invariant defined by

$$\overline{\eta }\left(D_{P_{M,0},gP_{M,0}g^1}\right)$$

$$=\frac{dim\mathrm{ker}\left(D_{P_{M,0},gP_{M,0}g^1}\right)+\eta \left(D_{P_{M,0},gP_{M,0}g^1}\right)}{2}.$$

### 5.3. An index theorem for $`𝑻_𝒈`$

Let $`^{TM}`$ be the Levi-Civita connection associated to the Riemannian metric $`g^{TM}`$. Let $`R^{TM}=(^{TM})^2`$ be the curvature of $`^{TM}`$. Also, we use $`d`$ to denote the trivial connection on the trivial vector bundle $`𝐂^N`$ over $`M`$. Then $`g^1dg`$ is a $`\mathrm{\Gamma }(\mathrm{End}(𝐂^N))`$ valued 1-form over $`M`$.

Let $`\mathrm{ch}(g,d)`$ denote the odd Chern character form (cf. \[Z\]) of $`(g,d)`$ defined by

$$\mathrm{ch}(g,d)=\underset{n=0}{\overset{(dimM1)/2}{}}\frac{n!}{(2n+1)!}\left(\frac{1}{2\pi \sqrt{1}}\right)^{n+1}\mathrm{Tr}\left[\left(g^1dg\right)^{2n+1}\right].$$

Let $`𝒫_M`$ denote the Calderón projection associated to $`D`$ on $`M`$ (cf. \[BW\]). Then $`𝒫_M`$ is an orthogonal projection on $`L^2((S(TM)𝐂^N)|_M)`$, and that $`𝒫_MP_{M,0}`$ is a pseudodifferential operator of order less than zero.

Let $`\tau _\mu (P_{M,0},gP_{M,0}g^1,𝒫_M)𝐙`$ be the Maslov triple index in the sense of Kirk and Lesch \[KL, Definition 6.8\].

We can now state the main result of \[DZ2\], which generalizes an old result of Douglas and Wojciechowski \[DoW\], as follows.

Theorem 5.2 The following identity holds,

$$\mathrm{ind}T_g=_M\widehat{A}\left(\frac{R^{TM}}{2\pi }\right)\mathrm{ch}(g,d)+\overline{\eta }\left(D_{P_{M,0},gP_{M,0}g^1}\right)$$

$$\tau _\mu (P_{M,0},gP_{M,0}g^1,𝒫_M).$$

The following immediate consequence is of independent interests.

Corollary 5.3 The number

$$_M\widehat{A}\left(\frac{R^{TM}}{2\pi }\right)\mathrm{ch}(g,d)\overline{\eta }\left(D_{P_{M,0},gP_{M,0}g^1}\right)$$

is an integer.

The strategy of the proof of Theorem 5.2 given in \[DZ2\] is the same as that of the heat kernel proof of (4.2). However, due to the appearance of the boundary $`M`$, one encounters new difficulties. To overcome these difficulties, one makes use of the recent result on the splittings of $`\eta `$ invariants (cf. \[KL\]) as well as some ideas involved in the Connes-Moscovici local index theorem in noncommutative geometry \[CM\] (see also \[CH\]). Moreover, the local index calculations appearing near $`M`$ is highly nontrivial. We refer to \[DZ2\] for more details.