text:
# Rationalizing right-handed neutrinos

## Abstract

A simple argument based on an SU(3) gauged horizontal symmetry is presented that connects the explanation for three generations of matter with the existence of a triplet of right-handed neutrinos. This rationale for right-handed neutrinos is analogous to, but completely independent of, grand unification or extra universal dimensions. A brief discussion of the supersymmetrized SU(3) model is also given, pointing out that certain problems in ordinary supersymmetric models such as fast proton decay via dimension-5 Planck-suppressed operators can be naturally solved.

preprint: hep-ph/0304256

One of the most remarkable features of the Standard Model (SM) is that matter fermions are chiral and yet all gauge gauge ; witten  and gravitational aw  anomalies vanish for each generation. A known but not often emphasized fact about the matter content is that, given one generation with unfixed hypercharges, anomaly cancellation determines the relative hypercharge assignment to be precisely what has been established by experiment SManomalies . In other words, electric charge quantization is essentially automatic without grand unification. This fact, taken at face value, is circumstantial evidence against the existence of right-handed neutrinos. By definition a candidate for a right-handed neutrino is any fermion that is uncharged under all of the SM gauge symmetries. Yet, gauge symmetries are precisely the reason that each type of matter ($`Q`$, $`u`$, $`d`$, $`L`$, $`e`$) is tied with the other matter fields together in a self-consistent, exclusive fashion. In addition, non-chiral matter allows a new mass scale unconnected to electroweak symmetry breaking that only further complicates our understanding of mass generation and mass hierarchies. Extensions of the SM with non-chiral matter, such as adding right-handed neutrinos, therefore appear to be contrary to all of the guiding wisdom gleaned from experiment, at least until recently. (Those who are still in doubt need only observe the agony that the $`\mu `$ problem causes avatars of supersymmetry.)

Neutrino experiments superk ; sno ; kamland , however, have firmly established that the neutrinos oscillate between each generation and thus they have mass. The largest mass of any one neutrino is constrained to be less than about 2 eV mass , and more likely their mass is one to a few orders of magnitude below this, depending on the generation. The mechanism of mass generation for neutrinos remains a mystery. If neutrinos acquire mass analogously to the SM matter fermions, preserving lepton number, then the particle content must be extended with at least two right-handed neutrinos $`\nu _{1,2}`$. Ordinary Yukawa terms $`=\lambda _\nu LH\nu ^c`$ with tiny couplings $`\lambda _\nu 10^{12}`$ suffice to explain the two undisputed mass differences found in neutrino oscillation experiments.

But, the global symmetry behind lepton number conservation is not expected to be exact. At dimension-5, the operator $`HLHL/M`$ violates lepton number by two units and leads to a tiny Majorana mass $`v^2/M`$ for left-handed neutrinos. This transmutes the neutrino mass hierarchy problem from explaining $`\lambda _\nu 10^{12}`$ to instead explaining $`v/M10^{12}`$. To embrace the dimension-5 neutrino mass explanation means the SM effective theory breaks down at $`M10^{14}`$ GeV. This is somewhat disconcerting since there are dimension-6 operators that violate lepton and baryon number, leading to a proton decay rate that is excluded by experiment unless $`M10^{16}`$ GeV. Hence, while lepton number must be be violated at $`M`$ to explain neutrino masses, baryon number must be preserved to keep the proton stable. The simplest phenomenological explanation for lepton number violation without baryon number violation at the cutoff scale $`M`$ is to add right-handed neutrinos to the SM with ordinary Yukawa couplings, plus a heavy Majorana mass term $`=M\nu ^c\nu `$ for the right-handed neutrinos. The resulting combination of a Dirac mass and a heavy Majorana mass leads to the famous “see-saw” neutrino mass matrix seesaw

$$\left(\begin{array}{cc}0& \lambda _\nu v\\ \lambda _\nu v& M\end{array}\right).$$
(1)
Diagonalizing this mass matrix, or equivalently integrating out the right-handed neutrino gives back the SM plus the dimension-5 operator $`HLHL/M`$ with a well-defined coefficient, $`\lambda _\nu ^2`$, and thus predicted neutrino mass, $`\lambda _\nu ^2v^2/M`$. A right-handed neutrino with Majorana mass $`M`$ therefore provides an ultraviolet completion of the effective theory beyond the cutoff $`M`$, explaining why only lepton number was violated at $`M`$.

The difficulty with such neutrino mass generation mechanisms is that they do not really solve the neutrino mass hierarchy problem, and worse still, require precisely those odd-ball fields – right-handed neutrinos – that are unconnected to SM matter through gauge anomalies. Furthermore, the see-saw explanation requires a new Majorana mass scale unconnected with electroweak symmetry breaking. These facts would ordinarily be highly distressing except for a remarkable coincidence: the scale $`M10^{14}`$ GeV is tantalizingly close to where the SM gauge couplings come to an approximate intersection. Such an intersection is predicted by grand unified theories (GUTs), providing justification for the new scale. Furthermore, in an SO(10) GUT each right-handed neutrino is elegantly fused with each generation of SM matter into a single $`\mathrm{𝟏𝟔}`$ representation so10 . This is really just an artifact of unifying into a GUT group with rank greater than that of the SM, since candidates for right-handed neutrinos in GUTs are those fields uncharged under the SM symmetries but charged under some additional gauge symmetry \[for SO(10) this is the extra U(1) under the decomposition SO(10) $``$ SU(5) $`\times `$ U(1)\]. Rank $`>4`$ GUTs therefore provide a rationale for $`n=0`$ mod $`3`$ right-handed neutrinos whenever each generation is unified into a single representation of the group.

Unfortunately, grand unification has many well-known problems of implementation. Non-supersymmetric grand unification proposals suffer from the hierarchy problem as well as a rather inexact unification of gauge couplings. Both non-supersymmetric and supersymmetric unification models predict proton decay at a rate that has been experimentally ruled out in the simplest models. Also, several theoretical problems pervade unification ranging from understanding how the Higgs is embedded into a GUT representation (the doublet-triplet splitting problem), how (or if) Yukawa couplings are unified, etc. Such experimental and theoretical problems ought to induce us to reconsider GUTs as the origin of right-handed neutrinos and the Majorana mass scale. Is there any rationale *independent* of unification that predicts right-handed neutrinos as well as the Majorana mass scale?

Suppose the explanation for the number of generations is that each field’s three generations ($`Q_{1,2,3}`$, $`u_{1,2,3}`$, etc.) correspond to three components of a multiplet of a “horizontal” flavor symmetry. There are only two continuous symmetries that are suitable for this purpose possessing a $`\mathrm{𝟑}`$ representation: SU(3) y  and SU(2) $``$ SO(3) wz ; ty . SU(2) can be summarily dismissed if right-handed neutrinos are required to be in a chiral representation of the new symmetry. It has already been emphasized that non-chiral fermions, and right-handed neutrinos in particular, seem to have no (aesthetic) place in the SM if anomaly cancellation is to connect all matter together. There is no hope with SU(2) since it is anomaly-free. SU(2) also does not predict the number of generations since representations of any dimension are possible so3comment . Instead, SU(3) admits chiral fermions with only certain dimensionality – there can be three but not two, four, five, seven, etc. generations. Moreover, SU(3) provides two additional key ingredients: (1) there is an additional anomaly cancellation condition on the matter content if SU(3) is at least weakly gauged, and (2) all fermion masses, including right-handed neutrino masses, arise from spontaneous symmetry breaking. Before proceeding, note that the connection between SU(3) anomaly cancellation and the existence of right-handed neutrinos was made some time ago in y . In this paper the argument in presented in detail, contrasting with grand unification and universal extra dimensions, and then implications for a supersymmetrized version are briefly discussed.

Gauging a new symmetry in which SM fermions transform is non-trivial and requires the cancellation of all gauge anomalies associated with the new symmetry. There are potentially eight new gauge anomalies associated with SU(3)f: \[SU(3)f\]3; SU(3)f $`\times `$ \[SU(3)c\]2; \[SU(3)f\]2 $`\times `$ SU(3)c; SU(3)f $`\times `$ \[SU(2)L\]2; \[SU(3)f\]2 $`\times `$ SU(2)L; SU(3)f $`\times `$ \[U(1)Y\]2; \[SU(3)f\]2 $`\times `$ U(1)Y; and SU(3)f $`\times `$ \[grav\]2. Six of these are trivially zero since $`\mathrm{tr}[t^a]=0`$ for SU(N) gauge groups with $`N>1`$. This leaves the mixed flavor symmetry/hypercharge anomaly \[SU(3)f\]2 $`\times `$ U(1)Y, and the \[SU(3)f\]3 anomaly. The mixed anomaly leads to a condition on the sum of the hypercharges of the SM fermions that is equivalent to the mixed \[grav\]2 $`\times `$ U(1)Y anomaly

$`[SU(3)_f]^2\times U(1)_Y:`$ $`\left(6Y_Q+3Y_u+3Y_d+2Y_L+Y_e\right)=0,`$
and so automatically cancels. The \[SU(3)f\]3 anomaly, however, does not cancel with just the SM fermion content wz ; y .

This is straightforward to see: Five types of matter ($`Q`$, $`u`$, $`d`$, $`L`$, $`e`$) can be assigned to either $`\mathrm{𝟑}`$ or $`\overline{\mathrm{𝟑}}`$ representations. Two of the five fields contribute an even number of $`\mathrm{𝟑}`$’s or $`\overline{\mathrm{𝟑}}`$’s to the anomaly ($`Q:\pm 6`$; $`L:\pm 2`$) while the remaining three fields contribute an odd number of $`\mathrm{𝟑}`$’s or $`\overline{\mathrm{𝟑}}`$’s ($`u:\pm 3`$, $`d:\pm 3`$, $`e:\pm 1`$). The sum of two even numbers and three odd numbers is an odd number, and so \[SU(3)f\]3 anomaly $`an_\mathrm{𝟑}n_{\overline{\mathrm{𝟑}}}`$ cannot be canceled no matter how SM matter is assigned to SU(3)f.

The simplest assignment of matter in $`\mathrm{𝟑}`$ or $`\overline{\mathrm{𝟑}}`$ representations allows ordinary Yukawa couplings. Here the Higgs scalar doublet is assumed to be a singlet under SU(3)f, since there is no need for (and various reasons that disfavor) more than one Higgs doublet in the SM. Gauge invariance of the three Yukawa couplings of the SM implies three relations among the anomaly coefficients of SM matter

$`QHu^c`$ $``$ $`a(Q)+a(u^c)=0`$ (2)
$`QH^{}d^c`$ $``$ $`a(Q)+a(d^c)=0`$ (3)
$`LH^{}e^c`$ $``$ $`a(L)+a(e^c)=0.`$ (4)
Without loss of generality $`Q`$ can be chosen to be a $`\mathrm{𝟑}`$, then $`u^c`$ and $`d^c`$ must both be $`\overline{\mathrm{𝟑}}`$’s. There are two choices for the leptons: \[$`L(\mathrm{𝟑})`$, $`e^c(\overline{\mathrm{𝟑}})`$\] or \[$`L(\overline{\mathrm{𝟑}})`$, $`e^c(\mathrm{𝟑})`$\]. In either case, the \[SU(3)f\]3 anomaly coefficient becomes

$$a=633\pm (21)=\pm 1.$$
(5)
Notice that the anomaly associated with colored fermions self-cancels, but with the leptons it does not cancel regardless of assigning $`(L,e^c)`$ into a $`(\mathrm{𝟑},\overline{\mathrm{𝟑}})`$ or $`(\overline{\mathrm{𝟑}},\mathrm{𝟑})`$.

Intriguingly, the \[SU(3)f\]3 anomaly is canceled by adding a single new field that transforms as a $`\overline{\mathrm{𝟑}}`$ \[for $`L(\mathrm{𝟑})`$, $`e^c(\overline{\mathrm{𝟑}})`$\] or $`\mathrm{𝟑}`$ \[for $`L(\overline{\mathrm{𝟑}})`$, $`e^c(\mathrm{𝟑})`$\] under SU(3)f. To avoid spoiling the SM anomaly cancellation conditions this field must be neutral under SM gauge symmetries. Hence, this anomaly-cancellation field has precisely the quantum numbers of a right-handed neutrino. Also, a Yukawa interaction connecting the left-handed with the right-handed neutrino, $`=LH\nu ^c`$, is automatically allowed by SU(3)f gauge invariance regardless of the initial choice of ($`\mathrm{𝟑}`$, $`\overline{\mathrm{𝟑}}`$, $`\overline{\mathrm{𝟑}}`$) versus ($`\overline{\mathrm{𝟑}}`$, $`\mathrm{𝟑}`$, $`\mathrm{𝟑}`$) for ($`L`$, $`e^c`$, $`\nu ^c`$).

This is a remarkable result. Let me restate the assumptions and the implication: Assuming (1) the explanation for the number of generations is a gauged SU(3)f flavor symmetry, (2) all matter is assigned to chiral representations ($`\mathrm{𝟑}`$ or $`\overline{\mathrm{𝟑}}`$) of SU(3)f, and (3) ordinary Yukawa couplings are SU(3)f gauge invariant, then there *must* exist one set of right-handed neutrinos $`\nu _{1,2,3}`$ transforming as a triplet of SU(3)f.

It is important to emphasize that this flavor symmetry rationale for right-handed neutrinos is completely independent of grand unification. In fact, the simplest assignment that allows Yukawa couplings to be gauge invariant under SU(3)f does not commute with the usual matter embeddings in unified representations of GUTs. For example, SU(5) \[as well as SO(10) and $`E_6`$\] unifies $`Q`$ and $`u`$ into a single representation; this is inconsistent with the SU(3)f assignment given above. However, Yukawa couplings are notoriously over-constrained in GUTs as well as flavor symmetry models. SU(5) predicts the down and lepton Yukawas of each generation should unify, and SO(10) predicts up, down, and lepton Yukawas to unify. These predictions are badly broken at low energies, and not much better at the GUT scale for all but perhaps $`\lambda _b`$ and $`\lambda _\tau `$. Analogously, the simplest SU(3)f assignment allows Yukawa couplings for all generations, but no generational differences. This must come from additional structure related to the flavor symmetry breaking that has not been specified here. Nevertheless, the matter (and Higgs) assignments under SU(3)f can be suitably modified to commute with grand unification. This was done in several early works on gauged SU(3)f $`\times `$ SU(5) SU3Hearly . There they found that many more triplets (or perhaps larger representations) of right-handed neutrinos were needed to cancel the \[SU(3)f\]3 anomaly. For the purposes of this paper, it is enough to observe that there must be *at least one* triplet of right-handed neutrinos to gauge SU(3)f.

The absence of a signal for new physics in flavor-changing neutral current processes places an important constraint on the scale of SU(3)f symmetry breaking. The constraint arises from the tree-level exchange of flavor gauge bosons that lead to transitions between same-charge, different-generation quarks or leptons. Integrating out heavy flavor gauge bosons results in a low-energy effective theory with new contributions to four-fermion, flavor-violating operators

$$\frac{g_f^4}{M_f^2}(\overline{f}_i\gamma _\mu f_i)(\overline{f}_j\gamma ^\mu f_j),$$
(6)
where $`g_f`$ is the SU(3)f gauge coupling and $`M_f`$ is the symmetry breaking scale. If the couplings are CP-conserving, one of the strongest constraints comes from the $`\mathrm{\Delta }s=2`$ process that contributes to the $`K_0\overline{K}_0`$ mass difference. Estimates of the bound on the four-quark operator suggest $`M_fg_f^2\times 1600`$ TeV lns . The bound is significantly stronger if the couplings maximally violate CP. In any case, for a flavor gauge coupling that is of order the SM gauge couplings, the bound on the symmetry breaking scale is at least hundreds of TeV. This is reminiscent of the constraints on extended technicolor de .

The benefit of right-handed neutrinos transforming under a chiral representation of the flavor symmetry is that the Majorana mass scale is no longer arbitrary. The right-handed neutrino Majorana mass is generated through flavor symmetry breaking, analogous to SM fermion masses generated through electroweak symmetry breaking. The scale $`M_f`$ is not predicted, but obviously there is no conflict between the lower bound $`M_f1000`$ TeV from flavor-changing constraints and the upper bound $`M_f10^{14}`$ GeV needed for a successful see-saw explanation of neutrino masses. If $`M_f`$ were near the lower bound, future experiments could search for deviations from (or as-yet unobserved) flavor-changing neutral current processes as a signal for SU(3)f. This would require neutrino Yukawa couplings $`\lambda _\nu 10^4`$ nearer in value to their lepton cousins.

How are right-handed neutrino Majorana masses generated from flavor symmetry breaking? Consider a pair of complex scalar fields in the fundamental representation $`\mathrm{\Sigma }_{1,2}(\mathrm{𝟑})`$ that acquire unaligned vacuum expectation values. This is sufficient to break SU(3)f $``$ nothing. A right-handed neutrino mass arises from the dimension-4 operator $`ϵ_{ijk}\nu _i^c\nu _j\mathrm{\Sigma }_k^{}`$ replacing $`\mathrm{\Sigma }`$ by its vev. Curiously, this two-field breaking model gives mass to just two off-diagonal components of the $`3\times 3`$ Majorana mass matrix in flavor space

$$\left(\begin{array}{ccc}& \mathrm{\Sigma }_1& \mathrm{\Sigma }_2\\ \mathrm{\Sigma }_1& & \\ \mathrm{\Sigma }_2& & \end{array}\right)$$
(7)
due to the anti-symmetric contraction of SU(3)f indices. This may be a useful starting point for generating an interesting neutrino mass texture. Also, the flavor symmetry could be broken in stages, such as SU(3) $``$ SU(2) $``$ nothing, that may be similarly useful for quark, lepton, or neutrino mass textures.

SU(3)f is not the only rationale for three generations and three right-handed neutrinos. In a recent proposal called “universal extra dimensions” (UED) univ , all matter, Higgs, and gauge bosons are promoted to six dimensional fields, and the more complicated gauge and gravitational anomaly structure of six dimensional theories is used to constrain the matter content dp . Ref. dp  found that cancellation of the global gauge anomaly en  required the number of generations to be $`n_g=0`$ mod $`3`$, and cancellation of the pure gravitational anomaly required $`n=n_g`$ fermionic fields uncharged under the SM gauge group. This is intriguingly similar to the SU(3)f symmetry argument, since the matter content is similarly restricted by anomaly cancellation of a larger symmetry structure. Other similarities are remarkable: dp  required that all matter was chiral in 6-D, analogous to requiring all matter to be in chiral representations of SU(3)f. This led to two possible chirality assignments in UED that are precisely analogous to the $`\mathrm{𝟑}`$ versus $`\overline{\mathrm{𝟑}}`$ “chirality” possibilities for the SM fermions under SU(3)f. Specifically, the quark doublet ($`Q`$) must have the opposite chirality to the quark singlets ($`u`$, $`d`$), and the lepton doublet ($`L`$) must have the opposite chirality to the lepton singlets ($`e`$, $`\nu `$). In UED the lepton doublet could have the same or the opposite chirality of the quark doublet, just as here the lepton doublet could be assigned to the same ($`\mathrm{𝟑}`$) or opposite ($`\overline{\mathrm{𝟑}}`$) representation of the quark doublet. Finally, the UED rationale for three generations and three right-handed neutrinos does not depend on the compactification scale, just as the SU(3)f argument does not depend on the flavor symmetry breaking scale.

There are a few important differences between the six dimensional UED model and the SU(3)f model. The higher dimensional nature of UED implies there is an effective theory cutoff scale that is only an order of magnitude above the compactification scale; in the SU(3)f model, there is no such restriction. Several gauge anomalies, such as \[SU(2)L\]2 U(1)Y that are automatically canceled in the SU(3)f model, are canceled in UED only via the Green-Schwarz mechanism with additional matter. Finally, the prediction of three generations is not easily extended to a supersymmetric six-dimensional “universal” model for a variety of reasons dp , whereas the SU(3)f model can be quite simply supersymmetrized as will be briefly sketched below.

Everything that has been said for the SM with a gauged SU(3)f flavor symmetry also applies to a straightforward extension of the minimal supersymmetric standard model (MSSM). This means promoting matter supermultiplets to (anti-)fundamental representations of SU(3)f while the Higgs supermultiplets remain singlets, in exact analogy with the non-supersymmetric case. (In the following discussion the same notation is used for the MSSM chiral superfields as for the SM fermion fields.) There are, however, new restrictions on the allowed operators in the supersymmetrized SU(3)f model. The most interesting, model-independent restriction is that the dimension-5 operators leading to proton decay

$$\frac{QQQL}{M_{\mathrm{Pl}}},\frac{u^cu^cd^ce^c}{M_{\mathrm{Pl}}}$$
(8)
are forbidden by SU(3)f. Technically the second operator in Eq. (8) could be made gauge-invariant if $`u^c`$ were assigned the conjugate representation to that of $`d^c`$ and $`e^c`$, but this does not happen for the SU(3)f model nor for the embeddings of matter into SU(5) or SO(10) representations. These operators can be made gauge-invariant by adding a pair of SU(3)f breaking superfields $`\mathrm{\Sigma }(\mathrm{𝟑})`$ and $`\overline{\mathrm{\Sigma }}(\overline{\mathrm{𝟑}})`$, whereby Eq. (8) becomes

$$\frac{QQQL\overline{\mathrm{\Sigma }}}{M_{\mathrm{Pl}}^2},\frac{u^cu^cd^ce^c\mathrm{\Sigma }}{M_{\mathrm{Pl}}^2}.$$
(9)
Below the SU(3)f symmetry breaking scale, these dimension-6 operators map onto the dimension-5 operators above with tiny coefficients of order $`\mathrm{\Sigma }/M_{\mathrm{Pl}}`$. This is sufficient to cure the fast proton decay problem that results from the ordinarily unsuppressed dimension-5 operators.

A supersymmetrized version of the SU(3)f model has even more interesting constraints. All dimension $`4`$ lepton number violating superpotential terms $`QLd^c`$, $`LLe^c`$, and $`LH_u`$ are forbidden by SU(3)f. Again, higher dimension operators with SU(3)f breaking fields will reintroduce these terms, but (for the first two) this leads to significant suppression. If the flavor symmetry were promoted to U(3)f, the dimension-4 baryon number violating term $`u^cd^cd^c`$ would also be forbidden. An exact flavor symmetry could serve in precisely the same role as matter parity on superfields ($`R`$-parity on fields). Of course the flavor symmetry is broken, and this reintroduces these so-called $`R`$-parity violating operators. It would be interesting to see if $`R`$-parity could be discarded in favor of a spontaneously broken U(3)f flavor symmetry without sacrificing a long-lived proton.

In summary, an extension of the Standard Model with an SU(3)f gauged flavor symmetry is presented that explains why there are three generations of matter and predicts the existence of three right-handed neutrinos. This argument is independent of grand unification or extra universal dimensions. The right-handed Majorana mass scale results from spontaneous SU(3)f symmetry breaking. If the breaking scale is “low”, less than of order 1000 TeV, deviations in flavor changing neutral current processes are expected due to tree-level flavor gauge boson exchange. It should be emphasized that such a Majorana mass scale is completely consistent with the see-saw explanation for neutrino mass generation so long as the Dirac masses of the neutrinos are less than but of order the muon mass. This is a perfectly reasonable possibility given that SU(3)f has freed us from thinking only in terms of grand unification. The supersymmetric extension including a gauged SU(3)f is straightforward. The fast proton decay problem from dimension-5 Planck-suppressed operators is automatically cured, and certain $`R`$-parity violating couplings are naturally suppressed. Combining the SU(3)f gauged flavor symmetry with models that attempt to explain the structure of the quark, lepton, or neutrino mass matrices is an extremely interesting direction left for future work.

###### Acknowledgements.

I have benefited from discussions with B. Balantekin, V. Barger, A. Nelson, Y. Nir, G. Shiu, and L.-T. Wang. I also thank the CERN theory group for hospitality where part of this work was completed. This work was supported in part by the U.S. Department of Energy under contract DE-FG02-95ER40896.


format: text

text:
# Geometry of Symplectic Intersections

## 1. Introduction

Symplectic geometry exhibits a range of intersection phenomena that cannot be predicted nor explained on the level of pure topology or differential geometry. The main players in this game are certain pairs of subspaces (e.g. Lagrangian submanifolds, domains, or a mixture of both) whose mutual intersections cannot be removed (or reduced) via the group of Hamiltonian or symplectic diffeomorphisms. The very first examples of such phenomena were conjectures by Arnold in the 1960’s, and eventually established and further explored by Gromov, Floer and others starting from the mid 1980s.

The first part of the paper will survey several intersection phenomena and the mathematical tools leading to their discovery. We shall not attempt to present the most general results and since the literature is vast the exposition will be far from complete. Rather we shall concentrate on various intersection phenomena trying to understand their nature and whether there is any relations between them.

The second part is dedicated to “non-intersections”, namely to situations where the principles of symplectic intersections break down. In the case of Lagrangian submanifolds this absence of intersections is reflected in the vanishing of a symplectic invariant called Floer homology. This vanishing when interpreted algebraically leads to restrictions on the topology of Lagrangian submanifolds. As a byproduct we shall explain how these restrictions can be used to study some problems in algebraic geometry concerning hyperplane sections and degenerations.

## 2. Various intersection phenomena

In this section we shall make a brief tour through the zoo of symplectic intersections, encountering three different species.

Before we start let us recall two important notions from symplectic geometry. Let $`(M,\omega )`$ be a symplectic manifold. A submanifold $`LM`$ is called Lagrangian if $`dimL=\frac{1}{2}dimM`$ and $`\omega `$ vanishes on $`T(L)`$. From now on we assume all Lagrangian submanifolds to be closed. The second notion is of Hamiltonian isotopies. An isotopy of diffeomorphisms $`\{h_t:MM\}_{0t1}`$, starting with $`h_0=\text{id}`$ is called Hamiltonian if the (time-dependent) vector field $`\xi _t`$ generating it satisfies that the $`1`$-forms $`i_{\xi _t}\omega `$ are exact for all $`0t1`$. Note that Hamiltonian isotopies preserve the symplectic structure: $`h_t^{}\omega =\omega `$ for all $`t`$. Finally, two subsets $`A,BM`$ are said to be Hamiltonianly isotopic if there exists a Hamiltonian isotopy $`h_t`$ such that $`h_1(A)=B`$. We refer the reader to  for the foundations of symplectic geometry.

### 2.1. Lagrangians intersect Lagrangians

The most fundamental Lagrangian intersection phenomenon occurs in cotangent bundles. Let $`X`$ be a closed manifold and $`T^{}(X)`$ be its cotangent bundle endowed with the canonical symplectic structure $`\omega _{\text{can}}=dp_idq_i`$. Denote by $`\lambda _{\text{can}}=p_idq_i`$ the Liouville form (so that $`\omega _{\text{can}}=d\lambda _{\text{can}}`$). Recall that a Lagrangian submanifold $`LT^{}(X)`$ is called exact if the restriction $`\lambda _{\text{can}}|_{T(L)}`$ is exact. Note that the property of exactness is preserved by Hamiltonian isotopies. Denote by $`O_XT^{}(X)`$ the zero-section. The following theorem was proved by Gromov in :

Theorem A. Let $`LT^{}(X)`$ be an exact Lagrangian submanifold. Then:
1) For every Lagrangian $`L^{}`$ which is Hamiltonianly isotopic to $`L`$ we have $`LL^{}\mathrm{}`$.
2) $`LO_X\mathrm{}`$. In particular, $`L`$ cannot be separated from the zero-section by any Hamiltonian isotopy.

If one assumes $`L`$ to be a Hamiltonian image of the zero-section a more quantitative version of Theorem A holds:

Theorem B. Let $`LT^{}(X)`$ be a Lagrangian submanifold which is Hamiltonianly isotopic to the zero-section and intersects it transversely. Then

$$\mathrm{\#}LO_X\underset{j=0}{\overset{dimX}{}}b_j(X),$$

where $`b_j(X)`$ are the Betti numbers of $`X`$.

Chronologically Theorem B preceded Theorem A. It was conjectured by Arnold (see  for the history), first proved for $`X=𝕋^n`$ by Chaperon  and generalized to all cotangent bundles by Hofer  and by Laudenbach and Sikorav . Now a days it can be viewed as a special case of Floer theory (see Section 2.4 below).

Note that the intersections described by both theorems above cannot in general be understood on a purely topological level. Indeed, in general topology predicts less than $`b_j(X)`$ intersection points, and sometimes even none. Finally, note that in general the statement of Theorem B fails if one assumes $`L`$ to be only symplectically isotopic to $`O_X`$, as the example $`X=𝕋^n`$ shows.

### 2.2. Balls intersect balls

Denote by $`B^{2n}(R)`$ the closed Euclidean ball of radius $`R`$, endowed with the standard symplectic structure induced from $`^{2n}`$. Denote by $`P^n`$ the complex projective space, endowed with its standard Kähler form $`\sigma `$, normalized so that $`_{P^1}\sigma =\pi `$. The following obstruction for symplectic packing was discovered by Gromov :

Theorem C. Let $`M`$ be either $`B^{2n}(1)`$ or $`P^n`$. Let $`B_{\phi _1},B_{\phi _2}M`$ be the images of two symplectic embeddings $`\phi _1:B^{2n}(R_1)M`$, $`\phi _2:B^{2n}(R_2)M`$. If $`R_1^2+R_2^21`$ then $`B_{\phi _1}B_{\phi _2}\mathrm{}`$.

Since symplectic embeddings are also volume preserving there is an obvious volume obstruction for having $`B_{\phi _1}B_{\phi _2}=\mathrm{}`$. However, volume considerations predict an intersection only if $`R_1^{2n}+R_2^{2n}1`$ (moreover for volume preserving embeddings the latter inequality is sharp).

When one considers embeddings of several balls things become more complicated and interesting. Here results are currently available only in dimension $`4`$.

Theorem D. Let $`M`$ be either $`B^4(1)`$ or $`P^2`$, and let $`B_{\phi _1},\mathrm{},B_{\phi _N}M`$ be the images of symplectic embeddings $`\phi _k:B^4(R)M`$, $`k=1,\mathrm{},N`$, of $`N`$ balls of the same radius $`R`$. Then there exist $`ij`$ such that $`B_{\phi _i}B_{\phi _j}\mathrm{}`$ in each of the following cases:

1. $`N=2`$ or $`3`$ and $`R^21/2`$.
2. $`N=5`$ or $`6`$ and $`R^22/5`$.
3. $`N=7`$ and $`R^23/8`$.
4. $`N=8`$ and $`R^26/17`$.

Moreover all the above inequalities are sharp in the sense that in each case if the inequality on $`R`$ is not satisfied then there exist symplectic embeddings $`\phi _1,\mathrm{},\phi _N`$ as above with disjoint images $`B_{\phi _1},\mathrm{},B_{\phi _N}M`$.

Statement 2 for $`N=5`$ was proved by Gromov . The rest was established by McDuff and Polterovich . Let us mention that for $`N=4`$ and any $`N9`$ this intersection phenomenon completely disappears in the sense that an arbitrarily large portion of the volume of $`M`$ can be filled by a disjoint union of $`N`$ equal balls (see  for $`N=4`$ and $`N=k^2`$, and  for the remaining cases).

### 2.3. Balls intersect Lagrangians

It turns out that there exist (symplectically) irremovable intersections also between contractible domains (e.g. balls) and Lagrangian submanifolds.

Denote by $`P^nP^n`$ the Lagrangian $`n`$-dimensional real projective space (embedded as the fixed point set of the standard conjugation of $`P^n`$). The following was proved in :

Theorem E. Let $`B_\phi P^n`$ be the image of a symplectic embedding $`\phi :B^{2n}(R)P^n`$. If $`R^21/2`$ then $`B_\phi P^n\mathrm{}`$. Moreover the inequality is sharp, namely for every $`R^2<1/2`$ there exists a symplectic embedding $`\phi :B^{2n}(R)P^n`$ whose image avoids $`P^n`$.

In fact this pattern of intersections occurs in a wide class of examples (see ):

Theorem E’. Let $`(M,\omega )`$ be a closed Kähler manifold with $`[\omega ]H^2(M;)`$ and $`\pi _2(M)=0`$. Then for every $`ϵ>0`$ there exists a Lagrangian CW-complex $`\mathrm{\Delta }_ϵ(M,\omega )`$ with the following property: every symplectic embedding $`\phi :B^{2n}(ϵ)(M,\omega )`$ must satisfy $`\text{Image }(\phi )\mathrm{\Delta }_ϵ\mathrm{}`$.

By a Lagrangian CW-complex we mean a subspace $`\mathrm{\Delta }_ϵM`$ which topologically is a CW-complex and the interior of each of its cells is a smoothly embedded disc of $`M`$ on which $`\omega `$ vanishes.

### 2.4. Methods for studying intersections

Lagrangian intersections. The first systematic study of Lagrangian intersections was based on the theory of generating function  (an equivalent theory was independently developed in contact geometry ). Gromov’s theory of pseudo-holomorphic curves  gave rise to an alternative approach which culminated in what is now called Floer theory. Each of these theories has its own advantage. Floer theory works in larger generality and seems to have a richer algebraic structure, on the other hand the theory of generating functions leads in some cases to sharper results (see ).

Since Floer theory will appear in the sequel, let us outline a few facts about it (the reader is referred to the works of Floer  and of Oh  for details). Let $`(M,\omega )`$ be a symplectic manifold and $`L_0,L_1(M,\omega )`$ two Lagrangian submanifolds. In “ideal” situations Floer theory assigns to this data an invariant $`HF(L_0,L_1)`$. This is a $`_2`$-vector space obtained through an infinite dimensional version of Morse-Novikov homology performed on the space of paths connecting $`L_0`$ to $`L_1`$. The result of this theory is a chain complex $`CF(L_0,L_1)`$ whose underlying vector space is generated by the intersection points $`L_0L_1`$ (one perturbs $`L_0,L_1`$ so their intersection becomes transverse). The homology of this complex $`HF(L_0,L_1)`$ is called the Floer homology of the pair $`(L_0,L_1)`$. The most important feature of $`HF(L_0,L_1)`$ is its invariance under Hamiltonian isotopies: if $`L_0^{},L_1^{}`$ are Hamiltonianly isotopic to $`L_0,L_1`$ respectively, then $`HF(L_0^{},L_1^{})HF(L_0,L_1)`$. From this point of view $`HF(L_0,L_1)`$ can be regarded as a quantitative obstruction for Hamiltonianly separating $`L_0`$ from $`L_1`$. Indeed, the rank of $`HF(L_0,L_1)`$ is a lower bound on the number of intersection points of any pair of transversally intersecting Lagrangians $`L_0^{},L_1^{}`$ in the Hamiltonian deformation classes of $`L_0,L_1`$ respectively.

Let us explain the “ideal situations” in which Floer homology is defined. First of all there are restrictions on $`M`$: due to analytic difficulties manifolds are required to be either closed or to have symplectically convex ends (e.g. $`^n`$, cotangent bundles or any Stein manifold). More serious restrictions are posed on the Lagrangians. For simplicity we describe them only for the case when $`L_1`$ is Hamiltonianly isotopic to $`L_0`$. From now on we shall write $`L=L_0`$ and $`L^{}=L_1`$. In Floer’s original setting  the theory was defined under the assumption that the homomorphism $`A_\omega :\pi _2(M,L)`$, defined by $`D_D\omega `$, vanishes. The reason for this comes from the construction of the differential of the Floer complex: the main obstruction for defining a meaningful differential turns out to be existence of holomorphic discs with boundary on $`L`$ or $`L^{}`$. These discs appear as a source of non-compactness of the space of solutions of the PDEs involved in the construction. Since holomorphic discs must have positive symplectic area the assumption $`A_\omega =0`$ rules out their existence. Under this assumption Floer defined $`HF(L,L^{})`$ and proved its invariance under Hamiltonian isotopies. Moreover he showed that $`HF(L,L)`$ is isomorphic to the singular cohomology $`H^{}(L;_2)`$ of $`L`$. This together with the invariance give:

Theorem F. Let $`(M,\omega )`$ be a symplectic manifold, either compact or with symplectically convex ends. Let $`L(M,\omega )`$ be a Lagrangian submanifold with $`A_\omega =0`$. Then for every Lagrangian $`L^{}`$ which is Hamiltonianly isotopic to $`L`$ and intersects $`L`$ transversally we have: $`\mathrm{\#}LL^{}\text{rank}HF(L,L^{})=\text{rank}H^{}(L;_2)`$. In particular $`L`$ cannot be separated from itself by a Hamiltonian isotopy.

Floer theory was extended by Oh  to cases when $`A_\omega 0`$. There are two assumptions needed for this extension to work: the Maslov homomorphism $`\mu :\pi _2(M,L)`$ should be positively proportional to $`A_\omega `$ (such Lagrangians are called monotone). The second assumption is that the positive generator $`N_L`$ of the subgroup $`\text{Image }\mu `$ is at least $`2`$. In this setting Oh defined $`HF(L,L^{})`$ and proved its invariance under Hamiltonian isotopies. It is however no longer true in general that $`HF(L,L)`$ is isomorphic to $`H^{}(L;_2)`$. Still, Oh proved  that $`HF(L,L)`$ is related to $`H^{}(L;_2)`$ through a spectral sequence. Recently the theory was considerably generalized by Fukaya, Oh, Ohta and Ono .

Intersections of balls. Theorems C and D were obtained using Gromov’s theory of pseudo-holomorphic curves. The hard-core of the proofs consists of existence of pseudo-holomorphic curves of specified degrees that pass through a prescribed number of points in the manifold (see ) for the details). From a more modern perspective it can be viewed as an early application of Gromov-Witten invariants.

Finally, Theorems E and E’ are proved by a decomposition technique introduced in  which enables to decompose symplectic manifolds as a disjoint union of a symplectic disc bundle and a Lagrangian $`CW`$-complex. A variation on the proof of Gromov’s non-squeezing theorem  gives an upper bound on the radius of a symplectic ball that can be squeezed inside that disc bundle. Hence, a larger ball must always intersect this $`CW`$-complex. For $`M=P^n`$, the corresponding $`CW`$-complex turns out to be a smooth copy of $`P^n`$. See  for the details.

## 3. Some questions and speculations

Cotangent bundles. The following questions show that even in the case of cotangent bundles the most fundamental invariants are not completely understood.

1. Let $`LT^{}(X)`$ be an exact Lagrangian (not necessarily Hamiltonianly isotopic to $`O_X`$). By Theorem A, $`LO_X\mathrm{}`$. Is it true that $`HF(L,O_X)0`$ ?
2. Let $`L_0,L_1T^{}(X)`$ be two exact Lagrangians (again, not necessarily Hamiltonianly isotopic neither to $`O_X`$ nor to each other). Is it true that $`L_0L_1\mathrm{}`$ ? Is it true that $`HF(L_0,L_1)0`$ ?

These questions are of a theoretical importance, since the zero section and its Hamiltonian images are the only known examples of exact Lagrangians in $`T^{}(X)`$.

Symplectic packing. Lack of tools (or new ideas) prevent us from understanding symplectic packings in dimension higher than $`4`$. The only packing obstructions known in these dimensions are described in Theorem C. Note that $`P^n`$ admits full packing by $`N=k^n`$ equal balls , but it is unclear what happens for other values of $`N`$. In view of this and Theorem C, the first unknown case (for $`n3`$) is of $`N=2^n+1`$ equal balls.

The situation in dimension $`4`$ is only slightly better. Except of $`P^2`$ and a few other rational surfaces no packing obstructions are known. It is known that for every symplectic $`4`$-manifold $`(M,\omega )`$ with $`[\omega ]H^2(M;)`$ packing obstruction (for equal balls) disappear once the number of balls is large enough (see ), but nothing is known when the number of balls is small. In fact even the case of one ball is poorly understood (namely, what is the maximal radius of a ball that can be symplectically embedded in $`M`$). The reason here is that the methods yielding packing obstructions strongly rely on the geometry of algebraic and pseudo-holomorphic curves in the manifold. The problem is that most symplectic manifolds have very few (or none at all) $`J`$-holomorphic curves for a generic choice of the almost complex structure. Thus, even in dimension $`4`$ it is unknown whether or not packing obstructions is a phenomenon particular to a sporadic class of manifolds such as $`P^2`$.

Is everything Lagrangian? Weinstein’s famous saying could be relevant for the intersection described in Theorems C,D and E. In other words, it could be that these intersections are in fact Lagrangian intersections under disguise. To be more concrete, let $`\frac{1}{2}<R^2<\frac{1}{2^{1/n}}`$ and consider a Lagrangian $`L_R`$ lying on the boundary $`B^{2n}(R)`$. Is it possible to Hamiltonianly separate $`L_R`$ from itself inside $`B^{2n}(1)`$ ?

If we can find a Lagrangian $`L_R`$ for which the answer is negative then this would strongly indicate that Theorem C is in fact a Lagrangian intersections result. Namely it would imply Theorem C for $`R_1=R_2`$ under the additional assumption that $`\phi _1,\phi _2`$ are symplectically isotopic. A good candidate for $`L_R`$ seems to be the split torus $`B^2(\sqrt{R/n})\times \mathrm{}\times B^2(\sqrt{R/n})B^{2n}(R)`$, but one could try other Lagrangians as well.

Attempts to approach this question with traditional Floer homology fail. The reason is that Floer homology is blind to sizes: both to the “size” of the Lagrangian $`L_R`$ as well as to the “size” of the domain in which we work $`B^{2n}(1)`$. Indeed it is easy to see that $`HF(L_R,L_R)`$ whether computed inside $`B^{2n}(1)`$ or in $`^{2n}`$ is the same, hence vanishes. The meaning of “sizes” can be made precise: the size of $`L_R`$ is encoded in its Liouville class, and the size of $`B^{2n}(1)`$ could be encoded here by the action spectrum of its boundary.

It would be interesting to try a mixture of symplectic field theory  with Floer homology. This would require a sophisticated counting of holomorphic discs with $`k`$ punctures (for all $`k0`$), where the boundary of the discs goe to $`L_R`$ and the punctures to periodic orbits on $`B^{2n}(1)`$.

It is interesting to note that when the radii of the balls are not equal things become more complicated. Indeed suppose that $`R_1^2+R_2^2>1`$ and consider two Lagrangian submanifolds $`L_{R_1},L_{R_2}`$ lying on the boundaries of the balls $`B_{\phi _1},B_{\phi _2}`$. Then clearly $`L_{R_1}`$ and $`L_{R_2}`$ can be disjoint even though the balls $`B_{\phi _1},B_{\phi _2}`$ do intersect (e.g. two concentric balls $`B_{\phi _1}B_{\phi _2}`$, where $`R_1<R_2`$). It would be interesting to see to which extent this mutual position can be detected on the level of the Lagrangians $`L_{R_1}`$ and $`L_{R_2}`$ alone. Or, in more pictorial (but less mathematical) terms, do the Lagrangians $`L_{R_1}`$ and $`L_{R_2}`$ know that they lie one “inside” the other ?

Returning to the case of equal balls, if the above plan is feasible, it would be interesting to try similar approaches for more than two balls as described in Theorem D. A similar approach could be tried in the situation of Theorem E. Here one could expect an irremovable intersection between a Lagrangian submanifold $`L_RB_\phi `$ and $`P^n`$.

Quantitative intersections. In contrast to the quantitative version of Lagrangian intersections given by Theorems B and F, Theorems C–E provide only existence of intersections. Is it possible to measure the size of these intersections ?

More concretely, consider two balls $`B_{\phi _1},B_{\phi _2}B^{2n}(1)`$ with $`R_1^2+R_2^2>1`$ but with $`R_1^{2n}+R_2^{2n}<1`$ (so that $`\text{Vol}(B_{\phi _1})+\text{Vol}(B_{\phi _2})<1`$). Is it possible to bound from below the size of $`B_{\phi _1}B_{\phi _2}`$ ?

It is not hard to see that volume is a wrong candidate for the size since for every $`ϵ>0`$ there exist two such balls with $`\text{Vol}(B_{\phi _1}B_{\phi _2})<ϵ`$. Symplectic capacities seem also to be inappropriate for this task. It could be that “size” should be replaced here by a kind of “complexity” or a trade-off between capacity and complexity: namely if the intersection has large capacity (e.g. when $`B_{\phi _1}B_{\phi _2}`$) the complexity is low, and vice-versa. Note that in dimension $`2`$ a possible notion of complexity of a set is the number of connected components of its interior.

A related problem is the following. Consider two symplectic balls $`B_{\phi _1},B_{\phi _2}P^n`$ of radii $`R_1,R_2`$, where $`R_1^2+R_2^2=1`$. Assume further that $`\text{Int }(B_{\phi _1})\text{Int }(B_{\phi _2})=\mathrm{}`$. Theorem C implies that the balls must intersect hence the intersection occurs on the boundaries: $`B_{\phi _1}B_{\phi _2}\mathrm{}`$. What can be said about the intersection $`B_{\phi _1}B_{\phi _2}\mathrm{}`$, in terms of size, dynamical properties etc. ?

It is easy to see that this intersection cannot be discrete. Moreover, an argument based on the work of Sullivan  shows that the intersection must contain at least one entire (closed) orbit of the characteristic foliation of the boundaries of the balls (see  for a discussion on this point). Looking at examples however suggests that the number of orbits in the intersection should be much larger.

The same problem can be considered also for (some of) the extremal cases described in Theorem D. Similarly one can study the intersection $`B_\phi P^n`$ where $`B_\phi P^n`$ is a symplectic ball of radius $`R^2=1/2`$ whose interior is disjoint from $`P^n`$. It is likely that methods of symplectic field theory  could shed some light on this circle of problems.

Stable intersections. The problems described here come from Polterovich . Let $`(M,\omega )`$ be a symplectic manifold and $`AM`$ a subset. We say that $`A`$ has the Hamiltonian intersection property if for every Hamiltonian diffeomorphism $`f`$ we have $`f(A)A\mathrm{}`$. We say that $`A`$ has the stable Hamiltonian intersection property if $`O_{S^1}\times AT^{}(S^1)\times M`$ has the Hamiltonian intersection property. Polterovich discovered in  that if there exists a subset $`AM`$ with open non-empty complement and with the stable Hamiltonian property then the universal cover $`\stackrel{~}{\text{Ham}}(M,\omega )`$ of the group of Hamiltonian diffeomorphisms has infinite diameter with respect to Hofer’s metric. Note that when $`\pi _1(\text{Ham}(M,\omega ))`$ is finite the same holds also for $`\text{Ham}(M,\omega )`$ itself. (See  for the details and references for other results on the diameter of Ham). This is applicable when $`(M,\omega )`$ contains a Lagrangian submanifold $`A`$ with $`HF(A,A)0`$, since then $`HF(O_{S^1}\times A,O_{S^1}\times A)=(_2_2)HF(A,A)0`$. For example, taking $`A=P^nP^n`$ Polterovich proved that $`\text{diam}\stackrel{~}{\text{Ham}}(P^n)=\mathrm{}`$ (for $`n=1,2`$ the same holds for $`diam\text{Ham}(P^n)`$).

In view of the above the following question seems natural: does every closed symplectic manifold contain a subset $`A`$ with open non-empty complement and with the stable Hamiltonian intersection property ? Note that besides Lagrangian submanifolds (with $`HF0`$) no other stable Hamiltonian intersection phenomena are known. It would also be interesting to find out whether the intersections described in Theorems C,D,E and especially $`E^{}`$ continue to hold after stabilization.

## 4. Intersections versus non-intersections

In contrast to cotangent bundles there are manifolds in which every compact subset can be separated from itself by a Hamiltonian isotopy. The simplest example is $`^n`$: indeed linear translations are Hamiltonian, and any compact subset can be translated away from itself. Clearly the same also holds for every symplectic manifold of the type $`M\times `$ by applying translations on the $``$ factor. Note that manifolds of the type $`M\times `$ sometime appear in “disguised” forms (e.g. as subcritical Stein manifolds, see Cieliebak ).

The “non-intersections” property has quite strong consequences on the topology of Lagrangian submanifolds already in $`^n`$. Denote by $`\omega _{\text{std}}`$ the standard symplectic structure of $`^n`$ and let $`\lambda `$ be any primitive of $`\omega _{\text{std}}`$. Note that the restriction $`\lambda |_{T(L)}`$ of $`\lambda `$ to any Lagrangian submanifold $`L^n`$ is closed. The following was proved by Gromov in :

Theorem G. Let $`L^n`$ be a Lagrangian submanifold. Then the restriction of $`\lambda `$ to $`L`$ is not exact. In particular $`H^1(L;)0`$.

Indeed if $`\lambda `$ were exact on $`L`$ then $`A_\omega :\pi _2(^n,L)`$ must vanish, hence by Theorem F it is impossible to separate $`L`$ from itself by a Hamiltonian isotopy. On the other hand, as discussed above, in $`^n`$ this is always possible. We thus get a contradiction. (Gromov’s original proof is somewhat different, however a careful inspection shows it uses the failure of Lagrangian intersections in an indirect way). Arguments exploiting non-intersections were further used in clever ways by Lalonde and Sikorav  to obtain information on the topology of exact Lagrangians in cotangent bundles (see also Viterbo  for further results).

An important property of symplectic manifolds $`W`$ having the “non-intersections” property is the following vanishing principle: for every Lagrangian submanifold $`LW`$ with well defined Floer homology we have $`HF(L,L)=0`$. Applying this vanishing to $`^n`$ yields restrictions on the possible Maslov class of Lagrangian submanifolds of $`^n`$. (Conjectures about the Maslov class due to Audin appear already in . First results in this directions are due to Polterovich  and to Viterbo . The interpretation in Floer-homological terms is due to Oh . Generalizations to other manifolds appear in  and . Finally, consult  for recent results answering old questions on the Maslov class).

### 4.1. Lagrangian embeddings in closed manifolds

The ideas described above can be applied to obtain information on the topology of Lagrangian submanifolds of some closed manifolds. Note that in comparison to closed manifolds the case of $`^n`$ can be regarded as local (Darboux Theorem). Of course, “local” should by no means be interpreted as easy. On the contrary, characterization of manifolds that admit Lagrangian embeddings into $`^n`$ is completely out of reach with the currently available tools.

Below we shall deal with the “global” case, namely with Lagrangians in closed manifolds. One (coarse) way to “mod out” local Lagrangians is to restrict to Lagrangians $`L`$ with $`H_1(L;)`$ zero or torsion (so that by Theorem G they cannot lie in a Darboux chart). The pattern arising in the theorems below is that under such assumptions in some closed symplectic manifolds we have homological uniqueness of Lagrangian submanifolds. Let us view some examples.

We start with $`P^n`$. It is known that a Lagrangian submanifold $`LP^n`$ cannot have $`H_1(L;)=0`$ (see Seidel , see also  for an alternative proof). However, $`LP^n`$ may have torsion $`H_1(L;)`$ as the example $`P^nP^n`$ shows.

Theorem H. Let $`LP^n`$ be a Lagrangian submanifold with $`H_1(L;)`$ a $`2`$-torsion group (namely, $`2H_1(L;)=0`$). Then:

1. $`H^{}(L;_2)H^{}(P^n;_2)`$ as graded vector spaces.
2. Let $`aH^2(P^n;_2)`$ be the generator. Then $`a|_LH^2(L;_2)`$ generates the subalgebra $`H^{\text{even}}(L;_2)`$. Moreover if $`n`$ is even the isomorphism in 1 is of graded algebras.

Statement 1 of the theorem was first proved by Seidel . An alternative proof based on “non-intersections” can be found in . Let us outline the main ideas from . Consider $`P^n`$ as a hypersurface of $`P^{n+1}`$. Let $`U`$ be a small tubular neighbourhood of $`P^n`$ inside $`P^{n+1}`$. The boundary $`U`$ looks like a circle bundle over $`P^n`$ (in this case it is just the Hopf fibration). Denote by $`\mathrm{\Gamma }_LL`$ the restriction of this circle bundle to $`LP^n`$. A local computation shows that $`U`$ can be chosen so that $`\mathrm{\Gamma }_LP^{n+1}P^n`$ becomes a Lagrangian submanifold. (This procedure works whenever we have a symplectic manifold $`\mathrm{\Sigma }`$ embedded as a hyperplane section in some other symplectic manifolds $`M`$). The next observation is that $`\mathrm{\Gamma }_LP^{n+1}P^n`$ is monotone and moreover its minimal Maslov number $`N_{\mathrm{\Gamma }_L}`$ is the same as the one of $`L`$. Due to our assumptions on $`H_1(L;)`$ this number turns out to satisfy $`N_{\mathrm{\Gamma }_L}n+1`$. The crucial point now is that $`HF(\mathrm{\Gamma }_L,\mathrm{\Gamma }_L)=0`$. Indeed, the symplectic manifold $`P^{n+1}P^n`$ can be completed to be $`^{n+1}`$ where Floer homology vanishes.

Having this vanishing we turn to an alternative computation of $`HF(\mathrm{\Gamma }_L,\mathrm{\Gamma }_L)`$. This computation is based on the theory developed by Oh  for monotone Lagrangian submanifolds. According to  Floer homology can be computed via a spectral sequence whose first stage is the singular cohomology of the Lagrangian. The minimal Maslov number has an influence both on the grading as well as on the number of steps it takes the sequence to converge to $`HF`$. In our case we have a spectral sequence starting with $`H^{}(\mathrm{\Gamma }_L;_2)`$ and converging to $`HF(\mathrm{\Gamma }_L,\mathrm{\Gamma }_L)=0`$. A computation through this process together with the information that $`N_{\mathrm{\Gamma }_L}n+1`$ makes it possible to completely recover $`H^{}(\mathrm{\Gamma }_L;_2)`$. It turns out that $`H^i(\mathrm{\Gamma }_L;_2)=_2`$ for $`i=0,1,n`$ and $`n+1`$, while $`H^i(\mathrm{\Gamma }_L;_2)=0`$ for all $`1<i<n`$. Going back from $`H^{}(\mathrm{\Gamma }_L;_2)`$ to $`H^{}(L;_2)`$ is now done by the Gysin exact sequence of the circle bundle $`\mathrm{\Gamma }_LL`$ and noting that the second Stiefel-Whitney class of this bundle is nothing but the restriction $`a|_L`$ of the generator $`aH^2(P^n;_2)`$.

Summarizing the proof, there are three main ingredients:

1. Transforming the Lagrangian $`L`$ into a related Lagrangian $`\mathrm{\Gamma }_L`$ living in a different manifold such that $`\mathrm{\Gamma }_L`$ can be Hamiltonianly separated from itself. Consequently we obtain $`HF(\mathrm{\Gamma }_L,\mathrm{\Gamma }_L)=0`$.
2. Relating $`HF(\mathrm{\Gamma }_L,\mathrm{\Gamma }_L)`$ to $`H^{}(\mathrm{\Gamma }_L)`$ via the theory of Floer homology (e.g. a spectral sequence).
3. Passing back from $`H^{}(\mathrm{\Gamma }_L)`$ to $`H^{}(L)`$.

Similar ideas work in various other cases (see ). For example, consider $`P^n\times P^n`$. This manifold has Lagrangians with $`H_1(L;)=0`$, e.g. $`P^n`$ which can be embedded as the “anti-diagonal” $`\{(z,w)P^n\times P^n|w=\overline{z}\}`$.

Theorem I. Let $`LP^n\times P^n`$ be a Lagrangian with $`H_1(L;)=0`$. Then $`H^{}(L;_2)H^{}(P^n;_2)`$, the isomorphism being of graded algebras.

Another application of this circle of ideas is for Lagrangian spheres. Recently Lagrangian spheres have attracted special attention due to their relations to interesting symplectic automorphisms  and to symplectic Lefschetz pencils .

Theorem J. 1) Let $`M`$ be a closed symplectic manifold with $`\pi _2(M)=0`$, and denote by $`m=dim_{}M`$ its complex dimension. If $`M\times P^n`$ (where $`m,n1`$) has a Lagrangian sphere then $`mn+1(mod2n+2)`$.
2) Let $`M=P^n\times P^m`$, $`m+n3`$, be endowed with the split symplectic form $`(n+1)\sigma (m+1)\sigma `$. If $`M`$ has a Lagrangian sphere then $`\mathrm{gcd}(n+1,m+1)=1`$.

Let us remark that when $`m=n+1`$ any product of the form $`P^n\times M`$ (with $`dim_{}M=n+1`$) indeed has a Lagrangian sphere, after a possible rescaling of the symplectic form on the $`M`$ factor (see ). We are not aware of any other examples, namely when $`mn+1(mod2n+2)`$ but $`mn+1`$).

## 5. Relations to algebraic geometry

The purpose of this section is to show how ideas from Section 4 are related to algebraic geometry. We shall not present new results here but rather try to outline a new direction in which symplectic methods can be used in algebraic geometry.

### 5.1. Hyperplane sections

Let $`\mathrm{\Sigma }`$ be a smooth projective variety. The classical Lefschetz theorem provides restrictions on smooth varieties $`X`$ that may contain $`\mathrm{\Sigma }`$ as their hyperplane section. It was discovered by Sommese  that there exist projective varieties $`\mathrm{\Sigma }`$ that cannot be hyperplane sections (or even ample divisors) in any smooth variety $`X`$. For example, Sommese proved that Abelian varieties of (complex) dimension $`2`$ have this property (see  for more examples).

Let us outline an alternative approach to this problem using symplectic geometry. Let $`XP^N`$ be a smooth variety. Denote by $`X^{}(P^N)^{}`$ the dual variety (namely, the variety of all hyperplanes $`H(P^N)^{}`$ that are non-transverse to $`X`$).

Theorem K. Suppose that $`\mathrm{\Sigma }=XH_0X`$ is a smooth hyperplane section of $`X`$ obtained from a projective embedding $`XP^N`$. Then either $`\mathrm{\Sigma }`$ has a Lagrangian sphere (for the symplectic structure induced from $`P^N`$), or $`\text{codim}_{}(X^{})>1`$.

Here is an outline of the proof. Suppose that $`\text{codim}_{}(X^{})=1`$. Choose a generic line $`\mathrm{}(P^N)^{}`$ intersecting $`X^{}`$ transversely (and only at smooth points of $`X^{}`$). Consider the pencil $`\{XH\}_H\mathrm{}`$ parametrized by $`\mathrm{}`$. Passing to the blow-up $`\stackrel{~}{X}`$ of $`X`$ along the base locus of the pencil we obtain a holomorphic map $`\pi :\stackrel{~}{X}\mathrm{}P^1`$. The critical values of $`\pi `$ are in 1-1 correspondence with the point of $`\mathrm{}X^{}`$. Moreover, the fact that $`\mathrm{}`$ intersects $`X^{}`$ transversely implies that $`\pi `$ is a so called Lefschetz fibration, namely each critical point of $`\pi `$ has non-degenerate (complex) Hessian (in other words, locally $`\pi `$ looks like a holomorphic Morse function). The condition $`\text{codim}_{}(X^{})=1`$ ensures that $`\mathrm{}X^{}\mathrm{}`$ hence at least one of the fibres of $`\pi `$ is singular. Let $`X_0`$ be such a fibre and $`pX_0`$ a critical point of $`\pi `$. The important point now is that the vanishing cycle (corresponding to $`p`$) that lies in the nearby smooth fibre $`X_ϵ`$ can be represented by a (smooth) Lagrangian sphere. By Moser argument all the smooth divisors in the linear system $`\{XH\}_{H(P^N)^{}}`$ are symplectomorphic. In particular $`\mathrm{\Sigma }`$ has a Lagrangian sphere too.

The existence of Lagrangian vanishing cycles was known folklorically for long time. Its importance to symplectic geometry was realized by Arnold , Donaldson  and by Seidel .

Theorem K can be applied as follows: given a smooth variety $`\mathrm{\Sigma }`$, use methods of symplectic geometry to prove that $`\mathrm{\Sigma }`$ contains no Lagrangian spheres, say for any symplectic structure compatible with the complex structure of $`\mathrm{\Sigma }`$. Then by Theorem K the only chance for $`\mathrm{\Sigma }`$ to be a hyperplane section is inside a variety $`X`$ with “small dual”, namely $`\text{codim}_{}(X^{})>1`$. Let us remark that smooth varieties $`XP^N`$ with $`\text{codim}_{}(X^{})>1`$ are quite rare, and have very restricted geometry (see e.g. Zak  and Ein ). Using the theory of “small dual varieties” we can either rule out this case or get strong restrictions on the pair $`(X,\mathrm{\Sigma })`$.

Let us illustrate this on the example mentioned at the beginning of the section. Let $`\mathrm{\Sigma }`$ be an Abelian variety of complex dimension $`n2`$. Note that $`\mathrm{\Sigma }`$ cannot have a Lagrangian sphere for any Kähler form. Indeed, if $`\mathrm{\Sigma }`$ had such a sphere then the same would hold also for the universal cover of $`\mathrm{\Sigma }`$ which is symplectomorphic to $`^n`$. But this is impossible in view of Theorem G. Thus if $`\mathrm{\Sigma }`$ is a hyperplane section of $`XP^N`$ then $`\text{codim}_{}(X^{})>1`$. It is well known  that in this case $`X`$ must have rational curves (in fact lots of them). In particular $`\pi _2(X)0`$. By Lefschetz’s theorem we get $`\pi _2(\mathrm{\Sigma })0`$. But this is impossible since $`\mathrm{\Sigma }`$ is an Abelian variety. We therefore conclude that $`\mathrm{\Sigma }`$ cannot be a hyperplane section in any smooth variety $`X`$.

An analogous (though symplectically more involved) argument should apply also to any algebraic variety $`\mathrm{\Sigma }`$ with $`c_1=0`$ and $`b_1(\mathrm{\Sigma })0`$ (see ). An application of more refined symplectic tools (e.g. methods described in Section 4.1 above) can be used to obtain many more examples.

Here is another typical application: let $`C`$ be a projective curve of genus $`>0`$. It was observed by Silva  that $`C\times P^n`$ can be realized as a hyperplane section in various smooth varieties. Note that by Theorem J, $`C\times P^n`$ cannot have any Lagrangian spheres. It immediately follows that the only smooth varieties $`X`$ that support $`C\times P^n`$ as their hyperplane section must have small dual. For $`n5`$ results of Ein  make it even possible to list all such $`X`$’s.

We conclude with a remark on the methods. The symplectic approach outlined above gives coarser results. Indeed Sommese  provides examples of varieties that cannot be ample divisors whereas the methods above only rule out the possibility of being very ample. On the other hand the symplectic approach has an advantage in its robustness with respect to small deformations (see , c.f. ).

### 5.2. Degenerations of algebraic varieties

The methods of the previous section can also be used to study degenerations of algebraic varieties. Let $`Y`$ be a smooth projective variety. We say that $`Y`$ admits a Kähler degeneration with isolated singularities if there exists a Kähler manifold $`X`$ and a proper holomorphic map $`\pi :XD`$ to the unit disc $`D`$ with the following properties:

1. Every $`0tD`$ is a regular value of $`\pi `$ (hence, all the fibres $`X_t=\pi ^1(t)`$, $`t0`$, are smooth Kähler manifolds).
2. $`0`$ is a critical value of $`\pi `$ and all the critical points of $`\pi `$ are isolated.
3. $`Y`$ is isomorphic (as a complex manifold) to one of the smooth fibres of $`\pi `$, say $`X_{t_0}`$, $`t_00`$.

As in the previous section this situation is related to symplectic geometry through the Lagrangian vanishing cycle construction. As pointed out by Seidel  one can locally morsify each of the critical points in $`X_0=\pi ^1(0)`$ and then by applying Moser’s argument obtain for each critical point of $`\pi `$ at least one Lagrangian sphere in the nearby fibre $`X_ϵ`$. Since all the smooth fibres are symplectomorphic we obtain Lagrangian spheres also in $`Y`$.

Applying results from Section 4 to this situation we obtain examples of projective varieties that do not admit any degeneration with isolated singularities. For example, let $`Y`$ be any of the following:

* $`P^n`$, $`n2`$. Or more generally $`P^n\times M`$, where $`M`$ is a smooth variety with $`\pi _2(M)=0`$ and $`dim_{}Mn+1(modn+1)`$.
* Any variety whose universal cover is $`^n`$, ($`n2`$), or a domain in $`^n`$.

Then by the results in Section 4, $`Y`$ has no Lagrangian spheres, hence does not admit any degeneration as above. More examples can be found in .

This point of view seems non-trivial especially when $`H_n(Y;)=0`$, where $`n=dim_{}Y`$. In these cases the vanishing cycles are zero in homology and it seems that there are no obvious topological obstructions for degenerating $`Y`$ as above. From the list above, the first non-trivial example should be $`P^n`$ with $`n=\text{odd}3`$. It would be interesting to figure out to which extent the above statement could be proved within the tools of pure algebraic geometry. Note that Lagrangian spheres are a non-algebraic object and it seems that their existence/non-existence cannot be formalized in purely algebro-geometric terms.

Another direction of applications should be to find an upper bound on the number of singular points of an algebraic variety $`X_0`$ that can be obtained from a degeneration of $`Y`$. Note that the vanishing cycles of different singular points of $`X_0`$ are disjoint. Thus the idea here is to obtain an upper bound on the number of possible disjoint Lagrangian spheres that can be embedded in $`Y`$. The simplest test case here should be the quadric $`Q=\{z_0^2+\mathrm{}+z_{n+1}^2=0\}P^{n+1}`$, where $`n2`$. Clearly $`Q`$ can be degenerated to a variety $`X_0`$ with isolated singularities (e.g. to a cone over a smaller dimensional quadric). It seems reasonable to expect that in every such degeneration the singular fibre $`X_0`$ will have only one singular point. Note that for $`n=\text{even}`$ this easily follows from topological reason but it may not be so when $`n=\text{odd}3`$ because $`H_n(Q;)=0`$. From a symplectic point of view the above statement would follow if we could prove that every two Lagrangian spheres in $`Q`$ must intersect. This is currently still unknown but there are evidences supporting this conjecture . It is likely that a refinement of the methods from  would be useful for this purpose. More generally, one could try to bound the number of singular fibres in a degeneration of other hypersurfaces $`\mathrm{\Sigma }P^{n+1}`$ (in terms of $`\mathrm{deg}(\mathrm{\Sigma })`$ and $`n`$). See  for the conjectured bounds.

Acknowledgments. I am indebted to Leonid Polterovich for sharing with me his insight into symplectic intersections in general and especially regarding the problems presented in Section 3. Numerous discussions with him have influenced my conceptions on the subject. I wish to thank him also for valuable comments on earlier drafts of the paper. I would like to thank also Jonathan Wahl who told me about Sommese’s work , Olivier Debarre for the reference to the works of Zak  and Ein , and Paul Seidel for useful discussions on symplectic aspects of singularity theory and the material presented in section 5.


text:
# Algebraic Structures on Valuations,Their Properties and Applications

## 0. Introduction

The theory of continuous valuations on convex sets generalizes, in a sense, both the measure theory and the theory of the Euler characteristic. Roughly speaking one should think of a continuous valuation $`\varphi `$ on a real linear space $`V`$ as a finite additive measure on a class of compact nice subsets of $`V`$ (say piecewise smooth submanifolds with corners) which satisfy the following additional property (instead of the usual sigma-additivity): the restriction of $`\varphi `$ to the subclass of convex compact domains with smooth boundary extends by continuity to the class $`𝒦(V)`$ of all convex compact subsets of $`V`$. Here the continuity is understood in the sense of the Hausdorff metric on $`𝒦(V)`$. Remind that the Hausdorff metric $`d_H`$ on $`𝒦(V)`$ depends on the choice of the Euclidean metric on $`V`$ and it is defined as follows: $`d_H(A,B):=inf\{\epsilon >0|A(B)_\epsilon \text{ and }B(A)_\epsilon \},`$ where $`(U)_\epsilon `$ denotes the $`\epsilon `$-neighborhood of a set $`U`$. This condition of continuity turns out to be very strong restriction and has a lot of consequences on purely algebraic level. These properties will be discussed in this paper. The simplest examples of such valuations are any smooth measure on $`V`$ and the Euler characteristic. Also it turns out that one of the main tools used recently in investigations of valuations is the representation theory of real reductive groups and the Beilinson-Bernstein theory of $`D`$-modules.

Now let us give the formal definition of valuation.

###### 0.1.1 Definition.

a) A function $`\varphi :𝒦(V)\text{ }\text{ }\text{C}`$ is called a valuation if for any $`K_1,K_2𝒦(V)`$ such that their union is also convex one has

$$\varphi (K_1K_2)=\varphi (K_1)+\varphi (K_2)\varphi (K_1K_2).$$

b) A valuation $`\varphi `$ is called continuous if it is continuous with respect the Hausdorff metric on $`𝒦(V)`$.

The linear space of all continuous valuations on $`V`$ will be denoted by $`CVal(V)`$. It is a Fréchet space with the topology of uniform convergence on compact subsets of $`𝒦(V)`$. In Section 1 we discuss its dense subspace of polynomial smooth valuations $`(PVal(V))^{sm}`$ (it has the topology of inductive limit of Fréchet spaces). It turns out that this space has a natural structure of associative commutative unital algebra (when the unity is the Euler characteristic). In Section 2 we discuss the space $`Val(V)`$ of translation invariant continuous valuations. Its dense subspace $`(Val(V))^{sm}`$ of so called smooth valuations is a subalgebra of $`(PVal(V))^{sm}`$. It has a natural grading and satisfies a version of Poincaré duality. This property follows from the Irreducibility Theorem 2.1.3 which is by itself key result in the investigation of valuations (see Subsection 2.1). Moreover even smooth translation invariant continuous valuations form a graded subalgebra of $`(Val(V))^{sm}`$ and satisfy a version of the hard Lefschetz theorem (Subsection 2.2). This property turns out to be closely related to the cosine transform problem in the (Gelfand style) integral geometry solved recently in . These properties of valuations turn out to be useful to obtain new explicit classification results on valuations with additional invariance properties. The classical Hadwiger theorem describes explicitly $`SO(n)`$\- and $`O(n)`$-invariant translation invariant continuous valuations on the Euclidean space $`\text{ }\text{ }\text{R}^n`$. The new result is the classification of unitarily invariant translation invariant continuous valuations on the Hermitian space $`\text{ }\text{ }\text{C}^n`$ (Subsection 2.3). The main application of the classification results on valuations is integral geometric formulas. Using our classification we obtain new results in (Chern style) integral geometry of real submanifolds of complex spaces (Section 3).

## 1. General continuous valuations

In order to study general continuous valuations let us remind the definition of polynomial valuation introduced by Khovanskii and Pukhlikov , .

###### 1.1.1 Definition.

A valuation $`\varphi `$ is called polynomial of degree $`d`$ if for every $`K𝒦(V)`$ the function $`x\varphi (K+x)`$ is a polynomial on $`V`$ of degree at most $`d`$.

Note that valuations polynomial of degree 0 are called translation invariant valuations. Polynomial valuations have many nice combinatorial-algebraic properties (, ). Also in  the author have classified explicitly rotation invariant polynomial continuous valuations on a Euclidean space.

Let us denote the space of polynomial continuous valuations on $`V`$ by $`PVal(V)`$. One has

###### 1.1.2 Proposition ().

The space $`PVal(V)`$ of polynomial continuous valuations is dense in the space of all continuous valuations $`CVal(V)`$.

The proof of this proposition is rather simple; it is a tricky use of a form of the Peter-Weyl theorem (for the orthogonal group $`O(n)`$), and in particular the convexity is not used in any essential way.

Let us remind the basic definition of a smooth vector for a representation of a Lie group. Let $`\rho `$ be a continuous representation of a Lie group $`G`$ in a Fréchet space $`F`$. A vector $`\xi F`$ is called $`G`$-smooth if the map $`g\rho (g)\xi `$ is infinitely differentiable map from $`G`$ to $`F`$. It is well known the the subset $`F^{sm}`$ of smooth vectors is a $`G`$-invariant linear subspace dense in $`F`$. Moreover it has a natural topology of a Fréchet space (which is stronger than that induced from $`F`$), and the representation of $`G`$ is $`F^{sm}`$ is continuous.

We will especially be interested in polynomial valuations which are $`GL(V)`$-smooth. This space will be denoted by $`(PVal(V))^{sm}`$.

Example. Let $`\mu `$ be a measure on $`V`$ with a polynomial density with respect to the Lebesgue measure. Let $`A𝒦(V)`$ be a strictly convex compact subset with smooth boundary. Then

$$\varphi (K):=\mu (K+A)$$

is a continuous polynomial smooth valuation (here $`K+A:=\{k+a|kK,aA\}`$).

Let us denote by $`𝒢(V)`$ the linear space of valuations on $`V`$ which are finite linear combinations of valuations from the previous example. It can be shown (using Irreducibility Theorem 2.1.3) that $`𝒢(V)`$ is dense in $`(PVal(V))^{sm}`$. Let $`W`$ be another linear real vector space. Let us define the exterior product $`\varphi \psi 𝒢(V\times W)`$ of two valuations $`\varphi 𝒢(V),\psi 𝒢(W)`$. Let $`\varphi (K)=_i\mu _i(K+A_i),\psi (L)=_j\nu _j(L+A_j)`$. Define

$$(\varphi \psi )(M):=\underset{i,j}{}(\mu _i\nu _j)(M+(A_i\times \{0\})+(\{0\}\times B_j)),$$

where $`\mu _i\nu _j`$ denotes the usual product measure.

###### 1.1.3 Proposition ().

For $`\varphi 𝒢(V),\psi 𝒢(W)`$ their exterior product $`\varphi \psi 𝒢(V\times W)`$ is well defined; it is bilinear with respect to each argument. Moreover

$$(\varphi \psi )\eta =\varphi (\psi \eta ).$$

Now let us define a product on $`𝒢(V)`$. Let $`\mathrm{\Delta }:VV\times V`$ denote the diagonal imbedding. For $`\varphi ,\psi 𝒢(V)`$ let

$$\varphi \psi :=\mathrm{\Delta }^{}(\varphi \psi ),$$

where $`\mathrm{\Delta }^{}`$ denotes the restriction of a valuation on $`V\times V`$ to the diagonal.

###### 1.1.4 Proposition ().

The above defined multiplication uniquely extends by continuity to $`(PVal(V))^{sm}`$. Then $`(PVal(V))^{sm}`$ becomes an associative commutative unital algebra where the unit is the Euler characteristic $`\chi `$.

## 2. Translation invariant continuous valuations

For a linear finite dimensional real vector space $`V`$ let us denote by $`Val(V)`$ the space of translation invariant continuous valuations on $`V`$. This is a Fréchet space with respect to the topology of uniform convergence on compact subsets of $`𝒦(V)`$. In this section we will discuss properties of this space.

### 2.1. Irreducibility theorem and Poincaré duality

It was shown by P. McMullen  that the space $`Val(V)`$ of translation invariant continuous valuations on $`V`$ has a natural grading given by the degree of homogeneity of valuations. Let us formulate this more precisely.

###### 2.1.1 Definition.

A valuation $`\varphi `$ is called homogeneous of degree $`k`$ if for every convex compact set $`K`$ and for every scalar $`\lambda >0`$

$$\varphi (\lambda K)=\lambda ^k\varphi (K).$$

Let us denote by $`Val_k(V)`$ the space of translation invariant continuous valuations homogeneous of degree $`k`$.

###### 2.1.2 Theorem (McMullen ).

$$Val(V)=\underset{k=0}{\overset{n}{}}Val_k(V),$$

where $`n=dimV`$.

Note in particular that the degree of homogeneity is an integer between 0 and $`n=dimV`$. It is known that $`Val_0(V)`$ is one-dimensional and is spanned by the Euler characteristic $`\chi `$, and $`Val_n(V)`$ is also one-dimensional and is spanned by a Lebesgue measure . The space $`Val_n(V)`$ is also denoted by $`Dens(V)`$ (the space of densities on $`V`$). One has further decomposition with respect to parity:

$$Val_k(V)=Val_k^{ev}(V)Val_k^{odd}(V),$$

where $`Val_k^{ev}(V)`$ is the subspace of even valuations ($`\varphi `$ is called even if $`\varphi (K)=\varphi (K)`$ for every $`K𝒦(V)`$), and $`Val_k^{odd}(V)`$ is the subspace of odd valuations ($`\varphi `$ is called odd if $`\varphi (K)=\varphi (K)`$ for every $`K𝒦(V)`$). The Irreducibility Theorem is as follows.

###### 2.1.3 Theorem (,).

The natural representation of the group $`GL(V)`$ on each space $`Val_k^{ev}(V)`$ and $`Val_k^{odd}(V)`$ is irreducible.

This theorem is the main tool in further investigations of valuations and classification of them (see Subsection 2.3). This immediately implies so called McMullen’s conjecture . Its proof is heavily based on the use of the representation theory of real reductive groups and the Beilinson-Bernstein theory of D-modules. Another key tool in the proof of this result is the Klain-Schneider characterization of simple translation invariant continuous valuations , .

By the results of Section 1 $`(Val(V))^{sm}`$ is a subalgebra of $`(PVal(V))^{sm}`$. It is easy to see that the algebra structure is compatible with the grading, namely

$$(Val_i(V))^{sm}(Val_j(V))^{sm}(Val_{i+j}(V))^{sm}.$$

In particular we have

$$(Val_i(V))^{sm}(Val_{ni}(V))^{sm}Dens(V).$$

A version of the Poincaré duality theorem says that this is a perfect pairing. More precisely

###### 2.1.4 Theorem ().

The induced map

$$(Val_i(V))^{sm}(Val_{ni}(V)^{})^{sm}Dens(V)$$

is an isomorphism.

### 2.2. Even translation invariant continuous valuations

Let us denote by $`Val^{ev}(V)`$ the subspace of even translation invariant continuous valuations. Then clearly $`(Val^{ev}(V))^{sm}`$ is a subalgebra of $`(Val(V))^{sm}`$. It turns out that it satisfies a version of the hard Lefschetz theorem which we are going to describe.

Let us fix on $`V`$ a scalar product. Let $`D`$ denote the unit ball with respect to this product. Let us define an operator $`\mathrm{\Lambda }:Val(V)Val(V).`$ For a valuation $`\varphi Val(V)`$ set

$$(\mathrm{\Lambda }\varphi )(K):=\frac{d}{d\epsilon }|_{\epsilon =0}\varphi (K+\epsilon D).$$

(Note that by a result of P. McMullen  $`\varphi (K+\epsilon D)`$ is a polynomial in $`\epsilon >0`$ of degree at most $`n`$.) It is easy to see that $`\mathrm{\Lambda }`$ preserves the parity of valuations and decreases the degree of homogeneity by 1. In particular

$$\mathrm{\Lambda }:Val_k^{ev}(V)Val_{k1}^{ev}(V).$$

The following result is a version of the hard Lefschetz theorem.

###### 2.2.1 Theorem ().

Let $`k>n/2`$. Then

$$\mathrm{\Lambda }^{2kn}:(Val_k^{ev}(V))^{sm}(Val_{nk}^{ev}(V))^{sm}$$

is an isomorphism. In particular for $`1i2kn`$ the map

$$\mathrm{\Lambda }^i:(Val_k^{ev}(V))^{sm}(Val_{ki}^{ev}(V))^{sm}$$

is injective.

Note that the proof of this result is based on the solution of the cosine transform problem due to J. Bernstein and the author , which is the problem from (Gelfand style) integral geometry motivated by stochastic geometry and going back to G. Matheron .

### 2.3. Valuations invariant under a group

Let $`G`$ be a subgroup of $`GL(V)`$. Let us denote by $`Val^G(V)`$ the space of $`G`$-invariant translations invariant continuous valuations. From the results of  and  follows the following result.

###### 2.3.1 Theorem.

Let $`G`$ be a compact subgroup of $`GL(V)`$ acting transitively on the unit sphere. Then $`Val^G(V)`$ is a finite dimensional graded subalgebra of $`(Val(V))^{sm}`$. It satisfies the Poincaré duality, and if $`IdG`$ it satisfies the hard Lefschetz theorem.

It turns out that $`Val^G(V)`$ can be described explicitly (as a vector space) for $`G=SO(n),O(n),\text{ and }U(n)`$. In the first two cases it is the classical theorem of Hadwiger , the last case is new (see ). In order to state these results we have to introduce first sufficiently many examples.

Let $`\mathrm{\Omega }`$ be a compact domain in a Euclidean space $`V`$ with a smooth boundary $`\mathrm{\Omega }`$. Let $`n=dimV`$. For any point $`s\mathrm{\Omega }`$ let $`k_1(s),\mathrm{},k_{n1}(s)`$ denote the principal curvatures at $`s`$. For $`0in1`$ define

$$V_i(\mathrm{\Omega }):=\frac{1}{n}\left(\genfrac{}{}{0pt}{}{n1}{n1i}\right)^1_\mathrm{\Omega }\{k_{j_1},\mathrm{},k_{j_{n1i}}\}𝑑\sigma ,$$

where $`\{k_{j_1},\mathrm{},k_{j_{n1i}}\}`$ denotes the $`(n1i)`$-th elementary symmetric polynomial in the principal curvatures, $`d\sigma `$ is the measure induced on $`\mathrm{\Omega }`$ by the Euclidean structure. It is well known that $`V_i`$ (uniquely) extends by continuity in the Hausdorff metric to $`𝒦(V)`$. Define also $`V_n(\mathrm{\Omega }):=vol(\mathrm{\Omega })`$. Note that $`V_0`$ is proportional to the Euler characteristic $`\chi `$. It is well known that $`V_0,V_1,\mathrm{},V_n`$ belong to $`Val^{O(n)}(V)`$. It is easy to see that $`V_k`$ is homogeneous of degree $`k`$. The famous result of Hadwiger says

###### 2.3.2 Theorem (Hadwiger, ).

Let $`V`$ be $`n`$-dimensional Euclidean space. The valuations $`V_0,V_1,\mathrm{},V_n`$ form a basis of $`Val^{SO(n)}(V)(=Val^{O(n)}(V))`$.

Now let us describe unitarily invariant valuations on a Hermitian space. Let $`W`$ be a Hermitian space, i.e. a complex vector space equipped with a Hermitian scalar product. Let $`m:=dim_{\text{ }\text{ }\text{C}}W`$ (thus $`dim_{\text{ }\text{ }\text{R}}W=2m`$). For every non-negative integers $`p`$ and $`k`$ such that $`2pk2m`$ let us introduce the following valuations:

$$U_{k,p}(K)=_{E{}_{}{}^{\text{ }\text{ }\text{C}}𝒜Gr_{mp}}V_{k2p}(KE)𝑑E.$$

Then $`U_{k,p}Val_k^{U(m)}(W)`$.

###### 2.3.3 Theorem ().

Let $`W`$ be a Hermitian vector space of complex dimension $`m`$. The valuations $`U_{k,p}`$ with $`0p\frac{min\{k,2mk\}}{2}`$ form a basis of the space $`Val_k^{U(m)}(W)`$.

It turns out that the proof of this theorem is highly indirect, and it uses everything known about even translation invariant continuous valuations including the solution of McMullen’s conjecture, cosine transform, hard Lefschetz theorem for valuations, and also results of Howe and Lee  on the structure of certain $`GL_n(\text{ }\text{ }\text{R})`$-modules. Namely in order to describe explicitly the (finite dimensional) space of unitarily invariant valuations it is necessary to study the (infinite dimensional) $`GL_{\text{ }\text{ }\text{R}}(W)`$-module $`Val^{ev}(W)`$.

Note that as algebra $`Val^{SO(n)}(V)`$ is isomorphic to $`\text{ }\text{ }\text{C}[x]/(x^{n+1}`$. The algebra structure of $`Val^{U(m)}(W)`$ is not yet computed.

## 3. Applications to integral geometry

In this section we state new results from (Chern style) integral geometry of Hermitian spaces. They are obtained by the author in  using the classification of unitarily invariant valuations described in Subsection 2.3 of this paper. They can be considered as a generalization of the classical kinematic formulas due to Chern, Crofton, Santaló, and others (see e.g. , ,,, ).

Let us remind first the principal kinematic formula following Chern . Let $`ISO(n)`$ denote the group of affine isometries of the Euclidean space $`\text{ }\text{ }\text{R}^n`$. Let $`\mathrm{\Omega }_1,\mathrm{\Omega }_2`$ be compact domains with smooth boundary in $`\text{ }\text{ }\text{R}^n`$. Assume also that $`\mathrm{\Omega }_1U(\mathrm{\Omega }_2)`$ has finitely many components for all $`UISO(n)`$.

###### 3.1.1 Theorem ().

$$_{UISO(n)}\chi (\mathrm{\Omega }_1U(\mathrm{\Omega }_2))𝑑U=\underset{k=0}{\overset{n}{}}\kappa _kV_k(\mathrm{\Omega }_1)V_{nk}(\mathrm{\Omega }_2),$$

where $`\kappa _k`$ are constants depending on $`k`$ and $`n`$ only which can be written down explicitly.

For the explicit form of the constants $`\kappa _k`$ we refer to  or , Ch.15 §4.

Let us return back to the Hermitian situation. Let $`IU(m)`$ denote the group of affine isometries of the Hermitian space $`\text{ }\text{ }\text{C}^m`$ preserving the complex structure (then $`IU(m)`$ is isomorphic to $`\text{ }\text{ }\text{C}^mU(m)`$). Let $`\mathrm{\Omega }_1,\mathrm{\Omega }_2`$ be compact domains with smooth boundary in $`\text{ }\text{ }\text{C}^m`$ such that $`\mathrm{\Omega }_1U(\mathrm{\Omega }_2)`$ has finitely many components for all $`UIU(m)`$. The new result is

###### 3.1.2 Theorem ().

$$_{UIU(m)}\chi (\mathrm{\Omega }_1U(\mathrm{\Omega }_2))𝑑U=\underset{k_1+k_2=2m}{}\underset{p_1,p_2}{}\kappa (k_1,k_2,p_1,p_2)U_{k_1,p_1}(\mathrm{\Omega }_1)U_{k_2,p_2}(\mathrm{\Omega }_2),$$

where the inner sum runs over $`0p_ik_i/2,i=1,2`$, and $`\kappa (k_1,k_2,p_1,p_2)`$ are certain constants depending on $`m,k_1,k_2,p_1,p_2`$ only.

Remark. We could compute explicitly the constants $`\kappa (k_1,k_2,p_1,p_2)`$ only in $`\text{ }\text{ }\text{C}^2`$.

For more integral geometric formulas of this and other type for real domains in $`\text{ }\text{ }\text{C}^m`$ we refer to .