text:
# Non-zero Degree Maps between 3-Manifolds*

## 0. Introduction

The topology of surfaces was completely understood by the end of 19th century, but maps between surfaces keep to be an active topic in the 20th century, and some basic results just appeared in the last 25 years, among which are the Nielsen-Thurston classification of surface automorphisms \[Th3\], and Edmonds’ standard form for surface maps \[E1\]. Then fine results followed, say the realization of Nielsen number in the isotopy class of surface automorphisms by Jiang \[J\], and the simple loop theorem for surface maps by Gabai \[Ga2\].

The topology of 3-manifolds was well-understood only in the later 20th century due to many people’s deep results, in particular Thurston’s great contribution to the geometrization of 3-manifolds, and the topic of non-zero degree maps between 3-manifolds becomes active only rather recently.

We will survey the results and questions in the topic indicated by the title, present its relations to 3-manifold topology and its applications to problems in geometry group theory, fixed point theory and dynamics. The methods used are varied, and we try to describe them briefly.

For standard terminologies of 3-manifolds and knots, see the famous books of J. Hempel, W. Jaco and D. Rolfsen. For a proper map $`f:MN`$ between oriented compact 3-manifolds, $`\text{deg}(f)`$, the degree of $`f`$, is defined in most books of algebraic topology. A closed orientable 3-manifold is said to be geometric if it admits one of the following geometries: $`H^3`$ (hyperbolic), $`\stackrel{~}{PSL_2R}`$, $`H^2\times E^1`$, Sol, Nil, $`E^3`$ (Euclidean), $`S^2\times E^1`$, and $`S^3`$ (spherical). A compact orientable 3-manifold $`M`$ admits a geometric decomposition if each prime factor of $`M`$ is either geometric or Haken. Thurston’s geometrization conjecture asserts that any closed orientable 3-manifold admits a geometric decomposition. Each Haken manifold $`M`$ with $`M`$ a (possibly empty) union of tori has a Jaco-Shalen-Johannson (JSJ) torus decomposition, that is, it contains a minimal set of tori, unique up to isotopy, cutting $`M`$ into pieces such that each piece is either a Seifert manifold or a simple manifold, which admits a complete hyperbolic structure with finite volume \[Th2\].

In the remainder of the paper, all manifolds are assumed to be compact and orientable, all automorphisms are orientation preserving, all knots are in $`S^3`$, all Kleinian groups are classical, and all maps are proper, unless otherwise specified.

Let $`M`$ and $`N`$ be 3-manifolds and $`d>0`$ an integer. We say that $`M`$ $`d`$-dominates (or simply dominates) $`N`$ if there is a map $`f:MN`$ of degree $`\pm d`$. Denote by $`D(M,N)`$ the set of all possible degrees of maps from $`M`$ to $`N`$. A 3-manifold $`M`$ is small if each closed incompressible surface in $`M`$ is boundary parallel.

Due to space limitation, quoted literature are only partly listed in the references; while the others are briefly indicated in the context.

## 1. Existence and finiteness

A fundamental question in this area (and in 3-manifold theory) is the following.

###### Question 1.1

Given a pair of closed 3-manifolds $`M`$ and $`N`$, can one decide if $`M`$ $`d`$-dominates $`N`$? In particular, can one decide if $`M`$ 1-dominates $`N`$?

The following two natural problems concerning finiteness can be considered as testing cases of Question 1.1.

###### Question 1.2 {\rm\[Ki, Problem 3.100 (Y. Rong)\]}

Let $`M`$ be a closed $`3`$-manifold. Does $`M`$ 1-dominate at most finitely many closed 3-manifolds?

###### Question 1.3

Let $`N`$ be a closed 3-manifold. When is $`|D(M,N)|`$ finite for any closed 3-manifold $`M`$?

An important progress towards the solution of Question 1.2 is the following:

###### Theorem 1.1 {\rm(\[So2\], \[WZh2\], \[HWZ3\])}

Any closed $`3`$-manifold $`1`$-dominates at most finitely many geometric $`3`$-manifolds.

Theorem 1.1 was proved by Soma when the target manifolds $`N`$ admit hyperbolic geometry \[So2\]. The proof is based on the argument of Thurston’s original approach to the deformation of acylindrical manifolds. Porti and Reznikov had a quick proof of Soma’s result, based on the volume of representations \[Re2\]. However Soma’s approach deserves attention as it proves that the topological types of all hyperbolic pieces in closed Haken manifolds 1-dominated by $`M`$ are finite \[So3\]. Theorem 1.1 was proved in \[WZh2\] when the target manifolds admit geometries of $`H^2\times E^1`$, $`\stackrel{~}{PSL_2(R)}`$, Sol or Nil. The proof for the case of $`H^2\times E^1`$ geometry invokes Gabai’s result that embedded Thurston Norm and singular Thurston Norm are equal \[Ga1\], and the proof for case of $`\stackrel{~}{PSL_2(R)}`$ geometry uses Brooks and Goldman’s work on Seifert Volume \[BG\]. Theorem 1.1 was proved in \[HWZ3\] when the target manifolds admit $`S^3`$ geometry, using the linking pairing of 3-manifolds. Note that only finitely many 3-manifolds admit the remaining two geometries.

For maps between 3-manifolds which are not necessarily orientable, there is a notion of geometric degree (See D. Epstein, Proc. London Math. Soc. 1969). It is worth mentioning that if $`d`$-dominating maps are defined in terms of geometric degree, then Rong constructs a non-orientable 3-manifold which 1-dominates infinitely many lens spaces \[Ro3\]. Actually there is a non-orientable hyperbolic 3-manifold which 1-dominates infinitely many hyperbolic 3-manifolds \[BW1\]. Such examples do not exist in dimension $`n>3`$ due to Gromov’s work on simplicial volume and H.C. Wang’s theorem that, for any $`V>0`$, there are at most finitely many closed hyperbolic $`n`$-manifolds of volume $`<V`$.

The answer to Question 1.2 is still unknown for closed irreducible 3-manifolds admitting geometric decomposition. The following result is related.

###### Theorem 1.2 {\rm(\[Ro1\], \[So3\])}

For any 3-manifold $`M`$ there exists an integer $`N_M`$, such that if $`M=M_0M_1\mathrm{}.M_k`$ is a sequence of degree one maps with $`k>N_M`$, and each $`M_i`$ admits a geometric decomposition, then the sequence contains a homotopy equivalence.

The situation for Question 1.3 can be summarized in the following theorem.

###### Theorem 1.3 {\rm(\[Gr1\], \[BG\], \[W2\])}

Suppose $`N`$ is a closed 3-manifold admitting geometric decomposition. Then

(1) $`|D(M,N)|`$ is finite if either a prime factor of $`N`$ contains a hyperbolic piece in its JSJ decomposition, or $`N`$ itself admits the geometry of $`\stackrel{~}{PSL_2(R)}`$.

(2) $`|D(N,N)|`$ is infinite if and only if either (i) $`N`$ is covered by a torus bundle over the circle or a surface$`\times S^1`$, or (ii) each prime factor of $`N`$ has a cyclic or finite fundamental group.

Part (1) of Theorem 1.3 follows from the work of Gromov \[Gr1\] and Brooks-Goldman \[BW\]. Part (2) can be found in \[W2\]. Note that if $`|D(N,N)|`$ is infinite and $`D(M,N)`$ contains non-zero integers, then $`|D(M,N)|`$ is also infinite. I suspect that Theorem 1.3 (2) indicates a general solution to Question 1.3.

There are many partial results for Question 1.1: When both $`M`$ and $`N`$ are Seifert manifolds with infinite fundamental groups Rong has an algorithm to determine if $`M`$ 1-dominates $`N`$ \[Ro3\]. When $`N`$ is the Poincare homology sphere and a Heegaard diagram of $`M`$ is given, Hayat-Legrand, Matveev and Zieschang have an algorithm to decide if $`M`$ $`d`$-dominates $`N`$ \[HMZ\]. There are simple answers to Question 1.1 in the following cases: (1) $`M`$ and $`N`$ are prism spaces and $`d=1`$ \[HWZ2\]; (2) $`M=N`$ admit geometry of $`S^3`$ and $`f_{}`$ an automorphism on $`\pi _1`$ \[HKWZ\]; (3) $`N`$ is a lens space. I will state (3) as a theorem, since both its statement and proof are short, and since it has rich connections with previous results and with different topics.

###### Theorem 1.4 {\rm(\[HWZ1\], \[HWZ3\])}

A closed $`3`$-manifold $`M`$ $`d`$-dominates the lens space $`L(p,q)`$ if and only if there is an element $`\alpha `$ in the torsion part of $`H_1(M,)`$ such that $`\alpha \alpha =\frac{dq}{p}`$ in $`/`$, where $`\alpha \alpha `$ is the self-linking number of $`\alpha `$.

A direct consequence of Theorem 1.4 is the known fact that $`L(p,q)`$ 1-dominates $`L(m,n)`$ if and only if $`p=km`$ and $`n=kqc^2`$ mod $`m`$. This fact has at least four different proofs: using equivariant maps between spheres by de Rham (J. Math. 1931) and by Olum (Ann. of Math. 1953), using Whitehead torsion by Cohen (GTM 10, 1972), using pinch in \[RoW\] and using linking pair in \[HWZ1\].

Degree one maps from general 3-manifolds to some lens spaces, in particular the $`RP^3`$, have been studied by Bredon-Wood (Invent. Math. 1969) and by Rubinstein (Pacific J. Math. 1976) to find one-sided incompressible surfaces, by Luft-Sjerve (Topo. Appl. 1990) to study cyclic group actions on 3-manifolds, by Shastri-Williams-Zvengrowski \[SWZ\] in theoretical physics, by Taylor (Topo. Appl. 1984) to define normal bordism classes of degree one maps, and by Kirby-Melvin (Invent. Math. 1991) to connect with new 3-manifold invariants.

Degree one maps induce epimorphisms on $`\pi _1`$. There are easy examples indicate that Question 1.2 does not have direct correspondence in the level of 3-manifold groups \[BW1\], \[RWZh\]. However the following related question was raised in 1970’s.

###### Question 1.4 {\rm\[Ki, Problem 1.12 (J. Simon)\]}

Conjectures:

(1) Given a knot group $`G`$, there is a number $`N_G`$ such that any sequence of epimorphisms of knot groups $`GG_1\mathrm{}.G_n`$ with $`nN_G`$ contains an isomorphism.

(2) Given a knot group $`G`$, there are only finitely many knot groups $`H`$ for which there is an epimorphism $`GH`$.

According to a conversation with Gonzalez-Acuna, who discussed Question 1.4 with Simon before it was posed, the epimorphisms in Question 1.4 are peripheral preserving in their minds.

###### Theorem 1.5 {\rm(\[So5\], \[RW\])}

The conjecture in Question 1.4 (1) holds if the knot complements involved are small. The conjecture in Question 1.4 (2) holds if the knot complements are small and the epimorphisms are peripheral preserving.

The first claim is due to Soma \[So5\] and the second claim is in \[RW\]. Both of them invoke Culler-Shalen’s work on the representation varieties of knot groups. It is also proved that any infinite sequence of epimorphisms among 3-manifold groups contains an isomorphism if all manifolds are either hyperbolic \[So5\] or Seifert fibered \[RWZh\]. In \[RWZh\], the proof uses the fact that epimorphisms between aspherical Seifert manifolds with the same $`\pi _1`$ rank are realized by maps of non-zero degree. Both this fact and Question 1.4 (1) are variations of the Hopfian property.

We end this section by mention that there are results about $`D(M,N)`$ in \[DW\] for $`(n1)`$-connected $`2n`$-manifolds, $`n>1`$, which are quite explicit and of interest from both topological and number-theoretic point of view.

## 2. Uniqueness

The following question is raised in 1970’s.

###### Question 2.1 {\rm\[Ki, Problem 3.16 (W. Thurston)\]}

Suppose a 3-manifold $`M`$ is not covered by (surface)$`\times S^1`$ or a torus bundle over $`S^1`$. Let $`f,g:MN`$ be two coverings, must $`deg(f)=deg(g)`$?

It is known \[WWu2\] that Question 2.1 has positive answer if $`M`$ admits geometric decomposition and is not a graph manifold ($`M`$ is a graph manifold if each piece of its JSJ decomposition is Seifert fibered.) For graph manifolds there are four different covering invariants introduced in middle 1990’s by \[WWu2\], Luecke and Wu \[LWu\], Neumann \[N\] and Reznikov \[Re1\] . Unfortunately all those four covering invariants are either vanishing or not well-defined for some non-trivial graph manifolds. It is also known that covering degree is uniquely determined if the graph manifold in the target is either a knot complement \[LWu\] or its corresponding graph is simple \[WWu2, N\]. The positive answer to Question 2.1 for graph manifolds was finally obtained in \[YW\], using the matrix invariant defined in \[WWu2\] and an elegant application of matrix theory due to Yu.

###### Theorem 2.1 {\rm(\[WWu2\], \[YW\])}

For 3-manifolds admitting geometric decomposition and not covered by either (surface)$`\times S^1`$ or a torus bundle over $`S^1`$, covering degrees are uniquely determined by the manifolds involved.

It is worth mentioning an interesting fact that any knot complement is non-trivially covered by at most two knot complements and any knot complement non-trivially covers at most one knot complement. The first claim follows from the cyclic surgery theorem of Culler-Gordon-Luecke-Shalen and the positive answer to the Smith Conjecture. The second claim is in \[WWu1\].

Question 2.1 is equivalent to asking the uniqueness of indices of finite index embeddings between 3-manifold groups. Recently there are also some discussions on the uniqueness of indices of self-embeddings of groups. A group $`G`$ is said to be co-Hopf if each self-monomorphism of $`G`$ is an isomorphism.

###### Question 2.2

Let $`G`$ be either a 3-manifold group, or a Kleinian group, or a word hyperbolic group. When is $`G`$ co-Hopf?

The cohopficity of groups were first considered by Baer (Bull. AMS 1944). For word hyperbolic groups it was first considered by Gromov in 1987 \[Gr2, p.157\], and subsequently by Rips-Sela (GAFA, 1994), Sela \[Se\], and Kapovich-Wise (Isreal J. Math. 2001). Cohopficity of 3-manifold groups was first studied in 1989 by Gonzalez-Acuna and Whitten \[GWh\], and then in \[WWu2\] and \[PW\]. The answer for 3-manifolds admitting geometric decomposition with boundary either empty set or a union of tori is known \[GWh\], \[WWu2\], and partial results for 3-manifolds with boundary of high genus surfaces are in \[PW\]. Cohopficity of Kleinian groups was first considered in 1992 in an early version of \[PW\], then in 1994 in an early version of \[WZh1\], also by Ohshika-Potyagailo (Ann. Sci. Ecole Norm. Sup. 1998) and Delzant-Potyagailo (MPI Preprint, 2000) for high dimensional Kleinian groups.

###### Theorem 2.2

Suppose $`K`$ is a non-elementary, freely indecomposable, geometrically finite Kleinian group and $`K`$ contains no $`ZZ`$ subgroup. Then

(1)\[Se\], \[PW\], \[WZh1\] $`K`$ is co-Hopf if $`K`$ is a group of one end.

(2)\[WZh1\] If the singular locus of the hyperbolic 3-orbifold $`H^3/K`$ is a 1-manifold, then $`K`$ is co-Hopf if and only if no circle component of singular locus meets a minimal splitting system of hyperbolic cone planes.

The proof of Theorem 2.2 (1) in \[WZh1\], influenced by that of torsion free case in \[PW\], use a generalization of Thurston-Gromov’s finiteness theorem on the conjugacy classes of group embeddings (Delzant, Comm. Math. Helv. 1995) and a proper conjugation theorem of Kleinian groups (Wang-Zhou, Geometriae Dedicata, 1995). Theorem 2.2 (2) is proved by using 3-dimensional hyperbolic orbifold structures and orbifold maps, which turn out to be useful geometric tools.

Note that groups in Theorem 2.2 are word hyperbolic groups. According to Sela (\[Se\] and his MSRI preprints in 1994), people once expected that a non-elementary word hyperbolic group is co-Hopf if and only if it has one end. Sela proved this expectation for the torsion free case \[Se\]. Theorem 2.2 (2) and examples in \[WZh1\] show that cohopficity phenomenon is very complicated in the torsion case. In particular there are co-Hopf word hyperbolic groups which have infinitely many ends.

Inspired by Questions 2.1 and 2.2 it is natural to ask

###### Question 2.3

Are the indices (including the infinity) of embeddings $`HG`$ between co-Hopf groups unique?

## 3. Interactions with 3-manifold topology

Degree one maps define a partial order on Haken manifolds and hyperbolic 3-manifolds. By Gordon-Luecke’s theorem knots are determined by their complements \[GL\]. We say that a knot $`K`$ 1-dominates a knot $`K^{}`$ if the complement of $`K`$ 1-dominates the complement of $`K^{}`$. 1-domination among knots also gives a partial order on knots. This partial order seems to provide a good measurement of complexity of 3-manifolds and knots. The reactions of non-zero degree maps between 3-manifolds and 3-manifold topology are reflected in the following very flexible

###### Question 3.1

Suppose $`M`$ and $`N`$ are 3-manifolds (knots) and $`M`$ 1-dominates ($`d`$-dominates) $`N`$.

(1) Is $`\sigma (M)`$ not “smaller” than $`\sigma (N)`$ for a topological invariant $`\sigma `$?

(2) If $`M`$ and $`N`$ are quite “close”, are they homeomorphic? do they admit the same topological structure?

Positive answers to Question 3.1 (1) are known in many cases. Suppose $`M`$ 1-dominates $`N`$. Then $`\sigma (M)\sigma (N)`$ when $`\sigma `$ is either the rank of $`\pi _1`$, or Gromov’s simplicial volume, or Haken number (of incompressible surfaces), or genus of knots; $`\sigma (N)`$ is a direct summand of $`\sigma (M)`$ when $`\sigma `$ is the homology group, and $`\sigma (N)`$ is a factor of $`\sigma (M)`$ if $`\sigma `$ is the Alexander polynomial of knots. The answer to Question 3.1 (1) is still unknown for many invariants of knots and 3-manifolds, for example crossing number, unknotting number, Jones polynomial, knot energy, and tunnel number, etc. Li and Rubinstein are specially interested in Question 3.1 (1) for Casson invariant in order to prove it is a homotopy invariant \[LRu\].

There are both positive and negative answers to Question 3.1 (2), depending on the interpretation of the problem. On the negative side, Kawauchi has constructed, using the imitation method invented by himself, degree one maps between non-homeomorphic 3-manifolds $`M`$ and $`N`$ with many topological invariants identical, see his survey paper \[Ka\]. On the positive side, there are many results. An easy one is that if $`M`$ $`d`$-dominates $`N`$ and both $`M`$ and $`N`$ are aspherical Seifert manifolds, then the Euler number of $`M`$ is zero if and only if that of $`N`$ is zero \[W1\]. A deeper result is Gromov-Thurston’s Rigidity theorem, which says that a degree one map between hyperbolic 3-manifolds of the same volume is homotopic to an isometry \[Th2\]. The following are some recent results in this direction.

###### Theorem 3.1 {\rm(\[So4\], \[So1\])}

(1) For any $`V>0`$, suppose $`f:MN`$ is a degree one map between closed hyperbolic 3-manifolds with $`Vol(M)<V`$. Then there is a constant $`c=c(V)`$ such that $`(1c)\text{V}ol(M)\text{V}ol(N)`$ implies that $`f`$ is homotopic to an isometry.

(2) If $`MN`$ is a map of degree $`d`$ between Haken manifolds such that $`M=dN`$, then $`f`$ can be homotoped to send $`H(M)`$ to $`H(N)`$ by a covering, where $`||||`$ is the Gromov norm and $`H()`$ is the hyperbolic part under the JSJ decomposition.

###### Theorem 3.2 {\rm(\[BW1\], \[BW2\])}

(1) Let $`M`$ and $`N`$ be two closed irreducible 3-manifolds with the same first Betti number and suppose $`M`$ is a surface bundle. If $`f:MN`$ is a map of degree $`d`$, then $`N`$ is also a surface bundle.

(2) Let $`M`$ and $`N`$ be two closed, small hyperbolic 3-manifolds. If there is a degree one map $`f:MN`$ which is a homeomorphism outside a submanifold $`HN`$ of genus smaller than that of $`N`$, then $`M`$ and $`N`$ are homeomorphic.

(1) and (2) of Theorem 3.1 provide a stronger version and a generalization of Gromov-Thurston’s Rigidity theorem, respectively. In respect of Theorems 3.2, the following examples should be mentioned: There are degree one maps between two non-homeomorphic hyperbolic surface bundles with the same first Betti number and between two non-homeomorphic small hyperbolic 3-manifolds \[BW2\]. The constructions of those maps are quite non-trivial. There are many applications of Theorem 3.2. We list two of them which are applications of Theorem 3.2 to Thurston’s surface bundle conjecture and to Dehn surgery respectively, where degree one maps constructed by surgery on null-homotopic knots are involved.

###### Theorem 3.3 {\rm(\[BW1\], \[BW2\])}

(1) There are closed hyperbolic 3-manifolds $`M`$ such that any tower of abelian covering of $`M`$ contains no surface bundle.

(2) Suppose $`M`$ is a small hyperbolic 3-manifold and that $`kM`$ is a null-homotopic knot, which is not in a 3-ball. If the unknotting number of $`k`$ is smaller than the Heegaard genus of $`M`$, then every closed 3-manifold obtained by a non-trivial Dehn surgery on $`k`$ contains an incompressible surface.

## 4. Standard forms

###### Question 4.1

What are standard forms of non-zero degree maps and of automorphisms of 3-manifolds?

Sample answers to analogs of Question 4.1 in dimension 2 are that each map of non-zero degree between closed surfaces is homotopic to a pinch followed by a branched covering \[E1\], and each automorphism on surfaces can be isotoped to a map which is either pseudo Anosov (Anosov), or periodic, or reducible \[Th3\].

###### Theorem 4.1 {\rm(Haken, Waldhausen, \[E2\], \[Ro2\])}

(1) A degree one map between closed 3-manifolds is homotopic to a pinch.

(2) A map of degree at least three between closed 3-manifolds is homotopic to a branched covering.

(3) A non-zero degree map between Seifert manifolds with infinite $`\pi _1`$ is homotopic to a fiber preserving pinch followed by a fiber preserving branched covering.

(1) is proved by Haken (Illinois J. Math. 1966), also by Waldhausen, and a quick proof using differential topology is in \[RoW\]. (2) is proved by Edmonds \[E2\] quickly after Hilden-Montesinos’s result that each 3-manifold is a 3-fold branched covering of 3-sphere. (3) is due to Rong \[Ro2\], which invokes \[E1\]. According to conversations with D. Gabai and with M. Freedman, people are still wondering if each map of degree 2 between closed 3-manifolds is homotopic to a pinch followed by a double branched covering.

For non-prime 3-manifolds, Cesar de Sa and Rourke claim that every automorphism is a composition of those preserving and permuting prime factors (Bull. AMS, 1979), and those so-called sliding maps. A proof is given by Hendricks and Laudenbach \[HL\], and by McCullough \[Mc\].

Standard forms of automorphisms on prime 3-manifolds admitting geometric decomposition have been studied in \[JWW\]. The orbifold version of Nielsen-Thurston’s classification of surface automorphisms is established, i.e., each orbifold automorphism is orbifold-isotopic to a map which is either (pseudo) Anosov, or periodic, or reducible. We then have the following theorem.

###### Theorem 4.2 {\rm(\[JWW\])}

Let $`M`$ be a closed prime 3-manifold admitting geometric decomposition. Let $`f:MM`$ be an automorphism. Let $`𝒯`$ be the product neighborhood of the JSJ tori. Then

(1) $`f`$ is isotopic to an affine map if $`M`$ is a 3-torus.

(2) $`f`$ is isotopic to an isometry if $`M`$ is the Euclidean manifold having a Seifert fibration over $`RP^2`$ with two singular points of index 2.

(3) $`f`$ is isotopic to a map which preserves the torus bundle structure over 1-orbifold if $`M`$ admits the geometry of Sol.

(4) for all the remaining cases, $`f`$ can be isotoped so that $`𝒯`$ is invariant under $`f`$, and for each $`f`$-orbit $`O`$ of the components in $`\{𝒯,\overline{M𝒯}\}`$, $`f|O`$ is an isometry if $`O`$ is hyperbolic, $`f|O`$ is affine if $`O`$ belongs to $`𝒯`$, otherwise there is a Seifert fibration on $`O`$ so that $`f`$ is fiber preserving and the induced map on the orbifold is either periodic, or (pseudo) Anosov, or reducible.

As in dimension 2, standard forms in Theorems 4.2 are useful in the study of fixed point theory and dynamics of 3-manifold automorphisms. The following is a result in this direction.

###### Theorem 4.3 {\rm(\[JWW\])}

Suppose $`M`$ is a closed prime 3-manifold admitting geometric decomposition and $`f:MM`$ is an automorphism. Then

(1) the Nielsen number $`N(f)`$ is realized in the isotopy class of $`f`$.

(2) $`f`$ is isotopic to a fixed point free automorphism unless some component of the JSJ decomposition of $`M`$ is a Seifert manifold whose orbifold is neither a 2-sphere with a total of at most three holes or cone points nor a projective plane with a total of at most two holes or cone points.

## References


format: text

text:
# 1 Introduction

## 1 Introduction

It is widely believed that most complex systems are unpredictable, with concrete implications in earthquake prediction111See the contributions in Nature debates on earthquake prediction at http://helix.nature.com/debates/earthquake/., in engineering failure \[Karplus (1992)\], and in financial markets \[Fama (1998)\] to cite a few. In addition to the persistent failures of predictive schemes for these systems, concepts such as self-organized criticality \[Bak (1996)\] suggest an intrinsic impossibility for the prediction of catastrophes. Several recent works suggest a different picture: catastrophes may result from novel mechanisms amplifying their size \[Sornette (1999), Sornette (2002), L’vov, Pomyalov & Procaccia (2001)\] and may thus be less unpredictable than previously thought. This idea has been mostly explored in material failure \[Anifrani, Lefloch, Sornette & Souillard (1995), Garcimartin, Guarino, Bellon, & Ciliberto (1997), Johansen & Sornette (2000)\], in earthquakes \[Keilis-Borok & Soloviev (2003)\], and in financial markets and emerged in the latter from the analysis of cumulative losses (drawdowns) \[Johansen & Sornette (1998), Johansen & Sornette (2001a)\], from measures of algorithmic complexity \[Mansilla (2001)\], and from agent-based models \[Lamper, Howison & Johnson (2002)\].

We present novel empirical tests that provide a strong support for the hypothesis that large events can be predicted. We focus our analysis on financial indices (typically the daily Dow Jones Industrial Average (DJIA) from 26-May-1896 to 11-Mar-2003) as they provide perhaps the best data sets that can be taken as proxies for other complex systems. Our methodology is based on the assumption that fast large market drops (crashes) are the results of interactions of market players resulting in herding behavior: exogenous shocks (due to changes in market fundamentals) often do not play an important role, which is at odds with standard economy theory. The fact that exogenous shocks may not be the most important driving causes of the structures found in financial time series has been shown to be the case for volatility shocks \[Sornette, Malevergne & Muzy (2003)\]: most of the bursts of volatility in major US indices can be explained by an endogenous organization involving long-memory processes, while only a few major shocks such as 9/11/2001 or the coup against Gorbachev in 1991 are found to leave observable signatures. Concerning financial crashes in indices, bonds and currencies, \[Johansen & Sornette (2003)\] have performed an extended analysis of the distribution of drawdowns (cumulative losses) in the two leading exchange markets (US dollar against the Deutsch and against the Yen), in the major world stock markets, in the U.S. and Japanese bond market and in the gold market and have shown the existence of “outliers,” in the sense that the few largest drawdowns do not belong to the same distribution. For each identified outlier, \[Johansen & Sornette (2003)\] have checked whether the formula (1) given below, expressing a so-called log-periodic power law signature (LPPL), could fit the price time series preceding them; if yes, the existence of the LPPL was taken as the qualifying signature for an endogenous crash (a drawdown outlier was seen as the end of a speculative unsustainable accelerating bubble generated endogenously). In the absence of LPPL, \[Johansen & Sornette (2003)\] were able to identify the relevant historical event, i.e., a new piece of information of such magnitude and impact that it is reasonable to attribute the crash to it, following the standard view of the efficient market hypothesis. Such drawdown outliers were thus classified as having an exogenous origin. Globally over all the markets analyzed, \[Johansen & Sornette (2003)\] identified 49 outliers, of which 25 were classified as endogenous, 22 as exogenous and 2 as associated with the Japanese “anti-bubble” starting in Jan. 1990. Restricting to the world market indices, \[Johansen & Sornette (2003)\] found 31 outliers, of which 19 are endogenous, 10 are exogenous and 2 are associated with the Japanese anti-bubble. The combination of the two proposed detection techniques, one for outliers in the distribution of drawdowns and the second one for LPPL, provided a systematic taxonomy of crashes. The present paper goes one step further to propose a prediction scheme for crashes, and by extension large and rapid changes of regime.

Our proposed learning algorithm belongs to a recent body of literature that challenges the Efficient Market Hypothesis (EMH). In the context of Information Theory which is related to the present pattern recognition approach, see for instance \[Shmilovici, Alon-Brimer, & Hauser (2003)\]. Our approach is also related to the field of nonlinearity applied to financial time series: see \[Brock (2000)\] for a review and \[Hsieh (1989), Brock (1993), Hsieh (1995), Kaboudan (1996), Brock and Hommes (1997), Harriff (1997), Brock and Hommes (1998)\] for tests and models of nonlinearity in financial time series. However, while we use a deterministic function to fit bubble periods, our approach is not deterministic because our underlying rational expectation bubble theory acknowledges the stochastic component introduced by the crash hazard rate and by the random regime shifts associated with the nucleation of bubble periods. In addition, the LPPL pattern that we propose to characterize these bubble periods, while rather rigid in its geometrical proportions exemplifying the concept of discrete scale symmetry (see below), is different from one bubble to the next as its time span and amplitude can change.

This paper is organized as follows. Section 2 presents the theoretical foundation of our approach. Section 3 describes the definition of the pattern recognition method and its implementation and tests on the DJIA. Section 4 shows further tests on the Hong Kong Hang-Seng index and section 5 concludes.

## 2 Theoretical foundation of our approach

Our key idea is to test for signatures of collective behaviors similar to those well-known in condensed-matter and statistical physics. The existence of similar collective behaviors in physics and in markets may be surprising to those not involved in the study of complex systems. Indeed, in physics, the governing laws are well established and tested whereas one could argue that there is no well-established fundamental law in stock markets. However, the science of complexity developed in the last two decades has shown that the emergence of collective phenomena proceeds similarly in physics, biology or social sciences as long as one is interested in the coarse-grained properties of large assemblies of constituting elements or agents (see for instance \[Anderson, Arrow, & Pines (1988), Bak (1996), Sornette (2003a), Sornette (2004)\] and references therein).

Collective behaviors in population of agents can emerge through the forces of imitation leading to herding \[Scharfstein & Stein (1990), Lux (1995), Graham (1999)\]. Herding behavior of investors is reflected in significant deviations of financial prices from their fundamental values \[Shiller (2000)\], leading to so-called speculative bubbles \[Flood & Garber (1994)\] and excess volatility \[Shiller (1989)\]. The similarity between herding and statistical physics models, such as the Ising model, has been noted by many authors \[Callen & Shapero (1974), Montroll & Badger (1974), Orléan (1995), Krawiecki, Holyst & Helbing (2002), Sornette (2003a), Sornette (2003b)\]. We use this similarity to model a speculative bubble as resulting from positive feedback investing, leading to a faster-than-exponential power law growth of the price \[Johansen, Sornette & Ledoit (1999), Johansen, Ledoit & Sornette (2000), Sornette & Andersen (2002)\]. In addition, the competition between such nonlinear positive feedbacks, negative feedbacks due to investors trading on fundamental value and the inertia of decision processes lead to nonlinear oscillations (nonlinear in the sense of amplitude-dependence frequencies) \[Ide & Sornette (2002)\] that can be captured approximately by adding an imaginary part to the exponent of the power law price growth (by power law price growth, we refer to the super-exponential growth typically associated with the solution of the equation $`dp/dt=r(p)p`$, with $`r(p)p^\delta `$ and $`\delta >0`$. For $`\delta =0`$, one recovers the standard exponential growth. For $`\delta >0`$, $`p(t)`$ can approach a singularity in finite time like $`p(t)|t_ct|^\alpha `$; its discrete time approximation takes the form of an exponential of an exponential of time \[Sornette, Takayasu & Zhou (2003)\]). The addition of an imaginary part to the exponent $`\alpha `$ such that it is written $`\alpha =\alpha ^{}+i\alpha ^{\prime \prime }`$ describes the existence of accelerating log-periodic oscillations obeying a discrete scale invariance (DSI) symmetry \[Sornette (1998), Sornette (2003a), Sornette (2003b)\]. This can be seen from the fact that $`|t_ct|^{\alpha ^{}+i\alpha ^{\prime \prime }}=|t_ct|^\alpha ^{}\mathrm{exp}[i\alpha ^{\prime \prime }\mathrm{ln}|t_ct|]`$ whose real part is $`|t_ct|^\alpha ^{}\mathrm{cos}[\alpha ^{\prime \prime }\mathrm{ln}|t_ct|]`$. More generally, it can be shown that the first terms in a systematic expansion can be written as

$$\mathrm{ln}[\pi (t)]=AB\tau ^m+C\tau ^m\mathrm{cos}\left[\omega \mathrm{ln}(\tau )\varphi \right],$$
(1)
where $`\pi (t)`$ is the price of a specific asset (it can be a specific stock, commodity or index), $`A`$ is the logarithm of the price at $`t_c`$, $`\omega `$ is the angular log-frequency, $`B>0`$ and $`0<m<1`$ for an acceleration to exist, $`C`$ quantifies the amplitude of the log-periodic oscillations, $`\varphi `$ is an arbitrary phase determining the unit of the time, $`t`$ is the current time, and $`\tau =t_ct`$ is the distance to the critical time $`t_c`$ defined as the end of the bubble. Equation (1) does not hold for any time but rather applies to those non-stationary transient phases, otherwise known as bubbles.

Expression (1) has been used to fit bubbles according to the following algorithm described in \[Johansen & Sornette (2001b)\], that we summarize here: (i) the crashes were identified within a given financial time series as the largest cumulative losses (largest drawdowns) according to the methodology of \[Johansen & Sornette (2001a)\]; (ii) the bubble preceding a given crash was then identified as the price trajectory starting at the most obvious well-defined minimum before a ramp-up ending at the highest value before the crash. The starting value (local minimum) of the bubble is unambiguous for large developed markets but it was sometimes necessary to change the interval fitted due to some ambiguity with respect to the choice of the first point in these smaller emergent markets \[Johansen & Sornette (2001a)\]. This algorithm was applied to over thirty crashes in the major financial markets and the fits of their corresponding bubbles with Eqn.(1) showed that the DSI parameter $`\omega `$ exhibits a remarkable universality, with a mean of $`6.4`$ and standard deviation of $`1.6`$ \[Johansen & Sornette (2003), Johansen (2003)\], suggesting that it is a significant distinguishing feature of bubble regimes. In these previous look back studies, the knowledge of the time of a given crash was used to identify its associated bubble with the purpose of testing for a common behavior. This was done in the usual spirit of a scientific endeavor which consists in general in the following steps: (a) collect samples; (b) test for regularities and commonalities in the samples to establish a robust classification of a new phenomenon; (c) develop a model accounting for this classification and calibrate it; (d) put the model to test using its prediction that are applied to novel data, that have not be used in the model construction. The previous works mentioned above \[Johansen & Sornette (2003), Johansen (2003)\] cover the steps (a-c). The present paper addresses step (d) by developing a systematic look forward approach in which the goal is to identify “in real time” the existence of a developing bubble and to predict times of increased crash probability.

The physics-type construction (i.e., emphasizing interactions and collective behavior) leading to Eqn.(1) is however missing an essential ingredient distinguishing the natural and physical from the social sciences, namely the existence of expectations: market participants are trying to discount a future that is itself shaped by market expectations. As exemplified by the famous parable of Keynes’ beauty contest, in order to predict the winner, recognizing objective beauty is not very important, but knowledge or prediction of others’ prediction of beauty is much more relevant. Similarly, mass psychology and investors’ expectations influence financial markets significantly. We use the rational-expectation (RE) Blanchard-Watson model of speculative bubbles \[Blanchard (1979), Blanchard & Watson (1982)\] to describe concisely such feedback loops of prices on agent expectation and vice-versa. Applied to a given bubble price trajectory, agents form an expectation of the sustainability of such bubble and of its potential burst. By this process, to each price $`\pi (t)`$ is associated a crash hazard rate $`h(t)`$, such that $`h(t)dt`$ is the probability for a crash or large change of regime of average amplitude $`\kappa `$ to occur between $`t`$ and $`t+dt`$ conditioned on the fact that it has not happened. The crash hazard rate $`h(t)`$ quantifies the general belief of agents in the non-sustainability of the bubble. The RE-bubble model predicts the remarkable relationship \[Johansen, Sornette & Ledoit (1999), Johansen, Ledoit & Sornette (2000), Sornette & Andersen (2002)\]

$$\mathrm{ln}\left[\frac{\pi (t)}{\pi (t_0)}\right]=\kappa _{t_0}^th(t^{})𝑑t^{}$$
(2)
linking price to crash hazard rate. This equation is the solution of the model describing the dynamics of the price increment $`d\pi `$ over a time increment dt: $`d\pi =\mu \pi dt\kappa \pi dj`$, where $`\mu `$ is the instantaneous return, $`dj`$ is a jump process equal to $`1`$ when a crash occurs with amplitude $`\kappa `$ and zero otherwise. The no-arbitrage condition $`\mathrm{E}[d\pi ]=0`$ implies that $`\mu \pi =\kappa \pi \mathrm{E}[dj/dt]`$, where $`\mathrm{E}[.]`$ denotes the expectation conditional on all information available up to the present time. By definition of the crash hazard rate, $`h(t)=\mathrm{E}[dj/dt]`$, this gives $`\mu \pi =\kappa \pi h(t)`$. Thus, conditioned on no crash having occurred until time $`t`$, the integration of the model yields (2).

Substituting Eqn.(1) in Eqn.(2) and using the fact, that by definition $`h(t)0`$, gives the constraint

$$bBm|C|\sqrt{m^2+\omega ^2}0.$$
(3)
This condition has been found to provide a significant feature for distinguishing market phases with bubbles ending with crashes from regimes without bubbles \[Bothmer & Meister (2003)\].

Note that $`h(t)`$ is not normalized to $`1`$: $`_0^+\mathrm{}h(t)𝑑t`$ gives the total conditional probability for a crash to occur, which is less than $`1`$ for the RE bubble model to hold. There is thus a finite probability $`1_0^+\mathrm{}h(t)𝑑t`$ for the bubble to land smoothly without a crash. This is necessary for the RE bubble model, as otherwise the crash would be certain. By having a non-zero probability for no crash to occur, it remains rational for investors to remain in the market and harvest the risk premium (the bubble) for the risk of a possible (but not certain) crash. Technically, using the standard rule of conditional probabilities, $`h(t)`$ is related to the unconditional probability $`p(t)`$ for a crash per unit time by $`h(t)=p(t)/_t^+\mathrm{}p(t^{})𝑑t^{}`$. By integrating, rearranging and differentiating, this gives $`p(t)=h(t)\mathrm{exp}[_0^th(t^{})𝑑t^{}]`$.

## 3 The pattern recognition method

### 3.1 Objects and classes

Let us now build on these insights and construct a systematic predictor of bubble regimes and their associated large changes. Due to the complexity of financial time series, to the existence of many different regimes, and to the constant action of investors arbitraging gain opportunities, it is widely held that the prediction of crashes is an impossible task 222A. Greenspan, Economic Volatility, Remarks at symposium sponsored by the Federal Reserve Bank of Kansas City, Jackson Hole, Wyoming, August 30, 2002.. The truth is that financial markets do not behave like statistically stationary physical systems and exhibit a variety of non-stationarities, influenced both by exogenous and endogenous shocks \[Sornette, Malevergne & Muzy (2003)\]. To obtain robustness and to extract information systematically in the approximately self-similar price series, it is useful to use a multiscale analysis: the calibrations of Eqn.(1) and Eqn.(2) are thus performed for each trading day over nine different time scales of $`60`$, $`120`$, $`240`$, $`480`$, $`720`$, $`960`$, $`1200`$, $`1440`$ and $`1680`$ trading days.

Each trading day ending an interval of length $`1680`$ or smaller is called an object and there are a total of 27428 objects from 06-Feb-1902 till 11-Mar-2003. We define a map from the calendar time $`t`$ to an ordered index $`\$t`$ of the trading day, such that $`\$t=t`$ if $`t`$ is a trading date and $`\$t=\mathrm{}`$ (empty set) otherwise. This mapping is just a convenient way for defining the set of objects to which our method is applied. It does not produce any bias or additional dynamics since it is used only in the classification scheme. We denote $`𝒯_𝒞=\{t_{c,i}:i=1,\mathrm{},n\}`$ a list of targets (crashes) that happened at $`t_{c,i}`$. The total number of targets is thus $`n`$. All objects are partitioned into two classes $`\mathrm{I}`$ and $`\mathrm{II}`$, where class $`\mathrm{I}(t_l)=\{\$t:t_{c,i}t_ltt_{c,i},t_{c,i}𝒯_𝒞\}`$ contains all the objects in $`[t_{c,i}t_l,t_{c,i}]`$ for $`i=1,\mathrm{},n`$ and class $`II(t_l)=\{T:T=1,\mathrm{},27428\}\mathrm{I}`$ includes all the remaining, where $`t_l`$ is a given number in unit of calender day. In the following, our target set are three well-known speculative bubbles that culminated on 14-Sep-1929, 10-Oct-1987 and 18-Jul-1998, respectively, before crashing in the following week or month. The first two bubbles are perhaps the most famous or infamous examples discussed in the literature. The last one is more recent and has been associated with the crisis of the Russian default on its debt and LTCM. Of course, the choice of this triplet is otherwise arbitrary. Our goal is just to show that our method can generalize to other bubbles beyond those which have been used for training.

### 3.2 Empirical tests of the relevance of patterns

Figure 1 presents the distributions of the $`\omega `$’s (defined in Eqn.(1)) and $`b`$’s (defined in Eqn.(3)), obtained from fits with Eqn.(1) in intervals ending on objects for all time scales and for three different values of $`t_l`$. The distributions for classes $`\mathrm{I}`$ and $`\mathrm{II}`$, which are very robust with respect to changes in $`t_l`$, are very different: for $`\omega [6,13]`$ (which corresponds to the values obtained in previous works), $`p(\omega |\mathrm{I})`$ is larger than $`p(\omega |\mathrm{II})`$; $`40\%`$ of objects in class $`\mathrm{I}`$ obey the constraint $`b0`$ compared to $`6\%`$ in class $`\mathrm{II}`$. A standard Kolmogorov-Smirnov test gives a probability that this difference results from noise much less than $`10^3`$. The very significant differences between the distributions in classes $`\mathrm{I}`$ and $`\mathrm{II}`$ confirm that the DSI and constraint parameters are distinguishing patterns of the three bubbles.

Figure 2 plots a vertical line segment for each time scale at the date $`t`$ of the object that fulfills simultaneously three criteria: $`6\omega 13`$, $`b0`$ and $`0.1m0.9`$. The last criterion ensures a smooth super-exponential acceleration. Each stripe thus indicates a potentially dangerous time for the end of a bubble and for the coming of a crash. The number of dangerous objects decreases with the increase of the scale of analysis, forming a hierarchical structure. Notice the clustering of alarms, especially at large scales, around the times of the three targets. Altogether, Figs. 1 and 2 suggest that there is valuable information for crash prediction but the challenge remains to decipher the relevant patterns in the changing financial environment. For this, we call in the statistical analysis of sparse data developed by the Russian mathematical school \[Gelfand, et al. (1976), Keilis-Borok & Soloviev (2003)\] in order to construct a systematic classifier of bubbles. The modified “CORA-3” algorithm \[Gelfand, et al. (1976)\] that we briefly describe below allows us to construct an alarm index that can then be tested out-of-sample.

Each object is characterized by five quantities, which are believed to carry relevant information: $`\omega `$, $`b`$, $`m`$, the r.m.s. of the residuals of the fits with Eqn.(1) and the height of the power spectrum $`P_N`$ of log-periodicity at $`\omega `$ \[Press, Teukolsky, Vetterling & Flannery (1996), Zhou & Sornette (2002a)\]. The residuals exhibit much less structure that the initial time series but some dependence remains as the fits with Eqn.(1) account only for the large time-scale structure. Indeed, using the r.m.s. of the residuals in the discrimination of the objects is useful only if some dependence remains in them. In other words, the residuals contain all the information at the smaller time scales and it is hoped that these sub-structures may be linked to the log-periodic structure at the largest time scale. This has actually been shown to be the case by \[Drozdz, Ruf, Speth & Wojcik (1999)\] in a specific case and by the construction of trading methods at short time scales \[Johansen and Sornette, unpublished\]. This also follows from the full solution of the underlying so-called renormalization group equation embodying the critical nature of a bubble and crash \[Zhou & Sornette (2003b)\]. The use of the power spectrum $`P_N`$ follows the same logic.

Integrated with the 9 scales, there are totally $`5\times 9`$ parameters for each object. One could worry that using so many parameters is meaningless in the deterministic framework of Eqn.(1) because the large number of degrees of freedom and their associated uncertainties lead to an effective randomness, but this view does not recognize that this approach amounts to a decomposition in multiple scales: (a) it is allowed for or recognized that the price trajectory has patterns at many scales; (ii) for each pre-selected scale, a rather parsimonious deterministic fit is obtained. The selection of the relevant parameters discussed below amounts actually to searching for those time scales that are the most relevant, since the data should decide and not some a priori preconception.

We set $`t_l=200`$ but have checked that all results are robust with large changes of $`t_l`$. Class I has 444 objects. By proceeding as in Fig. 1, for each of the 45 parameters, we qualify it as relevant if its distributions in classes I and II are statistically different. We have played with several ways of quantifying this difference and find again a strong robustness. Out of the 45 parameters, 31 are found to be informative and we have determined their respective intervals over which the two distributions are significantly different. We use these 31 parameters to construct a questionnaire of 31 questions asked to each object. The $`i`$th question on a given object asks if the value of the $`i`$th parameter is within its qualifying interval. Each object $`A`$ can then be coded as a sequence of binary decisions so that $`A=A_1A_2\mathrm{}A_{31}`$ where $`A_i`$ is the answer (Y/N) to the $`i`$th question.

### 3.3 Construction of the alarm index

A distinctive property of the pattern recognition approach is to strive for robustness notwithstanding the lack of sufficient training data (here only three bubbles) \[Gelfand, et al. (1976)\]. In this spirit, we define a trait as an array $`(p,q,r,P,Q,R)`$ where $`p=1,2,\mathrm{},31`$, $`q=p,p+1,\mathrm{},31`$, $`r=q,q+1,\mathrm{},31`$ and $`P,Q,R=`$ Y or N. There are $`2\left(\genfrac{}{}{0pt}{}{31}{1}\right)+4\left(\genfrac{}{}{0pt}{}{31}{2}\right)+8\left(\genfrac{}{}{0pt}{}{31}{3}\right)=37882`$ possible traits. If $`P=A_p`$, $`Q=A_q`$ and $`R=A_r`$, we say that the object $`A`$ exhibits a trait $`(p,q,r,P,Q,R)`$. The number of questions (three are used here) defining a trait is a compromise between significance and robustness. A feature is a trait that is present relatively frequently in class I and relatively infrequently in class II. Specifically, if there are no less than $`k_\mathrm{I}`$ objects in class $`\mathrm{I}`$ and no more than $`k_{\mathrm{II}}`$ objects in class $`\mathrm{II}`$ that exhibit a same trait, then this trait is said to be a feature of class $`\mathrm{I}`$. Calling $`n(\$t)`$ the number of distinctive features of a given object at time $`t`$, we define the alarm index $`AI(t)`$, here amounting to perform a moving average of $`n(\$t)`$ over a time window of width $`\mathrm{\Delta }`$. We then normalize it: $`AI(t)AI(t)/\mathrm{max}_t\{AI(t)\}`$.

### 3.4 Synoptic view of the approach

Let us summarize our construction leading to the Alarm Index.

* We select a few targets (the three well-known speculative bubbles ending in a crash at Oct. 1929, Oct. 1987 and Aug. 1998 for the US index and two targets for the Hang Seng), which serve to train our system (independently for the two indices).
* An object is defined simply as a trading day ending a block of trading days of a pre-defined duration.
* Those objects which are in a neighborhood of the crashes of the targets are defined to belong to class I. All other objects are said to belong to class II.
* For each object, we fit the price trajectory with expression (1) over a given time scale (defining the duration of the block of days ending with the object used in the fit) and obtain the corresponding values of the parameters, which are considered as characteristic of the object. This step is repeated nine times, once for each of the nine time scales used in the analysis. We keep $`5`$ parameters of the fit by expression (1) for each time scale, thus giving a total of $`5\times 9`$ parameters characterizing each object.
* We construct the probability density functions (pdf) of each parameter over all objects of class I and of class II separately. Those parameters which are found to exhibit sufficiently distinct pdf’s over the two classes are kept as being sufficiently discriminating. In this way, out of the total of $`45`$ parameters, $`31`$ are kept.
* Each selected parameter gives a binary information, Y or N, as whether its value for a given object falls within a qualifying interval for class I.
* As a compromise between robustness and an exhausive description, we group the parameters in triplets (called traits) to obtain a characterization of objects. Ideally, one would like to use all $`31`$ parameters simultaneously for each object, but the curse of dimensionality prevents doing this.
* We study the statistics of all traits and look for those which are frequent in objects of class I and unfrequent in objects of class II. Such traits are called features.
* The Alarm Index at a given time is defined as a moving average number of distinctive features found at that time. Large values of the Alarm Index are proposed to be predictors of changes of regime in the stock market.

### 3.5 Tests on the DJIA

Figure 3 shows the alarm index $`AI(t)`$ for $`\mathrm{\Delta }=100`$ with $`k_\mathrm{I}=200`$ (corresponding to at least $`45.1\%`$ of the objects in class I) and $`k_{\mathrm{II}}=1500`$ (corresponding to no more than $`5.6\%`$ of the objects in class II). The alarm index is found to be highly discriminative as only 13.0% (respectively 9.5%, and 4.7%) of the total time is occupied by a high level of alarm larger than $`0.2`$ (respectively $`0.3`$, and $`0.4`$). We have performed extensive experiments to test the robustness of the results. We have varied simultaneously $`k_\mathrm{I}`$ from 100 to 400 with spacing 50 and $`k_{\mathrm{II}}`$ from $`1000`$ to $`4000`$ with spacing $`500`$. The results are remarkably robust with no change of the major peaks.

The three bubbles used in the learning process are predicted (this is expected of course): peak 3 is on 01-Jul-1929 (market maximum on 14-Sep-1929); peak 9 is on 10-Oct-1987 (market maximum on 04-Sep-1987 and crash on 19-Oct-1987); peak 13 is on 23-Apr-1998 (market maximum on 18-Jul-1998). This timing is a good compromise for a prudent investor to exit or short the market without losing much momentum. This success is however not a big surprise because this corresponds to in-sample training. The most remarkable property of our algorithm is its ability for generalizing from the three bubbles ending with crashes to the detection of significant changes of regimes that may take a large variety of shapes (for instance, a crash is not a one-day drop of, say, +15% but can be spread over several days or a few weeks with rather complex trajectories, leading to very large cumulative losses).

The following peaks are predictors of crashes or of strong corrections with an $`AI=0.3`$ alarm threshold: peak 1 (Jan, 1907), peak 4 (Apr, 1937), peak 5 (Aug, 1946), peak 6 (Jan, 1952), peak 7 (Sep, 1956), peak 8 (Jan, 1966), peak 12 (Jul, 1997) and peak 14 (Sep, 1999). The more than $`20\%`$ price drops in 1907 (peak 1) and 1937 (peak 4) have been previously identified as crashes occurring over less than three months \[Mishkin & White (2002)\]. Peak 12 is occurring three months before the turmoil on the stock market (one day $`7\%`$ drop on 27-Oct-1997) that was followed by a three month plateau (see discussion in Chapter 9 of Ref. \[Sornette (2003a), Sornette (2003b)\]). Peak 10 is a false alarm. Peaks 2 and 11 can be regarded as early alarms of the following 1929 crash and 1997, 1998, and 2000 descents. Peak 15-17 are smaller peaks just barely above $`AI=0.2`$. Peaks 15 and 16 are false alarms, while peak 17 (31-May-2000) is just two months before the onset of the global anti-bubble starting in August of 2000 \[Sornette & Zhou (2002), Zhou & Sornette (2003a), Zhou & Sornette (2003b)\].

Figure 4 is constructed as follows. Each alarm index peak is characterized by the time $`t_c`$ of its maximum and by its corresponding log of price $`\mathrm{ln}[\pi (t_c)]`$. We stack the index time series by synchronizing all the $`t_c`$ for each peak at the origin of time and by translating vertically the prices so that they coincide at that time. Figure 4 shows four years of the DJIA before and two years after the time of each alarm index maximum. We choose this representation to avoid the delicate and still only partly resolved task of defining what is a crash, a large drop or a significant change of regime. On this problem, the literature is still vague and not systematic. Previous systematic classifications \[Johansen & Sornette (1998), Johansen & Sornette (2001a), Mishkin & White (2002)\] miss events that are obvious to the trained eyes of professional investors or on the contrary may incorporate events that are controversial.

Figure 4 shows that these peaks all correspond to a time shortly before sharp drops or strong change of market regimes. In a real-time situation, a similar prediction quality can be obtained in identifying the peaks by waiting a few weeks for the alarm index to drop or alternatively by declaring an alarm when $`AI(t)`$ reaches above a level in the range $`0.20.4`$. The sharpness and the large amplitudes of the peaks of the alarm index ensures a strong robustness of this latter approach. Our method however misses several large price drops, such as those in 1920, 1962, 1969, 1974 \[Mishkin & White (2002), Sornette (2003a), Sornette (2003b)\]. This may be attributed to a combination of the fact that some large market drops are not due to the collective self-organization of agents but result from strong exogenous shocks, as argued in Refs. \[Sornette, Malevergne & Muzy (2003), Johansen & Sornette (2003), Johansen (2003)\] and that the method needs to be improved. However, the generalizing ability of our learning algorithm strengthens the hypothesis that herding behavior leads to speculative bubbles and sharp changes of regimes.

### 3.6 Error diagrams for evaluation of prediction performance

#### 3.6.1 Theoretical background

This brief presentation is borrowed from Chap. 9 of the book in \[Sornette (2003a), Sornette (2003b)\]. In evaluating predictions and their impact on (investment) decisions, one must weight the relative cost of false alarms with respect to the gain resulting from correct predictions. The Neyman-Pearson diagram, also called the decision quality diagram, is used in optimizing decision strategies with a single test statistic. The assumption is that samples of events or probability density functions are available both for correct signals (the crashes) and for the background noise (false alarms); a suitable test statistic is then sought which optimally distinguishes between the two. Using a given test statistic (or discriminant function), one can introduce a cut which separates an acceptance region (dominated by correct predictions) from a rejection region (dominated by false alarms). The Neyman-Pearson diagram plots contamination (misclassified events, i.e., classified as predictions which are thus false alarms) against losses (misclassified signal events, i.e., classified as background or failure-to-predict), both as fractions of the total sample. An ideal test statistic corresponds to a diagram where the “Acceptance of prediction” is plotted as a function of the “acceptance of false alarm” in which the acceptance is close to $`1`$ for the real signals, and close to $`0`$ for the false alarms. Different strategies are possible: a “liberal” strategy favors minimal loss (i.e., high acceptance of signal, i.e., almost no failure to catch the real events but many false alarms), a “conservative” one favors minimal contamination (i.e., high purity of signal and almost no false alarms but many possible misses of true events).

\[Molchan (1990), Molchan (1997)\] has shown that the task of predicting an event in continuous time can be mapped onto the Neyman-Pearson procedure. He has introduced the “error diagram” which plots the rate of failure-to-predict (the number of missed events divided by the total number of events in the total time interval) as a function of the rate of time alarms (the total time of alarms divided by the total time, in other words the fraction of time we declare that a crash or large correction is looming) (see also \[Keilis-Borok & Soloviev (2003)\] for extensions and reviews). The best predictor corresponds to a point close to the origin in this diagram, with almost no failure-to-predict and with a small fraction of time declared as dangerous: in other words, this ideal strategy misses no event and does not declare false alarms! These considerations teach us that making a prediction is one thing, using it is another which corresponds to solving a control optimization problem \[Molchan (1990), Molchan (1997)\].

#### 3.6.2 Assessment of the quality of predictions by the error diagram

To assess quantitatively the prediction procedure described above, we thus construct an error diagram, plotting the failure to predict as a function of the total alarm duration. The targets to be predicted are defined as large drawdowns (cumulative losses as defined in \[Johansen & Sornette (1998), Johansen & Sornette (2001a)\]) whose absolute values are greater than some given value $`r_0`$. Each drawdown has a duration covering a time interval $`[t_1,t_2]`$. For a given alarm index threshold $`AI_0`$, if there exist a time $`t[t_1\mathrm{\Delta }t,t_1)`$ so that $`AI(t)AI_0`$, then the drawdown is said to be successfully predicted; in contrast, if $`AI(t)<AI_0`$ for all $`t[t_1\mathrm{\Delta }t,t_1)`$, we say it is a failure. This definition ensures that the forecast occurs before the occurrence of the drawdown. The error diagram is obtained by varying the decision threshold $`AI_0`$. The quantity “failure to predict” is the ratio of the number of failures over the number of targets as said above. The total alarm duration is the total time covered by the objects whose alarm index is larger than $`AI_0`$ divided by the total time. By varying the value of $`AI_0`$, we obtain different pairs of failure to predict and total alarm duration.

Figure 5 presents the error diagram for two target definitions ($`r_0=0.1`$ and $`r_0=0.15`$) for the DJIA with $`\mathrm{\Delta }t=40`$, $`k_\mathrm{I}=250`$ and $`k_{\mathrm{II}}=3500`$. The results do not change significantly as long as $`k_\mathrm{I}`$ is much smaller than $`k_{\mathrm{II}}`$. The anti-diagonal corresponds to the completely random prediction scheme. The error diagrams with different $`\mathrm{\Delta }t`$, $`k_\mathrm{I}`$ and $`k_{\mathrm{II}}`$ are found to be similar, with small variations, showing a strong robustness of our construction. The inset shows the prediction gain, defined as the ratio of the fraction of targets correctly predicted (equal to one minus the fraction of missed targets) to the total alarm duration. Theoretically, the prediction gain of a random prediction strategy is $`1`$. The error diagram shows that half of the targets with $`r_0=0.15`$ are predicted with a very small alarm duration and all targets are predicted for an alarm duration less than 40%. This gives rise to large prediction gains for small alarm durations, confirming the very good quality of our predictors. Confidence in this conclusion is enhanced by finding that the quality of the prediction increases as the targets are evolved toward largest losses ($`10\%`$ loss for $`r_0=0.1`$ to $`15\%`$ loss for $`r_0=0.15`$). This trend continues for largest losses but there are too few events to draw statistically meaningful conclusions for larger drawdowns of $`20\%`$ or more.

## 4 Tests on the Hong Kong Hang Seng composite index (HSI)

The validity of our construction should be ascertained by testing it without modification on other independent time series. Here, we present similar promising results obtained for the Hong Kong Hang Seng composite index (HSI) from 24-Nov-1969 to present. The Hong Kong Hang Seng composite index is particularly interesting as it can be considered as a “textbook” example of an unending succession of bubbles and crashes \[Sornette & Johansen (2001)\]. The nine biggest crashes since 24-Nov-1969 were triggered approximately at the following dates: 20-Sep-1971, 5-Mar-1973, 04-Sep-1978, 13-Nov-1980, 01-Oct-1987, 15-May-1989, 4-Jan-1994, 8-Aug-1997, and 28-Mar-2000. Except for the last one which was posterior to the study published in \[Sornette & Johansen (2001)\], all of them have been studied previously in \[Sornette & Johansen (2001)\]. The distinctive properties of the first eight bubbles are consistent with those reported previously \[Sornette & Johansen (2001)\] (see page 461 concerning the parameters $`\omega `$ and $`m`$). In contrast with Ref. \[Sornette & Johansen (2001)\] in which the positivity constraint was not considered, we find here that it plays a significant role in screening out solutions. We use the two crashes in 1987 and 1997 to train the parameters of our algorithm because they give the best defined log-periodic power law patterns over the longest time intervals.

Figure 6 illustrates the typical result for the Alarm Index obtained with $`t_l=200`$, $`k_\mathrm{I}=210`$ and $`k_{\mathrm{II}}=1700`$. Four out of the five crashes that were not used in the training phase are identified very clearly by the alarm index peaks (note that the first two crashes in 20-Sep-1971 and 5-Mar-1973 are omitted in our counting since they occur before the time when our Alarm Index can be constructed). This encouraging result seems to confirm the hypothesis that the seven crashes, posterior to 1976 where the Alarm Index can be defined, are triggered by endogenous stock market instabilities preceded by log-periodic power-law speculative bubbles \[Johansen & Sornette (2003), Johansen (2003)\]. Varying the values of the parameters $`t_l`$, $`k_\mathrm{I}`$ and $`k_{\mathrm{II}}`$ of our algorithm gives results which are robust.

In addition, one can identify more peaks in Fig. 6 and it is an interesting question to test whether they are associated with an anomaly in the market. In total, we count 15 peaks in Fig. 6 around 15-Sep-1978 (S), 09-Apr-1980 (?), 29-Sep-1981 (F), 30-Nov-1982 (F), 01-Oct-1986 (?), 30-Sep-1987 (S), 20-May-1989 (S), 18-Aug-1990 (F), 09-Oct-1991 (F), 19-Oct-1992 (F), 28-Oct-1993 (S), 05-May-1996 (?), 06-Aug-1997 (S), 10-Apr-1999 (F), and 25-Mar-2000 (S), whose alarm indexes are greater than $`0.1`$. Six peaks whose dates are identified by “S” correspond to successful predictions, among which two are trivial since they were used in the training process. Six peaks identified with “F” are false alarms. The remaining three peaks marked with “?” are neither successful predictions nor complete false alarms as they fall close to crashes. The date with an alarm index of 0.1 on the east side of the first “?” alarm peak is 31-Jul-1980 and can probably be interpreted as a forerunner of the crash on 13-Nov-1980. The two other “?” alarm peaks can probably interpreted as “fore-alarms” of two main alarm peaks used for training the algorithm.

Similar to Fig. 4, Fig. 7 shows a superposed epoch analysis of the price trajectories a few years before and one year after these 15 peaks, to see whether our algorithm is able to generalize beyond the detection of crashes to detect changes of regimes. The ability to generalize is less obvious for the Hang-Seng index that it was for the DJIA shown in Fig. 4.

Figure 8 presents the error diagram for three target definitions ($`r_0=0.1`$, $`r_0=0.15`$, and $`r_0=0.2`$) for the HSI with $`\mathrm{\Delta }t=40`$, $`k_\mathrm{I}=210`$ and $`k_{\mathrm{II}}=1700`$. The anti-diagonal corresponds to the random prediction scheme. Again, the error diagrams with different $`\mathrm{\Delta }t`$, $`k_\mathrm{I}`$ and $`k_{\mathrm{II}}`$ are robust. The inset plots the corresponding prediction gains. More than half of the targets with $`r_0=0.20`$ are predicted with a very small alarm time and all targets are predicted for a finite alarm time less than 50%. These results are associated with a very significant predictability and strong prediction gains for small alarm durations.

## 5 Concluding remarks

We find that different stock markets have slightly different characteristics: for instance, the parameters trained on the DJIA are not optimal for the Hang-Seng index. When we use the parameters obtained from the training of our algorithm on the DJIA on the HSI, we find that two of the seven crashes in the HSI are identified accurately (01-Oct-1987 and 4-Jan-1994) while the five other crashes are missed. Interestingly, these two crashes are the most significant. This suggests the existence of a universal behavior333 The term “universality” is used here in the technical sense developed in thermodynamics and statistical physics. Its roots go back to the 1960s when pioneers, such as B. Widom, L. Kadanoff, M.E. Fisher, K. Wilson and many others explored and established the theory of critical phenomena in natural sciences. This theory was fully developed in the 1970s to describe the peculiar change of organization that may occur in fluids or magnets and many other condensed matter systems. In any system in nature, there are at least two tendencies that oppose each other: interactions between constituents favor order while “noise” or thermal fluctuations promote disorder. The referee of this fight between order and disorder is called a “control parameter”: by varying it, the fluid or magnet may undergo a transition from an ordered to a disordered state. The transition may be “critical” in the technical sense that fluctuations of both competing states occur at all space and time scale (bounded of course by the size of the system) and become intimately intertwinned. This leads to specific signatures in the form of power law dependences of physical observables (such as density difference or magnetization, correlation length, susceptibility) as a function of the distance of the control parameter to its critical value. The concept of universality enters in this picture from the remarkable empirical discovery later understood within the framework of the renormalization group theory that the critical exponents of these power laws characterizing a critical point are universal: they are the same for a magnet or a fluid within the same “universality class” defined only by very general properties of the system (such as the dimension of the embedding space, the dimension of the order paremeter and symmetries). The exponents are otherwise completely independent of the nature of the system, whether it is constituted of atoms, molecules or magnetic spins. In other words, the properties of a critical point are independent of many of the details of a system. A major challence in the theory of complex systems is to evaluate to what degree this concept of universality remains valid when extended to more complex out-of-equilibrium dynamical systems. only for the “purest” event cases, while smaller events exhibit more idiosyncratic behavior. As we have shown, training our algorithm on these two events on the HSI improves very significantly the prediction of the other events. This shows both a degree of universality of our procedure and the need to adapt to the idiosyncratic structure of each market. This situation is similar to that documented in earthquake predictions based on pattern recognition techniques \[Keilis-Borok & Soloviev (2003)\].

How does the performance of our system compare with those of commonly used stochastic models of stock indices, such as GARCH? To answer this question, we have used a standard method to calibrate a GARCH(1,1) model to the S&P500. Then, the GARCH model was run to provide a prediction of the volatility for the next period (here taken in unit of days), but not of the sign of the next return (which is unpredictable according to the GARCH model). We found a quite strong predictability of the volatility “predicted” by GARCH by preceding large drops (and often large ups) of the market but not the reverse. In other words, GARCH is not predicting in any way future large drops, it is the other way around: large realized drops predict future large GARCH-predicted values of the volatility. Actually, this is not surprising for two reasons: (i) from the structure of the GARCH model, a previous large daily drop (or gain) immediately increases the present amplitude of the volatility; (ii) the fact that losses predict future increases of volatility (and not the reverse) has a rich literature associated with the “leverage” effect (see for instance \[Figlewski and Wang (2000), Chan, Cheng & Lung (2003)\] and references therein). Thus, we can conclude that the GARCH model can in no way reproduce or even approach the detection of impending changes of regime as we have documented here.

In summary, we have developed a multiscale analysis on the analysis of stock market bubbles and crashes. Based on a theory of investor imitation, we have integrated the log-periodic power-law patterns characteristic of speculative bubbles preceding financial crashes within a general pattern recognition approach. We have applied our approach to two financial time series, DJIA (Dow Jones Industrial Average) and HSI (Hang-Seng Hong Kong index). Training our algorithm on only a few crashes in each market, we have been able to predict the otherwise crashes not used in the training set. Our work provides new evidence of the relevance of log-periodic power-law patterns in the evolution of speculative bubbles supporting the imitation theory.

We are grateful to T. Gilbert for helpful suggestions. This work was partially supported by the James S. Mc Donnell Foundation 21st century scientist award/studying complex system.