text:
# A differential analog of a theorem of Chevalley

## Abstract

In this note a proof of a differential analog of Chevalley’s theorem \[C\] on homomorphism extensions is given. An immediate corollary is a condition of finitenes of extensions of differential algebras and several equivalent definitions of a differentially closed field, including Kolchin’s Nullstellensatz.

In this note I give a proof of the following differential analog of Chevalley’s theorem \[C\] on homomorphism extensions.

###### Theorem 1.

Let $`S`$ be a differential algebra over $``$ with no zero divisors and let $`b`$ be a non-zero element of $`S`$. Let $`R`$ be a differential subalgebra of $`S`$ over which $`S`$ is differentially finitely generated. Let $``$ be a differentially closed field of characteristic $`0`$. Then there exists a non-zero element $`a`$ of $`R`$ such that any homomorphism $`\phi :R`$ which does not annihilate $`a`$ extends to a homomorphism $`\psi :S`$ which does not annihilate $`b`$.

An almost immediate consequence of the proof of Theorem 1 is

###### Theorem 2.

Let $``$ be a differentially closed field of characteristic $`0`$ and let $`SR`$ be differentially finitely generated differential algebra and subalgebra over $``$. Suppose that there exists a non-zero element $`b`$ of $`S`$ such that any homomorphism $`\phi :R`$ has only finitely many extensions $`\psi :S`$ satisfying $`\psi (b)0`$. Then the field extension $`FractSFractR`$ is finite. In particular, if any homomorphism $`\phi :R`$ has at most $`d`$ extensions $`\psi :S`$ with $`\psi (b)0`$, then the degree of $`FractS`$ over $`FractR`$ is at most $`d`$.

An immediate corollary of Theorem 1 is Kolchin’s Nullstellensatz \[K\] and its earlier weaker versions by Ritt \[R\], Cohn \[Cohn\] and Seidenberg \[S\].

As far as I can understand it, Theorem 1 is closely related to Blum’s elimination of quantifiers theorem \[Blum\], \[M\] in the model theory of differentially closed fields.

In Section 1 I explain the necessary background on Differential Algebra, in Sections 2 and 3 give proofs of Theorems 1 and 2 and in Section 4 give about a dozen of equivalent definitions of a differentially closed field of characteristic $`0`$ (Theorem 1 and Kolchin’s Nullstellensatz being among them).

This note is an offshoot of a course in Differential Algebra that I gave at M.I.T. in the fall of 2000. The motivation for teaching this course came from close connections of the theory of conformal algebras to differential algebras \[BDK\]. Namely, each Lie conformal algebra defines a functor from the category of differential algebras to the category of Lie algebras (very much like a group scheme defines a functor from the category of algebras to the category of groups). I am grateful to my students for their enthusiasm, especially to A. De Sole for several corrections and improvements. I would like to thank A. Buium for very useful and enlightening correspondence and J. Young for bringing Marker’s paper \[M\] to my attention and explaining some parts of it.

1.  Here I recall some terminology and facts from Differential Algebra. All of this can be easily found in two excellent books \[K\] and \[B\], the primary source of both being Ritt’s foundational book \[R\].

By an algebra we always mean a commutative associative unital algebra. A *differential algebra* $`R`$ is an algebra over a field with a fixed derivation $`\delta `$. By a homomorphism of differential algebras we always mean a homomorphism commuting with derivations. Likewise subalgebras and ideals are assumed to be $`\delta `$-invariant, though to emphasize this we shall often call them differential subalgebras and ideals.

The first important observation of Ritt is that in a differential algebra over $``$ the radical of a differential ideal is a differential ideal (see \[R\], \[Ra\], \[K\] Lemma 1.8). (A counterexample in characteristic $`p`$ is the zero ideal in $`R=[x]/(x^p)`$, $`\delta =d/dx`$.) Another important fact is the differential Krull theorem: any differential radical ideal is an intersection of differential prime ideals (see \[Ra\], \[K\] Theorem 2.1). These facts imply, in particular, that maximal (among) differential ideals are prime.

If a differential algebra $`R`$ has no zero divisors, we can form the field of fractions $`FractR`$ and extend the derivation $`\delta `$ to this field.

One says that a differential algebra $`S`$ is differentially *generated* by elements $`x_1,\mathrm{},x_n`$ over a differential subalgebra $`R`$, and writes $`S=R\{x_1,\mathrm{},x_n\}`$, if the algebra $`S`$ is generated by all elements from $`R`$ and all derivatives $`x_i^{(k)}`$, $`i=1,\mathrm{},n`$, $`k_+=\{0,1,2,\mathrm{}\}`$, of the elements $`x_i`$.

Let $`S=R\{x\}`$ be a differential algebra with no zero divisors. One says that $`x`$ is *differentially transcendental* over $`R`$ if all elements $`x^{(k)}`$, $`k_+`$, are algebraically independent over $`FractR`$; otherwise $`x`$ is called *differentially algebraic* over $`R`$.

The algebra of *differential polynomials* over a differential algebra $`R`$ in the differential indeterminates $`y_1,\mathrm{},y_n`$ is the algebra of polynomials $`R[y_i^{(k)},i=1,\mathrm{},n,k_+]`$ with the derivation $`\delta `$ extended from $`R`$ by the rule $`\delta (y_i^{(k)})=y^{(k+1)}`$. This algebra is denoted by $`R\{y_1,\mathrm{},y_n\}`$.

Consider the differential algebra $`R\{y\}`$ of differential polynomials over $`R`$ in one differential indeterminate $`y`$. Let $`A(y)R\{y\}\backslash R`$, be a “non-constant” differential polynomial. The largest $`r`$ for which $`y^{(r)}`$ is present in $`A(y)`$ is called the *order* of $`A(y)`$ and is denoted by $`ordA`$. One can write in a unique way:

$$A(y)=I_A(y)y^{(r)d}+I_1(y)y^{(r)d1}+\mathrm{}+I_d(y),$$

where $`ordI_A(y)<r`$, $`ordI_j(y)<r`$ and $`I_A(y)0`$. Then $`d`$ is called the *degree* of $`A(y)`$ and is denoted by $`\mathrm{deg}A(y)`$, and $`I_A(y)`$ is called the *initial* of $`A(y)`$. The differential polynomial $`S_A(y)=A(y)/y^{(r)}`$ is called the *separant* of $`A(y)`$. The important property of the characteristic 0 case is that $`S_A(y)0`$ if $`AR`$.

For $`A,BR\{y\}\backslash R`$ we write $`A<B`$ if either $`ordA<ordB`$, or $`ordA=ordB`$ and $`\mathrm{deg}A<\mathrm{deg}B`$. We also write $`A<B`$ if $`AR`$, $`BR`$. Note that $`I_A<A`$ and $`S_A<A`$.

The basic result of Ritt’s theory \[R\] (see \[K\] Lemma 7.3 or \[B\] (2.3)) is the following division algorithm:

Given a non-constant differential polynomial $`A(y)`$, for any $`F(y)R\{y\}`$ there exists$`G(y)R\{y\}`$ with $`G<A`$ and $`m,n_+`$ such that:

$$I_A(y)^mS_A(y)^nF(y)G(y)mod[A(y)],$$

where $`[A(y)]`$ is the ideal of $`R\{y\}`$ generated by $`A(y)^{(k)}`$, $`k_+`$. Furthermore, one can take $`m=0`$ if only $`ordGordA`$ is required, which is called the weak division algorithm.

The important notion of a differentially closed field was introduced by Robinson \[Rob\]. His axioms have been considerably simplified by Blum \[Blum\], and it is her definition, given below, that is commonly used. A differential field $``$ is called *differentially closed* if for any two differential polynomials $`A(y),B(y)\{y\}`$ with $`B0`$ and $`ordB<ordA`$ there exists $`\alpha `$ such that $`A(\alpha )=0`$ and $`B(\alpha )0`$.

Any differentially closed field is algebraically closed (i.e., has no non-trivial finite extensions), but it always has differentially algebraic extensions. Nevertheless, it turned out to be the right substitute for Differential Algebra of the notion of an algebraically closed field.

The existence of a differentially closed field containing a given differential field of characteristic 0 is easy to establish in the framework of Model Theory (see e.g. \[M\]). An elementary proof (i.e., without reference to model theory) may be found in \[B\] (5.2).

2. *Proof of Theorem 1.*  The general scheme of the proof is the same as Chevalley’s \[C\]. We have:

(1)
$$S=R\{x_1,\mathrm{},x_{n1}\}\{x_n\}R\{x_1,\mathrm{},x_{n1}\}R.$$

By induction on $`n`$ we reduce the proof to the case $`n=1`$. Indeed, from being true for $`n=1`$, we conclude that there exists a non-zero $`b_1R\{x_1,\mathrm{},x_{n1}\}`$ such that any homomorphism $`\psi _1:R\{x_1,\mathrm{},x_{n1}\}`$ with $`\psi _1(b_1)0`$ extends to $`\psi :R\{x_1,\mathrm{},x_n\}`$ with $`\psi (b)0`$, and by the inductive assumption, there exists a non-zero $`aR`$ such that any homomorphism $`\phi :R`$ with $`\phi (a)0`$ extends to $`\psi _1:R\{x_1,\mathrm{},x_{n1}\}`$ with $`\psi _1(b_1)0`$.

Thus, we may assume that $`S=R\{x\}`$. Given a homomorphism $`\phi :R`$, we may extend it to the algebras of differential polynomials $`A(y)A^\phi (y)`$ by applying $`\phi `$ to coefficients. Let $`B(y)R\{y\}`$ be such that $`B(x)=b`$. We consider separately two cases.

Case 1: $`x`$ is differentially transcendental over $`R`$. Let $`aR`$ be any non-zero coefficient of $`B(y)`$. If $`\phi (a)0`$, then $`B^\phi (y)\{y\}`$ is a non-zero differential polynomial, hence there exists $`\alpha `$ which is not a root of $`B^\phi (y)`$ (we take $`B=B^\phi (y)`$ and $`A`$ with $`ordA>ordB`$ in the definition of a differentially closed field). Then $`\psi (Q(x))=Q^\phi (\alpha )`$ is a well defined homomorphism $`S`$ with $`\psi (b)0`$.

Case 2: $`x`$ is differentially algebraic over $`R`$. Let $`A(y)R\{y\}`$ be a minimal in the partial ordering $`<`$ irreducible (in the usual sense) over $`FractR`$ differential polynomial such that $`A(x)=0`$. If $`F(y)R\{y\}`$ is such that $`F(x)=0`$, apply the division algorithm:

$$S_A(y)^mI_A(y)^nF(y)G(y)mod[A(y)],$$

where $`G<A`$. Since $`F(x)=0`$ and $`A(x)=A^{}(x)=\mathrm{}=0`$, we see that $`G(x)=0`$, hence, due to minimality of $`A`$, $`G(y)=0`$, and we have

(2)
$$S_A(y)^mI_A(y)^nF(y)[A(y)].$$

Let $`a_1R`$ be a non-zero coefficient of $`I_A(y)`$. Let $`D(y)R\{y\}`$ be the discriminant of $`A(y)`$ viewed as a polynomial in $`y^{(ordA)}`$. Note that $`D(y)0`$ since $`A(y)`$ is an irreducible polynomial, and that $`ordD(y)<ordA(y)`$. Let $`a_2R`$ be a non-zero coefficient of $`D(y)`$.

Suppose that $`\phi (a_1a_2)0`$. Then $`ordA^\phi (y)=ordA(y)>ordI_A^\phi (y)D^\phi (y)`$ and$`I_A^\phi (y)D^\phi (y)0`$. Since $``$ is differentially closed, there exists $`\alpha `$ which is a root of $`A^\phi (y)`$ but not a root of $`I_A^\phi (y)D^\phi (y)`$. Since $`D^\phi (\alpha )0`$, $`A^\phi (y)`$ and $`S^\phi (y)`$ have no common roots, hence $`S_A^\phi (\alpha )0`$. Since also $`I_A^\phi (\alpha )0`$, but $`A_A^\phi (\alpha )=0`$, we conclude from (2) that $`F^\phi (\alpha )=0`$. Therefore $`\psi (Q(x))=Q^\phi (\alpha )`$ is a well defined homomorphism $`S`$ which extends $`\phi :R`$.

It remains to take care of the condition $`\psi (b)0`$ by an appropriate choice of $`\alpha `$ (satisfying the above conditions as well). By the weak division algorithm we have:

(3)
$$S_A^n(y)B(y)=B_1(y)mod[A(y)],$$

where $`n_+`$ and $`ordB_1(y)ordA(y)`$. Letting $`y=x`$ in (3) we get

(4)
$$B_1(x)=S_A^n(x)b.$$

Since $`S_A^\phi (\alpha )0`$, we conclude that $`S_A(x)0`$, hence $`B_1(x)0`$, due to (3). Let $`r(y)R\{y\}`$ be the resultant of the polynomials $`B_1(y)`$ and $`A(y)`$ viewed as polynomials in $`y^{(ordA)}`$. Since $`A(x)=0`$ and $`B_1(x)0`$, we conclude that $`r(y)0`$ (otherwise $`A(y)`$ and $`B_1(y)`$ would have a common root in $`FractR`$ and therefore $`B_1(y)`$ would be divisible by $`A(y)`$ due to its irreducibility). Note that $`ordr(y)<ordA(y)`$. Let $`a_3R`$ be a non-zero coefficient of $`r(y)`$.

We let $`a=a_1a_2a_3`$ and suppose that $`\phi (a)0`$. Choose $`\alpha `$ such that, as before, $`A^\phi (\alpha )0`$, $`I_A^\phi (\alpha )D^\phi (\alpha )0`$, and, in addition, $`r^\phi (\alpha )0`$. As before, define $`\psi (Q(x))=Q^\phi (\alpha )`$. As before, this is a well defined homomorphism $`S`$, and, by (4): $`S_A^\phi (\alpha )\psi (b)=B_1^\phi (\alpha )`$. But $`B_1^\phi (\alpha )0`$, since $`r^\phi (\alpha )0`$ and therefore $`A^\phi (y)`$ and $`B_1^\phi (y)`$ have no common roots. Hence $`\psi (b)0`$.

###### Corollary 1.

If $`S`$ is a differentially finitely generated differential algebra over $``$, then for any non-zero $`bS`$ there exists an $``$-algebra homomorphism $`\psi :S`$ such that $`\psi (b)0`$.

3.  In order to prove Theorem 2, we need the following lemma.

###### Lemma 1.

Let $``$ be a differentially closed filed. Then

1. For any non-zero $`A(y)\{y\}`$ there exists infinitely many $`\alpha `$ such that $`A(\alpha )0`$.
2. If $`A(y)\{y\}\backslash `$ and there exists $`B(y)\{y\}`$ such that $`B(y)0`$, and $`B(y)<ordA(y)`$ and only finitely many roots of $`A(y)`$ are not roots of $`B(y)`$, then $`ordA(y)=0`$.

*Proof.*  (a) Take a sequence $`A_j(y)\{y\}`$, $`j_+`$, of increasing order, such that $`A_0(y)=A(y)`$. For each $`j=1,2,\mathrm{}`$ there exists $`\alpha _j`$ which is a root of $`A_j`$, but not of $`A_0\mathrm{}A_{j1}`$. Hence all $`\alpha _1,\alpha _2,\mathrm{}`$ are not roots of $`A(y)`$.

(b) Let $`\alpha _1,\mathrm{},\alpha _n`$ be all roots of $`A(y)`$ which are not roots of $`B(y)`$. Note that $`n1`$ since $``$ is differentially closed. Let $`B_1(y)={\displaystyle \underset{i=1}{\overset{n}{}}}(y\alpha _i)B(y)`$ and suppose that $`ordA(y)>0`$. Then $`ordB_1(y)<ordA(y)`$, hence there exists $`\beta `$ which is a root of $`A(y)`$, but not of $`B_1(y)`$. But then $`\beta \alpha _i`$ for all $`i`$, a contradiction. $`\mathrm{}`$

*Proof of Theorem 2.*  Again, using (1), we reduce the proof to the case $`S=R\{x\}`$. Certainly any homomorphism $`R\{x_1,\mathrm{},x_{n1}\}`$ extends in only finitely many ways to $`S`$ that does not annihilate $`b`$. Hence from $`n=1`$ case we conclude that $`FractS`$ is finite over $`FractR\{x_1,\mathrm{},x_{n1}\}`$. By Theorem 1, there exists $`b^{}0`$ in $`R\{x_1,\mathrm{},x_{n1}\}`$ such that any homomorphism $`\phi `$ to $``$ with $`\phi (b^{})0`$ extends to $`\psi :S`$ with $`\psi (b)0`$. Hence there exists only finitely many homomorphisms $`\psi :R\{x_1,\mathrm{},x_{n1}\}`$ which extend $`\phi :R`$ such that $`\psi (b^{})0`$. Hence, by the inductive assumption, $`FractR\{x_1,\mathrm{},x_{n1}\}`$ is finite over $`FractR`$ and $`FractS`$ is finite over $`FractR`$.

Since $`R`$ is differentially finitely generated over $``$, by Corollary 1, for any non-zero $`aR`$ we can find a homomorphism $`\phi :R`$ with $`\phi (a)0`$. Again we consider separately two cases of $`S=R\{x\}`$, keeping notations of the proof of Theorem 1.

Case 1: $`x`$ is differentially transcendental over $`R`$. Let $`aR`$ be a non-zero coefficient of $`B(y)`$, so that $`B^\phi (y)0`$. By Lemma 1a, $`B^\phi (y)`$ has infinitely many non-roots $`\alpha `$, and for each of them the homomorphism $`\psi (B(x))=B^\phi (\alpha )`$ extends $`\phi `$ such that $`\psi (b)0`$. Thus, this case is impossible.

Case 2: $`x`$ is differentially algebraic over $`R`$. Take $`a=a_1a_2a_3`$ from the proof of Theorem 1 and $`\phi :R`$ with $`\phi (a)0`$. From the proof of Theorem 1 we know that $`\phi `$ extends to $`\psi :S`$ with $`\psi (b)0`$ by letting $`\psi (Q(x))=Q^\phi (\alpha )`$, where $`\alpha `$ is a root of $`A^\phi (y)`$ and is not a root of $`I_A^\phi (y)D^\phi (y)r^\phi (y)`$. Since, by conditions of the theorem, there exists only finitely many such $`\alpha `$, we conclude by Lemma 1, that $`ordA^\phi (y)=0=ordA(y)`$. In other words, $`A(y)`$ is an ordinary (non-zero) polynomial over $`R`$ such that $`A(x)=0`$. Hence $`FractS`$ is finite over $`FractR`$.

$`\mathrm{}`$

4. Theorem 3.  The following properties of a differential filed $``$ of characteristic $`0`$ are equivalent:

1. $`F`$ is differentially closed.
2. If $`A,B\{y\}`$ are such that $`A`$ is irreducible, $`B`$ is not divisible by $`A`$ and $`ordAordB`$, then there exists $`\alpha `$ such that $`A(\alpha )=0`$, $`B(\alpha )0`$ and $`S_A(\alpha )0`$.
3. If $`𝒥`$ is a prime differential ideal of $`\{y\}`$ and $`B\{y\}\backslash 𝒥`$, then there exists $`\alpha `$ such that $`f(\alpha )=0`$ for all $`f𝒥`$ and $`B(\alpha )0`$.
4. The same as (c), where $`𝒥`$ is a radical differential ideal.
5. The same as (c) (resp. d) where $`\{y\}`$ is replaced by $`\{y_1,\mathrm{},y_n\}`$, $`n1`$.
6. If $`𝒥`$ is a proper prime differential ideal of $`\{y_1,\mathrm{},y_n\}`$, $`n1`$ arbitrary, then there exists $`\alpha `$ such that $`f(\alpha )=0`$ for all $`f𝒥`$.
7. The same as (f), where $`𝒥`$ is a proper differential ideal (resp. radical differential ideal).
8. The same as (f), where $`𝒥`$ is a proper maximal differential ideal.
9. Given any $`S`$, $`R`$ and $`b`$ as in Theorem 1, the conclusion of Theorem 1 holds for $``$.

*Proof.*  (a) implies (i) by Theorem 1. (i) implies (c) by taking $`S=\{y\}/𝒥`$, $`b=`$ image of $`B`$ in $`S`$ and applying Corollary 1 (and the differential Krull theorem). (e) trivially implies (c) and (d). A standard argument of adding extra indeterminates shows that (f) is equivalent to (e). By the differential Krull theorem, (g) is equivalent to (f). Since every prime differential ideal can be included in a maximal one, which is prime too, (h) is equivalent to (f). (b) trivially implies (a) (by taking in (a) an irreducible factor of $`A`$ of the same order). Finally, the implication (c) $``$ (b) is proved in the same way as 3) $``$ 1) in Theorem 5.1 of \[B\]. Indeed, $`\{A\}:S_A`$ is a prime ideal by Ritt’s structure theorem \[B\] Theorem 2.5 or \[K\] Lemma 7.9, and this ideal does not contain $`S_AB`$ by Ritt’s divisibility theorem \[B\] Theorem 2.4 or \[K\] Lemma 7.8. $`\mathrm{}`$

*Remark.* All proofs can be extended without difficulty to the case of several commuting derivations $`\delta _1,\mathrm{},\delta _m`$.


format: text

text:
# Optical studies of the X-ray transient XTE J2123–058 – II. Phase-resolved spectroscopy

## 1 Introduction

The soft X-ray transient (SXT) XTE J2123–058 was discovered by the All Sky Monitor (ASM) on the RXTE satellite, and confirmed by the Proportional Counter Array (PCA), on 1998 June 27 (Levine, Swank & Smith 1998). The object was promptly identified with a 17th magnitude blue star with an optical spectrum typical of SXTs in outburst (Tomsick et al. 1998a). The discovery of apparent type-I X-ray bursts (Takeshima & Strohmayer 1998) confirmed that the compact object was a neutron star and detection of twin high frequency quasiperiodic oscillations (Homan et al. 1998, 1999; Tomsick et al. 1999), as seen in other neutron star LMXBs (van der Klis 1999 and references therein) is consistent with this. Casares et al. (1998) reported the presence of a strong optical modulation and attributed this to an eclipse. The orbital period was subsequently determined to be 6.0 hr both photometrically (Tomsick et al. 1998b; Ilovaisky & Chevalier 1998) and spectroscopically (Hynes et al. 1998). Tomsick et al. (1998b) suggested that the 0.9 mag modulation is due to the changing aspect of the heated companion in a high inclination system, although partial eclipses appear also to be superimposed on this (Zurita, Casares & Hynes 1998). Ilovaisky & Chevalier (1998) reported that the light curve shows a further 0.3 mag modulation on a 7.2 d period which they suggested was associated with disc precession. By 1998 August 26 XTE J2123–058 appeared to have reached optical quiescence, at a magnitude of $`R21.7`$ (Zurita & Casares 1998; Zurita et al. 2000, hereafter Paper I).

XTE J2123–058 is an important object to study for several reasons. Firstly, neutron star transients with low-mass companions (in contrast to the high-mass Be neutron star X-ray transients and the low-mass black hole X-ray transients) are relatively rare; most neutron star low-mass X-ray binaries (LMXBs) are persistently active. It is important to compare neutron star transients with the apparently more common black hole systems in order to determine how the presence or absence of a compact object hard surface affects the behaviour of the system. Such a comparison can also be used to test models for the formation and evolution of SXTs which predict different evolutionary histories for black hole and neutron star transients (King, Kolb & Szuszkiewicz 1997 and references therein). Secondly, XTE J2123–058 has a high orbital inclination, and possibly exhibits partial eclipses. High inclination systems provide excellent tools with which to probe the system geometry. Thirdly, XTE J2123–058 is at high Galactic latitude ($`36\stackrel{}{.}2`$) and distant ($`d=8\pm 3`$ kpc, Paper I), placing it in the Galactic halo; Tomsick et al. (1999) estimate a distance of at least 2.6 kpc below the Galactic plane. For comparison, the mean altitude of the low-mass X-ray binary distribution (i.e. $`z_{\mathrm{rms}}`$) is only 1 kpc (van Paradijs & White 1995). We must therefore ask if this system is intrinsically a halo object, or if it was formed in the disc and then kicked out at high velocity when the neutron star was formed. This issue is also relevant to LMXB evolution (King & Kolb 1997).

In this paper we present the results of a spectrophotometric study of XTE J2123–058 using the William Herschel Telescope (WHT), La Palma. We will address the questions and possibilities raised above. Our photometric observations were described Paper I.

## 2 Data reduction

We observed XTE J2123–058 using the ISIS dual-beam spectrograph on the 4.2-m William Herschel Telescope (WHT) on the nights of 1998 July 18–20. A log of exposures is given in Table 1. The R300B grating combined with an EEV $`4096\times 2048`$ CCD gave an unvignetted wavelength range of $``$4000–6500 Å with some useful data outside this range. An 0.7 arcsec slit was used on July 19/20 and 1.0 arcsec on July 20/21. On both of these nights the slit was aligned to pass through a second comparison star. An additional pair of exposures were taken on July 18/19, both using an 0.7 arcsec slit, with R300B/EEV10 on the blue arm and R158R/TEK2 on the red arm. No flux calibration was attempted for these latter spectra and flat fields were not available for the blue side spectrum.

Standard iraf procedures were used to de-bias the images. Small scale CCD sensitivity variations were removed using a high signal-to-noise flat field obtained at the beginning of the night of July 20. Fringing (significant above $`5900`$ Å) and illumination variations were removed using contemporaneous flats taken at the same time as arc calibrations. One-dimensional spectra were extracted from the processed images using the optimal extraction method (Horne 1986; Marsh 1989).

Wavelength calibration was interpolated between contemporaneous exposures of a copper-argon arc lamp. Second-order flexure effects were corrected using the O \[I\] 5577.37 Å night sky emission line. This correction was always $`0.3`$ Å. We performed flux calibration as a two-stage process. The spectrograph slit was aligned to pass through another star to serve as a comparison. Spectra of both stars were extracted from each image, and the spectrum of XTE J2123–058 was then calibrated relative to the on-slit comparison star in each case. A final image using a 4 arcsec slit was used to calibrate the comparison star relative to the spectrophotometric standard Feige 110 (Oke 1990). The absolute flux calibration is sensitive to uncertainty in the extinction curve. The wide slit observation used for absolute calibration was obtained at airmass 1.72, whereas the spectrophotometric standard, Feige 110, was observed at airmass 1.25. An average extinction curve for La Palma (King 1985) was assumed in correcting for this difference but the presence of significant dust in the atmosphere at the time of the observations means that the extinction curve may be far from average and that the extinction may also depend on the azimuth of the star. Differential slit losses represent an additional problem as they vary between spectra. This difficulty will be discussed in Sect. 3.

To obtain the average blue spectrum from July 19–21 shown in the upper two panels of Fig. 1, two combined spectra were generated. The first was a straight sum of recorded counts before slit loss and extinction corrections. As the slit losses in particular are strongly dependent on wavelength and time, this improves the signal-to-noise ratio dramatically compared to averaging calibrated spectra. The second was an average of calibrated spectra resampled onto a uniform phase grid. This should give a true average energy distribution. This second spectrum was then used to calibrate the first high signal-to-noise one. The main drawback of this process is that relative fluxes of widely separated lines may be subject to systematic errors. For example, the line structure in the average blue-end spectrum is predominantly determined from the phase range $``$0.0–0.5 when slit losses were minimised, whereas the red-end line spectrum is closer to a uniformly weighted average. Our single red spectrum from July 18/19 is also shown in the lower panel of Fig. 1.

## 3 The continuum flux distribution and light curve

We show the dereddened spectral energy distribution of XTE J2123–058 in Fig. 2. For comparison, we also show the photometry of Tomsick et al. (1998a) and a spectrum of the black hole candidate GRO J0422+32 in outburst (Shrader et al. 1994). Dereddening assumes $`E(BV)=0.12`$ for XTE J2123–058 (Sect. 4.3) and $`E(BV)=0.3`$ for GRO J0422+32 (as discussed in Hynes & Haswell 1999). Power-laws of spectral index $`\alpha =1.9`$ for XTE J2123–058 and $`\alpha =0.9`$ for GRO J0422+32 (where $`F_\nu \nu ^\alpha `$) are overplotted to characterise the spectral energy distributions (SEDs) of the two objects. A comparison of these objects is appropriate as GRO J0422+32 has an orbital period of 5.1 hr and is the closest black hole analogue of XTE J2123–058. Its SED is representative of other short period black hole systems, but is redder than that of XTE J2123–058 (compared to both spectroscopy and photometry); the latter SED is closer to an $`\alpha =2`$ Rayleigh-Jeans spectrum. This suggests that the emission regions in XTE J2123–058 are hotter than in GRO J0422+32, although the possibility of errors in extinction correction, discussed in Sect. 2 weaken this conclusion.

We show in Fig. 3 light curves for three ‘continuum’ bins at 4500 Å, 5300 Å and 6100 Å. The light curves show very similar shapes, with no significant differences in profile or amplitude within this wavelength range. The apparent differences between them, most noticeable around phase 0.6, are likely due to calibration uncertainties: coverage on each night was approximately from phase 0.6 through 1.6.

The light curve morphology is fully discussed in Paper I. We believe it is mainly due to the changing aspect of the heated companion star. This is the dominant light source at maximum light, near phase 0.5. At minimum light (phase 0.0) the heated face is obscured and we see the accretion disc only. At all phases the unilluminated parts of the companion star are expected to contribute negligible flux as the outburst amplitude is $`5`$ magnitudes. We see no strong dependence of this modulation amplitude on wavelength. This is expected if the emission regions are sufficiently hot that we only see the $`F_\nu \nu ^2`$ Rayleigh-Jeans part of the spectrum and is consistent with the very steep SED shown in Fig. 2.

## 4 The line spectrum

At first glance, XTE J2123–058 presents a nearly featureless spectrum, with only the Bowen blend (N iii/C iii 4640 Å) and He ii 4686 Å prominent. A number of weaker emission lines are present too, however, with complex Balmer line profiles and weak interstellar absorption features also seen.

### 4.1 N iii-C iii and He ii 4686 Å emission

The equivalent widths of the N iii/C iii Bowen blend and He ii 4686 Å imply a N iii/C iii-He ii ratio of $`1.03\pm 0.02`$, thoroughly consistent with the ratios observed in other Galactic LMXBs (typically $`0.51.5`$; Motch & Pakull 1989). In particular, this ratio is significantly higher than seen in LMXBs in low-metallicity environments (e.g. $`<0.12`$ for LMC X-2 and $`<0.20`$ for 4U 2127+11 in the low-metallicity globular cluster M15; Motch & Pakull 1989). This suggests that XTE J2123–058 originates in the Galactic population; see Sect. 7.6 for further discussion. Note that as these two lines are very close, this ratio is not affected by the uncertainty in line flux calibration noted in Sect. 2.

### 4.2 H i Balmer and He ii Pickering lines

The Balmer lines exhibit a complex profile with broad absorption and a structured emission core. This broad absorption plus core emission line structure is common in SXTs in outburst. Balmer line profiles in GRO J0422+32 (Shrader et al. 1994; Callanan et al. 1995) look very similar, showing a double-peaked core in broad absorption.

The interpretation of the profiles is complicated by the presence of coincident He ii Pickering lines ($`n=`$ 5,6,7…to 4 transitions) within 2 Å of each Balmer line. The cores are compared in Fig. 4. We show H$`\alpha `$ and H$`\beta `$, together with the He ii 6–4, 7–4 and 8–4 lines (Pickering $`\beta `$$`\delta `$). The He ii 4686 Å line is also shown for comparison. Higher order lines show similar behaviour, but with lower resolution and signal-to-noise. The two peaks marked are clearly present in both H$`\alpha `$ and H$`\beta `$, and can also be identified in H$`\gamma `$ and H$`\delta `$. The blue peak is also seen in He ii 5411 Å (7–4), but the red peak apparently is not. Peak 4686 Å emission also coincides with the blue peak. The blue peak, however, appears weaker in flux in He ii 5411 Å than in H$`\alpha `$ or H$`\beta `$. As noted in Sect. 2 we should be cautious about systematic errors in comparing widely separated lines, and the difference is not large. We therefore cannot rule out the possibility that the blue peak is purely He ii emission. Our suggested decomposition of the line profile is therefore that the blue peak contains He ii emission, and possibly also some Balmer emission. The red peak is pure Balmer emission. The broad absorption also appears to be a Balmer feature.

There is a further complication. As will be described in more detail in Sect. 5, H$`\alpha `$, and possibly also H$`\beta `$, are also subject to transient absorption, strongest in the phase range 0.35–0.55. The average H$`\alpha `$ line profile in this phase range is shown in Fig. 7. As the absorption component clearly lies between the two peaks, it may be that weaker absorption at all phases is partly responsible for the apparent separation of the two peaks.

### 4.3 Interstellar lines

We measure equivalent widths for the Na D lines of $`0.29\pm 0.02`$ and $`0.30\pm 0.02`$ Å. Based on the strength of the D1 line and the calibration of Munari & Zwitter (1997) we estimate $`E(BV)=0.12\pm 0.05`$. Assuming $`R_\mathrm{V}=3.1`$, this implies $`A_\mathrm{V}=0.37\pm 0.15`$, agreeing with the value of $`A_\mathrm{V}=0.36`$ quoted by Homan et al. based on dust maps in Schlegel, Finkbeiner & Davis (1998). We note, however, that Soria et al. (1999) estimate a somewhat lower value of $`E(BV)=0.054`$ using the reddening maps derived by Burstein & Heiles (1982) from H i column densities and galaxy counts.

## 5 Line variability

The emission lines show significant changes over an orbital cycle, with differences in behaviour between lines. N iii/C iii 4640 Å (Bowen blend) shows a flux modulation, as does He ii 4686 Å emission which also shows complex changes in line position and structure. He ii 6560 Å and H$`\alpha `$ are blended and in addition to probably showing flux and wavelength modulations are subject to a transient absorption feature that is strongest around phase 0.5. The changes are illustrated with integrated light curves in Fig. 5 and with trailed spectrograms in Fig. 6.

The Bowen blend light curve appears similar to the continuum light curves shown in Fig. 3. Bowen emission may therefore originate on the heated face of the companion star, with the modulation arising from the varying visibility of the heated region.

He ii light curves show a double peaked structure with a strong peak near phase 0.75 and a weaker one near 0.25. We should be cautious in interpreting this light curve in detail, however, as the complex line behaviour seen in the trailed spectrogram indicates multiple emission sites; the integrated light curve is an average of different light curves of several regions.

H$`\alpha `$ emission is blended with He ii 6560 Å, and as discussed in Sect. 4 also comprises several components. The most striking is the transient absorption component seen near phase 0.5. This can be clearly seen in the light curve and the trailed spectrogram. The light curve is deceptive as it also reflects changes in the emission flux; it suggests that the absorption feature is centred at phase 0.5, but, as can be seen from the trailed spectrogram, this is not really the case. The feature begins quite sharp and red, around phase 0.3 and then becomes progressively more extended to the blue, fading out by phase 0.6. Whilst it is sometimes difficult to distinguish an absorption component from an absence of emission, that is not a problem here; the absorption definitely extends below the continuum as can clearly be seen in Fig. 7.

## 6 Doppler tomography

We have used the technique of maximum entropy Doppler tomography (Marsh & Horne 1988) to identify He ii 4686 Å emission sites in velocity space. Our results are shown in Fig. 8. The analytical back-projection method gave similar results. We should beware, however, that one of the fundamental assumptions of Doppler tomography, that we always see all of the line flux at some velocity, is clearly violated, as the integrated line flux is not constant. We attempt to account for this by normalising the spectra (dividing by a smooth fit to the He ii 4686 Å light curve in Fig. 5). If we do not normalise then the structure of the derived tomogram is very similar, so our results do not appear to be sensitive to this difficulty.

Ideally, to perform Doppler tomography we should know the systemic velocity of the binary. We have no a priori knowledge of this, so we estimate it from the He ii 4686 Å line itself. This is done by repeating the tomographic analysis for different assumed mean velocities and seeking to minimise the residuals of the fit of the reconstructed spectrogram to the data (Marsh & Horne 1988). Our measured heliocentric systemic velocity is $`90`$ km s<sup>-1</sup> The implications of this velocity are discussed in Sect. 7.6.

In order to interpret the tomogram quantitatively, we also require estimates of system parameters. These were derived from results of fits to outburst light curves (see Paper I). The parameters we assume are given in Table 3. The masses deserve comment. The light curve fits only provide a significant constraint on the mass ratio, of $`q=M_1/M_2=4.6_{0.2}^{+0.5}`$. To obtain actual masses, one must make physical assumptions. We assume that the neutron star has a typical mass of $`M_11.4`$ M (see Thorsett & Chakrabarty 1999 for a review of neutron star masses) and hence that the companion should have a mass of 0.3 M. As discussed in Paper I, a main sequence companion should have a mass of 0.6 M so it is likely that the companion is evolved and undermassive. Other SXTs do appear to have undermassive companions (Smith & Dhillon 1998), so this is a very plausible scenario, and one which is consistent with theoretical predictions (King, Kolb & Burderi 1996) that short period neutron star LMXBs should only be transient if they have evolved companions.

The dominant emission site (corresponding to the main S-wave) appears on the opposite side of the neutron star from the companion. For any system parameters, it is inconsistent with the heated face of the companion and the stream/disc impact point. There is a fainter spot near the $`\mathrm{L}_1`$ point, corresponding to the fainter S-wave, which is possibly associated with the accretion stream. There also appears to be a loop connecting these two spots and a fainter arc on the right completing the circle. As noted above, however, the assumption that all the line flux is always visible at some velocity is not true for these data, and normalising the total flux is only an approximate solution when several emission components are present. It is likely, therefore, that some low-level artefacts are present and so weaker features in the tomogram may not represent real emission sites. To test this, we have also generated tomograms in which data along the S-wave corresponding to the bright spot are given a very low weight (see Steeghs et al. 2000). This should prevent artefacts due to misfitting of the time-dependence of this feature. We find that the upper left loop appears robustly, and is likely to be a real structure. The apparent arc completing the circle on the right does not appear and so is probably spurious. It is notable that nearly all emission has a lower velocity than Keplerian material at the outer disc edge could have, and so if associated with the disc requires sub-Keplerian emission.

## 7 Discussion

### 7.1 A neutron star SW Sex system?

In several respects XTE J2123–058 resembles the SW Sex class of cataclysmic variables. These are all novalike variables, typically seen at a relatively high inclination. Their main defining characteristics are (Thorstensen et al. 1991; Horne 1999; Hellier 2000):

1. Balmer lines are single-peaked and their motions lag behind that of the white dwarf. In a Doppler map emission is typically (though not always) concentrated in the lower-left quadrant at velocities lower than any part of the disc. A high velocity component is detectable with velocity semi-amplitude $`1000`$ km s<sup>-1</sup>. This has the same phasing as the low velocity component; and their separate identity is vigorously debated.
2. Transient absorption lines are seen. These are strongest near phase 0.5 or slightly earlier and move from red to blue between phases 0.2 and 0.6.
3. Continuum eclipses have a V-shaped profile implying a flatter temperature distribution than expected theoretically. Balmer line eclipses are shallow and occur earlier than continuum eclipses. Red shifted emission remains visible at mid-eclipse.

On the first two points XTE J2123–058 shows striking similarities to the SW Sex systems. Anomalous emission line behaviour is most clearly seen in He ii 4686 Å in this higher excitation system. This line appears, on average, single peaked. The emission is concentrated in the lower-left quadrant at sub-Keplerian velocities. A high velocity component is not detected, but cannot be ruled out without higher quality data. Transient absorption is seen in the Balmer lines before and around phase 0.5. It begins slightly redshifted and moves to the blue. XTE J2123–058 is the SXT with the highest known inclination ($`i=73^{}\pm 4^{}`$; Paper I), strengthening the similarity to SW Sex systems. It does not deeply eclipse, however, so it is impossible to comment on the third point.

Various models have been proposed for the SW Sex phenomenon including bipolar winds, magnetic accretion and variations on a stream overflow theme. Current opinion favours the latter interpretation, with the stream either being accelerated out of the Roche lobe by a magnetic propeller, or re-impacting the disc. Both of these models are discussed in more detail below.

### 7.2 Is this behaviour typical of LMXBs?

The only other SXT for which outburst Doppler tomography of adequate quality is available is the black hole candidate GRO J0422+32 (Casares et al. 1995). Using ephemerides determined subsequently (Harlaftis et al. 1999), emission in this object appears to originate from the accretion stream/disc impact point. These data were obtained in the December 1993 mini-outburst, $`500`$ d after the peak of the primary outburst.

Amongst other short-period neutron star LMXBs in an active state, only 2A 1822–371 (Harlaftis, Charles & Horne 1997) has Doppler tomography. This was observed in H$`\alpha `$ and exhibited disc emission enhanced towards the stream impact point. Similar behaviour is suggested by a radial velocity analysis of He ii (Cowley, Crampton & Hutchings 1982). This is different to what we see in XTE J2123–058. Radial velocity analyses of other systems, however, suggest similar behaviour to that seen here. 4U 2129+47 shows a very similar blue spectrum to XTE J2123–058 (Thorstensen & Charles 1982). Clear variations in the He ii 4686 Å radial velocity are seen with maximum radial velocity around phase 0.7–0.8 and semi-amplitude $`216\pm 70`$ km s<sup>-1</sup>. Spectroscopy of EXO 0748–676 (Crampton et al. 1986) also reveals radial velocity modulations in the He ii 4686 Å line. The radial velocity curve measured from the line base shows maximum velocity at phase $`0.76\pm 0.16`$ with semi-amplitude $`210\pm 92`$ km s<sup>-1</sup>. For comparison, a similar radial velocity curve fit to our He ii spectra (Hynes et al. 1998) yielded maximum velocity close to phase 0.75 with semi-amplitude $`180`$ km s<sup>-1</sup>. Augusteijn et al. (1998) studied 4U 1636–536 and 4U 1735–44 and found the two systems similar to each other. Both show He ii modulations with semi-amplitude 140–190 km s<sup>-1</sup> with maximum radial velocities measured at phases 0.87–1.04.<sup>1</sup><sup>1</sup>1These phases have shifted by 0.5 with respect to those quoted by the authors as they define phase 0 as photometric maximum. We therefore suggest that the behaviour we see is common, but not universal, in short period neutron star LMXBs in a high state.

### 7.3 The magnetic propeller interpretation

It is possible to explain the behaviour of the He ii 4686 Å line in terms of the magnetic propeller model. This model also provides a possible mechanism for explaining the transient Balmer absorption, although this is less satisfactory.

The essence of this model is that in the presence of a rapidly rotating magnetic field, accreting diamagnetic material may be accelerated tangentially, leading to it being ejected from the system. It is easy to see how such a scenario may arise due to a rapidly spinning, magnetised compact object. This mechanism has been invoked to explain the unusual behaviour of the cataclysmic variable AE Aqr in terms of a magnetic field anchored to a spinning white dwarf (Wynn, King & Horne 1997; Eracleous & Horne 1996). It is also likely that magnetic propellers driven by neutron star magnetic fields exist, as proposed by Illarianov & Sunyaev (1975). These may be very important in suppressing accretion in neutron star SXTs in quiescence; see Menou et al. (1999) and references therein for a discussion of this situation. There is an important difference of scale, however; spin periods of white dwarfs are $`30`$ s; spin periods of neutron stars are typically of order milliseconds. For XTE J2123–058, $`P_{\mathrm{spin}}=3.92\pm 0.22`$ ms was suggested by Tomsick et al. (1999) based on the beat-frequency model for kilohertz QPOs. This identifies the separation of the QPO peaks with the spin period. While the peak separation is not always exactly constant, Psaltis et al. (1999) examine the model and conclude that the separation is still likely to be close to the spin period. Thus the spin period of the neutron star in XTE J2123–058 is probably not far from 4 ms. For such rapid rotation we must consider the light-cylinder, defined by the radius at which the magnetic field must rotate at the speed of light to remain synchronised with the compact object spin. Outside of this radius, the magnetic field will be unable to keep up and hence becomes wound up. This will make the propeller less effective, although it may still produce some acceleration (Wynn et al. 1997; Wynn priv. comm.). For a $`4`$ ms spin period in XTE J2123–058, this is $`6\times 10^5R_{\mathrm{disc}}`$; for comparison, the closest approach of a ballistic stream is $`0.2R_{\mathrm{disc}}`$. We would therefore not expect a neutron star propeller to strongly affect the stream overflow material, although simulations are needed to test this. An alternative mechansim was suggested by Horne (1999) for SW Sex systems, in which a magnetic propeller can arise from the magnetic fields of the disc itself. This has problems, in that it should very efficiently remove angular momentum from the disc (Wynn & King priv. comm.), but this could perhaps be overcome if only a small fraction of stream material passes through the propeller. This mechanism allows much lower angular velocities for the field and may be the only large scale propeller mechanism possible in a neutron star system.

We adopt the parameterisation used by Wynn, King & Horne (1997) to describe the magnetic propeller in AE Aqr. The magnetic field is simplistically modelled as a dipole with angular velocity $`\omega _\mathrm{f}`$. While such a field may be a reasonable representation of that of a compact star (white dwarf or neutron star), it is clearly a gross simplification of that of a disc-anchored propeller. In the latter case the net effect would probably arise from a mixture of unstable field structures of varying strengths and anchored at a range of radii. For simplicity and consistency with existing work on magnetic propellers, however, we approximate the complex field structure with a single rotating dipole. The propeller mechanism assumes that the flow breaks up into diamagnetic blobs of varying size and density. The field can only partially penetrate the blobs, and hence they are accelerated, but not locked to the field lines. The magnetic acceleration on a blob is given by $`g_{\mathrm{mag}}=k[vv_\mathrm{f}]_{}`$, where $`v`$ is the stream velocity, $`v_\mathrm{f}=r\omega _\mathrm{f}`$ is the field velocity and the component of the velocity difference perpendicular to the field is taken. The parameter $`k`$ parameterises our ignorance of the magnetic field strength and blob size and density, and following Wynn & King (1995) we adopt the functional dependence $`k=k_0(r/r_0)^2`$, where $`r_0=10^{10}`$ cm. Different blob sizes and densities will lead to a spread of values of $`k_0`$ and hence different trajectories. The uncertainty in the angular velocity of the magnetic field is also factored into the uncertainty in $`k_0`$ provided it is high enough that $`v_\mathrm{f}v`$ at all points along the stream trajectory, and low enough that all regions of interest are within the light cylinder. For XTE J2123–058, this corresponds to a field period of approximately $`500\mathrm{s}P_{\mathrm{spin}}10`$ s (i.e. a field anchored at $`0.16r_{\mathrm{disc}}R0.012r_{\mathrm{disc}}`$). Within this regime varying $`P_{\mathrm{spin}}`$ has the same effect as varying $`k_0`$, and different trajectories can be described by a single parameter. This gives a plausible range of radii for anchoring the magnetic field and so we adopt this simple case for our model and arbitarily set $`P_{\mathrm{spin}}=50`$ s, corresponding to a field anchored at radius $`0.03`$ of the disk radius. As noted above, this is only an approximation to a more chaotic field structure.

As we are considering a system in which most of the accretion likely proceeds through the disc, we assume that the propeller mechanism acts only on overspill from the stream-disc impact point. Our model therefore calculates a trajectory which is purely ballistic until it reaches the disc edge and is subject to a propeller force thereafter. Quantitatively, this makes little difference to the results compared to allowing the magnetic force to act everywhere on the stream.

We adjust the $`k_0`$ parameter to find a trajectory which passes through the central emission on the tomogram. This is shown in Fig. 9 together with two bracketting trajectories to show the dependence on $`k_0`$. For this to be a plausible model, we also require an explanation for why one particular place on the trajectory is bright. Such an explanation is offered by Horne (1999): there is a point outside the binary at which trajectories for different values of $`k`$ intersect. At this point, faster moving blobs cross the path of slower blobs and enhanced emission might be expected. This can be seen in Fig. 9. We identify the region where the trajectories collide with the emission region.

To make a quantitative comparison of our data with the model, we transform the brightest emission spot of the tomogram into real space. Such a conversion is not in general possible, of course, but can be performed if system parameters and a velocity field are assumed, provided that the velocity–space mapping is one–one or many–one. That is true in this case if we only consider emission from the first loop around the binary. We construct a series of trajectories in velocity space defined by uniform steps in $`k_0`$, and select points for which the tomogram intensity is above a certain value. The selected points are shown spatially in the right hand panel of Fig. 9 as region A. We can see that, subject to the assumptions made, emission does occur at the point where trajectories cross. Our data are therefore consistent with the emission mechanism suggested by Horne (1999). We emphasise that this spatial map is only valid in the context of the propeller model.

One point should be noted. Hellier (2000) suggests that the propeller model should predict a continuous S-wave, which is not seen. This is not necessarily the case. If the region in which accelerated blobs collide is optically thick in the line then emission is expected predominantly from the inner edge of this region where fast moving blobs impact slow moving blobs. For the geometry we have considered, as can be seen in Fig. 9, this will result in emission predominantly towards the top of the figure. This corresponds to seeing maximum emission near phase 0.75, exactly as is observed in our data and in that of SW Sex systems, such as PX And (Hellier & Robinson 1994). The phase dependence of emission line flux is therefore not a problem with the propeller model, but instead a prediction made by it.

We have demonstrated that both our He ii tomogram and the line-flux light curve are consistent with emission from the region where material ejected by a propeller is expected to converge. Can we interpret the phase-dependent Balmer absorption as arising from the same region, as was suggested by Horne (1999) in the context of SW Sex systems? This model certainly predicts some ejecta in front of the disc and/or companion at phases 0.3–0.6 where absorption is observed. As can be seen in Fig. 6, however, we also have information on the absorption kinematics. We can thus perform a similar mapping to that used for the emission to determine if absorption is predicted with the right phasing and kinematics and where it would have to originate. We construct a fine grid of trajectories and for each point calculate the phase and velocity at which this would pass in front of the centre of the disc. If this lies within the absorption region in the trailed spectrogram then we mark that point as within region B in Fig. 9. We find that the resulting points do populate the observed absorption region fully, so the model can produce the absorption. The material involved, however, is not on the same trajectories as the emission, but occurs on lower velocity tracks. The absence of emission from region B may indicate that only high velocity material undergoes sufficiently energetic collisions to produce He ii emission. The converse, that absorption does not reach to region A, could arise because the emission region has a higher brightness temperature in the lines than the disc continuum. It also remains unexplained why absorption is only seen along part of the trajectories. For the low velocity trajectories, some material may fall back onto the disc. The phase dependence may also be related to the height of the absorbing material above the disc plane, as may the lack of absorption from region A. Clearly these are further questions to address in developing a full propeller model for these data. Such a three-dimensional treatment is beyond the scope of our simplistic model however, and we leave these points as observational requirements that a full model must satisfy.

### 7.4 The stream overflow and disc re-impact interpretation

The most popular alternative explanation for the SW Sex phenomenon is the accretion stream overflow and re-impact model. This was originally suggested by Shafter, Hessman & Zhang (1988); for a recent exposition see Hellier (2000).

In this model the overflowing stream can be thought of in two regions. The initial part of the stream produces the transient absorption when seen against the brighter background of the disc. The latter part, where the overflow re-impacts the disc, is seen in emission giving rise to the high velocity component. To reproduce observations, this model requires a number of refinements.

In the simplest form of the model, the overflow stream should produce absorption at all phases; it always obscures some of the disc. This problem is overcome by invoking a strongly flared disc so that the overflow stream is only visible near phase 0.5. We see an absorption depth of 15 per cent of the continuum flux. At this phase, at least 40 per cent of the continuum flux actually comes from the companion star, so we require absorption of 25 per cent of the visible disc light to achieve the observed depth, and hence at least 25 per cent of the visible disc to be covered by the obscuring material; more if absorption is not total. With a strongly flared disc, however, the near side may be self-occulted so only $`10`$ per cent of the total disc area need be covered by the overflowing material. This is not implausible, and simulations can produce such a messy splash (Armitage & Livio 1998). There are difficulties in invoking a strongly flared disc in XTE J2123–058, however. It has lower inclination than typical SW Sex systems: $`73^{}\pm 4^{}`$ (Paper I) and does not show X-ray eclipses. The strong photometric modulation due to heating of the companion also constrains the amount that the disc can be flared; too much and the companion will be shielded from X-rays. In Paper I an upper limit to the flare angle of $`8\stackrel{}{.}6`$ was derived (99 per cent confidence) from fitting of optical lightcurves $`10`$ days after the WHT observations. Comparing this with the viewing angle of $`17^{}\pm 4^{}`$ it is clear that the near side of the disc, and hence the overflowing material, should never be completely obscured, rendering the phase dependence difficult to account for.

The re-impact model requires that the strong emission in the lower-left quadrant be produced by overlap of weaker emission from the high-velocity component (the re-impact point) and the disc. To produce emission at the low velocities observed, however, requires the ‘disc’ component to be sub-Keplerian by a significant amount. This problem is not peculiar to XTE J2123–058, for which parameters are uncertain, but common to the SW Sex class in general. In our case, such a sub-Keplerian ring may be present, but it is likely that this is not a real feature. There is no evidence for a high velocity component, but given our data quality, this is also inconclusive.

### 7.5 X-ray signatures of the absorbing material

In principal X-ray observations can provide a diagnostic tool not available in SW Sex systems, and could constrain the location of the absorbing material. In the overflow and reimpact model it is necessary for the disc to be highly flared to reproduce the phase dependence of the absorption, and for the overflow to splash over a moderately large area to reproduce the absorption depth. One might then expect that some of the overflowing material would cross our line of sight to the neutron star and produce X-ray dips in the phase range $`0.6`$–1.0, as seen in other high inclination LMXBs, the dipping sources. Such dips are also possible in the propeller model, although not necessary as this does not require a flared disc or extended splash. This model would, however, be expected to produce X-ray dips in the same range as the transient H$`\alpha `$ absorption, $`0.3`$–0.6.

Tomsick et al. (1998b) reported no evidence for an X-ray orbital modulation in RXTE/PCA data. We have obtained archival PCA data from 1998 July 14 and 22 and examined Standard-2 mode lightcurves to reexamine this issue. Background subtraction was done using bright source background models dated 1999 September 17 and lightcurves were extracted for both the whole PCA bandpass and for 3–5 keV only. We find no evidence for dips of longer than 16 s with depth greater than 10 per cent at any orbital phase. We have also phase-binned all of these data to search for broader structures. Approximately 60 per cent of phase bins have data combined from at least two binary orbits; amongst these, with the exception of the X-ray bursts (Takeshima & Strohmayer 1998; Tomsick et al. 1998b) there are no excursions of greater than $`\pm 10`$ per cent about the mean.

It is unfortunate that the RXTE data is not simultaneous with our WHT observations and phase coverage is incomplete. Thus we can say that X-ray observations provide no evidence in favour of either the reimpact or propeller models, but they do not rule either out.

### 7.6 The origin of XTE J2123–058

The question of the origin of XTE J2123–058 was raised in Sect. 1: is it intrinsically a halo object or was it kicked out of the Galactic plane? The strongest evidence is provided by the ratio of N iii/C iii-He ii. This is typical of Galactic LMXBs and is inconsistent with known halo objects. In principal, the systemic velocity is also an important clue. For this object to have reached its present location it must have received a kick when the neutron star was formed, and hence it may now have a large peculiar velocity (150–400 km s<sup>-1</sup>, Homan et al. 1999). Our measured radial component of velocity is $`90`$ km s<sup>-1</sup> which is not inconsistent with the predictions of Homan et al., which could also be lowered if the source distance is less than the 10 kpc value assumed by those authors. We should add a note of caution, however, as systemic velocities inferred from emission lines are sometimes unreliable; a definitive measurement of the systemic velocity must await a study based on photospheric absorption lines of the companion star.

## 8 Conclusions

Three key issues were raised in Sect. 1 and we conclude by returning to these. As a neutron star transient, XTE J2123–058 is valuable for comparison with black hole systems. Of the latter, GRO J0422+32 is the most similar, and appears to show a less steep optical spectral energy distribution than XTE J2123–058, consistent with the suggestion of King et al. (1997) that irradiation is more efficient in neutron star systems. The high orbital inclination of XTE J2123–058 has made possible Doppler tomography of He ii emission, and the detection of transient Balmer line absorption. These effects suggest similarities to SW Sex type cataclysmic variables. Unlike SW Sex systems, XTE J2123–058 shows characteristic SW Sex emission line kinematic effects (in He ii) decoupled from transient absorption providing constraints not possible when both effects occur in the same line. We have examined our results in the context of stream overflow models applied to these systems, both with stream reimpact and propeller ejection. The reimpact model, while plausible physically, has difficulties accounting for our observations. The propeller model can successfully explain both emission and absorption behaviour, but has yet to overcome theoretical difficulties. Both theoretical work, and higher quality observations will be required to conclusively decide between the models, and it remains possible that neither is correct. As an object currently located in the Galactic halo, the origin of XTE J2123–058 is also of interest. Bowen line strengths, relative to He ii, indicate abundances similar to Galactic sources but different to globular cluster and Magellanic LMXBs and hence we suggest that this object was formed in the plane and subsequently kicked out.

## Acknowledgements

RIH was initially supported by a PPARC Research Studentship and now by grant F/00-180/A from the Leverhulme Trust. Thanks to Chris Shrader for providing the spectrum of GRO J0422+32 for comparison. Doppler tomography used molly, doppler and trailer software by Tom Marsh. RIH would like to thank Danny Steeghs, and Daniel Rolfe for enlightening discussions and useful software; Graham Wynn, Jan van Paradijs, Keith Horne and Tom Marsh have also provided valuable suggestions and advice, as has our referee, Coel Hellier. The William Herschel Telescope is operated on the island of La Palma by the Isaac Newton Group in the Spanish Observatorio del Roque de los Muchachos of the Instituto de Astrofísica de Canarias. This research has made use of the SIMBAD database, operated at CDS, Strasbourg, France, the NASA Astrophysics Data System Abstract Service and of data obtained through the High Energy Astrophysics Science Archive Research Center Online Service, provided by the NASA/Goddard Space Flight Center.


text:
# Data clustering and noise undressing of correlation matrices

## I Introduction

Statistical mechanics typically addresses the question of how structures and order arising from interactions in extended systems are dressed, and eventually destroyed, by stochastic – so-called thermal – fluctuations. The inverse problem, unraveling the structure of correlations from stochastic fluctuations in large data sets, has only recently been addressed using ideas of statistical mechanics. This is the case of data clustering problems, where the goal is to classify $`N`$ objects, defined by $`D`$ dimensional vectors $`\{\stackrel{}{\xi }_i\}_{i=1}^N`$, in equivalence classes. The general idea consists in postulating a cost function that measures how a possible data structure compares with the sample one is studying. This cost function can be considered as an Hamiltonian whose low energy states correspond to the cluster structures which are mostly compatible with the data sample. Structures are identified by configurations $`𝒮=\{s_i\}_{i=1}^N`$ of class indices, where $`s_i`$ is the equivalence class to which object $`i`$ belongs. Regarding $`s_i`$ as Potts spins, a Potts Hamiltonian $`H_q=_{i<j}J_{i,j}\delta _{s_i,s_j}`$ has been recently proposed  as a cost function, with couplings $`J_{i,j}`$ decreasing with the distance $`d_{i,j}=\stackrel{}{\xi }_i\stackrel{}{\xi }_j`$ between objects $`i`$ and $`j`$. The underlying structure of data sets emerges as the clustering of Potts variables at low temperatures.

In the present work we address the question of data clustering. Rather than postulating the form of the Hamiltonian, we start from a statistical Ansatz and invoke maximum likelihood and maximum entropy principles. In this way, the structure of the Hamiltonian arises naturally from the statistical Ansatz, without the need of assumptions on its form. The method is particularly suited to study high dimensional data sets, where each object is characterized by a large number $`D1`$ of properties. Time series are an ideal example of high dimensional objects. The study of the structure of correlations between time series is therefore a crucial benchmark for our method.

First we derive the form of the Hamiltonian in the general case. Then we study the thermal and the ground state properties of this Hamiltonian by Monte Carlo methods, in three different cases: 1) a synthetic uncorrelated data set 2) a synthetic data set with a known correlation structure and finally 3) a data set composed of financial time series with unknown correlations. We find that 1) for random uncorrelated time series no persistent structure emerges at low temperatures; 2) if the time series are generated with some cluster structure $`𝒮^{}`$, we find a phase transition to a low temperature phase which is dominated by cluster configurations close to $`𝒮^{}`$. Hence the method does not introduce spurious correlations and is able to recover known correlation structures. The nature of the transition is investigated by a simple mean field calculation in a simple case. This reveals that the phase transition is of first order.

The financial time series that we will study consists of the returns of the assets composing the S&P500 index, whose correlations have been the subject of much recent interest. On one side it has been observed  that the S&P500 correlation matrix is affected by considerable noise-dressing. Indeed its spectral properties are close to those of random, uncorrelated time series. On the other hand, these same correlation matrices have revealed a non-trivial structure of correlations when analyzed by minimal spanning tree methods  and by the method  of ref. . These apparently contradictory results raise the issue of disentangling in a systematic way the effects of fluctuations from real correlations in a large but finite data set. This is the main issue we shall focus here.

Quite interestingly, our analysis of the S&P500 data set reveals a low temperature behavior dominated by few clusters of correlated assets with scale invariant properties. We shall not enter into the details of the economic meaning of our findings, which shall be discussed elsewhere . Our aim is rather to address the problem of revealing the structure of bare correlations hidden in a finite data set. We show that a thermal average over the relevant cluster structures provides a good fit of the financial correlations, which allows us to estimate the noise-undressed correlation matrix. Finally, we discuss several generalizations of our approach to generic data clustering.

## II The Method

Let the data set $`\mathrm{\Xi }=\{\stackrel{}{\xi }_i\}_{i=1}^N`$ be composed of $`N`$ sets $`\stackrel{}{\xi }_i=\{\xi _i(d)\}_{d=1}^D`$ of $`D`$ measures. These are normalized to zero mean $`_d\xi _i(d)/D=0`$ and unit variance $`_d\xi _i^2(d)/D=1`$. For example, in our application below $`\xi _i(d)`$ is the normalized daily returns of asset $`i`$ of the S&P500 index, in day $`d`$. The data set can also refer to a set of $`N`$ objects which are characterized by $`D`$ measured quantities. In this case $`\xi _i(d)`$ is the “normalized” value of property $`d`$ for object $`i`$. We assume that $`\xi _i(d)`$ are Gaussian variables. The reason is that we want to focus exclusively on pairwise correlations and the Gaussian model is the only one which is completely specified at this level. We shall discuss later how deviations from Gaussian statistics can be accounted for. The key quantity of interest is the matrix

$$C_{i,j}(D)\frac{1}{D}\underset{d=1}{\overset{D}{}}\xi _i(d)\xi _j(d).$$
(1)
In order to investigate the structure of correlations, let us assume that $`\xi _i(d)`$ were generated by the equation

$$\xi _i(d)=\frac{\sqrt{g_{s_i}}\eta _{s_i}(d)+ϵ_i(d)}{\sqrt{1+g_{s_i}}}.$$
(2)
Here $`g_s>0`$ and $`s_i`$ are integer variables (so-called Potts spins), $`\eta _s(d)`$ and $`ϵ_i(d)`$ are iid Gaussian variables with zero average and unit variance.

The Ansatz of Eq. (2) was proposed by Noh  to explain the spectral properties found in ref. . The idea behind it is that each set $`i`$ belongs to one cluster $`s_i`$ and that sets $`i`$ and $`j`$ in the same cluster ($`s_i=s_j=s`$) are correlated ($`C_{i,j}g_s/(1+g_s)`$) whereas sets in different clusters ($`s_is_j`$) are independent. The $`s^{\mathrm{th}}`$ cluster is composed of $`n_s`$ sets with internal correlation $`c_s`$, where

$$n_s=\underset{i=1}{\overset{N}{}}\delta _{s_i,s},c_s=\underset{i,j=1}{\overset{N}{}}C_{i,j}\delta _{s_i,s}\delta _{s_j,s}.$$
(3)
In order to allow for totally uncorrelated sets, we allow $`s_i`$ to take all integer values up to $`N`$. Hence $`𝒮=\{s_i\}_{i=1}^N`$ describes the structure of correlations whereas the parameters $`𝒢\{g_s\}_{s=1}^N`$ tune the strength of these correlations.

Note that this Ansatz is different from the explicative factor model used in financial applications , which is discussed in the Appendix.

The correlation matrix generated by Eq. (2) for $`D\mathrm{}`$ is

$$C_{i,j}=\frac{g_{s_i}\delta _{s_i,s_j}+\delta _{i,j}}{1+g_{s_i}}.$$
(4)
Its distribution of eigenvalues is simple: To each $`s`$ with $`n_s1`$ there correspond one eigenvalue

$`\lambda _{s,0}={\displaystyle \frac{1+g_sn_s}{1+g_s}}`$

and $`n_s1`$ eigenvalues $`\lambda _{s,1}=1/(1+g_s)`$. Hence, large eigenvalues correspond to groups of many ($`n_s1`$) sets. For $`D`$ finite, we expect noise to lift degeneracies between $`\lambda _{s,1}`$ but to leave the structure of large eigenvalues unchanged.

In order to fit the data set $`\mathrm{\Xi }`$ with Eq. (2), let us compute the likelihood. This is the probability $`P(\mathrm{\Xi }|𝒮,𝒢)`$ of observing the data $`\mathrm{\Xi }`$ as a realization of Eq. (2) with structure $`𝒮`$ and parameters $`𝒢=\{g_s\}_{s=1}^N`$, and it reads

$`P(\mathrm{\Xi }|𝒮,𝒢)={\displaystyle \underset{d=1}{\overset{D}{}}}{\displaystyle \underset{i=1}{\overset{N}{}}}\delta \left(\xi _i(d){\displaystyle \frac{\sqrt{g_{s_i}}\eta _{s_i}(d)+ϵ_i(d)}{\sqrt{1+g_{s_i}}}}\right)`$

where the average is over all the $`\eta `$’s and $`ϵ`$’s variables and $`\delta (x)`$ is Dirac’s delta function. Gaussian integration and elementary algebra leads to

$`P(\mathrm{\Xi }|𝒮,𝒢)`$ $``$ $`e^{DH\{𝒮,𝒢\}}`$ (5)
$`H\{𝒮,𝒢\}`$ $`=`$ $`{\displaystyle \frac{1}{2}}{\displaystyle \underset{s}{}}[(1+g_s)(n_s{\displaystyle \frac{g_sc_s}{1+g_sn_s}})`$ (7)
$`n_s\mathrm{ln}(1+g_s)+\mathrm{ln}(1+g_sn_s)].`$
For any given structure $`𝒮`$ and $`D1`$, the likelihood $`P(\mathrm{\Xi }|𝒮,𝒢)`$ is maximal for $`g_s=\widehat{g}_s`$, where

$$\widehat{g}_s=\frac{c_sn_s}{n_s^2c_s}$$
(8)
for $`n_s>1`$ and $`\widehat{g}_s=0`$ for $`n_s1`$. Inverting Eq. (8) gives $`c_s=(\widehat{g}_sn_s^2+n_s)/(\widehat{g}_s+1)`$ which is exactly what one would get combining Eqs. (3,4). Hence the maximum likelihood estimators $`\widehat{g}_s`$ are consistent with our Ansatz (2).

Note that for uncorrelated sets $`C_{i,j}=\delta _{i,j}`$ we have $`c_s=n_s`$ for each $`s`$ and hence $`\widehat{g}_s=0`$. The coupling strength $`\widehat{g}_s`$ instead diverges for totally correlated sets ($`C_{i,j}=1`$) because $`c_s=n_s^2`$.

An expansion to second order in $`g_s\widehat{g}_s`$ of Eq. (7) shows that the likelihood quickly vanishes for $`|g_s\widehat{g}_s|1/\sqrt{D}`$. Hence, for $`D1`$, we can simplify things considerably by setting $`g_s=\widehat{g}_s`$ in Eq. (7). The likelihood of structure $`𝒮`$ under Ansatz (2) then takes the form $`P(\mathrm{\Xi }|𝒮)e^{DH_c}`$, where

$$H_c\{𝒮\}=\frac{1}{2}\underset{s:n_s>0}{}\left[\mathrm{log}\frac{c_s}{n_s}+(n_s1)\mathrm{log}\frac{n_s^2c_s}{n_s^2n_s}\right].$$
(9)
The ground state $`𝒮_0`$ of $`H_c`$ yields the maximum likelihood fit with Eq. (2). This would probably take the Ansatz (2) too seriously. In general, it is preferable to consider probabilistic solutions $`P\{𝒮\}`$ and, following ref. , we invoke the maximum entropy principle: Among all distributions $`P\{𝒮\}`$ with the same average log-likelihood, we select that which has maximal entropy. This, as usual, leads to the Gibbs distribution $`P\{𝒮\}e^{\beta H_c\{𝒮\}}`$ where the inverse temperature $`\beta `$ arises as a Lagrange multiplier.

The Hamiltonian $`H_c`$ depends implicitly on the Potts spins $`s_i`$ through the cluster variables $`n_s`$ and $`c_s`$ of Eq. (3). Unlike the Potts Hamiltonian $`H_q`$, the dependence on $`\delta _{s_i,s_j}`$ is non-linear and it is modulated by $`C_{i,j}`$. For $`s_is_j`$ for all $`ij`$ we have $`n_s=c_s=1`$ for all $`s`$ and hence $`H_c=0`$. This state is representative of the high temperature ($`\beta 0`$) limit. The low temperature physics of $`H_c`$ is instead non-trivially related to the correlation matrix $`C_{i,j}`$. Note, that the ferromagnetic state $`s_i=1`$ for each $`i`$, which dominates as $`\beta \mathrm{}`$ in clustering methods based on Potts models, is in general not the ground state of $`H_c`$. Intuitively we expect that, if the model of Eq. (2) is reasonable, $`H_c`$ should have a well defined ground state and low temperature phase which is energetically dominated by this state. In these cases, as in ref. , we expect a thermal phase transition.

The form of the Hamiltonian $`H_c`$ clearly depends on the Ansatz (2). For example if one takes a factor model for the correlations one finds a different Hamiltonian which depends on different variables, as discussed in the Appendix. Also note that the present model only describes positive correlations. It is straightforward to generalize this approach to the case where a sizeable number of matrix elements $`C_{i,j}`$ are negative and not small. The idea is to introduce spin variables $`\sigma _i=\pm 1`$ for each set and modify Eq. (2) by multiplying the right hand side by $`\sigma _i`$. This leads us to the analysis of a system where the Potts variables $`s_i`$ and the spin variables $`\sigma _i`$ interact. An account of this method shall be given elsewhere .

## III The data

We consider three different data sets, i.e. three different matrices $`C_{i,j}`$. For all of them we took $`N=443`$ time series of length $`D=1599`$. The results with shorter time series will also be discussed below.

The first data set refers to $`N`$ totally uncorrelated time series of length $`D`$. This is obtained, for example, from Eq. (2) with $`s_i=i`$. The second also is obtained from Eq. (2), but this time with preassigned structure $`𝒮^{}`$ and coupling strengths $`𝒢^{}`$. We shall discuss below how the specific structure was chosen. These two data sets help us to understand how the method performs when no structure at all is present and to check whether a predefined structure can be recovered.

Our third data set is made of financial time series of asset prices relative to the Standard & Poors 500 index (S&P500). More precisely $`\xi _i(d)`$ is the normalized daily returns of asset $`i`$ of the S&P500 index, in day $`d`$; this is defined as

$$\xi _i(d)=\frac{\mathrm{log}[p_i(d)/p_i(d1)]r_i}{\sigma _i},$$
(10)
where $`p_i(d)`$ is the price of asset $`i`$ in day $`d`$. The parameters $`r_i`$ and $`\sigma _i`$ are determined in order to have zero mean and unit variance for all $`i`$.

Correlation matrices of financial time series are of great practical interest. Indeed they are at the basis of risk minimization in the modern portfolio theory. This states that, in order to reduce risk, the investment needs to be diversified (i.e. divided) on many uncorrelated assets, so that price fluctuations are averaged out. However the measure of correlation in samples with a number of observation times comparable to the number of assets was recently found to be affected by considerable noise-dressing . For example the S&P500 is composed of $`N=500`$ assets and considering daily data from July 3rd 1989 to October 27th 1995 one has $`D=1600`$ data points for each asset. This data set is then an ideal instance of a problem where our method applies.

In addition, this data set has been studied by other authors with several methods including spectral analysis , minimal spanning tree  and super-paramagnetic clustering . This allows us to compare the results of our method with those of other methods. Finally, in order to better appreciate the performance of our method in a real application, we choose the synthetic correlated data set $`\{𝒮^{},𝒢^{}\}`$ as a “large likelihood” structure of the S&P500 data set. In other words we performed a simulated annealing experiment on the S&P500 data set, where the fictitious temperature $`1/\beta `$ was gradually decreased to $`0`$. This allowed us to compare how well the real S&P500 data set can be described by a maximum likelihood structure***The maximum likelihood structure may be computationally hard to find and simulated annealing may get trapped into a local minimum. Indeed in our case we found slightly different structures in different experiments. The structure $`𝒮^{}`$ was that with largest likelihood..

Figure 1 shows a comparison of the spectral properties of the three correlation matrices. The spectrum of eigenvalues for uncorrelated time series are known exactly. It extends over an interval of size $`N/D`$ around $`\lambda =1`$, as shown in Fig. 1. The spectrum of eigenvalues of the S&P500 correlation matrix has a similar shape for $`\lambda 1`$. This suggests that significant noise-dressing due to finite $`D`$ occurs. The tail of the distribution ($`\lambda 1`$) implies that some correlation is however present. Within our framework, large eigenvalues are associated with large clusters. Indeed the synthetic correlated data set has a broad distribution of cluster sizes (see Fig. 3) and a correspondingly fat tail in the distribution of eigenvalues.

## IV Clustering by Monte Carlo simulations

In order to study the properties of $`H_c`$ we resort to Monte Carlo (MC) method with Metropolis algorithm. This, at equilibration, allows us to sample the Gibbs distribution $`P\{𝒮\}`$ and compute average quantities, such as the internal energy $`E_\beta =H_c_\beta `$ where $`\mathrm{}_\beta `$ stands for thermal average. To detect the occurrence of spontaneous magnetization – which correspond to the $`s_i`$ remaining locked into energetically favorable configurations at low temperature – we measure the autocorrelation function

$$\chi (t,\tau )=\frac{_{i<j}\delta _{s_i(t),s_j(t)}\delta _{s_i(t+\tau ),s_j(t+\tau )}}{_{i<j}\delta _{s_i(t),s_j(t)}}.$$
(11)
This quantity tells us how many pairs of sites belonging to the same cluster at time $`t`$ are still found in the same cluster after $`\tau `$ MC steps. For $`t`$ large enough, $`\chi `$ becomes a function of $`\tau `$ only. This function decreases rapidly to a plateau value $`\chi _\beta =\chi (t,\tau )_\beta `$ for $`t\tau 1`$. Clearly $`\chi _\beta 0`$ implies that no persistent structure is present whereas, at the other extreme, $`\chi _\beta =1`$ implies that all sites are locked in a persistent structure of clusters.

We monitored these quantities for the three data sets. Let us start with a truly uncorrelated time series. We generate the time series $`\xi _i(d)`$ and then compute $`C_{i,j}`$ by Eq. (1). With this we compute the Hamiltonian $`H_c`$ and study its thermal properties by the MC method. We do not expect any clustering to emerge in this case. Indeed, the internal energy $`E_\beta `$ stays very close to $`0`$ (see Fig. 2a) for all values of $`\beta `$ investigated up to $`\beta =512`$. Correspondingly no persistent cluster arises, i.e. $`\chi _\beta 0`$.

The results change turning to correlated data. Let us first discuss the S&P500 data (for $`D=1599`$). As Fig. 2a shows, for $`\beta 20`$ the energy $`E_\beta `$ starts deviating significantly from zero. For $`\beta >20`$ persistent clusters are present: $`\chi _\beta `$ rapidly raises from zero and it has a maximum at $`\beta 40`$ (see Fig. 2b). The energy fluctuations reported in the inset shows a broad peak of intensity marking the onset of an ordered low temperature phase. As $`\beta `$ increases the dynamics is significantly slowed down. At $`\beta 200`$ the energy reaches a minimal value $`E_\beta 0.11N`$ and does not decrease significantly increasing $`\beta `$ at least up to $`\beta =4095`$. This energy is smaller than that of the ferromagnetic state ($`E_f=0.086N`$), with all sets in the same cluster. The system in this range of temperatures visits only few configurations.

The statistical properties of cluster configurations, as $`\beta `$ varies, are shown in Fig. 3. For small $`\beta `$ only small clusters survive to thermal fluctuations. As $`\beta `$ increases a distribution of cluster sizes develops. At low temperatures the rank order plot of $`n_s`$ reveals a broad distribution of clusters with the largest aggregating more than $`190`$ sets. By a power law fit of this distribution, we find that the number of clusters with more than $`n`$ sets decays as $`n^{0.83}`$. The scatter plot of $`c_s`$ versus $`n_s`$ also reveals a non-trivial power law dependence $`c_sn_s^{1.66}`$. This gives a statistical characterization of the dominant configurations of clusters at low energy. The clusters structure we obtain is reasonable from the economic point of view: companies in the same economic sector belong to the same cluster. These issues will be discussed in detail elsewhere. Here we restrict our attention to the clustering method.

It is instructive to compare these results with those obtained form shorter time series. We performed a second set of simulations with $`C_{i,j}`$ computed using the quotes of the S&P500 assets for the last $`D=400`$ days. We find two inverse temperatures $`\beta _120`$ and $`\beta _280`$ which separates three regimes. This is signalled by the bending in the $`E_\beta `$ curve and by peaks in the $`\delta E_\beta ^2`$ vs $`\beta `$ plot. At the first temperature clusters start to appear. For $`\beta <\beta _2`$ the largest cluster groups less than $`30`$ sets and for $`\beta >\beta _2`$ larger clusters $`n_s100`$ appear. This hints at a time dependence of correlations, which are averaged in the $`D=1599`$ data set. For even shorter time series we found that sampling errors, acting like a temperature, destroy large clusters and only relatively small clusters ($`n_s<40`$ for $`D=60`$) were found.

Finally let us discuss the results for the synthetic correlated data set (for $`D=1599`$). As already mentioned, the structure $`𝒮^{}`$ is a typical low energy configuration for the S&P500 data set extracted from the previous simulations (with $`D=1599`$). The parameters $`g_s^{}`$ where deduced from the $`n_s`$ and $`c_s`$ of this configuration, via Eq. (8). We recall that this data set is useful for two reasons: first it allows one to understand to what extent a structure of correlation put by hand with the form dictated by Eq. (2) can be correctly recovered. Secondly it allows us to compare the results found for the S&P500 data with those of a time series with correlations described by Eq. (2).

For $`\beta <150`$, the behaviors of $`E_\beta `$, $`\delta E_\beta ^2`$ and $`\chi _\beta `$ are similar to those found for the S&P500 data (see Fig. 2). A second, sharp peak in $`\delta E_\beta ^2`$ at $`\beta 170`$ signals a new clustering transition. Below this temperature, as shown by the plot of $`\chi _\beta `$ (Fig. 2b), the MC dynamics freezes into the original structure $`𝒮^{}`$. The overlap with the configuration $`𝒮^{}`$, defined as in Eq. (11) as the fraction of “bonds” $`s_i=s_j`$ for which $`s_i^{}=s_j^{}`$, quickly converges to $`1`$ (see Fig. 2b) for the synthetic time series, whereas it remains around $`60\%`$ for the S&P500 data set. This, on one hand means that the original structure $`𝒮^{}`$ can be recovered quite efficiently. On the other hand, it suggests that several cluster configuration compete at low temperatures in the S&P500 data set.

## V Mean Field model

In this section we would like to determine the nature of the clustering transition that takes place in our system. We apply our method to an unrealistically simple situation that will allow us to extract analytical expressions for the order parameter associated with the phase transition. Our analysis is rather similar to that in .

We take $`N`$ time series that belong to $`M`$ clusters of the same size $`n=N/M`$. Let $`\overline{s}_i`$ be the cluster index of the $`i^{\mathrm{th}}`$ time series , and let

$$C_{i,j}=\gamma \delta _{\overline{s}_i,\overline{s}_j}+(1\gamma )\delta _{i,j}$$
(12)
be the correlation matrix for $`D=\mathrm{}`$. This means that time series with $`\overline{s}_i=\overline{s}_j`$ have correlation $`\gamma `$ whereas $`\overline{s}_i\overline{s}_j`$ have $`C_{i,j}=0`$. The matrix $`C_{i,j}`$ has $`N/M`$ blocks of size $`M`$ along the diagonal and is zero elsewhere. A finite $`D`$ sample of this problem is generated with Eq. (2) with $`g_s=\gamma /(1\gamma )`$ and $`s_i=\overline{s}_i`$.

To fix ideas we can imagine to have the problem of putting $`N`$ balls of $`M`$ different colors in $`M`$ boxes. Colors represent the original structure $`\overline{s}_i`$ whereas boxes represent the actual clustering configuration $`s_i`$. When the balls contained in each box have the same color, the original cluster structure has been recovered. Let $`m_s^c`$ be the number of balls of color $`c`$ in box (or cluster) $`s`$. Now $`_sm_s^c=n=N/M`$ is the overall number of balls of color $`c`$, assumed equal for all colors, and $`_cm_s^c=n_s`$ is the number of balls in box $`s`$, as in Eq. (3). With the above choice of parameters the *internal correlation* of box $`s`$ for a given configuration $`\{m_s^c\}`$ of clusters is

$$c_s=(1\gamma )n_s+\gamma \underset{c}{}m_{s}^{c}{}_{}{}^{2}.$$
(13)
To compute the free energy $`F=UTS`$ of the system we use the energy $`H_c`$ as in Eq. (9) and we estimate the configuration entropy in the following way: The number of ways in which one can distribute the balls of color $`c`$ by putting $`m_s^c`$ of them in box $`s`$ is

$$\frac{(_sm_s^c)!}{_s(m_s^c!)}=\frac{n!}{_s(m_s^c!)},$$
(14)
and the total number of configurations for all the colors is the product over $`c`$ of this expression. Taking the logarithm of this product we obtain the configuration entropy

$$S=\mathrm{log}\left(\underset{c}{}\frac{(_sm_s^c)!}{_s(m_s^c!)}\right)=\underset{s,c}{}m_s^c\mathrm{log}\left(\frac{m_s^c}{n}\right),$$
(15)
where we have approximated in the usual way the logarithm of the factorial. Finally the free energy is:

$`F`$ $`=`$ $`{\displaystyle \underset{s}{}}F_s`$ (16)
$`F_s`$ $`=`$ $`{\displaystyle \frac{1}{2}}\left[\mathrm{log}{\displaystyle \frac{c_s}{n}}+(n1)\mathrm{log}({\displaystyle \frac{n^2c_s}{n^2n}})\right]+`$ (18)
$`+T{\displaystyle \underset{c}{}}m_s^c\mathrm{log}\left({\displaystyle \frac{m_s^c}{n}}\right).`$
After substitution of Eq. (13) we find an expression which depends on the occupation variables $`m_s^c`$. The occupation in different boxes are related by the overall constraints $`_sm_s^c=N/M`$. We take the mean-field approximation, which is legitimate in cases like this, where we neglect these effects. In other words, we minimize each of the $`F_s`$ independently and we suppress therefore the subscript $`s`$ from now on.

We can then focus on just one box and look for solutions of the form

$$m^c=\frac{N}{M}\{\begin{array}{cc}\varphi \hfill & c=1\hfill \\ \frac{1\varphi }{M1}\hfill & c1\hfill \end{array},$$
(19)
with $`0\varphi 1`$. In this Ansatz, the balls of color $`c=1`$ are more (or less) numerous that those of other colors $`c>1`$. In the spirit of mean-field approximation, we neglect the possibility that the number of balls of colors $`c>1`$ may be unevenly distributed. Hence the parameter $`\varphi `$ plays the role of the order parameter.

The paramagnetic solution $`\varphi =1/M`$, which corresponds to an uniform distribution of colors, is always a solution of the saddle point equations $`\frac{F}{\varphi }=0`$. This state is expected to be stable (the minimum of $`F`$) at high temperature. A second solution of $`\frac{F}{\varphi }=0`$, which corresponds to the clustered “ferromagnetic” state, appears at intermediate temperatures with $`\varphi 1`$. For $`T=T_c`$ the values of the free energy corresponding to the two states are equal and a first order phase transition to a ferromagnetic state takes place. The order of the transition is independent of the values of the parameters, while the critical temperature is determined mainly by the number of time series $`N`$.

We checked that this result is compatible with that obtained from Monte Carlo simulations. We find that the mean-field approach provides a good qualitative picture of the transition and a reliable estimate of the critical temperature at which it takes place (see Fig. 4).

## VI Noise Undressing

Eq. (2) with a single cluster configuration ($`\beta \mathrm{}`$), is inadequate to capture the full complexity of the correlations in the S&P500 data set. Probabilistic clustering, where several cluster structures $`𝒮`$ are allowed with their Gibbs probability $`P\{𝒮\}`$, provides an alternative approximation. In this approach the parameter $`\beta `$ can be tuned to determine the optimal spread in configuration space, which best describes the correlation structure built in the original data set. This line of reasoning will lead us to a method to “fit” the correlation structure of a data set with a single parameter $`\beta `$. This will finally allow us to undress the correlation matrix $`C_{i,j}(D)`$ of its noise-dressing and to reveal the bare correlations.

Let us start by remarking that the problem with Eq. (2) is that it stipulates that a set $`i`$ can belong to only one cluster. This suggests to consider the generalized model

$$\xi _i(d)=\frac{_s\sqrt{g_{s,i}}\eta _s(d)+ϵ_i(d)}{\sqrt{1+_sg_{s,i}}},$$
(20)
where each set $`i`$ can belong to any cluster $`s`$. Eq. (20) has, on the other side, the disadvantage that it depends on too many variables, and it leads to overfitting stochastic fluctuations.

The finite temperature distribution $`P\{𝒮\}`$ provides a natural way out of this situation. Indeed at finite $`\beta `$ each set $`i`$ visits different clusters $`s`$ and we can define

$$g_{s,i}(\beta )=\frac{c_sn_s}{n_s^2c_s}\delta _{s,s_i}_\beta .$$
(21)
The parameters $`g_{s,i}(\beta )`$ can be measured in a MC simulation and provide us with a measure of the strength of the correlation between set $`i`$ and cluster $`s`$.

These parameters and Eq. (20) also allow us to generate synthetic data sets $`\stackrel{~}{\xi }_i(d)`$, whose statistical properties can be compared to those of the original data set. We make this comparison using the spectral properties of the correlation matrix. In other words, with Eq. (1) and $`\stackrel{~}{\xi }_i(d)`$ we compute a “$`\beta `$-synthetic” correlation matrix; we determine the spectrum of eigenvalues and compare it to that of the original matrix. The parameter $`\beta `$ can be tuned in order to get the best fit.

This procedure was carried out for the S&P500 data set. The eigenvalue spectra of the two matrices are compared in Fig. 5 for $`\beta =48`$. The value of $`\beta `$ was chosen by visual inspection as that giving the best fit. The curves are remarkably close, suggesting that Eq. (20) provides a good statistical description of the correlations among assets.

Once the value $`\beta ^{}`$ which gives the best fit is found, we can compute the noise undressed correlation matrix

$$C_{i,j}^{}(\mathrm{})=\frac{\delta _{i,j}+_s\sqrt{g_{s,i}^{}g_{s,j}^{}}}{\sqrt{1+_sg_{s,i}^{}}\sqrt{1+_sg_{s,j}^{}}}$$
(22)
from the parameters $`g_{s,i}^{}=g_{s,i}(\beta ^{})`$. This is the correlation matrix of a synthetic data set obtained from Eq. (20) with $`D\mathrm{}`$. Fig. 5 shows the eigenvalue distribution of the noise undressed matrix $`C_{i,j}^{}(\mathrm{})`$. This allows one to appreciate the effect of noise dressing. As expected, noise mainly affects small eigenvalues.

## VII Discussion and Conclusions

The applicability of the method can be extended considerably to a generic data set $`\{\stackrel{}{x}_i\}_{i=1}^N`$. $`\stackrel{}{x}_i`$ need not be a time series. The distribution of $`x_i(d)`$ need not be Gaussian and it does not even need to be the same across $`i`$. For example, $`x_i(d)`$ may be the measure of the $`d^{\mathrm{th}}`$ feature of the $`i^{\mathrm{th}}`$ object or the concentration of species $`i`$ in the $`d^{\mathrm{th}}`$ sample of an experiment. The idea is to map the data set $`\stackrel{}{x}_i`$ into a Gaussian time series $`\stackrel{}{\xi }_i`$ to which we apply Eq. (2). The mapping results from requiring that non-parametric cross-correlations $`\tau _{i,j}^x=\tau _{i,j}^\xi `$ are preserved. To do this in practice we compute Kendall’s $`\tau `$  for the $`\stackrel{}{x}_i`$ data sets: $`\tau _{i,j}^x=\text{sign}[x_i(d)x_i(d^{})]\text{sign}[x_j(d)x_j(d^{})]_{d<d^{}}`$. For two Gaussian time series with correlation $`C_{i,j}`$ one can compute analytically $`\tau _{i,j}^x`$ in the limit $`D\mathrm{}`$. This leads to the relation

$$C_{i,j}=\frac{\mathrm{tan}(\pi \tau _{i,j}^x/2)}{\sqrt{1+\mathrm{tan}^2(\pi \tau _{i,j}^x/2)}}$$
(23)
between Gaussian and non-parametric correlations. This equation allows us to translated non-parametric correlations into Gaussian correlations. From these one can build $`H_c`$ of Eq. (9) and study the clustering properties.

This procedure has been tested for the S&P500 data set, for which it is known that $`\xi _i(d)`$ has non-Gaussian statistics . We have found indistinguishable results which indicate that the deviation from Gaussian behavior have little or no effect on the results. We expect that this approach breaks down when the marginal distribution of $`\xi _i(d)`$ is such that the second moment is not defined. In that case $`C_{i,j}`$ computed from $`\tau _{i,j}`$ and Eq. (23) can even fail to be positive definite.

With respect to ref. , our approach does not need any assumption on the form of the Hamiltonian. As input, the method only needs the correlation matrix $`C_{i,j}`$ (or $`\tau _{i,j}`$). The range of interactions is set by the correlations themselves. Indeed our method predicts a non-trivial ground state $`𝒮_0`$ which is not, in general, the ferromagnetic one.

For small $`D`$, the local interaction of ref.  may well be more efficient in capturing the structure of data. Our method is most useful in cases where $`DN1`$. These ideas can clearly be extended to models of correlations different from Eq. (2) as shown, for example, in the Appendix.

We acknowledge R. Zecchina, R. Pastor-Satorras and D. Vergni for interesting discussions and R. N. Mantegna for providing the S&P500 data.

## A

We define here the explicative factor model for stocks returns (also known as multi index model, see e. g. ):

$$\xi _i(t)=\frac{\stackrel{}{v}_i\stackrel{}{\eta }(t)+ϵ_i}{\sqrt{1+v_i^2}},v_i^2=\stackrel{}{v}_i\stackrel{}{v}_i=\underset{\alpha =1}{\overset{L}{}}(v_i^\alpha )^2$$
(A1)
where $`\stackrel{}{v}_i`$ are $`L`$ dimensional vectors and $`\stackrel{}{\eta }(t)`$ is a $`L`$ dimensional Gaussian random vector $`\eta ^\alpha (t)=0`$ and $`\eta ^\alpha (t)\eta ^\beta (t^{})=\delta _{\alpha ,\beta }\delta _{t,t^{}}`$.

The idea is that there are $`L`$ explicative factors $`\eta ^1(t),\mathrm{},\eta ^L(t)`$ which describe the fluctuations of each stock price. This model is different from the one we considered in the text in that each time series is coupled to all other with a different strength. This can be easily understood by observing that the model (2) can be cast in the form of an explicative factor model with $`v_i^\alpha =g_{s_i}\delta _{\alpha ,s_i}`$. This is a rather particular form of Eq. (A1). We observe, however, that $`L`$ must be much smaller than $`N`$ in order to avoid problems of overfitting with Eq. (A1), while Eq. (2) requires $`LN`$.

As we did in Sect. II we look at the probability of observing the time series $`\xi _i(t)`$ given the model Eq. (A1) and the parameters $`\stackrel{}{v}_i`$:

$$P\{\xi _i(t)|\stackrel{}{v}_i\}=\underset{t=1}{\overset{D}{}}\underset{i=1}{\overset{N}{}}\delta \left(\xi _i(t)\frac{\stackrel{}{v}_i\stackrel{}{\eta }+ϵ_i}{\sqrt{1+v_i^2}}\right)_{\stackrel{}{\eta },ϵ}.$$
(A2)
After taking the average over the Gaussian variables one obtains the equivalent of Eqs. (7) and (5)

$`P\{\xi _i(t)|\stackrel{}{v}_i\}=e^{DH\{\stackrel{}{v}\}}`$

$`H\{\stackrel{}{v}\}`$ $`=`$ $`{\displaystyle \frac{1}{2}}{\displaystyle \underset{i}{}}\left[(1+v_i^2)\mathrm{log}(1+v_i^2)\right]`$ (A4)
$`{\displaystyle \frac{1}{2}}\text{Tr}\mathrm{log}(1+V){\displaystyle \frac{1}{2}}\text{Tr}{\displaystyle \frac{\chi }{1+V}},`$
where we have defined the matrices

$`V_{\alpha ,\beta }`$ $`=`$ $`{\displaystyle \underset{i=1}{\overset{N}{}}}v_i^\alpha v_i^\beta `$ (A5)
$`\chi _{\alpha ,\beta }`$ $`=`$ $`{\displaystyle \underset{i,j=1}{\overset{N}{}}}C_{i,j}\sqrt{1+v_i^2}v_i^\alpha \sqrt{1+v_j^2}v_j^\beta ,`$ (A6)
and $`C_{i,j}`$ is defined in Eq. (1). We note that the second term in Eq. (A4) is sub-extensive, and could be neglected; nevertheless in the presence of the matrix $`\chi `$ a Monte Carlo simulation becomes excessively time consuming, since a change in $`\stackrel{}{v}_k`$ requires order $`N`$ operations to compute the new matrix. This may considerably limit the practical applicability of this method.