text:
# JLAB-THY-01-05 Evidence Against Instanton Dominance of Topological Charge Fluctuations in QCD

## I Introduction

In the late 1970’s it became clear through the study of instantons  that gauge field topology plays an important role in the resolution of the $`U_A(1)`$ problem: topological transitions could produce an $`\eta ^{}`$ mass via the axial anomaly. It is now quite well established from lattice QCD that topological charge is indeed important, with the $`\eta ^{}`$ mass quite nicely explained by its coupling to the $`U_A(1)`$ current anomaly and with topological charge playing a significant role in other parts of QCD as well .

In two remarkable 1979 papers , Witten argued that, while topological charge fluctuations were clearly involved, the dynamics underlying the $`\eta ^{}`$ mass need not be associated with the semiclassical tunnelling events called instantons since the large vacuum fluctuations associated with confinement would also necessarily entail large fluctuations of topological charge. In fact Witten argued that not only are instantons not required for resolving the $`U_A(1)`$ problem, but that the instanton resolution is in conflict with predictions based on the large-$`N_c`$ approximation. Instantons would produce an $`\eta ^{}`$ mass that vanishes exponentially for large $`N_c`$, while considerations based on large-$`N_c`$ chiral dynamics strongly suggest that the $`\eta ^{}`$ mass should be of order $`1/N_c`$. Given the incompatibility of large-$`N_c`$ dynamics and instantons, and the strong empirical support for the applicability of the large-$`N_c`$ limit, Witten speculated that the true dynamical origin of the $`\eta ^{}`$ mass would be the coupling of the $`U_A(1)`$ anomaly to topological charge associated with confinement-related vacuum fluctuations and not instantons.

From this point of view, the particular kind of quantized, locally self-dual or anti-self-dual gauge fluctuations associated with an instanton gas or liquid model of the QCD vacuum are only a small subset of all the configurations containing significant fluctuations of topological charge. Although instantons are favored by the Yang-Mills action, apparently the much greater entropy of the many non-instanton configurations causes them to dominate the QCD path integral. As a result, the instanton expansion is invalidated, presumably by large quantum fluctuations associated with confinement. A very analogous situation in the two-dimensional CP(N-1) model is cited by Witten as further evidence that even a theory with classical instanton solutions can have a quantum mechanical ground state which is not even approximately described by an instanton expansion . Witten’s arguments are also consistent with a view of the QCD vacuum based on the strong-coupling expansion in which the gauge fields are highly disordered. This disorder leads very directly to confinement, as seen from the area law exhibited by large Wilson loops. The semiclassical gauge configurations associated with an instanton picture are highly ordered and quite atypical in a strong-coupling framework.

In this paper we investigate the prevalence of instantons in the QCD vacuum using numerical lattice QCD. In the absence of analytic control in the continuum, this is the most direct way of resolving this nonperturbative issue from first principles. To achieve this it is necessary to design a test that (a) is capable of distinguishing the typical vacuum configurations implied by the instanton picture from the bulk of generic configurations containing topological charge fluctuations, and (b) can lead to a meaningful result using available lattice techniques. With respect to (a) we propose to test the local (anti)self-duality properties of the gauge fields in the regions where the field strength is large. Indeed, it is inherent to an instanton picture of the gauge vacuum (such as the Instanton Liquid Model (ILM) ), that strong fields are concentrated in small regions of space-time, with the fields being locally (anti)self-dual. Checking gauge field duality properties directly on Monte-Carlo generated lattice gauge configurations is problematic because, as is frequently pointed out, the short distance fluctuations associated with lattice artifacts can mask the expected continuum-like behavior. To avoid artificially changing the configuration in some type of smoothing or cooling procedure, we use instead fermionic methods. In particular, we will argue that local self-duality properties of gauge fields are encoded in the local chirality properties of low-lying Dirac eigenmodes. The main point of this paper is to present evidence that the local chiral structure of low-lying Wilson-Dirac eigenmodes is not consistent with instanton dominance.

The gauge configurations used in this study are a subset of the Fermilab ACPMAPS b-ensemble recently used in extensive calculations  which confirmed in considerable detail the chiral properties expected from continuum chiral Lagrangian arguments. These included the magnitude and time-dependence of the $`\eta ^{}`$ hairpin correlator, the value of the pure-glue topological susceptibility, and a number of “quenched chiral logarithm” effects associated with the $`\eta ^{}`$. Since these previously reported results also probe the chiral structure of low-lying Dirac eigenmodes, they provide us with reasonable confidence that the qualitative conclusions reported here represent the behavior of continuum QCD.

Given the conclusion that instantons are not the dominant source of topological charge fluctuations, how does this affect our picture of low energy QCD? Instantons are usually invoked in qualitative explanations of phenomena associated with topological charge. However, as was explained by Witten , these physical effects do not necessarily require instantons, and can be qualitatively understood in any picture involving confinement. (Note that it has long been recognized that instantons alone are not likely to lead to confinement, so it is natural to suspect that their relevance to low energy QCD is limited.) An important implication of our result is that it calls into question phenomenological approaches based on semiclassical methods in favor of approachs where confinement plays a central role. In contrast to the situation for instantons, there is no all-encompassing confinement-based framework comparable to the ILM, providing a physics picture for how the key low energy phenomena of QCD are driven by confinement. However, in this paper we will refer to the new Strong Coupling Quark Model  as possibly providing such a framework. In his discussion of these matters, Witten used not only the generic properties of the confining vacuum, but also specific properties following from the large-$`N_c`$ expansion of QCD . The derivation of the Witten-Veneziano formula  for the $`\eta ^{}`$ mass serves as an example of how the large-$`N_c`$ limit, can be used as a tool in quantitative calculations relevant to these questions.

In fact, the success of the large-$`N_c`$ expansion is in itself evidence for a non-instanton picture of the QCD vacuum. In this regard, we remind the reader that the successes of large $`N_c`$ are by now very broad and include a basis for the valence quark model , Regge theory , the utility of the narrow resonance approximation, quasi-two-body dominance of hadronic decays, the OZI rule , and the general systematics of hadronic spectra and matrix elements . Also, a growing body of lattice results appear to confirm the empirical evidence that the large-$`N_c`$ expansion is good, i.e. that $`1/N_c=1/3`$ is reasonably close to $`1/N_c=0`$. Most directly relevant to the $`\eta ^{}`$ mass problem are the lattice results showing that the topological suceptibilities $`\chi _t`$ at $`N_c=2`$ and $`N_c=3`$ are almost equal within errors. With the string tension $`\sigma `$ used to set the scale, it is found  that $`\chi _t^{\frac{1}{4}}/\sqrt{\sigma }=0.455\pm 0.015`$ in $`SU(3)_c`$ while this same dimensionless ratio takes the value $`0.487\pm 0.012`$ in $`SU(2)_c`$. Finally, it should be pointed out that Witten has recently provided new evidence in favor of his conjecture on the purely theoretical front . Using the Anti-de Sitter space-conformal field theory (AdS/CFT) connection which has emerged from superstring theory, and assuming that there are no phase transitions as a certain parameter $`\eta `$ varies between the region $`\eta 0`$ (where it approaches 4D gauge theory) and $`\eta >>1`$ (where it becomes a weakly coupled string theory which can be studied as a long wavelength supergravity theory), Witten showed that the $`\theta `$-dependence of $`SU(N_c)`$ gauge theory is of leading order in $`N_c`$ as $`N_c\mathrm{}`$, as anticipated . In an instanton picture, the $`\theta `$-dependence would vanish exponentially. The AdS/CFT correspondence also reveals the existence of stable nondegenerate vacua and nonanalytic $`\theta `$-dependence of the vacuum energy in the large $`N_c`$ limit, confirming other aspects of Witten’s no-instanton hypothesis.

In Section II we begin by introducing the reader to the circle of ideas that lead Witten to conclude that at the quantum level, instantons might simply “disappear” from QCD . This includes an elementary discussion of how the physical effects usually ascribed to instantons naturally occur in a confining vacuum even without instantons. In Section III we discuss the relation between Dirac near-zero modes, the chiral condensate, and the $`\eta ^{}`$ mass. Sections II and III both have a pedagogical character and serve as theoretical background for Section IV, where we present the main results of the paper. There we show how an instanton-dominated vacuum would be reflected in the local chiral structure of near-zero modes, describe our lattice methods, and present the results. Finally, Section V summarizes our conclusions.

## II Witten’s Conjecture

### A The Vulnerability of QCD Instantons

To illustrate how large quantum corrections could make instantons irrelevant (“evaporate”) in QCD, Witten considered two $`U(1)`$ gauge theories in $`1+1`$ dimensions, namely

$$_1=D_\mu \varphi ^{}D_\mu \varphi \mu ^2\varphi ^{}\varphi \frac{1}{4}F_{\mu \nu }F_{\mu \nu }$$
(1)
$$_2=D_\mu \varphi ^{}D_\mu \varphi \lambda (\varphi ^{}\varphi a^2)^2\frac{1}{4}F_{\mu \nu }F_{\mu \nu },$$
(2)
where $`D_\mu _\mu +ieA_\mu `$, $`\varphi `$ is a charge scalar field, $`F_{\mu \nu }=_\mu A_\nu _\nu A_\nu `$, and the theories are defined in an infinite volume. The first theory has an unbroken $`U(1)`$ symmetry and a spectrum of neutral $`\varphi \overline{\varphi }`$ bound states (since in one dimension the Coulomb potential is a linear confining one). The second has a characteristic Higgs spectrum with a real physical scalar and a massive “photon”. Both $`_1`$ and $`_2`$ possess the topological charge $`Q=\frac{e}{2\pi }d^2xϵ_{\mu \nu }F_{\mu \nu }`$; in $`_2`$ it is quantized to integer values and an instanton gas plays an important role, while in $`_1`$ it is not quantized and instantons play no role. Of course, even in theory (1) there is a global quantization of Euclidean topological charge on a finite periodic torus, but in this case the quantization is strictly a finite volume effect (analogous to the quantization of momentum in a finite box) and has no relevance to the local structure of the vacuum.

Another distinguishing feature of the non-instanton theory is in the dependence of physics on the vacuum parameter $`\theta `$ (Fourier conjugate to the topological charge). For the spontaneously broken theory (2), $`\theta `$-dependence arises via an instanton expansion and physical quantities are smooth analytic functions of $`\mathrm{cos}\theta `$, while for the unbroken theory (1), $`\theta `$ represents a background electric field, and physical quantities (e.g., the vacuum energy) are singular at certain values of $`\theta `$, corresponding to the physical effect of a $`\varphi ^+\varphi ^{}`$ pair popping out of the vacuum and screening and cancelling a unit of electric flux. Witten’s recent consideration of this issue in 4D gauge theory via the AdS/CFT correspondence  has demonstrated that a similar nonanalytic behavior of the vacuum energy as a function of $`\theta `$ arises in the large $`N_c`$ limit of QCD (with wrapped sixbranes playing a role analogous to units of electric flux in the 2D case). Thus, at least at large $`N_c`$, the $`\theta `$ dependence of four-dimensional $`SU(N_c)`$ gauge theory resembles unbroken 2D scalar electrodynamics (1), and not the instanton dominated Higgs theory (2).

Although it is not believed to happen in two-dimensional scalar electrodynamics, it is easily imaginable that a theory with classical instantons, like $`_2`$, could be converted into a theory like $`_1`$ by quantum corrections, if those corrections changed the broken Higgs-type effective potential to an unbroken one. Given the apparent incompatibility of instanton and large-$`N_c`$ ideas and the strong phenomenological evidence for the latter, Witten conjectures that, in spite of the fact that QCD at the classical level resembles a theory of type $`_2`$, quantum effects most likely change it into a theory of type $`_1`$. To illustrate this view more convincingly, he then argues in detail that precisely this kind of behaviour occurs in the two-dimensional CP(N-1) model. This model has a global SU(N) symmetry and, like QCD, is asymptotically free and classically scale invariant. The condition of finite action leads to a field boundary condition at infinity that breaks the SU(N) symmetry, and results in a quantized topological charge at the classical level. There are instantons for all N, but the solution of the theory in terms of a 1/N expansion reveals that instantons in fact evaporate . This can be interpreted as invalidation, by large quantum corrections, of the boundary condition that breaks the SU(N) symmetry. In the true vacuum, the symmetry is actually restored.

At the classical level, the situation in QCD is quite analogous to that of a CP(N-1) model. Furthermore, Witten argues, the phenomenon of confinement in QCD plays an analogous role to that of an SU(N) symmetry restoration in the CP(N-1) case. Indeed, the standard semiclassical condition of finite action results in the requirement that the field is pure gauge at infinity. However, with the assumption of confinement, this condition is not reasonable in QCD. The correct boundary condition at infinity should reflect the behaviour that is typical of a confining vacuum, while the pure gauge behaviour corresponds to a perturbative, non-confining vacuum. This suggests that the QCD path integral is dominated by configurations which invalidate the semiclassically motivated pure gauge boundary condition, and that the instantons “evaporate” due to large quantum corrections.

### B Low Energy QCD without Instantons

An important role in the qualitative arguments of Subsection II.A is clearly played by the assumption of confinement. While the structure of the confining vacuum is not fully understood, Witten argued that large gauge field fluctuations, generically present in the confining vacuum, are sufficient for a qualitative explanation of the physical effects usually associated with instantons. This is quite easily seen, since many of these effects, including the $`\theta `$ dependence of physics and the solution of the $`U_A(1)`$ problem, depend only on the fact that, for the configurations typical of the QCD vacuum, topological charge fluctuations can be large (or, more precisely, that the topological susceptibility of the pure-glue vacuum is nonzero). The Euclidean topological charge is defined as

$$Q\frac{1}{32\pi ^2}d^4x\frac{1}{2}ϵ_{\mu \nu \rho \sigma }G_{\mu \nu }^aG_{\rho \sigma }^a\frac{1}{32\pi ^2}d^4xG_{\mu \nu }^a\stackrel{~}{G}_{\mu \nu }^a$$
(3)
where $`G_{\mu \nu }^a=_\mu A_\nu ^a_\nu A_\mu ^a+f^{abc}A_\mu ^bA_\nu ^c`$. A basic property of $`Q`$ is that its density is a total divergence and thus it can be represented as an integral over the large hypersphere

$$Q=d^4x_\mu K_\mu =𝑑S_\mu K_\mu $$
(4)
with $`dS_\mu `$ the surface element, and

$$K_\mu =\frac{1}{16\pi ^2}ϵ_{\mu \nu \rho \sigma }\left(A_\nu ^a_\rho A_\sigma ^a+\frac{1}{3}f^{abc}A_\nu ^aA_\rho ^bA_\sigma ^c\right)$$
(5)
the topological current.

Let us now consider the $`\theta `$ dependence of physics. Before the discovery of instantons, the inclusion of the perfectly acceptable term $`S^{}\theta d^4xG_{\mu \nu }^a\stackrel{~}{G}_{\mu \nu }^a`$ into the gauge action was believed to have no effect on physical observables, since, according to (4) it is just a surface term. The behaviour of fields at infinity should reflect the “typical behaviour in the vacuum” which, according to usual (perturbative) lore inherited from QED, corresponds to physical fields being zero (pure gauge). Hence, it appeared that there was no reason to believe that the $`\theta `$ term would contribute, and $`S^{}`$ was discarded.

The discovery of instantons revealed that even if one retains the picture of the vacuum as asymptotically pure gauge, the integral $`d^4xG_{\mu \nu }^a\stackrel{~}{G}_{\mu \nu }^a`$ does not necessarily have to vanish. Indeed, the BPST instanton solution , even though pure gauge at infinity, winds nontrivially around the group and yields a nonzero value for the integral ($`Q=1`$). More generally, any gauge configuration that is pure gauge at infinity (not just solutions of the equation of motion) and winds nontrivially around the group will have a nonzero, integer value of Q. Since the number of windings is stable under a continuous deformation of gauge fields, this pointed to the role of topology in QCD and, more importantly lead to the fundamental discovery that physics actually will depend on $`\theta `$ (except in the presence of massless fermions).

At the same time, from the confinement point of view, $`\theta `$ dependence is rather obvious. If confinement is assumed, one simply should not describe the vacuum as being asymptotically pure gauge. Consequently, the correct boundary condition at infinity should involve a nonzero $`G_{\mu \nu }^a`$ in some form. For such configurations there is absolutely no reason why $`d^4xG_{\mu \nu }^a\stackrel{~}{G}_{\mu \nu }^a`$ should vanish, hence the $`\theta `$ dependence. A concrete model which illustrates that instantons do not have to play any significant role in this respect is the Schwinger model, which is a theory of type (1) without quantized topological charge. The $`\theta `$ term corresponds to a background electric field and the physics depends on it, except when fermions are massless.

By the same argument, confinement also provides an alternative mechanism for $`\eta ^{}`$ mass generation. The flavor singlet axial current $`j_5^\mu \mathrm{\Sigma }_\alpha \overline{\psi }_\alpha \gamma ^\mu \gamma _5\psi _\alpha `$ ($`\alpha `$ being the flavour index) exhibits an axial anomaly, but the corresponding divergence is proportional to the topological charge density, $`_\mu j_5^\mu G_{\mu \nu }^a\stackrel{~}{G}_{\mu \nu }^a`$. Consequently, one can use the topological current (5) to define a conserved, but gauge variant current. To solve the $`U_A(1)`$ problem, it is necessary to show that this conserved current does not lead to a Goldstone pole , so that the $`\eta ^{}`$ remains massive in the chiral limit, unlike the other pseudoscalar mesons. t’Hooft has shown  that in the semiclassical approximation, using the instanton gas picture, this is indeed what happens. At the same time, as Witten argues, this conclusion only relies on the fact that the semiclassical framework naturally emphasizes configurations that give nonzero $`d^4xG_{\mu \nu }^a\stackrel{~}{G}_{\mu \nu }^a`$. While it is interesting that the $`U_A(1)`$ problem is solved already at the semiclassical level, the fluctuations of a confining vacuum, as we noted above, will also naturally include configurations with nonvanishing topological charge. As a result, the $`U_A(1)`$ problem is expected to be solved in any confinement-based picture of the QCD vacuum. This is exhibited, for example, by the derivation of the Witten-Veneziano formula  for the $`\eta ^{}`$ mass, which is carried out in the large $`N_c`$ framework without reference to semiclassical arguments.

### C Discussion

Early attempts at studying topological effects in QCD were framed in the context of a dilute instanton gas. While it is now well known that this framework is not capable of providing an accurate description of low energy QCD, the associated basic picture of the vacuum has survived in more elaborate scenarios based on instanton methods, such as the Instanton Liquid Model . In particular, the typical vacuum gauge configuration is imagined as being very inhomogeneous, containing lumps of very strong (anti)self-dual fields, surrounded by regions of very weak fields. When the non-interacting instanton gas picture is refined to include correlations, one can no longer speak of first-principles semiclassical calculations, but rather of a semiclassically-motivated phenomenology. An extensive examination of the possible roles of instantons in low-energy QCD (including their role in spontaneous chiral symmetry breaking and the propagation of light quarks in the vacuum) has been made in the context of the ILM . Ref.  also provides an excellent overview of the history of instantons and some very beautiful pedagogical discussion of low energy QCD. For other recent important work on instantons, see .

To further motivate our conclusion that such models are not viable, we note that from the confinement point of view, the instanton picture is unnaturally ordered. Indeed, with the (anti)self-dual restriction entailed by instanton excitations, the chromoelectric and chromomagnetic fields are locked together $`(𝐄^𝐚=\pm 𝐁^𝐚)`$. The important property of this condition is that the role of topological charge is maximally emphasized in the sense that for a given value of the action density, (anti)self-duality implies the maximal topological charge density. This can be seen from the fact that $`S(𝐄𝐄+𝐁𝐁)`$, while $`Q𝐄𝐁`$. The resulting lumpiness of topological charge then affects the propagation of quarks. While the large value of topological susceptibility in the pure gauge vacuum obtained from lattice simulations suggests that topological charge is indeed lumpy , this lumpiness is perfectly natural for a highly fluctuating confining vacuum as well: confinement automatically entails large localized fluctuations of $`𝐄^𝐚`$ and $`𝐁^𝐚`$ with energy densities of the order of $`1GeV/fm^3`$, generating “hot spots” in both $`(𝐄𝐄+𝐁𝐁)`$ and $`𝐄𝐁`$. What is not natural from the confinement point of view is the maximally ordered situation implied by the instanton picture. There appears to be no good reason to expect that the confining vacuum would fluctuate in such a way as to only generate (anti)self-dual lumps. In the SCQM , these fluctuations are associated with virtual scalar glueballs mixed into the bare strong-coupling vacuum (leading to a strong space-time localization of the gauge field fluctuations) and a loosely correlated angle between $`𝐄^𝐚`$ and $`𝐁^𝐚`$ associated with the fields internal to the scalar glueball.

## III The Role of Fermionic Near-Zero Modes

Since the approach pursued in this paper relies on fermionic near-zero modes, we now discuss their role in determining the low energy properties of QCD. In particular, we will concentrate on spontaneous chiral symmetry breaking (s$`\chi `$SB) and the $`\eta ^{}`$ mass. The common origin of these phenomena is an attractive feature of the ILM, but we will argue below that the same connection will arise in any picture which incorporates large fluctuations of topological charge density.

We begin by recalling the derivation of the Banks-Casher relation  which clarifies the role of near-zero modes of the Dirac operator in forming the chiral condensate. (Although we will ultimately address these issues numerically, using the lattice Wilson-Dirac operator, we restrict our theoretical considerations to the continuum Dirac operator. The effects of lattice discretization will be discussed in the next Section in the context of the Monte Carlo results.) The spectral requirement for a nonvanishing chiral condensate is found by considering the eigenmode expansion of a scalar quark loop with mass $`m`$ in a background gauge field $`A`$ in Euclidean 4-space,

$$\overline{\psi }(x)\psi (x)_A=\mathrm{Tr}G(x,x)=\underset{i}{}\frac{\psi _i^{}(x)\psi _i(x)}{i\lambda _im}.$$
(6)
Here the $`\psi _i`$’s and $`\lambda _i`$’s are the eigenvectors and eigenvalues of the massless Dirac operator,

$$i\overline{)}D\psi _i=\lambda _i\psi _i$$
(7)
Because of the anticommutator $`\{\gamma ^5,\overline{)}D\}=0`$, nonzero eigenvalues come in chiral pairs $`\pm \lambda _i`$ with eigenvectors $`\psi _i`$ and $`\gamma ^5\psi _i`$, respectively. Symmetrizing the eigenvalue expansion (6) over chiral pairs, we have

$$\overline{\psi }(x)\psi (x)_A=\underset{\lambda _i>0}{}\left(\frac{2m}{\lambda _i^2+m^2}\right)\psi _i^{}(x)\psi _i(x)\frac{1}{m}\underset{\lambda _i=0}{}\psi _i^{}(x)\psi _i(x).$$
(8)
After averaging over gauge fields, the expectation value on the left hand side becomes translation invariant, so we may integrate over the 4-volume $`V`$ and use the fact that the eigenfunctions are normalized to $`\psi _i^{}\psi _id^4x=1`$ to get

$$\overline{\psi }\psi =\frac{1}{V}\underset{i}{}\frac{m}{\lambda _i^2+m^2}\frac{N_0}{mV}.$$
(9)
At this point it is important to take the volume to infinity before taking the chiral limit $`m0`$. If we were to take the chiral limit first, we would conclude that only the modes with exactly zero eigenvalue contribute to $`\overline{\psi }\psi `$. Instead, the infinite volume limit at small but finite mass replaces the sum by an integral over a density function $`\rho (\lambda )`$, which receives contributions from a large number of near-zero modes. In fact, the contribution of exact zero modes to the integral becomes negligible compared to the near-zero modes in the infinite volume limit. (For constant topological susceptibility, the typical number of exact zero modes $`N_0`$ grows like $`\sqrt{V}`$, while the total number of low-lying modes grows linearly with the volume.) As a result, only the density of near-zero modes is relevant for chiral symmetry breaking,

$$\overline{\psi }\psi =_0^{\mathrm{}}𝑑\lambda \rho (\lambda )\left(\frac{m}{\lambda ^2+m^2}\right)$$
(10)
In the chiral limit, this reduces to the Banks-Casher relation

$$\overline{\psi }\psi =\pi \rho (0)$$
(11)
where $`\rho (0)=lim_{\lambda 0}\rho (\lambda )`$. Thus, the requirement for spontaneous chiral symmetry breaking is that the density of near-zero modes $`\rho (0)`$ in a very large box must be finite. The number of low-lying modes needed to achieve this is of course much larger than what would be available for free fermions in four dimensions, where the density vanishes like $`\lambda ^3`$. In the instanton liquid model, the required excess of near-zero eigenmodes is posited to be supplied by the topological zero-modes of the instantons and anti-instantons, which mix and spread into a band of eigenvalues near zero. But a finite value of $`\rho (0)`$, and hence a chiral condensate, can arise under much more general circumstances. The instanton liquid model populates the vacuum with a finite density of gauge field “lumps” which act like attractive potentials in the quark eigenvalue problem. In this case, each lump supports one low-lying eigenmode, so the total number of low-lying modes grows linearly with the volume and the finite density of states required for $`s\chi SB`$ can be achieved. But note that the Atiyah-Singer index theorem  tells us that any gauge configuration with an overall topological charge (which is an integer in a periodic box, with or without instantons) will exhibit at least that number of zero eigenvalues in its Dirac spectrum. More generally, large gauge fluctuations of all types will act as attractive potentials in the Dirac operator (see Section IV), giving rise to low-lying “bound state” eigenmodes. Just as in the instanton model, for a given density of large gauge fluctuations we would expect the number of such modes to grow linearly with the volume. This argument shows that a finite density of near-zero Dirac eigenmodes, and hence $`s\chi SB`$, can be a result of the types of large gauge fluctuations which are implied by confinement and do not require any of the more specific restrictions of the instanton picture.

Given the finite density of low-lying Dirac eigenvalues implied by $`s\chi SB`$ and the Banks-Casher relation, we now want to consider the resolution of the $`U_A(1)`$ problem from the point of view of low Dirac eigenmodes. The most direct way to address this issue is to consider the eigenmode expansion for the pseudoscalar “double-hairpin”, i.e. the loop-loop correlator associated with the $`\eta ^{}`$ mass,

$$\mathrm{\Delta }_{dh}^{(A)}(x,y)=\mathrm{Tr}\gamma ^5G(x,x)\mathrm{Tr}\gamma ^5G(y,y)_A$$
(12)
As before, the eigenmode expansion represents the quark propagators in a particular background gauge configuration $`A`$. In the end, the expressions are averaged over an ensemble of gauge configurations. As in the derivation of the Banks-Casher relation, each loop can be symmetrized over the contribution of chiral pairs, giving

$$\mathrm{\Delta }_{dh}^{(A)}(x,y)=\underset{i,j}{}\left(\frac{m}{\lambda _i^2+m^2}\right)\left(\frac{m}{\lambda _j^2+m^2}\right)\psi _i^{}(x)\gamma ^5\psi _i(x)\psi _j^{}(y)\gamma ^5\psi _j(y)$$
(13)
Because of the two factors in parentheses, we see that, just like $`\overline{\psi }\psi `$, the pseudoscalar hairpin correlator is dominated by the low eigenmodes in the chiral limit $`m0`$. The size and space-time dependence of this correlator, after averaging over gauge configurations, can be estimated by chiral Lagrangian arguments. In either quenched or full QCD, the hairpin correlator will be large and will have a long range component in the chiral limit which falls off exponentially according to the pion mass. In quenched QCD, the hairpin correlator at zero 3-momentum falls off like $`te^{m_\pi t}`$, while in full QCD it falls off like $`e^{m_\pi t}`$ (cancelling the Goldstone pole of the valence propagator).

A recent study of the hairpin correlator , using the same (quenched) gauge configurations that we study here, has confirmed that both the size and time-dependence of the hairpin correlator are in excellent agreement with chiral Lagrangian predictions, in which the double hairpin vertex is treated as an $`\eta ^{}`$ mass insertion. The study in Ref.  uses a different method for computing the hairpin correlator (i.e. not an eigenmode expansion), but we take those results as strong evidence that the correct physical mechanism for $`\eta ^{}`$ mass generation, in particular the structure of the low eigenmodes which should dominate the result, is well represented by the lattice data. The interpretation of these results is discussed in both Ref.  and , where the $`\eta ^{}`$ double hairpin is an ordinary $`q\overline{q}q^{}\overline{q}^{}`$ OZI-violating diagram of the quark model.

Before discussing the lattice results of this paper, some further remarks on the eigenmode formula for the hairpin correlator (13) will be useful. A finite density of modes near $`\lambda 0`$ is already guaranteed by the existence of the chiral condensate. The additional property required to have a large hairpin correlator (and hence a finite $`\eta ^{}`$ mass) in the chiral limit is that the typical low-lying eigenmodes must have significant space-time regions in which the pseudoscalar charge $`\psi _i^{}\gamma ^5\psi _i`$ is large. Note that, in the Banks-Casher relation, the contribution of a given mode to the chiral condensate was fixed by the wave function normalization condition $`d^4x\psi _i^{}\psi _i=1`$. By contrast, the pseudoscalar charge for a mode $`\psi _i`$ with nonzero eigenvalue must integrate to zero,

$$d^4x\psi _i^{}\gamma ^5\psi _i=0$$
(14)
since $`\psi _i`$ and $`\gamma ^5\psi _i`$ have opposite eigenvalues of $`i\overline{)}D`$ and are therefore orthogonal to each other. What is required to produce a non-vanishing $`\eta ^{}`$ mass and resolve the $`U_A(1)`$ problem is for the low-lying eigenmodes to exhibit local regions in which $`\psi _i^{}\gamma ^5\psi _i`$ is large, even though the integrated quantity (14) vanishes. This does not happen, for example, for the eigenmodes of a free fermion in a periodic box, for which the pseudoscalar product vanishes locally, $`\psi _i^{}\gamma ^5\psi _i=0`$. Eigenmodes which resemble free plane wave states will thus not contribute to the $`\eta ^{}`$ hairpin correlator. In the instanton model, the low eigenmode wave functions will exhibit local peaks on the instantons and anti-instantons in which $`\psi _i^{}\gamma ^5\psi _i=\pm \psi _i^{}\psi _i`$, respectively. Thus, if the instanton picture is correct, the low eigenmodes will produce a large hairpin correlator via lumps of right-handed chiral charge $`\psi _i^{}(1+\gamma ^5)\psi _i=\psi _{iR}^{}\psi _{iR}`$ interspersed with other lumps of left-handed chiral charge $`\psi _i^{}(1\gamma ^5)\psi _i=\psi _{iL}^{}\psi _{iL}`$. This is a definitive prediction of the instanton model which follows directly from the self-duality and anti-self-duality of the gauge lumps, and it is this prediction that we address here. What we want to emphasize is that the resolution of the $`U_A(1)`$ problem only requires that the low eigenmodes exhibit patches or lumps which contain significant amounts of $`\psi ^{}\gamma ^5\psi `$ charge, not that these lumps be purely right-handed or left-handed. The view espoused by Witten leads to this less restrictive prediction, which is strongly supported by our lattice results.

## IV The Chiral Structure of the Near-Zero-Modes

### A Methods

As we discussed in the preceeding section, the structure of the QCD gauge vacuum in any instanton-based model involves a particular assertion about the structure of typical gauge excitations in the regions in which the field strength is large. Specifically, these excitations are supposed to be dominantly self-dual or anti-self-dual. We will test this assertion by studying the chiral structure of the low Dirac eigenmodes. In an instanton picture, these low eigenmodes are approximately given by linear combinations of topological zero-mode wave functions centered around each instanton and anti-instanton in the configuration. The fermion zero mode associated with an instanton (anti-instanton) is a lump of purely right-(left-)handed chiral charge, $`\psi ^{}(1\pm \gamma ^5)\psi `$. For more general large gauge field fluctuations, we expect that the right- and left-handed components of the low-lying eigenfunctions will clump around the same large gauge fluctuations, with relative strength determined by the relative size of self-dual and anti-self-dual gauge components. To see why this is true, consider the eigenfunctions of the continuum Dirac operator in a background gauge field. We note that the Dirac eigenfunctions satisfy $`i\overline{)}D\psi =\lambda \psi `$, from which we can obtain second order differential equations for the left and right components,

$$(i\overline{)}D)^2\psi =\left[D^2+\frac{1}{2}\sigma _{\mu \nu }G_{\mu \nu }\right]\psi =\lambda ^2\psi .$$
(15)
Separating the gauge field into self-dual and anti-self dual components,

$$G_{\mu \nu }=\frac{1}{2}\left(G_{\mu \nu }+\stackrel{~}{G}_{\mu \nu }\right)+\frac{1}{2}\left(G_{\mu \nu }\stackrel{~}{G}_{\mu \nu }\right)G_{\mu \nu }^{(+)}+G_{\mu \nu }^{()},$$
(16)
we obtain

$`\left[D^2+{\displaystyle \frac{1}{2}}\sigma _{\mu \nu }G_{\mu \nu }^{(+)}\right]\psi _L`$ $`=`$ $`\lambda ^2\psi _L`$ (17)
$`\left[D^2+{\displaystyle \frac{1}{2}}\sigma _{\mu \nu }G_{\mu \nu }^{()}\right]\psi _R`$ $`=`$ $`\lambda ^2\psi _R.`$ (18)
These are 4-D Schrodinger-like eigenvalue equations, with the self-dual and anti-self-dual components of the gauge field playing the role of a potential term for the left- and right-handed components of the eigenvector. If the gauge lump is purely self-dual or anti-self-dual, then only one of the two chiral components will be attracted by the gauge fluctuation, and the fermion lump will be purely left-handed or right-handed. For more general gauge fluctuations, we expect the chiral structure of the fermion lump to vary arbitrarily, as determined by the relative size of the $`G^{(+)}`$ and $`G^{()}`$ components of the gauge lump.

In the lattice results discussed here, we study the chirality of low Dirac eigenmodes by calculating the left and right chiral charges $`\psi _L^{}\psi _L`$ and $`\psi _R^{}\psi _R`$ for particular eigenvectors in the regions where their wave functions are large. Specifically, we scan through each eigenvector site by site and pick out the sites for which the overall size of the wave function $`\psi ^{}\psi =\psi _L^{}\psi _L+\psi _R^{}\psi _R`$ is greater than some minimum. We have chosen this threshold so that we sample about $`1\%`$ of the sites on the lattice for a typical eigenmode. Thus we are looking at the very highest peaks of the fermion eigenfunctions. If an instanton-dominated picture of low eigenmodes is at all valid, we would expect the peaks of the wave function to closely resemble instanton zero modes. Thus, if instantons dominate, a local peak in the wave function for a low-lying eigenmode (a fermion lump) should be dominantly a lump in $`\psi _L^{}\psi _L`$ or $`\psi _R^{}\psi _R`$, but not both. On the other hand, fermion lumps without a definite chirality would be an indication of non-self-dual gauge fluctuations, as expected from a confinement-related mechanism.

In the discussion of our results, an important distinction is made between exactly real modes and near-real modes of the Wilson-Dirac operator, which correspond to zero modes and near-zero modes, respectively, of the continuum Dirac operator. In the continuum theory, an exact zero mode is associated with the global topology of the gauge field on a 4-torus. In any gauge configuration there would be a minimum number of exact zero modes equal to the integrated topological charge. Such zero modes are unpaired and should be chiral, independent of whether the topological charge comes in the form of instantons or some other gauge fluctuations. The true test of instanton- vs. confinement-related fluctuations comes from studying the near-real modes, which also should be locally chiral if instantons are dominant. For example, a configuration of a nearby instanton-anti-instanton pair should produce a pair of near-real eigenmodes, each of which exhibit two lumps of opposite chirality. For more general gauge fluctuations which are not instantons, the attached fermion lumps will not have a definite chirality but will have both $`\psi _L^{}\psi _L`$ and $`\psi _R^{}\psi _R`$ charges.

### B Results

Using the procedure outlined in the previous section, we have carried out a detailed study of the low-lying eigenmodes of the Wilson-Dirac operator on an ensemble of Monte Carlo generated lattice gauge configurations. The gauge configurations used were a 30-configuration subset of the b-lattice quenched ensemble from the Fermilab ACPMAPS library. Low eigenmodes were obtained using the Arnoldi algorithm . The lattice size is $`12^3\times 24`$, and the gauge coupling is $`\beta =5.7`$ (corresponding to a lattice spacing of $`a^11.18`$ GeV). Although the lattice spacing is relatively coarse, recall that an extensive study of chiral symmetry and quenched chiral logarithms has been carried out on this gauge ensemble , showing that this lattice spacing is fine enough to reproduce in remarkable detail a variety of quenched chiral log and chiral loop effects related to the $`\eta ^{}`$ hairpin diagram. Combined with the clear observation of chirality for exactly real modes (see below), we are reasonably confident that lattice spacing effects do not invalidate our conclusions.

As described above, we select a subset of lattice points for each eigenmode, picking out the regions in which the wave function is large. For each of the chosen points, we calculate the magnitudes of the left and right components and parametrize their ratio by a “chiral orientation” parameter $`X`$, defined by

$$\mathrm{tan}\left(\frac{\pi }{4}(1+X)\right)=\frac{|\psi _L|}{|\psi _R|}.$$
(19)
Note that $`X`$ is just an angle in the $`|\psi _L|`$-$`|\psi _R|`$ plane, rescaled to run between -1 and +1. In particular, if the fermion eigenmode lumps have a random chiral orientation, this will be reflected as a flat distribution in $`X`$. On the other, hand if the eigenmodes are locally chiral, as with instanton zero-modes, the distribution will be peaked near $`X\pm 1`$. For any particular eigenmode or set of eigenmodes we wish to examine, we construct a histogram of values of the chirality parameter $`X`$ for all the space-time points at which the size of the wave function is above threshold. In our examination of 1592 low-lying eigenmodes on 30 gauge configurations, we found three distinct types of chirality histograms:
(1) The exactly real (unpaired) Wilson-Dirac eigenmodes in the continuum band (i.e. near $`\kappa _c`$) show a significant degree of chirality, as seen in Fig. 1, i.e. the histogram exhibits clear peaks at the two ends of the histogram $`X\pm 0.5`$. (Lattice effects have shifted these peaks from their continuum location of $`\pm 1`$, but the peaks are still clearly visible.) Fig. 1 includes results from 37 real modes.
(2) The near-real (paired) modes show a distribution which is strikingly flat over the central region of $`X`$ between $`.5`$ and $`.5`$, as seen in Fig. 2. This plot shows the chirality distribution for 350 complex modes with imaginary parts between 0.004 and 0.1 in lattice units (about 5 MeV to about 120 MeV in physical units).
(3) The eigenmodes with larger imaginary parts givea a chirality distribution that is peaked in the central region $`X0`$. This is seen in Fig. 3. This plot includes 336 modes with complex eigenvalues that have $`|\mathrm{Im}\lambda |>0.5`$.

We now discuss the significance of each of these three types of chirality distribution. As we have discussed, the chirality of the exactly real modes is expected from general considerations, and has nothing to do with instantons per se. For continuum Dirac eigenmodes, $`\gamma ^5`$ reflection relates conjugate pairs of eigenmodes at $`\pm \lambda `$. Thus, an unpaired, nondegenerate mode with zero eigenvalue must be an eigenstate of $`\gamma ^5`$, i.e. $`\gamma ^5\psi _0(x)=\pm \psi _0(x)`$. Of course, this argument is rendered inexact by lattice effects and the non-antihermiticity of the Wilson-Dirac operator. Nevertheless, the lattice results show clear evidence of the required chirality of the exactly real modes. This becomes even more clear if we look at histograms for individual real modes. In nearly all cases, the histogram for a single real eigenmode only exhibits one chiral peak, not two, as seen in the plots of three typical real modes (taken from three different gauge configurations) in Fig. 4. Thus the zero-modes exhibit a global chirality, as expected. Again we emphasize that the expectation that a zero-mode will exhibit an overall chirality has nothing to do with instantons but is required by general principles: an exact zero mode will appear in the spectrum if the overall integrated topological charge differs from zero . In an instanton picture, a positive chirality zero mode will have wave function peaks at the locations of some or all of the instantons but will be small in the vicinity of the anti-instantons. But chiral zero modes will occur even if the topological charge appears in the form of non-self-dual (e.g., random) gauge fluctuations.

Next we turn to the near-real modes which come in conjugate pairs and are therefore not required to be chiral. In fact, their global chirality must vanish as a consequence of $`\gamma ^5`$ hermiticity of the Wilson-Dirac operator. The chirality histogram for near-real modes is shown in Fig. 2. In assessing our lattice results, we must be aware of the possibility that finite lattice spacing effects may obscure the chiral behavior expected from continuum arguments. It is therefore particularly reassuring that we are able to clearly observe chirality peaks in the histogram of exactly real modes, Fig. 1, where chirality is required on general principles. It is difficult to imagine lattice effects which would destroy the chirality of the near-real modes without also destroying it in the exactly real modes. The contrast between the chiral structure of the exactly real eigenmodes (Fig. 1 and Fig. 4) and the flat random chirality seen in the near-real modes (Fig. 2 and Fig. 5), thus makes the latter result quite compelling. One very identifiable lattice effect is that the histograms are suppressed near the boundaries $`X=\pm 1`$, so that, for example the chirality peaks for the real modes are at $`X\pm 0.5`$ instead of $`\pm 1`$. The same effect is visible at the ends of the histogram for the near-real modes, Fig. 2, where the flat central plateau falls off rapidly for $`|X|>0.5`$. Like the exactly real modes, the chiral structure of the near-real modes is further clarified by studying histograms for individual modes. The histograms for three typical modes are shown in Fig. 5. The individual modes displayed in Fig. 5 were intentionally chosen to be particularly close to the real axis, with imaginary parts of 5 MeV, 10 MeV, and 16 MeV, respectively. The Arnoldi algorithm has no trouble resolving such nearly real eigenvalues into complex conjugate pairs, so in practice we found no difficulty in distinguishing between unpaired, exactly real modes, and nearly real, paired modes. The point emphasized by the modes shown in Fig. 5, and by the other near-real modes that we have inspected individually, is that even modes very close to the real axis show no tendency to be locally chiral, in marked contrast to the exactly real, unpaired modes.

The flat chirality distribution for the near-real modes exhibited in Fig. 2 is our central result. This may be contrasted not only with the chiral peaking of the real-mode histogram, but also with the central peaking of the higher momentum modes in Fig. 3 (with eigenvalues $`0.5<\mathrm{Im}\lambda <1.0`$). The central peaking at $`X0`$ can be interpreted as approximate “plane wave” behavior, in that these modes exhibit relatively little pseudoscalar charge. Note that the flat chiral distribution of the low-lying modes in Fig. 2 corresponds to generally nonvanishing $`\psi ^{}\gamma ^5\psi `$ charge in the wave function peaks. Thus, these modes provide the mechanism for producing the $`\eta ^{}`$ mass. To the extent that the higher plane-wave-type modes are centrally peaked, they have $`\psi ^{}\gamma ^5\psi 0`$, and do not contribute significantly to the $`\eta ^{}`$ mass. Of course, even the modes in Fig. 3 have modest-sized eigenvalues of $`600`$ MeV to a GeV, and so the peak around $`X=0`$ is still fairly broad. By examining histograms for various ranges of eigenvalues, we see a gradual transition from the plateau-like behavior for small values of Im $`\lambda `$ to the more centrally peaked distribution of Fig. 3. By contrast, the transition from Fig. 1 to Fig. 2 is sudden, with no indication of chiral peaks even for complex modes very close to the real axis.

## V Conclusions

It is now widely accepted that topological charge plays an important role in low energy QCD. This was originally discovered through the realization that vacuum topological transitions can lead to a resolution of the $`U_A(1)`$ problem. However, since Witten’s 1979 papers, it has been unclear whether the dynamics underlying the $`\eta ^{}`$ mass is associated with the semiclassical tunnelling events called instantons or with the large vacuum fluctuations typical of confinement. More generally, it has been unclear whether fluctuations of topological charge in the QCD vacuum are predominantly in the form of instantons.

In this paper we have approached this question from the fermionic point of view. After reviewing the role of fermionic near-zero modes in $`s\chi SB`$ and in the generation of the $`\eta ^{}`$ mass, we have argued that the presence or absence of instantons is conveniently encoded in the local chirality properties of these near-zero modes. The main points of this paper are (1) to propose the local chirality calculation as a way of testing the instanton picture in the framework of lattice QCD, and (2) to present our lattice data, which convincingly indicate that the fluctuations of the QCD vacuum are not instanton-dominated, in agreement with Witten’s conjecture.

While our work uses the Wilson-Dirac operator which lacks exact lattice chiral symmetry, we are reasonably confident (as we argued extensively in the text) that our conclusions are not invalidated by lattice artifacts. In this respect, we would also like to point out that the connection between (anti)self-duality and local chiral properties of low-lying Dirac eigenmodes is particularly transparent in QED2. We have tested this on the lattice using the Wilson-Dirac operator and the expected chiral structure is indeed exposed even without exact lattice chiral symmetry (see the Appendix). Nevertheless, it would obviously be quite interesting to see our calculations repeated using a Ginsparg-Wilson fermionic operator. While this is more computationally demanding due to non-ultralocality , a study analogous to ours is actually quite feasible .

The apparent validity of Witten’s conjecture has broad implications for hadron phenomenology. Some of these implications are explored in a companion paper by one of us . An ongoing effort to construct a data base of low eigenmodes for a much larger ensemble of gauge configurations is currently underway. Further studies of the chiral and space-time structure of low Dirac eigenmodes should enable us to extract the detailed structure of topological charge fluctuations in typical gauge configurations, and hopefully lead to satisfactory phenomenological models. This should also provide us with a much more precise understanding of chiral symmetry breaking and the $`U_A(1)`$ problem, and allow us to begin to explore the dynamics of phenomenologically critical quark-pair creation processes, both in hairpin diagrams and in ordinary OZI-allowed decays.

ACKNOWLEDGEMENTS

The work of N.I. was supported by DOE contract DE-AC05-84ER40150 under which the Southeastern Universities Research Association (SURA) operates the Thomas Jefferson National Accelerator Facility (Jefferson Lab). The work of Ivan Horváth, John McCune and H.B. Thacker was supported in part by the Department of Energy under grant DE-FG02-97ER41027. I.H. was also supported by DOE contract DE-FG05-84ER40154.

## A Local Chirality in QED2

To further examine the suitability of local chirality properties of Dirac near-zero modes as a test of local self-duality for the underlying gauge fields, we studied this connection also in the case of QED2. This is an ideal testing ground for the underlying ideas we introduced in Section IV because, in some sense, the gauge field is automatically (anti)self-dual. This can be seen from the fact that $`F_{\mu \nu }_\mu A_\nu _\nu A_\mu `$ only has one independent component (the electric field $`E=F_{12}`$), yielding for the Euclidean action density $`F_{\mu \nu }F_{\mu \nu }E^2`$, and for the topological charge density $`Q(x)ϵ_{\mu \nu }F_{\mu \nu }E`$. Thus, the magnitude of $`Q(x)`$ is always locked in with the magnitude of $``$, as is the case for the (anti)self-dual field in four dimensions. We would like to stress, though, that in case of QED2 this has nothing to do with relevance of instantons, but rather it is a generic property of the gauge field.

In terms of the Dirac eigenvalue problem (analogous to Eq. (17)-(18)), one finds that the left and right components of the eigenvector $`\psi `$ in $`i\overline{)}D\psi =\lambda \psi `$ have to satisfy

$`\left[D^2+eE\right]\psi _L=\lambda ^2\psi _L`$ (A1)
$`\left[D^2eE\right]\psi _R=\lambda ^2\psi _R`$ (A2)
where $`D_\mu _\mu +ieA_\mu `$, and we have used the convention $`\gamma _5i\gamma _1\gamma _2`$. From this one can see that in the region around the local positive maximum of $`E`$ it is energetically favorable for the near-zero mode to have a large $`\psi _R`$ and relatively small $`\psi _L`$. Similarly, around local negative maxima of $`E`$, one expects the dominance of $`\psi _L`$. In other words, there should be a certain degree of local chirality exhibited by the near-zero modes. (Again, we emphasize that in QED2, unlike QCD4, the local chirality of the near-zero modes is expected in the vicinity of any strong fluctuation of the gauge field, and thus does not imply the existence instantons.)

We have tested this scenario on the lattice using a procedure analogous to the one we have discussed at length for QCD. In particular, we have calculated the eigenmodes of a Wilson-Dirac operator in the background of Monte Carlo generated configurations of pure gauge compact QED2, and calculated local chirality in the regions where the low-lying eigenfunction is large. In Figures 6 and 7 we show the results from $`100`$ configurations on a $`24\times 24`$ lattice at $`\beta =2`$ (we use the standard normalization for Wilson’s plaquette action). The two histograms show the local chirality of real modes and of near-real modes lying in the vicinity of the continuum branch of the Wilson-Dirac spectrum, respectively. In Figure 7 near-real modes with imaginary part less than $`0.3`$ in absolute value were included, while the maximal imaginary part for the high modes is typically around $`1.3`$ (we use the standard normalization for the Wilson-Dirac operator). In both cases, about $`1\%`$ of the highest points in a given mode were used. Similar to the situation in QCD, significant local chirality is observed for the real modes as expected from the index theorem in the continuum. However, contrary to the situation in QCD, chiral peaks are also apparent for the non-zero modes in accordance with the above arguments. It should be emphasized here that these results not only illustrate the validity of the general approach proposed in this paper, but also further confirm that the required effects can be captured by the Wilson-Dirac operator. Indeed, we can observe the local chirality of near-zero modes even though the gauge fields are rather rough, even though the cut for imaginary part of the included modes is rather high, and even though the lumpiness of the topological charge is not expected to be as pronounced here as in QCD.

REFERENCES


format: text

text:
# The effect of distant large scale structure on weak lensing mass estimates

## 1 Introduction

11footnotetext: Present address: CITA, University of Toronto, 60 St. George Street, M5S 3H8 Ontario, Canada

Light rays are deflected as they travel through the universe, and as a result our view of the distant, faint galaxies is somewhat distorted. Measuring these lensing induced distortions has proven to be a valuable tool in observational cosmology (see the review by Mellier 1999): the lensing signal is sensitive to the projected mass surface density of the deflector, and through lensing one can probe the mass distribution without having to rely on assumptions about the dynamical state and nature of the deflecting matter.

When the surface density of the lens is sufficiently high, the distortions can be so large that multiple images can be seen, which provide strong constraints on the mass distribution in cluster cores. In this paper we will concentrate on the weak lensing regime, which allows us to study the cluster mass distribution on larger scales.

Weak lensing is a powerful tool to probe the distribution of matter in the universe on various scales. One exciting application is the measurement of the lensing signal induced by large scale structure (e.g. Blandford et al. 1991; Kaiser 1992; Bernardeau, van Waerbeke, & Mellier 1997; Schneider et al. 1998, SWJK hereafter), and recently several groups have reported detections of this signal (Bacon et al. 2000; Kaiser, Wilson, & Luppino 2000; van Waerbeke et al. 2000; Wittman et al. 2000).

The first successful weak lensing results were obtained by studying the systematic distortion of the images of faint galaxies behind rich clusters of galaxies (e.g. Tyson, Wenk, & Valdes 1990). Measuring the masses and mass-to-light ratios of rich clusters is an important application of weak lensing, because comparison of the weak lensing result with other mass estimators (e.g., X-ray, S-Z effect, or velocities) can give important clues about the geometry and dynamical state of the system (e.g, Zaroubi et al. 1998).

Weak lensing also has some drawbacks. First of all, the amplitude of the lensing signal depends on the redshift distribution of the faint background galaxies. These galaxies are generally too faint to be included in spectroscopic redshift surveys, but photometric redshifts are thought to provide ample information on the redshift distribution of these galaxies (e.g., Hoekstra, Franx, & Kuijken 2000). Also the observed weak lensing signal is a measure of a surface density contrast, and one of the problems associated with this is the mass sheet degeneracy (Gorenstein, Shapiro, & Falco 1988). One could try to get around this problem by measuring the lensing signal out to large distances from the centre, and arguing that the cluster surface density should vanish there.

Another potential problem is the fact that the lensing signal is sensitive to all matter along the line of sight. In general, weak lensing analyses of clusters assume that the signal induced by the cluster under investigation is so dominant that the contribution from other structures along the line of sight can be neglected.

The effect of structures, close to the cluster, such as the filaments that connect the cluster to the ‘cosmic web’, or an elongation along the line of sight, have been studied through N-body simulations (e.g, Cen 1997, Metzler et al. 1999). Local structures introduce both a bias and an additional uncertainty in the cluster mass estimate, in particular when one for instance tries to measure the mass within $`r_{200}`$.

In this paper we study how distant structures affect weak lensing mass estimates of clusters of galaxies. Unlike the local structure, these are uncorrelated with the cluster, and they can be filaments, galaxy groups, or even other clusters. That such structures affect the mass estimates is easily seen in the case of a massive high redshift cluster, with a low redshift group in front of it. Assume the cluster is at a redshift of unity, with a group at say $`z=0.3`$, and that the mass of the group is one tenth of that of the cluster: the contribution of the group to the total lensing signal is about one third, compared to the 10% in mass.

In this paper we will quantify the uncertainty in weak lensing mass estimates of rich clusters caused by intervening matter. To do so we examine the predicted statistical properties of the contribution by large scale structure. We use an approach similar to what is done to predict the amplitude of the cosmic shear signal, which is used to constrain the cosmological parameters. Given a cosmological model, we estimate the uncertainty in cluster masses derived from weak lensing analyses.

The structure of this paper is as follows. In section 2 we discuss two popular methods to measure cluster masses. In section 3 we derive the equations that describe the effect of large scale structure on such mass estimates. The effect of large scale structure on mass estimates for clusters is quantified in section 4. We examine the dependence on cluster redshift, aperture size and source redshift distribution.

## 2 Estimating masses from weak lensing

Cluster masses can be measured in several ways. In this paper we will use aperture mass statistics, which provide a filtered measurement of the of the surface mass density in an aperture (Schneider 1996, S96 hereafter):

$$M_{ap}(\theta )=_0^\theta d^2\theta ^{}U(|\theta ^{}|)\kappa (\theta ^{}),$$
(1)
where $`U(\theta )`$ is the weight or filter function. Provided that $`U(\theta )`$ is compensated, i.e.

$$_0^\theta 𝑑bbU(b)=0,$$
(2)
the aperture statistic can be related to the tangential shear $`\gamma _T`$. Unfortunately $`\gamma _T`$ cannot be observed directly, but one can measure the distortion $`g_T=\gamma _T/(1\kappa )`$. In the weak lensing regime, where $`\kappa 0`$, one can use $`\gamma _Tg_T`$, which is what we do throughout this paper. Then we have

$$M_{ap}(\theta )=_0^\theta d^2\theta Q(|\theta |)\gamma _T(\theta ).$$
(3)
Here, $`Q(\theta )`$ is related to $`U(\theta )`$ through:

$$Q(b)=\frac{2}{b^2}_0^b𝑑b^{}b^{}U(b^{})U(b).$$
(4)
S96 has shown that the signal-to-noise ratio of the aperture mass measurement is optimal when $`Q(|\theta |)`$ is the same as the (expected) tangential shear as a function of $`|\theta |`$.

We want to quantify the uncertainty in mass measurements of clusters, and therefore require that the aperture statistic has a clear physical meaning. Studies of cosmic shear (see SWJK) use different choices for $`Q(|\theta |)`$. Below we discuss two statistics that have been used to estimate cluster masses.

### 2.1 SIS model fit

A popular model to describe the cluster mass distribution is the singular isothermal sphere (SIS)

$$\kappa (r)=\frac{r_E}{2r},$$
(5)
where $`r_E`$ is the Einstein radius. The Einstein radius (in radians) is related to the velocity dispersion through

$$r_E=4\pi \left(\frac{\sigma }{c}\right)^2\beta .$$
(6)
The tangential shear as a function of radius is simply given by

$$\gamma _T(r)=\kappa (r)=\frac{r_E}{2r}.$$
(7)
The best fit SIS model is obtained by minimizing

$$\underset{i}{}\left(\frac{\gamma _{T,i}r_E/(2r_i)}{\sigma _\gamma }\right)^2,$$
(8)
with respect to $`r_E`$, where $`\gamma _{T,i}`$ are the individual measurements of the lensing signal at radius $`r_i`$. The uncertainty in the measurement of the tangential shear, $`\sigma _\gamma `$ is given by

$$\sigma _\gamma ^2=\frac{\sigma _{\mathrm{gal}}^2}{2\pi \overline{n}r_i\mathrm{\Delta }r_i},$$
(9)
with $`\sigma _{\mathrm{gal}}`$ the uncertainty in the measurement of an individual galaxy, $`\overline{n}`$ the number density of sources, and $`\mathrm{\Delta }r_i`$ the width of the $`i`$th radial bin. If we take the widths of all the bins the same, and assume that the number density of sources is uniform, minimizing with respect to $`r_E`$ gives

$$r_E=2\frac{_i\gamma _{T,i}}{_i\frac{1}{r_i}}.$$
(10)
For a continuous profile the summation can be replaced by an integral. As mentioned before one measures the distortion and not the shear, and equation 10 only works well when $`\kappa `$ is small. Also substructure in the cluster core changes the average tangential distortion (e.g., Hoekstra et al. 2000). To come around these problems one has to choose an inner radius for the fit, such that equation 10 is a good approximation.

If we assume that the tangential shear has been measured from a distance $`R_0`$ out to $`R`$, the Einstein radius satisfies

$$r_E=\frac{2}{\mathrm{ln}(R/R_0)}_{R_0}^R𝑑r\gamma _T(r).$$
(11)
Comparison with equation 3 shows that the best fit SIS model is an aperture mass measurement where $`Q(|\theta |)`$ is given by

$$Q(|\theta |)=\{\begin{array}{cc}(\pi \theta \mathrm{ln}(R/R_0))^1& ,R_0|\theta |R\hfill \\ 0& ,\mathrm{elsewhere}\hfill \end{array}.$$
(12)
The corresponding weight function for the surface density $`U(\theta )`$ is given by

$$U(|\theta |)=\frac{1}{\pi \mathrm{ln}(R/R_0)}\{\begin{array}{cc}2/R_02/R& ,0|\theta |<R_0\hfill \\ 1/|\theta |2/R& ,R_0|\theta |R\hfill \\ 0& ,|\theta |>R\hfill \end{array}.$$
(13)
The mass profiles of many clusters that have been studied to date are consistent with a SIS model. Thus the weight function suggested above is close to the optimal choice. However, this choice for $`U(|\theta |)`$ is not a continuous function. A functional form which is continuous, yet similar is suggested by S96, who discusses the use of aperture mass statistics to find mass concentrations in deep images.

### 2.2 $`\zeta `$-statistic

Another statistic which has been widely used is the $`\zeta `$-statistic (Fahlman et al. 1994):

$$\zeta (\theta _1,\theta _2)=\overline{\kappa }(<\theta _1)\overline{\kappa }(\theta _1<|\theta |<\theta _2).$$
(14)
It gives the difference of the average dimensionless surface mass density in a circular aperture of radius $`\theta _1`$ and that in an annulus with $`\theta _1<|\theta |<\theta _2`$. The corresponding weight function $`U(|\theta |)`$ is given by:

$$U(|\theta |)=\{\begin{array}{cc}(\pi \theta _1^2)^1& ,|\theta |<\theta _1\hfill \\ (\pi (\theta _2^2\theta _1^2))^1& ,\theta _1<|\theta |<\theta _2\hfill \\ 0& ,\mathrm{elsewhere}\hfill \end{array}.$$
(15)
This gives

$$Q(|\theta |)=\{\begin{array}{cc}\theta _2^2(\pi \theta ^2(\theta _2^2\theta _1^2))^1& ,\theta _1<|\theta |<\theta _2\hfill \\ 0& ,\mathrm{elsewhere}\hfill \end{array}.$$
(16)
Thus the $`\zeta `$-statistic can be related to the tangential shear through:

$$\zeta (\theta _1,\theta _2)=\frac{2\theta _2^2}{\theta _2^2\theta _1^2}_{\theta _1}^{\theta _2}d\mathrm{ln}\theta \gamma _T(|\theta |).$$
(17)
The SIS model fit has only a simple physical meaning when the cluster mass profile is indeed isothermal. The interpretation of the $`\zeta `$-statistic is independent of the mass distribution, although the statistic is not optimal in terms of signal-to-noise ratio.

Under the assumption that one can estimate or minimize $`\overline{\kappa }(\theta _1<|\theta |<\theta _2)`$, the average surface density in the control annulus, one obtains a measurement of $`\overline{\kappa }(<\theta _1)`$, and thus of the cluster mass. One way to do so is to assume a model for the cluster surface mass density which is used to estimate the average surface density in the annulus (e.g. Hoekstra et al. 1998). This way the cluster mass estimate is less dependent on the assumed model compared to the SIS model fit.

## 3 Contribution by LSS

Under the assumption that the cluster is the only lensing structure in the field, the aperture mass $`M_{\mathrm{obs}}`$ provides a direct measure of the cluster mass. However, the weak lensing signal is sensitive to all matter along the line of sight, and other structures, which we will refer to as large scale structure, will introduce an additional signal:

$$M_{\mathrm{obs}}(\theta )=M_{\mathrm{cluster}}(\theta )+M_{\mathrm{LSS}}(\theta ).$$
(18)
The aperture mass statistic measures a density contrast, and the expectation value for the contribution by uncorrelated large scale structure $`M_{\mathrm{LSS}}=0`$ (see also SWJK). Thus on average, distant structures do not introduce a bias in cluster mass estimates, but an uncertainty in the cluster mass of $`M_{\mathrm{LSS}}^2^{1/2}`$. As a result the effect of distant large scale structure can be treated as an additional (statistical) source of error. Its relative importance can be decreased by averaging the results for several clusters, i.e. measuring their average mass. We note that on certain scales $`M_{\mathrm{LSS}}`$ might be distributed skewly.

The effect of ‘local’ (or correlated) large scale structure has been studied using N-body simulations (e.g., Cen 1997; Metzler et al. 1999). In this case either projection effects, or filaments connected to the cluster bias the weak lensing mass estimates towards somewhat higher values. This becomes particularly important when one tries to estimate the enclosed mass within several Mpc’s from the cluster centre (Cen 1997).

SWJK have shown how one can relate the aperture mass statistic to the power spectrum of density fluctuations. Measuring $`M_{\mathrm{LSS}}^2^{1/2}`$ for a number of random fields allows one to constrain the projected power spectrum. In this paper the problem is reversed: given the power spectrum, what is the resulting $`M_{\mathrm{LSS}}^2^{1/2}`$? A detailed discussion about the use of aperture mass statistics and cosmic shear can be found in SWJK and Bartelmann & Schneider (1999).

SWJK showed that the effect of large scale structure on the variance in the aperture mass statistic is given by:

$$M_{\mathrm{LSS}}^2(\theta )=2\pi _0^{\mathrm{}}𝑑llP_\kappa (l)g(l,\theta )^2,$$
(19)
where the effective projected power spectrum $`P_\kappa (l)`$ is given by:

$$P_\kappa (l)=\frac{9H_0^4\mathrm{\Omega }_m^2}{4c^4}_0^{w_H}𝑑w\left(\frac{\overline{W}(w)}{a(w)}\right)^2P_\delta (\frac{l}{f_K(w)};w)$$
(20)
Here $`w`$ is the radial coordinate, $`a(w)`$ the cosmic scale factor, and $`f_K(w)`$ the comoving angular diameter distance. $`\overline{W}(w)`$ is the source averaged ratio of angular diameter distances $`D_{ls}/D_s`$ for a redshift distribution of sources $`p_w(w)`$:

$$\overline{W}(w)=_w^{w_H}𝑑w^{}p_w(w^{})\frac{f_K(w^{}w)}{f_K(w^{})}.$$
(21)
The function $`g(l,\theta )`$ in equation 19 depends on the filter function $`U(|\theta |)`$, and is given by:

$$g(l,\theta )=_0^\theta 𝑑\varphi \varphi U(\varphi )J_0(l\varphi ).$$
(22)
As has been demonstrated in Jain & Seljak (1997) and SWJK it is crucial to use the non-linear power spectrum in equation 20. This power spectrum can be derived from the linearly evolved cosmological power spectrum following the prescriptions of Hamilton et al. (1991), and Peacock & Dodds (1996).

### 3.1 Filter functions

The form of the function $`g^2(l,\theta )`$, which probes the projected power spectrum, depends on the choice of $`U(|\theta |)`$. For the SIS model fit we take $`R_0=\alpha R`$, which yields

$$g_{\mathrm{SIS}}(l,\theta )=\frac{\theta }{\pi \mathrm{ln}(\alpha )l\theta }\left[2\left\{J_1(\alpha l\theta )J_1(l\theta )\right\}+\underset{\alpha l\theta }{\overset{l\theta }{}}𝑑sJ_0(s)\right].$$

(23)
We will use a typical value for $`\alpha =0.15`$ (e.g., Hoekstra et al. 1998). Larger values are used when much substructure is found in the cluster core (e.g., Hoekstra et al. 2000). The resulting $`g_{\mathrm{SIS}}^2(l,\theta )`$ is presented in figure 1a.

For the $`\zeta `$-statistic we find

$$g_\zeta (l,\theta )=\frac{J_1(\alpha l\theta )\alpha J_1(l\theta )}{\pi \alpha l\theta (1\alpha ^2)},$$
(24)
where we have taken the inner radius $`\theta _1=\alpha \theta `$. A typical value for $`\alpha `$ used in weak lensing mass estimates of clusters is $`\frac{2}{3}`$, i.e., the control annulus runs from the boundary of the region in which we want to determine the mass out to 1.5 times this radius. Throughout this paper we will use this value for $`\alpha `$. In figure 1b the filter function $`g_\zeta ^2(l,\theta )`$ is shown (taking $`\alpha =\frac{2}{3}`$).

### 3.2 Photometric redshifts

The projected power spectrum $`P_\kappa (k)`$ depends on the redshift distribution of the sources, and consequently we need to know, or assume, a redshift distribution for the sources. We will use the photometric redshift distributions inferred from the northern and southern Hubble Deep Fields (Fernández-Soto, Lanzetta, Yahil 1999; Chen et al. 1998), which generally work well (e.g., Hoekstra, Franx, & Kuijken 2000). There is, however, some field to field variation in the redshift distribution the sources, but we assume that the distributions obtained from the HDFs are representative for the whole sky. We note that most of the lensing by large scale structure is caused by structures at intermediate redshifts, and as a result the signal induced by the LSS is fairly robust against variations in the source redshift distribution.

Using the colours of the galaxies in the HDFs we have computed their $`R`$ magnitudes, which allows a direct comparison with ground based observations. We use galaxies fainter than $`R=20`$. The resulting photometric redshift distributions for different limiting magnitudes are presented in figure 2.

### 3.3 Aperture statistics

Given the photometric redshift distribution discussed above we calculate the contribution of the LSS to the measurements of $`r_E`$ and the $`\zeta `$-statistic as a function of aperture size. We have considered three different cosmologies, which are listed in table 1.

Figure 3a shows the scatter in the value of the Einstein radius $`r_E`$ as a function of aperture size for the SCDM (thick lines) and the OCDM (thin lines) models. For clarity, we limit the number of curves shown in figure 3 and omit the results for the $`\mathrm{\Lambda }`$CDM model. For this model the scatter is similar to the OCDM model, with a somewhat higher amplitude on large scales and a somewhat lower amplitude on arcminute scales. The scatter in the value of $`r_E`$ increases for larger aperture sizes. The value of $`r_E`$ is independent of aperture size, and as a result the relative contribution of large scale structure to the error budget increases with increasing aperture size. Also the scatter is larger for fainter limiting magnitudes, i.e. higher source redshifts.

Figure 3b shows the scatter for the $`\zeta `$-statistic. Because both $`\zeta _{LSS}^2^{1/2}`$ and $`\overline{\kappa }(<\theta )`$ decrease with increasing aperture size the interpretation of figure 3b is not as straightforward as is the case for figure 3a. The effect of aperture size on the accuracy of weak lensing mass estimates is discussed in detail in section 4.3.

Comparison of the results for the SCDM and OCDM cosmologies indicates that the differences are fairly small. We found that the $`\mathrm{\Lambda }`$CDM cosmology gives similar estimates. Although we consider these different cosmologies throughout the paper (and plot the results for the OCDM and SCDM models) we note that the results hardly depend on the assumed cosmology.

## 4 Results

We examine the uncertainty in the weak lensing mass estimate caused by uncorrelated large scale structure. The results are derived for a cluster with a SIS mass profile, and a velocity dispersion of $`\sigma =1000`$ km/s, typical for rich clusters that have been studied in the past. We study the dependence on cluster redshift, where we discuss the results for low redshift clusters in more detail. We also examine how the accuracy of the mass measurements depends on aperture size.

### 4.1 Dependence on cluster redshift

First we examine how the mass estimates for clusters at different redshifts are affected when the lensing signal is measured out to a radius of $`1.5h_{50}^1`$ Mpc.

For the SIS model we use the measurements in an annulus from 0.225 to $`1.5h_{50}^1`$ Mpc ($`R=1.5h_{50}^1`$ Mpc, $`\alpha =0.15`$). In the case of the $`\zeta `$statistic we use an inner radius of $`\theta _1=1h_{50}^1`$ Mpc and an outer radius of $`\theta _2=1.5h_{50}^1`$ Mpc $`(\alpha =\frac{2}{3})`$. The value of the $`\zeta `$-statistic only provides a lower limit to the cluster mass, and one way to circumvent this is to estimate the average surface density in the annulus around the aperture, for instance by using the results from a model fit. However, this model fit is also affected by large scale structure. In general the region where the model is fitted to the data largely coincides with the annulus used to compute the value of the $`\zeta `$-statistic. Thus if the value of the $`\zeta `$-statistic is increased by additional structures, the mass from the model will also be higher. Although the uncertainty in the model parameters and the $`\zeta `$-statistic are not completely correlated, we will assume they are. This gives a fair upper limit on the uncertainty in the estimate of $`\overline{\kappa }`$

$$\sigma _{\overline{\kappa }}^{\mathrm{LSS}}=\sigma _\zeta ^{\mathrm{LSS}}+\frac{\sigma _{r_E}^{\mathrm{LSS}}}{(1+\alpha )\theta _2}.$$
(25)
The results are presented in figure 4. Panels c and f show respectively the value of the Einstein radius and $`\overline{\kappa }`$ as a function of cluster redshift. We show the results for the SCDM (thick lines) and the OCDM (thin lines) cosmologies. The solid lines are for a limiting magnitude of $`R=24`$ and the dashed lines for $`R=26`$. Panels b and e show the scatter in the aperture mass statistics introduced by the large scale structure. Finally the ratios of the scatter and the lensing signal are presented in figure a and d.

For clusters at low redshifts $`(z<0.2)`$ the relative importance of large scale structure for the $`\mathrm{\Lambda }`$CDM model is similar to that of the SCDM model. For clusters at high redshift, or deep observations, the results for the $`\mathrm{\Lambda }`$CDM are close to the OCDM results.

Comparison of figures 4a and d shows that the relative importance of the distant large scale structure is smaller in the case of the SIS model fit. For this mass estimator the uncertainty in the cluster mass estimate is about 6% for rich clusters at intermediate redshifts. The uncertainty in the mass is large for high redshift clusters when the limiting magnitude of the sources is rather bright (solid lines). However, in practice, to detect a significant lensing signal of a high redshift cluster deep observations are needed to ensure a sufficiently high number density of sources. As a result the importance of large scale structure is minimized (dashed lines). In the case of nearby clusters, the presence of structure along the line of sight results in a relatively large uncertainty in the cluster mass (see section 4.2).

The uncertainty introduced by large scale structure is much larger when the $`\zeta `$-statistic is used. This is not surprising as one only uses the shear measurements at relatively large radii, where the cluster mass surface density has become fairly low. In addition to this annulus, the SIS model fit uses data at much smaller radii, where the cluster signal is much higher.

The noise in weak lensing analyses is generally dominated by the intrinsic shapes of the source galaxies, and not by the noise in the large scale structure. Both sources of error depend differently on the limiting magnitude of the background galaxies, and as a result the total noise is a rather complex function of cluster redshift and limiting magnitude, as is demonstrated in figure 5a for the OCDM cosmology.

Figure 5b shows the the signal-to-noise ratio for a SIS model fit to the tangential shear of a cluster of $`\sigma =1000`$ km/s, when the contribution of large scale structure is ignored. It should be noted here that we assume that the shapes of the sources have been measured accurately. In reality the shapes of very faint galaxies are difficult to determine. From figure 5b one would expect to be able to measure accurate masses for low redshift clusters. However, the results change significantly when we include the contribution from large scale structure as is shown in figure 5a. The change in signal-to-noise ratio is small for high redshift clusters, where the shapes are still the major contributor to the noise. The true signal-to-noise ratio is lowered significantly in the case of deep observation of clusters at intermediate redshifts, but the largest difference is seen for clusters at low redshifts.

The signal-to-noise ratios for the $`\mathrm{\Lambda }`$CDM model are intermediate to those found for the OCDM and SCDM models, resembling the OCDM better for high redshift clusters. However, at high redshifts the differences between the various models become less important (they are fairly small anyway), because the noise is dominated by the intrinsic shapes of the sources.

### 4.2 Low redshift clusters

Figure 5a demonstrates that large scale structure introduces a large uncertainty in the mass measurement of low redshift clusters. To date not many low redshift clusters have been studied through weak lensing. This is because the amplitude of the lensing signal peaks at intermediate redshifts, and the signal is extremely small for nearby clusters. Furthermore their angular sizes were too large for the CCD cameras. Panoramic CCD cameras have become available recently, and a number of low redshift clusters have been imaged for weak lensing purposes. For instance Joffre et al. (2000) have analysed the mass distribution of Abell 3667 $`(z=0.055)`$.

Although the lensing signal of low redshift clusters is small, there are several reasons to study these systems. First of all, they can, or have been studied extensively by other techniques. Also, the uncertainty in the redshift distribution of the sources is almost non-existent, and one can use relatively large background galaxies.

In this section we examine how the weak lensing mass measurement of the Coma cluster $`(z=0.023)`$ is affected by structures at higher redshifts. A weak lensing analysis of the most nearby rich cluster of galaxies is challenging, but could provide important insights in the cluster dynamics. We assume a velocity dispersion of 1000 km/s for the cluster (e.g., Colles & Dunn 1996).

In figure 6a we plot the sources of noise in the measurement of the Einstein radius as a function of limiting magnitude of the sources. The figure shows the contribution from the large scale structure (dashed lines) and the intrinsic shapes of the sources (dotted lines). For the intrinsic shapes of the sources we have assumed $`\gamma _T^2^{1/2}=0.25`$, similar to what is observed in deep HST images (e.g., Hoekstra et al. 2000) Reaching the depth of the HDFs one would expect to obtain a signal-to-noise ratio of more than 20. However, figure 6a demonstrates that the total uncertainty in the measurement of the Einstein radius reaches a minimum value for a limiting $`R=24`$.

Figure 6b shows the expected signal-to-noise ratio as a function of limiting magnitude, and it demonstrates that observations with $`R_{\mathrm{lim}}>24.5`$ no longer improve the signal-to-noise ratio of the mass measurement. The maximum signal-to-noise ratio that can be reached by fitting a SIS model to the observed tangential shear around the Coma cluster is $`5`$ for the SCDM model, and $`7`$ for the OCDM model. The results for the $`\mathrm{\Lambda }`$CDM model lie between the curves for the other two models.

In figure 7 we show how the accuracy of the mass measurement is affected by large scale structure when the $`\zeta `$-statistic is used. Comparison with figure 6 shows that the SIS model fit yields a better signal-to-noise ratio.

### 4.3 Large apertures

The intrinsic shapes of the sources result in a statistical uncertainty in the measurement of the Einstein radius. If the SIS model is fitted to the measurements in an annulus from $`R_0`$ out to $`R`$, the error in $`r_E`$ is given by

$$\sigma _{r_E}=\sqrt{\frac{2}{\pi \overline{n}\mathrm{ln}(R/R_0)}}\sigma _{\mathrm{gal}},$$
(26)
where $`\overline{n}`$ is the number density of sources, and $`\sigma _{\mathrm{gal}}`$ is the uncertainty in the measurement of the tangential distortion for a single galaxy.

From equation 26 we find that this contribution to the error budget decreases slowly for increasing aperture sizes. However, figure 3a shows that the noise contribution from large scale structure increases rather rapidly with aperture size.

Figure 8a shows how the contributions of the intrinsic shapes of the sources, and the large scale structure depend on the aperture size. The results are for a cluster at a redshift $`z=0.5`$, and the lensing signal has been measured from $`R_0=0.225h_{50}^1`$ Mpc out to a radius $`R_{\mathrm{out}}`$. The uncertainty in $`r_E`$ caused by the intrinsic shapes of the sources decreases slowly with aperture size (dotted curve), whereas the contribution by the distant large scale structure increases (dashed line). Figure 8b shows the resulting signal-to-noise ratio. The signal-to-noise ratio is virtually constant for outer radii larger than $`1h_{50}^1`$ Mpc. Measuring the lensing signal in larger apertures does not improve the accuracy of the cluster mass. One has also to keep in mind that we have assumed that the mass profile is well described by a SIS model, which might not be true in the outer parts of clusters.

For clusters at higher redshifts the curves look similar, but the maximum signal-to-noise ratio is reached at somewhat larger (physical) aperture sizes. The noise caused by the large scale structure dominates the error for low redshift clusters, and for these systems, the signal-to-noise ratio decreases slowly with increasing aperture size.

So far we have used an inner radius for the SIS model fit of $`R_0=0.225h_{50}^1`$ Mpc, but one cannot always fit the model at such small distances from the centre. For instance in the case of the $`z=0.83`$ cluster MS 1054-03 Hoekstra et al. (2000) showed that the mass distribution in the centre is rather complex and the substructure lowers the azimuthally averaged tangential shear. Consequently they fit the SIS model from $`0.7`$ out to $`2h_{50}^1`$ Mpc, which corresponds to $`\alpha =0.35`$.

We also have examined how the accuracy of the mass estimate changes when the SIS model is fitted to the data within a given aperture, while the inner radius is increased. In this situation both the noise caused by the shapes of the sources and the contribution by the large scale structure increase. Thus small values for the inner radius improve the signal-to-noise of the mass measurement, but one should keep in mind that substructure in the cluster centre complicates a safe choice for the inner radius.

For the $`\zeta `$-statistic the noise contribution caused by the shapes of the galaxies is given by

$$\sigma _\zeta =\sqrt{\frac{1}{\pi (1\alpha ^2)\theta _1^2\overline{n}}}\sigma _{\mathrm{gal}}.$$
(27)
However, we want to estimate $`\overline{\kappa }(<1h_{50}^1\mathrm{Mpc})`$, and thus we have to add the average surface density in the annulus from 1 to $`R_{\mathrm{out}}`$ $`h_{50}^1`$ Mpc. As before we estimate $`\overline{\kappa }(1<r<R_{\mathrm{out}})`$ from a SIS model fit to the tangential shear. The errors in the $`\zeta `$statistic and the Einstein radius from the model fit are combined according to equation 25, because the errors are correlated. We note that the size of the correction depends on the assumed mass profile, but for increasing values of $`R_{\mathrm{out}}`$ the correction becomes smaller (i.e., $`\overline{\kappa }(1<r<R_{\mathrm{out}})`$ becomes smaller).

The resulting noise estimates are presented in figure 9a. The corresponding signal-to-noise ratio of $`\overline{\kappa }(<1h_{50}^1\mathrm{Mpc})`$ for a cluster at a redshift $`z=0.5`$ is presented in figure 9b. It shows that using a larger $`R_{\mathrm{out}}`$ improves the accuracy in the measurement of $`\overline{\kappa }(<1h_{50}^1\mathrm{Mpc})`$.

A similar calculation shows that the contribution of the large scale structure becomes more important when one tries to measure the average surface density in larger apertures. For instance the uncertainty in $`\overline{\kappa }(<2h_{50}^1\mathrm{Mpc})`$ almost doubles compared to $`\overline{\kappa }(<1h_{50}^1\mathrm{Mpc})`$.

Attempts to measure the average cluster mass profile out to large projected distances from the cluster centre will also be affected by large scale structure, given the fact that it is already difficult to measure the total mass. Unfortunately it will be difficult to quantify the effect of distant large scale structure on such measurements.

## 5 Conclusions

The observed weak lensing signal is sensitive to all matter along the line of sight, and as a result this additional matter acts as a source of uncertainty in mass estimates of clusters of galaxies. In this paper we have investigated the effect of distant large scale structure, uncorrelated with the cluster mass distribution. It can be treated as an additional source of noise.

Given a cosmological model it is possible to compute the statistical uncertainty in the weak lensing mass measurement caused by the distant large scale structure, and quantify its contribution to the total error budget. We have considered two methods to estimate the mass from a weak lensing analysis. One way is to fit a SIS model to the observed lensing signal. The other method uses the $`\zeta `$-statistic (e.g., Fahlman 1994) to estimate the mass. Both methods give similar results.

We find that the importance of matter along the line sight is fairly small for rich clusters ($`\sigma =1000`$ km/s) at intermediate redshifts, provided that the bulk of the sources are at high redshift compared to the cluster. If the lensing signal is measured out to $`1.5h_{50}^1`$ Mpc from the cluster centre, the typical $`1\sigma `$ relative uncertainty in the mass cause by the large scale structure is about $`6\%`$.

In certain situations the effect of large scale structure is more important. For nearby clusters, such as the Coma cluster, background structures introduce a considerable uncertainty in the mass. Considering only the noise caused by the shapes of the sources used in the weak lensing analysis one expects to achieve very accurate weak lensing mass measurements of nearby clusters, but we find that the large scale structure limits the maximum achievable signal-to-noise ratio to $`7`$, irrespective of the depth of the observations.

We also have examined how the accuracy of the weak lensing mass estimate changes when the lensing signal is measured out to large projected distances from the cluster centre. In the case of the SIS model fit we find that the signal-to-noise ratio does not increase significantly when one includes measurements at radii larger than $`1h_{50}^1`$ Mpc. When the $`\zeta `$-statistic is used to estimate the mass within a $`1h_{50}^1`$ Mpc radius aperture the signal-to-noise ratio does increase with $`R_{\mathrm{out}}`$. If the cluster mass profile well described by a SIS model, the model fit results in a better signal-to-noise ratio.

The effect of large scale structure is important in the outskirts of clusters, where the cluster signal itself is low. In this paper we have quantified the uncertainty in the measurement of the cluster mass, but distant large scale structure will also introduce noise in studies that try to constrain the mass profiles of clusters at large projected distances. Furthermore studies of the 3-D structure of clusters by combining weak lensing, X-ray, and S-Z data (e.g., Zaroubi et al. 1998) will be affected by uncorrelated large scale structure.

Photometric redshifts of the sources can be used to decrease the effect of large scale structure, either by modeling the contribution of large scale structure to the lensing signal, or by selecting sources in an optimal redshift range. This can be only done at the expense of a large amount of additional data. Because the effect of large scale structure can be treated as an additional (statistical) source of noise. Thus an alternative procedure is to average the results of an ensemble of clusters, thus decreasing the noise on the (average) cluster mass estimate.

###### Acknowledgements.

We thank the referee for several useful comments which improved the paper. It is a pleasure to thank Marijn Franx and Konrad Kuijken for useful discussions and comments to improve the paper. Also the comments from Ludo van Waerbeke are acknowledged. Peter Schneider made available his algorithms, which were very helpful.